Archive

Posts Tagged ‘how-to’

How to Install Domoticz Home Automation System in NanoPi NEO and Other ARM Linux Boards

January 19th, 2017 1 comment

I’ve recently started experimenting with IoT projects, and the first hurdle is to select the hardware and software for your projects are there are simply so many options. For the hardware your first have to choose the communication protocols for your sensors and actuators, and if you are going to go with WiFi, ESP8266 is the obvious solution, used together with your favorite low cost Linux development board such as Raspberry Pi or Orange Pi to run some IoT server software locally or leveraging the cloud. But the most difficult & confusing part for me was to select the server software / cloud services as there are just so many options. I prefer having a local server than something running only in the cloud, as my Internet goes a few hours a month, so I started with a solution combining ThingSpeak with MQTT gathering data from Sonoff power switches running ESPurna firmware and vThings CO2 monitor. This works OK, but while ThingSpeak.com cloud service is continuously update, its open source version has not been updated since mid 2015. Among the many service and software framework available, one seems to have come more often than other, is supported by vThings air monitoring platforms, and recently been added to ESPurna. I’m talking about Domoticz described as:

a Home Automation system that lets you monitor and configure various device like lights, switches, various sensors/meters like temperature, rain, wind, UV, Electra, gas, water and much more. Notifications/Alerts can be sent to any mobile device.

The system can run on Linux, Mac OS, Windows on x86 platform, but also on 32-bit and 64-bit ARM Linux boards such as Raspberry Pi and Cubieboard with just 256MB memory recommended, and 200MB free hard disk space. It can also generate charts from the data like the ones below.

Click to Enlarge

On top of that, the forums appear to be very active, and the last stable version was released in November 2016, and the last beta release yesterday according the download page.

I’m going to take it slow, so today I’ve just tried to install it on NanoPi NEO since it’s compact and runs Linux. However, it does not appear to be officially supported by Domoticz, so we’ll have to see whether it’s possible to install it on the board.

Domoticz is not a Linux distributions but a framework, so first we need to install a Linux distributions on the board, and the obvious choice for NanoPi NEO is to use the latest Armbian release either Debian Jessie or Ubuntu Xenial.

I downloaded Debian, extracted the image, and flashed it to a micro SD card on a Ubuntu computer:

Replace /dev/sdX with your own SD card device, which you can find with lsblk command.  If you are a Windows user, you can flash the firmware like you’d do for a Raspberry Pi using Win32DiskImager after uncompress Armbian firmware.

Now we can insert the micro SD card into the board, and connect the power to start the board. If you have not connected the serial console to your board, please be patient for the first boot as the system may take around 3 to 4 minutes to boot before you can login to it, as it expands the micro SD card to full capacity, and creates a 128MB emergency SWAP file.

Once it’s done we can login through the serial console or SSH using root / 1234 credentials. The first time, you’ll be asked to go through the first setup, changing the root password, and creating a new user with sudo privileges.

So now that we have Linux running on the board, and after login again as the new user, we can follow the instructions for Raspberry Pi board and other ARM boards to install domoticz with a single command line that works on systems running Debian/Ubuntu:

After a minute or two, as the system update the packages, and download domoticz, the setup wizard should start.

At some points we’ll need a fixed IP address, either by configuring Linux with static IP, or setting a permanent IP linked to the board MAC address in the router. The second option is usually my favorite option. Nevertheless, let’s click on OK to proceed.

You’ll be asked whether you want to enabled HTTP or/and HTTPS access. I selected both for now, but it’s probably a good idea to only select HTTPS for better security.Next is the HTTP port number set to 8080 by default, followed by the HTTPS port number to 443 by default (no screenshot), and finally the installation folder which defaults to ~/domoticz. You should now have reached the Installation Complete! window, and you can click Ok to exit the installation wizard.
Wow.. That was easy, and no errors. But does it work? Let’s access https://192.168.0.110:443 from a web browser.

We have a “Your Connection is not secure” error, but it’s expected as Domoticz simply generated a self-certificate, you can safely add exception to your browser to avoid this issue next time. Your data will still be encrypted, but if you plan to access your Domoticz setup from the Internet, you should probably install an other certificate using Let’s Encrypt certificate authority for example.
Once we have added an exception to the web browser we can indeed access Domoticz web interface, so the installation worked, but it will only show “No favorite devices defined…” Again that’s normal, because we need to configure it for example by clicking on the Hardware link.

Adding Hardware to Domoticz – Click to Enlarge

This will allow you to configure the system with MQTT, local I2C sensors, all sort of gateways, and even Kodi Media Center.  I’m pretty sure all devices working over the network or USB should work, but things like “Local I2C Sensors” which may be connected directly to the board may or may not work. Anyway, that looks promising, but I’ll stop here for today, as I have a lot more to study before going further, including upgrading Sonoff firmware, and configuring vThings CO2 monitor for Domoticz.

Getting Started with Onion Omega2+ LEDE WiFi IoT Board and Expansion Dock

January 16th, 2017 39 comments

Onion Omega2 LEDE (OpenWrt fork) WiFi board is powered by Mediatek MT7688 MIPS SoC, targets IoT projects, and sells for as low as $5. There are actually two versions: Omega2 with 64MB RAM, 16MB flash, and Omega2+ with 128MB RAM, 32MB flash and a micro SD slot. Onion sent me the latter for review, together with an expansion dock that allows powering up the board though USB , and adds a USB host port, an RGB LED, buttons, and access to GPIO via a female header. In this quick start guide, I’ll start by taking some unboxing pictures, and then report my experience following the documentation to configure the board, blink the RGB LED, and control a LED on a breadboard using a GPIO from the header.

Onion Omega2+ Unboxing

I received the two boards in their respective package, and which are both stored in anti-static bags.

Click to Enlarge

Click to Enlarge

Let’s check Onion Omega2+ board first. The top includes a chip antenna and an u.FL connector for an external antenna, as well as the main components covered by a shield with some info like FCC ID, and the MAC address with the last four digits (hexits?) in bold since they are used to access the board. The bottom of the board are two rows of headers, and a micro SD card slot. There’s also a footprint for another header or connector, but I could not find out the details.

Click to Enlarge

Click to Enlarge

Next up is the dock. We have a 2x 15-pin female header with clear marking for the pins that include power signals, GPIOs, I2C, UART, and USB.

Click to Enlarge

Click to Enlarge

The button on the top is for power, the one at 45 angle is the reset button, and we also have a micro USB port for power, a USB port for storage, an RGB LED, and the header for Onion Omega2 board.

Click to Enlarge

Click to Enlarge

Plugging Omega2 into the dock is very easy, and the only thing you have to check is that it is inserted correctly.

Onion-Omega2-vs-LinkIt-Smart-7688Onion Omega2+ is not my first Mediatek MT7688 board, as I’ve reviewed LinkIt Smart 7688 too, and took side-by-side picture of both boards for comparison. Omega2+ is smaller, but LinkIt Smart board already include a micro USB port for power.

Initial Setup for Onion Omega2 and Expansion Dock

I normally test the documentation as much as I test the board, and after a web search, I ended up on that Getting Started page. However, it was for Onion Omega, the first version of the board introduced in 2015, and while the instructions are similar, they are not quite the same. Finally, I found the actual Omega2 Wiki, and could successful complete the setup with some efforts.

I’ll be using a computer running Ubuntu 16.04 to access the board, but it also works with Windows with Bonjour Service, and Mac OS X.

The Zeroconf services is needed to play with the board unless you access the board directly with its IP, but it’s normally already installed in Linux distriutions, so we are good to go. First we need to connect a micro USB to USB cable to a power source like the USB port of your computer, and turn on the board with the power switch.

Click to Enlarge

Click to Enlarge

At first both the RGB LED on the dock and Omega2+ LED will turn on for a short time, after which the RGB LED will turn off, and Omega2 LED will blink for a few seconds, and once the LED stops blinking and remains solid the boot is done.

Omega-Onion2-Access-Point

You should then see an new “Omega-XXXX” access point in your WiFi networks, where XXXX is the last digits of your MAC address shown on bold on the board. We now need to connect to the access point using password: 12345678

Omega2 Web Configuration

One it’s done, open a web browser and go ti http://192.168.3.1 or http://omega-XXXX.local/ to access Omega2 Setup Wizard.

Omega-Onion2-Setup-Wizard

Click Start to login with the default credentials (username: root ; password:onioneer), and the next page will let you connect the board to your WiFi router.Omega-Onion2-WiFi-Configuration

Selection you ESSID, input you WiFi password. and clikc on “Configure WiFi“. Omega-Onion2-Cloud

The wizard offers you to register your board to the cloud, but this is completely optional, and you could simply select Skip Step to go to the next step (firmware update). But I tried to registered the device to the cloud for this review.Onion-Cloud-RegistrationYou’ll need to provide your name, an email address and a password to register an account first.Onion-Omega2-Cloud-NameYou’ll then be asked for a device name and a description to confirm the registration.Onion-Cloud-Connection-failedSadly this step failed and I got the window above. Clicking on the red cross button did nothing. If I login to the cloud service, I can see the board listed, but detected as offline. I’m not the only one to have this issue, and Onion developers are looking into it.

Onion-Omega2+-Firmware-Update-ConsoleSo instead I went to the next step to upgrade the firmware and install Console web-based virtual desktop.Onion-Omega2-Firmware-Download-StuckThis also failed as the progress bar did not move at all, and I waited for around 20 minutes. I could also see my router’s DHCP server gave an IP address to the board, so it should have been able to connect to the Internet.

Omega2 Command Line Configuration

So I used to backup configuration method, using the command line as explained in the documentation. You just need to SSH the board as root with the same password as in the web configuration (onioneer):

Note Ω-ware firmware version is 0.1.5 b130.

wifisetup allows you to scan the network, and connect the board to your router:

Good. Firmware update failed in the web setup wizard, but we can retry it with oupgrade command line:

The firmware could be downloaded, and it looked like the system rebooted as I lost access to SSH terminal. The LED was still on for a while after it happened, then the LED went off (forever), at least longer than the 15 seconds listed in the documentation, and in that case they explain you need to power cycle the board. I used the power switch on the expansion dock to do so.

The board LED blinked for a pretty long time (maybe 2 minutes), but eventually it stopped and remained solid, and I could login to the board:

The firmware was updated to version 0.1.7 b139, so all good even though the whole setup did not go 100% smoothly. In case something goes really wrong and you can’t access the board at all, you could try to do a Factory Restore by pressing and holding the reset button for 10 seconds then releasing it.

Omega2 LEDE System Info

Since we’re done with the configuration, let’s quickly check some system info:

So we have a relatively recent kernel (Linux 4.4), 24.4MB space available to the user, 125664 KB total memory, and a MIPS 24KEc processor…

Controlling Omega2’s Dock RGB LED (via PWM)

We can start playing with the GPIO on the board starting with the RGB LED on the dock  that should be connected to pin 15, 16 and 17. The documentation explains expled sample can be used for this and we can see the R, G, B hexadecimal values. I want to show red color only, and I set blue and green to zero:

Oops, segfault. Let’s try something else like a pinkish color:

It runs, but the RGB LED remains off. It’s not an hardware problem since the RGB LED turns on at boot time. expled is actually a bash script that can be found in /usr/bin/expled and calls “fast-gpio“program which access GPIOs directly without using sysfs. Maybe it’s another firmware issue.

Controlling Omega2 GPIOs – LED example

In order to play with the expansion header, I connected a 5V LED to a breadboard together with two resistors and a transistor (for 3.3 to 5V conversion), and connected it to pin 1 on the header.

Click to Enlarge

We’ve already seen fast-gpio tools in expled script, but I used another GPIO tools for the LED, namely gpioctl that relies on sysfs.

We first need to set the GPIO pin as an output pin using the dirout command (dirin would set it as an input):

We can now turn the LED on by setting GPIO 1 to HIGH with dirout-high option:

The get command above will check the value of the pin. The LED did turn on as it should, and we can turn it off with dirout-low option:

Success.

If you want to use multiplexed pin with I2C, SPI, UART, PWM, I2S… you’ll need to check out omega2-ctrl program. I have not tried it for this guide to keep it short.

Onion Omega2 and BreadBoards

Many similar small IoT board are designed to fit on a breadboard, but Onion Omega2 board’s header pins are using 2 mm pitch, not 2.5 mm pitch, so they can’t be used with a breadboard directly. Instead, you’d have run wires from the board to the breadboard or purchase a BreadBoard Dock as pictured below.

If you do not have a dock, or breadboard expansion board, you can still power the Omega2 module/board using a 3.3V power source for example with a regulator such as LD1117, or something like YwRobot MB102 breadboard power supply.

If you are interested in getting a board, you may have to wait as while Omega Expansion Dock sells for $14.99 on Onion store, Omega2 boards are not listed yet. For reference, Omega2 board went for $5, and Omega2+ board for $9 on Kickstarter. [Update: While the Kickstarter campaign is now finished, you can still get on Indiegogo for the same price, and that includes shipping].

Self-hosted OpenGL ES Development on Ubuntu Touch

January 15th, 2017 4 comments

Blu wrote BQ Aquaris M10 Ubuntu Edition review – from a developer’s perspective – last year, and now is back with a new post explaining how to develop and deploy OpenGL ES applications directly on the Ubuntu Touch tablet.

Ever since I started using a BQ M10 for console apps development on the go I’ve been wanting to get something, well, flashier going on that tablet. Since I’m a graphics developer by trade and by heart, GLES was the next step on the Ubuntu Touch for me. This article is about writing, building and deploying GLES code on Ubuntu Touch itself, sans a desktop PC. Keep that in mind if some procedure seems unrefined or straight primitive to you – for one, I’m a primitive person, but some tools available on the desktop are, in my opinion, impractical on the Touch itself. That means no QtCreator today, nor Qt, for that matter.

The display of any contemporary Ubuntu Touch device is powered by Mir – a modern compositor/surface manager taking care of all (rectangular-ish) things eventually appearing on screen. We won’t be delving much into Mir beyond obtaining an EGL context (EGL being the binding layer between GLES and the native windowing system). But enough ado – let’s get to work.

Preparations for doing GLES on a Ubuntu Touch box:

The above, as of the time of this writing, should provide you with gcc/g++-4.9, make and gdb-7.9, among other things. The last package and its dependencies provide you with up-to-date Mir headers. Git comes out of the box, IIRC, but if it’s missing just apt-get it.

We need a primer to step on, so here’s my adaptation of Don Bright’s Mir/GLES adaptation of Joe Groff’s OpenGL tutorials, using Daniel van Vugt’s Mir/EGL examples (yes, that’s a quite a chain-work):

I’ve taken the liberty to expand on the work of those gentlemen by bringing the Mir integration up to date, handling Touch’s novelty Desktop Mode and throwing in my own dusty GLES sample code, for good measure.

To build and install the primer, just do:

That will provide you with an original police-car flashing-lights primer. An alternative primer featuring tangential-space bump-mapping can be built by passing arg ‘guest’ to the build script:

Both versions of the primer use a fundamentally identical interface — a resource-initialization procedure and a frame-drawing procedure, so it’s not much of an effort to use the respective routines from either primers in the framework of the host app hello.cpp, and thus get a running render loop.

A few words about the peculiarities of the GLES development for Ubuntu Touch. It took me some time to show anything on screen, despite the fact I had a valid draw context and a render loop soon after the primer was building successfully. The reason is Unity8 on the Touch will not simply let you run a window-painting app from the terminal – you would get your Mir and EGL contexts alright, but the target surface will never be composited to the screen of the device upon eglSwapBuffers() unless you take certain actions. You have two alternatives here:

  • Produce a valid Click package from your app and subsequently install that to the Apps pane (what our build script does), where you can launch from an icon, or…
  • Use a launcher app to start your window app (info courtesy of Daniel van Vugt):

Unfortunately the second (much quicker and convenient) approach is not currently usable due to a bug, so we’ll stick with the first. Any command-line args we’d want to pass to the app will need to be written to the app’s .desktop file, which can be found at the official app location after installation:

In that file, set the desired args on the ‘Exec’ line, like this:

Another peculiarity was that in Desktop Mode the app window does a classical ‘zoom to full size’ animation at start. Nothing extraordinary in that, if not for the fact that the Mir surface itself resizes along with the window. Now, a default viewport in a GLES context spans the geometry of the target surface at the time of its creation, which, in our case, is the start of the window-zoom animation, with its tiny surface geometry. One needs to wait for the zoom animation to finish, and then set the viewport geometry to the final geometry of the Mir surface, or live with a post-stamp-sized output in the lower left corner of the window, if the viewport is left unchanged.

Once we get past those teething hurdles we actually get quite a nicely behaving full-screen app on our hands – it composites smoothly with all other Ubuntu Touch desktop elements like the Launcher tab at the desktop’s left edge and the pull-down Indicator pane on right (see screenshot). Our app even does live output to the Scopes selector screen (i.e. the task-switching screen) — behold the miracles of modern-day screen compositors! ; )

Click for Original Size (1920×1080)

But hey, don’t just take my word for it – try out GLES coding on a Ubuntu Touch device – you have the basics covered:

  • App’s rendering loop and the entirety of the flashing-screen primer are in hello.cpp
  • Mir context creation and subsequent EGL context binding are in eglapp.cpp
  • Bump-mapping primer is entirely in app_sphere.cpp
  • Various helpers are spread across util_* TUs and hello.cpp
  • All files necessary for the generation of the Click package are in resource folder.

In conclusion, self-sustained development on the Ubuntu Touch is a perfectly viable scenario (take that, iOS!). Moreover, the GPU in the BQ M10 turned out to have a very nice modern GLES3 (3.1) stack – see excerpts from the app logs below. Actually, this is my first portable device with a GLES 3.1 stack, so I haven’t started using it properly yet — the GLES2 primer above doesn’t make use of the new functionality.

If I have to complain about something from the development of this primer, it’d be that I couldn’t use my arm64 code on the primer, since there are only armhf (32-bit) EGL/GLES libraries available for the Touch. So 64-bit code on the Ubuntu Touch remains in console land for now.

Excerpts from the primer logs:

egl version, vendor, extensions:

1.4 Android META-EGL
Android
EGL_KHR_get_all_proc_addresses EGL_ANDROID_presentation_time EGL_KHR_image EGL_KHR_image_base EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_renderbuffer_image EGL_KHR_fence_sync EGL_KHR_create_context EGL_ANDROID_image_native_buffer EGL_KHR_wait_sync EGL_ANDROID_recordable EGL_HYBRIS_native_buffer2 EGL_HYBRIS_WL_acquire_native_buffer EGL_WL_bind_wayland_display

gl version, vendor, renderer, glsl version, extensions:

OpenGL ES 2.0 (OpenGL ES 3.1)
ARM
Mali-T720
OpenGL ES GLSL ES 3.10
GL_EXT_debug_marker GL_ARM_rgba8 GL_ARM_mali_shader_binary GL_OES_depth24 GL_OES_depth_texture GL_OES_depth_texture_cube_map GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_EXT_read_format_bgra GL_OES_compressed_paletted_texture GL_OES_compressed_ETC1_RGB8_texture GL_OES_standard_derivatives GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_sync GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_required_internalformat GL_OES_vertex_array_object GL_OES_mapbuffer GL_EXT_texture_format_BGRA8888 GL_EXT_texture_rg GL_EXT_texture_type_2_10_10_10_REV GL_OES_fbo_render_mipmap GL_OES_element_index_uint GL_EXT_shadow_samplers GL_OES_texture_compression_astc GL_KHR_texture_compression_astc_ldr GL_KHR_texture_compression_astc_hdr GL_KHR_debug GL_EXT_occlusion_query_boolean GL_EXT_disjoint_timer_query GL_EXT_blend_minmax GL_EXT_discard_framebuffer GL_OES_get_program_binary GL_OES_texture_3D GL_EXT_texture_storage GL_EXT_multisampled_render_to_texture GL_OES_surfaceless_context GL_OES_texture_stencil8 GL_EXT_shader_pixel_local_storage GL_ARM_shader_framebuffer_fetch GL_ARM_shader_framebuffer_fetch_depth_stencil GL_ARM_mali_program_binary GL_EXT_sRGB GL_EXT_sRGB_write_control GL_EXT_texture_sRGB_decode GL_KHR_blend_equation_advanced GL_OES_texture_storage_multisample_2d_array GL_OES_shader_image_atomic

GL_MAX_TEXTURE_SIZE: 8192
GL_MAX_CUBE_MAP_TEXTURE_SIZE: 4096
GL_MAX_VIEWPORT_DIMS: 8192, 8192
GL_MAX_RENDERBUFFER_SIZE: 8192
GL_MAX_VERTEX_ATTRIBS: 16
GL_MAX_VERTEX_UNIFORM_VECTORS: 1024
GL_MAX_VARYING_VECTORS: 15
GL_MAX_FRAGMENT_UNIFORM_VECTORS: 1024
GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS: 48
GL_MAX_VERTEX_TEXTURE_IMAGE_UNITS: 16
GL_MAX_TEXTURE_IMAGE_UNITS: 16

RetrOrangePi 3.0 Retro Gaming & Media Center Firmware Released for Orange Pi H3 Boards and Beelink X2 TV Box

December 28th, 2016 9 comments

RetrOrangePi is a Linux distribution based on armbian transforming Allwinner H3 boards – mostly Orange Pi boards, but also Banana Pi M2+ and NanoPi boards – into entertainment centers to play retro games, and watch/listen media files (videos/music) using Kodi. If you don’t have a development board, or would prefer a complete solution with casing and power supply, Beelink X2 TV box is also supported. The developers had been recently working on rectifying some GPL issues, and they have released RetrOrangePi 3.0 images right before Christmas.

retrorangepi

RetrOrangePi 3.0 changelog and key features:

  • Full Armbian 5.23 Jessie Desktop version with kernel 3.4.113 (backdoors fixed)
  • Slim version 1st release (less than 2 GB) coming soon
  • OpenELEC (Kodi Jarvis 16.1) with CEC support by Jernej Škrabec
  • RetroPie-Setup version 4.1
  • New Kodi Krypton beta6 version
  • New emulationstation-ROPI branch forked from jacobfk20 with gridview, on screen keyboard with easy wifi config and storage check with additional features added by ROPi team: display settings, OpenELEC / Desktop launcher and background music switcher integrated into main menu.
  • New Plug n’ Play feature – USB roms autoload (reads from /media/usb0) (buggy)
  • New dummy roms feature (most common platform shown)
  • New splash video on 1st boot by Rafael Spirax
  • New default splashscreen (from Libretro)
  • New custom ES splashscreen by Francois Lebel @MagicFranky
  • OpenELEC ROPI addon already installed
  • Retroarch with XMB menu driver (Lakka)
  • Better looking video with bilinear filtering (smoothness) or scanlines by default
  • Most retroarch cores updated (FBA, PCSX etc)
  • New and improved content:
    • AdvanceMAME (newer romset, more compatibility, better performance in some games: Elevator Action Returns, Street Fighter the Movie, Star Wars Arcade, Judge Dredd, Sega Sonic The Hedgehog etc)
    • Amiga (FS-UAE emulator, fullscreen now, diskette sound, launcher)
    • Atari 5200
    • Atari 8bit (models 400 800 XL XE)
    • Coco / Tandy
    • Colecovision (ColEm emu Custom Coleco BlueMSX core)
    • Creativision
    • Daphne (Philips Cdi emulator)
    • Dosbox (GLES version)
    • Dreamcast (fixed reicast-joyconfig)
    • Duke Nukem port (fixed tint color)
    • Game and Watch (fixed shortcuts)
    • Intellivision
    • OpenMSX (with .dsk support) PPSSPP (new version 1.3 from odroid repo)
    • TI99/4A (Texas Instruments)
    • Wolfenstein3D port

There are two ways to download the images:

  • BitTorrent – 16.0 GB download with images for all boards
  • Main server (http) – 1.6 GB compressed firmware image for your board.

If you download from the main server, you’ll get a warning saying you can’t sell hardware pre-installed with the image:

RetrOrange Pi is a non profit project.
It consists of a basic Retropie setup with most Libretro cores on top of an Armbian Jessie Desktop version pre-installed.
It includes an OpenELEC fork as well.
Much of the software included in the image have non-commercial licences. Because of this,
selling a pre-installed RetrOrange image is not legal, neither is including it with your commercial product.
As it relies on other people’s work with our own features, we won’t be offering any help in customizations to avoid rebranding or reselling.

It will be interesting to see what happens with RetroEngine Sigma project on Indiegogo that is very likely based on RetrOrangePi image for Orange Pi Lite board.

Anyway, since BitTorrent download was very slow, I downloaded RetrOrangePi-3.0.Orangepione.img.tar.gz from the main server for my $3.69 Orange Pi One board (there was a promo in September), extracted it, and flashed it to a 32GB card (8GB is enough) in Linux:

Replace sdX by your own SD card device in the 3rd command above. You can also do this in Windows with Win32DiskImager. Once it is done, insert the micro SD card in your board or TV box, prepare a gamepad, and connect all relevant cables.

orange-pi-orange-gaming

If you have connected the serial console (completely optional), or want to access the system through ssh, you can login with pi/pi or root/orangepi credentials:

Most people will just follow the instructions on the TV. We’ll get through a bunch of animation and logos during the boot.Note: Please ignore the vertical lines on the photos, as there’s just an issue with my TV.

retrorangepi-3-0-logo
The first time the system will resize the SD card to make use of the full SD card capacity, and generate SSH keys.
retrorangepi-installationOne more “Loading…” logo…

retrorangepi-loading

If you have connected a gamepad (highly recommended), you’ll be ask to configure the keys. Tronsmart Mars G01 gamepad was automatically detected, and I could easily set all keys up.

retrorangepi-gamepad-configurationOnce all is well and done, you’ll get to the main menu to select emulator or Kodi.

retrorangepi-user-interfaceMost emulators do not come with ROMs due to license issues, so you’d have to find the ROMs yourself, and install them via a USB drive, or copy them directly into the board over the network, for example with scp. If you want to try to play some games straightaway, you can do so by going to the PORTS sections with 13 games available including Doom, Quake, Wolfenstein 3D, CannonBall, Duke Nukem 3D, Super Mario War, etc…
retrorangepi-ports-pre-installed-games
I tested shortly tested Wolfenstein 3D and Quake, as well as launched Kodi 17 (Beta 6) in the demo video below.

vThings WiFi CO2 Monitor Quick Start Guide

December 28th, 2016 5 comments

I’ve already checked out vThings CO2 Monitor hardware and we’ve seen it’s based on ESPrino ESP8266 board, and my model includes CM1106 CO2 sensor and BMP180 temperature and pressure sensor. I’ve now installed it in my kitchen, about 3 to 4 meters from the gas stove, and getting data to ThingSpeak.

vair-monitor-co2-sensor

The door and window of my kitchen are open all day, and the wall have ventilation holes. That’s important for CM1106 sensor since it auto calibrates every 3 days in clear air. If you plan to use such sensor in a closed environment, you should buy Vthings with CM1102 CO2 sensor that costs more, but does not require calibration.

Since all WiFi systems I’ve just so far starting AP mode for configuration, I first looked for an access point, but… nothing… Then I decided to read the documentation (might be useful at times), and the monitor is actually configured via a Chrome (desktop only) add-on through USB. There are three types of devices made by vair-monitor, and I first used  vThings Configuration Utility add-on, but eventually found out I had to use vThings – Dual Beam Configuration Utility.

vthings-chrome-apps

vThings Configuration vs Things – Dual Beam Configuration Utility

I used Ubuntu (Linux), but if you are using a Windows or Mac computer, you’ll need to install drivers first. Once you’ve connected the monitor through USB and started “vThings- Device Configuration Tool” the following windows should be shown.

Click to Enlarge

Click to Enlarge

The fist thing to do is to connect the monitor to your WiFi router by entering its SSID and password, and click on Set WiFi.

Click to Enlarge

Click to Enlarge

It should connect to your router, and the first time updated the firmware automatically. Wait a couple of minutes for it to complete, and you can go to the next step to configure one or more of the following Public, Private or Generic services:

Public Private Generic
BeeBottle DomoticGa HTTP
Blynk.cc DomoticZ MQTT
dweet.io FHEM RF 433/315
EmonCMS Homeseer
ThingSpeak HomeAssistant
UbiDots JeeDom
OpenHAB
Pimatic

I decided to go with ThingSpeak since I got familiar with it while writing Sonoff POW tutorial.

Click to Enlarge

Click to Enlarge

Select the data provided by the sensors inside your vthings Co2 monitor, in my case CO2 levels, temperature, and pressure, and nothing else, or connection will fail, as I found out when I used 4 default fields including humidity, and ThingSpeak was not updated at all. You’ll also need ThingSpeak API write key, that you can get my create a channel on ThingSpeak.com as shown below.

thingspeak-co2-monitor-thingspeak-channel-configuration

Once the channels is create on ThingSpeak website, and you’ve added the API write key in vThings Device Configuration Tool, you could go to Generic Services->HTTP and notice an HTTP request has been created, so if you have installed ThingSpeak locally, you could change api.thingspeak.com to your own IP address.

Click to Enlarge

Click to Enlarge

By default the data will be updated every 120 seconds, but you can change that in Settings->Update Interval. Once configuration is done, you can unplug it from your PC, and connected to the location you want to monitor. vThings Device Configuration Tool requires a USB connection to find the device, it can not find it over WiFi, so if you want to change configuration, you’ll need to connect it back to your computer. There’s a function to (auto)start a webserver in vESPrino, but it did not seem to work for me.

After a few hours or minutes depending on your update internal you should get some nice charts on ThingSpeak with CO2 levels, temperature and pressure, or other data based on the sensors you’ve selected while purchasing the hardware.

Click to Enlarge

Click to Enlarge

The channel is public if you are interested/curious in seeing the data. ThingSpeak will show 60 samples (2 hours in my case) by default, but let’s see what happened over the last 12 hours with CO2 levels.

Click to Enlarge

Click to Enlarge

The CO2 levels started at about 500 to 600 ppm while I did the configuration in my office (windows closed), and dropped to around 404 ppm once I installed in the kitchen. That value correspond roughly to the current CO2 ppm value in the atmosphere (in Hawaii). Three times around 18h00 people warmed food and CO2 jumped to around 500 ppm. During the night, CO2 levels slowly increased to 480 ppm, likely because of the plants cycle (producing oxygen during the day, and carbon dioxide during the night). This morning CO2 levels spiked at around 900 ppm when cooking right after 6am and 8am.

That’s all fun, but is there a real benefit to measuring CO2 levels in your house? In the kitchen I could probably trigger an alert over 1,500 ppm in which case it may mean something is burning, but smoke detectors are much cheaper and better suited to the task. Vladimir Savchenko, vThings developer, found a study claiming that high CO2 levels may decrease creative thinking and lead to bad sleep, so he used vThings CO2 monitor in his bedroom and discovered CO2 levels reached close to 4,000 ppm, and that just open the door or window would greatly reduce the concentration of the gas.

sleepwithcloseddoortext-co2-levelsvThings CO2 monitor does not only monitor CO2 levels as we’ve seen above, as temperature, humidity, and/or pressure sensor can be included in the case, as well as a PM2.5 & PM10 laser dust sensor.

vThings CO2 Monitor v3 is sold for 60 Euros with CM1106 CO2 sensor, and more if you use a better CO2 sensor, or add extra environmental sensors. 135 Euros would get you a top of line monitor with a laser dust sensor, CDM7160 CO2 sensor, temperature and humidity sensor, and RF connectivity.

Something is “Eating” my Android TV Box Internal Storage!

December 21st, 2016 6 comments

No, it’s not a joke. I’ve been playing for a while with Eweat R9 Plus Android TV box after inserting a 1TB hard drive in the SATA bay, but while in most other reviews, the apps and files used for testing are just taking around 3 GB, the 9.31 GB storage in that device was completely full, and I have yet to install some apps, and copy some files part of my testing procedure…

android-storage-fullI did not immediately find out about the “storage full” issue, as I first I just noticed Kodi would not start anywhere, and the default video app would just reboot after adjusting the volume… But let’s check what takes all that space…

Click to Enlarge

Click to Enlarge

I had to uninstall 3Dmark in order to be able to take screenshots, which explains why I have a bit more space now, but the system reports all that space (and more???) is taken by apps. I clicked on Apps, to find out Es File Explorer was the worse offender but with only “285 MB” used. That did not add up, so I started to adb shell to try to find the exact files:

Media directory takes 3.1GB and data directory 4.5GB. After some more checking, I eventually the space mostly is taken by two large “external.db” databases for com.android.providers.media:

Reading from Android’s developer website:

Android Provider provides convenience classes to access the content providers supplied by Android.

Android ships with a number of content providers that store common data such as contact informations, calendar information, and media files. These classes provide simplified methods of adding or retrieving data from these content providers.

I have a lot of files (millions) on my SATA hard drive, many of them not media files, but the box is probably scanning the files and storing some metadata for faster access or/and search within Android. Earlier this year, I had a problem with Media Scanner in Zidoo X1 II TV box, as it slowed down the system greatly while scanning a USB hard drive and affected video playback. I solved the issue with media.Re.Scan: app which allowed me to stop the scanning process at the time. We don’t really need the app however, and since we don’t have space, you’d have to uninstall other apps to install the utility. Instead go to Settings->Apps, select “Show System” on the top right, and scroll down until you find “Media Storage”.

Click to Enlarge

Click to Enlarge

Now click on Media Storage, select Disable, and click on “Storage” in order to finally select “Clear Data” to free up (a lot of) space.

android-media-storage-clear-data

You can now have fun with your TV box with plenty of storage, and a lot less (apparent) bugs…android-storage-usbHappy to have gotten rid of Media Storage (aka The grinch) right before Christmas :).

Facebook Zstandard “zstd” & “pzstd” Data Compression Tools Deliver High Performance & Efficiency

December 19th, 2016 11 comments

Ubuntu 16.04 and – I assume – other recent operating systems are still using single-thread version of file & data compression utilities such as bzip2 or gzip by default, but I’ve recently learned that compatible multi-threaded compression tools such as lbzip2, pigz or pixz have been around for a while, and you can replace the default tools by them for much faster compression and decompression on multi-core systems. This post led to further discussion about Facebook’s Zstandard 1.0 promising both smaller and faster data compression speed. The implementation is open source, released under a BSD license, and offers both zstd single threaded tool, and pzstd multi-threaded tool. So we all started to do own little tests and were impressed by the results. Some concerns were raised about patents, and development is still work-in-progess with a few bugs here and there including pzstd segfaulting on ARM.

Zstd vs Zlib Compression Ratio vs Speed

Zstd vs Zlib Compression Ratio vs Speed

Zlib has 9 levels of compression, while Zstd has 19, so Facebook has tested all compression levels and their speed, and drawn the chart above comparing compression speed to compression ratio for all test points, and Zstd is clearly superior to zlib here.

They’ve also compared compression and decompression performance and aspect ratio for various other competing fast algorithms using lzbench to perform this from memory to prevent I/O bottleneck from storage devices.

Name Ratio C.speed D.speed
MB/s MB/s
zstd 1.0.0 -1 2.877 330 940
zlib 1.2.8 -1 2.730 95 360
brotli 0.4 -0 2.708 320 375
QuickLZ 1.5 2.237 510 605
LZO 2.09 2.106 610 870
LZ4 r131 2.101 620 3100
Snappy 1.1.3 2.091 480 1600
LZF 3.6 2.077 375 790

Again everything is a comprise, but Zstd is faster than algorithms with similar compression ratio, and has a higher compression ratio than faster algorithm.

But let’s not just trust Facebook, and instead try ourselves. The latest release is version 1.1.2, so that’s what I tried in my Ubuntu 16.04 machine:

This will install the latest stable release of zstd to your system, but the multi-thread is not build by default:

There are quite a lot of options for zstd:

Since we are going to compare results to other, I’ll also flush the file cache before each compression and decompression using:

I’ll use the default settings to compress Linux mainline directory stored in a hard drive with tar + zstd (single thread):

and pzstd (multiple threads):

Bear in mind that some time is lost due to I/O on the hard drive, but I wanted to test a real use case here, and if you want to specifically compare the raw performance of compressor you should use lzbench. Now let’s decompress the Zstandard tarballs:

My machine is based on an AMD FX8350 octa-core processor, and we can clearly see that by comparing real and user time, the test is mostly I/O bound. I’ve repeated those test with other multi-threaded tools as shown in the summary table below.

Compression Decompression File Size (bytes) Compression Ratio
Tools Time (s) “User” Time (s) Time (s) “User” Time (s)
ztsd 130.056 91.608 45.124 21.26 1,881,020,744 1.48
pzstd 58.929 86.56 38.175 23.39 1,883,697,296 1.48
lbzip2 84.216 353.84 37.109 167.416 1,855,837,345 1.50
pigz 61.121 121.332 34.36 15.26 1,903,915,372 1.47
pixz 177.596 1233.88 36.24 78.116 1,782,756,524 1.57
pzstd -19 275.361 1939.536 26.85 21.832 1,794,035,552 1.56

I’ve included both “real time” and “user time”, as the latter shows how much CPU time the task has spent on all the cores of the system. If user time is large that means the task required lots of CPU power, and if a task completes in about the same amount of “real time”, but a lower “user time”, it means it was likely more efficient, and consumes less power. pigz is the multi-threaded version of xz algorithm relying on lzma compression which delivers a high compression ratio, at the expense of longer compression time, so I also run pzstd with level 19 compression to compare:

Zstandard compression ratio is similar to the one of lbzip2 with default settings, but compression is quite faster, and much more power efficient. Compared to gzip, (p)zstd offers a better compression ratio, against with default settings, and somewhat comparable performance. pixz offers the best compression ratio, but takes a lot more time to compress, and uses more resources to decompress compared to Zstandard and Pigz. Pzstd with compression level 19 takes even more time to compress, and is getting close to pixz compression, but has the advantage of being much faster to decompress.

Compress & Decompress Files Faster with lbzip2 multi-threaded version of bzip2

December 16th, 2016 30 comments

Bzip2 is still one of the most commonly used compression tools in Linux, but it only works with a single thread, and I’ve been made aware that lbzip2 allows multi-threaded bzip2 compressions which should lead to much better performance on multi-core systems.

Tar with lbzip2 on a 8-core Processor - Click to Enlarge

Tar with lbzip2 on an 8-core Processor – Click to Enlarge

lbzip2 was not installed by default in my Ubuntu 16.04 machine, but it’s easy enough to install:

I have cloned mainline linux repository on my machine, so let’s see how long it takes to compress the directory with bzip2 (one core compression):

9 minutes and 22 seconds. Now let’s repeat the test with lbzip2 using all 8 cores from my AMD FX8350 processor:

2 minutes 32 seconds. Almost 4x times, not bad at all. It’s not 8 times faster because you have to take into account I/Os, and at the beginning the system is scanning the drive, using all 8-core but not all full throttle. The files were also stored in a hard drive, so I’d assume the performance difference should be even more noticeable from an SSD.

We can see both files are about the same size as they should be:

I’m not exactly sure why there’s about 771 KB difference as both tools offer the same compression.

That was for compression. What about decompression? I’ll decompress the lbzip2 compressed file with bzip2 first:

2 minutes and 49 seconds. Now let’s decompress the bzip2 compressed file with lbzip2:

45 seconds! Again the performance difference is massive.

If you want tar to always use lbzip2 instead of bzip2, you could create an alias:

Please note that this will cause a conflict (“Conflicting compression options”) when you try to compress files using -j /–bzip2 or -J, –xz options, so instead of tar, you may want to create another alias, for example tarfast.

lbzip2 is not the only tool to support multi-threaded bzip2 compression, as pbzip2 is another implementation. However, one report indicates that lbzip2 may be twice as fast as pbzip2 to compress files (decompression speed is about the same), which may be significant if you have a backup script…

tkaiser also tested various compression algorithms (gzip, pbzip2, lz4, pigz) for a backup script for Orange Pi boards running armbian, and measured overall performance piping his eMMC through the different compressors to /dev/null:

pigz looks the best solution here (25.2 MB/s) compared to pbzip2 (15.2 MB/s). lbzip2 has not been tested, and could offer an improvement over pigz both in terms of speed and compression based on the previous report, albeit actual results may vary depending on the CPU used.