tutorials

Posted on by Jean-Luc Aufranc (CNXSoft) - 12 Comments on Facebook Zstandard “zstd” & “pzstd” Data Compression Tools Deliver High Performance & Efficiency

Facebook Zstandard “zstd” & “pzstd” Data Compression Tools Deliver High Performance & Efficiency

Ubuntu 16.04 and – I assume – other recent operating systems are still using single-thread version of file & data compression utilities such as bzip2 or gzip by default, but I’ve recently learned that compatible multi-threaded compression tools such as lbzip2, pigz or pixz have been around for a while, and you can replace the default tools by them for much faster compression and decompression on multi-core systems. This post led to further discussion about Facebook’s Zstandard 1.0 promising both smaller and faster data compression speed. The implementation is open source, released under a BSD license, and offers both zstd single threaded tool, and pzstd multi-threaded tool. So we all started to do own little tests and were impressed by the results. Some concerns were raised about patents, and development is still work-in-progess with a few bugs here and there including pzstd segfaulting on ARM. Zlib has 9 levels of […]

Posted on by Jean-Luc Aufranc (CNXSoft) - 30 Comments on Compress & Decompress Files Faster with lbzip2 multi-threaded version of bzip2

Compress & Decompress Files Faster with lbzip2 multi-threaded version of bzip2

Bzip2 is still one of the most commonly used compression tools in Linux, but it only works with a single thread, and I’ve been made aware that lbzip2 allows multi-threaded bzip2 compressions which should lead to much better performance on multi-core systems. lbzip2 was not installed by default in my Ubuntu 16.04 machine, but it’s easy enough to install:

I have cloned mainline linux repository on my machine, so let’s see how long it takes to compress the directory with bzip2 (one core compression):

9 minutes and 22 seconds. Now let’s repeat the test with lbzip2 using all 8 cores from my AMD FX8350 processor:

2 minutes 32 seconds. Almost 4x times, not bad at all. It’s not 8 times faster because you have to take into account I/Os, and at the beginning the system is scanning the drive, using all 8-core but not all full throttle. […]

Posted on by Jean-Luc Aufranc (CNXSoft) - 11 Comments on How to Use Sonoff POW ESP8266 WiFi Power Switch with MQTT and ThingSpeak

How to Use Sonoff POW ESP8266 WiFi Power Switch with MQTT and ThingSpeak

ITEAD Studio’s Sonoff is a family of cheap home automation products based on ESP8266 WiSoC, and I’ve already tested Sonoff TH16 wireless switch with a humidity and temperature sensor using the stock firmware and eWelink app for Android or iOS. It works, but up to recently it required a registration to a cloud service (the company will now allow use from the local network), and the source code is closed. So for the second device under review, namely Sonoff POW wireless switch with a power consumption monitor, I decided to install ESPurna firmware working on ESP8266 Sonoff devices and NodeMCU, as it’s open source, supports Sonoff POW natively, includes a web interface to control the device from the LAN, and includes an MQTT client. MQTT (Message Queuing Telemetry Transport) is a lightweight publish/subscribe messaging protocol used to control IoT sensors and devices, and it’s a popular method to gather data […]

Posted on by Jean-Luc Aufranc (CNXSoft) - 11 Comments on How to Install ThingSpeak in Ubuntu 16.04

How to Install ThingSpeak in Ubuntu 16.04

Last week-end I installed ESPurna open source firmware with MQTT server on Sonoff POW WiFi switch, and the next step is find a way to draw power consumption charts in some web based interface. We could do this in the IoT cloud with services like Xively or ThingSpeak, but since one of the goals of replacing the default firmware was not to rely on a proprietary cloud based solution, I decided to find a way to draw those chart in a local server, and it so happens that ThingSpeak is also open source with the code available on Github. Hardware platforms like NanoPi NEO / NEO Air, or Orange Pi Zero boards appear to be particularly well suited for the task of running an MQTT broker and Thingspeak, but at first I wanted to install ThingSpeak in my own Ubuntu 16.04 computer to have a try. As you can see […]

Posted on by Jean-Luc Aufranc (CNXSoft) - 8 Comments on Getting Started with Pine64 PADI IoT Stamp – Part 2: Serial Console, GCC SDK, Flashing & Debugging Code

Getting Started with Pine64 PADI IoT Stamp – Part 2: Serial Console, GCC SDK, Flashing & Debugging Code

PADI IoT Stamp module powered by Realtek RTL8710AF ARM Cortex M3 WiFi SoC is a potential competitor to Espressif ESP8266 modules.  Pine64, the manufacturer of the module, sent me their kit with a $2 IoT stamp, a breakout board, a USB to TTL debug board and a J-Link debug board. In the first part of the review I’ve shown the hardware and how to assemble PADI IoT stamp kit. In the second part I’m going to write a tutorial / getting start guide showing how to control the board with AT commands, build the firmware with GCC SDK, and finally demonstrate how to flash and debug the firmware with the J-Link debugger. The Quick Start Guide indicates you need to connect the USB to TTL debug board to UART2 instead of UART1 as I did on the very similar B&T RTL-00 RTL8710AF module, and set connection settings to 38400 8N1. […]

Posted on by Jean-Luc Aufranc (CNXSoft) - 76 Comments on How to Create a Bootable Recovery SD Card for Amlogic TV Boxes

How to Create a Bootable Recovery SD Card for Amlogic TV Boxes

I reviewed Rikomagic MK22 TV box about two weeks ago, and with the firmware I had, online firmware update was not enabled, and the company only released .IMG firmware for Amlogic USB Burning Tool, a windows-only tool that’s not well designed, and requires some procedure that vary slightly from boxes to boxes which in some cases forces to buy a male to male USB cable. I’ve now started reviewing R-Box Pro TV box also based on Amlogic S912 processor, online firmware update is not working either, and again I only managed to find .IMG firmware for the box on GeekBuying. However, I’ve been informed that “USB Burning Tool” firmware can now be flashed through a micro SD card, or USB flash drive with all recent Amlogic TV boxes, so I’ve tried this method instead of R-Box Pro, and decided to report my experience in this post following some instructions on […]

Posted on by Jean-Luc Aufranc (CNXSoft) - 1 Comment on Getting Started with RabbitMax Flex IoT and Automation Hat for Raspberry Pi

Getting Started with RabbitMax Flex IoT and Automation Hat for Raspberry Pi

At the beginning of the month I showed how to assemble RabbitMax Flex, a Raspberry Pi HAT compliant add-on board for Raspberry Pi boards with 40-pin header, that targets IoT and home automation project with its relay, IR transmitter and receiver, I2C headers for sensors, buzzer, RGB LED, and more.  Since I’ve already described the hardware, I’ve spend some time this week-end following the user’s guide to play around with the board using a Raspberry Pi 2 board, and try various features. The user’s manual explains that you need the latest version of Raspbian, but I’d not played with my Raspberry Pi 2 board for a while, so the kernel and firmware were quite old:

So the first thing I had to do was to upgrade Raspbian. There are basically two options to upgrade, either downloading and dumping the latest Raspbian firmware image to your micro SD card, and […]

Posted on by Jean-Luc Aufranc (CNXSoft) - 9 Comments on How to Write ESP8266 Firmware from Scratch (using ESP Bare Metal SDK and C Language)

How to Write ESP8266 Firmware from Scratch (using ESP Bare Metal SDK and C Language)

CNXSoft: This is a guest post by Alexander Alashkin, software engineer in Cesanta, working on Mongoose Embedded Web Server. Espressif’s ESP8266 had quite an evolution. Some may even call it controversial. It all started with ESP8266 being a WiFi module with a basic UART interface. But later it became clear that it’s powerful enough for embedded system. It’s essentially a module that can be used for running full-fledged applications. Espressif realized this as well and released an SDK. As first versions go, it was full of bugs but since has become significantly better. Another SDK was released which offered FreeRTOS ported to ESP. Here, I want to talk about the non-OS version. Of course, there are third-party firmwares which offer support for script language to simplify development (just Google for these), but ESP8266 is still a microchip (emphasis on MICRO) and using script language might be overkill. So what we […]