Archive

Posts Tagged ‘thingspeak’

vThings WiFi CO2 Monitor Quick Start Guide

December 28th, 2016 5 comments

I’ve already checked out vThings CO2 Monitor hardware and we’ve seen it’s based on ESPrino ESP8266 board, and my model includes CM1106 CO2 sensor and BMP180 temperature and pressure sensor. I’ve now installed it in my kitchen, about 3 to 4 meters from the gas stove, and getting data to ThingSpeak.

vair-monitor-co2-sensor

The door and window of my kitchen are open all day, and the wall have ventilation holes. That’s important for CM1106 sensor since it auto calibrates every 3 days in clear air. If you plan to use such sensor in a closed environment, you should buy Vthings with CM1102 CO2 sensor that costs more, but does not require calibration.

Since all WiFi systems I’ve just so far starting AP mode for configuration, I first looked for an access point, but… nothing… Then I decided to read the documentation (might be useful at times), and the monitor is actually configured via a Chrome (desktop only) add-on through USB. There are three types of devices made by vair-monitor, and I first used  vThings Configuration Utility add-on, but eventually found out I had to use vThings – Dual Beam Configuration Utility.

vthings-chrome-apps

vThings Configuration vs Things – Dual Beam Configuration Utility

I used Ubuntu (Linux), but if you are using a Windows or Mac computer, you’ll need to install drivers first. Once you’ve connected the monitor through USB and started “vThings- Device Configuration Tool” the following windows should be shown.

Click to Enlarge

Click to Enlarge

The fist thing to do is to connect the monitor to your WiFi router by entering its SSID and password, and click on Set WiFi.

Click to Enlarge

Click to Enlarge

It should connect to your router, and the first time updated the firmware automatically. Wait a couple of minutes for it to complete, and you can go to the next step to configure one or more of the following Public, Private or Generic services:

Public Private Generic
BeeBottle DomoticGa HTTP
Blynk.cc DomoticZ MQTT
dweet.io FHEM RF 433/315
EmonCMS Homeseer
ThingSpeak HomeAssistant
UbiDots JeeDom
OpenHAB
Pimatic

I decided to go with ThingSpeak since I got familiar with it while writing Sonoff POW tutorial.

Click to Enlarge

Click to Enlarge

Select the data provided by the sensors inside your vthings Co2 monitor, in my case CO2 levels, temperature, and pressure, and nothing else, or connection will fail, as I found out when I used 4 default fields including humidity, and ThingSpeak was not updated at all. You’ll also need ThingSpeak API write key, that you can get my create a channel on ThingSpeak.com as shown below.

thingspeak-co2-monitor-thingspeak-channel-configuration

Once the channels is create on ThingSpeak website, and you’ve added the API write key in vThings Device Configuration Tool, you could go to Generic Services->HTTP and notice an HTTP request has been created, so if you have installed ThingSpeak locally, you could change api.thingspeak.com to your own IP address.

Click to Enlarge

Click to Enlarge

By default the data will be updated every 120 seconds, but you can change that in Settings->Update Interval. Once configuration is done, you can unplug it from your PC, and connected to the location you want to monitor. vThings Device Configuration Tool requires a USB connection to find the device, it can not find it over WiFi, so if you want to change configuration, you’ll need to connect it back to your computer. There’s a function to (auto)start a webserver in vESPrino, but it did not seem to work for me.

After a few hours or minutes depending on your update internal you should get some nice charts on ThingSpeak with CO2 levels, temperature and pressure, or other data based on the sensors you’ve selected while purchasing the hardware.

Click to Enlarge

Click to Enlarge

The channel is public if you are interested/curious in seeing the data. ThingSpeak will show 60 samples (2 hours in my case) by default, but let’s see what happened over the last 12 hours with CO2 levels.

Click to Enlarge

Click to Enlarge

The CO2 levels started at about 500 to 600 ppm while I did the configuration in my office (windows closed), and dropped to around 404 ppm once I installed in the kitchen. That value correspond roughly to the current CO2 ppm value in the atmosphere (in Hawaii). Three times around 18h00 people warmed food and CO2 jumped to around 500 ppm. During the night, CO2 levels slowly increased to 480 ppm, likely because of the plants cycle (producing oxygen during the day, and carbon dioxide during the night). This morning CO2 levels spiked at around 900 ppm when cooking right after 6am and 8am.

That’s all fun, but is there a real benefit to measuring CO2 levels in your house? In the kitchen I could probably trigger an alert over 1,500 ppm in which case it may mean something is burning, but smoke detectors are much cheaper and better suited to the task. Vladimir Savchenko, vThings developer, found a study claiming that high CO2 levels may decrease creative thinking and lead to bad sleep, so he used vThings CO2 monitor in his bedroom and discovered CO2 levels reached close to 4,000 ppm, and that just open the door or window would greatly reduce the concentration of the gas.

sleepwithcloseddoortext-co2-levelsvThings CO2 monitor does not only monitor CO2 levels as we’ve seen above, as temperature, humidity, and/or pressure sensor can be included in the case, as well as a PM2.5 & PM10 laser dust sensor.

vThings CO2 Monitor v3 is sold for 60 Euros with CM1106 CO2 sensor, and more if you use a better CO2 sensor, or add extra environmental sensors. 135 Euros would get you a top of line monitor with a laser dust sensor, CDM7160 CO2 sensor, temperature and humidity sensor, and RF connectivity.

How to Use Sonoff POW ESP8266 WiFi Power Switch with MQTT and ThingSpeak

December 11th, 2016 10 comments

ITEAD Studio’s Sonoff is a family of cheap home automation products based on ESP8266 WiSoC, and I’ve already tested Sonoff TH16 wireless switch with a humidity and temperature sensor using the stock firmware and eWelink app for Android or iOS. It works, but up to recently it required a registration to a cloud service (the company will now allow use from the local network), and the source code is closed. So for the second device under review, namely Sonoff POW wireless switch with a power consumption monitor, I decided to install ESPurna firmware working on ESP8266 Sonoff devices and NodeMCU, as it’s open source, supports Sonoff POW natively, includes a web interface to control the device from the LAN, and includes an MQTT client.

MQTT (Message Queuing Telemetry Transport) is a lightweight publish/subscribe messaging protocol used to control IoT sensors and devices, and it’s a popular method to gather data from client to a MQTT broker to push the data to the cloud or a local database.

iot-sensors-mqtt-cloud

So typically, you’d have a bunch of sensor nodes (like Sonoff devices) communicating over MQTT to an MQTT Broker in your local network, which could be an OpenWrt router or a Linux development board like a Raspberry Pi, which in turns gets the data the the cloud to services like AWS IoT, Xively, or ThingSpeak. It’s also possible to use Cloud services to control MQTT devices remotely through the MQTT broker.

I eventually plan to use NanoPi NEO board to run both MQTT and ThingSpeak locally (not connected to the cloud) in order to monitor the power consumption of my small office, but since I’m all new to this, I’ve started experimenting by connecting a 30W light to Sonoff POW, and use a desktop computer running Ubuntu 16.04 for MQTT and ThingSpeak.

sonoff-pow-connection

Click to Enlarge

Click to Enlarge

Since I’ve already installed ESPurna firmware to the device, I disconnected the USB to serial board (important since Sonoff POW board has a hot ground), and connected it to the mains (220V in my location). That means we already have an MQTT client which first I had to configure.

Click to Enlarge

Click to Enlarge

Since it was the first time I connected a load to the device, I went to ESPurna’s status menu to check power usage was reported, and my 30 Watts light bulb was drawing 27 Watts. Close enough. I changed the hostname to sonoff-office, and setup two SSID in order to connect Sonoff POW to my local network in client mode, instead of using it in Access Point mode by default. You’ll need to tap on Update each time you modify the settings. Since the SSID must be entered manually, please note that SSID are case sensitive, e.g. CNX-SOFTWARE is different from cnx-software.

Click to Enlarge

Click to Enlarge

I wanted to calibrate the power using the 30W light bulb, so I entered 30W in AC RMS Active Power field, and tapped on Update, but the web interface reported “no changes”. I’m not sure how to use that part. Finally the most important part for this tutorial is to set the MQTT settings with MQTT IP address, and leaving other fields unchanged. However, you can change MQTT Topic field for example replacing /test/switch/{identifier} by /myiotstuff/{identifier}.

Now that our MQTT client is configured, I need to install mosquitto MQTT broker in Ubuntu:

mosquitto-clients is not really needed, but I’ll use it to test the MQTT broker a little later. Once you installed it, the MQTT Broker should already run automatically.

The last line of the log above shows a client connection from Sonoff POW. Now, we need to check the topic, and since ESPurna documentation is still work in progress, you could either check out the source code, or IMHO more fun, capture MQTT packet with tcpdump or Wireshark as I’ve done below.

Wireshark MQTT Capture - Click to Enlarge

Wireshark MQTT Capture – Click to Enlarge

Here we can see that Sonoff POW will send a Publich Message with the power level using the topic “/test/switch/sonoff-office/power29”.  “/test/switch” is the string we’ve defined in the web interface, “sonoff-office” the hostname we’ve given to Sonoff Pow, and “power29” indicates 29 Watts of power is currently used.

We can also start a client in Ubuntu 16.04 terminal window to check more MQTT topics with # wildcard for sonoff-office host:

We can use MQTT to get the IP address, firmware and file system version, hearbeat message, power use, and relay status (on or off).

It’s all good, but now we need to do something to draw the data, and possibly analyze it. I selected ThingSpeak for this purpose since it can be installed in the local network, or through their service in the cloud. By the end of my testing, I’ve noticed ThingSpeak has a new MQTT API, meaning it should be possible to connect your MQTT broker directly to it, but for this guide I use mqspeak instead as a bridge between MQTT and ThingSpeak. It may still be useful, as the open source version of ThingSpeak is not updated anymore, and lacks the MQTT API.

You’ll need Python 3 and pip3 to install mqspeak:

Once it’s done, we’ll need to create a config files as explained on mqspeak’s github repo, and I created /etc/mqspeak.conf with the following content:

Brokers are used to configure MQTT broker IP address and port, as well as the topic(s) to subscribe to, while Channels take care of ThingSpeak configuration with the channel’s Id and write API key, update rate in seconds (15s minimum), update type (see github for details), and fields defined in your ThingSpeak’s channel(s), which will create later on. I wrote one broker for the power consumption topic, and other for the relay status. However, I eventually ignored the relay status, as it’s not updated often enough and cause ThingSpeak’s channel to only be updated when the relay changes status, even if there are power updates in the meantime. A workaround is to use two different channels for ThingSpeak.

mqspeak connects directly to api.thingspeak.com, so if you are using ThingSpeak cloud services, the next step is to register an account and setup one or more channels.

Extra Instructions for a local installation of ThingSpeak

However, if you’ve installed ThingSpeak in Ubuntu 16.04 or other Linux operating systems locally or on your own server, you’ll need to change the server in the source code, and reinstall mqspeak.

  1. Get the source code:
  2. Modify mqspeak/sending.py to replace api.thingspeak.com using HTTPS with localhost (or other IP address/URL where you’ve installed ThingSpeak) with HTTP:
  3. Install mqspeak

An improvement would be to install a signed SSL certificate, like the one offered by LetsEncrypt and configure the rails server to use https instead. I have not setup ThingSpeak server to start automatically yet, so I have to start it manually for now:

End of instructions specific to local installation.

The instructions specific for the local installation of ThingSpeak are now done, and all instructions below are valid for both the local installation and cloud service. Now open a web browser, go ThingSpeak (cloud or local), and click on “Get Started Now” in order to register an account.

Click to Enlarge

Click to Enlarge

Once it is done, login and click on “New Channel”.

Click to Enlarge

Click to Enlarge

Give it a name, a description, create fields as needed, for example power-consumption and power-status, and click on Save Channel.  Update /etc/mqspeak.conf accordingly with the fields’ name, and channel Id.

thingspeak-api-keyNow select API Keys tab to copy and paste the write API key into mqspeak.conf.

Now we can start mqspeak:

ESPurna firmware will send a power update every 60 seconds (this can be changed in code/src/pow.ino), so you’ll need a new message pop-up every 60 seconds with your channels Id and write API key. I’ve let it run for about one hour, and got the follow chart in ThingSpeak after turning on and off the lights from time to time.
thingspeak-power-consumptionThat’s pretty cool, so it only shows the current power in watt, and we’d probably want to get power consumption in kW/h per day, week and month at some time, and I have yet to study how to do that, Exporting the data to excel would be a workaround if this can not be handled in ThingSpeak. ThingSpeak.com (but not the open source version) offers some Matlab processing of the data, so that’d be another options.

The next steps would be to install MQTT and ThingSpeak in NanoPi NEO board, enable HTTPS in ThingSpeak, autostart rails server and mqspeak at boot time, make ESPurna firmware publish the “Power” topic more often than every 6 second, and find some way to generate useful kW/h consumption charts from the data stored in ThingSpeak within ThingSpeak, or but exporting the data.

How to Install ThingSpeak in Ubuntu 16.04

December 7th, 2016 11 comments

Last week-end I installed ESPurna open source firmware with MQTT server on Sonoff POW WiFi switch, and the next step is find a way to draw power consumption charts in some web based interface. We could do this in the IoT cloud with services like Xively or ThingSpeak, but since one of the goals of replacing the default firmware was not to rely on a proprietary cloud based solution, I decided to find a way to draw those chart in a local server, and it so happens that ThingSpeak is also open source with the code available on Github. Hardware platforms like NanoPi NEO / NEO Air, or Orange Pi Zero boards appear to be particularly well suited for the task of running an MQTT broker and Thingspeak, but at first I wanted to install ThingSpeak in my own Ubuntu 16.04 computer to have a try.

Click to Enlarge

Click to Enlarge

As you can see from the screenshot above I manage to do it, but it requires a bit more efforts than expected, as the project has not been updated since 2015, and does not work out of the box with the latest operating system.

I used various resources on the web including the instructions on Github, as well as this ThingSpeak script for Ubuntu 14.04, and a few other resources.

First we have to make sure Ubuntu 16.04 is fully upgraded:

Ubuntu 16.04 comes with Ruby 2.3, but we need the older Ruby 2.1.0 version for ThingSpeak, so let’s uninstall ruby to avoid conflicts:

Now we can install dependencies, Ruby 2.1.0, and Bundle:

Once this is done, we can get ThingSpeak source code and install it:

This looked successful so I moved on to database configuration:

It’s recommended to change the username and password in config/database.yml for test, development and production databases with your own for security purpose. Once it’s done, let’s try to create the databases:

Sadly it starts with an error:

So I checked mysql2 version and upgraded it to see if it would fix the issue:

The previous error is gone, but only to be replaced by a new one…

Finally, I found out (can’t find where anymore) that I had to edit Gemfile in ThingSpeak directory, and add an older version to mysql2:

Let’s update mysql2, and try to create the databases again:

Damn a permission error. I could not find a proper fix, so at this point the title of the post should possibly become “How NOT to install Thingspeak in Ubuntu 16.04”, as although it will work, the steps below makes the installation insecure since I simply give full databases’ access to thing user. But that will do since I’ll only use it in my LAN, and maybe somebody will point to a secure solution to the issue.

[Update: Thanks to Arthur, I’ve got a more secure solution . I’ve left both insecure and secure workaround for reference, but obviously you should use the secure one, especially it’s not hard]

Insecure (don’t use it, I just left it to show what you should not do):

Secure way (strongly recommended):

This time I can create the databases for Thingspeak:

So now we can go to the next step to load the database with some data required by Thingspeak to work:

Great! Yet another error:

After spending a while for a solution I eventually found it in Rails Github with the reason being that MySQL 5.7 used in Ubuntu 16.04 does ot allows for NULL key.

We’ll need to create config/initializers/abstract_mysql2_adapter.rb file with:

Then we need add the following line at the end of config/environment.db

and run the command again:

Success! Finally…

The final step is to start the server:

Now start your web browser and you can access your local Thingspeak installation @ http://localhost:3000.
I’ll now have to study a little more about Thingspeak, install MQTT, as well as one of the MQTT to Thingspeak bridges available on the web, and see if I can plot power consumption data there.