Karl’s Home Automation Project – Part 1: Home Assistant & YAML, MQTT, Sonoff, and Xmas Lights

Karl here. I am here to write about my home automation project. First thing I want to say is that I am very cost conscious and I don’t mind putting in extra effort into the setup of things to keep costs down. I did invest a lot of time and had to do a lot of reading to get my project going. It took while and I received a lot of groans from my wife while testing. I am still in the process of tweaking things.

I started watching a series of videos on YouTube from Bruh Automation. He introduced me to Home Assistant. It got me really excited. He uses a Raspberry Pi as a server but I already had a Wintel Pro CX-W8 Smart TV Box which I use as a server. I run 3 Minecraft Servers, Emby Server, iSpyConnect DVR (2 IP Cameras), Unifi wifi controller, and now MQTT Server, and Home Assistant. Below is screenshot of mostly idle.

If it weren’t for iSpy it would be around 5-10% most of the time. Emby transcoding is the only thing that is stressful and it is not used much. The reason I mention this is because after purchasing a Raspberry Pi with power supply and case, you are not far off from getting a z8300 box. Only downfall is dreaded Windows update auto reboot. I finally looked into it and disabled it. If you decide to use a Windows box, I would make sure you are running 64bit windows. One advantage to using a Raspberry Pi is there is an image on Home assistant with the basics pre-configured and just need to write it to an SD card.

Server side Setup

I won’t go into too much detail on server side, as I installed Python, Mosquitto, and Home Assitant (I followed the guide on their site for Windows)

Python was a breeze to install and just ran the executable and went with defaults. I already had it installed for something else and I am running 3.5.2 64-bit. There are newer versions now. Mosquitto was the most difficult. I followed this guide but substituted Win32OpenSSL_Light-1_0_2j.exe approx 2MB. A k version is available now. Home assistant was easy and used pip.

Christmas Lights

It was a little before Christmas when I started researching home automation. I had been reading about these inexpensive Sonoff devices here on CNX and I found a project on Github for some custom firmware by arendst that enabled them to be controlled by MQTT. (While getting the link it looks like a new project has started with some additional features here). My wife really likes decorating for Xmas and we have 3 trees and lots of lights. She mentioned getting some timers and boom I had my opportunity and ordered them the same night. After receiving It took me a couple nights and I had a simple automation turning Xmas lights on and off at specific times and life was good. I got an extra one to play with until Xmas was over. I redeployed the rest  around the house after Xmas.

MQTT

I really had no idea what this was and it took me a while to grasp. You can use a cloud based MQTT if you would like, but I prefer to run my own. MQTT is a service that relays messages between devices. There are 2 main items topics and payloads. To be able to tell a switch to turn on you send payload “on” to a topic, for example, “cmnd/testbench/power”. The light turns on and it replys back to a topic “stat/testbench/POWER” confirming that the light is on and the message is received. Because we are sending “on” to the topic each device using MQTT will need its own topic. Topics are case sensitive. I made a batch file to subscribe to all topics for troubleshooting so I could monitor the messages. The # indicates all sub topics.

Sonoff

I picked the Sonoff basic but there are also different varieties that add additional features which are supported by arendst software.

Arendst  has been very active with this project and adding/tweaking daily. When I first flashed the device, I did find a defect and notified him and he had it fixed and uploaded within the hour. He has very detailed instructions on the Wiki. First step before flashing is soldering headers. (I link to bent headers…which I initially thought I made a mistake but turned out it was good. They are easy to straighten) A USB to TTL adapter is also needed to upload from Arduino IDE. I recommend one like this because it provides both 3.3 and 5V.  After downloading and setting Arduino up, I only set my WiFi password and SSID in the sketch. After it boots the first time, it connects to your wireless network. Find the IP address in your router, and pop the IP address in your browser to finish the configuration. Set the MQTT server credentials and topic and your done. I never setup credentials on the MQTT server so it accepts any login. Finally after everything is programmed you need to connect it to mains. Beware do not connect mains while TTL is connected.  I bought some extension cords locally. Cut them in half and stripped back a ¼ inch of the insulation. Extension cords use stranded wire so I tinned them with solder to avoid any stray strands from shorting out. Then I screwed them down on the terminals making sure polarity was correct.

Click to Enlarge

YAML

YAML is unforgiving. It is the formatting that you configure Home Assistant in. A single space will stop Home Assistant from starting. Luckily on this last update if you restart Home Assistant through the browser it will test the configuration file before actually restarting. I purposefully put an extra space on line 54 to show it is easy to find any mistakes.

Click to Enlarge

I also recommend Notepad++ for editing in windows. You can break your configuration down into different files but I like one. Notepad ++ allows you to collapse the parts you aren’t currently working on.

I recommend adding one thing at a time and restarting to make it easier to find errors. And making a copy of the last working config before adding more. In the config below there are 5 sonoff’s and an automation to turn the lights on and off at specific times. This is extremely basic. I also recommend setting up one new device and be conscious of naming. When you get your config working properly on your first new device I copy the config to a new blank text window and do a find/replace.

Below is the screen capture of collapsed parts, and and full config (minus personal info).



Notice the test bench is on later firmware and the MQTT topic is slightly different

Next Steps

So now I have a smart home, right? Not in my opinion. I can turn lights on and off with a schedule or with my smart phone or at the light by pressing the button on the Sonoff. To me this is not smart. Setting a schedule is OK, but then you have the lights on unnecessarily and wasting electricity. Only real option is to press a button on the Sonoff but what difference is that than flipping a switch. Taking your phone out takes way too long, and I feel like it is going backwards. Below are estimated costs so far. By far the Windows Box will be the most expensive part if you choose to go that way. You can re-purpose just about anything that runs Linux to be a server. One other option is to run Linux on an S905x.

Money Spent

Cost of server not included nor shipping.

ItemQtyPriceTotal
Sonoff Basic5$4.85$24.25
Headers1$1.50$1.50
USB to TTL1$2.54$2.54
Total$28.29

If you find this entertaining or want me to go more in depth on a specific aspect let me know in the comments. I have been finding my time setting it up very satisfying. I am able to do some hardware and software work. I hope this might get your interest in home automation going, and find out it is not hard nor expensive. I would like to state none of the products linked were provided by the sites. I purchased with my own money.

The plan right now is to do a 3 part post. In the next post, we will integrate some inexpensive motion sensors and door sensors using 433mhz, then finally modifying the sensors to include a light intensity sensor.

Continue reading “Part 2: 433 MHz / WiFi MQTT Bridge, Door & PIR Motion Sensors“.

Karl is a technology enthusiast that contributes reviews of TV boxes, 3D printers, and other gadgets for makers.

Support CNX Software - Donate via PayPal, become a Patron on Patreon, or buy review samples
Subscribe
Notify of
guest
30 Comments
oldest
newest most voted
Harley
Harley
3 years ago

Will you use RFLink Gateway for 433MHz (and 315MHz) sensors? It is the best: http://www.nemcon.nl/blog2/ Bought mine from nodo-shop.nl and Arduino clone from Aliexpress on the cheap: https://www.nodo-shop.nl/nl/21-rflink-gateway Slowly migrating away from 433MHz but still have light and door sensors. I also use Sonoff too, but have Domoticz instead now, and looking at OpenHAB. Tip; also bought DHT22 temperature/humidity sensors to connect to each Sonoff. http://tinkerman.cat/tag/sonoff/ Xose Pérez at tinkerman.cat linked above has some tips for cheap Sonoff hacks. I have also bought cheap Z-Wave USB stick as well, but devices are expensive. Another tip is to buy TeckNet BlueTek… Read more »

Theguyuk
Theguyuk
3 years ago

So you are worried about the cost of powering a light bulb, so you turn it off by powering a windows atom TV box.

So cost of atom TV box and part, plus labour and postage, how long to recover your initial costs + cost of running atom TV box?

Karl Johnson
Karl Johnson
3 years ago

Harley : Will you use RFLink Gateway for 433MHz (and 315MHz) sensors? It is the best: http://www.nemcon.nl/blog2/ Bought mine from nodo-shop.nl and Arduino clone from Aliexpress on the cheap: https://www.nodo-shop.nl/nl/21-rflink-gateway Slowly migrating away from 433MHz but still have light and door sensors. I also use Sonoff too, but have Domoticz instead now, and looking at OpenHAB. Tip; also bought DHT22 temperature/humidity sensors to connect to each Sonoff. http://tinkerman.cat/tag/sonoff/ Xose Pérez at tinkerman.cat linked above has some tips for cheap Sonoff hacks. I have also bought cheap Z-Wave USB stick as well, but devices are expensive. Another tip is to buy… Read more »

cnxsoft
Admin
3 years ago

I think that’s OK because he would run the box anyway for his other purposes. It probably consumes about 4 to 6Wh, but would run whether he had a control system for his light bulbs or not.

Karl Johnson
Karl Johnson
3 years ago

Theguyuk : So you are worried about the cost of powering a light bulb, so you turn it off by powering a windows atom TV box. So cost of atom TV box and part, plus labour and postage, how long to recover your initial costs + cost of running atom TV box? I already had it running those other servers for a while now. A PI can be used if starting from scratch. I use the Atom specifically for it’s built in Quick Sync transcoding support supported by Emby. I would need a considerably beefier system to do CPU transcoding.… Read more »

Harley
Harley
3 years ago

@Theguyuk No you have misunderstod. We are not really worried of the cost of electricity per lightbulb. When we say “cost” here we are refering to the cost of home automation devices, which we don’t want or can’t spend a fortune on. Modern home automation devices such as the latest wireless power relays and sensors are generally known for being expensive. This means that everyone thinks that you have to pay a lot to build a home automation system. What we are saying here is that you can build a relativly cheap home automation system by using Sonoff devices ,… Read more »

Harley
Harley
3 years ago

@Karl Johnson
RFLink Gateway is a DIY project as well, and it is fully open source. Both open source hardware and open source firmware/software. No other RF Gateway supports as many devices as RFLink. Highly recommend it over all others:

http://www.nemcon.nl/blog2/devlist

RFLink firmware also supports communication via 2.4Ghz as well as reception at 868 and 915 Mhz (but not on the same board, as you currently will need a seperate board per frequency).

Kris
Kris
3 years ago

I loved the article.

Theguyuk
Theguyuk
3 years ago

@Karl Johnson
Have more fun, use a Nanopi !

Katalin
3 years ago

Nice project Karl. If you have patience for few more weeks to finish some projects I intend to make public the MQTT broker on an ESP8266, I just need to create a nice web interface to allow users to create an account on one of my servers so they will get the broker address with a simple HTTP call ( no need to check the router for IP addresses). See more details on https://myesp8266.blogspot.nl/2016/11/mqtt-broker-on-esp8266-5.html .This projecct started on an early development board from EspressIf ESP32. Then I moved to ESP8266 just to check its capabilities and is working fine. Up… Read more »

Petr
Petr
3 years ago

As you suggest flipping the light manually or some time trigger is not particularly smart. A box standard timer socket could do it. Why don’t you use the sunset as a trigger for turning the lights on? A number of triggers spring to mind for the off trigger. Simple time, sunrise, motion detection, TV socket power, late shift detection with a calendar, etc. Depending on personal circumstances.

I too am often struggling to think of the ‘smart’ condition, so my smart devices make sense.

Harley
Harley
3 years ago

@Katalin Sounds like a interesting project with a lot of potential if you would build on it. So I was wondering if you have thought anything about using a Mesh networking concept to setup a distributed system of MQTT brokers that could share the load and that way both get redundant MQTT brokers and allow more clients to connect to that MQTT mesh? That is, I don’t like the idea of relying of a cloud service as I want everything to always be 100% locally controlled on my local network, however I think that it would be very cool if… Read more »

Harley
Harley
3 years ago

@Petr Cheap 433Mhz twilight relay/switch (light sensor) can also help to make ‘smart’ condition/scenario if you have a good place to put it, as unlike sunset and sunup a twilight relay can also detect if it is an cloudy overcast so that you therefor want to turn the light of earlier or leave them on later than your normal sunset and sunup + delay scenario. If you combine the data from a twilight relay with a the sunset/sunup then you can for example create a ‘smart’ condition/scenario where the twilight relay can turn the lights on within 2-hours before sunset… Read more »

tkaiser
tkaiser
3 years ago

@Harley I wonder whether this ‘cheap 433Mhz’ stuff is safe/secure to operate (especially when doing a web search for ‘433mhz replay attack’ or ‘rf replay attack’)? Same with people storing their Wi-Fi credentials on cheap IoT nodes running vendor firmwares or prone to physical access by unauthorised people (I would never do this but instead use at least something as an Orange Pi Zero as AP to create an own IoT Wi-Fi for ESP8266/SONOFF devices where instrusion detection could work at least somehow)? Talking to neighbours recently I was reminded that most people still think Wi-Fi pseudo security like MAC… Read more »

BCatalin
3 years ago

@Harley
That’s the beauty of my broker. If I am at home my mobile app will connect over the websockets to the broker and will send/receive MQTT data to/from my devices. I am using the cloud only to control the devices when I am not at home. I don’t rely on the cloud data also, but if I am home I am connecting direct to the ESP8266 if not through the cloud. If I am at home and the internet connection is down, I am not affected at all. See this link: https://myesp8266.blogspot.com/2016/04/since-i-received-lot-of-questions-about.html

tkaiser
tkaiser
3 years ago

Doing a web search for ‘hta-r03-light-weight-protocol-serious-equipment-critical-implications.pdf’ is also somewhat frightening (slides and talk on youtube about ~60,000 publicly available insecure MQTT brokers)

cnxsoft
Admin
3 years ago

@tkaiser
Nothing is secure, even electronics in cars are not secure according to Masayoshi Son (ARM’s new owner).
I don’t think things will change unless some laws are in place to make somebody responsible (e.g. not only the hacker himself) in case of a security breach. The problem is that currently security also relies on the user not doing something stupid, and it’s not realistic to think end user will understand it all. So products would have to be secure whatever the user does.

tkaiser
tkaiser
3 years ago

cnxsoft :
@tkaiser
Nothing is secure, even electronics in cars

Just do the web search for ‘hta-r03-light-weight-protocol-serious-equipment-critical-implications.pdf’, there are examples about publicly available MQTT brokers that collect/send data from/to cars. It’s scary to be able to subscribe to a MQTT broker with F_SPEED, F_SW_BRAKE and so on exposed (and most probably accessible too). Same applies to ‘dumb homes’ (that’s IMO the better name for so called ‘smart homes’ connected to some ‘cloud’)

Petr
Petr
3 years ago

@Harley
Good idea.

Petr
Petr
3 years ago

@tkaiser
So I have just “wasted” an hour to play with Shodan and am gobsmacked how much info is available publicly. Let’s see this out of interest https://www.shodan.io/host/24.67.211.138

excerpt:
Eric and Cailyn from Salmo in Canada
running an unsecured MQTT broker

Topics:
owntracks/eric/MotoX
owntracks/caitlyn/caitlyn’s phone
topic
arduino/temperature/living_room
arduino/living_room/temperature
arduino/living_room/touch_sensor
arduino/living_room/touch_sensor/set
arduino/livingroom/temperature
arduino/livingroom/touch_sensor

You can easily subscribe to the phones’ location updates via owntracks publisher. Read their home sensor statuses and possibly push a payload to switch things on or off.

Oh, and Look Eric is on the move…

Scary stuff.

Encrypted MQTT even as a first line of defence.

Harley
Harley
3 years ago

tkaiser : @Harley I wonder whether this ‘cheap 433Mhz’ stuff is safe/secure to operate (especially when doing a web search for ‘433mhz replay attack’ or ‘rf replay attack’)? I only use 433Mhz sensors, like a twilight light sensor and a few door sensors, not any power-relays. So the 433MHz sensors is only for non critical infrastructure. To turn lights off or on if someone opens or close a door. Also remember that 433Mhz sensors is not connected to the internet in any way, so I’m not paranoid enough to think someone will physicially come to my house and send 433MHz… Read more »

tkaiser
tkaiser
3 years ago

@Petr
Now add ‘cloud’ to the picture. This here (cloud connected ‘toys’) is a random example how everything cloud connected (Hey, Alexa) ends up eventually: Everything exposed to anyone.

https://www.troyhunt.com/data-from-connected-cloudpets-teddy-bears-leaked-and-ransomed-exposing-kids-voice-messages/

Theguyuk
Theguyuk
3 years ago

Here is a lovely silly one from BBC http://www.bbc.co.uk/news/technology-38966285

Petr
Petr
3 years ago

@Theguyuk

Actually, I’ve found similar vulnerability in my system, by yelling loudly “OK, Google, open the garage door” from the outside of my house.

Charis
Charis
3 years ago

Hi! I want to ask you why you choose Sonoff switches for home automation, and not Xiaomi mi home ? I’m asking you because Xiaomi has very cheap sensors for your home automation system, is working with zigbee and a hub. I ‘m also on this dilemma between those 2 solutions.

Karl Johnson
Karl Johnson
3 years ago

Really depends on the requirements. I liked the universal MQTT method to communicate. Have to weigh pros and cons. When I started i wasn’t aware or it wasn’t around. Not sure this helps but good luck. Also be sure if you choose a home automation software it works with the hardware. I need to check it node-red. It’s another option.

Petr
Petr
3 years ago

@Charis As Karl has pointed out Sonoff has been around possibly longer and now there is enough community development to upload a custom software on it, so you don’t need to rely on chinese apps to customise/control your device. I prefer the TASMOTA firmware that gives you a nice web interface, OTA updates, MQTT and many, many functions. I actually now use it on the Wemos D1 Mini boards too, these costing US$3-4 and have a ecosystem of add-on shields. This paired with HomeAssistant makes for very robust setup. Mind you yaml is a bit of a pain, but not… Read more »

Charis
Charis
3 years ago

Itead sonoff problem is that there are not much trigger devices for auto scenes. On the other side Xiaomi smart home has too many sensors that trigger events. But for 2 floor house i think is inposible to work with zigbee protocol because of lower range. And i think i need more than one gateways…

Charis
Charis
3 years ago

@Charis
(sorry for bad english… inposible = impossible)

Karl Johnson
Karl Johnson
3 years ago

Petr : @Charis As Karl has pointed out Sonoff has been around possibly longer and now there is enough community development to upload a custom software on it, so you don’t need to rely on chinese apps to customise/control your device. I prefer the TASMOTA firmware that gives you a nice web interface, OTA updates, MQTT and many, many functions. I actually now use it on the Wemos D1 Mini boards too, these costing US$3-4 and have a ecosystem of add-on shields. This paired with HomeAssistant makes for very robust setup. Mind you yaml is a bit of a pain,… Read more »

Advertisements