Saturday, 23 December 2017

Home Assistant + Node-RED with BTRFS on Arch

After using Home Assistant for a few months and having set it up to use external MySQL databases, etc. I have decided to re-install in a robust manner. I found that the ext4 that I was using kept going corrupt after power outages. I stupidly turned off the journal to "save" the sdcard but this was the last straw and many of my config files went missing... Time to start afresh!

This post pretty much contains a list of commands. I won't go into detail on each of them but they should be fairly self explanatory. I probably missed a bunch of stuff too but this is the gist of it!

First off, I am using Arch because I like the amount of control it gives me... I am also going to go with btrfs instead of ext4 this time around. My hardware is a Cubieboard2 with an external SATA 1TB drive which I use to store CCTV footage, the MySQL database, etc. I want to minimise writes to the sdcard and so that is the thing to keep in mind.

Following the Arch installation tutorial here:

Using an Ubuntu 16.04 VM to get the sdcard prepped, install btrfs-tools first.

Partition and create btrfs instead of ext4 and create a boot partition too. Make a 100MB boot partition and use the rest for root. Follow the guide for the start location.

sudo mkfs.ext2 /dev/sdc1
sudo mkfs.btrfs /dev/sdc2

I would like to use the btrfs compression for a performance boost (allegedly), space savings and perhaps a boost in reliability because of less space usage...

cd /tmp

mkdir boot
mkdir root

sudo mount -o compress=lzo /dev/sdc2 root
sudo mount /dev/sdc1 boot

Download the arch tar.gz file and decompress as per their guide. I got a few warnings about flags due to the btrfs so I used tar instead of bsdtar. I'll ignore them for now and hope for the best! It may take a while to do the extraction - depending on the speed of your SD Card.

sudo tar -xzvf ArchDownloadFile.tar.gz -C root

Create a file on the boot partition called boot.cmd and put this inside it:

if test -n ${distro_bootpart}; then setenv bootpart ${distro_bootpart}; else setenv bootpart 1; fi
part uuid ${devtype} ${devnum}:${bootpart} uuid

setenv bootargs console=${console} root=/dev/mmcblk0p2 rootfstype=btrfs rootflags=autodefrag,compress=lzo rw rootwait

if load ${devtype} ${devnum}:${bootpart} ${kernel_addr_r} /zImage; then
  if load ${devtype} ${devnum}:${bootpart} ${fdt_addr_r} /dtbs/${fdtfile}; then
    if load ${devtype} ${devnum}:${bootpart} ${ramdisk_addr_r} /initramfs-linux.img; then
      bootz ${kernel_addr_r} ${ramdisk_addr_r}:${filesize} ${fdt_addr_r};
      bootz ${kernel_addr_r} - ${fdt_addr_r};

This is essentially what is in the default boot.scr, with mods for btrfs and the boot partition. We need to compile this file. Get the u-boot-tools and the run with appropriate modifications to your paths if you aren't following mine:

mkimage -C none -A arm -T script -d boot/boot.cmd boot/boot.scr

Unmount everything and pop the sdcard into the Cubieboard. It should boot up to a login prompt. If not, start from scratch and try again :) the serial console helps a lot! It's probably worthwhile to edit the boot.scr to output to HDMI rather that serial.

When you are up and running, before installing uboot or other update stuff (as per the Arch tutorials), update your fstab to include a line for the boot partition:

/dev/mmcblk0p2 / btrfs defaults,relatime 0 0
/dev/mmcblk0p1 /boot ext2 defaults,noatime 0 0

tmpfs   /tmp tmpfs  rw,nodev,nosuid,size=512M  0  0

It's also important to install btrfs-tools and a couple of other things with pacman:

pacman -Syu btrfs-prog ntp wget nano

General Stuff

Do the usual things at:

Also, install yaourt for AUR stuff:

su (the password is root by default)
useradd ...
pacman -S sudo

Follow the instructions and then check if your new user has sudo by typing su newuser and trying out a sudo command. If all goes well, ssh in via the new user and sudo userdel alarm.

Make sure your boot partition is mounted properly and then:

sudo pacman -Syu

Home Assistant and Node-Red Installation

yaourt -S home-assistant --noconfirm
yaourt -S nodejs-node-red --noconfirm

The default home assistant AUR install puts the config is /var/lib/hass and runs home assistant with user hass. Let's keep it this way but I want read write access to the config as my normal user.

sudo usermod -a -G hass username
sudo systemctl start home-assistant
sudo systemctl enable home-assistant

The first run of home-assistant (hass going forward) takes a while as it downloads and sets up all of the components.

Node-Red doesn't seem to come with a predefined service, user, etc. so let's create one...

sudo useradd -m -s /sbin/nologin nodered
sudo nano /lib/systemd/system/nodered.service

Paste this inside:

# systemd service file to start Node-RED
Description=Node-RED graphical event wiring tool.
# Run as root user in order to have access to gpio pins
ExecStart=/usr/bin/env node-red-pi $NODE_OPTIONS $NODE_RED_OPTIONS

sudo systemctl start nodered
sudo systemctl enable nodered

The Node-Red config is in /home/nodered/.node-red

I then modified the permissions on the nodered home directory so that I could edit as my normal user:

sudo usermod -a -G nodered username
sudo chmod 770 /home/nodered

There's lots more to do such as installing pi-hole, samba, etc!

Friday, 22 December 2017

Sonoff External Button Input

It seems like an obvious thing that's missing from the excellent Sonoff modules out there: an input for an external button (or two or four!). A common thing to do is to solder some wires across the existing switch on the PCB, but this isn't the best idea. In this post I will quickly show what I did, which is a more robust solution for a permanent installation.

The built in button(s) on the Sonoff's are hooked up directly to one of the GPIO inputs on the ESP8266. This is fine for a button on a PCB as no ESD or surge or wrong connections are likely. On the other hand, hooking a pin to a long wire which will be handled during installation, is susceptible to interference coupling, etc. is not the best idea. The ESD protection diodes on the chip may be able to handle some punishment but over the years it's going to be a real hassle to replace the Sonoff...

In the photo and rough circuit diagram below you can see I have simply soldered everything (with some heatshrink for insulation) to a pin header which plugs onto the Sonoff serial header. A 2.2k pullup resistor is there to provide a strong pull up. I have found that the built in pullups aren't strong enough, causing the inputs to trigger sporadically and sometimes even "randomly" - possibly when lightning strikes... A series 470 ohm resistor prevents too much current going into or coming out of the pin in case of mis-connection, surge or even static to some extent.

I plan on designing some small input protection PCBs to use with all of my Sonoffs which will have proper ESD and surge protection by using clamp diodes, etc. These will be totally bullet-proof and could probably even handle a mains wire shorting to them!

Sunday, 3 December 2017

Home Assistant Hue Emulation

My Home Assistant setup uses nginx to proxy the web interface to HTTPS. I bought a Google Assistant for black friday and I want it to be able to control my lights... I haven't tested it yet but I figure I may as well start doing some setup!

The Home Assistant docs say that the emulated_hue component needs to run on port 80. This won't work by default because it doesn't have root permissions. I also didn't want to grant python the ability using cap_net_bind. My solution in the end was to create a NAT firewall rule on my server (see the end of this post). This takes anything that comes in via TCP on port 80 and redirects it to port 8300 (which is the port that Home Assistant was told to use for the emulated hue).

The issue with this is that nothing else that uses port 80 will work anymore - in particular nginx pages such as pi-hole. I figured I could just run pi-hole through HTTPS!

sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8300

and for localhost redirects too (optional):

sudo iptables -t nat -I OUTPUT -p tcp -d --dport 80 -j REDIRECT --to-ports 8300

To remove these rules:

iptables -t nat --line-numbers -n -L

HDMI to HDMI+Audio Teardown

I recently purchased an HDMI to HDMI+Optical and stereo analog output box from Banggood. I was curious at to what was inside as well as the build quality. It's pretty good! This post has photos as well as links to some of the components.

First off, why do I need one of these? My current TV and media PC setup is a little too spread out for my liking. The TV is mounted above the fireplace and the media PC, PS4, Chromecast, etc. are scattered around, but not above the fireplace obviously! I don't want a bunch of separate HDMI cables running up the wall (not to mention power cables) and then optical or coax S/PDIF cables running back down to the HiFi...

The solution is to use an HDMI selector ($10 or so!) and then afterwards place the audio extractor shown above. That way I get surround sound for every device (and I don't need to use the TV passthrough which doesn't really work anyway since it has to be on!), and I only need one remote control to select what I want to watch / listen to. This is really convenient for my home automation system too, since I can use one IR blaster to turn on everything, and simply send the commands to the selector and everything works... The selector actually has an external IR receiver so perhaps I'll make it work without IR :) stay tuned!

This is what I found inside when I cracked it open:

It's a good quality looking PCB without any dodgy soldering and nice looking solder mask / silk screen. What's important it the components though. In terms of power supply, the input is 5V and there is a 3.3V and 1.8V linear regulator immediately visible. Linear regulators have good noise performance compared to switch mode so this bodes well for potential buzzing on the analog audio output.

Interestingly the heart of the device is a large looking IC with markings rubbed off. This is probably to avoid royalties and other issues. I'm not complaining...

The Analog to Digital Converter (ADC) is the small 8 pin chip close to the RCA outputs. It's a CS4344 from Cirrus Logic. It's a pretty good ADC: 24 bit, 192 kHz stereo (obviously). The THD+N is pretty average at -90 dB which is about 0.003%. This is still a pretty good THD given that most affordable HiFi power amplifiers can only do 0.01% (and sometimes as bad as 0.1%). You'll probably find that your typical power amplifier at home is as bad as 10% THD when you crank it up - so nothing to complain about here!

Overall, having used this device for a few weeks now I'm happy with the performance! The only issue is that sometimes when I change HDMI source using the selector, this audio extractor doesn't immediately figure out that the source has changed and I need to power-cycle to get the audio back. It's pretty rare so no big deal! One other issue is that my BluRay player seems to know something is up, and it won't work through this device due to the HDMI encryption.

Wednesday, 29 November 2017

Mikrotik RB435/RB433 3D Printed Case

Although Mikrotik do sell a nice metal case for their excellent RB433 RouterBoards, I have a RB435 because I need gigabit Ethernet and there is no case for it... I figured a 3D printed one might do the trick!

I recently discovered Autodesk Fusion 360 which, as it turns out, is super easy to use. It took me about an hour to get to grips with the workflow and design the case below for my RB435:

I didn't really bother with accurate cutouts for each plug but it's a snug fit and works well. I also didn't want to deal with six screws. Instead, I opted to extrude 3.4 mm diameter pegs which the PCB slots onto. The top part of the case fits "tightly" over the pegs and holds the PCB in place too.

In addition, I added two SMA sized holes at the back for when I add a WiFi card and want to mount a couple of antennas.

You can download the STL files and print your own:

Wednesday, 30 August 2017

Arduino Sliding Gate Controller

As a part of my budding home automation system I am planning on adding some monitoring and control functionality to my home gate. The gate is of the sliding variety, and to be specific is the Centurion D5 which has a nice controller board with several good (and probably quite common) features. The datasheet is here.

I am using my ESPLive (ESP8266) board (without the mains power supply) and hooking it directly to the 12 V battery supply available on the controller board. I am in the process of implementing firmware with the following features:

  1. MQTT over WiFi for monitoring and control
  2. Output to trigger the gate to fully open
  3. Output to trigger the gate pedestrian open
  4. Monitor the gate's battery voltage (when it's low on charge, needs replacing, etc.)... it really sucks having a power failure and being stuck outside!
  5. Monitor the gate status LED to determine whether the gate is closed, opening, open, closing, faulty, etc.
All of the features are pretty straight forward, requiring simple IO and ADC. The trigger outputs could use a relay, but since the ground connection is the same for the ESP and the gate I am simply using an open collector transistor to pull the "PED" and "TRG" inputs low for pedestrian and full open operation.

The battery voltage monitoring is also simple. I am using a 10k and 150k resistor divider to the ADC input. This provides a nice round 0.0625 division of the battery voltage which is nominally 13.5V (0.84V at the ADC which has a 1V reference) and can safely measure up to 16V to cater for charger over-voltages, headroom, etc. If the battery measures less than 12.5V when the charger is connected then it's on it's way out or dead already.

The most interesting feature is monitoring the STATUS LED. According to the datasheet, this LED flashes at a different frequency, representing different things. I have summarised the statuses from page 45, below:
  • Off: gate is closed (if the gate lost power and didn't fully close, will it show as open or closed?)
  • On: gate is open
  • Slow continuous flash (frequency to be determined): gate opening
  • Fast continuous flash (frequency to be determined): gate closing
  • 1 flash/s: pillar light on
  • 2 flash/s: no mains
  • 3 flash/s: battery low
  • 4 flash/s: collision detected (need to wait 60s before trying to open/close again)
  • 5 flash/s: microprocessor reset
I need to do some testing to determine what "continuous flashing" entails. Hopefully the frequency of fast flashing is less than 1 flash/s - that way it's easy to tell the difference between the states.

I plan to implement a simple finite state machine to determine the statuses. I'll write more details when it's done!

Friday, 18 August 2017

ESPLive v2.0 : The "ultimate" mains connected ESP8266

I decided that although there are many ESP8266 boards out there these days - and even some which are "mains connected" such as the fantastic Sonoff boards - none really fit my requirements for flexibility. As a result, the ESPLive was born.

The ESPLive v2.0 prototype with USB-Serial connected.

The ESPLive is simply an ESP8266 based ESP12F module with all of the pins broken out and a few nifty features. Of course, the most obvious feature is the mains power supply... I opted for a MeanWell IRM-02-5S to supply 5V at 2W (for relays and other power hungry devices).
  1. I like the NodeMCU way of flashing and so I added a couple of transistors so that I never have to push any buttons while developing firmware. 
  2. Since there is only one ADC on the ESP8266, I also added an analog multiplexor IC which lets me switch between two inputs.
  3. Through selectively soldering only certain resistors and jumpers it is possible to have 2 separate inputs, one of which has an optional voltage divider for 3.3 V or 5 V range (or more). The second input is direct or has a burden resistor that is biased to 0.5 V for current measurement with a current transformer.
  4. There are two transistor driven outputs capable of 300 mA for driving relays. These outputs also have robust flyback diode protection with a zener and rectifier diode. See this app note for why the zener is a good idea.


I have used both espurna and Tasmota on these boards and I currently have one installed and running well with Tasmota in my Man Cave. The board is really small (about 3.5 x 5 cm) and so I wrapped it up in insulation tape and squeezed it into a light switch which had neutral available. A little bit dodgy but it works! ;)

I am working on some mods to the above firmwares (I haven't decided which is best yet) to control my how water geyser for maximum energy efficiency and also something to control my jacuzzi, gate, etc. 

I'm using Home-Assistant and Node-Red, which seem great so far! I'll post some articles on the setup at some point.

Home Assistant + Node-RED with BTRFS on Arch

After using Home Assistant for a few months and having set it up to use external MySQL databases, etc. I have decided to re-install in a rob...