Category Archives: MQTT

Android Phone as Server

Why am I showing you a picture of a cracked-screen phone?

Linux on Android PhoneWell because this particular phone is a bust Oukitel K10000, the phone with the world’s longest-lasting battery and an excellent workhorse. My friend Aidan gave me this (rather than chucking it away) some time ago and it has been sitting doing nothing. All that is wrong with it is a cracked (and exceedingly dangerous on the fingers) screen. I’ll bet I’m not the only one with such a phone lying around wasting space.

Well, as of yesterday, it is a Debian server with all my usual stuff going on quietly in the background – with the screen normally off – running freezing cold and hopefully being super reliable.

This is an experiment only – beware – if your phone or tablet dies it is your problem….  oh and your Android phone/tablet needs to be ROOTED. 

Imagine turning your old, dust-covered phone into a sleek, battery backed-up server with unfeasibly long backup time, immunity to any mains spikes, a silent, fast Debian base for all the stuff in my script – which includes Node-Red, Apache/PHP, SQLITE (+ PHPLiteAdmin), MQTT, MC, Ha-Bridge and more!  If you’ve been following this blog you’ll know about the script.

So this article applies to ROOTED Android phones and we’ve only tested around Android 5.0 onwards.  In my case I rooted the phone with KingRoot (note, NOT the one in the PlayStore which is a con – avoid it - but the one at kingroot.net ) – please note that rooting phones is beyond the scope of this article and if you are not confortable with this you should not do it. There are a lot of links out there on the subject and many of them are fraudulent.

tmpCD52There is an APP in the Play Store called Linux Deploy. It is also on GitHub. Beware that this is essentially undocumented unless you are Russian – so please don’t ask how you can use your phone’s GPS or Sound from Debian – not a clue!

You should have a modern-ish WiFi enabled (or hardwired if you like) Android phone or tablet with at least 16GB of internal FLASH (which should give you 10GB working space).   If you only have 8GB that will leave you only 2GB which - really – isn’t enough.

Getting Debian 8 on the phone:  After ensuring your phone/tablet is rooted, Install the App.

Linux on Android PhoneIn the app, on the top LEFT menu – find  REPOSITORIES and select Debian 8 for ARM.

On the bottom right is a drop down – you should go in there and select INSTALLATION TYPE – directory (OR FILE with a reasonable file size limit – say 5GB – the 2GB default will NOT work). Change the user to “pi” with suitable password in the same menu. TICK for allowing initialisation – and use of SSH server. Also where you see INIT SYSTEM change that to sysv. 

Then TOP RIGHT menu  - INSTALL – that might take some time – top right again CONFIGURE – then bottom menu START – and you should have a working Linux you can get into with for example WINSCP for that “pi” user. The IP address is clearly shown in the App.

I suggest not going further until you are comfortable with the above – took me several attempts because I didn’t follow the above exactly (well, and Antonio and I were learning at the same time).

Running the script: Via WinSCP or similar, put the script into the pi directory – changing permissions as normal - run the script – and ensure PHONE is selected – it isn’t by default. Come back after lunch. The script will no doubt suggest a reboot. Instead, hit the STOP button on the bottom of the phone screen – wait for it complete, hit the START button – wait – and you should now have everything in the script running!

Now – I’m using this to run my script – but I’m thinking you COULD use it to serve files etc. – I’ve not tried this but I’m guessing it would be easy to access an SD card…. and make that a folder…. hmmm.

Anyway, it is now the day after I installed all this – the phone is sitting there “off” and unlike my FriendlyArm board with it’s whirling fan, is sitting silently and freezing cold yet ran the script much faster than any of my SBCs – around 40 minutes.

K10000 running Debian 8No guarantees as there just isn’t enough info about Linux Deploy out there (unless you find/translate some) – but it does seem to work well now that we’ve made sufficient alterations to the script to take this particular setup into account. A fun project, to be sure. Now, I know this is a not a fair comparison and tomorrow we might come back and say … no good (for whatever reason).. but at £107 for that particular phone… compare – how much would it costs for a Raspberry Pi 3, uninterruptable power supply able to keep the phone going for something like a couple of days with no power, a high-def touch screen, a solid case for the whole thing.. indeed ,it might be possible to use a cheap tablet…  I was looking on Ebay – 16GB Android tablet – perfectly working except for missing buttons and cracked digitiser – around £10

One thing to note – I’ve turned rsyslog messages off – it was spitting out all sorts of unwanted helpful messages when you changed brilliance on the phone or disconnected the USB etc –REALLY annoying.. If you feel you need logging on -

sudo service rsyslog start

That will stay on until the next stop-start…

Node-Red running on a K10000 phoneSuch a shame it isn’t clear how to access some of the peripherals like audio. But who knows – someone reading this article may already have solved such challenges.

Please note: the pretty colours on the right do not come as standard. Click images to see larger versions.

This is really starting to look GOOD!!!!

Revelation: I’ve now taken on-board ideas from others and thanks for that – both people in here and on Google+ – most of the other solutions are longwinded and need other packaged  so up to now Linux Deploy – I’m now installing on my old Nexus 7 2012 UTTERLY SUCCESSFULLY (though not the fastest kid on the block) ( after rooting it with the Nexus toolkit ) - using Linux Deploy (which is completely self-contained and offers full root access – is looking the best). The ONLY thing you can’t do is use the Android peripherals – because of lack of UK info but this morning I figured it all out. 

We’ve also tested this one OnePlus One (model BACON) and a Xiaomi Redmi 3 model IDO). The K10000 has now been up for several days.

Ok, bear with me – you have Node-Red on Linux – and MQTT. So, you run Tasker on the phone (in the Android section) with MQTT – and now you have access to and control of all of the Android facilities that TASKER can handle (i.e. just about the lot) from within the Debian environment. Doddle.. now all I need is some time!!

Facebooktwittergoogle_pluspinterestlinkedin

Fauxmo Alexa Delights

Amazon EchoI’m going to cut a very long story short here.  I’ve been looking at ESP8266 emulations of WEMOs because up to now this seems to be a popular device to emulate when using with your talking Amazon Echo. If anyone wants to come up with something that will take ANY word, not just ON or OFF – please do feel free to do the hard work. In another article, I described how, with the aid of a small SKILL at the Amazon end, an SSL certificate and some Node-Red code, we managed to make a generalised system to extract words out of Echo – and that is controlling my heating and all sorts right now -  marvellous. If you’re not up to speed with ECHO, MQTT or NODE-RED see earlier blogs in here.

Simple On Off

But what if you just want ON and OFF with no HTTPS, no messing at the Amazon end?  I checked out ESP8266 emulations. AWFUL. I’ll not mention names and I’m sure this was just done as a demo – but the one I tested was written in Arduino/ESP code – connected to WIFI and acted as an ON/OFF Wemo – MARVELOUS. Except that with the first sign of WIFI trouble – it fell over and died – absolutely no attempt at making it work regardless. Well as you know, WIFI dies from time to time. Bin.

So what next?

I found something others will have spotted called FAUXMO which is basically a single Python script –that emulates one or more WEMO type devices. At first it did not work – so half a day later having learned enough Python to get me by I realised that this was intended to call web pages – to turn things on and off. I understood there was an MQTT version of it – but that a friend of mine was having trouble getting that working – so  - I decided to go for broke and modify the original.

All you need here is the Pi or similar equipped with MQTT and with Python installed – Node-Red is not needed. Indeed once you see how this works – it would be very easy to take the cheapest of these little SBCs – the FriendlyArm NEO for example and make it look like several devices – instead of MQTT you could turn ports on and off in Python.

Cheapest multi-output Alexa-enabled device on the planet?

Orange Pi ZeroBefore I go further – you might have a simple requirement – far simpler than we usually cover in here – a single box that “does it all”, talking to Amazon Alexa with multiple outputs looking like multiple devices… well, the Orange Pi ZERO available at under £9 (inc shipping) from AliExpress could just be the ticket? Install Armbian, MQTT and Python – and… read on.

PAHO Mqtt

You need a client called Paho MQTT to bring MQTT to Python… so here’s what I did in order… we’re looking at a Pi or similar, running Node-Red and MQTT broker. In my case with username and password.

sudo pip install paho-mqtt

Seconds later, done – MQTT now available inside Python – that was easy.

I had to add a header into the code – to import mqtt – and I had to define username, password, mqtt address.

That all went into the top of the file and should be pretty obvious.

I then modified some code – and the definitions of two sample devices – “kitchen lights” and “bedroom lights”.

Here are the modifications.

Top:

import paho.mqtt.client as paho

mqtt_host="192.168.aa.bb"
mqtt_user="xxxxxxx"
mqtt_pass="yyyyyyyyy"

Change to how recognised terms are actually used

class rest_api_handler(object):
    def __init__(self, on_cmd, off_cmd):
        self.on_cmd = on_cmd
        self.off_cmd = off_cmd

    def on(self):
        dbg(self.on_cmd)
        topic,payload=self.on_cmd.split("|")
        client.publish(topic,payload,0,0)
        return 200

    def off(self):
        dbg(self.off_cmd)
        topic,payload=self.off_cmd.split("|")
        client.publish(topic,payload,0,0)
        return 200

And finally the bit you're interested in - device definitions... you only have to change the user settings at the top then this lot.

FAUXMOS = [
    ['bedroom lights', rest_api_handler('toesp/bedlights|{out0:1}', 'toesp/bedlights|{out0:0}')],
    ['kitchen lights', rest_api_handler('toesp/kitchenlights|{out0:2}', 'toesp/kitchenlights|{out0:3}')],
	]

So what you’re looking at are the words that Alexa will accept (“kitchen lights” for example) followed by ON OR OFF. The two following strings are MQTT topic and PAYLOAD separated by | for firstly ON and secondly OFF.

As you can see it is easy to add new devices – you do with the MQTT what you will – in my case this would be talking directly to devices – more likely I’ll fire instructions off that Node-Red can handle for manual override of lighting etc.

To get this all to work you need to be running the program and say “Alexa find devices” to your ECHO – it will find the devices and you’re ready to go. If you make changes – restart the program – re-find devices.

And here  is the complete – modified Python code – complete with the original author info – after all I didn’t make THAT many changes even if it did end up taking me all day.

Don’t worry about it looking over-powering – you only need to know about the stuff above – the rest is just a case of pasting into into a file.

#!/usr/bin/env python

"""
The MIT License (MIT)
Copyright (c) 2015 Maker Musings
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""

# For a complete discussion, see http://www.makermusings.com

import email.utils
import requests
import select
import socket
import struct
import sys
import time
import urllib
import uuid
import paho.mqtt.client as paho

mqtt_host="192.168.aa.bb"
mqtt_user="xxxxxxxx"
mqtt_pass="yyyyyyy"


# This XML is the minimum needed to define one of our virtual switches
# to the Amazon Echo

SETUP_XML = """<?xml version="1.0"?>
<root>
  <device>
    <deviceType>urn:MakerMusings:device:controllee:1</deviceType>
    <friendlyName>%(device_name)s</friendlyName>
    <manufacturer>Belkin International Inc.</manufacturer>
    <modelName>Emulated Socket</modelName>
    <modelNumber>3.1415</modelNumber>
    <UDN>uuid:Socket-1_0-%(device_serial)s</UDN>
  </device>
</root>
"""


DEBUG = False

def dbg(msg):
    global DEBUG
    if DEBUG:
        print msg
        sys.stdout.flush()


# A simple utility class to wait for incoming data to be
# ready on a socket.

class poller:
    def __init__(self):
        if 'poll' in dir(select):
            self.use_poll = True
            self.poller = select.poll()
        else:
            self.use_poll = False
        self.targets = {}

    def add(self, target, fileno = None):
        if not fileno:
            fileno = target.fileno()
        if self.use_poll:
            self.poller.register(fileno, select.POLLIN)
        self.targets[fileno] = target

    def remove(self, target, fileno = None):
        if not fileno:
            fileno = target.fileno()
        if self.use_poll:
            self.poller.unregister(fileno)
        del(self.targets[fileno])

    def poll(self, timeout = 0):
        if self.use_poll:
            ready = self.poller.poll(timeout)
        else:
            ready = []
            if len(self.targets) > 0:
                (rlist, wlist, xlist) = select.select(self.targets.keys(), [], [], timeout)
                ready = [(x, None) for x in rlist]
        for one_ready in ready:
            target = self.targets.get(one_ready[0], None)
            if target:
                target.do_read(one_ready[0])
 

# Base class for a generic UPnP device. This is far from complete
# but it supports either specified or automatic IP address and port
# selection.

class upnp_device(object):
    this_host_ip = None

    @staticmethod
    def local_ip_address():
        if not upnp_device.this_host_ip:
            temp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
            try:
                temp_socket.connect(('8.8.8.8', 53))
                upnp_device.this_host_ip = temp_socket.getsockname()[0]
            except:
                upnp_device.this_host_ip = '127.0.0.1'
            del(temp_socket)
            dbg("got local address of %s" % upnp_device.this_host_ip)
        return upnp_device.this_host_ip
        

    def __init__(self, listener, poller, port, root_url, server_version, persistent_uuid, other_headers = None, ip_address = None):
        self.listener = listener
        self.poller = poller
        self.port = port
        self.root_url = root_url
        self.server_version = server_version
        self.persistent_uuid = persistent_uuid
        self.uuid = uuid.uuid4()
        self.other_headers = other_headers

        if ip_address:
            self.ip_address = ip_address
        else:
            self.ip_address = upnp_device.local_ip_address()

        self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.socket.bind((self.ip_address, self.port))
        self.socket.listen(5)
        if self.port == 0:
            self.port = self.socket.getsockname()[1]
        self.poller.add(self)
        self.client_sockets = {}
        self.listener.add_device(self)

    def fileno(self):
        return self.socket.fileno()

    def do_read(self, fileno):
        if fileno == self.socket.fileno():
            (client_socket, client_address) = self.socket.accept()
            self.poller.add(self, client_socket.fileno())
            self.client_sockets[client_socket.fileno()] = client_socket
        else:
            data, sender = self.client_sockets[fileno].recvfrom(4096)
            if not data:
                self.poller.remove(self, fileno)
                del(self.client_sockets[fileno])
            else:
                self.handle_request(data, sender, self.client_sockets[fileno])

    def handle_request(self, data, sender, socket):
        pass

    def get_name(self):
        return "unknown"
        
    def respond_to_search(self, destination, search_target):
        dbg("Responding to search for %s" % self.get_name())
        date_str = email.utils.formatdate(timeval=None, localtime=False, usegmt=True)
        location_url = self.root_url % {'ip_address' : self.ip_address, 'port' : self.port}
        message = ("HTTP/1.1 200 OK\r\n"
                  "CACHE-CONTROL: max-age=86400\r\n"
                  "DATE: %s\r\n"
                  "EXT:\r\n"
                  "LOCATION: %s\r\n"
                  "OPT: \"http://schemas.upnp.org/upnp/1/0/\"; ns=01\r\n"
                  "01-NLS: %s\r\n"
                  "SERVER: %s\r\n"
                  "ST: %s\r\n"
                  "USN: uuid:%s::%s\r\n" % (date_str, location_url, self.uuid, self.server_version, search_target, self.persistent_uuid, search_target))
        if self.other_headers:
            for header in self.other_headers:
                message += "%s\r\n" % header
        message += "\r\n"
        temp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        temp_socket.sendto(message, destination)
 

# This subclass does the bulk of the work to mimic a WeMo switch on the network.

class fauxmo(upnp_device):
    @staticmethod
    def make_uuid(name):
        return ''.join(["%x" % sum([ord(c) for c in name])] + ["%x" % ord(c) for c in "%sfauxmo!" % name])[:14]

    def __init__(self, name, listener, poller, ip_address, port, action_handler = None):
        self.serial = self.make_uuid(name)
        self.name = name
        self.ip_address = ip_address
        persistent_uuid = "Socket-1_0-" + self.serial
        other_headers = ['X-User-Agent: redsonic']
        upnp_device.__init__(self, listener, poller, port, "http://%(ip_address)s:%(port)s/setup.xml", "Unspecified, UPnP/1.0, Unspecified", persistent_uuid, other_headers=other_headers, ip_address=ip_address)
        if action_handler:
            self.action_handler = action_handler
        else:
            self.action_handler = self
        dbg("FauxMo device '%s' ready on %s:%s" % (self.name, self.ip_address, self.port))

    def get_name(self):
        return self.name

    def handle_request(self, data, sender, socket):
        if data.find('GET /setup.xml HTTP/1.1') == 0:
            dbg("Responding to setup.xml for %s" % self.name)
            xml = SETUP_XML % {'device_name' : self.name, 'device_serial' : self.serial}
            date_str = email.utils.formatdate(timeval=None, localtime=False, usegmt=True)
            message = ("HTTP/1.1 200 OK\r\n"
                       "CONTENT-LENGTH: %d\r\n"
                       "CONTENT-TYPE: text/xml\r\n"
                       "DATE: %s\r\n"
                       "LAST-MODIFIED: Sat, 01 Jan 2000 00:01:15 GMT\r\n"
                       "SERVER: Unspecified, UPnP/1.0, Unspecified\r\n"
                       "X-User-Agent: redsonic\r\n"
                       "CONNECTION: close\r\n"
                       "\r\n"
                       "%s" % (len(xml), date_str, xml))
            socket.send(message)
        elif data.find('SOAPACTION: "urn:Belkin:service:basicevent:1#SetBinaryState"') != -1:
            success = False
            if data.find('<BinaryState>1</BinaryState>') != -1:
                # on
                dbg("Responding to ON for %s" % self.name)
                success = self.action_handler.on()
            elif data.find('<BinaryState>0</BinaryState>') != -1:
                # off
                dbg("Responding to OFF for %s" % self.name)
                success = self.action_handler.off()
            else:
                dbg("Unknown Binary State request:")
                dbg(data)
            if success:
                # The echo is happy with the 200 status code and doesn't
                # appear to care about the SOAP response body
                soap = ""
                date_str = email.utils.formatdate(timeval=None, localtime=False, usegmt=True)
                message = ("HTTP/1.1 200 OK\r\n"
                           "CONTENT-LENGTH: %d\r\n"
                           "CONTENT-TYPE: text/xml charset=\"utf-8\"\r\n"
                           "DATE: %s\r\n"
                           "EXT:\r\n"
                           "SERVER: Unspecified, UPnP/1.0, Unspecified\r\n"
                           "X-User-Agent: redsonic\r\n"
                           "CONNECTION: close\r\n"
                           "\r\n"
                           "%s" % (len(soap), date_str, soap))
                socket.send(message)
        else:
            dbg(data)

    def on(self):
        return False

    def off(self):
        return True


# Since we have a single process managing several virtual UPnP devices,
# we only need a single listener for UPnP broadcasts. When a matching
# search is received, it causes each device instance to respond.
#
# Note that this is currently hard-coded to recognize only the search
# from the Amazon Echo for WeMo devices. In particular, it does not
# support the more common root device general search. The Echo
# doesn't search for root devices.

class upnp_broadcast_responder(object):
    TIMEOUT = 0

    def __init__(self):
        self.devices = []

    def init_socket(self):
        ok = True
        self.ip = '239.255.255.250'
        self.port = 1900
        try:
            #This is needed to join a multicast group
            self.mreq = struct.pack("4sl",socket.inet_aton(self.ip),socket.INADDR_ANY)

            #Set up server socket
            self.ssock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM,socket.IPPROTO_UDP)
            self.ssock.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)

            try:
                self.ssock.bind(('',self.port))
            except Exception, e:
                dbg("WARNING: Failed to bind %s:%d: %s" , (self.ip,self.port,e))
                ok = False

            try:
                self.ssock.setsockopt(socket.IPPROTO_IP,socket.IP_ADD_MEMBERSHIP,self.mreq)
            except Exception, e:
                dbg('WARNING: Failed to join multicast group:',e)
                ok = False

        except Exception, e:
            dbg("Failed to initialize UPnP sockets:",e)
            return False
        if ok:
            dbg("Listening for UPnP broadcasts")

    def fileno(self):
        return self.ssock.fileno()

    def do_read(self, fileno):
        data, sender = self.recvfrom(1024)
        if data:
            if data.find('M-SEARCH') == 0 and data.find('urn:Belkin:device:**') != -1:
                for device in self.devices:
                    time.sleep(0.1)
                    device.respond_to_search(sender, 'urn:Belkin:device:**')
            else:
                pass

    #Receive network data
    def recvfrom(self,size):
        if self.TIMEOUT:
            self.ssock.setblocking(0)
            ready = select.select([self.ssock], [], [], self.TIMEOUT)[0]
        else:
            self.ssock.setblocking(1)
            ready = True

        try:
            if ready:
                return self.ssock.recvfrom(size)
            else:
                return False, False
        except Exception, e:
            dbg(e)
            return False, False

    def add_device(self, device):
        self.devices.append(device)
        dbg("UPnP broadcast listener: new device registered")


# This is an example handler class. The fauxmo class expects handlers to be
# instances of objects that have on() and off() methods that return True
# on success and False otherwise.
#
# This example class takes two full URLs that should be requested when an on
# and off command are invoked respectively. It ignores any return data.
# 
# Clearly somewhat modified by PS to take in a string - split in two and send out to MQTT
class rest_api_handler(object):
    def __init__(self, on_cmd, off_cmd):
        self.on_cmd = on_cmd
        self.off_cmd = off_cmd

    def on(self):
        dbg(self.on_cmd)
        topic,payload=self.on_cmd.split("|")
        client.publish(topic,payload,0,0)
        return 200

    def off(self):
        dbg(self.off_cmd)
        topic,payload=self.off_cmd.split("|")
        client.publish(topic,payload,0,0)
        return 200


# Each entry is a list with the following elements:
#
# name of the virtual switch
# object with 'on' and 'off' methods
# port # (optional; may be omitted)

# NOTE: As of 2015-08-17, the Echo appears to have a hard-coded limit of
# 16 switches it can control. Only the first 16 elements of the FAUXMOS
# list will be used.

# Clearly the above is changed. Here we have MQTT commands in the form topic|payload

FAUXMOS = [
    ['bedroom lights', rest_api_handler('toesp/bedlights|{out0:1}', 'toesp/bedlights|{out0:0}')],
    ['kitchen lights', rest_api_handler('toesp/kitchenlights|{out0:2}', 'toesp/kitchenlights|{out0:3}')],
	]


if len(sys.argv) > 1 and sys.argv[1] == '-d':
    DEBUG = True

# Set up our singleton for polling the sockets for data ready
p = poller()

# Set up our singleton listener for UPnP broadcasts
u = upnp_broadcast_responder()
u.init_socket()

# Add the UPnP broadcast listener to the poller so we can respond
# when a broadcast is received.
p.add(u)

# Create our FauxMo virtual switch devices
for one_faux in FAUXMOS:
    if len(one_faux) == 2:
        # a fixed port wasn't specified, use a dynamic one
        one_faux.append(0)
    switch = fauxmo(one_faux[0], u, p, None, one_faux[2], action_handler = one_faux[1])

dbg("Entering main loop\n")

client = paho.Client()
client.username_pw_set(mqtt_user, mqtt_pass)
client.connect(mqtt_host)

while True:
    try:
        # Allow time for a ctrl-c to stop the process
        p.poll(100)
        time.sleep(0.1)
    except Exception, e:
        dbg(e)
        break

client.disconnect()

(if this lot complains about spacings - get the original article and make my changes

So – put the file in a directory – I put it into /home/pi/fauxmo/fauxmo.py and gave the file execute permissions (no idea if that’s needed or not but as I do that with scripts I thought it best - 0754 if you're interested).

I ran the file in the /home/pi/fauxmo folder at a terminal as:

python fauxmo.py –d

The –d is for debug – you can scratch that once you are running – and then find a way to make it run reliably on power up. That is possibly described here:

https://github.com/makermusings/fauxmo  if someone has the working command line for this please do send it in – might save some experimenting.

And that’s it. Great fun and instantly useful without lots of hassle. With Node-Red you could fire the MQTT in to do with what you want – or fire straight to MQTT-accepting ESP8266s (what use are ESPs that DON’T support MQTT you might ask) – or if you look at the chunk of code handling MQTT – you could pull that apart and instead just control ports!!

So here’s a thought – one of our readers was kind enough to remind me that the Orange Pi Zero is now available and it is CHEAP. Normally I’d not touch Orange with a bargepole but the Armbian Jessie implementation seems to be now up to speed. SO - https://www.aliexpress.com/store/product/New-Orange-Pi-Zero-H2-Quad-Core-Open-source-development-board-beyond-Raspberry-Pi/1553371_32760774493.html

Under £9 for the cheapest, might not be that happy running Node-Red but it will certainly run PYTHON and MQTT (well, you don’t even need MQTT for that matter if you just want to control the odd output or send stuff through serial etc)… so for under £9 you could have several Alexa-enabled devices !!! That can’t be bad.

This will no doubt mutate as time goes on!

Facebooktwittergoogle_pluspinterestlinkedin

A5 V11 Router

A5-11 RouterOr – the cheapest router on the planet – probably. I should say at the outset this experiment did not end too well – currently waiting for another unit to arrive – but worth a read in case you’re about to embark on using this unit.

I had to pick one up – at just a few quid (I paid £3.82 but I note they’ve put the price up to £4.41) I could not help myself. 2 weeks ago I ordered this unit from China and this morning it turned up.

Of course once I’d received it (in a very pretty package comprising the 3g router, a USB lead and instructions in Chinese and English) I had to go off to find out how to actually make use of it.

Here’s the specification for those interested:

  • SoC – Mediatek/Ralink) RT5350F MIPS processor @ 360MHz
  • System Memory – 32MB RAM (W9825G6EH-75).
  • Storage – 4MB NAND flash (Pm25LQ032)
  • Connectivity – Wi-Fi 802.11b/g/n up to 150 Mbps; 1x 10/100M Ethernet
  • USB – 1x USB 2.0 host port, 1x micro USB port for power
  • Misc – Power LED, factory reset pinhole
  • Power – 5V via micro USB port
  • Dimensions –  6.1 x 2.3 x 1.4 cm

     

    So – typically instructions on the web were not at first a lot of use - “The unit will default to 192.168.100.1. Plug into your network and connect” – log into it… well that’s fine except you CAN’T unless you happen to have a 192.168.100.x setup !!

    shellAnyway as it turns out  - when I plugged power into the little unit, and plugged a network cable in, it adopted an address on my network – 192.168.1.42

    web interfaceAnd so off I went to find Telnet on my PC – and sure enough with username and password admin and admin I found an utterly useless shell! But then I noted in the instructions something about a web page – seems reasonable as routers usually have web pages – and lo – a web interface!!

    A good start –of course the idea of buying this was to fit it out with openWRT and MQTT to truly make the cheapest MQTT-based home network on the planet.. but I had to have a play in the meantime. Considering these things have 4Mbit of Flash, this is not a bad start.

    I could not test the 3g side as I don’t have a 3g dongle (my phone has data for months over here in Spain before THREE decide I’ve been here too long so no need for one). So I tried changing the WLAN settings – surprising amount of info in there – well impressed at the price.

    The WIFI signal, with the unit lying horizontally was somewhat down from a “proper” router but sitting vertically, from the next room I noted a signal almost as good as my TP-Link.

    SpeedtestI decided, silly as it seemed to go for a speed test. I mean – what DO you expect for £4?

    I connected my phone via WIFI to the router. Incidentally all of the operations I tried needed a reboot of the unit - I guess they had to take shortcuts somewhere - but no problem as it only took a few seconds each time. With the dish-connected WIFI we have here in Galera, I get around 10Mbps download and 3Mbps upload. I ran the speed test (Speedtest.net) on my phone while using the unit as a router – no more than a metre away from the little unit. I have to say, impressed I was!!! I’m thinking of ordering another one just for that use alone! 

    I mean, if nothing else, just for some extra range in the house without cluttering the place up with kit – this thing is smaller than the power supply you need to power it!!! Apparently we’re looking at 300ma at 5v (so, 1.5w) so the most rubbish USB supply would do the job. The unit itself was at this point slightly warm, nothing more. For heaven’s sake it even has NTP time synchronisation.

    Next, I noted something about “network storage” – which was enabled… I grabbed a handy FAT32-formatted USB stuck and put a (780MB) movie on it – and plugged the USB stick into the router unit (this is all guesswork)… I noted that the stick was not hing special speed-wise as it had taken an age to copy the movie over (nearly 3 minutes) on my fast PC.

    Here’s where I hit a snag – despite network storage being enabled-  I could not figure out how I might use this – the IP address was already being used for the web interface – and the instructions for network storage were non-existent. A quick look on my PC for devices showed nothing.

    So, on the off-chance, I went into file explorer on the PC and typed \\192.168.1.42  -  sure enough it asked for a user and password – the default was still set at admin and admin – and lo – a directory called MEDIA appeared. Within that directory was a folder for system information-  and my movie.  I clicked on the movie…. and sure enough – it worked.

    RouterWell impressed by all of this – I left that running on the PC… no stuttering…. just working. I went to the phone which was still at this point connected to the little unit wirelessly. ES File Explorer – streaming media player – sure enough – the movie played simultaneously on my PC (hardwired to the net work) and the HTC One M8 phone.

    At this point I was beginning to wonder if I really wanted to upgrade this little unit to something else – it just works SO well for it’s size and price. DMZ, DDNS – oh and of course the point is it is supposed to provide WIFI and WIRED access from a dongle - but I could not test the latter.

    Static routing and MAC-Binding – how on EARTH did they fit all this stuff in such a small space and for that price – this thing has more useful features than the utterly useless router that Plusnet gave me, back in the UK!!!

    I used the little SHELL software I referred to earlier – and despite the list of instructions being minimal – actually, it responds to far more instructions including cat /proc/meminfo which immediately told me this unit had the full 32MB of memory -  apparently other units often have only 16GB.

    Feeling REALLY ambitious? Check out this guy’s video and blog – he apparently has DEBIAN running on it!!!

    http://board.nwrk.biz/viewtopic.php?pid=342

    https://www.youtube.com/watch?v=pZLU15zx1EU

    upgradingIn reality, all I wanted was to try Open Router for the ability to run MQTT on this little box.  So, using software from here… I used the standard upgrade option in the menu to load the image – with a warning that a bad image would BRICK the router…. 2 minutes later – everything said “100%” – but of course – no more visuals as it was sitting on the same address as my main router at 192.168.1.1  presumably?

    By now I was wondering if this was such a good idea…. the original blue light on the little router was now flashing white and blue!!!  I figured that MUST be a good sign and took the whole lot over to my laptop and plugged the router into the laptop – so that this was a completely separate network.

    Erm, no. After a couple of reboots, I ended up with the device still fitted with original Qualcomm software – and now on it’s own address range of 192.168.100.1 – I guessed that – and sure enough – it was running DHCP. I changed the address to that of my network – and disabled DHCP and I was back to square one. (it turns out you need a better bootloader as this one won’t take unofficial software updates!!)

    And that’s where I am up to now – this I can see is going to be a long blog… but I thought you might like to see where I’m up to – even at this point I think this is a must-have gadget at the price. If I can get MQTT on it – that’ll be a bonus.

    Right now – to update this unit to OpenWrt – you need to put in a new bootloader, the original won’t let you use a non-official image…. that’s fine – except the TFTP to let you put that in – isn’t supplied with the module. Also you need to load a smaller version of OpenWrt to load the bigger one – and the links for that are dead (along with most of the links for the bootloader). Some very much out of date material out there.  If anyone has a known working Busyboxlink (with known working links in it) I’m sure lots of us would benefit. I found a usable link for the bootloader - https://github.com/wertwert4pda/rt5350f-uboot

    But I don’t THINK that is useful on it’s own. Some links suggest putting the file on a website and WGET it – no wget in BusyBox shell!!!

    After changing the IP address of the unit and turning off DHCP – I found that all worked well until the next reboot – and then the unit would not come back to the same address or indeed respond to any address -  and needed restoring to factory settings with a pin. 

    At this point I was considering cutting my wrists when I stumbled across this. https://hackaday.io/project/11037-running-rtl-sdr-on-a5-v11-3g4g-router

    Off I went – and sure enough the files I needed were there.  I realised there was a wget command in there and as usual “just whip out your Linux computer”  to run a webserver to serve the file up – right. Anyway, I discovered MONGOOSE web server – a one-file webserver that sets up shop in the directory in which you run the program – marvelous.

    With that I was able to follow the instructions in the Hackaday blog. Well, almost – the file name was not the same as the on in the blog – with 6 extra letters on the end (bear that in mind if you try this), the file once grabbed with WGET was not in the /tmp directory contrary to the instructions – (I used ./ instead) and once all of that was done  - the address of the new router was not 192.168.1.1 as promised but 192.168.100.1 – that cost me half an hour Smile Oh – and the –vn command in the instructions turns out to be a –n command !!

    Anyway at the end of it all I ended up being able to Telnet (Windows command line with address as parameter) to the new OpenWrt router on my little sub-£4 box.

    But all was not well (again) – There was an instruction to change the password – and this was also suggested in the blog above.  Simple enough except when I did that it said “No space left on device” – not enough even to put a new password!!?!?!! Then I remembered of course that I’d left the openWrt install file in place – so using RM I got rid of that and NOW I could update my password. I was expecting at this point the Telnet service to stop working and SSH to start working.

    And so it did – but can I HELL log in with SSH – I have Bitwise SSH client on the PC – I can see it knows about the router as other addresses provide different error messages but it simply WILL NOT log in.. 

    Update 7/7/2016: WELL it turns out that the version of openWRT I used didn’t have SSH activated – yes, amazingly after telling people to put a new password in – and then rely totally on SSH – it was not installed.

    It turns out that inside the unit  there are 4 connectors – 3 are relevant as they are ground, serial in, serial out. They can be attached to an FTDI at 57600 giving access to the shell (a 2013 version I may add). I changed the address in /etc/config/networks  using the built in VI editor – to one in my address range and added a gateway. This then enabled me to grab another version of the openWRT software… here ..https://wiki.openwrt.org/toh/unbranded/a5-v11 and the link I used was  http://192.168.100.249:8080/openwrt-15.05-ramips-rt305x-a5-v11-squashfs-sysupgrade.bin

    On reboot I could see I had a NEW shell (2015) and I set the password again – and set the address again as this was all lost in the update.

    NOW I could SSH into the unit and use the shell in a terminal.

    In theory all that is left is to mount a memory stick – expand available space and add in the full graphical environment? Well, plugging in a memory stick works – it is acknowledged in the shell – erm, but no SDA1 appears in DEV…

    I started to follow these instructions for adding block device support

    opkg update
    opkg install kmod-usb-storage block-mount kmod-fs-ext4
    mkswap /dev/sda1
    swapon /dev/sda1
    mkdir -p /mnt/share
    mount -t ext4 /dev/sda2 /mnt/share -o rw,sync

    And about half way through the second line I ran out of storage on the device!! However the installation continued. At the end of line 2 above I was getting pretty depressed – but checked – and sure enough I had an SDA and SDA1 in /DEV/

    But no room to do anything – not even to create /mnt/sda1 or similar. Just utterly out of room. and finally – after trying various image with varying success I tried the image in this project - https://github.com/sternlabs/RT5350F-cheap-router and – BRICKED. Project on hold until another unit arrives Smile

  • Facebooktwittergoogle_pluspinterestlinkedin

    Gear S2 Classic and Home Control

    For a change from processor boards, let me tell you about my new WATCH.

    Scargill watchFor a long time I’ve been a faithful Pebble user, the watch has a week’s battery life, a display you can read in sunlight and a host of applications.  I had one of the very originals which sadly did not survive a swimming expedition in a saltwater hotel pool and the guys at Pebble replaced it without question. However things move on and I was sitting with a neighbour perhaps 3 weeks ago now and I notice his wrist hovering as if to say “you need this, Peter”.

    He flicked his wrist and the most wonderful Tag Heuer display popped up.  a beer later and he showed me the watch again – this time with a Gucci watch face. I could bear it no longer.

    So, several days later my Gear S2 Classic turned up. It is a work of art, to put it mildly.  I’ll get the bad bit out of the way first – the battery lasts 2-3 days – but the good part of that is the proximity charger, another work of art you just drape the watch over with no connection to charge – in VERY little time, less than an hour and it’s ready to go.

    This alone with the pictures should be enough to make you get your wallet out – but then I remembered I have TASKER on my mobile phone (HTC ONE M8) – and that talks to MQTT. Could I be so lucky?

    YES – there’s an APP for that. A small add-on APP for the watch lets me scroll through all my Tasker tasks and run them at a touch. If you’ve used tasker – you’ll know you can do just about anything with it – including (with an add-on) send MQTT messages.

    If you, like me, have no artistic skills then you can create your own watch-face in no time at all like the one I’ve shown on the right – done right in Mr Time Maker’s web page – but if you have a little skill you can make a DAMNED good emulation of just about any watch on the planet.

    Just as an aide the watch has technology underneath to do your pulse rate and there is also the usual health stuff like how many steps you’ve taken, weather locally, news, alerts etc.. the list is endless and growing by the minute.

    So now not only do I have a black-leather-strapped work of art attached to my wrist (you really DO have to see the display – which, depending on watch face (and there are many hundreds of them thanks to Mr Time Maker app) can be seen in reasonably bright (Spanish) sunshine, but also I can control all my household gadgets in comfort.

    The links in the above article should help you. Enjoy. If interested do check pricing though for the watch. It varies dramatically.

    Facebooktwittergoogle_pluspinterestlinkedin

    Remote checking Home Control Units

    Those of you who are aware of the Home Control project (Home Control 2016) will know that the software/hardware combination of a Raspberry Pi (running Jessie, Node-Red, SQLite and Mosquitto) and ESP12 (running the software described in the Home Control project) is proving very reliable – but there are occasions when you may wish to be absolutely sure that units are actually doing something.  One way to do that is simply to put timeouts on every response you expect… but that makes for complicated coding. A simpler way is to regularly ask each unit for the status of one of the outputs. It doesn’t matter what the response is, as long as there is a response.

    As it happens I’m having an issue right now – the heating controller part of my system back home in the UK is working perfectly except of an on-going issue with Plusnet broadband. They are denying it is their fault but the broadband fails perhaps once a day, maybe more, maybe once every two days but eventually it will fail. Most of the time the TP-Link TD-9980 router reconnects within a minute or so – but for reasons I don’t yet understand, occasionally it just will not reconnect. The solution to that was simple and two-fold, a standard timer which turns the router off for a minute in the early hours every day. That’s the “belt and braces” approach. In addition, the Raspberry Pi is polling Google every few minutes. If it fails totally over a 15 minute period to contact Google, an ESP8266 unit will reboot the router.

    All of this worked perfectly for several days and then… the heating relay ESP8266 refused to log back in after one of these “cuts”. I can only put it down to being right on the edge of signal. The reason I say that is that late last year we had horrendous problems with WIFI here in Spain, some kind of attack. It went away but during that time over several days I was losing WIFI constantly so I made damned sure the ESP units would reliable log back in, in the event of temporary failure of MQTT or WIFI or both etc. And yet just occasionally this particular unit will not log back in – a power cycle fixes the problem but that means I need to know about it and ask someone to pull the plug for a moment.

    So – I’ve implemented this for the two most important devices in the building – the main heating temperature sensor – and the ESP that controls the relay to turn the heating on and off.

    watchdog

    Timeout NodeWhat you see above is as follows: 2 Node-Red “inject” nodes (standard) fire out requests every 5 minutes to MQTT to ask the relay and temperature units for the state of output 0. 

    Which command is irrelevant as long as it is a query which returns a result and otherwise has no effect on anything.  So for example with a topic holly1/toesp and a payload of {out0?} the system is expecting a response of OFF or ON (irrelevant here) from topic holly1/fromesp/out0

    The two incoming MQTT nodes (left, purple) pick up on the response and feed them into my “timeout” nodes  (node-red-contrib-timeout).

    As you’ll see below, the operation is simple – the timeout node doesn’t care what is incoming as long as something is. if it gets nothing for 60 minutes it will time out and send a payload out – in this case thanks to the standard email node (green) an email to me. 

    During that time, normally, the MQTT nodes would have responded every 5 minutes – ie 20 times, topping the timeout node up to 3600 seconds even if most of the messages were missing (and there is no reason why any should be missing).  In this case I’ve not put messages for “safe state” or “warning” – so nothing goes out unless the timeout actually runs down to zero.

    Click on any of these images for larger versions.

    So while this does not solve the problem, it does mean that I’m alerted to the issue and can ask someone to disconnect and reconnect the relay unit. With what has been 100% reliability of Raspberry Pi 2 boards (with a decent Samsung microSD or similar) I can be pretty sure I’ll get an email if something goes wrong.

    I hope this proves useful to others.

    Facebooktwittergoogle_pluspinterestlinkedin

    Home Control 2016

    Introduction (Latest update December 15, 2016)

    HomeSick of reading about other people’s home control? Want to make your own? Got some electronics and software experience? You’re going to LOVE THIS! Rock-solid home control – low cost, DIY.

    If you’ve read my early blogs you’ll know that Aidan Ruff and I have been working in home control since the late 20th century. Together, we ran an electronics R&D company for many years and one of our products was a home control system called APPCON.

    That range of products  achieved much coverage in the UK tech press at the time and many loved it but the design involved spouse-unfriendly WIRES – bad mistake. So now we’re back – this time for fun –  with a vengeance!

    Home Control 2016

    Add to that lot above, the SI1132 unit which returns light values for visible, UV and IR - and a host of other I2c boards around the corner.

    By the time you look at the diagram above it will likely be out of date as new features are added – thankfully there’s a manual along with source code and diagrams available online and kept up to date. This is all free incidentally and without warranty. We have software, diagrams, board layouts… and this blog has a range of articles on the subject.

    The diagram pretty much says it all – a Raspberry Pi (2 or 3) or similar,  talking to your PC or smartphone via a range of potential technologies – and using a powerful communications protocol called MQTT over WIFI to talk to small yet incredibly powerful ESP8266 units – these can be bought very cheaply and programmed – or you can take our board designs and have some boards made in China if you really want to DIY (we do!).

    There are no real limits to the number of boards you can have talking to the central controller – that is down, in the main, to your WIFI network which could cover one room or extend across the globe.

    In my own system I have 3 properties talking to each other – one in the UK, one in Spain. No matter where I am I have full access – great fun, great utility, low cost. Who wants to pay £50 to control a light! This is definitely aimed at the DIYer.

    In the coming link there is a complete project, suitable for Windows development using the ESP8266 “Unoffical Development Environment” for the ESP8266 – but that won’t stop Linux users from bending the makefile to their will!!

    There is a DOC file here which is fully up to date with all commands – this blog entry just shows a selection. Even if you ignore or don’t understand the source and just download the ROMS available – you’ll be able to keep up to date via OTA (over the air updating) – and you’ll need that manual!

    You may also need some new – and very useful skills as you’ll see the term Node-Red appear very often throughout the considerable range of blog entries in here – this is by NO means a single item – there’s a Nextion-based, prize-winning WIFI touch display blog and a range of discussions (and there are LOTS of valuable comments) throughout the blog – enjoy learning and please do feel free to share knowledge – there is SO much more we can do with this.

    Update on flashing ROMS here:  http://tech.scargill.net/esp8266-home-control-update/  - Blog reader Jay has written a great blog on setting up the programming environment for Windows.

    ESP8266-terminalWhat is missing from the diagram above is my first experiments with SPI. Using up GPIO13 to 16, I have a 320*240 display working experimentally as a fast scrolling terminal on the ESP8266. Handy for debugging the system without tying up your PC monitor.

    How useful this is depends on how many other pins you need! We need an ESP8266 with many more IO pins – I guess that will be the ESP-32…

    Background: I’ve been using ESP8266 chips since they first came out – and I have the development kit for their new ESP-32 –just waiting for some decent software tools to emerge but right now the ESP8266 is a lot of fun and that’s part of what keeps this all going.  In the summer of 2015 I made myself so visible on the ESP8266 front that I was invited to be part of an Espressif-led delegation to MIT in Boston for the FAB11 Conference – demonstrating the ESP8266 with MQTT and Node-Red and how to build this into practical systems. I’ve also recently been involved in a project in Spain for the MOD researching ESP8266 and IOT security and more.  If you wish, follow my ramblings on the subject of control using these chips in THIS blog – and also on Twitter (@scargill), Facebook (esp8266wifi and IOT pages) and LinkedIn.

    Nextion DisplayRegular readers will know that for some time now, the ESP8266 boards have been turning the world of IOT upside-down. I’ve long-since scrapped various radio designs (NRF24L01 etc.), blighted by short range or lack of network ability - and have gone “hell for leather” into using these boards.

    Home ControlThe original plan involved an Arduino Mega as a “master controller” and Arduino “slaves running over an NRF24L01 mesh network. Here in this link is one of my first attempts at blogging on the subject – along with pictures of the original “Appcon” system from back in the ‘90s.

    The original Arduino-based system worked well worked and indeed controlled heating and lighting for a couple of years at my home, but the controller went out of the window when the Raspberry Pi 2 came out which was dirt cheap but with more than enough power to control a house or ten. Now of course we have the RPI3 and other exciting boards such as the tiny FriendlyArm NanoPi Neo.

    Hardwired to Ethernet (or WIFI if you prefer), the system allows for a comprehensive wireless home control setup  and to answer any questions about ESP8266 reliability – with the code you’ll see in here,  I can say with a lot of experience, that the ESP8266 units given reliable power, are rock solid – stunning considering the price.

    ESP-12The ESP8266 boards provide a REALLY low-cost entry solution into the world of IOT making it possible to create in the simplest sense an Internet-controlled LED for under £2 (it could just as easily be a 2KW heater of course for a little more outlay).

    There are a number of variations on the ESP8266 boards, the first shot at this being the ESP-01 – a simple board with power, a couple of IO lines and serial in and out. The thing is – they all use the same chip and that particular board (the ESP-01) is very I/O and FLASH limited – and for no good reason – the ESP-12 and variants for example have access to the A/D converter, several IO lines, have more FLASH memory and cost the same amount!

    The only downside to the latter is 2mm spacing pins but for VERY little you can buy adaptors to 0.1” – or of course use a board that will take the ESP-12 or the ESP-12E (which is the same – but with a few extra essentially useless pins) – same price. There are also a number of ESP-12-based boards today which offer 0.1” spacing. For this reason I’m giving up on supporting the ESP-01 with it’s limited FLASH memory though as you’ll see in another blog entry,  with a steady hand and fine soldering iron it is trivial to replace the old 1MB Flash on the ESP-01 with a cheap 4MB Flash making it more compatible with ESP-12 etc (well, you still have hardly any pins to use).

    Oh, here’s a link to the various source files. Here are original Board files designed by Aidan Ruff. No guarantees – not even any guarantee they won’t change or disappear if we find bugs. Since this article was started I’ve added I2c and a software serial port (optional) on GPIO4 and 5 to control Nextion serial displays directly. Instructions here may not be fully up to date but always check the WORD document in the repository. You might also want to look at the Nextion WIFI touch display board and links – I discuss another control board in there as it can be used for general purpose. This board has neither a relay nor power supply (other than 5v to 3v3 conversion) on-board but it’s a better job for general use.

    Also check elsewhere in the blog to see what we’ve done with the Itead Studio Sonoff boards and their plug-in-the-wall remote mains socket. All of this stuff works together as a whole or you can cherry-pick.

    I’ve added i2c to the ESP8266 software which as you’ll see in the picture at the start of this blog as made a big difference to the number of supported devices. What a roller-coaster this has been an a great way to learn all about i2c – I’ve improved the original Espressif offering a lot!  I’ve recently added version numbering to power up info and a powerful debug command and also, the power-up info indicates which GPIO is currently used for Web setup and which pin for the visual LED or RGB LED indicator.  At the time of writing the code is using the latest Espressif SDK version 2.0

    tmpA4AFThe later PCB you see here was designed (by Aidan Ruff) for my award-winning (ok I had to get that in) Nextion Serial Terminal which  in April 2016 grabbed second prize at the esp8266.com 1st Annual awards. The latest update to this board includes the option to use an RGB LED as the status indicator – something I see as becoming the standard – once you have a selection of colours an intensities to indicate status, all of a sudden a normal LED looks boring.

    The Boards

    For the reasons stated I’ll discuss here the ESP-12 (and variations, ESP-12F being the latest with a marginally better antenna) as that is my favourite sub-module - but the same code applies to other variations where pins allow. We have working software – and circuit boards to go with it. For reasons best known to my pal Aidan we called this original the Hackitt & Bodgitt board. We have a new board version with an RGB LED as the status indicator on GPIO13.

    If you are in any way unhappy using 240v AC mains power, you may want to give this a miss or fit one of those 12v to 5v down-converters so cheap and popular on EBay – I do this to power them from a solar-charged deep-discharge battery in some cases. This version does NOT tie GPIO16 to reset (the saving in power on this board wasn't really worth the bother – and note that GPIO16 is usable as an output). Again in the latest version we’re moving to using GPIO16 as the default relay output – freeing up GPIO0 to be an input only (programming – and web-setup).

    Front view

    Here is the rear view. Note the MOSFETS on the back (this is looking straight through hence letter reversal)

    rearview

    Note that a particularly nice feature of this board is that it can be programmed from a standard, cheap 5v FTDI.

    So in terms of “Features”, the diagram at the start of this blog entry says a lot – but for clarity:

    • Mains-powered WIFI controller board
    • FTDI-compatibility
    • Accommodation for solid state or mechanical relay output
    • Temperature sensing using DS18B20
    • Temperature and pressure sensing using the BMP280
    • Temperature, pressure and humidity sensing via the BME280
    • Temperature and humidity using the DHT-22 or DHT-11
    • All signals brought out to edge connectors
    • Uses inexpensive power supply (or DC/DC converter) available on EBay
    • De-bounced input with automatic debounce and messaging
    • Analog in for checking battery state etc., will handle up to 20v
    • Several Outputs expandable via I2c
    • RGB WS2812b output on most pins able to handle 300 or more serial LED STRIP
    • 3 outputs for high definition PWM – with MOSFETS to control LED STRIP
    • Talks WIFI and MQTT - ideal for being controlled by a Raspberry Pi or similar using, say, Node-Red
    • Flashing status (software supports both simple LED and serial RGB LED)  indication that the unit is functioning and has the correct time which it maintains internally in between updates*
    • Second serial output to support Nextion displays (see the Nextion WIFI Touch Display project)
    • Software mono-stable out for use as a watchdog
    • Outputs can be timers for watering systems for example
    • OTA updating from internal or external website
    • I2C support for an ever-increasing range of devices
    • Hitachi LCD and Seeed OLED support
    • 16-channel 12-bit PWM support via I2c
    • 4-channel 16-bit ADC support via I2c
    • A new NANO-based peripheral via I2c
    • A new SPi-based experimental terminal
    • and more to come…

    * (The time comes from an MQTT broker, for example using a Raspberry Pi or similar running Node-Red – see below)

    ** I now use Node-Red on the Pi to do thermostatic control. Consider the thermostat code to be legacy.

    And another board: we’ve also put together a board for the Nextion displays - but WELL suited as a general purpose board and it is detailed on the WIFI Nextion  blog entry but I’ve included a picture here for reference. I can see a variation on this one being my main board eventually – just needs an add-on with a couple of relays and 3 MOSFETs to make it a perfect development board.  There is a cap on the reset line so that typically an FTDI is able to reset it automatically. Note: The current revision of the board has extra grounds and  4 way sensor connector (DS18B20, DHT22 etc.). Note that an even later version optionally uses an RGB LED for a status indicator – WAY better idea.

    Nextion control board

    The Controller

    ESP-12We have chosen to commit to a communications protocol called MQTT as reliable code for this is freely available, MQTT “brokers” are also freely available and once you spend a little time on the subject it is both easy and the obvious way to control stuff in a network. Lookup “tuanpm mqtt” on Google.

    As MQTT is so central to this, I’ll go into this in a little depth.  There are many ways you could have units talking to each other – a mesh network, a radio network, polling etc…over time I have come to see MQTT as the ideal tool for home control because it is simple, the tools are free and it WORKS.

    Essentially an MQTT broker is a piece of software – a server if you like,  available online or (my preference) on the likes of a Raspberry Pi inside your network) which monitors incoming messages and sends messages out according to who they are for. Addresses are simply text. Messages are simply text. MQTT clients (software that receives MQTT messages for one unit and sends messages out to an MQTT broker) are available freely for the likes of Arduino, ESP8266 and in Node-Red. It really is all very simple once you get playing. So MQTT is an incredibly simple yet powerful format comprising a message “topic” and a message “payload” both in the form of simple text. Units can “subscribe” or listen for specific topics. A great way to play with this is to use an online free MQTT broker – and something like MQTT-SPY which is a free “client” which runs for example on a PC.

    The ESP-12-based controller has a number of inputs and outputs which can be controlled equally well by MQTT commands or serial data - by the current software.

    To talk to the unit wirelessly (WIFI) via the popular MQTT message protocol, a message can be sent to the topic “toesp” to send to all units in the network or to a specific unit by prefixing the topic with the unit ID, or example “kitchen_light/toesp”. The only difference between these two topics is that the former will go to all units yet never elicit a response from any unit, the latter will go to unit “kitchen_light” only and MAY, depending on the context, elicit a response from unit “kitchen_light” – unit ID is simple, programmable text.

    Whatever controls these boards(Raspberry Pi?)  should now send

    topic: toesp

    payload: {heartbeat}

    every minute. All the boards will see this - and all will pre-set a counter in themselves - which if allowed to drop to zero will re-initialise the MQTT code. In the event that boards lose the WIFI for some time, they may reset their WIFI  - then reset the MQTT code. Reset is used only as a last resort and the software can handle up to two SSIDs in case you happen to have two access points. If one fails it will try the other.

    Hence in theory they should be bullet-proof. In practice they are bullet-proof unless of course the power goes off. Last summer in Spain I endured horrendous WIFI and power issues and the current code stood up to this and has done ever since – learned the hard way. That experience is also why I worked hard to ensure the OTA works so I can update the units remotely.

    Each COMMAND or payload starts with an open brace and ends with a closing brace (my choice, kind of like JSON). In the case of MQTT, this information is stored in the message “payload”. For serial, there is no “topic” – just the message (payload) which you can fire into the serial port at 115k baud.

    An example of a simple command might be to turn output 4 ON (output 4 goes to pin GPIO4).

    {out4:1}

    It is possible to send multiple commands at once by separating them with semicolons. i.e.

    {out4:1;out4:0}

    So to use serial – this is all that is required (newline delimiter) – for MQTT – the topic should be as referred to earlier and the payload should be in the format shown just above.

    clip_image002[4]For more on MQTT go here. http://mqtt.org/

    I chose to use a Raspberry Pi 2 (you could use 3 but 2 is ok) as my controller – and chose to use Node-Red as the controlling software when this was a project unknown to most  – turns out I picked a winner as Node-Red is now embedded in “Raspbian” on the Pi and is hence guaranteed a well-deserved and bright IOT future. It is also wonderful and really NOT hard to use. Want to play with Node-Red without even doing an install? https://fred.sensetecnic.com/ – enjoy.

    The following document details the commands the units will respond to – the actions which will occur along with any return serial message or MQTT message created as a result of the incoming command. But please – read the manual.

    The Pins

    Here we see a drawing of the ESP-12 board, please note there is a question mark over the positioning of GPIO-4 and GPIO-5 – on many units, these are reversed.

    You are looking at the FRONT of the unit with the metal cover visible.

    clip_image002[6]clip_image004

    Note: Contrary to the diagram, in prototyping I’m using a 1k to pull GPIO15 down as it’s still a grey area (some say the board won’t program unless GPIO15 is held down) – it does NOT have to be fully grounded and as you’ll see in fact I added PWM output on it (March 26, 2015).

    Preliminaries

    The ESP-12 units operate on 3v3 – NOT 5v. Extensive experience suggests a good 3v3 linear regulator fed from, say a 5v supply provides more reliable results than a 3v3 switched-mode supply – but that will vary with suppliers. I’ve recently been part of a project where we used WELL over 100 ESP-01 units all attached to one 20 amp switched mode power supply – they worked… but I prefer per-unit linear regulation.

    We done just that in all our own boards, using a switched-mode 5v supply (or DC/DC convertor for 12v use) with a 3v3 linear regulator. I suspect most complaints about these boards boil down to power supplies. I can honestly say – that having used countless ESP boards since they came out – I’ve not once come across a bad board!!

    To talk to these units from any 5v system through serial, you should ensure that inputs to the ESP-12 never exceed 3v3 (resistive divider). Typically the serial output will work just fine fed into, say the serial input of a 5v Arduino – I have NEVER had any trouble with this at 115k baud. Simple resistive level conversion can easily be implemented so that a 5v FTDI will talk to the serial without issue. In practice I’ve yet to see a 5v FTDI destroy an ESP8266 but if you go without the divider, be it on your head.

    Certain pins are special and in particular, grounding GPIO-0 on power-up is used to trigger “flash” programming of the chips and also used to put the units into access point mode for setup – so you should be aware of this. When designing outputs I base all of mine on Vcc. So for example when driving a LED…

    clip_image006

    Why do I do that? Well, when the unit powers up – GPIOs are all input – and if you preset them to “1” on power up you’ll not get any flashing lights as the units boot up.

    Below describes SOME of the commands currently implemented.  We have both WEB-based and SERIAL setup, full MQTT integration and the development focuses on using NODE-RED – specifically but not exclusively on Raspberry Pi (I’ve installed Node-Red on many SBCs and consider it as standard as adding Apache and PHP.  We’re using the free and excellent Mosquito for the MQTT “broker” on the Pi (i.e all units connect to this “broker” by address and password” and the broker stores and releases on request stored messages – hence any unit can talk to any other unit as well as making requests of the Pi).

    The manual that comes on BitBucket with the code is way more comprehensive than this blog in terms of commands.

    Command set

    Controlling and monitoring outputs

    The potential outputs are GPIO0 (that’s the one on the board with the relay – future boards will not use GPIO0), GPIO4, GPIO5, GPIO12[1]  GPIO13[2] and GPIO16. These are referred to as OUT0[3], OUT4, OUT5, OUT12 OUT13 and OUT16. You can also use GPIO2 as an output via a command – normally it is used with the likes of temperature sensors.

    Typical outputs respond to the commands {outX:1} , {outX:0} etc where X is the GPIO number. If requesting the status of an output this form is used {out0?}

    There are 3 variations on this:

    {out13} (GPIO13) by default should not be used as its main purpose is that of a LED “status” indicator (though you can override that) (see the relevant command).

    {out0:X} responds to the following for X:

    0 – off (if command “invert=1” then the state of that output is inverted (for positive-based relays) – see the invert command…

    1 – on (see above)

    2 – on from dusk till midnight (see above)

    3 – on from dusk till dawn (see above)

    4 – on from dawn till dusk (see above)

    5 – under local thermostatic control – see relevant command (see above)

    6 – timer for output 0. Another parameter determines ON time in minutes – i.e. {out0,6,100} sets the output on for 100 minutes.

    There are a set of related commands for reading the state of outputs – these are {out1?} {out2?} etc. Should unit “kitchen_light”  be asked for the state of out1 – the MQTT message sent back will be in the form

    topic: kitchen_light/fromesp/out0

    and the payload will contain the appropriate value.

    Outputs can be inverted by the invert command by setting bits 0,1 and 2  etc., in the invert command. See the manual.

    PWM

    We’ve added PWM onto 4,5 and GPIO15 using the latest Espressif library which allows for up to around 95% duty cycle at very high resolution (14 bits) for the cheap 12v serial RGB LED STRIP though we have brought that down to 100 smoothly transitioning levels – but decent ones (i.e. non-linear – with more emphasis at the bottom end for smooth transition). Attached to a logic-level control MOSFET you can drive a 12v LED strip with this – initially off, use the command {pwm:X,Y,Z,I} where X,Y,Z are 0-99 for duty cycle and I is false if you want instant transition, or any value if you want a smooth transition. For example if I =50 (50ms) then 0-99 would take 5 seconds. A lookup table is used to get a reasonably smooth fade (needs work).  Due to the Espressif library, you can get up to around 90% or so duty cycle at value 99 – but you really would not notice the extra on a light. Note also that the PWM even when off i still running – so if you run PWM with a board, don’t expect reliable SERIAL LED operation as the PWM has a minor effect on really critical timing as needed for serial LEDS.

    Heating controls

    In controlling heating, the unit will generally use the internally connectable (on GPIO2) temperature sensor DHT22 (which also handles humidity) or the Dallas DS18b20 or DS18b20p. These heating commands are deprecated as it is a lot easier to do heating controls in Node-Red.

    It is possible right now to set the maximum temperature, the fall-back temperature, frost temperature and 2 on-off times per day locally. Of course using an external MQTT control the possibilities are endless.

    Relevant commands are (X is general a value, degrees C or minute after midnight):

    {peak:X}

    {off-peak:X}

    {frost:X}

    {on_1:X}

    {off_1:X}

    {on-2:X}

    {off-2:X}

    Time settings

    The units are able to keep time but clearly do not have a source of time – this can be provided to each unit separately or more sensibly to all units at once.

    {time:X} standard Unix time value the number of seconds since January 1970

    {dawn:X } number of minutes past midnight considered “dawn” or street lights off time

    {dusk:X} number of minutes past midnight considered “dusk” or light up time

    {When they power up, units attempt to log on. This is an ideal time to send them the time.

    The ADC

    The internal analog to digital convertor reads from 0 to 1.024 volts and outputs a signal accordingly. This can be read by the {adc?} command.

    Debugging

    From the serial console, we can view various settings by using the command {debug}.

    Temperature and Humidity

    The relevant commands here are {temperature?} and {humidity?} They will send out, by return, for example in the case of unit 999 :

    Topic: 999/fromesp/temperature

    Data: X where X is in degrees C

    Humidity follows the same format but is a percentage.

    Expansion

    If we want the unit not to respond to a command but to pass everything on to the serial line to control, say an Arduino, use the EXT command followed by a string  – this can be used part way through a multi-line command if required.

    {out0:1;out2:0;ext:”out0:1”}

    The two outputs will be switched and the command {out0:1} will be send out of the serial port.

    Of course, one COULD send these commands off to an Arduino – OR you could send them off to another ESP-12 with it’s WIFI disabled… the logic of that? It’s cheaper than an Arduino and faster!! You can develop in the C language for the ESP boards using the unofficial Eclipse development environment.

    http://programs74.ru/udkew-en.html

    and you can learn more about my own work at http://tech.scargill.net

    clip_image008

    Setting up for WIFI

    There are commands to set the unit ID (defaults to “999” note this is a string not a number), ssid, ssid2, pass, pass2, mqtt_host, mqtt_port, mqtt_user and mqtt_pass. In each case surround the argument with double-quotes.

    i.e.

    {id:”123”}

    You can also set “desc” for a short description and “attribute” for any short message like “Dallas” to indicate what facilities the board has.

    The above can also be set by WIFI.  Turn the device on and immediately short GPIO0 to ground AFTER power up for several seconds. This will put the unit in to web setup mode. Have your laptop, tablet or phone browser log into Hack-setup and go to 192.168.4.1/wifi/wifi.tpl to set everything up.

    There are two SSID and passwords – the idea being that if the unit fails to get through to an access point it will try the second one, if that fails it will go back to trying the first one etc.

    Reset

    If for any reason we want to reset the device, the {reset} command will do that.

    Inputs

    GPIO14 is an input. We can read the state with the {in14?} command.

    It is also de-bounced and you can read (and simultaneously reset) the count using the in14_count? command. The inbounce:X command will let you set the bounce value in milliseconds.

    When the state of the input changes (de-bounced) an MQTT message will be send out in the form of TOPIC: XXX/fromesp/in14 where XXX is the unit ID and  MESSAGE: 1 or 0

    There is also a bounce value for in2 – i.e. {in2_count?}

    Manual Override

    If we wish to manually over-ride the status of GPIO-0 – for example wall control of a light – set {override:1} to do this – the input can then be attached to an on-off switch and will over-ride the output for a length of time in minutes determined by “override_time”.

    RGB LEDs

    One of the more useful features of the board is to be able to directly control an array of WS2812B LEDs. Right now, overall control is implemented including programmable fade-in and fade-out and a “rainbow” effect. However, buffer space is available to implement full individual control of up to 900 LEDs (warning use separate 5v supply for these – current can be up to 5amps for such a run). Note you cannot currently run PWM and RGB LEDS at the same time. Initiating PWM starts a timer that disrupts RGB timing and a reboot is needed between these. I’ve contacted Espressif about this.

    The command takes the form of:

    {rgb: port,red,green,blue,number,duration}

    So if we want to go to bright red over 10 seconds and you’ve attached an array of 20 serial LEDs to the output GPI12, the command is:

    {rgb:12,255,0,0,20,10}

    And that will happen and we can do other things while it is fading in. The duration is a 32 bit number.

    For amusement purposes we can also try {rainbow: port,number_of_leds,duration[,timer speed in ms]}

    A neat toy: requires 60 RGB LEDs and can run on a port pin… a clock… {clock: port} if is 255, the clock is not running else runs on the relevant port.

    If you want to just light up SOME LEDs in a strip – the command {rgbset: start_led, number, r, g, b} will let you set up various LEDS to various colours and the command {rgbgo: port,number_of_leds} will start the ball rolling. That number should not exceed 300 (not for any reason – I could have made it larger).

    If you want to play with animation… {rgbstart: port,number_of_leds} will clear a command buffer so you can interactively add sequences {rgbadd: start_led,number_of_leds,r,g,b,milliseconds} and when you want it all to end, {rgbstop}. These commands can be sent via serial of MQTT.

    Other

    When the board powers up it sends a message “esplogin” and in the body the the name if its ID with a slash after it.. so “999/”. This can be read to send the board, say, the time. See the next section.

    I’ve implemented {sleep:x} where x=microseconds (32 bit number) hence 10000000 is 10 seconds. You need GPIO16 attached to reset and that just isn’t worth the saving on this particular board but it’s in there for completeness. This is deprecated.

    Node-Red

    BlynkOriginally we’d planned on using something like OpenHab for home control and of course anyone armed with the software can do just that. But for our purposes we like Node-Red as it is extremely flexible and becoming more powerful by the day. If you have a Raspberry Pi2 or better – you can run Node-Red on that but it also runs on Linux, PCs etc.

    Node-Red allows one to “program” using visual boxes via a web interface– so an MQTT input box can respond to a particular topic, you can then do something with that and send a message back – or elsewhere – or maybe send a TWEET, or store some info in a database or send an email – honestly it is VERY simple to use and very powerful once you get to grips with it.

    If you look at my earlier blogs I’ve written about graphing locally, using data pulled in by Node-Red and stored in a MySQL database with ease. There are alternatives – I’ve written a “node” for Node-Red to let you simply send data to Grovestreams.com and others have written nodes for the likes of EmonCMS which has some very pretty dials.

    Here’s a screenshot of my Emoncms data and yes, the battery is really flat – due to this being solar charged out in Spain – and it is 6am in the morning – clearly yesterday wasn’t that sunny.

    EmonCMS used at Bedrock in Spain

    Node-Red has a timing module… when lets you trigger things at certain times. It also has available a suncalc which sets a flag when it is sunset. I’ve improved this significantly in a node called BIGTIMER (see http://flows.nodered.org/ -  check out my bigtimer, esplogin and grove nodes– my contributions – for example – “npm install node-red-contrib-esplogin”) -  to produce an output directly suitable for the current design – to log the boards in and to tell them about the time, sunset and sunrise.

    The esplogin node can send time updates when Node-Red powers up and on a regular basis (every 12 hours)to all boards – but also send to a specific board on request, and all boards will happily send that request when they power up – so typically a few seconds after power-up the boards know what time it is and what dusk and dawn times are.

    Here’s what it looks like in action.. MQTT coming in from boards in the form of a json payload (purple), esplogin node in yellow doing the time/dusk/dawn updating and outputting the relevant MQTT back to the board (or in the case of time top-ups – all boards). The outputs from esplogin are, in order..

    1. MQTT output

    2. Output for a MYSQL database (legacy)

    3. Output to a straight text logging file

    4. Output to a SQLITE database

     

    tmp3A16

    All boards subscribe to “toesp” – and to a version of that with their name (ID) prefixed – so for example fred/toesp. That way I can talk to any board – or all of them.

    When the board first powers up – it sends…  in the case of board “fred” which has description “my board”

    Topic: esplogin

    Payload: {“id”:”fred”,”desc”:”my board”,”attribute”:””}

    Desc and attribute at not that important but can be shoved away in a database.  So that sets off the esplogin node  putting info to it’s outputs – I only use the SQLITE output and that ends up in a database – that’s up to you but my setup is in the Raspberry Pi script referred to elsewhere.

    So assuming the node gets that login – it will send a bunch of time info back to the ESP board in question – and will then do that every day to all boards. This stops the boards from watchdog resetting and ensures they always have the correct time without needing a real time clock etc.

    See the time, dusk and dawn commands – the ESPs can be updated though right now this isn’t necessary as the home control software on the Raspberry Pi is doing all the work.

    How to get all the software the easy way

    In order to use these boards effectively – you’ll need a Linux setup (a Raspberry Pi 2 or 3 for example) with MQTT, NODE-RED and preferably other stuff like PHP, SQLITE etc. Personally I’m not a Linux lover!!! And so I like things to be easy – so we made a SCRIPT. You can grab a Raspberry Pi(2) and put everything on there. The script works with JESSIE version of Raspbian. I made this script for “Jessie”. Follow the instructions precisely or it will not work. You must be user PI and NOT ROOT. Read more about the script here...  and

    I use the Pi2 – after testing many boards this seemed a while ago to be the best for the job IMHO and for reliability use a good SD such as Samsung EVO and to make life really easy. If you have a Pi3 all the better.

    For controlling and monitoring – I am currently using the Nextion (serial) touch display boards for the wall and vary between using Blynk Android/IOS app for remote control, the node-red-dsahboard and ImperiHome.  All work well depending on your requirements.

    You’ll read about all of this stuff elsewhere in the blog – use the search facility – there is a LOT of info.

    Notes:

    [1] GPIO5 (out5) can be manually over-ridden by the input and the “override” command.

    [2] GPIO13 (out13) is generally used as an indicator to show that all is well. It activates on receipt of the correct time either serially or via MQTT and will flash briefly once every couple of seconds from there on. This activity can be disabled by a command. Increasingly we’re using a serial RGB LED on this pin to provide more comprehensive status info.

    [3] GPIO0 (out0) is special in that it responds to more than “1” and “0” and can be used as part of a heating control system. Ultimately we will scrap using GPIO0 as an output and merely use it for programming and setup. GPIO16 makes a good relay output as it is not much use for other types of output.

    Haptic feedback – when the board is being used in “Nextion” mode – we use GPIO12 as a beeper – simply fasten one of those cheap round haptic feedback devices to it and glue that to the back of the board. Also note that with limits (the line buffer is only 80 characters in length) you can concatenate commands so that for example if you want to turn GPIO12 on and get feedback as to the state at the same time {out12:1;out12?}  - note the two payloads are joined together by the semicolon.

    But the REAL source of info on commands is in the WORD manual with the project – that is where you go to keep fully up to date.

     

    Enjoy!

    Facebooktwittergoogle_pluspinterestlinkedin

    Itead Slampher and Sonoff

    Slampher - in Scargill's review of Slampher and SonoffRemind you of IKEA? No? Heard of ESP8266? Yes? Well, Slampher and Sonoff are two products from Itead and I think you’ll like at least one of them. They arrived at my desk today. Slampher is a screwfit lamp fitting – take the E27 lamp out – put Slampher in – screw the lamp back in and you’re in business with a remote controlled lamp. It really is that easy.

    This blog updated May 10, 2016.

    There’s a tiny button on the unit that you press to turn the lamp on or off manually if you ever need to – or hold it down and you can make it listen for a controller – the handy – and i have to say very nice quality – keychain controller which can control 4 devices.

    Handset - in Scargill's review of Slampher and SonoffThe REALLY good thing is that both of these products use ESP8266 so if you want to go in and re-program them – as I have  – to handle MQTTon your WIFI!!!! You CAN do but there is an APP if you don’t want to do it the hard way. I don’t have the App at this time so I can’t comment… but it’s an ESP8266 – how hard can it be to get that right.

    Slampher and SonoffThe other product is called Sonoff and it’s just a little box that you wire mains into – and out the other end you fasten it to whatever you want – and control it the same way. SO the units obviously have both WIFI (ESP) and a 433Mhz receiver – because the little handset is 433Mhz.  Sonoff would not look out of place on a skirting board at all (proper little screw connectors inside – no soldering needed).

    These are NOT expensive – in fact that really IS the point – Sonoff in particular – I could not build one at that price – and I guess that is part of the charm.  I can make this stuff with my eyes shut but by the time I put it all together and in a box….

    The units I have are solid looking – the little mains controller Sonoff – the PCB inside is well put together with proper isolation on the PCB (some of you will have seen some HORRIBLY DANGEROUS pcbs in the past when it comes to UK mains but little Sonoff is ok at least at first glance).  I’ve not had a chance to pull the lamp unit apart yet – that will come later.

    Pricing: I found this – $4.85 – if you live in the UK the postage brings it to $9.37 but even then! I checked out the cost of 3 – $14.55 all in to UK that is $5 each – that’s around £3 or so.

    https://www.itead.cc/sonoff-wifi-wireless-switch.html

    DIY programming: You’ll like this if you know how to program ESP8266 chips. I now have a diagram for Sonoff – initially the connectors did not seem to match those on the diagram – so I contacted Itead – they were keen to point out that they have a cloud system and App and of course the remote control via radio – all of which I’ll lose when I reprogram the board – but hey…. I want to control these with MY kit.

    Look at the diagram below… this is looking at the TOP of the board. There is a 4 way or 5 way connector. If the latter you might want to get the meter out – but on the 4-way (mine)….  the SQUARE pin is not as you might think ground – it is 3v3 – then we have serial, serial and ground. If you wire your FTDI as per the wiring diagram below it should work.  The relay is controlled by GPIO12.

    I was happy to see cooperation from Itead – obviously they would prefer that you use their cloud software and most will – but we are hackers and would never be satisfied without control, right? Ignore the LED, it is fastened to the RF device so you don’t have control over that. Right now I don’t have enough info to make any use of the RF device. If someone knows better please write in. Would be nice to have both.

    Warning – you are on your own if you start this – you will need your own software and messing with the board will kill any warranty.  Clearly this must NOT be plugged into the mains while programming – use FTDI at 3v3 settings. No doubt 5v would cause damage.

     

    Sonoff board

     

    Image635909899921896626

    CLARIFICATION – the square pin is NOT ground – the square hole is 3v3. You should see that the round hole is connected to the flood filling (if you have good eyesight).

    Use FTDI at 3v3 setting – in the original boards,  press the large button –THEN power up FTDI. DO NOT connect mains power while doing this – you will fry. Test MQTT connectivity, disconnect, put back in the box, use. NOTE – the newer boards marked RF do not seem to attach GPIO0 to the button – see later blog – it may be necessary to make a VERY find short at power=up – of two surface mount components.

    As for actual use.. here’s the interface in Node-Red

    tmp427F

    Complicated, isn’t it. The node on the left is one of Andrei’s UI components – the node on the right is the standard MQTT connection – all of course running on my Raspberry Pi2 and already in charge of the heating and a load of experiments.

    Here’s what it looks like on the phone…

    tmpEE3C

    and here it is - waiting for the screws which for the LIFE of me I can’t find… works a TREAT. The output relay is on GPIO12.

    tmp66FB

    I have modified my general purpose MQTT software to run with the Sonoff devices and it works a treat.  https://bitbucket.org/scargill/esp-mqtt-dev   - see the home control 2016 project in here where you’ll also find binary files.  A couple of settings need changing from normal – I normally use GPIO2 for WIFI setup and the SONOFF WIFI units use GPIO0…  also  - GPIO13 which I use as a status indicator – turns out the Sonoffs use them but reference 3v3 so I have an inversion setting so they flash off most of the time instead of on all the time – it’s all in the DOC file in the project above. With both of those settings updated... you can also use the button as an on-off button if you like (toggle).

    The SONOFFs have only 1MByte of FLASH which would kill off OTA for all but the simplest of projects – my code needs the larger size IF you want OTA. With this in mind, is it worth paying extra for the Electrodragon? Some say the quality of the latter is not so good? But they do use an ESP-12 which will have the normal 4Mbytes of FLASH… what do you think. Update Jly 2016:  The Electrodragons didn't turn out so well - but now I've just proven that it is in face easy with a fine soldering iron to fit a £0.37 32Mbit FLASH chip to these, replacing the original - which means full OTA and no problems expanding.  See http://tech.scargill.net/32mb-esp01/ for more info.

     

    Facebooktwittergoogle_pluspinterestlinkedin

    More Websockets

    In a previous blog item here I covered the use of MQTT, jQuery Mobile and Websockets to make a controller web page for mobile phones/tablets etc. here http://tech.scargill.net/mqttjquerywebsockets-controller/

    I showed just a basic and not too pretty page with buttons that would control real items, in my case via Websockets on a Raspberry Pi, firing off MQTT (also on the Pi) to ESP8266 modules (using our own code based around TuanPMs MQTT project).

    Well, I’ve been doing a bit more playing around – here’s a better way to make the buttons – but all based on the previous article so you’ll need to read that first.  I’ve also added a second page – merely to show you how that is done easily – and you could expand that ad-infinitum – but I’ve ALSO shown how to get information back in from the same web socket as I was asked about this aspect. It’s very easy (there was a time when I was terrified of the subject).

    So first of all this time my Node-Red code is slightly different – I’ve separated off the incoming websocket node from the outgoing one – as long as they refer to the same item they don’t have to be joined together.

    Websockets

    So in this version – the same code in “process commands” – as instructions from the web page go out to MQTT to control various gadgets. See other blog item.

    Here, a Timestamp (standard Node-Red item) pushes out a timestamp every second – and another function converts that into a simple string as payload – which is then sent back, every second, to the web page. Normally you might send JSON data back for further parsing in the web page but in this case it’s a simple string.

    Here’s the code inside timeConvert.

    var dt = new Date(msg.payload);
    var msg = {    'payload':  dt.getHours()+":"+dt.getMinutes()+":"+dt.getSeconds() }
    return msg;

    Page 1And so to the web page itself: I’ve improved it some way as you’ll see as I figured out how to get 4 buttons wide (trivial – I just didn’t spot it in jQuery Mobile the last time around).

    As you can see it is quite pretty, has a page button near the top to switch to another page (which is empty as yet so I won’t show it here but it will be obvious in the code how it works and how you can keep adding more.  I’m not happy about the look of the paging button but it’ll do for now.

    Then way down at the bottom left – you’ll see a timestamp – that is coming from Node-Red and is in this case updating every second. It COULD be status information – or anything – but for the purpose of demonstration (and it’s also nice to know that something is actually working by visual feedback like this) it is just a string as described above.

    I wish I could figure out a simple additional class to colour these buttons individually instead of making themes for each button type – perhaps someone else could contribute here.

    So now this is a quite reasonable control page with the option for more pages (not going to get any more buttons on this page).   I did think of having Node-Red return a string with the states of the various outputs and changing the buttons accordingly – but that would not have made for easy reading.  Anyone implementing this will soon figure out how to add such functionality.  Instead of returning simple text, it would make sense to similarly send back jSon as there is a very easy to use function to split this up.

    Recall too from the previous blog that I’ve implemented (in the .htaccess file) simple password protection not shown here and not of much use locally but pretty essential if you want to control from outside.

    Here is the web page code for the above, note I’ve changed the name of the websocket to /myapp3 – could have called it anything, really.  See the “onmessage” function – you don’t get any easier than this. There you go… https://bitbucket.org/snippets/scargill/AGMq5

    newmenuUpdate: 23/08/2015 – there is a better way to do navigation in jQuery mobile (I’m still learning) but it seems incomplete. A slide-in-from-the-left menu system is easy to create – but if you try to make one menu system for all pages – there is no formatting – I guess this will be available in a future version – however, replicating a menu system for each page has it’s advantages as you can disable the current page link! 

    Here is a version with the awful button missing hence saving some space on the page. So this is a little like the swipe side menu in NETIO except I’ve implemented it by touching the top title bar because swiping doesn’t work too well on a PC and you might be testing this in a normal browser!!!

    The title bar is sitting doing nothing anyway, so why not.

    Actually I like this – on a typical phone you could easily get half a dozen pages and some explanatory text or imagery in that side menu… hell, you could even get some more buttons in there and a logo!

    Here you go..

    https://bitbucket.org/snippets/scargill/yREGd

    Facebooktwittergoogle_pluspinterestlinkedin

    MQTT/JQUERY/WEBSOCKETS Controller

    I wrote a while ago about using web sockets as against something like NETIO for controlling the home – most folk liked the article but I think part of it was a little complicated – and at the time I’d not really thought it out to make it as simple as possible.

    So, this morning I started again and now I have a decent and easy way to make control interfaces from the mobile phone – to ESP boards. I won’t go into MQTT in detail here – I’ll assume you have an MQTT interface of some description on your WIFI boards – if you need more info on that, look elsewhere on the blog.

    jquery mobile themesI’ll also assume you have node-red running somewhere. In my case I have a Raspberry Pi2 sitting in a corner with battery backup, running node-red, MQTT and Apache web server.

    So – in order to make a half-decent looking phone interface, I decided to go for jQuery Mobile – it is cheap, easy to run and there is a theme generator.

    I went along to https://themeroller.jquerymobile.com/ and made myself a set of THEMES – that is some CSS. You can’t get any easier – you just drag the colours onto the various items – save the ZIP file – and then refer to the THEMES folder in your web page.

    Here’s what I did – you might want less colours, more colours, different colours etc.… you  can change all aspects of the jQuery Mobile interface which is basically a bunch of items that work well on mobile phones and tablets. Click on these images for larger versions.

    So – what’s needed then is a web page…. that shows a series of buttons (jQuery Mobile), a means to send off button presses to Node-Red (web sockets) and some means of then controlling things (MQTT).

    In my case I put the web page and dependencies in a folder within /var/www on my Raspberry Pi (could be anywhere – a Windows PC, whatever).  I can access these pages externally and hence there is a password on the folder – that is done in my case with an.htaccess file but you might have other ideas.

    The page should begin with a header for accessing jQuery, jQuery Mobile and the themes etc.…  I figured as the phone has to have an Internet connection to work anyway, there was no point in filling up the Pi with libraries other than my custom theme..

        <head>
            <meta name="viewport" content="width=device-width, initial-scale=1">
            <title>Simple Controller</title>
            <link rel="stylesheet" href="themes/jquery.mobile.icons.min.css" />
            <link rel="stylesheet" href="http://code.jquery.com/mobile/1.4.5/jquery.mobile-1.4.5.min.css" />
            <script src="http://code.jquery.com/jquery-1.11.1.min.js"></script>
            <script src="http://code.jquery.com/mobile/1.4.5/jquery.mobile-1.4.5.min.js"></script>
            <link rel="stylesheet" href="themes/petes.min.css" />
           
        </head>
       

    phoneAnd so above, in the HEAD section of the web page (which I simply edit with NodePad++) links to the local theme and remote jquery. That local stuff all comes in the zip file from the themeroller site – well it does if you make the title when saving “petes” – you might want to call it something else.

    Here’s my test page which works equally well on a mobile phone as in a web browser  - I’ve checked this out in Chrome and Edge and on my Android 5 phone – identical results.

    As you can see I’ve gone for gross, using matching colours (more or less) for the colour controls I’m actually firing off.

    The only difference between the buttons is a simple addition of data-theme="g" or whatever letter – if you look at the theme image at the top, as you add more themes, they start off as “a” then “b” etc. We’ll see this code later.

    Rather than put a ton of code in here – here’s a snippet with the entire page.

    https://bitbucket.org/snippets/scargill/jgBnK

    You’ll see the header information followed by some setup for web sockets.

    The basis of my controller as you’ll know if you’ve read Home Control 2015 in this blog, is to send MQTT data to my ESP8266 boards, so for example to control the RGB output on GPIO0 of a board called RED1,  I’ll send the following…. the topic would be red1/toesp and the payload would be for example {rgb:0,255,0,0,70,1}

    The above means – send a message to board red1 and tell it to fire out a WS2812b signal on GPIO0 to turn 70 LEDs on fully bright red.

    I wanted the web page to be as simple a possible – adding to a button something like….

    onClick='send("red1","rgb:0,255,0,0,70,1")'

    And that is exactly what I’ve done. But firstly the mechanics of the Node-Red web sockets interface.

    In Node-Red, I have this:

    node-red

    Simpler than it looks, I’ve dropped in a WS input module (standard node-red node unit) and out – and connected them together.

    the input listens to “/myapp2”  - could be /fred  or anything you like to call it.  Yes folks that is ALL there is to the setup – but…. assuming my web page connects to this – I need to do something with incoming data (in this blog I’m ignoring firing data back to the page but that is also easy – I referred to that in the last blog – the point of web sockets of course is that this is a continuous connection between the web page – and node-red web socket control.

    So what I need is an incoming MQTT message and topic – and that needs firing out to MQTT to control things. The only bit of code here is in “process commands” – this is a function module (drag and drop) and this is what I put in it.

    var jsondata=JSON.parse(msg.payload);

    if (jsondata.topic)
    {
        msg.topic=jsondata.topic+"/toesp";
        msg.payload=jsondata.payload;
        return msg;
    }

    Simple enough – any incoming information is assumed to be in json format and is “parsed” (built in function) so that I can extract a payload and a topic to simply fire off to MQTT and on to my boards.

    As I’m always going to need “/toesp” on the end of the topic – I add it in here rather than needlessly replicate it in the web page.

    The web page itself needs to fire out something like (in this case)….

    {“topic”:”red1”,”payload”:”{rgb:0,255,0,0,70,1}”}

    The inner braces are for my boards…

    So – my board ends up with:

    topic:  red1/toesp

    payload: {rgb:0,255,0,0,70,1}

    I wanted to keep the actual buttons as simple as possible so there is a function in the web page to glue the topic and payload together.

    function send(topic,payload)
    {
    ws.send('{"topic":"'+topic+'","payload":"{'+payload+'}"}');
    }   

    and so all we need now is the connection between the web page and node-red web socket interface (simple as they’re both on the same server) hence if you look at the web page you’ll see a definition for wsUriC which is then used in wsConnect() which is called on page load. This makes the connection.

    ws.send() sends a string to Node-Red and my function ensures the string is in the right format.

    A typical button looks like this..

                            <div class="ui-block-a" ><input data-theme="a" type="button" value="red" onClick='send("red1","rgb:0,255,0,0,70,1")'></div>

    Note the very simple onClick function call to send a command off to a particular board…. and if you study the web page example, putting aside the formatting commands for jQuery Mobile (which you can learn all about here https://jquerymobile.com/)  - there is WAY more than just buttons available to you - but I’ll leave that to the reader to explore (or not).

    So I press a button on the phone – and the light comes on or changes colour or whatever. There is no feedback – that would be covered by a call-back function in the page and jQuery to alter the shape or colour of elements accordingly – really quite easy once you get started. I thought of putting that in here but I’ve probably given you enough to think about as it is.

    Hope you find this useful. When the connection is working incidentally you get a little TICK at the top of the page – or a little cross if not.

    tick

    Pete.

    p.s.  If you’re wondering what all this cave reference is about – head over to http://www.bedrock.es

    Coming up soon: I hope to review the inexpensive NEXION touch display (one is on it’s way) to make a nice wall-mounted thermostatic control – which will talk via serial to an ESP-01 which in turn will control a relay and have a DHT-22 chip for temperature sensing – and Node-Red for background timing control – and of course I will have phone-based monitoring and over-ride based on code very similar to what you see here.

    Facebooktwittergoogle_pluspinterestlinkedin

    Lua Revisited

    Back in the dark ages when the ESP8266 was something new, I had a go at the nodeMCU software, essentially implementing the Lua language (or a subset of it) on the ESP8266.  At the time I was not familiar with Lua but thanks to some of the excellent resources out there on the web, I quickly caught up.

    Sadly, my expectations exceeded the ability of the implementation to match and every time I went past flashing a few lights – CRASH. I gave up in disgust.  I’m not complaining as that gave me the impetus to learn the hard way how best to program these chips – which I maintain is using the C language – and in my case via the unofficial development kit which is just an excellent piece of work by Mikhail Grivorev, leading to a complete Windows environment with a ton of examples. Between that and TuanPMs excellent MQTT code as a starting point  I’ve been happily programming away and even built up a working relationship with Espressif in the process. My home control project is well under way and now just waiting for some new boards – and a function that Espressif have promised to shut down PWM.

    LuaMeanwhile this morning, I can’t remember what I was reading that triggered me off, I decided to have a quick look at the node-MCU code again.  I had it sitting in the development environment but figured it would be out of date and so went off to the repository to study any changes – which I duly put into the code. The GIT Makefile is useless for Windows but fortunately there was one in the environment – and as it happens, a good one. I wasn’t about to go backward so I made sure I was linking to the very latest SDK which at this time is 1.1.2 with at least one patch – all available from the Espressif forum.

    I checked the settings in the Makefile, adjusted the SPI speed up to 80Mhz, set the FLASH size to 4Mb to suit the sparkly new ESP-210 that is sitting here looking for a job – and started the compile process. Within a minute and without any issues, the code was flashed. I have to say I was expecting at least one or two issues – but no.

    I set up a 9600 baud terminal, in the process discovering for the first time that you can have multiple terminals at once in the Eclipse environment – handy (it would be more handy if it always reported the right location of bugs but you can’t have everything). Sure enough – a working Lua board.

    I told it about my WIFI setup and sure enough it grabbed that too.  The project has come a long way since the early days and now (apparently) supports json, a file system, timers, pwm, i2c, spi, 1-wire, net, mqtt, coap, gpiio, wifi, adc, uart and more. I’ll believe all of that when I see it.

    I was delighted to see that both the integer and float versions are available – that’s a plus! I also noted support for the WS2812b chips – I noted that as I had to make a change to the source code and I’m delighted to note that they’ve made use of a local compiler directive that I was not even aware of to ensure accurate timing. Wish I’d seen that a month ago when I started work on my own WS2812b support. 

    WS2812B LEDs on nodeMCU LuaI plugged a (short) length of WS2812b strip (a custom board we had made) into the ESP-210, fired the instruction and… worked first time out of the box. Impressive.

    I’m pleased to see the nodeMCU docs have come on a long way – sadly the website was running like a DOG at the time but here’s a few stats I managed to get off the board.

    print(node.heap()) returned 20kbytes which is more than when I tried it – but then it used to die when the heap got down to 5K so we’ll have to wait and see how reliable that is.

    3 Channels of PWM are available – at 10 bit resolution. Clearly some work to be done here as Espressif have just released a library handling more outputs than that at nearer to 14 bit resolution – and it works as I’ve been using it in my own research and projects. However – handy enough.

    There are libraries for sending to displays complete with fonts – now that I’m interested in – if this will do reliably MQTT and report results to a display I have a use for this already – wall thermostats.

    So, overall, a very brief test – it is possible with the right environment to get interactive and easy to use nodeMCU up and running in minutes – you don’t need special boards and the software is free. Can’t be bad – how complex a program you can make withou running out of RAM remains to be seen. Here is some more info on nodeMCU.

    Facebooktwittergoogle_pluspinterestlinkedin