Monthly Archives: March 2015

ESP826 + Pi + MQTT + Node-Red Heaven

NetioIf you’ve been following the blog you’ll note I’ve spent a lot of time getting to grips with the Raspberry Pi, Node-Red, the ESP chips using C and of course my earlier home control attempts with NETIO, the NRF24L01 boards and Arduino clones (all of which are working 24/7 – I’m just getting more ambitious).

Well, it’s all starting to come together now. This morning I just put together all the pieces I’ve discussed earlier. On my workbench I have an ESP-01 based board controlling a mains light, I have the Pi with a couple of LEDs on it and I have an ESP-12 based board with some RGB LEDs and a MOSFET driving a PWM channel.

Now, I have to tell you that I’ve discovered something I’d rather not have… the PWM output is , it would seem, driven by software, NOT by hardware. I say that as I’ve realised that if I turn on the PWM output and then start fading RGB LEDs – there is some temporary interference to the PWM – and it is visible. It goes away when the RGB LEDs stop changing however.  This does suggest that it would be better not to run both at once – or indeed palm off the PWM to a serially-driven Arduino-Micro or similar from the serial output of the ESP..  I turn off the interrupts when handling RGB and this no doubt is the cause. It’s not a big deal, it’s just slightly annoying.

However I’ve now put NETIO in charge of this lot and I have to say it’s working a treat. Here is my current test screen on the Samsung S4. I was going to use the NETIO text input box but the author doesn’t seem to have finished this off as it looks awful (it works, it just looks awful) so I’ve missed that off for now.

You can see the S4 screen over on the right. The RGB wheel works a treat, the slider controls the PWM, the LEFT on-off buttons control the LEDs on the Pi, and one of the rightmost pairs controls the mains light on another board.  Node-Red acts as the arbitrator for this lot. TCP in from the NETIO app on the phone is sent through a function block in Node-Red in which I parse the incoming data and make sure I sent back the appropriate response. It’s trivial actually once you get familiar with Node-Red.  For now I’m storing the Pi Pin states in a database but ultimately I’ll store the lot. By and large I’m just sending a message back to NetIO and then sending off an MQTT message to the relevant device as dictated to by the NETIO message.

Node Red


var newMsg = { payload: msg.payload.trim() };
var myMsg=newMsg.payload.split("|");

if (myMsg[0]=="gpio") {    newMsg.payload=myMsg[2];
        switch (myMsg[1]) {
            case "0":

newMsg.topic="Update pins set gpio0=" + myMsg[2];
                return [newMsg,null,null,null,msg,null];
            case "1":
                newMsg.topic="Update pins set gpio1=" + myMsg[2];
                return [null,newMsg,null,null,msg,null];
            case "2":
                newMsg.topic="Update pins set gpio2=" + myMsg[2];
                return [null,null,newMsg,null,msg,null];
            case "3":
                newMsg.topic="Update pins set gpio3=" + myMsg[2];
              return [null,null,null,newMsg,msg,null];
                return [null,null,null,null,msg,null];
else if (myMsg[0]=="gpio?")
    switch (myMsg[1]) {   
        case "0":
              return [null,null,null,null,msg,null];
        case "1":
              return [null,null,null,null,msg,null];
        case "2":
              return [null,null,null,null,msg,null];
        case "3":
              return [null,null,null,null,msg,null];
              return [null,null,null,null,msg,null];

else if (myMsg[0]=="mqtt") // message and topic passed
     return [null,null,null,null,msg,msg];

return [null,null,null,null,msg,null];


ESP01 versus ESP12

And for today’s intriguing question…  I’ve been working on the ESP-12 for some time now – the program has never failed to work on power up… today I wanted to put the code back into an ESP-01 as I only wanted to use GPIO0… same code – same kit – ESP-01 powers up – gives up the usual rubbish on powerup and… nothing.

Works in the ESP-12 – doesn’t work in ESP-01.  Tell me I’m dreaming but are they not identical apart from the actual pinout?

I feel like I’ve entered a parallel universe… used to work. I’ve tried spare boards – makes no difference.


ESP8266 Start up Crap

I’m pretty pleased with my ESP8266 developments so far. Ditching Lua was a good move as I’m no longer tied for resources… there is however one major annoyance that I’ve not seen a cure for…

When you power up the board, in the ESPRESSIF rom there is some setup info – along the lines of…


See that garbage at the top – that is something like 76Kbaud debug material at the start, then everything up to “connected with loft-east”. If anyone has discovered a way to turn that off – please do let me know. I’m assuming it is in the binary files and beyond our reach.


ESP8266 Tidy PWM

pwmIf you check out an earlier post I managed to get PWM working nicely on the ESP-12 on the otherwise unused GPIO15. Well, it was a little messy – so I’ve tidied it up with the use of a struct.

Here it is..  You’ll need to make minor mode to the PWM .C page as per my earlier blog.

In your variable setup…

typedef struct {
    uint8_t channel;
    uint16_t frequency;
    uint8_t actual;
    uint8_t bright;
    uint32_t timeout;
    uint8_t minimum;
} PWM;
PWM pwm;

And a 100 step gamma correction table…

static const uint8_t PWMTable[100] = {0, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3,

4, 4, 4, 5, 5, 5, 6, 6, 6, 7, 7, 8, 8, 9, 9,10, 11, 11, 12, 13, 13, 14, 15, 16,

17, 18, 19, 20, 21, 22, 23, 24, 25, 27, 28, 30, 31, 33, 34, 36, 38, 40, 42, 44,

46,48, 51, 53, 56, 59, 62, 64, 68, 71, 74, 78, 81, 85, 89, 93, 97, 102, 107, 111,

116, 122, 127, 133, 139, 145, 151, 157, 164, 171, 178, 186, 194, 202, 210, 218,

226, 234, 244, 250 ,255 };

In your INIT code = 0;
pwm_init(pwm.frequency, &;
pwm_set_duty(, 0);

In your timer callback (mine is every 20ms)

if (pwm.timeout) {
    if (--pwm.timeout == 0) pwm.bright = pwm.minimum;
if (pwm.actual != pwm.bright) {
    if (pwm.bright > 99) pwm.bright = 99;
    if (pwm.actual > pwm.bright) pwm.actual--;
    else if (pwm.actual < pwm.bright) pwm.actual++; = PWMTable[pwm.actual];
    pwm_set_duty(, 0);

And that’s it – just adjust pwm.bright from 0-99 and if you set pwm.timeout the light will fade out after a while –and if you set pwm.minimum when it times out it will time out to that value.

Handy for general LED strip lighting, SAD lighting etc.


ESP8266 PWM on One Output

According to the ESP8266 API - you have control over ports 12,13 and 15 for PWM.

I'm interested only in using GPIO15 as I have uses for the other port bits - so I took PWM.H and removed reference to the other two ports - leaving only GPIO15. I changed PWM_CHANNEL to 1 and deleted references to the other two pins.

#ifndef __PWM_H__
#define __PWM_H__
#define PWM_CHANNEL 1
struct pwm_single_param {
   uint16 gpio_set;
   uint16 gpio_clear;
   uint16 h_time;
struct pwm_param {
   uint16 period;
   uint16 freq;
   uint8 duty[PWM_CHANNEL];
#define PWM_DEPTH 255
#define PWM_1S 1000000
#define PWM_0_OUT_IO_NUM 15
void pwm_init(uint16 freq, uint8 *duty);
void pwm_start(void);
void pwm_set_duty(uint8 duty, uint8 channel);
uint8 pwm_get_duty(uint8 channel);
void pwm_set_freq(uint16 freq);
uint16 pwm_get_freq(void);

I then performed an init... and LO - 0% duty cycle out of GPIO15

Here's my code..

uint8_t PWM_CH[]= {0};
uint16_t freq = 1000;
pwm_init(freq, PWM_CH);
pwm_set_duty(pwm_CH[0], 0);

and to change the duty cycle (actual values 0-255)

pwm_set_duty(pwm_CH[0], 0);

Voila. The otherwise wasted GPIO15 now usable.


ESP8266 SDK v1.0.0 Out Now

esp8266 v1.0.0The new esp8266 SDK v1.0.0 is out now including various optimisations and a half-decent English manual. How far we have come since only a few short months ago when the code was full of bugs and the English documentation almost non-existent. Today you can get paid to report bugs!!

According to Espressif…  they have fixed various bugs including “interrupt during flash erasing causing wdt reset”, read and write issues with RTC memory, reconnects.. they have added API code to update the cpu frequency, set a mac address filter during “sniffer” and set which interface the UDP broadcast is sent from.

ESP8266Various optimisations included smartconfig, AT updates, PWM and much more.

So does it work? Is if going to give you headaches? I have just downloaded the lot and set my printer off on the 96 page SDK document.. I renamed the original SDK directory in the ESPRESSIF folder structure and added in the new one – I compiled the MQTT code I’ve been working with – and voila – failed – then I realised I’ve added a macro in there (digitalWrite) – which if nothing else proves it was trying to compile the new version) – I fixed the missing macro and the whole thing works – I’ve not had time to extensively test my code but the compilation process works flawlessly and a quick test of my code suggests all is well.

For those still using the AT code, AT+CIPSTATUS now returns the right code!!! the wdt no longer resets on connect failure and much more – it’s all in their blog. Here’s the Espressif BBS link but you’ll need a free account..  You can even now many money for reporting new bugs – perhaps Microsoft should consider this.

Update August 2016: We are now up to SDK v2.0.0 and the SDK goes from strength to strength – well worth downloading the documents from Espressif.


WS2812B LEDS flicker

If you recall my earlier post on the code for handling WS2812B LEDs with an ESP-12 or similar – I’ve been discovering the odd flicker on the first LED – recall I lengthened the reset pulse to get rid of this – well, I was still getting it very occasionally during a long fade of the lights. I’ve lengthened the reset pulse and it has now completely gone (see code in bold). Enjoy.


void ICACHE_FLASH_ATTR WS2812OutBuffer(uint8_t * buffer, uint16_t length, uint16_t repetition) {
    uint16_t i;

   for (i = 0; i < 4; i++)
    while (repetition--) {
        for (i = 0; i < length; i++) {
            uint8_t mask = 0x80;
            uint8_t byte = buffer[i];
            while (mask) {
                (byte & mask) ? SEND_WS_12_1() : SEND_WS_12_0();
                mask >>= 1;


Raspberry Pi Remote Root Access

Now some of you will write in no doubt and say how easy this is – how could I possibly not know this. Well, I didn’t. I found out two other things this week – so here’s the lot in one go.

SD Card Cloning

But before I start, here’s something else I just cleared up – cloning of microSD cards. I was SURE I read there was an issue cloning the Pi SD cards. So having set up my Pi2 the way I wanted it with just about everything but the kitchen sink in it, I decided to start the second Pi2 from scratch – well that didn’t work out because about 2.3rds the way through setting up the many programs, the think ground just about to a halt – I mean like 5 minutes to power up. Was it the microSD?, was it the Pi? was it the setup?  I decided I had nothing to lose by cloning the microSD with Windows 8.1 using Win32DiskManager.

So that I did and sure enough – I now have two fully functioning Pi2 boards, one on and the other on – oh and that includes WIFI – I just plugged an identical WIFI module into the second Pi, powered it up and it just worked.

SD Card Lifetime

Something ELSE I just discovered. So I was getting worried about the lifetime of the microSD given that I’m using mySQL for example which surely must hammer the disk. WELL, it turns out that as well as looking for memory cards with SPEED you need to consider automatic levelling. EH? Don’t worry you don’t have to do anything. So imagine the Pi is writing to the card over and over in the same spot. Well, you only get 10s of thousands of writes and depending what you’re doing that could be wrapped up in weeks never mind years. It turns out that SD cards with automatic levelling (Samsung Ultra comes to mind as against cheap unbranded crap) actually have a controller in there that monitor usage and move stuff around for you!! The downside is that there’s always the possibility you finish what you’re doing, shut off and.. lose some data – so – shut the Pi down properly and use some kind of uninterruptable supply. SOME phone chargers will act like an uninterruptable, some won’t… you’ll have to take pot luck on that, I have 4 different battery packs (JuiceBar etc) and only two of them make reliable uninterruptable power supplies.

Remote Access to Pi Files

and so finally onto remote access…. up to now I’m using WinSCP on the PC talking to TightVNCServer on the Pi2 boards.. but could never get past user pi level access – in other words it I typed in root and a password, no matter what password it would not work. I could have SWORN I had typed in a root password.. however, obviously not.

So off I went to the web to find out how to fix  that, most of the instructions said “just enable a root access password and you can then log in using WinSCP as root”. HAH, all very nice but how the hell do you do THAT?

Turns out it’s easy… open a terminal in the Pi (unless you’re already in command line mode) – and type…

sudo passwd root

at which point you get asked for a password twice and Bob’s your uncle!! So now I can access the Pi files from my PC from either device – as Pi or Root as easy as clicking a couple of buttons.



NODE-RED Development

While I’m sitting in a hotel in Birmingham waiting to do my bit in this month’s #FSBCONF I thought I’d drop down a few notes about Node-Red which is definitely my new toy of choice when it comes to a central hub for home control. I say toy, sitting on a Raspberry Pi 2 it is definitely NOT a toy.


So what you see above is one tiny part of what I’m working on for the home controller. The boards mentioned a couple of articles ago need to know what time it is as they don’t have a clock or battery. So when you plug them in – they need to know the time – and as they are also aware of dusk and dawn which clearly changes from time to time, there’s a whole package goes to them on a regular basis, once a day.

There is a “sunrise” node available to Node-Red, but my requirements are slightly different to that which the standard unit allows – so using the Sunrise node as a base I made my own node.

It turns out that as an absolute mininum you need a .JS file and an HTML file. You put them in a suitable named directory under your pi/node-red/nodes directory and that’s about it – provided you’ve not made any mistakes – it just appears as a block in Node-Red.

So what I wanted was a block which would respond to incoming requests from individual WIFI units with their ID and return time info to the individual nodes, usually after a power up or reset – but also send a message to ALL nodes on a regular basis, say, once a day. The WIFI units are programmed to request info at regular intervals until they get an answer – though I’ve yet to see this miss a beat.

The Boards Login block is an MQTT subscribe block – standard. The Daily Update is a standard inject block – and the Send to Boards is an MQTT publish – against standard.

The bit in the middle is a heavily modified Sunshine block – called duskdawn. I made a directory for it – I put the JS and HTML files in there and started gutting them. So now if you click the block you get…


So here are the two files….

The first is as you can see VERY similar to the suncalc node and uses the same functions – but it does not use it’s own timer – it uses an input.  Also it does not generate simple yes/no outputs but generates a message suitable for MQTT.  If a message comes in that says “fromesp” – it sends out the new message to all. If it comes in from a particular unit, it sends that info back to that unit. The info comprises the time in seconds since 1970, the number of minutes after midnight we call DUSK and the number of minutes past midnight we call DUSK.

module.exports = function(RED) {
    "use strict";
    var SunCalc = require('suncalc');

    function SunNode(n) {
        RED.nodes.createNode(this,n); =;
        this.lon = n.lon;
        this.start = n.start;
        this.end = n.end;

        var node = this;

        this.on("input", function (msg2) {
            var now = new Date();
            var times = SunCalc.getTimes(now,, node.lon);
            var nowMillis = Date.UTC(now.getUTCFullYear(),now.getUTCMonth(),now.getUTCDate(),now.getUTCHours(),now.getUTCMinutes(), now.getUTCSeconds());
            var midnightMillis = Date.UTC(now.getUTCFullYear(),now.getUTCMonth(),now.getUTCDate(),0,1);
            var startMillis = Date.UTC(times[node.start].getUTCFullYear(),times[node.start].getUTCMonth(),times[node.start].getUTCDate(),times[node.start].getUTCHours(),times[node.start].getUTCMinutes());
            var endMillis = Date.UTC(times[node.end].getUTCFullYear(),times[node.end].getUTCMonth(),times[node.end].getUTCDate(),times[node.end].getUTCHours(),times[node.end].getUTCMinutes());
            var moon = parseInt(SunCalc.getMoonIllumination(now).fraction*100+0.5)/100;
            var msg = { payload:0, topic:"sun", moon:moon };
            var dawn = (startMillis - midnightMillis);
            var dusk = (endMillis - midnightMillis);

            if ((nowMillis>dawn)&& (nowMillis<dusk)) { node.status({fill:"yellow",shape:"dot",text:"day"}); }
            else { node.status({fill:"blue",shape:"dot",text:"night"}); }

            msg.topic = msg2.payload + 'toesp';
            msg.payload = '{time:' + nowMillis/1000 + ';dusk:' + dusk/60000 + ';dawn:' + dawn/60000 + '}';




This code is complemented by HTML which identifies what inputs are needed, how many outputs etc. This is even closer to but simpler than the Sunrise files. All of the copyright notices in the originals apply, my modifications are not serious enough to claim any ownership though I should say that when I started playing with this I knew nothing about creating these files and now it seems pretty straightforward.


<script type="text/x-red" data-template-name="duskdawn">
  <div class="form-row">
    <label for="node-input-lat"><i class="fa fa-globe"></i> Latitude</label>
    <input type="text" id="node-input-lat" placeholder="51.025">
  <div class="form-row">
    <label for="node-input-lon"><i class="fa fa-globe"></i> Longitude</label>
    <input type="text" id="node-input-lon" placeholder="-1.4">
  <div class="form-row">
    <label for="node-input-name"><i class="fa fa-tag"></i> Name</label>
    <input type="text" id="node-input-name" placeholder="Name">

<script type="text/x-red" data-help-name="duskdawn">
    <p>Uses the suncalc module to generate an MQTT output which contains time, dusk and dawn on demand for a specified longitude and latitude.</p>


<script type="text/javascript">
        category: 'advanced-input',
        defaults: {
            name: {value:""},
            lat: {value:"", required:true, validate:RED.validators.number()},
            lon: {value:"", required:true, validate:RED.validators.number()},
            start: {value:"sunrise", required:true},
            end: {value:"sunset", required:true}
        icon: "sun.png",
        label: function() {
            return||"Sun rise/set";
        labelStyle: function() {
        oneditprepare: function() {
            if (($("#node-input-lat").val() === "") && ($("#node-input-lon").val() === "")) {
                if ("geolocation" in navigator) {
                    navigator.geolocation.getCurrentPosition(function(position) {


So there you have it – with luck you should find this information useful. I AM told by a very kind reader that I should amend any copyright because of the substantial changes I’ve made – but then without the original I could not have gotten this far (I deal with business forums in another life and people are often very unhelpful – whereas in here I’m constantly amazed at the great feedback and help I get)  so I’ll ask merely this – if you find it useful, please somewhere put a link back to the blog so that more knowledgeable people will come back to me with potential improvements to everything I’m doing – something that we’ll all benefit from.

And if you happen to spot a Node-Red node that can create a nice web page to set up a central heating system – please DO let me know as that’s my next requirement…


LED Quick Test

Quick test of the new ESP8266 code for driving the ws2812b LEDs… not far off 2 amps at 5v in at one end – GND and ESP8266 signal other end… fully tested with 300 serial LEDS – this is my next WIFI SAD LIGHT once I find something with enough power and some very thick cable. 70 LEDs here.

purple  green  red