ESP8266 and MQTT Exciting Times

Peter ScargillEclipse, ESP-01, ESP-02, ESP-03, ESP-04, ESP-05, ESP-06, ESP-07, ESP-08, ESP-09, ESP-10, ESP-11, ESP-12, ESP-201, ESP12, ESP8266, MQTT, Openhab, Scargill, WIFI, Windows 812 Comments on ESP8266 and MQTT Exciting Times

I don’t want to jump in prematurely here but those of you who have been reading the blog will know that I’ve had no end of problems with the MQTT stand-alone software on the ESP8266, but as each problem has emerged, the author (Minh Tuan) has been keeping in touch and working with me.  

MQTTLast week having apparently gotten everything working (that’s a real project continuously firing out temperature, responding to incoming time correction signals and accepting commands to turn GPIO0 on and off,  connected to a solid state relay controlling a lamp) when I realised that one of the real-life tests I’d missed out was … what happens if the broadband went off.

To my horror, it resulted in absolute failure.  The software would not reliably reconnect. I had some great suggestions from people – including one to think outside of the box, could it possibly be the MQTT broker not reconnecting properly. I worked on everyone’s suggestions to no avail.  I even updated my development environment (Windows 8 with Eclipse) to the latest SDK + patch (0.9.5) – no difference.  I can’t tell you now many evening hours I spent on this.

ESP-01Then this morning I had a contact from Minh to say he’d sorted the problem. I downloaded the code – and tested it – turned the WIFI off for a few seconds – turned it back on – HURRAY, reconnect no problem, re-subscribe no problem, queued outputs sent no problem. I was just about  to report success when I thought I’d see what happened if I left the WIFI off long enough to fill/overflow the queue (there is a RAM queue for messages and it includes incoming subscribes and outgoing messages, I had a hunch that a full queue might stop re-subscribes). Sure enough… the grim news on leaving the WIFI off for several minutes and reconnecting was that the software started up again – but all subscribes were lost – no room for them.

Thankfully Minh was around and immediately spotted this – we agreed that if the queue filled up completely and the WIFI restarted, the queue would be SCRAPPED.  Ok there are better ways and I’ve asked him to look at this – but for a quick fix he sent me a replacement mqtt.c file and… I put it to the test.

MQTT[6]3 hours later I cannot crash the software. No matter how long the WIFI is off it always reconnects properly. Some earlier strange issues on power up have disappeared (I feared they might be my own non-volatile variable storage additions.. but no).

The library has been updated accordingly just minutes ago and is here https://github.com/tuanpmt/esp_mqtt.

You really should take a look at this. For the first time we potentially have the means to make an ultra-reliable control system with the ESP8266 boards. Recent mods to the code free up RAM (an issue for example with the Lua software) and so I have no doubt there is lots of room in there to add your own bits.

To recap for those who’re looking in for the first time, anything I refer to here will be covered in other blog pages – but I’m using MOSQUITTO – in my case running on a Diskstation NAS Drive for reliability but it will also run on various platforms and there are servers out there for people to use as well. I’m testing this with the excellent MQTT-SPY and all of this without having learn a word of Linux Smile  thanks to the excellent feedback I’ve had on this blog and the work of others around the world who like me are so keen to get this working.  If all goes well I can now concentrate on mastering the likes of Openhab so that once I have boards in place controlling stuff, I can easily monitor and control the boards themselves from a mobile phone – and for the first time with a little security in place – the point of investigating MQTT in the first place!

phpIncidentally, the PHP page I use under CRON to push the time out to my MQTT broker – I had some fun with that as I realised it was pushing out rubbish (wrong time) and further to a PHP upgrade at my provider – was throwing in warning messages to boot. Here is the latest working version (been up there running 24/7 for a week now) with security information altered to protect the innocent. To keep things simply, the dawn and dusk information is only accurate to the nearest minute. That lowers the number of times it will be updated and as I keep this in FLASH, the number of times the FLASH is updated.

<?php
error_reporting(E_ALL ^ E_NOTICE);
require(“../phpMQTT.php”);

function varcheck($var, $default)
{
return isset($var) ? stripslashes($var) : $default;
}

$locn = varcheck($_GET[‘loc’], “Europe/London”);
$lon = varcheck($_GET[‘lon’], 55);
$lat = varcheck($_GET[‘lat’], -2);

date_default_timezone_set($locn);   
$dateTimeZoneLocal = new DateTimeZone($locn);
$dateTimeLocal = new DateTime(“now”, $dateTimeZoneLocal);
$localDateTime = date(“H:i:s d-m-Y”, time());
$localDisplayDateTime = date(“H:i l d-m-Y”, time());
$localTime=strtotime($localDateTime);

$mqtt = new phpMQTT(“your.mqtt.server”, 1884, “whateverYouLike”);
if ($mqtt->connect(TRUE,NULL,”yourwifilogin”,”yourwifipass”)) {   
            $mqtt->publish(“time”,$localTime,0);
            $mqtt->publish(“timestring”,$localDisplayDateTime,0);
            $sun_info = date_sun_info($localTime, $lon, $lat);
            foreach ($sun_info as $key => $val) {
                if ($key==’civil_twilight_end’) $mqtt->publish(“dusk”,(int)(($val %86400)/60),0);
                if ($key==’civil_twilight_begin’) $mqtt->publish(“dawn”,(int)(($val %86400)/60),0);
            }
    $mqtt->close();
}
?>

Hopefully now I can stop worrying about reliability and start having some fun. I have suggestions for better ways to store things in FLASH and there is much to learn about Node-Red, OpenHab and much more so this is really just the beginning.

12 thoughts on “ESP8266 and MQTT Exciting Times”

    1. I can’t but I’m not having any issues with uart now, haven’t for a while – other than a slightly bodged operation where the interrupt driven uart handler sets a flag when there is a line full of data and a timer callback loads up that info. It works perfectly but I’d prefer something a tad more polished. MQTT similarly, I’ve updated my code whenever there have been changes to TuanPMs original and I’m aware of no issues with it.

  2. Thanks for blogging your process, I am having a lot of problems with lua resetting on my ESP8266-01. I complied tuanpmt’s code but still can’t get it to work. Do you have your source up on git to share? Would love to use it as a reference. I think it will be great to have a working example for a particular model, eg. ESP8266-01, with 1 input and output gpio. I will create 1 once it get it working. I the mean time I will read through your blog, hopefully it will solve my problems.

    1. I don’t have it on GIT and no matter how hard I try I just cannot get it through my head how GIT works. And right now every alternative compilation produces a duffer so you really don’t want to be using my code quite yet and my doc is still in my head.

    2. The instructions are quite clear, what exactly goes wrong? – I am on Debian 8 – I changed the code to log onto wifi and the settings are all in include/user_config.h. I get stuck lots and get lots of help so keen to help where I can?

  3. Hi Pete,

    I too am a bit confiused as a newbie to these modules. Ive been playing around using LUA script and mqtt ( basically what this thread is all about http://www.esp8266.com/viewtopic.php?f=19&t=1278 )

    but i think and correct me if im wrong, but you are using the esp chips differently as in not using LUA, and instead ( i think i understand) you are using Eclipse to write the code (in C ), compile and update the chip?

    Cheers,
    Greg

    1. That’s correct Greg. Every time I’ve tried to use Lua for anything non-trivial I end up running out of RAM. I need the kind of control I can give myself using C – and RAM is much less of an issue. The Eclipse system with the many examples that come with that setup make it reasonably easy to get a start using C and the latest MQTT code. Right now I’m just figuring out how to use the remaining IO on the likes of the ESP-12.

      1. Same thing for me with LUA and MQTT, keeps resetting even for trivial code.
        Too bad because LUA makes prototyping very easy with quick code turnaround…

        So, what’is a good environment to setup eclipse to build C code for the ESP? Any pointer?

        (Another) Greg

        1. I’m using Windows 8.1 and the unofficial Eclipse setup. I detailed it in an earlier blog I believe. Grab coffee however it does take a little while – but one there you have ready made examples.

  4. I am sure with the Red Node no problems. It is very easy to put together and easy to understand. MQTT blocks are built and work well.
    Please tell me there is an instruction or a step by step example for setting https://github.com/tuanpmt/esp_mqtt ? I’m just starting to learn and want a quick start. How to compile the code. How to flash. ?? A lot of questions. Give links please.
    Thank U.

      1. I too am looking for some information on how to get the esp_mqtt project imported into Eclipse to build. Did you write anything on that? I haven’t found it 🙂 I enjoy your blog a lot though!!

Comments are closed.