Category Archives: Eclipse

Simple Scope

SPIYou may have noticed that I’ve been working on my ESP8266 home control software (see the updates elsewhere along with new diagram), specifically upgrading and adding to the number of OLED and LCD displays I can handle, either by I2c or SPI.

I’ve been doing a lot of optimising and simplifying – for example with SPI – assuming that there will only be the one SPI device at once on an ESP means you don’t need the CS line which can be grounded.  But there’s more….

Continue reading Simple Scope


ESP8266 Remote Access

Notice I titled that remote access as against remote programming – but that’s part of it.

Years ago, having gotten sick and tired of dismantling my home control Arduinos just to upgrade the code, I purchased a couple of small units from China which I’ve just thrown in the bin.   One had a USB plug on it and plugged into the PC, the other had an FTDI output on it. Both had those black 2.4Ghz antenna.

So the purpose of these units was to wirelessly replace an FTDI – or serial programmer, the idea being that instead of bringing the Arduino project back to wherever one programs the devices, you plugged the unit into the project and over the air programmed it.

At the time I was using Arduinos and those CHEAP £1.50 radio modules – the NRF24L01s, the worst radio boards in the world who’s range could be so bad you’d be lucky to get through one wall. I had high hopes for them with a great mesh network called RADIOHEAD, but ultimately their range and lack of ability to read signal strength doomed them.   However, as you can imagine they were dotted all over the house and I thought this little radio pair would solve the issue of programming them without ripping everything up.

It turns out the “radio FTDI” had the same problem – it was based on the NRF24L01.

Fast forward to September 2015. Here I am in Spain with a new setup of ESP-12 modules controlling lights, watering system and a lot more.  I’ve made some improvements to our “home control 2015” software in the ESPs and once again I find myself having to dismantle kit.

Now, I know that is it possible to program over the air – a few guys have tackled this successfully and indeed much of the code is built into the Espressif SDK. Right now my friend Aidan is working on adding this to our code but there are two issues, he’s not had luck up to now and secondly as our code comes to something like 300k you’re never going to be able to add this to the ESP-01 modules (we use ESP-01 sometimes, ESP-12 other times…)  as you need as much spare storage as you have code for OTA programming!

It was with that in mind that last night I dragged out the old FTDI boards to see if I could save myself the effort of dismantling the Pergola lighting system to update it.  Suffice it to say that I managed to get a few bytes programmed before that all came to an end despite several attempts. With around 25 WIFI units of one description or another dotted around the place, the old wireless FDTI (same frequency range) simply could not handle more than a few feet.


I figured it would not hurt to look around and see if someone has alternative wireless solutions – lots of time has passed since I bought that kit.

And so it was that I stumbled upon this. – as it happens it is quite new – essentially what we are looking at is a single unit (yes, that had me going for a while) which will hook into the WIFI system and act like an FTDI – given that you can fire the serial out of your PC over TCP to a port – port 23.   I grabbed the code (the stable 1.04 version) and very excitedly blew it into an ESP-01 unit I had lying around.

Really, nicely done web interface (with remnants of Martin Hanzarov’s code in there) and a really good write-up – indeed there may be enough info in there to help us implement our own FOTO (over the air) updates to our code.

But this has far wider implications – a virtual serial link would also let you monitor the serial line of a remote unit for debugging info – and our units do indeed put out lots of serial debugging info. It would also let you remote program and monitor un-wirelessly enabled Arduino projects.

So the first problem was – I had this unit sitting as on my network and I needed to blast info into port 23 to have it come out as 115K serial.  I have ECLIPSE sitting on the PC running my projects and there’s a line in the MAKE file that says “COM6”.  I have absolutely NO idea how to turn that into a TCP statement if indeed it can be done (FEEL FREE TO START TYPING IF YOU KNOW HOW).

So I turned to a 30 day trial of a PC program that promised to provide a “virtual com port” – and that it did. Serial/IP Redirector let me create a link to port 23 and ship raw data back and forth to COM22 (which doesn’t exist on my PC)- LO AND BEHOLD I set it running, put PUTTY on the PC talking to COM22 and with a real FTDI attached to the little ESP-01 board on real COM6 I set that running on a terminal –at 115K. Utterly flawless raw serial communications – rock solid.

I was getting excited by now. I took my ESP-01 board and cross-connected the serial to an ESP-12 test board. I held GPIO0 to ground and turned the power on, putting the ESP12 into programming mode.

I punched COM22 into the ECLIPSE project and hit the FLASH button on the project. Lights flashed on both units. With a scope attached  to the serial in on the ESP-12 I could see clean 115k data going into my project and… nowt, zilch, nothing.

I was up until the early hours ensuring the baud rate was ok and that my level conversion wasn’t getting in the way – you see all my projects use a resistive divider on the serial input of ESP boards so the whole lot will work just fine with 5v logic on the FTDI and elsewhere.  I wondered if hooking two of these units together with serial cross connected might prove too much but the scope says no – it says I’ve a perfectly reasonable serial signal going into the ESP-12 – but it’s not having it – no responding output whatsoever. Eclipse tries sending out data over and over and eventually gives up just as if there’s nothing connected.

So near and yet so far.

I got up this morning and the unit was still rock-solidly connected to the WIFI and typing a few characters provided perfect results at the other end – but I cannot program ESP units – yet.

Very exciting, yet very frustrating.

Perhaps coffee and a new start to the day will help. BUT I had to bring this to you as the article is really good, the software seems to WORK and there’s a super-bumper version with it’s own OTA updating in the works for larger targets like an ESP-12 (more FLASH) than the ESP-01.

I’m sure this is going to be a winner – just need to get it to work. After all, the ability to add remote programming to any FTDI-enabled project for what, a couple of quid.. well, you just can’t ignore it.

Meanwhile if anyone reading this firstly knows how to set COMX on Eclipse (on Windows) to TCP – do let me know – and if you really think you know what I’m doing wrong here and want to jump in before I figure it out – by all means – you have the keyboard…


Lua Revisited

Back in the dark ages when the ESP8266 was something new, I had a go at the nodeMCU software, essentially implementing the Lua language (or a subset of it) on the ESP8266.  At the time I was not familiar with Lua but thanks to some of the excellent resources out there on the web, I quickly caught up.

Sadly, my expectations exceeded the ability of the implementation to match and every time I went past flashing a few lights – CRASH. I gave up in disgust.  I’m not complaining as that gave me the impetus to learn the hard way how best to program these chips – which I maintain is using the C language – and in my case via the unofficial development kit which is just an excellent piece of work by Mikhail Grivorev, leading to a complete Windows environment with a ton of examples. Between that and TuanPMs excellent MQTT code as a starting point  I’ve been happily programming away and even built up a working relationship with Espressif in the process. My home control project is well under way and now just waiting for some new boards – and a function that Espressif have promised to shut down PWM.

LuaMeanwhile this morning, I can’t remember what I was reading that triggered me off, I decided to have a quick look at the node-MCU code again.  I had it sitting in the development environment but figured it would be out of date and so went off to the repository to study any changes – which I duly put into the code. The GIT Makefile is useless for Windows but fortunately there was one in the environment – and as it happens, a good one. I wasn’t about to go backward so I made sure I was linking to the very latest SDK which at this time is 1.1.2 with at least one patch – all available from the Espressif forum.

I checked the settings in the Makefile, adjusted the SPI speed up to 80Mhz, set the FLASH size to 4Mb to suit the sparkly new ESP-210 that is sitting here looking for a job – and started the compile process. Within a minute and without any issues, the code was flashed. I have to say I was expecting at least one or two issues – but no.

I set up a 9600 baud terminal, in the process discovering for the first time that you can have multiple terminals at once in the Eclipse environment – handy (it would be more handy if it always reported the right location of bugs but you can’t have everything). Sure enough – a working Lua board.

I told it about my WIFI setup and sure enough it grabbed that too.  The project has come a long way since the early days and now (apparently) supports json, a file system, timers, pwm, i2c, spi, 1-wire, net, mqtt, coap, gpiio, wifi, adc, uart and more. I’ll believe all of that when I see it.

I was delighted to see that both the integer and float versions are available – that’s a plus! I also noted support for the WS2812b chips – I noted that as I had to make a change to the source code and I’m delighted to note that they’ve made use of a local compiler directive that I was not even aware of to ensure accurate timing. Wish I’d seen that a month ago when I started work on my own WS2812b support. 

WS2812B LEDs on nodeMCU LuaI plugged a (short) length of WS2812b strip (a custom board we had made) into the ESP-210, fired the instruction and… worked first time out of the box. Impressive.

I’m pleased to see the nodeMCU docs have come on a long way – sadly the website was running like a DOG at the time but here’s a few stats I managed to get off the board.

print(node.heap()) returned 20kbytes which is more than when I tried it – but then it used to die when the heap got down to 5K so we’ll have to wait and see how reliable that is.

3 Channels of PWM are available – at 10 bit resolution. Clearly some work to be done here as Espressif have just released a library handling more outputs than that at nearer to 14 bit resolution – and it works as I’ve been using it in my own research and projects. However – handy enough.

There are libraries for sending to displays complete with fonts – now that I’m interested in – if this will do reliably MQTT and report results to a display I have a use for this already – wall thermostats.

So, overall, a very brief test – it is possible with the right environment to get interactive and easy to use nodeMCU up and running in minutes – you don’t need special boards and the software is free. Can’t be bad – how complex a program you can make withou running out of RAM remains to be seen. Here is some more info on nodeMCU.


ESP8266 and MQTT Exciting Times

I don’t want to jump in prematurely here but those of you who have been reading the blog will know that I’ve had no end of problems with the MQTT stand-alone software on the ESP8266, but as each problem has emerged, the author (Minh Tuan) has been keeping in touch and working with me.  

MQTTLast week having apparently gotten everything working (that’s a real project continuously firing out temperature, responding to incoming time correction signals and accepting commands to turn GPIO0 on and off,  connected to a solid state relay controlling a lamp) when I realised that one of the real-life tests I’d missed out was … what happens if the broadband went off.

To my horror, it resulted in absolute failure.  The software would not reliably reconnect. I had some great suggestions from people – including one to think outside of the box, could it possibly be the MQTT broker not reconnecting properly. I worked on everyone’s suggestions to no avail.  I even updated my development environment (Windows 8 with Eclipse) to the latest SDK + patch (0.9.5) – no difference.  I can’t tell you now many evening hours I spent on this.

ESP-01Then this morning I had a contact from Minh to say he’d sorted the problem. I downloaded the code – and tested it – turned the WIFI off for a few seconds – turned it back on – HURRAY, reconnect no problem, re-subscribe no problem, queued outputs sent no problem. I was just about  to report success when I thought I’d see what happened if I left the WIFI off long enough to fill/overflow the queue (there is a RAM queue for messages and it includes incoming subscribes and outgoing messages, I had a hunch that a full queue might stop re-subscribes). Sure enough… the grim news on leaving the WIFI off for several minutes and reconnecting was that the software started up again – but all subscribes were lost – no room for them.

Thankfully Minh was around and immediately spotted this – we agreed that if the queue filled up completely and the WIFI restarted, the queue would be SCRAPPED.  Ok there are better ways and I’ve asked him to look at this – but for a quick fix he sent me a replacement mqtt.c file and… I put it to the test.

MQTT[6]3 hours later I cannot crash the software. No matter how long the WIFI is off it always reconnects properly. Some earlier strange issues on power up have disappeared (I feared they might be my own non-volatile variable storage additions.. but no).

The library has been updated accordingly just minutes ago and is here

You really should take a look at this. For the first time we potentially have the means to make an ultra-reliable control system with the ESP8266 boards. Recent mods to the code free up RAM (an issue for example with the Lua software) and so I have no doubt there is lots of room in there to add your own bits.

To recap for those who’re looking in for the first time, anything I refer to here will be covered in other blog pages – but I’m using MOSQUITTO – in my case running on a Diskstation NAS Drive for reliability but it will also run on various platforms and there are servers out there for people to use as well. I’m testing this with the excellent MQTT-SPY and all of this without having learn a word of Linux Smile  thanks to the excellent feedback I’ve had on this blog and the work of others around the world who like me are so keen to get this working.  If all goes well I can now concentrate on mastering the likes of Openhab so that once I have boards in place controlling stuff, I can easily monitor and control the boards themselves from a mobile phone – and for the first time with a little security in place – the point of investigating MQTT in the first place!

phpIncidentally, the PHP page I use under CRON to push the time out to my MQTT broker – I had some fun with that as I realised it was pushing out rubbish (wrong time) and further to a PHP upgrade at my provider – was throwing in warning messages to boot. Here is the latest working version (been up there running 24/7 for a week now) with security information altered to protect the innocent. To keep things simply, the dawn and dusk information is only accurate to the nearest minute. That lowers the number of times it will be updated and as I keep this in FLASH, the number of times the FLASH is updated.

error_reporting(E_ALL ^ E_NOTICE);

function varcheck($var, $default)
return isset($var) ? stripslashes($var) : $default;

$locn = varcheck($_GET[‘loc’], "Europe/London");
$lon = varcheck($_GET[‘lon’], 55);
$lat = varcheck($_GET[‘lat’], -2);

$dateTimeZoneLocal = new DateTimeZone($locn);
$dateTimeLocal = new DateTime("now", $dateTimeZoneLocal);
$localDateTime = date("H:i:s d-m-Y", time());
$localDisplayDateTime = date("H:i l d-m-Y", time());

$mqtt = new phpMQTT("your.mqtt.server", 1884, "whateverYouLike");
if ($mqtt->connect(TRUE,NULL,"yourwifilogin","yourwifipass")) {   
            $sun_info = date_sun_info($localTime, $lon, $lat);
            foreach ($sun_info as $key => $val) {
                if ($key=='civil_twilight_end') $mqtt->publish("dusk",(int)(($val %86400)/60),0);
                if ($key=='civil_twilight_begin') $mqtt->publish("dawn",(int)(($val %86400)/60),0);

Hopefully now I can stop worrying about reliability and start having some fun. I have suggestions for better ways to store things in FLASH and there is much to learn about Node-Red, OpenHab and much more so this is really just the beginning.