Category Archives: Arduino

Cheap Serial Terminal

How about a cheap serial terminal for less than a fiver all-in?

In a previous blog entry I’ve been working on more developments for the increasingly powerful home control system using an Arduino as a kind of universal I2c peripheral to the ESP8266. The reason for that is just the vast range of driver software our there and the fact that from China you can get an Arduino-Nano-type-board for under £2. There are drivers for just about everything so it seems daft to just ignore that. It ends up cheaper than most actual i2c peripheral boards.

First thing I had to do with the Arduino base code was increase the size of the buffer used for storing I2c and that is covered elsewhere – but finally I got that working. In the process that led me to the idea of a prototyping board with both the ESP8266, running the home control 2016 software AND a 328 to power some peripheral devices and give extra IO pins.

In the process of THAT I got the QDTECH boards running (128*160 LCD – cheap) – and that also is covered in a previous blog entry. These are great – and cheap (though not as cheap since Banggood jacked the price up but thankfully there is still AliExpress – thanks to Squonk for that particular link) but they are a little limited in terms of resolution as any kind of terminal. I even found a little bit of code to do vertical scrolling. Fast enough but then it is a low resolution display.

In the meantime however I pulled out another favourite display of mine, the ILI9340-driven displays at 320*240 – at £3.47 – mind you if you go to the Italian version of Aliexpress they seem to go down to just over £3. The increased resolution of these means you really COULD make a nice monitoring terminal but for one thing – speed.  I checked out several implementations including the Adafruit ones and the drawing speed is atrocious – attempts to scroll the display are generally abysmal unless you use a fast processor.

Terminal[1]And so it was that I stumbled on THIS article.  The funny thing is that this is on an Arduino site – but points to an article that is anything but Arduino –the fellow has written some FAST code to chuck out small text in the form of a (subset of a) VT-100-compatible terminal. But the code needed you to use the C++ compiler on the command line, NOT in the Arduino environment – and that was no good for me as I have to add other stuff. But if you take a look, you’ll see my late hours have been FAR from wasted.

In the picture on the right (depending on your screen) you see this terminal running – it has  2 fixed areas and a scrolling area and it is fast.

The first problem was that this needed to be compiled outside of any environment and it simply would not work for me – the linker had issues and so on. I realised if this was to be useful – it needed to be inside the Arduino IDE where I could shove in all sorts of other code including my I2c connection.

I opened up a NEW Arduino project, copied all the files in there and renamed the primary file to be the same as the project. I then put the contents of his MAIN() code into “LOOP” – pulling out initialisation and putting that into “SETUP()”.  Stunningly, it all SEEMED to work – I just don’t have that much luck normally. However any attempts to use the serial port failed miserably. So I threw away his serial code and put in the normal Arduino serial – that worked just fine – but when I actually tried to do anything with it other than “hello world” – I got garbage characters.

I was reminded of the work I’d done to expand the I2c buffer on the Arduino and that reminded my that the normal Arduino serial buffer is a staggeringly useless 64 bytes.  I’m including hopefully helpful links here as you can fix this stuff. Despite the limits of the Arduino default software settings, I need way more than that so I increased the buffer to 1K, using up a significant amount of Arduino RAM. Voila – perfect.

Debug infoAs it happens I’ve been experimenting on a 1284 chip which while being Arduino-compatible kind of, it does have another serial port, so without disconnecting my FTDI – I hooked up the second serial port to the home control serial and fired the DEBUG command at the latter.

As you can see on the left, a perfectly usable terminal and now with programmable on-screen LEDS and baud rate control stored in EEPROM. At this point despite YEARS of VT-100 wilderness I’d mastered the colours and scrolling area controls – thanks to this handy VT-100-related link

I’d even noticed that in the code, there was a fast horizontal line command. Putting the driver code side by side with the Adafruit code for the same chipset, there were similarities. I should be able to drop in a PIXEL command to get a dot up. YES, with slight mods it worked. For my next trick I would add in a fast VERTICAL line routine from Adafruit – and then – the general LINE drawing routine and the world would be my oyster.

Erm, not quite – the Adafruit SWAP routine caused me endless issues – but I replaced that and all is now well.  What’s important here is that this is FAST.  I’ve put a little character count on the bottom status line so we know that something is actually happening in the case of repeating commands… and this could form a great plug-in for our prototyping board – or indeed any kind of experimental setup where having a dedicated little serial monitor costing next to nothing would be handy.

The code is currently sitting on BitBucket. Oh - and if you use these displays, don’t make the mistake of thinking, as they run on 5v that the LED power should be 5v. It works for a while but I just burned a LED out on the larger display – and yes I’d noted that it was running a tad warm and didn’t twig.  Now the LED supply is connected to 3v3 and it is just fine (apart from the one bust LED). I could probably do PWM brilliance control but that’s something for later. Right now I’m running the entire display on 3v3 and it is just fine. My tiny 3v3 regulator on my boards isn’t even breaking a sweat.

Block LEDs

In the VT-100 command-set, they refer to a command to turn LEDs on and off, 4 of them in fact and I thought that might be fun to implement – now bearing in mind I’ve no prior experience of the switch statements used in the VT-100 code I managed to get this lot running this morning.

On the top right of the image here, you’ll see 4 simple rectangles in red – filled or not. The state of those LEDs can be controlled as follows…

ESC [ parm q

Where parm is one of 5 values 0=all off, 1=one on, 2=2 on, 3=3 on, 4=4 on.

That apparently is the standard though WHY it won’t let you turn an individual LED off I don’t understand so I’ve extended this so that 5 turns off  LED1, 6 turns off LED2 etc.

That’s how I’m interpreting that particular VT-100 command.

I’ve added a non-standard command to set baud rate as I could not find a standard command to do it. This is stored in EEPROM.

ESC [ parm X

Where parm= 1,2,3,4,5 or 6 for 300,2400,9600,57500, 76800 or 115200 baud. Why that 76K? Because on power up that’s the speed the ESP8266 chucks out debug info. 


I now have code running native on the ESP8266 to handle this display and the terminal – so as not to detract from this article I’ve split this into another article – but you can see the first attempts running here…


HC 2016 Experiments

Throughout the development of the Home Control 2016 project I’ve constantly had the software tied a via a serial umbilical to my PC to monitor passing messages, make sure all is well etc. and to periodically check on RAM space to make sure nothing I’m doing is running away with precious memory.

You may have seen elsewhere that I’ve been working with a prototyping board which is basically an ESP12 with a prototyping area in which I’ve put rows of pins for ground, 3v3, 5v and the two lines I use for I2c as I add various peripherals culminating in the recent idea to use an Arduino 328 chip as an “ultimate peripheral”.

Well, we’ve taken that one a stage further and we’ve been discussing making our own prototyping board (Aidan’s a whizz at PCBs and of course we use the likes of Dirty PCBs in China so it is quite cheap to knock up a few boards) and the last few days I’ve been formulating a planas Aidan puts together the schematic. What I’ve realised is that I always run out of the power and ground lines no matter what I do. Add to that the FTDI so I can monitor serial Comms and it all gets a bit messy.

Display for HC2016

So right now we’ve been discussing a board with the ESP-12 module with lots of pins and additionally a 328 SMT chip with Xtal. Of course that means 2 serial monitors. I’ve been playing with the old QDTech 128*160 displays since I discovered that they work pretty well when driven from the proper SPI pins on Arduinos (don’t even think of using software SPI on them) and ramping up the SPI speed – and so it was that in conversation, we were saying “if only you could scroll them” – that led to thoughts of having a RAM buffer larger than the total available on the ESP8266 or Arduino and… well, I thought I’d use our friend Google to go take a look. It turns out – I was completely unaware of this and VERY pleasantly surprised to find – that the QDTECH chip has the ability to scroll up built in – all that is needed is to scroll up an area then paint a fast horizontal black line after each scroll. 

There is a version of the driver for the board here and at some point in 2014, it turns out that an M J Sage added this scrolling ability. Well done.  I wonder if he or she realised what a useful addition this could be.

So now we have a display, easily driven by the 328 which can scroll constantly (I’ve had it on several night’s test) and hence provide a boatload of status info. Top and bottom can if needed remain static.

First Stab

We’re going to incorporate the display onto the prototyping board and it won’t take any additional space as it will sit neatly over the other components! The image above probably bears no relation to the final board but I’m just trying to convey an idea here. The prototyping area on the right of the board will be chock-full of PTH holes but there will be long runs of connected holes for power and I2c as these end up being needed the most with connector wires. All 0.1” of course.

With the QDTech display you can connect the reset line to the 328 reset and hence here are my first thoughts at the use of the 328 which will have access to A6 and A7. I’ve pretty much got this working – just need to add in the temperature handling code I already have for Dallas and DHT chips.

D0-7 – general purpose port extender – inputs or outputs or any mix
D8     D/C for QDTECH
D9     16- BIT PWM out
D10  16-BIT PWM out
D12   MISO
A0      CS for QDTECH
A1      Temperature
A2      Temperature
A3       Debounced input
A4      I2C to ESP
A5      I2C to ESP
A6      Analog in 1
A7      Analog in 2

Aside from losing GPIO2 and GPIO14 inputs (as they’ll run the I2c) the ESP use will remain the same.  The only restriction being that we cannot run ESP PWM at the same time as the I2c due to timing interference– a pain but not something I can see away around. The I2c 16-channel PWM controller does seem a good way around this as it offloads all PWM work onto a cheap board – for single colour PWM lighting the new 16-bit PWM on the 328 works fabulously.

Given that the 328 can handle inputs for the main board we can do away with our normal GPIO2 and GPIO14 and they can have the job of I2c, leaving all normal outputs as for the existing system.

I guess the end-game here will be the smallest possible board that contains both the 328 and the ESP wired to this spec for general use. But that comes later after we see how well all of this works in practice. First the prototyping board. Watch this space.

Sadly in the link I’ve enclosed, the author refers to Banggood who used indeed to offer these displays for under £3 at a time when there were no software drivers for them – (I think we paid £2.60) but I’ve noticed recently that Banggood have been getting greedy and jacking their prices up – a cursory check of the link shows they now want £5.48 for these simple LCD displays – well good luck with that Banggood.

The only ones I’ve found up to now – and the price is good – as usual – is AliExpress.

Someone have another link?  the board has an 8-way connector at one end and a larger connector at the other – with a full size SD socket on the underside. See the AliExpress link.

This blog entry could well change dramatically as we spot fatal flaws in the plan above Smile


Faster ESP I2C

Experimenting with I2c on the ESP8266? I am – and I’m having a great time with it. If you’re using my code you don’t really need to think about it but if you’re hunting around for better I2c for the ESP – or maybe interfacing the ESP8266 to Arduino – then you’ll love the stuff I’m doing right now – and yes, I AM looking for I2c experts to tell me if I’m doing something wrong because it is all going rather too smoothly.

I decided to take the end from the info from the last blog entry and separate it off as I learn more about I2c (which to recall is a simple 2-wire multi-drop serial communication protocol with separate clock and data).

Why read this? Well, up to now because I spotted a howling issue in the WIRE library for Arduino, found an improvement on the ESP8266 SDK I2C code in the ESPRUINO changes to the Espressif I2c code - I think I found a mistake in it, fixed it then SAILED past the original speeds, reducing a 15ms package best case to well under 3ms  while adding clock stretching into the bargain.  I should clarify at this point that I am NOT an expert in I2C but hey – I can’t get this to fail so I might be onto something! Read on!

So firstly let’s recall -  I’ve been trying to make an Arduino peripheral for the ESP8266 Home Control 2016 project.  I started with the WIRE library in Arduino, working on a small NANO device. I discovered that there are two buffers in WIRE that are 32 bytes in length and hence limit packages to less than that  - I replaced them with a couple of 128 byte buffers and solved lots of issues I’d been having – that is detailed in the last blog.

Where we left off, I’d made a test setup to send a message to the newly-created NANO peripheral which right now on my bench handles a QTECH display and some IO. The ESP8266 I2C is also talking to a BME280 temperature/pressure/humidity sensor and a PWM expander board – so there are 3 devices – some input, some output.

In the last blog entry I’d added a test command for the ESP which could be sent to the board serially or by MQTT and went like this…

{nano:9,7,"how are you today and the weather is nice - NO REALLY it certainly is.",0}

Simple enough – send a message to device 9 (my chosen default address for the NANO peripheral), command 7, a string – and a dummy 0 on the end to tell the device not to bother sending any return info.

As you can see below, this operation takes, using the standard Espressif Ic2 Master code, 15ms in total – not stunningly fast but ok.i2c_thumb2

Flush with the success of finding that buffer issue, I decided to go into the ESP I2C library and have a tinker – changing all the delays into ‘'#defines so I could mess with them. So without any real effort I could reduce delays from 14ms to 6ms – a very worthwhile improvement. bear in mind however that the ESP code has no clock-stretching and hence no way for the peripheral to slow things down. I am told that the call-back routine within WIRE holds the clock down until it is finished, in order to slow things down – this would do nothing but cause havoc for the Espressif code.

Below is a read operation which I got to work properly after reading  this video – as I had no idea what a re-start was – see the little green circle just after the third byte.


As you can see in the image above I address device 9 (18 as it is shifted left by one – i.e. 7 bit addresses but the bottom bit is reserved to indicate a read or write) – I fire command 2 to read port 4 – then with an I2s restart (start without a preceding stop), I  resend to the device with the LSB set (ie 19) and get a value 0 back (from a pin)  All in a matter of a millisecond.

I was at this point getting worried about lack of clock-stretching – in which a slave can hold the clock low to “stretch” things out a bit – about the only control the slave has!

It looked as if this forum might have some answers as their modification of the ESPRESSIF code certainly had clock stretching in it. It involved some changes and I decided to pick the best bits from their modification.

So they’ve simplified the code a little – while adding in clock-stretching – by replacing a simple delay with a check for clock low – easy enough when you look at it… but they also seemed to have completely messed up master_start – at least, they had it the opposite way to Espressif – and accordingly, repeated starts – as I use in getting data out of a Nano above – simply failed. The logic probe said this was all wrong.

So I reversed master_start back to the way Espressif have it  –so I ended up with a mix of the old and new – and the result – well, up to now everything works – and that long 15ms string send was now reduced to under 5ms from 15ms.

5ms string_thumb[2]

I tested the code with the PWM chip I’ve been playing with and the BME280 – so that is both reading and writing and up to now both were working perfectly. 

It was notable that the clock low period was now much longer than the clock high period and I wondered if there is any way to shorten that.  You know when you get deeply into code and all of a sudden it all becomes clear.  I realised that setting and checking ACK signals was a function – with all that entails – I changed that to a macro. I realised that delays in loops were really un-necessary as the loop itself would cause delays.. I took them out.

At this point I had backed everything up expecting my experiments to fail miserably. I tested the new code and everything worked. I got the logic probe out again.


My original test string at 15ms was now down to 3ms and according to the probe – all is well. I DID get it down to 2.5ms but the probe said I was doing restarts in the middle of the string – I’ll find another way around that one. Current state of the art – 2.9ms

As for reads – my original byte read was taking somewhat over 1ms – it now takes 0.25ms.

fastish read

And of course I’ve concentrated on the read and write routines leaving the inter-byte handshaking pretty much along so maybe there’s another 10% improvement to make here without dipping into ESP assembly language. 

The driver and firmware libraries in the repository above contain the modified Espressif i2c_master code – and also other I2c wrappers for sending packages – they are fine – any further optimisation needs a good solid look at the one file (and it’s header) for i2c_master.   Open to ideas (that don’t involve converting Arduino assembly language to work with the ESP SDK – been there, failed).

I have just noticed that in the write cycles, there is still more off-time than on for the clock – and so I just took a delay out – which SHOULD mean the off-time is way too slow but because of the time taken to call and process the function – we still end up with 0.9us on time and 1.3us off time – everything continues to work on the tests I have – yet the total time for that read drops from 0.25ms to 0.21ms and the string write time from 2.9ms to about 2.2ms – again – worthwhile improvements.

That simple change was made in the write routine…

i2c_master_writeByte(uint8 wrdata)
    uint8 dat;
    sint8 i;

    for (i = 7; i >= 0; i--) {
        dat = wrdata >> i;
      //  i2c_master_wait(I2C_DELAY_5);

Logic analyser says yes, 3 test items say yes… how low can it go!


I2C the Easy Way

I2c on IOTBEARIf you’re going to experiment with I2C – may as well do it the easy way. Having spent the past few days with a desk that looks remarkably like a bowl of spaghetti, I’ve finally gotten around to making a special IOTBEAR board up for the job. 18 each of ground, +3v3 and +5v lines – and 16 each of GPIO4 and GPIO5.

This gave me the opportunity to tackle that long string problem in Arduino Wire. A brief attempt with the logic analyser suggested that my ESP i2c was sending out at least one byte more than the Arduino was receiving – impossible to tell if more because the ESP would stop sending as soon as the Arduino would stop receiving.

And that brings me to a question – is anyone aware of a nice, pretty I2c and other protocol analyser based on a Raspberry Pi? Seems to me that would be a good use for an old RPI2?

Anyway I digress… I’ve had issues with I2c experiments in the past – usually when sending strings. I’ve looked up the issue on Google and found nothing. I was convinced it was a timing issue and at that point I Skyped my friend Peter Oakes in Canada – just as with Aidan who you’ll have read about in here, I often find that “two heads are better than one” when I’m getting bogged down.

We started sending I2c strings to the Arduino who’s receive buffer I’d put a Serial.println() statement in – to see how many characters it THOUGHT it was receiving.

29…. 30… 31… 32… 32   - EH

As I increased the length of string sent from the ESP (using ESP software I2c) to the Arduino (using the WIRE library) at 33 characters the Arduino thought it was getting 32 – at 34 it went to meet it’s maker.

VOILA – the WIRE library clearly has a buffer to store stuff – I did not believe at first that 1. the buffer would be so small and 2. this would result in a crude crash.  I went into the Wire Library (that’s a long story – I have dozens of WIRE libraries and it took a while to find the right one) – updated the buffer size in wire.h and… nothing – made no difference. I introduced an error into the .h file to make sure I had the right one – sure enough  - but no joy on fixing the problem.

I don’t know if Peter or I twigged first but in my ESP SERIAL buffering which I wrote myself, I have a 256 buffer for incoming characters – which fills until it gets a CRLF then transfers that to an output buffer of the same size – so that incoming characters can continue to arrive while processing the buffer. It occurred to us that maybe the Wire library has the same setup – SURE ENOUGH. Not only is there a buffer definition in arduino/hardware/arduino/avr/libraries/wire/src/wire.h but also in arduino/hardware/arduino/avr/libraries/wire/src/utility/twi.h

And yes, the directory structure IS that complicated (I’m on Arduino 1.69) – sure enough two separate 32 byte buffers are created.  As I have never seen this covered so I assume I’m the only person in the world who’s ever sent 32 bytes via I2c… but if you’re about to try the same – and you’ll need to if you want to try my peripheral software, then I suggest despite the deep hole this will leave in RAM (all 2K of it) – making both of these 128 bytes.

Since amending and re-compiling – I can now send long strings to the Arduino absolutely to my heart’s content!

{nano:9,7,"how are you today and the weather is nice - NO REALLY it certainly is.",0}

Around 15ms in total – not stunningly fast but fast enough not to interfere with the running of anything.

For more on I2C – see the blog entry “Faster ESP I2C”


Arduino Libraries and ESP8266

DallasSo I’m working on a side project – which may or may not go anywhere – to make a working version of my home control 2016 ESP8266 software in the Arduino environment. 

One popular device is the Dallas DS18b20 temperature sensor, a simple 3 wire device that looks like a transistor, has a single communications line and returns accurate temperature digitally.

Well, I’ve had this running for AGES on my code and can handle both the normal and P-suffix variety – so I was a bit dismayed after wasting an hour and throwing a chip away to find that the Arduino version does not.  Also it seems people are still writing libraries out there which wait for the chip – wasting valuable time – so I thought I’d do a short write-up on my own code.

Another annoyance I discovered was the need to predefine which GPIO pin you’re using for the temperature sensor – making dynamic change impossible. There really is no need for this.

So – how do I get around the delay – simple – I swap things around – instead of priming the unit, waiting and taking a reading, I take a reading and prime the unit – making the first ever reading after power-up a waste of time – and I use a flag to return zero the first time. This means no waiting and hence VERY fast reading. I also check the result and if the DS18b20 fails I try for a DS18b20P.

In my own code I use EASYGPIO which isn’t available on the Arduino version – which means (as I understand it) putting the GPIO pin from output to input means two operations, one to reverse the state of the pin and another to set the output value – but hey, it works.

No libraries needed of any kind though if you were cleverer than I you could dump my routines into a library. Of course nothing is new and some of my code is simply refactored from elsewhere as you’ll see.

Firstly – I’m not going for accuracy here, I’m only interested in temperature to the nearest degree – if you want more you’ll have to make minor changes. I’m also not interested in multiple devices per pin. I am however interested in changing pins without static declarations. Funny enough this all started for the Arduino, got adapted for ESP8266 and ended up back in an ESP8266/Arduino setup.

   Adaptation of Paul Stoffregen's One wire library to the ESP8266 and
   Necromant's Frankenstein firmware by Erland Lewin <>

   Paul's original library site:

   See also

   Stripped down to bare minimum by Peter Scargill for single DS18B20 or DS18B20P integer read

// Perform the onewire reset function.  We will wait up to 250uS for
// the bus to come high, if it doesn't then it is broken or shorted
// and we return;

void ds_reset(void)
  uint8_t retries = 125;

  pinMode(temperaturePort, INPUT_PULLUP);
  // wait until the wire is high... just in case
  do {
    if (--retries == 0) return;
  } while (!digitalRead(temperaturePort));
  pinMode(temperaturePort, OUTPUT);
  pinMode(temperaturePort, INPUT_PULLUP);

// Write a bit. Port and bit is used to cut lookup time and provide
// more certain timing.
static inline void write_bit(int v)
  pinMode(temperaturePort, OUTPUT);
  if (v) {
  else {

// Read a bit. Port and bit is used to cut lookup time and provide
// more certain timing.
static inline int read_bit(void)
  int r;
  pinMode(temperaturePort, OUTPUT);
  pinMode(temperaturePort, INPUT_PULLUP);
  r = digitalRead(temperaturePort);
  return r;

// Write a byte. The writing code uses the active drivers to raise the
// pin high, if you need power after the write (e.g. DS18S20 in
// parasite power mode) then set 'power' to 1, otherwise the pin will
// go tri-state at the end of the write to avoid heating in a short or
// other mishap.
void ds_write(uint8_t v, int power)
  uint8_t bitMask;

  for (bitMask = 0x01; bitMask; bitMask <<= 1) {
    write_bit((bitMask & v) ? 1 : 0);
  if (!power) {
    pinMode(temperaturePort, INPUT_PULLUP);
    pinMode(temperaturePort, OUTPUT);

// Read a byte
uint8_t ds_read()
  uint8_t bitMask;
  uint8_t r = 0;

  for (bitMask = 0x01; bitMask; bitMask <<= 1) {
    if (read_bit()) r |= bitMask;
  return r;

So what you see above are the basic routines for talking to the two  chip variations. Here’s the actual code I call once a minute (you could use any interval) to store the temperature. I don’t stop interrupts as I might be running an RGB fader and I don’t want any flashing.

Doesn’t seem to present issues, never has, but if you wanted you could average temperature over time.


I run my heating without any of that and I don’t get glitches on my Home Control software – this is the same code but with changes for the Arduino-style port handling.  I wonder if I’m missing a single instruction to set an input to an output and make it zero at the same time?

   ds_write(0xcc, 1);
   ds_write(0xbe, 1);
   temperature = (int) ds_read();
   temperature = temperature + (int) ds_read() * 256;
   temperature /= 16;
   if (temperature > 100) temperature -= 4096;
   ds_write(0xcc, 1);
   ds_write(0x44, 1);
   if (gotDsReading == 0) { gotDsReading=1; temperature = 0; }    // ignore first reading
    Serial.printf(“Here’s your temperature %dc”,temperature);

It would be nice to see this in a stand-alone library somehow. temperaturePort is the number of the GPIO pin. I generally use 2 or 14 on the ESP and I don’t want them dedicated in case I want to use them for something else. Define gotDSReading and make it zero on power-up or change of pin. If the dummy value of zero is no good – change it.

Oh and in the process of putting this together I realised a slight silly in my main Home Control software – and that is now fixed….

Works for me!!


I2C Continuum

Updated August 07. 2016: This article which started off discussing the “new addition” of i2c to the home control software, is now NOT ONLY about an I2c  2/4 line LCD display facility recently added to the ESP8266 boards – but there’s a PARALLEL version as well, making use of GPIO 4,5,12,13,15 and 16. And NOW – I’m adding a pretty crude interface for the Seeeed OLED displays with other OLEDs to follow.

Comments below about format apply equally to the I2c and parallel versions – for the latter, simply use device 255. Note when you do that you lose GPIO13 as an indicator automatically until next power-up.

Success with I2c – once I realised there was something up at the Arduino end which I could work around, my ESP8266 I2c endeavours have been coming on in leaps and bounds!

And here is my test rig – or one of them – I have one for I2c display and another for parallel display  - the latter is just as messy and has more wires:


So what you’re looking at there from top to bottom – is a prototyping ESP8266 board (the author will recognise it) just because I had it handy – ESP-12 based, running my rapidly developing software (which now has loads of spare RAM thanks to the SDK 2.0). I put a simple set of i2c commands in there then promptly realised that a series of commands sent by MQTT would not be a lot of use. I then proceeded to waste the day, having already talked successfully to an 8-bit port expander, trying to get one of those back-of-an-LCD I2C boards to work – as it was based on the same chip.

Parallel versionAfter several hours of considering taking up brick-laying I realised the damned thing was bust – and went off in search of my  cheap Chinese port expander – that’s the red thing you see in the middle. The only difference is this does not have a transistor to power an LCD backlight so I just hardwired that. At the bottom – the original test 4-line LCD. Over on the right, an irrelevant Arduino-type board which is also talking I2c successfully.

Having decided I liked the idea of plugging one of the cheap LCDs onto the odd home control board for information purposes, I set about doing something I’ve never done before – reading the Hitachi data sheet. These things are quite cute once you get over starting in 8-bit mode then switching to 4-bit to save on wires etc… (so in total you only need 6 signals to talk to the LCD) and after some timing experimenting (clear screen command takes a while – over 1.5ms) as you can see I finally have a working LCD – and by the look of it, rock-solid reliable.

So what is driving that display (which is updating every second virtually instantly)…



and inside that inject?

Topic: freddy/toesp

Payload: {hitachi:39,"$1MQTT test$2$i$3Time $t$4Date $d"}

That’s all. So assuming you’re familiar with MQTT (if not – look at other articled in here) the destination is the little board “freddy” – and the new command I’ve just added – “hitachi” talks to an i2c board (the expander) device #39 and sends out that string.

Rather than have a boatload of commands which would be unwieldy in the home control setup – or special characters which might cause issues – I used the dollar as an escape character. Here’s what I’ve implemented up to now

  1. $$ -  well that shows a dollar on the screen !!
  2. $s -  setup – sets up a virgin LCD and clears the screen
  3. $c  - clear the screen
  4. $1 – set the cursor to line 1 -  ($2 $3 $4)
  5. $t  -  fire out the time
  6. $d – fire out the date
  7. $i  -  fire out the current ip address

I’ll probably settle on a 2-liner as they are sub-£2 from China – indeed for 20-off just over £1 – and at £1.34 for the port expander – around £3 to add a nice little display to some of the ESP8266 boards – can’t be bad. indeed, using one of the port expanders for an ultra-low-cost i2c keypad isn’t a bad idea. 45p for the keyboard, £1.34 for the expander… Sub-£2 keypad.

Clearly one way to wipe a line would be “$1               $1” which is probably fast enough for most purposes. I’ll likely think of some more commands.

A reminder that the right hand side of this blog contains links to the Home Control 2016 project, ROMs etc.

And now I have this big decision.. whether to spend £1.34 on a PCF-base port expander… which I need to poll regularly if attaching a keyboard – and no spare pins for a beeper – OR to spend £1.68 on an Arduino Nano which can be turned into an I2c peripheral (granted I might need 2 pull-up resistors) with keyboard buffering and a beep facility…..   OR…. go the whole hog and use another ESP8266 to make a completely wireless keypad.

OLED from SeeedAnd on that latter note – if you abandoned serial I/O which would net one more output – that gives up to 10 control pins – enough for a keyboard and beeper and light. Hmm…£1.43 for an ESP12, 17 pence for a 0.1” adaptor board.   Fully buffered keypad…. Big decision…

And finally – device 255 is reserved for  a PARALLEL version of this using 6 GPIO pins – 3,4,12,13,15 and 16 – operation is identical. All in the code – including the OTA ROMS. Documented in the WORD manual.

Update: On the right you see a Seeed OLED – I starting with this library, code intended for Arduino, I’ve heavily modified it to run in the normal ESPRESSIF SDK environment – but I have to say – the original Arduino version was SLOW, so VERY slow and this is still a little slow – especially the screen clear – I’ve pulled that into one function with an inner loop of I2c bytes, way faster than the original  but even THEN it is hardly nippy – but the important thing is that “Hello World” works – so soon I’ll add the above commands and then figure out a way to speed it up. Right now I can manage {seeed:”Hello world”} or similar. Current software and ROMs are on the web.


ESP8266 Meets Arduino via I2C

If you’ve been reading the blog regularly you’ll know I added I2c to the ESP8266 code some time ago – that is, the ability to send an I2c message either to read or write – originally intended and still working with cheap I/O expanders – so you lose 2 wires (GPIO4 and 5) and gain another 6 or 14 for one or two I/O expander boards.

Well, I’ve updated it as of tonight.

Of course, ESP12 talking to Arduino is nothing new but given what we’ve built up here with the ESP boards and Node-Red I thought it might be useful to have the option to expand further.

What a few hours ago seems like something in the distance – well, it is done – and it works – and it has LOTS of potential. Read on.

ESP8266 and Arduino and I2c

What you see above (the FTDI is just stuck into the ESP8266 board for power) is a 1284 Aiduino boards flashing lights  - controlled by MQTT – that is, the ESP12 board on the RIGHT is taking in the MQTT and sending commands via I2c to the 1284-based board and to the little red expander in the middle (on the same 2 pins).

Hence, I’ve spent the day working on the I2c code adding the ability to send multiple byte parameters and to receive a byte (I could have made it receive multiple bytes but for simple control a byte will do).

ATMEGA1284The next step was to set up an Arduino – or in my case specifically an ATMEGA1284 chip to talk to the ESP – hence giving us the ability to control Arduino ports via Node-Red.  Now as yet another blog entry here will testify, you can of course connect (now that the node is fixed) an Arduino directly to the Raspberry Pi in charge – but puts too many physical constraints up.  I had no intention of developing Firmata on the ESP – and so a simple i2c protocol was develop and SO much more can be done.

Right now I have an ESP8266  on the bench with a 2-wire I2c connection to an I/O expander – and an Arduino-type board using the 1284. I can talk to both from the ESP (and hence from my Arduino wirelessly via MQTT) and can control the following:

  • Any pin as an output (and PWM outputs where appropriate
  • Any pin as an input (and analog inputs where appropriate)

So first things first – I used  the Atmega1284 simply as I have loads of them – both DIP40 and surface mount in our little AIDUINO boards. The 1284 has twice the RAM of a 2560 and is easy to use. However this would work equally well in a normal Arduino or in the massive 2560 boards.. the only difference being which two pins are used for i2c. The red chart above, shamelessly stolen from ManicBug’s website shows the pins (in parentheses the numbering system I use).

So in the ESP8266 (grab the latest ROMS or SOURCE) I have a simple method of communication which can be used by MQTT or serial. The format is similar to that used elsewhere in the home control system.

i2c:  device, return, param1, param2, param3, param4, param5.

Not all parameters are needed.

So – to talk to an i2c expander sitting as device 39, connected to pins 4 and 5 of the ESP board…


device 39, no return value, send out 3 (which lights up the bottom 2 LEDS.


This returns the value in the port expander.  this is covered in two previous blogs.

So what has changed is that the code can now send out multiple parameters – and optionally receive information back.


The above sends out to device 8, expecting nothing back, type 1 means set a port bit, bit 20 in this case (see red table above) – to 1.


Above – device 8, 1 means return a byte, type 4 means read analog value and port number is 30 – the instruction will return the value of the analog input.

1 is digital out, 2 is digital in, 3 is PWM out, 4 is analog in – you must use appropriate ports – not all are able to handle analog in or PWM out… and you don’t have to worry about port direction setting – this is checked before any operation and set.

There is SO much more potential here but only so many hours.  I will likely make the settings for outputs non-volatile at some point – and add in all manner of device monitoring and probably also make use of the multiple serial ports on these devices.

For now that’s not bad for a Sunday session.  If you’re seriously interested I’ll make the Arduino code available but right now it is highly volatile – but it works!!


Home Control 2016

Introduction (Latest update December 15, 2016)

HomeSick of reading about other people’s home control? Want to make your own? Got some electronics and software experience? You’re going to LOVE THIS! Rock-solid home control – low cost, DIY.

If you’ve read my early blogs you’ll know that Aidan Ruff and I have been working in home control since the late 20th century. Together, we ran an electronics R&D company for many years and one of our products was a home control system called APPCON.

That range of products  achieved much coverage in the UK tech press at the time and many loved it but the design involved spouse-unfriendly WIRES – bad mistake. So now we’re back – this time for fun –  with a vengeance!

Home Control 2016

Add to that lot above, the SI1132 unit which returns light values for visible, UV and IR - and a host of other I2c boards around the corner.

By the time you look at the diagram above it will likely be out of date as new features are added – thankfully there’s a manual along with source code and diagrams available online and kept up to date. This is all free incidentally and without warranty. We have software, diagrams, board layouts… and this blog has a range of articles on the subject.

The diagram pretty much says it all – a Raspberry Pi (2 or 3) or similar,  talking to your PC or smartphone via a range of potential technologies – and using a powerful communications protocol called MQTT over WIFI to talk to small yet incredibly powerful ESP8266 units – these can be bought very cheaply and programmed – or you can take our board designs and have some boards made in China if you really want to DIY (we do!).

There are no real limits to the number of boards you can have talking to the central controller – that is down, in the main, to your WIFI network which could cover one room or extend across the globe.

In my own system I have 3 properties talking to each other – one in the UK, one in Spain. No matter where I am I have full access – great fun, great utility, low cost. Who wants to pay £50 to control a light! This is definitely aimed at the DIYer.

In the coming link there is a complete project, suitable for Windows development using the ESP8266 “Unoffical Development Environment” for the ESP8266 – but that won’t stop Linux users from bending the makefile to their will!!

There is a DOC file here which is fully up to date with all commands – this blog entry just shows a selection. Even if you ignore or don’t understand the source and just download the ROMS available – you’ll be able to keep up to date via OTA (over the air updating) – and you’ll need that manual!

You may also need some new – and very useful skills as you’ll see the term Node-Red appear very often throughout the considerable range of blog entries in here – this is by NO means a single item – there’s a Nextion-based, prize-winning WIFI touch display blog and a range of discussions (and there are LOTS of valuable comments) throughout the blog – enjoy learning and please do feel free to share knowledge – there is SO much more we can do with this.

Update on flashing ROMS here:  - Blog reader Jay has written a great blog on setting up the programming environment for Windows.

ESP8266-terminalWhat is missing from the diagram above is my first experiments with SPI. Using up GPIO13 to 16, I have a 320*240 display working experimentally as a fast scrolling terminal on the ESP8266. Handy for debugging the system without tying up your PC monitor.

How useful this is depends on how many other pins you need! We need an ESP8266 with many more IO pins – I guess that will be the ESP-32…

Background: I’ve been using ESP8266 chips since they first came out – and I have the development kit for their new ESP-32 –just waiting for some decent software tools to emerge but right now the ESP8266 is a lot of fun and that’s part of what keeps this all going.  In the summer of 2015 I made myself so visible on the ESP8266 front that I was invited to be part of an Espressif-led delegation to MIT in Boston for the FAB11 Conference – demonstrating the ESP8266 with MQTT and Node-Red and how to build this into practical systems. I’ve also recently been involved in a project in Spain for the MOD researching ESP8266 and IOT security and more.  If you wish, follow my ramblings on the subject of control using these chips in THIS blog – and also on Twitter (@scargill), Facebook (esp8266wifi and IOT pages) and LinkedIn.

Nextion DisplayRegular readers will know that for some time now, the ESP8266 boards have been turning the world of IOT upside-down. I’ve long-since scrapped various radio designs (NRF24L01 etc.), blighted by short range or lack of network ability - and have gone “hell for leather” into using these boards.

Home ControlThe original plan involved an Arduino Mega as a “master controller” and Arduino “slaves running over an NRF24L01 mesh network. Here in this link is one of my first attempts at blogging on the subject – along with pictures of the original “Appcon” system from back in the ‘90s.

The original Arduino-based system worked well worked and indeed controlled heating and lighting for a couple of years at my home, but the controller went out of the window when the Raspberry Pi 2 came out which was dirt cheap but with more than enough power to control a house or ten. Now of course we have the RPI3 and other exciting boards such as the tiny FriendlyArm NanoPi Neo.

Hardwired to Ethernet (or WIFI if you prefer), the system allows for a comprehensive wireless home control setup  and to answer any questions about ESP8266 reliability – with the code you’ll see in here,  I can say with a lot of experience, that the ESP8266 units given reliable power, are rock solid – stunning considering the price.

ESP-12The ESP8266 boards provide a REALLY low-cost entry solution into the world of IOT making it possible to create in the simplest sense an Internet-controlled LED for under £2 (it could just as easily be a 2KW heater of course for a little more outlay).

There are a number of variations on the ESP8266 boards, the first shot at this being the ESP-01 – a simple board with power, a couple of IO lines and serial in and out. The thing is – they all use the same chip and that particular board (the ESP-01) is very I/O and FLASH limited – and for no good reason – the ESP-12 and variants for example have access to the A/D converter, several IO lines, have more FLASH memory and cost the same amount!

The only downside to the latter is 2mm spacing pins but for VERY little you can buy adaptors to 0.1” – or of course use a board that will take the ESP-12 or the ESP-12E (which is the same – but with a few extra essentially useless pins) – same price. There are also a number of ESP-12-based boards today which offer 0.1” spacing. For this reason I’m giving up on supporting the ESP-01 with it’s limited FLASH memory though as you’ll see in another blog entry,  with a steady hand and fine soldering iron it is trivial to replace the old 1MB Flash on the ESP-01 with a cheap 4MB Flash making it more compatible with ESP-12 etc (well, you still have hardly any pins to use).

Oh, here’s a link to the various source files. Here are original Board files designed by Aidan Ruff. No guarantees – not even any guarantee they won’t change or disappear if we find bugs. Since this article was started I’ve added I2c and a software serial port (optional) on GPIO4 and 5 to control Nextion serial displays directly. Instructions here may not be fully up to date but always check the WORD document in the repository. You might also want to look at the Nextion WIFI touch display board and links – I discuss another control board in there as it can be used for general purpose. This board has neither a relay nor power supply (other than 5v to 3v3 conversion) on-board but it’s a better job for general use.

Also check elsewhere in the blog to see what we’ve done with the Itead Studio Sonoff boards and their plug-in-the-wall remote mains socket. All of this stuff works together as a whole or you can cherry-pick.

I’ve added i2c to the ESP8266 software which as you’ll see in the picture at the start of this blog as made a big difference to the number of supported devices. What a roller-coaster this has been an a great way to learn all about i2c – I’ve improved the original Espressif offering a lot!  I’ve recently added version numbering to power up info and a powerful debug command and also, the power-up info indicates which GPIO is currently used for Web setup and which pin for the visual LED or RGB LED indicator.  At the time of writing the code is using the latest Espressif SDK version 2.0

tmpA4AFThe later PCB you see here was designed (by Aidan Ruff) for my award-winning (ok I had to get that in) Nextion Serial Terminal which  in April 2016 grabbed second prize at the 1st Annual awards. The latest update to this board includes the option to use an RGB LED as the status indicator – something I see as becoming the standard – once you have a selection of colours an intensities to indicate status, all of a sudden a normal LED looks boring.

The Boards

For the reasons stated I’ll discuss here the ESP-12 (and variations, ESP-12F being the latest with a marginally better antenna) as that is my favourite sub-module - but the same code applies to other variations where pins allow. We have working software – and circuit boards to go with it. For reasons best known to my pal Aidan we called this original the Hackitt & Bodgitt board. We have a new board version with an RGB LED as the status indicator on GPIO13.

If you are in any way unhappy using 240v AC mains power, you may want to give this a miss or fit one of those 12v to 5v down-converters so cheap and popular on EBay – I do this to power them from a solar-charged deep-discharge battery in some cases. This version does NOT tie GPIO16 to reset (the saving in power on this board wasn't really worth the bother – and note that GPIO16 is usable as an output). Again in the latest version we’re moving to using GPIO16 as the default relay output – freeing up GPIO0 to be an input only (programming – and web-setup).

Front view

Here is the rear view. Note the MOSFETS on the back (this is looking straight through hence letter reversal)


Note that a particularly nice feature of this board is that it can be programmed from a standard, cheap 5v FTDI.

So in terms of “Features”, the diagram at the start of this blog entry says a lot – but for clarity:

  • Mains-powered WIFI controller board
  • FTDI-compatibility
  • Accommodation for solid state or mechanical relay output
  • Temperature sensing using DS18B20
  • Temperature and pressure sensing using the BMP280
  • Temperature, pressure and humidity sensing via the BME280
  • Temperature and humidity using the DHT-22 or DHT-11
  • All signals brought out to edge connectors
  • Uses inexpensive power supply (or DC/DC converter) available on EBay
  • De-bounced input with automatic debounce and messaging
  • Analog in for checking battery state etc., will handle up to 20v
  • Several Outputs expandable via I2c
  • RGB WS2812b output on most pins able to handle 300 or more serial LED STRIP
  • 3 outputs for high definition PWM – with MOSFETS to control LED STRIP
  • Talks WIFI and MQTT - ideal for being controlled by a Raspberry Pi or similar using, say, Node-Red
  • Flashing status (software supports both simple LED and serial RGB LED)  indication that the unit is functioning and has the correct time which it maintains internally in between updates*
  • Second serial output to support Nextion displays (see the Nextion WIFI Touch Display project)
  • Software mono-stable out for use as a watchdog
  • Outputs can be timers for watering systems for example
  • OTA updating from internal or external website
  • I2C support for an ever-increasing range of devices
  • Hitachi LCD and Seeed OLED support
  • 16-channel 12-bit PWM support via I2c
  • 4-channel 16-bit ADC support via I2c
  • A new NANO-based peripheral via I2c
  • A new SPi-based experimental terminal
  • and more to come…

* (The time comes from an MQTT broker, for example using a Raspberry Pi or similar running Node-Red – see below)

** I now use Node-Red on the Pi to do thermostatic control. Consider the thermostat code to be legacy.

And another board: we’ve also put together a board for the Nextion displays - but WELL suited as a general purpose board and it is detailed on the WIFI Nextion  blog entry but I’ve included a picture here for reference. I can see a variation on this one being my main board eventually – just needs an add-on with a couple of relays and 3 MOSFETs to make it a perfect development board.  There is a cap on the reset line so that typically an FTDI is able to reset it automatically. Note: The current revision of the board has extra grounds and  4 way sensor connector (DS18B20, DHT22 etc.). Note that an even later version optionally uses an RGB LED for a status indicator – WAY better idea.

Nextion control board

The Controller

ESP-12We have chosen to commit to a communications protocol called MQTT as reliable code for this is freely available, MQTT “brokers” are also freely available and once you spend a little time on the subject it is both easy and the obvious way to control stuff in a network. Lookup “tuanpm mqtt” on Google.

As MQTT is so central to this, I’ll go into this in a little depth.  There are many ways you could have units talking to each other – a mesh network, a radio network, polling etc…over time I have come to see MQTT as the ideal tool for home control because it is simple, the tools are free and it WORKS.

Essentially an MQTT broker is a piece of software – a server if you like,  available online or (my preference) on the likes of a Raspberry Pi inside your network) which monitors incoming messages and sends messages out according to who they are for. Addresses are simply text. Messages are simply text. MQTT clients (software that receives MQTT messages for one unit and sends messages out to an MQTT broker) are available freely for the likes of Arduino, ESP8266 and in Node-Red. It really is all very simple once you get playing. So MQTT is an incredibly simple yet powerful format comprising a message “topic” and a message “payload” both in the form of simple text. Units can “subscribe” or listen for specific topics. A great way to play with this is to use an online free MQTT broker – and something like MQTT-SPY which is a free “client” which runs for example on a PC.

The ESP-12-based controller has a number of inputs and outputs which can be controlled equally well by MQTT commands or serial data - by the current software.

To talk to the unit wirelessly (WIFI) via the popular MQTT message protocol, a message can be sent to the topic “toesp” to send to all units in the network or to a specific unit by prefixing the topic with the unit ID, or example “kitchen_light/toesp”. The only difference between these two topics is that the former will go to all units yet never elicit a response from any unit, the latter will go to unit “kitchen_light” only and MAY, depending on the context, elicit a response from unit “kitchen_light” – unit ID is simple, programmable text.

Whatever controls these boards(Raspberry Pi?)  should now send

topic: toesp

payload: {heartbeat}

every minute. All the boards will see this - and all will pre-set a counter in themselves - which if allowed to drop to zero will re-initialise the MQTT code. In the event that boards lose the WIFI for some time, they may reset their WIFI  - then reset the MQTT code. Reset is used only as a last resort and the software can handle up to two SSIDs in case you happen to have two access points. If one fails it will try the other.

Hence in theory they should be bullet-proof. In practice they are bullet-proof unless of course the power goes off. Last summer in Spain I endured horrendous WIFI and power issues and the current code stood up to this and has done ever since – learned the hard way. That experience is also why I worked hard to ensure the OTA works so I can update the units remotely.

Each COMMAND or payload starts with an open brace and ends with a closing brace (my choice, kind of like JSON). In the case of MQTT, this information is stored in the message “payload”. For serial, there is no “topic” – just the message (payload) which you can fire into the serial port at 115k baud.

An example of a simple command might be to turn output 4 ON (output 4 goes to pin GPIO4).


It is possible to send multiple commands at once by separating them with semicolons. i.e.


So to use serial – this is all that is required (newline delimiter) – for MQTT – the topic should be as referred to earlier and the payload should be in the format shown just above.

clip_image002[4]For more on MQTT go here.

I chose to use a Raspberry Pi 2 (you could use 3 but 2 is ok) as my controller – and chose to use Node-Red as the controlling software when this was a project unknown to most  – turns out I picked a winner as Node-Red is now embedded in “Raspbian” on the Pi and is hence guaranteed a well-deserved and bright IOT future. It is also wonderful and really NOT hard to use. Want to play with Node-Red without even doing an install? – enjoy.

The following document details the commands the units will respond to – the actions which will occur along with any return serial message or MQTT message created as a result of the incoming command. But please – read the manual.

The Pins

Here we see a drawing of the ESP-12 board, please note there is a question mark over the positioning of GPIO-4 and GPIO-5 – on many units, these are reversed.

You are looking at the FRONT of the unit with the metal cover visible.


Note: Contrary to the diagram, in prototyping I’m using a 1k to pull GPIO15 down as it’s still a grey area (some say the board won’t program unless GPIO15 is held down) – it does NOT have to be fully grounded and as you’ll see in fact I added PWM output on it (March 26, 2015).


The ESP-12 units operate on 3v3 – NOT 5v. Extensive experience suggests a good 3v3 linear regulator fed from, say a 5v supply provides more reliable results than a 3v3 switched-mode supply – but that will vary with suppliers. I’ve recently been part of a project where we used WELL over 100 ESP-01 units all attached to one 20 amp switched mode power supply – they worked… but I prefer per-unit linear regulation.

We done just that in all our own boards, using a switched-mode 5v supply (or DC/DC convertor for 12v use) with a 3v3 linear regulator. I suspect most complaints about these boards boil down to power supplies. I can honestly say – that having used countless ESP boards since they came out – I’ve not once come across a bad board!!

To talk to these units from any 5v system through serial, you should ensure that inputs to the ESP-12 never exceed 3v3 (resistive divider). Typically the serial output will work just fine fed into, say the serial input of a 5v Arduino – I have NEVER had any trouble with this at 115k baud. Simple resistive level conversion can easily be implemented so that a 5v FTDI will talk to the serial without issue. In practice I’ve yet to see a 5v FTDI destroy an ESP8266 but if you go without the divider, be it on your head.

Certain pins are special and in particular, grounding GPIO-0 on power-up is used to trigger “flash” programming of the chips and also used to put the units into access point mode for setup – so you should be aware of this. When designing outputs I base all of mine on Vcc. So for example when driving a LED…


Why do I do that? Well, when the unit powers up – GPIOs are all input – and if you preset them to “1” on power up you’ll not get any flashing lights as the units boot up.

Below describes SOME of the commands currently implemented.  We have both WEB-based and SERIAL setup, full MQTT integration and the development focuses on using NODE-RED – specifically but not exclusively on Raspberry Pi (I’ve installed Node-Red on many SBCs and consider it as standard as adding Apache and PHP.  We’re using the free and excellent Mosquito for the MQTT “broker” on the Pi (i.e all units connect to this “broker” by address and password” and the broker stores and releases on request stored messages – hence any unit can talk to any other unit as well as making requests of the Pi).

The manual that comes on BitBucket with the code is way more comprehensive than this blog in terms of commands.

Command set

Controlling and monitoring outputs

The potential outputs are GPIO0 (that’s the one on the board with the relay – future boards will not use GPIO0), GPIO4, GPIO5, GPIO12[1]  GPIO13[2] and GPIO16. These are referred to as OUT0[3], OUT4, OUT5, OUT12 OUT13 and OUT16. You can also use GPIO2 as an output via a command – normally it is used with the likes of temperature sensors.

Typical outputs respond to the commands {outX:1} , {outX:0} etc where X is the GPIO number. If requesting the status of an output this form is used {out0?}

There are 3 variations on this:

{out13} (GPIO13) by default should not be used as its main purpose is that of a LED “status” indicator (though you can override that) (see the relevant command).

{out0:X} responds to the following for X:

0 – off (if command “invert=1” then the state of that output is inverted (for positive-based relays) – see the invert command…

1 – on (see above)

2 – on from dusk till midnight (see above)

3 – on from dusk till dawn (see above)

4 – on from dawn till dusk (see above)

5 – under local thermostatic control – see relevant command (see above)

6 – timer for output 0. Another parameter determines ON time in minutes – i.e. {out0,6,100} sets the output on for 100 minutes.

There are a set of related commands for reading the state of outputs – these are {out1?} {out2?} etc. Should unit “kitchen_light”  be asked for the state of out1 – the MQTT message sent back will be in the form

topic: kitchen_light/fromesp/out0

and the payload will contain the appropriate value.

Outputs can be inverted by the invert command by setting bits 0,1 and 2  etc., in the invert command. See the manual.


We’ve added PWM onto 4,5 and GPIO15 using the latest Espressif library which allows for up to around 95% duty cycle at very high resolution (14 bits) for the cheap 12v serial RGB LED STRIP though we have brought that down to 100 smoothly transitioning levels – but decent ones (i.e. non-linear – with more emphasis at the bottom end for smooth transition). Attached to a logic-level control MOSFET you can drive a 12v LED strip with this – initially off, use the command {pwm:X,Y,Z,I} where X,Y,Z are 0-99 for duty cycle and I is false if you want instant transition, or any value if you want a smooth transition. For example if I =50 (50ms) then 0-99 would take 5 seconds. A lookup table is used to get a reasonably smooth fade (needs work).  Due to the Espressif library, you can get up to around 90% or so duty cycle at value 99 – but you really would not notice the extra on a light. Note also that the PWM even when off i still running – so if you run PWM with a board, don’t expect reliable SERIAL LED operation as the PWM has a minor effect on really critical timing as needed for serial LEDS.

Heating controls

In controlling heating, the unit will generally use the internally connectable (on GPIO2) temperature sensor DHT22 (which also handles humidity) or the Dallas DS18b20 or DS18b20p. These heating commands are deprecated as it is a lot easier to do heating controls in Node-Red.

It is possible right now to set the maximum temperature, the fall-back temperature, frost temperature and 2 on-off times per day locally. Of course using an external MQTT control the possibilities are endless.

Relevant commands are (X is general a value, degrees C or minute after midnight):








Time settings

The units are able to keep time but clearly do not have a source of time – this can be provided to each unit separately or more sensibly to all units at once.

{time:X} standard Unix time value the number of seconds since January 1970

{dawn:X } number of minutes past midnight considered “dawn” or street lights off time

{dusk:X} number of minutes past midnight considered “dusk” or light up time

{When they power up, units attempt to log on. This is an ideal time to send them the time.


The internal analog to digital convertor reads from 0 to 1.024 volts and outputs a signal accordingly. This can be read by the {adc?} command.


From the serial console, we can view various settings by using the command {debug}.

Temperature and Humidity

The relevant commands here are {temperature?} and {humidity?} They will send out, by return, for example in the case of unit 999 :

Topic: 999/fromesp/temperature

Data: X where X is in degrees C

Humidity follows the same format but is a percentage.


If we want the unit not to respond to a command but to pass everything on to the serial line to control, say an Arduino, use the EXT command followed by a string  – this can be used part way through a multi-line command if required.


The two outputs will be switched and the command {out0:1} will be send out of the serial port.

Of course, one COULD send these commands off to an Arduino – OR you could send them off to another ESP-12 with it’s WIFI disabled… the logic of that? It’s cheaper than an Arduino and faster!! You can develop in the C language for the ESP boards using the unofficial Eclipse development environment.

and you can learn more about my own work at


Setting up for WIFI

There are commands to set the unit ID (defaults to “999” note this is a string not a number), ssid, ssid2, pass, pass2, mqtt_host, mqtt_port, mqtt_user and mqtt_pass. In each case surround the argument with double-quotes.



You can also set “desc” for a short description and “attribute” for any short message like “Dallas” to indicate what facilities the board has.

The above can also be set by WIFI.  Turn the device on and immediately short GPIO0 to ground AFTER power up for several seconds. This will put the unit in to web setup mode. Have your laptop, tablet or phone browser log into Hack-setup and go to to set everything up.

There are two SSID and passwords – the idea being that if the unit fails to get through to an access point it will try the second one, if that fails it will go back to trying the first one etc.


If for any reason we want to reset the device, the {reset} command will do that.


GPIO14 is an input. We can read the state with the {in14?} command.

It is also de-bounced and you can read (and simultaneously reset) the count using the in14_count? command. The inbounce:X command will let you set the bounce value in milliseconds.

When the state of the input changes (de-bounced) an MQTT message will be send out in the form of TOPIC: XXX/fromesp/in14 where XXX is the unit ID and  MESSAGE: 1 or 0

There is also a bounce value for in2 – i.e. {in2_count?}

Manual Override

If we wish to manually over-ride the status of GPIO-0 – for example wall control of a light – set {override:1} to do this – the input can then be attached to an on-off switch and will over-ride the output for a length of time in minutes determined by “override_time”.


One of the more useful features of the board is to be able to directly control an array of WS2812B LEDs. Right now, overall control is implemented including programmable fade-in and fade-out and a “rainbow” effect. However, buffer space is available to implement full individual control of up to 900 LEDs (warning use separate 5v supply for these – current can be up to 5amps for such a run). Note you cannot currently run PWM and RGB LEDS at the same time. Initiating PWM starts a timer that disrupts RGB timing and a reboot is needed between these. I’ve contacted Espressif about this.

The command takes the form of:

{rgb: port,red,green,blue,number,duration}

So if we want to go to bright red over 10 seconds and you’ve attached an array of 20 serial LEDs to the output GPI12, the command is:


And that will happen and we can do other things while it is fading in. The duration is a 32 bit number.

For amusement purposes we can also try {rainbow: port,number_of_leds,duration[,timer speed in ms]}

A neat toy: requires 60 RGB LEDs and can run on a port pin… a clock… {clock: port} if is 255, the clock is not running else runs on the relevant port.

If you want to just light up SOME LEDs in a strip – the command {rgbset: start_led, number, r, g, b} will let you set up various LEDS to various colours and the command {rgbgo: port,number_of_leds} will start the ball rolling. That number should not exceed 300 (not for any reason – I could have made it larger).

If you want to play with animation… {rgbstart: port,number_of_leds} will clear a command buffer so you can interactively add sequences {rgbadd: start_led,number_of_leds,r,g,b,milliseconds} and when you want it all to end, {rgbstop}. These commands can be sent via serial of MQTT.


When the board powers up it sends a message “esplogin” and in the body the the name if its ID with a slash after it.. so “999/”. This can be read to send the board, say, the time. See the next section.

I’ve implemented {sleep:x} where x=microseconds (32 bit number) hence 10000000 is 10 seconds. You need GPIO16 attached to reset and that just isn’t worth the saving on this particular board but it’s in there for completeness. This is deprecated.


BlynkOriginally we’d planned on using something like OpenHab for home control and of course anyone armed with the software can do just that. But for our purposes we like Node-Red as it is extremely flexible and becoming more powerful by the day. If you have a Raspberry Pi2 or better – you can run Node-Red on that but it also runs on Linux, PCs etc.

Node-Red allows one to “program” using visual boxes via a web interface– so an MQTT input box can respond to a particular topic, you can then do something with that and send a message back – or elsewhere – or maybe send a TWEET, or store some info in a database or send an email – honestly it is VERY simple to use and very powerful once you get to grips with it.

If you look at my earlier blogs I’ve written about graphing locally, using data pulled in by Node-Red and stored in a MySQL database with ease. There are alternatives – I’ve written a “node” for Node-Red to let you simply send data to and others have written nodes for the likes of EmonCMS which has some very pretty dials.

Here’s a screenshot of my Emoncms data and yes, the battery is really flat – due to this being solar charged out in Spain – and it is 6am in the morning – clearly yesterday wasn’t that sunny.

EmonCMS used at Bedrock in Spain

Node-Red has a timing module… when lets you trigger things at certain times. It also has available a suncalc which sets a flag when it is sunset. I’ve improved this significantly in a node called BIGTIMER (see -  check out my bigtimer, esplogin and grove nodes– my contributions – for example – “npm install node-red-contrib-esplogin”) -  to produce an output directly suitable for the current design – to log the boards in and to tell them about the time, sunset and sunrise.

The esplogin node can send time updates when Node-Red powers up and on a regular basis (every 12 hours)to all boards – but also send to a specific board on request, and all boards will happily send that request when they power up – so typically a few seconds after power-up the boards know what time it is and what dusk and dawn times are.

Here’s what it looks like in action.. MQTT coming in from boards in the form of a json payload (purple), esplogin node in yellow doing the time/dusk/dawn updating and outputting the relevant MQTT back to the board (or in the case of time top-ups – all boards). The outputs from esplogin are, in order..

1. MQTT output

2. Output for a MYSQL database (legacy)

3. Output to a straight text logging file

4. Output to a SQLITE database



All boards subscribe to “toesp” – and to a version of that with their name (ID) prefixed – so for example fred/toesp. That way I can talk to any board – or all of them.

When the board first powers up – it sends…  in the case of board “fred” which has description “my board”

Topic: esplogin

Payload: {“id”:”fred”,”desc”:”my board”,”attribute”:””}

Desc and attribute at not that important but can be shoved away in a database.  So that sets off the esplogin node  putting info to it’s outputs – I only use the SQLITE output and that ends up in a database – that’s up to you but my setup is in the Raspberry Pi script referred to elsewhere.

So assuming the node gets that login – it will send a bunch of time info back to the ESP board in question – and will then do that every day to all boards. This stops the boards from watchdog resetting and ensures they always have the correct time without needing a real time clock etc.

See the time, dusk and dawn commands – the ESPs can be updated though right now this isn’t necessary as the home control software on the Raspberry Pi is doing all the work.

How to get all the software the easy way

In order to use these boards effectively – you’ll need a Linux setup (a Raspberry Pi 2 or 3 for example) with MQTT, NODE-RED and preferably other stuff like PHP, SQLITE etc. Personally I’m not a Linux lover!!! And so I like things to be easy – so we made a SCRIPT. You can grab a Raspberry Pi(2) and put everything on there. The script works with JESSIE version of Raspbian. I made this script for “Jessie”. Follow the instructions precisely or it will not work. You must be user PI and NOT ROOT. Read more about the script here...  and

I use the Pi2 – after testing many boards this seemed a while ago to be the best for the job IMHO and for reliability use a good SD such as Samsung EVO and to make life really easy. If you have a Pi3 all the better.

For controlling and monitoring – I am currently using the Nextion (serial) touch display boards for the wall and vary between using Blynk Android/IOS app for remote control, the node-red-dsahboard and ImperiHome.  All work well depending on your requirements.

You’ll read about all of this stuff elsewhere in the blog – use the search facility – there is a LOT of info.


[1] GPIO5 (out5) can be manually over-ridden by the input and the “override” command.

[2] GPIO13 (out13) is generally used as an indicator to show that all is well. It activates on receipt of the correct time either serially or via MQTT and will flash briefly once every couple of seconds from there on. This activity can be disabled by a command. Increasingly we’re using a serial RGB LED on this pin to provide more comprehensive status info.

[3] GPIO0 (out0) is special in that it responds to more than “1” and “0” and can be used as part of a heating control system. Ultimately we will scrap using GPIO0 as an output and merely use it for programming and setup. GPIO16 makes a good relay output as it is not much use for other types of output.

Haptic feedback – when the board is being used in “Nextion” mode – we use GPIO12 as a beeper – simply fasten one of those cheap round haptic feedback devices to it and glue that to the back of the board. Also note that with limits (the line buffer is only 80 characters in length) you can concatenate commands so that for example if you want to turn GPIO12 on and get feedback as to the state at the same time {out12:1;out12?}  - note the two payloads are joined together by the semicolon.

But the REAL source of info on commands is in the WORD manual with the project – that is where you go to keep fully up to date.




ESP8266 Remote Access

Notice I titled that remote access as against remote programming – but that’s part of it.

Years ago, having gotten sick and tired of dismantling my home control Arduinos just to upgrade the code, I purchased a couple of small units from China which I’ve just thrown in the bin.   One had a USB plug on it and plugged into the PC, the other had an FTDI output on it. Both had those black 2.4Ghz antenna.

So the purpose of these units was to wirelessly replace an FTDI – or serial programmer, the idea being that instead of bringing the Arduino project back to wherever one programs the devices, you plugged the unit into the project and over the air programmed it.

At the time I was using Arduinos and those CHEAP £1.50 radio modules – the NRF24L01s, the worst radio boards in the world who’s range could be so bad you’d be lucky to get through one wall. I had high hopes for them with a great mesh network called RADIOHEAD, but ultimately their range and lack of ability to read signal strength doomed them.   However, as you can imagine they were dotted all over the house and I thought this little radio pair would solve the issue of programming them without ripping everything up.

It turns out the “radio FTDI” had the same problem – it was based on the NRF24L01.

Fast forward to September 2015. Here I am in Spain with a new setup of ESP-12 modules controlling lights, watering system and a lot more.  I’ve made some improvements to our “home control 2015” software in the ESPs and once again I find myself having to dismantle kit.

Now, I know that is it possible to program over the air – a few guys have tackled this successfully and indeed much of the code is built into the Espressif SDK. Right now my friend Aidan is working on adding this to our code but there are two issues, he’s not had luck up to now and secondly as our code comes to something like 300k you’re never going to be able to add this to the ESP-01 modules (we use ESP-01 sometimes, ESP-12 other times…)  as you need as much spare storage as you have code for OTA programming!

It was with that in mind that last night I dragged out the old FTDI boards to see if I could save myself the effort of dismantling the Pergola lighting system to update it.  Suffice it to say that I managed to get a few bytes programmed before that all came to an end despite several attempts. With around 25 WIFI units of one description or another dotted around the place, the old wireless FDTI (same frequency range) simply could not handle more than a few feet.


I figured it would not hurt to look around and see if someone has alternative wireless solutions – lots of time has passed since I bought that kit.

And so it was that I stumbled upon this. – as it happens it is quite new – essentially what we are looking at is a single unit (yes, that had me going for a while) which will hook into the WIFI system and act like an FTDI – given that you can fire the serial out of your PC over TCP to a port – port 23.   I grabbed the code (the stable 1.04 version) and very excitedly blew it into an ESP-01 unit I had lying around.

Really, nicely done web interface (with remnants of Martin Hanzarov’s code in there) and a really good write-up – indeed there may be enough info in there to help us implement our own FOTO (over the air) updates to our code.

But this has far wider implications – a virtual serial link would also let you monitor the serial line of a remote unit for debugging info – and our units do indeed put out lots of serial debugging info. It would also let you remote program and monitor un-wirelessly enabled Arduino projects.

So the first problem was – I had this unit sitting as on my network and I needed to blast info into port 23 to have it come out as 115K serial.  I have ECLIPSE sitting on the PC running my projects and there’s a line in the MAKE file that says “COM6”.  I have absolutely NO idea how to turn that into a TCP statement if indeed it can be done (FEEL FREE TO START TYPING IF YOU KNOW HOW).

So I turned to a 30 day trial of a PC program that promised to provide a “virtual com port” – and that it did. Serial/IP Redirector let me create a link to port 23 and ship raw data back and forth to COM22 (which doesn’t exist on my PC)- LO AND BEHOLD I set it running, put PUTTY on the PC talking to COM22 and with a real FTDI attached to the little ESP-01 board on real COM6 I set that running on a terminal –at 115K. Utterly flawless raw serial communications – rock solid.

I was getting excited by now. I took my ESP-01 board and cross-connected the serial to an ESP-12 test board. I held GPIO0 to ground and turned the power on, putting the ESP12 into programming mode.

I punched COM22 into the ECLIPSE project and hit the FLASH button on the project. Lights flashed on both units. With a scope attached  to the serial in on the ESP-12 I could see clean 115k data going into my project and… nowt, zilch, nothing.

I was up until the early hours ensuring the baud rate was ok and that my level conversion wasn’t getting in the way – you see all my projects use a resistive divider on the serial input of ESP boards so the whole lot will work just fine with 5v logic on the FTDI and elsewhere.  I wondered if hooking two of these units together with serial cross connected might prove too much but the scope says no – it says I’ve a perfectly reasonable serial signal going into the ESP-12 – but it’s not having it – no responding output whatsoever. Eclipse tries sending out data over and over and eventually gives up just as if there’s nothing connected.

So near and yet so far.

I got up this morning and the unit was still rock-solidly connected to the WIFI and typing a few characters provided perfect results at the other end – but I cannot program ESP units – yet.

Very exciting, yet very frustrating.

Perhaps coffee and a new start to the day will help. BUT I had to bring this to you as the article is really good, the software seems to WORK and there’s a super-bumper version with it’s own OTA updating in the works for larger targets like an ESP-12 (more FLASH) than the ESP-01.

I’m sure this is going to be a winner – just need to get it to work. After all, the ability to add remote programming to any FTDI-enabled project for what, a couple of quid.. well, you just can’t ignore it.

Meanwhile if anyone reading this firstly knows how to set COMX on Eclipse (on Windows) to TCP – do let me know – and if you really think you know what I’m doing wrong here and want to jump in before I figure it out – by all means – you have the keyboard…


A GroveStreams Node Red Node

One of the more obvious things to do with a home control system is to capture information like temperature and humidity – and to graph it.  For example I spent part of my time in the UK and part of it in Spain. When I’m not in Spain it is usually raining in the Northeast of England and so I take great delight in checking the weather back at base in Galera in Andalucía.

There are of course many online systems for logging your data and I have demonstrated elsewhere in this blog doing the job locally on a Raspberry Pi. One of my favourite online services is GroveStreams. They have both paid and free offerings and I have chosen the latter as I’m only logging the information out of interest.

I am using a setup in Spain which is about to be replaced… a series of Arduino type modules with either DS128b20 chips or the combined temperature/humidity devices, the DHT22. These talk by short range radio to a large Arduino (1284-based) which is connected to the Internet. These will soon be replaced by ESP8266 devices connected directly to WIFI and using MQTT as the protocol.

Accordingly I want to interrogate those devices and send off the information somewhere.

There are a number of ways of sending information to GroveStreams via their API – and as far as Node-Red is concerned there is an http response node – you could tie that to a function block but I thought it would be nice to have a dedicated GroveStreams node – and so here it is.

GroveStreams node by Peter Scargill

What you see above is an MQTT listener awaiting a regularly-sent value from a module – and firing it off to GroveStreams. What I like about their service is simplicity. I can tell it the name of a stream and if it does not exist, it will be created automatically.


GroveStreams setup

I made a nice simple dialog box. You can get an API key from your GroveStreams control panel and this is an example (a dummy example). As you can see I’ve given the node a meaningful name, the API key, the name of a “component” and the name of a “stream” under that component – a lot simpler than it looks. All the node needs is an incoming value and that’s it.


In the example below, neither RaspberryPi nor temperature existed initially – I put these in the dialog box, ran Node-Red and magically they appeared in the web interface for GroveStreams.  The API key is what you call a “Feed put API key with auto registration rights”. I won’t go into GroveStreams setup or we’ll be here all night.

And that is all there is to it.

sudo npm install node-red-contrib-grove

or depending how you set up your node-red

sudo npm install -g node-red-contrib-grove

(a quick restart of node-red and you should find the nice purple icon appear)

I hope this helps, if you do manage to get this working on any particular system do let us know what steps you took. Might save someone else some work.

Here’s that temperature graph from the sensor in Spain. GroveStreams does an excellent job of overlaying internal and external temperatures with humidity.