ESP8266 Home Control Update

As the blog entry on using my home control code ROMS for ESP8266 is filling up and of course as much of the information is now dating I thought I’d do a new blog to bring everyone up to speed and move the conversation to this blog entry. The good news is  – the code is now running under SDK 2.0 – i.e. bang up to date at the time of writing.

Firstly – yes, everything works – but the procedure for blowing ROMS has changed ever so slightly. I have updated the binary files to run on SDK 2.0 and updated the RBOOT code I use for OTA (over-the-air updating) to the latest version.

To backtrack a little

I’ve written software for the ESP8266 which has been in operation for some time and is under constant development. It allows the ESP8266 to control things like simple lights, serial LEDs, PWM LEDs and read sensors etc, talking via WIFI and MQTT to whatever central system you have – for example a Raspberry Pi with Node-Red. I’ve written a node to allow endless units to log in and to provide central communications for them. The software is rock solid and runs 24-7.

I use Windows to develop the software using the Cherts unofficial development environment. This provides an IDE using Eclipse for running and compiling C code. It is not necessary to use any of this if you simply FLASH the ROMS which are available – but of course if you do use the IDE you can customise code for your own use.  The projects Home Control 2016  and Nextion WiFi Touch Display both use the same software and are the end result of many years developing home control. Personally I access much of this equipment using Apps like Blynk and more recently Imperihome (see also this article, this and this).  Tools I use in Node-Red include my own BigTimer node.  All of these options were chosen after much deliberation.

So what have I learned and what has changed?

All has been going well until recently when the Espressif SDK for ESP8266 was updated, firstly to 1.54 then very quickly followed by 2.0 and all of a sudden I and other started to have issues starting up ESP8266 boards. This is now resolved incidentally.

Firstly one of my problems was lack of serial output on power-on. Some serial (115Kb) status messages I have on power up, just point-blank refused to show. I figured the code might be starting at the wrong place. But no, a quick port flick at power-up proved that everything was working – but no serial out.

After much debugging, I discovered that there would be no serial output until this function:


had been called. Prior to SDK 1.54 this was NOT the case. Simple enough I put this at the start of my code – I need to ensure that web updating still works but merely moving this function to the start solved all the serial output issues.

The second problem was a tendency for the unit to run away, firing out garbage as if the ESP-12 was broken. It would SEEM that post-SDK-1.53 – it is necessary on initial run, to flash location starting 0x3fc000 with the default file esp_init_data_default.bin which is found in the Espressif development kit.  If this has already been blown (previous installation) then it is not necessary but with a CLEAN FLASH, failure to flash this area results in the continuous rubbish on output. Again this was not the case before SDK 1.54 – I have NEVER flashed that location in the past. Once blown it is not necessary to blow this again for OTA. Still – once warned….

System Data

There are 2 relevant files. A file called blank.bin sits at 0x3fe000 (top of the third MB in a 4MB Flash i.e. ESP-12) and contains default system parameters produced in the SDK.  NOT setting this initially seems to have no effect on my code.

A file called esp_init_data_default.bin sits at 3fc000 and contains default system parameters stored in the SDK. This is a 1K file. This HAS to be blown when first setting up a chip.

What to do

There is a binary file called esp_init_data_default.bin which I have added to the server at – this is in addition to rboot.bin and romx.bin

In the Espressif setup there is a file called ESPTOOL – there are exe versions and .py versions of this floating around. This is what I use to flash the ESP8266 chips – there are others – you may use them – but I can’t VOUCH for them!

In Windows this works for me – you will have to check for yourself.

To completely wipe an ESP12 – including all data and setup etc and assuming COM3 in this case.

esptool.exe  -p COM3 -b 115200 erase_flash

To initially flash the code… assuming you have the 3 files mentioned above…

esptool.exe -p $(COM3) -b 115200 write_flash -fs 32m -ff 80m -fm qio 0x00000 rboot.bin 0x02000 romx.bin   0x3FC000 esp_init_data_default.bin 

Obviously you can change port, speed and file locations. I usually double up on that baud rate but this one is safe.

I’ve tried NODEMCU FIRMARE PROGRAMMER and that seems to work (but this does not totally wipe everything including user data etc.



And so there it is – the source is updated on the web and requires SDK 2.0 or better to run. The ROMS are updated and the new file is up there.

And why?

Why worry about updates indeed? Well, one good reason – is that recently, updates to the SDK have used LESS precious RAM – the kind that programs (not data) run in – so with 2.0 there is more margin for expansion – which means more features and more room for more features!

A complete and utter aside

I don’t use the Arduino setup but as I was thinking about user data, I just checked the Arduino EEPROM library SOURCE for ESP8266 and it would appear that does NOT use this 3-sector trick. When you update the EEPROM (which is actually FLASH on the ESP8266), the sector is wiped (even if you have user data smaller than 4k, a 4k sector is still used) and then updated from RAM – not ideal. I don’t know if that is the case in the actual ESP8266 Arduino environment as I can’t seem to find the actual EEPROM source!


Banana Pi M2

Technically referred to as the BP1-M2+  I received this board in the post this morning. It is SMALL – that is, the same width but just a tad wider than the FriendlyArm M1 and M2.

But there the similarity ends. Firstly this little thing has EMMC, microSD, Bluetooth, WIFI, Wired Ethernet, 2 USB sockets (a tad limiting), 1GHZ Arm quad-core processor, 1GB Ram, 8GB EMMC (which to me is too small but I’m sure people will find it useful – and as for backup… I’d rather use microSD), and it claims to run Android 4.4 smoothly – hence my comment about the EMMC – what use is 8GB for Android??!?!?

The Android version is a little old for me so I thought I’d try the recently updated Raspbian. The blurb SAYS the GPIO is compatible with Raspberry Pi (really? Does it run the Raspberry Pi modules for Node-Red? If so I’ll be impressed).

The downloads and docs are all here which is a big improvement on some others – Orange Pi with hopelessly out-dated images – others with non-working Google Drive images etc.. I got the image no problem for Debian and as I received no documentation with the board, I was grateful for the online version.

I DID notice no audio output other than that on HDMI – which is a bit of a bummer – I like to see a 3.5mm jack for audio. Sadly the manual refers to plugging in a 3.5mm jack for audio – but there is definitely no 3.5mm jack socket on the board.

There is also an IR receiver. I’ve yet to see one of these boards run it out of the box but there IS a reference to this in the manual! – would be awfully handy if this worked in Android for remote controlling stuff.

At this point I just about had the image downloaded and things were going downhill a little.

At the end of the instructions, sure it was obvious you could be now running your operating system – but from WHERE was unclear. I didn’t want to load it into EMMC.

I was encouraged to see WIRINGPI available – but it was not clear if this was a special – if it WAS a special I found myself wondering why they claimed above that the board was PI-image compatible??

THIS page – on copying images – got my interest – up to now in all the boards I’ve tested, the Raspberry Pi is the ONLY board that has a simple to use copy/backup facility that will make duplicate images even on different size SDs!! This would prove to be no good.

SO – first things first… claims about being Raspberry Pi compatible – MYTH (like all the rest) if it were compatible it would run the RPI ROMS and it does NOT – I just tested it – result – nothing.

But on the other hand for the FIRST TIME their package they describe as “Raspbian booted first time – and had an “expand file system” which after a reboot opened up the operating system to the full size of the SD (others – ARE YOU READING THIS!!!). Marvellous.

Not only that but their “Raspbian” which features the Raspberry Pi logo and looks really like a Pi – apart from the monitor overhang which made closing programs difficult – has the latest file backup system that ONLY (as far as I know) the Pi has had up to now – would it work? I took the Raspberry Pi image disk that was supposed to work with the Banana Pi – now defunct as it does not – and used that as the backup.

I booted up “Raspbian” as supplied on the Banana Pi site – and ensured the WIFI worked – it DID (however it only found one access point which compares badly to other systems which find even my neighbour’s access point). It said I had a connection – but a poor one – no more than 12ft away from the access point!!! 2ft away my PC streams movies on that connection!

I plugged in the USB with my microSD in the BP1-M2+ and ran the graphical backup program. All looked well as it found the USB3 drive and started partitioning. “preparing partitions” it said. After what seemed like a similar time to the Raspberry Pi, maybe a pit longer, the software went off to start copying the two partitions, just like the Pi. If I were honest it SEEMED a little slower than the Pi2 but there are so many factors to take in here. It copied partition one and then…

“Could not mount partition” – I have NEVER seen that in a Pi2 or 3 before (and I make live backups all the time)  so I took the chip out and formatted it on a PC – and re-inserted…Once again – “preparing partitions”… I’m sure it took longer than normal… (and remember when I do this normally it is on a system doing all SORTS of jobs with all SORTS of software. This is a simple empty system).

Partition one started to copy – 60%… 70%… 90%… slow.   Not in the same league as Pi3… it stuck at 100% for AGES – I was convinced it was going to fall over… and…

“Could not mount partition”. I tried this three times in total with different SD holders – same result.  Having failed to get anywhere I took the same chip in the same container, put it back into a Raspberry Pi 2 and initiated a backup. This worked PERFECTLY.

I’m sorry guys – this is NOT Raspberry Pi compatible – STOP CLAIMING COMPATIBILITY. the RPI backup program WORKS. This does NOT.

At this point I noted, having received a heatsink with no glue and having written back to ask if it was necessary, that the main ARM chip was running hot enough to cook an egg. Fortunately I found a little heatsink I had lying around and that improved matters.

I wondered if it was worthwhile doing the usual apt-get update/upgrade – and checked to ensure I had a WIFI signal. Sure enough my WIFI was connected – but I could not browse the web or do anything Internet-related. I got that IP address which means – no.   I even tried putting the address in manually – no.

As I was looking at the WIFI – I noted the volume control top right was on mute. I clicked on the slider to adjust it – nothing – would not come out of mute.

With no audio and no WIFI I thought I’d go off on a tangent and try the recommended ARMBIAN. Aside from (again) overscan on my monitor (which works perfectly with a Pi and various other boards) Armbian came up – with a very nice screen and packed full of utilities (but no SD backup). Once again the WIFI would not have it. I plugged in Ethernet and decided to give the video a try – I opened up Firefox – and went to the BBC iPlayer.  Sorry – HTML5 content will not work – you need the FLASH player – and we all know what getting that running is like.

At this point I was ready to give up… but there was one thing left to try.

Android – a particularly old Android 4.x but I figured it might be worth a try. I followed the instructions which unlike any other board I’ve tried did not include blowing an image with Win32DiskManager but instead a piece of converted Chinese software. I tried several times and failed but eventually got a complete, verified image. Put it into the M2, the Banana Pi info came up and then… blank. The instructions said wait a while the first time – I waited 15 minutes – still blank.

Such a promising start, it looked like an RPI, acted like an RPI but… I have to say, disappointed.


I2C Conundrums

Trying to find examples of ESP8266 C I2C is like looking for the proverbial needle. Even the manual doesn’t have examples..

So I’m looking for ideas… I have an ESP8266 talking to an Arduino – both in C – the ESP using SDK 1.53 – the Arduino using the latest Arduino IDE and code.

So I have the I2c code for the ESP acting as a MASTER.

I can send multiple bytes to the Arduino – no problem – and I can get a byte back into the ESP.

Now bearing in mind that [a] my experience with i2c is minimal [b] the Arduino I2c looks completely different to the ESP I2c….

If I return one byte to the ESP – all is fine – works perfectly – if I try two – it doesn’t work – and eventually the ESP reboots – clearly I’m missing something…

So — Arduino…

//function that executes whenever data is requested by master
// this function is registered as an event, see setup()
void requestEvent() {
Wire.write(retParam); // *****

That works – and now the ESP

if (!i2c_master_checkAck())
os_sprintf(strValue, “duff i2c”);
uint8_t a;
a=i2c_master_readByte(); //*******
os_sprintf(strValue, “%d”,a);

That also works… but add another write for the Arduino where I’ve put stars *** and another read in the ESP code where I’ve put stars – and it does not work – one byte comes back in the wrong place – the other is 255 – and usually the ESP dies after a while.
What am I missing? I’d LIKE to be able to accept variable numbers of bytes but I’ll settle for getting past one for now??

Ideas guys?


WIFI Smart Socket

Smart socketIt looks like Itead have done it again – another winner – well that’s all down to the price and I’ll leave you to look that one up…the site says £9.76 for the unit which is reasonable – but I didn’t check postage.

The first modern WIFI wall socket I had was the Orbvibo Smart socket. I could not re-program it but I did manage to get Node-Red to talk to it over a websocket interface. I think it lasted a week before losing the info, then I discovered that Amazon had stopped selling them because of some regulation or other.  Not a nice experience – but they DID look nice.

Smart Socket[6]Similarly – the Itead  unit is good looking (though you would not tell it was theirs based on the one I have – there’s no reference to Itead on there at all and there were no instructions in the (very pretty) box. Probably because this is new (but available). This is an EU socket not UK so if you’re in the UK you’ll need a simple adaptor unless you want to live dangerously.

There is a programming button – which handily goes to GPIO0 on the ESP8266. Again a slight gripe here, once again they’ve Smart Socket[8]used a 1MB FLASH –  fine for their purposes but if ONLY they’d used the same as the ESP-12 (4MB) I could have done OTA – I believe there is other software that might be suitable which fits into the smaller space. However – not that big a deal. When my new 4MB Flash chips turn up I may just replace theirs.

pinsThe Smart Socket is designed to work with Itead’s APP – and if that’s what you want – read no further. It makes for a nice cheap WIFI controlled socket.

In my case of course I wanted it to work with our own software and hence remotely over Node-Red and MQTT – I suspect readers in here will want the same. So – out with the screwdriver – it turns out this was to be incredibly simple, once screw then finger-nails in the edge and it pops open  – there’s a 4-way set of holes which you can just shove wires into – ground, TX, RX and VCC.  I applied the relevant wires from my FTDI (obviously RX to TX etc)  to the holes – ensuring that [a] my finger was on the programming button at the time and [b] my FTDI was set to 3v3 – not 5v (important).  I blew the ROMs into the unit – this is covered elsewhere in the blog and is easy) and lo – one WIFI controlled switch I can control over MQTT etc.

I mean – it was that easy  – I’m missing the bit out about making a  change to my program to slow the flashing light down as the lights in this thing are BRIGHT.  So the green light is the general indicator (GPIO13) which I use all the time (thank heavens) and as in the SONOFF they use GPIO12 for the relay AND a nice blue light – so when the unit is running it is flashing green – if the output is on there is also a blue glow. Very nice.

So – there you have it – another winner.   It’s all down to postal charges really… oh, the relay claims 2KW – I’ll leave it up to someone else to see if it is up to that – I’d suggest that putting a 2KW electric heater on it will probably not work too well. It would be nice if a 1KW heater would work… anyone up to testing that ?


ESP8266 Meets Arduino via I2C

If you’ve been reading the blog regularly you’ll know I added I2c to the ESP8266 code some time ago – that is, the ability to send an I2c message either to read or write – originally intended and still working with cheap I/O expanders – so you lose 2 wires (GPIO4 and 5) and gain another 6 or 14 for one or two I/O expander boards.

Well, I’ve updated it as of tonight.

Of course, ESP12 talking to Arduino is nothing new but given what we’ve built up here with the ESP boards and Node-Red I thought it might be useful to have the option to expand further.

What a few hours ago seems like something in the distance – well, it is done – and it works – and it has LOTS of potential. Read on.

ESP8266 and Arduino and I2c

What you see above (the FTDI is just stuck into the ESP8266 board for power) is a 1284 Aiduino boards flashing lights  – controlled by MQTT – that is, the ESP12 board on the RIGHT is taking in the MQTT and sending commands via I2c to the 1284-based board and to the little red expander in the middle (on the same 2 pins).

Hence, I’ve spent the day working on the I2c code adding the ability to send multiple byte parameters and to receive a byte (I could have made it receive multiple bytes but for simple control a byte will do).

ATMEGA1284The next step was to set up an Arduino – or in my case specifically an ATMEGA1284 chip to talk to the ESP – hence giving us the ability to control Arduino ports via Node-Red.  Now as yet another blog entry here will testify, you can of course connect (now that the node is fixed) an Arduino directly to the Raspberry Pi in charge – but puts too many physical constraints up.  I had no intention of developing Firmata on the ESP – and so a simple i2c protocol was develop and SO much more can be done.

Right now I have an ESP8266  on the bench with a 2-wire I2c connection to an I/O expander – and an Arduino-type board using the 1284. I can talk to both from the ESP (and hence from my Arduino wirelessly via MQTT) and can control the following:

  • Any pin as an output (and PWM outputs where appropriate
  • Any pin as an input (and analog inputs where appropriate)

So first things first – I used  the Atmega1284 simply as I have loads of them – both DIP40 and surface mount in our little AIDUINO boards. The 1284 has twice the RAM of a 2560 and is easy to use. However this would work equally well in a normal Arduino or in the massive 2560 boards.. the only difference being which two pins are used for i2c. The red chart above, shamelessly stolen from ManicBug’s website shows the pins (in parentheses the numbering system I use).

So in the ESP8266 (grab the latest ROMS or SOURCE) I have a simple method of communication which can be used by MQTT or serial. The format is similar to that used elsewhere in the home control system.

i2c:  device, return, param1, param2, param3, param4, param5.

Not all parameters are needed.

So – to talk to an i2c expander sitting as device 39, connected to pins 4 and 5 of the ESP board…


device 39, no return value, send out 3 (which lights up the bottom 2 LEDS.


This returns the value in the port expander.  this is covered in two previous blogs.

So what has changed is that the code can now send out multiple parameters – and optionally receive information back.


The above sends out to device 8, expecting nothing back, type 1 means set a port bit, bit 20 in this case (see red table above) – to 1.


Above – device 8, 1 means return a byte, type 4 means read analog value and port number is 30 – the instruction will return the value of the analog input.

1 is digital out, 2 is digital in, 3 is PWM out, 4 is analog in – you must use appropriate ports – not all are able to handle analog in or PWM out… and you don’t have to worry about port direction setting – this is checked before any operation and set.

There is SO much more potential here but only so many hours.  I will likely make the settings for outputs non-volatile at some point – and add in all manner of device monitoring and probably also make use of the multiple serial ports on these devices.

For now that’s not bad for a Sunday session.  If you’re seriously interested I’ll make the Arduino code available but right now it is highly volatile – but it works!!


Sunday Morning Experimenting

I figured after a couple of horrendous days figuring out what was wrong with my ROMS, it was time for a morning’s relaxing.  Some time ago I added I2c to the Home Control software but never actually got around to doing anything with it.  The i2c parallel expansion modules I got from Ebay have their own pull-up resistors on them… could I get away with hooking 2 boards up without having to dismantle surface-mount pull-ups on on?

test ledsSo first things first I wired up some test LEDs comprising a some AQUA LEDs I had lying around and 120r resistors. I looked all over for pre-made leads with female 0.1” connectors on them but as usual ended up making up my own. If anyone knows of a good cheap source of these – do let us know in here.

Next stop I needed a test board and the obvious one to use was one of the samples I got from Bob Elmour.

A few extra ground, 3v3 and 5v lines (0.1” connector strip) along with GPIO4 and GPIO5 brought out and I was in business. Ok, not the prettiest layout but good enough for testing.

And… yes, I’m not sure I’d want to go beyond two without removing resistors – but they work. With one DIP switch set to ON-ON-ON and the other to ON-ON-OFF I found I could easily use i2c to turn on and off LEDS on both boards at addresses 39 and 38 respectively.

ESP8266 board

It is worth noting that the output DRIVE capability of the little red boards is not that good (as the outputs are also used as inputs with pull-ups) and so to get a decent output I fastened the LED + leads to +3v3 and used the expander boards to SINK current (which means the output logic is inverted).

Using the come control software, lighting up those 4 LEDS is simple.



(The zero is a recent addition and simply says there is nothing being returned. See future blogs for more extensive use involving multiple parameters and an optional return value).

So for the loss of 2 wires, you get 16 outputs using I2c and at a cost for boards of around a fiver. Not entirely sure how well that compares to sticking an ATMEGA24560 onto the ESP and doing some custom software to let you control the outputs (I have thought about this)… but it is certainly one easy way to get more inputs and outputs.


I2C Expansion for Pi and ESP8266

Pi ExpansionIf like me you are not THAT familiar with I2c, you might find the results of my  experiments interesting and perhaps even useful.

PCF8574TSo I bought a couple of these i2c port expanders from China – mainly because I wanted something simple to mess with i2c on the ESP8266. It occurred to me that if I could get these working on a Pi, so I was sure of the addresses and commands etc., then on the ESP, I’d then get the confidence to do something more dramatic on the ESP8266 with i2c.

So ignoring for a minute the interrupt capability, these are pretty basic devices – using up 2 port bits (SDA and SCL) on your Pi or ESP, send an i2c start to them followed by an address then a byte to them – and the outputs light up accordingly. Set the outputs to 255 and read back a byte and you get the state of the pins as inputs. It doesn’t get any easier.

Well, not unless you completely mis-interpret the addressing as I did and spend ages chattering away to the wrong address. Anyway, let’s not dwell on that.

I noted that the outputs are HIGH by default.  Also note that in my experiments I have set the 3 DIP switches to ON (NOT as in the photo above).

Armed with the latest version of Raspbian Jessie on a pi2 or Pi3, connect ground on the device to ground on the Pi, VCC to 3v3 on the Pi, SDA to SDA (blue) on the Pi, SCL to SCL on the Pi. Simples. The boards have built-in pull-up resistors for i2c (which could pose an issue if you parallel a bunch of them up of course) so that’s it – no other new components needed other than a LED for testing. I used a 470r resistor in series with the LED.

Open a terminal on the Pi and type:

sudo pigpiod

That starts the new GPIO library daemon running in the background. 

Now here is a short file that will set all the outputs to 0 – running Python…

import pigpio

pi1 = pigpio.pi() 
pi1.write(18, 0)


Nice but then my pal Peter Oakes pointed out to me that I’d end up loading the entire Python environment  every time I wanted to change an output.. so I started experimenting with C code… just turning GPIO18 (on the Pi) on and off for starters…. see the line that says gpioWrite(18,0);  that turns the port off. Substitute a “1” to get the opposite effect.   All of this worked a treat.. “b” here ends up containing a handle.

#include <pigpio.h>
#include <stdio.h>

void main()
    if (gpioInitialise() < 0)
      puts(“erm no”); // pigpio initialisation failed.
    gpioSetMode(18, PI_OUTPUT);
    // pigpio initialised okay.
    gpioWrite(18, 0);

The code above once compiled failed the first time – I realised you must NOT have the daemon running when using this. so a quick reboot later and I was in business.

Oh, here’s how to compile a simple C program like that – make sure it’s a text file, say in your /home/pi directory.

gcc -Wall -pthread -o prog prog.c -lpigpiod_if2 –lrt

See where it says “prog” – change that to the name of your program. Takes seconds.

Anyway, I was just about to set everything up in C for i2c etc. when I discovered… PIGS

sudo pigpiod
pigs w 18 0
pigs w 18 1

Note – no sudo needed for the commands and presumably one would run that daemon (pigpiod) at startup. This looked like a nice simple route – dead easy for Node-Red as you can just issue the commands in an EXEC function and pass the parameters in the payload – so next would be to try i2c….

pigs i2co 1 39 0 
pigs i2cwb 0 0 0
pigs i2cwb 0 0 255
pigs i2cwb 0 0 1
pigs i2cwb 0 0 2
pigs i2cc 0

The first command visually returned 0 – hence my use of 0 later in the code as the “handle”. I order, I set the expander to all off, all on, then the first bit only on – then the second bit only on and finally I closed the handle.

Something to note is that I2c lines need pull-up resistors – and this board has them already built in – unfortunately they are 1k pullups – fine if you only have one board, not a lot of use if you want to put several in parallel. After discussion we think that possibly the two relevant resistors might be replaced by 10k in which case you could then run several in parallel (with different addresses) but we’ve not tested that.

Oh, making that daemon permanent… I did that with a command line edit “sudo nano /etc/rc.local” –  and added the line “sudo /usr/bin/pigpiod” – and rebooted…. no problem.

The final stage of this experiment gives my ESP8266 software the ability to achieve the same thing, losing 2 wires to get 8 back, a net benefit of 6 I/O lines, could be worthwhile as the ESP8266 isn’t exactly brimming with IO lines.


Above you see (blue) our ESP-12 board, fastened to an FTDI for power, and wired by jumper to one of the PCF8574T boards.

With a typical Chinese PCF8574T board which includes pullups, I’ve added new commands to the ESP8266 Home Control software as of my software version 1.3.9 – the I2C command. 



sets the lowest bit of the expander high (the 0 is a mandatory argument above – see future blogs) whereas:


returns the byte with the state of all 8 input bits. 

Clearly again you need to set a bit high before you can use it as an input. Here is the datasheet for this chip – and here is a typical Chinese expansion board.  With GPIO4 on our little boards hooked to SCL and GPIO5 hooked to SDA – the new commands work a treat.

In the above photo – address 39 equates to all DIP switches set to ON (that’s high or 7). If you set number 3 to off – that is address 38 etc. (simple binary selections – you can make the device work as anything from 32 (all switches OFF) to 39 (all switches ON) but before you go connecting eight of them up – bear in mind the comments about pull-ups above.

Hope you found the above useful. For more information on the ESP software – go to the relevant page on the blog. There is of course the main Home Control 2016 page.


ESP SDK and RBoot Woes

Remember I did an article entitles Serial Woes? Well that’s been changed to serial success as we found bugs in tools and gained an understanding of the USB FTDI handshaking lines – which altogether led to improvements in the Node-Red Arduino node and a completely working solution for USB Serial on the Raspberry Pi.

Well for the last few days I’ve been at the WOES part of a serious issue with our ESP8266 code.   I’m hoping if Espressif see this and Richard of RBoot fame see this and we can get a little interaction – so that what looks like an issue with Espressif SDK 1.5.4 and Rboot can be resolved. I’ll update this as we go along. 

So – those of you who’ve been reading this blog for a while will know that I’ve put a lot of work into developing my own code (along with a lot of help from others) for the ESP8266 to control ESP-12-based boards designed by Aidan Ruff – the software talking via MQTT back to a central controller – this is all working and is regularly updated. The software is BIG (well over 0.5 meg and hence for OTA purposes needs boards with a 4MB FLASH like the ESP-12 though currently it will work without OTA on the likes of the Sonoff controllers) – and is available as source code and .ROM files.

So a few days ago some guys in the blog wrote in to say they were having horrible difficulty flashing my ROMs and indeed that their boards were apparently broken!!!

I could not replicate this behaviour – I would take my ROMS, blow them – and all was well, my code worked. The code uses Richard Burton’s RBOOT to allow for OTA (over the air updating).

And then… I tried blowing the ROMS into a virgin ESP8266 board (and I mean virgin with no software on it – not one stocked with AT software). DEAD. But not just dead. Dead sometimes… and when it worked – it worked kind of.


Here’s what I mean. Look at the first image above – that’s  a normal boot up of my code. “ESP starting…”  some info on the use of GPIOs etc., software version and right at the end, another function is called which outputs “Web page control enabled” (or disabled).

Now look at the code below.. this is the output when I CAN get the code to work with SDK


Do you see what is wrong – the “web page control enabled” message comes up FIRST – you might say that is IMPOSSIBLE – that code does not run until after the other stuff is all done… and there are bits missing – but more often than not the code does not run at all – and that is what pointed me to the problem – and the “solution”.

Here are sequences of events..

  • Start with an existing ESP
  • Flash my code
  • Output probably ok
  • Start with something like the NodeMCU software rom
  • Flash my code
  • Output seems ok but has the above nonsense (but works)
  • Entirely WIPE the FLASH (yes, there is a quick way)
  • Flash the new code
  • Nothing works
  • Load the NodeMCU rom (works mainly)
  • Flash my code
  • Works sometimes – might not work after reboot – but might.

This is cutting a LONG story short – a story of using various programs to try to wipe the 4MBYTES of the ESP-12 only to find in one case that the final meg would not erase – or in another case that no matter what address you put in it was always ONLY the bottom MB that was being erased – that wasted a few hours I can tell you!!!

Finally I hit on this – more by accident than anything else:

C:\Espressif\utils> -p COM3 -b 115200 erase_flash

This took an instant and appears to wipe the entire Flash perfectly.

Even using the official tool would seem to work – but when you watched the addresses go by – it was obvious that only the bottom 1 meg was being erased!!

Wipe the Lot

But I digress.

So when I finally got to analysing the problem (scratching my last hairs out) and realising that something was wrong – maybe a start up vector in RAM not being set properly, I decided to try something – instead of using the latest Espressif SDK 1.541 (which coincidentally I loaded and compiled at about the time people started griping about non-working boards – but of course I was simply OTA-ing my existing, working boards and seeing nothing) – I went back a version to 1.53 – and….

IMMEDIATELY (thank heavens) the problem went away – I could now program the boards in the normal way and they worked (work) perfectly. 

So at there are some working roms using SDK 1.53

rboot.bin and romx.bin

The above work (for example

Here is the same compiled under 1.541

rboot_duff_154.bin  and romx_duff_154.bin

These are the ones that were up there until now – and were giving trouble. I’ve just renamed them.  Easy to test – put the first at 0x00000 and the second at 0x2000 (that’s 0x2000 NOT 0x20000) after running that erase procedure above – and one will work – the other will not.

So – the question is now – is there something wrong with the Espressif SDK 1.541 – or have they moved or changed something that is giving Richard’s RBOOT (which has not changed for a long while and which has proven to work perfectly) trouble. Surely it has to be one or the other.   My impression, rightly or wrongly is that the code is starting up in the wrong place – sometimes in the middle of no-where, sometimes almost in the right place (hence the re-arrangement of output).

I’ve pointed both Richard and Espressif to this as it is possible to replicate the issues quite easily on an ESP-12 (my viewable serial output at powerup is 115k baud after the usual 78k debug info).

Hopefully this will become a shorter success story with a fix somewhere. If it is affecting my code – it has to be affecting other code (and no I’m not running out of variable RAM – thought of that). Right now I simply cannot use SDK 1.541



NanoPi M2

NanoPi M2More from FriendlyArm – this time the NanoPi M2.  I’ve had this board running both Debian and Android – but first – a little about the boards. Like the M1 these are small – around 65mm by 60mm so much smaller than Raspberry Pi. They are also cheaper than the Pi but but this time more powerful than the M1 and most likely the Pi, also.

Unlike the M1, the board comes with a connector for FriendlyArm’s touch-LCD display and also unlike the M1 which comes with an awful Android with no Play Store, this one is a 5.11 version with full playstore.  I’ve tried it with BBC video and it runs flat out M1so I have no doubt this would make a media centre for anyone interested. Indeed I’m waiting for a wireless keyboard/mouse arrangement turning up and one of these might just end up as a media centre.

The board has 2 USBs connectors and it has 0.1” connectors on board for a further two. It has up to 1GBPS Ethernet connection, audio jack, microphone, camera connector, standard HDMI out and a microSD slot on the underside.  Mine came with 1GB RAM. There is also a header for a serial console.

The processor is a Samsung S5P4418 quad core Cortex A9 running between 400Mhz and 1.4Ghz. There is also a Mali GPU. I’ve not yet done speed tests but I’d expect this to give the Pi a run for it’s money – again – how they compare depends on postal charges in your area.

My next step will be to install my usual software script on this board in Debian and we’ll see then how easy or otherwise it is to get port and serial access. Keep looking in.


Raspberry Pi Node Red Serial Delight

Or – how to get USB serial ports working on the Raspberry Pi

Serial SetupThis originally was a cry for help – I wanted a second serial port on the Raspberry Pi AND I made the mistake of trying out Firmata on the Arduino at the same time – resulting collection of failures had me scratching out my few remaining hairs – but thanks to a lot of work – two lots of changes to the Arduino node itself and some reader insights in here, I can now confidently say that:

1. You CAN add more serial ports to your Raspberry Pi

2. You CAN talk from the Pi (or similar) to Arduino gaining access to ports for things like port on/off, PWM etc. with ease.

It all started when I innocently plugged a USB FTDI into the Raspberry Pi USB connector to give me a second serial port to play with as I was using the first one to talk to a Nextion display.  I checked in Debian in the /dev directory and it up came /dev/ttyUSB0. When I checked in Node-Red it came up with the same – magic. Great start.

I added in the serial node to Node-Red – set up an inject – and a debug node.

I shorted TRX and TX on the serial and… nothing.

(Serial set to a decent baud rate with 100ms timeout).

FTDII could see on the FTDI flashing lights indicating that not only was data coming out  – but also going back in.

I decided to test this and hooked up the internal serial on the Raspberry Pi.  I had mistakenly thought that the graphical control in the new Jessie allowed you to turn the serial login terminal on and off – so I turned off serial. BAD mistake – not only did the serial not work but the GPIO node failed as well!  I turned the serial back on – rebooted, back to square one.

I updated /boot/cmdline.text to remove reference to the serial port… and read THIS –  so now I had serial working.

I tested as you see above.  I shorted TX and RX on the internal serial – and sent some text – smashing – the text shows up in the debug.

So now I connected the serial out from the USB serial – to the serial in on the Pi (ensuring 3v3 output) – MAGIC – that worked.

So all that appeared to be NOT working – was the serial in on the USB FTDI serial – yet there are were error messages.

Is this familiar to anyone? I tried various combinations of the RTS and DTR lines – no difference. All looks fine and connected but the USB0 serial in appeared to do nothing.

Stage 2:

I took out the FTDI – put another one in – that appeared as USB1 (yes, the SAME ONE)… but it worked – I took that out and plugged the ORIGINAL back in – now this appeared as USB2  and worked – but how the hell are you supposed to use a port if it keeps renaming itself. Read on.

I rebooted the Pi (NOT removing power, just rebooted) – it went back to being USB0 – it worked.  I turned the power off, them on – no serial in.

So…. I took the FTDI out – and put it back in – now it appeared as USB1 – adjusted the setting on Node-Red accordingly – it worked.  Rebooted…No USB 1 – looked in the node settings – USB0 was back. Adjusted. All was well on USB0.

I turned the power back off then on (for the whole lot)…. Green connected lights on the in and out nodes, all’s well – BUT – no input. I could again see both lights on the FTDI indicating that it is not only sending but pulling back in – but nothing in the debug window when injecting text.

But this had WORKED after a reboot… but not after power up… so – I tried rebooting again…surely it would work? No.

I pulled the FTDI out – and plugged it back in… USB0 disappeared but USB1 was valid…. I changed the node to work with USB1 – it worked.

So it seems that the only situation when the input would not work – was when both the Pi and the FTDI had been powered down and back up. Which of course is every time you turn the stuff off and on!! And the name was changing – which is just useless.

The Solutions:

Well, thanks to readers in here I managed to split this problem down into sections. The first was this issue of the name keep changing. As it happens this one is RELATIVELY simple.

pi@bedrock2:~ $ sudo lsusb -v | more

Bus 001 Device 005: ID 0403:6001 Future Technology Devices International, Ltd FT
232 USB-Serial (UART) IC
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0
  bDeviceProtocol         0
  bMaxPacketSize0         8
  idVendor           0x0403 Future Technology Devices International, Ltd
  idProduct          0x6001 FT232 USB-Serial (UART) IC
  bcdDevice            6.00
  iManufacturer           1 FTDI
  iProduct                2 FT232R USB UART
  iSerial                 3 A50285BI
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           32
    bNumInterfaces          1

So looking at the code above created by running the command at the top above – starting “sudo”…

This told me all about the last USB item plugged in – sure enough my Future technology USB FTDI

I made a note as instructed by reader Alex – of idVentor, idProduct and iSerial and put them into a file:

sudo nano /etc/udev/rules.d/99-usb-serial.rules

The one and only line in that file would be..


Tried it – failed.  Of course I was trying to use vendor 0x0403 – when I took off the 0x it worked a lot better. I’ve no idea why. With further help from Alex the solution appeared as


All of that was great – on power up the unit would come up with my chosen name – also after reboot – and after disconnecting.


All worked well…one problem apparently solved…

But… from time to time I would lose the output – the lights on the FTDI would show up – but nothing would come back in to Node-Red. Another reader – Christopher had suggested that in some cases you need to short DTR and DCD, also RTS and CTS.  if these were floating and if they WERE important – I could see why I’d be getting strange results. I put links on these.

Voila. Reliable serial connection. It would be nice to clarify why the word “serial” and not “iSerial” – and why you need to miss off the “0x” – but it worked reliably.

Pi and Arduino

Though it works perfectly as a serial port in Node-Red – Node-Red cannot see the new port probably because this is just a shortcut or symbol – if you type it in – it worked – but Node-Red could not (and will not)  search for it – not too much of a hassle to put in manually.

But – the ARDUINO node for Node-Red would NOT accept the manual input.

By specifying ttyUSB0 I managed to flash a light with this Firmata-based node – but any attempts to change things around, try PWM on a pin etc. would result in  Node-Red dying temporarily. I talked to the developer of the Firmata node as I noted the serial node NEVER crashed Node-Red.  Result – thank heavens – a bug – now removed.

Reading a reply in the Node-Red Google group, it was suggested for a while NOT to use the Arduino node but to scrap that and use the contrib-gpio node (the yellow one) – as it happened I had that installed. I used my shortcut (may as well go in at the deep end) – and LO – no more Node-Red crashing. 

I tested port digital control as well as PWM and they all worked.. BUT – changing mid-stream – i.e. opening the node and changing the port bit FAILED like the Arduino node – but did not crash Node-Red – this let me experiment a LOT more. I killed the Node-Red service and ran it manually. But at this point, making a change to the node and then running deploy would result in the node sitting with “connecting” and just refusing to connect to the serial port. Simply stopping and starting Node-Red fixed the issue every time.

The author was playing with this and could not replicate this – but it turns out he was using Node-Red’s DEPLOY ALL mode – whereas I just deploy all changed nodes (so my house control doesn’t fall to bits when I’m experimenting). I just had a call to say he can now replicate this.

WHICH MEANS the yellow GPIO node also has this issue! I’ll let them know.

So now that it looks like Firmata will work but does not like change.

Dave at IBM who has been working on the Arduino node has now FIXED this issue in Arduino node version 0.09   (npm install node-red-node-arduino).  I have tried port 23 to make sure the MEGA version of this is running – and sure enough – it is running.

Lovely. Multiple Serial ports on the Pi – and LOTS of GPIO cheap add on – what more could you want. My thanks to Dave for quick action and to everyone for their helpful comments which brought us this far.

The node-red-contrib-gpio node has exactly the same issue that the Arduino node HAD – I’ve left an issue in the repository of the latter.