Category Archives: ESP8266 SDK

ESP8266 Code Update

The code that forms a series of articles on this blog has now not been updated for some time – simply because it “just works” and has been doing so reliably for many months. However, I’ve been wanting to add bits to it and my problem was I was running dangerously low on iRAM. This would not affect the stack or reliability but would just stop me from continuously adding code.

Well, thanks to Espressif that is now history. The latest Esp8266 SDK, released just days ago, fixes a number of things on the ESP8266 – you can find the SDK here – you will need it if you wish to compile code but of course ROMS are available as usual.  The upshot for me is the return of nearly 800 bytes of iRAM which is something of a breath of fresh air.

The software has been updated to version 1.8.5, the code is available here…  and the SDK here.

And if you’re wondering what on EARTH I’m talking about… if you’ve seen “the script” – this is what it was put together for in the first place.

Blowing the home control 2016 code

Home control 2016

Extract from the manual (which is the best source of info).

Home Control

Facebooktwittergoogle_pluspinterestlinkedin

VT100 Terminal for HC2016

rear view ESP8266 VT100Well, I had to give it a go, didn’t I – porting the code for the cheap VT100-type serial terminal into the main ESP8266 home control software.

BOY was that difficult but… after 2 days of head-scratching – check out the home control manual in the source code repository for Home Control 2016 – this is classed as experimental as I’ve not thoroughly checked what interaction it has with other stuff and it uses up lots of precious port bits (GPIO 13,14,15,16).

However I have the terminal code up and running (minus baud rate controls… and the bottom line is now a general status line) and I have to say, fast doesn’t start to explain it.

As you can see in the image above, all we have here is the display with a slim ESP8266 board behind it – the FTDI you see attached to the back is merely there to provide 5v power and, erm, stand the unit up!!!

I need to separate the code out into C files – it was just an experiment about which I did not have high hopes so it is all inline – but as it develops I think it needs it’s own source files.

ESP8266 VT100 front viewI did some tests to see if how fast I could make it -  I’ve already optimised the SPI and character routines – the board will not operate at all under ESP8266 at the fastest SPI speed so that is probably a dead-end, I tried caching the ROM reads (which are 8 bits – meaning you have to read 32 bits at a time and select the right 8 bits.

Caching that info actually very marginally slowed things down – I tried all sorts, writing 16 bits at a time to SPI – and finally after being able to obtain no more speed improvements, I stopped – not sure why I needed any of this as it was already blazingly fast.  Now, writing an X to every location on the screen (that’s 1600 character writes) takes 330ms so that is 200us per character (5*7). I think that is fast enough for now. Clear screen is near enough instant and scrolling is instant.

See this demo video of the ESP8266 version – the 328 version isn’t  quite THIS fast but it is still fast.

https://www.youtube.com/watch?v=YXqLVmoyKPE

So I’ve made 4 commands in the HC2016 project

{ili_set:1}
The above will setup the display and make it available to accept data – once set the display will set itself up on power up of the unit.  Setting that command to 0 stops any data and from there on the display will NOT initialise on powerup.
{ili_text:"\e[35;40mHello there \e[32;40mmate\r\n"}
{ili_status:"All working exceedingly well"}
{ili_title:"\e[37;40mESP8266 Status Monitor"}

I may well change the first one – as it should be automatic on first use… and also It may be there’s a good use for a version without scrolling and the top and bottom lines – but for now that’s it.

Hence by firing serial or MQTT commands at the board, I can get stuff onto the display.  To monitor all MQTT traffic was easy – over to the Pi and MQTT..

In Node-Red – a simple MQTT node picks up all traffic, passes it to a function – which then passes the result to the board which in this case is called “thisone”.

tmpPayload=”  “ + msg.payload;
tmpTopic="
\\e[36;40m"+msg.topic+"\\e[32;40m\r\n";
if (msg.topic!="thisone/toesp")
{
msg.topic="thisone/toesp";
msg.payload="{ili_text:\"" + tmpTopic + tmpPayload + "\r\n\"}";
return msg;
}

DisplayHence the board gets all traffic except for traffic destined for the board (which could end up in an infinite loop).

And now thanks to a conversation with reader Bob Green –  a WIFI signal strength (RSSI) indicator for the board and the time in the bottom left. I deliberately left the seconds out as this would typically not be refreshed that often – maybe every couple of seconds...

Bob suggested that by plugging the board into a battery pack you have a simple range tester and he’s absolutely right of course. Now, how to teach it to make coffee…

On the subject of terminals

No need to talk about VT100-type terminals for PCs – they’re coming out of our ears – but…

Android

I’d put this all together and I thought…an 80 line or 132 line version of this would be nice – I’ll put one on my Android phone. Well, you may well write in and tell me I’m wrong but I cannot find a VT-100 compatible serial modem for Android anywhere (I did find one but it was not clear if scrolling regions worked and it had a limited range of serial settings). Surprising considering that it can be done on a  relatively simple unit like the ESP8266

Linux

And that led me to Linux - or rather - I was thinking about the various single board computers I have lying around - a Pi would do but I have a rather nice FriendlyArm NanoPC T2 which was not quite fast enough for Android but runs a nice Debian.  I started looking for fancy graphical terminals - not a lot - nothing to anywhere near TOUCH some of the stuff on Windows - however I had this T3 with a little 7" LCD touch screen lying around and was determined it would not go to waste.

It turns out that the humble LX terminal does at least some VT 100 - but that pulls up a command line and lets you interact with it  -was there any way to get it to talk to serial instead - preferably one of it's own rather than a USB serial as it has FOUR serial ports on it.

Well, yes. I discovered this..

cu -l /dev/device -s baud-rate-speed

It looked as if this would let me use the terminal with serial instead of a keyboard- but of course when I tried it using:

cu -l /dev/ttyAMAT3-s 115200

I got zilch. The system didn't have a clue what CU is (neither did I for that matter).

Anyway, just for a laugh I tried SUDO APT-GET CU

And it worked. I tried again. THIS time all was well - but it could not contact the serial port - as it was "busy" - yeah right.

I added user FA (the user in control of that terminal session) to the relevant group - no difference - so as is often the case I just gave 777 permissions to the serial port and VOILA.

Terminal on Debian running serial

I tested some colour escape sequences from my PC Serial Terminal I wrote some time ago (and just recently updated to let me put ESC in there) and all worked well but for some visible representation of the ESCAPE sequences (which still worked). I continued experimenting and the UX terminal that comes with Debian LDXE does not suffer that particular issue – so it has the job!!!

Facebooktwittergoogle_pluspinterestlinkedin

Faster ESP I2C

Experimenting with I2c on the ESP8266? I am – and I’m having a great time with it. If you’re using my code you don’t really need to think about it but if you’re hunting around for better I2c for the ESP – or maybe interfacing the ESP8266 to Arduino – then you’ll love the stuff I’m doing right now – and yes, I AM looking for I2c experts to tell me if I’m doing something wrong because it is all going rather too smoothly.

I decided to take the end from the info from the last blog entry and separate it off as I learn more about I2c (which to recall is a simple 2-wire multi-drop serial communication protocol with separate clock and data).

Why read this? Well, up to now because I spotted a howling issue in the WIRE library for Arduino, found an improvement on the ESP8266 SDK I2C code in the ESPRUINO changes to the Espressif I2c code - I think I found a mistake in it, fixed it then SAILED past the original speeds, reducing a 15ms package best case to well under 3ms  while adding clock stretching into the bargain.  I should clarify at this point that I am NOT an expert in I2C but hey – I can’t get this to fail so I might be onto something! Read on!

So firstly let’s recall -  I’ve been trying to make an Arduino peripheral for the ESP8266 Home Control 2016 project.  I started with the WIRE library in Arduino, working on a small NANO device. I discovered that there are two buffers in WIRE that are 32 bytes in length and hence limit packages to less than that  - I replaced them with a couple of 128 byte buffers and solved lots of issues I’d been having – that is detailed in the last blog.

Where we left off, I’d made a test setup to send a message to the newly-created NANO peripheral which right now on my bench handles a QTECH display and some IO. The ESP8266 I2C is also talking to a BME280 temperature/pressure/humidity sensor and a PWM expander board – so there are 3 devices – some input, some output.

In the last blog entry I’d added a test command for the ESP which could be sent to the board serially or by MQTT and went like this…

{nano:9,7,"how are you today and the weather is nice - NO REALLY it certainly is.",0}

Simple enough – send a message to device 9 (my chosen default address for the NANO peripheral), command 7, a string – and a dummy 0 on the end to tell the device not to bother sending any return info.

As you can see below, this operation takes, using the standard Espressif Ic2 Master code, 15ms in total – not stunningly fast but ok.i2c_thumb2

Flush with the success of finding that buffer issue, I decided to go into the ESP I2C library and have a tinker – changing all the delays into ‘'#defines so I could mess with them. So without any real effort I could reduce delays from 14ms to 6ms – a very worthwhile improvement. bear in mind however that the ESP code has no clock-stretching and hence no way for the peripheral to slow things down. I am told that the call-back routine within WIRE holds the clock down until it is finished, in order to slow things down – this would do nothing but cause havoc for the Espressif code.

Below is a read operation which I got to work properly after reading  this video – as I had no idea what a re-start was – see the little green circle just after the third byte.

read_thumb2

As you can see in the image above I address device 9 (18 as it is shifted left by one – i.e. 7 bit addresses but the bottom bit is reserved to indicate a read or write) – I fire command 2 to read port 4 – then with an I2s restart (start without a preceding stop), I  resend to the device with the LSB set (ie 19) and get a value 0 back (from a pin)  All in a matter of a millisecond.

I was at this point getting worried about lack of clock-stretching – in which a slave can hold the clock low to “stretch” things out a bit – about the only control the slave has!

It looked as if this forum might have some answers as their modification of the ESPRESSIF code certainly had clock stretching in it. It involved some changes and I decided to pick the best bits from their modification.

So they’ve simplified the code a little – while adding in clock-stretching – by replacing a simple delay with a check for clock low – easy enough when you look at it… but they also seemed to have completely messed up master_start – at least, they had it the opposite way to Espressif – and accordingly, repeated starts – as I use in getting data out of a Nano above – simply failed. The logic probe said this was all wrong.

So I reversed master_start back to the way Espressif have it  –so I ended up with a mix of the old and new – and the result – well, up to now everything works – and that long 15ms string send was now reduced to under 5ms from 15ms.

5ms string_thumb[2]

I tested the code with the PWM chip I’ve been playing with and the BME280 – so that is both reading and writing and up to now both were working perfectly. 

It was notable that the clock low period was now much longer than the clock high period and I wondered if there is any way to shorten that.  You know when you get deeply into code and all of a sudden it all becomes clear.  I realised that setting and checking ACK signals was a function – with all that entails – I changed that to a macro. I realised that delays in loops were really un-necessary as the loop itself would cause delays.. I took them out.

At this point I had backed everything up expecting my experiments to fail miserably. I tested the new code and everything worked. I got the logic probe out again.

3ms

My original test string at 15ms was now down to 3ms and according to the probe – all is well. I DID get it down to 2.5ms but the probe said I was doing restarts in the middle of the string – I’ll find another way around that one. Current state of the art – 2.9ms

As for reads – my original byte read was taking somewhat over 1ms – it now takes 0.25ms.

fastish read

And of course I’ve concentrated on the read and write routines leaving the inter-byte handshaking pretty much along so maybe there’s another 10% improvement to make here without dipping into ESP assembly language.

https://bitbucket.org/scargill/esp-mqtt-dev/src 

The driver and firmware libraries in the repository above contain the modified Espressif i2c_master code – and also other I2c wrappers for sending packages – they are fine – any further optimisation needs a good solid look at the one file (and it’s header) for i2c_master.   Open to ideas (that don’t involve converting Arduino assembly language to work with the ESP SDK – been there, failed).

I have just noticed that in the write cycles, there is still more off-time than on for the clock – and so I just took a delay out – which SHOULD mean the off-time is way too slow but because of the time taken to call and process the function – we still end up with 0.9us on time and 1.3us off time – everything continues to work on the tests I have – yet the total time for that read drops from 0.25ms to 0.21ms and the string write time from 2.9ms to about 2.2ms – again – worthwhile improvements.

That simple change was made in the write routine…

void ICACHE_FLASH_ATTR
i2c_master_writeByte(uint8 wrdata)
{
    uint8 dat;
    sint8 i;

    for (i = 7; i >= 0; i--) {
        dat = wrdata >> i;
        I2C_MASTER_SET_DC(dat,0);
      //  i2c_master_wait(I2C_DELAY_5);
        I2C_MASTER_SET_DC(dat,1);
        while (!GPIO_INPUT_GET(GPIO_ID_PIN(I2C_MASTER_SCL_GPIO))) {}
       I2C_MASTER_SET_DC(dat,0);
    }
}

Logic analyser says yes, 3 test items say yes… how low can it go!

Facebooktwittergoogle_pluspinterestlinkedin

BMP280 for HC 2016

I have just added a working BMP280 implementation into the Home Control 2016 code (see right menu in blog) returning temperature and pressure.  Add that to the (already implemented) BME280 code, Seeed display, 16,channel PWM, port expansion and more, it’s not been a bad development week really! Manual updated.

The BMP280 as you know does not have humidity – but is cheaper than the BME280.The manual is updated and has details on addresses etc.  Just waiting now for the A/D board to turn up from China.  I’m quite getting into this i2c expansion.

This little picture attempts to show the current state of play for Home Control 2016. No doubt I’ve missed loads off but as you’ll see quite a bit has been added recently.

Home Control 2016

Facebooktwittergoogle_pluspinterestlinkedin

16 Channels of PWM for ESP8266

PCA9685This morning a little board turned up for me – the advert said “Smart Electronics” – but the board says Adafruit 16-channel 12-bit PWM board. At £1.38 each these represent EXCELLENT value – especially as the Adafruit originals are much more expensive. Nicely made with gold connections and all the connectors provided.

Actually when I bought these I was unaware of the Adafruit product – I didn’t see the likeness until I went looking for code to modify for my own use.

http://www.aliexpress.com/item/Smart-Electronics-PCA9685-16-Channel-12-bit-PWM-Servo-Driver-I2C-Interface-for-Arduino-Raspberry-Pi/32464046768.html?spm=2114.13010608.0.73.QEOjVb

https://www.adafruit.com/product/815

(the Adafruit board is blue, the one I have is dark purple) - In essence a very nice little  board which takes in 5v and I2C (in this case software I2C on the ESP8266) and gives out up to 16 channels of 12-bit PWM at up to 1,600hz without external clocks etc.

So right now I’ve added only basic control – a single command that has two uses – one to set up the board, the second to control individual channels. I’ve added this to the ESP8266 home control.

The command works like this – two examples – one for setup, one to control output 0 (ie bit 0 as you can control multiple bits at once) – and assuming device 0x40 (decimal 64)

{pca9685:64,0,1600}

{pca9685:64,1,4000}

where 0 is OFF and 4095 would be full on.

Here's another example - set all 16 outputs to 100

{pca9685:64,0xffff,100}

PWM ControlI checked out the Adafruit code for Arduino and made a very simplified version – I’m not sure I fully understand the two parameters (ON and OFF – the last two) because setting the first to 0 seems to give the full range using the last parameter only – maybe someone who’s already used this might enlighten us. Anyway, it works, reliably and it’s available. I’ve updated the source and the ROMS.

To test, I wired from the ESP ground, GPIO4 and 5 (already connected to an I2c device with pullups) and Vcc. I also connected +5v (that goes to the 5v rail on the board) and then I connected a LED to the +5v rail and PWM output 0.  really very simple. I guess what I need is some kind of timer-based control to allow slow ramping up and down of brilliance – which would mean you could arrange at least 16 channels of lighting from the ESP8266. Mind you – you can do that with serial RGB LEDS but they’re quite expensive compared to other lighting.

In the photo above I connected 8 outputs to +V and a bar-led – I bought these for testing just like this – as you can see I’ve put values from 2000 to 4060 in there and there’s a nice variation of brilliance on all of them. The speed this is working (1,600hz) in the background (all done by the chip) – waggling the display does not produce any kind of strobing effect). As for levels – at the very dimmest levels with a bright LED you can just tell the steps – but WAY, WAY better than simple 8-bit PWM.

All works – if anyone wants to take the software further who’s already been there, happy to incorporate any working additions.

Follow the Adafruit link to get their Arduino code – here (minus the ESP master library) is what’s left once I get what I wanted…

This is all in the home control software – just reproduced here so you can see what’s involved, maybe point out any improvements etc.

uint8_t PWMAddr=0x40;
#define PCA9685_MODE1 0x0
#define PCA9685_PRESCALE 0xFE

#define LED0_ON_L 0x6
#define LED0_ON_H 0x7
#define LED0_OFF_L 0x8
#define LED0_OFF_H 0x9

#define ALLLED_ON_L 0xFA
#define ALLLED_ON_H 0xFB
#define ALLLED_OFF_L 0xFC
#define ALLLED_OFF_H 0xFD

void IFA pwmWrite(unsigned char addr,unsigned char d)
{
i2c_master_start();
i2c_master_writeByte(PWMAddr << 1);
if (!i2c_master_checkAck())
{
i2c_master_stop();                   // End I2C communication
iprintf(RESPONSE, "Bad PCA9685 I2C\r\n");
}
else
{
i2c_master_writeByte(addr);
i2c_master_checkAck();
i2c_master_writeByte(d);
i2c_master_checkAck();
i2c_master_stop();                   // End I2C communication
}
}

uint8_t IFA pwmRead(uint8_t addr)
{
uint8_t a;
i2c_master_start();
i2c_master_writeByte(PWMAddr << 1);
if (!i2c_master_checkAck())
{
i2c_master_stop();                   // End I2C communication
}
else
{
i2c_master_writeByte(addr);
i2c_master_checkAck();
i2c_master_stop();     i2c_master_start();
i2c_master_writeByte((PWMAddr << 1)|1);
if (!i2c_master_checkAck())
{
i2c_master_stop();
}
else
{
a = i2c_master_readByte();
i2c_master_stop();
}
}
return a;
}

int IFA ifloor(float x) {
int xi = (int)x;
return x < xi ? xi - 1 : xi;
}

void IFA pwmFrequency(uint8_t chipAddr, float freq)
{
PWMAddr=chipAddr;
pwmWrite(PCA9685_MODE1,0); // saves a reset function
freq *= 0.9;  // Correct for overshoot in the frequency setting
float prescaleval = 25000000;
prescaleval /= 4096;
prescaleval /= freq;
prescaleval -= 1;
uint8_t prescale = ifloor(prescaleval + 0.5);
uint8_t oldmode = pwmRead(PCA9685_MODE1);
uint8_t newmode = (oldmode&0x7F) | 0x10; // sleep
pwmWrite(PCA9685_MODE1, newmode); // go to sleep
pwmWrite(PCA9685_PRESCALE, prescale); // set the prescaler
pwmWrite(PCA9685_MODE1, oldmode);
os_delay_us(5000);
pwmWrite(PCA9685_MODE1, oldmode | 0xa1);  //  This sets the MODE1 register to turn on auto increment.
}

void IFA pwmSet(uint8_t chipAddr,uint8_t num, uint16_t on, uint16_t off)
{
PWMAddr=chipAddr;
i2c_master_start();
i2c_master_writeByte(PWMAddr << 1);
if (!i2c_master_checkAck())
{
i2c_master_stop();                   // End I2C communication
}
else
{
i2c_master_writeByte(LED0_ON_L+4*num);
i2c_master_checkAck();
i2c_master_writeByte(on);
i2c_master_checkAck();
i2c_master_writeByte(on>>8);
i2c_master_checkAck();
i2c_master_writeByte(off);
i2c_master_checkAck();
i2c_master_writeByte(off>>8);
i2c_master_checkAck();
i2c_master_stop();                   // End I2C communication
}
}

And so there it is – working 16-channels of PWM (or 32 or more) added to the home control setup so you can control these lights via MQTT or serial via a simple command.

If anyone wants to tinker – the lights all start up as ON – I’d rather they started up as OFF and I’d also like a master ON/OFF. Ok, I could do it the hard way I guess.

I could see a daughter board coming up with 16 MOSFETS on it… actually you could just use it for on-off control if you wanted  - at the price.

One of the BIG benefits for me is – the Espressif PWM is simply not that good – I use it – but for example, you cannot use that AND I2c at the same time because both the Espressif implementation and another I’ve tried both continue to mess with interrupts in the background even when on 100% or off 100%.  This neatly bypasses the issue.

Facebooktwittergoogle_pluspinterestlinkedin

ESP8266 Home Control Update

As the blog entry on using my home control code ROMS for ESP8266 is filling up and of course as much of the information is now dating I thought I’d do a new blog to bring everyone up to speed and move the conversation to this blog entry. The good news is  - the code is now running under SDK 2.0 – i.e. bang up to date at the time of writing.

Firstly – yes, everything works – but the procedure for flashing ESP-12 etc has changed ever so slightly. I have updated the binary files to run on Espressif SDK 2.0 and updated the RBOOT code I use for OTA (over-the-air updating) to the latest version.

To backtrack a little

For anyone new in here – some background - I’ve written software for the ESP8266 which has been in operation for some time and is under constant development. It allows the ESP8266 to control things like simple lights, serial LEDs, PWM LEDs and read sensors etc, talking via WIFI and MQTT to whatever central system you have – for example a Raspberry Pi with Node-Red. I’ve written a node to allow endless units to log in and to provide central communications for them. The software is rock solid and runs 24-7.

I use Windows to develop the software using the Cherts unofficial development environment. This provides an IDE using Eclipse for running and compiling C code. It is not necessary to use any of this if you simply FLASH the ROMS which are available – but of course if you do use the IDE you can customise code for your own use.  The projects Home Control 2016  and Nextion WiFi Touch Display both use the same software and are the end result of many years developing home control. Personally I access much of this equipment using Apps like Blynk and more recently Imperihome (see also this article, this and this).  Tools I use in Node-Red include my own BigTimer node.  All of these options were chosen after much deliberation.

So what have I learned and what has changed?

All has been going well until recently when the Espressif SDK for ESP8266 was updated, firstly to 1.54 then very quickly followed by 2.0 and all of a sudden I and other started to have issues starting up ESP8266 boards. This is now resolved incidentally.

Firstly one of my problems was lack of serial output on power-on. Some serial (115Kb) status messages I have on power up, just point-blank refused to show. I figured the code might be starting at the wrong place. But no, a quick port flick at power-up proved that everything was working – but no serial out.

After much debugging, I discovered that there would be no serial output until this function:

  wifi_set_opmode(0);

had been called. Prior to SDK 1.54 this was NOT the case. Simple enough - I put this at the start of my code – I’ve just checked and web updating works just fine - but merely moving this function to the start solved all the serial output issues.

Incidentally much is made of the “garbage” that comes out of the ESP8266 on power up – it is just that Espressif chose to use 78,600 baud instead of a more popular one. If user software is set to this speed rather than, say, 11500 (which is what I use) then things look very different – diagnostic on power up followed by user code – all very seamless. Problem is – not that many serial terminals support 78k.. but I digress.

The second problem was a tendency for the unit to run away with itself, firing out garbage as if the ESP-12 was broken. It would SEEM that post-SDK-1.53 or thereabouts – it is NECESSARY on initial run, to flash location starting 0x3fc000 (end of the 4th page in a 4MB Flash chip) with the default file esp_init_data_default.bin which is found in the Espressif development kit.  If this has already been blown (previous installation) then it is not necessary to redo that, but with a CLEAN FLASH, failure to flash this area results in the continuous rubbish on output. Again this did not seem to be the case before SDK 1.53/4 – I have NEVER flashed that location in the past. Once blown it is not necessary to blow this again for OTA. Still – now we know.

I’ve just had a Skype chat with Espressif engineers to try to dig deeper. They have confirmed the small file at the end is essential (I probably didn’t notice it before as I’d generally be using boards already programmed) – the serial issue we’re still trying to understand – but it all works.

System Data

There are 2 relevant files here. A file called blank.bin sits at 0x3fe000 (top of the third MB in a 4MB Flash i.e. ESP-12) and contains default system parameters produced in the SDK.  NOT setting this initially seems to have no effect on my code.

A file called esp_init_data_default.bin sits at 3fc000 and contains default system parameters stored in the SDK. This is a 1K file. This HAS to be blown when first setting up a chip.

What to do

There is a binary file called esp_init_data_default.bin which I have added to the server at www.scargill.net – this is in addition to rboot.bin and romx.bin

In the Espressif setup there is a file called ESPTOOL – there are exe versions and .py versions of this floating around (see below). This is what I use to flash the ESP8266 chips – there are others – you may use them – but I can’t VOUCH for them!

https://github.com/themadinventor/esptool

https://github.com/metalheart/esp8266/raw/master/bin/esptool.exe

In Windows this works for me – you will have to check for yourself.

To completely wipe an ESP12 – including all data and setup etc and assuming COM3 in this case (not necessary but if it makes you feel better)…

esptool.exe  -p COM3 -b 115200 erase_flash

To initially flash the code from scratch… assuming you have the 3 files mentioned above…

esptool.exe -p $(COM3) -b 115200 write_flash -fs 32m -ff 80m -fm qio 0x00000 rboot.bin 0x02000 romx.bin   0x3FC000 esp_init_data_default.bin 

Obviously you can change port, speed and file locations. I usually double up on that baud rate but this one is safe.

If you insist on trying this with an ESP-01 (no way will OTA work) – that 3FC000 address becomes just 0xFC000 as they only have 1MB of Flash unless you upgrade them (discussed in another blog in here – done – easy).

As an alternative, I’ve tried NODEMCU FIRMARE PROGRAMMER and that seems to work…

NodeMCU

Nodemcu[6]

And so there it is – the source is updated on the web and requires SDK 2.0 or better to run. The ROMS are updated and the new file is up there.

Bear in mind there is some data here NOT being erased (see full erase command higher up) – in my case at 0x79000 – or around half way up the first megabyte – this is USER data – stored passwords etc. it was put here for compatibility with ESP-01 but I’m about to move this way up to 0xf800 leaving the best part of a MB for code – plenty of room.

And why?

Why worry about updates indeed? Well, one good reason – is that recently, updates to the SDK have used LESS precious RAM – the kind that programs (not data) run in – so with 2.0 there is more margin for expansion – which means more features and more room for more features! It is fine having 1MB to put code in but that is generally NOT the limiting factor with the ESP8266. The code ultimately has to run in RAM and there’s a slight bottleneck there – the gains of later SDK updates help a lot.

Update 31/07/2016

Having now  added i2c and parallel LCD displays to the board, I set a parallel LCD running from MQTT and updating all 4 lines of the display, once a second from Node-Red. While this was running (it’s been running all afternoon – I thought I’d try a test)… I set my serial terminal program fire out lots of on-off commands in a row – which is WAY over the top…

I set up 30 on/off commands for GPIO0 – to run at 80ms each – as you can see clean as a whistle.  I dropped that to 50ms per instruction and the board rebooted – I’m guessing that is a buffer overflow – but then – that simply is not going to happen in real life or anything like it.

I figured I’d go for broke - with 390 (I got sick of cut and paste) commands sent at 80ms it lasts almost to the end then rebooted – again a sure sign of the buffer being in trouble either overflow or wrap – backing that off to 100ms gave zero errors so not a wrap issues – Very happy with this – typically the boards get no more than a few instructions a seconds – often MUCH less. For the sake of it I think this coming week I may just take a look at this – firstly to see if at such speed, my terminal is screwing up – and if not – I’ll see if I can narrow that down but it is utterly academic

serial tester

A complete and utter aside

I don’t use the Arduino setup but as I was thinking about user data, I just checked the Arduino EEPROM library SOURCE for ESP8266 and the Arduino environment does NOT use the 3-sector trick employed in the SDK. When you update the EEPROM (which is actually FLASH on the ESP8266), the sector is wiped (even if you have user data smaller than 4k, a 4k sector is still used) and then updated from RAM – not ideal.  I have suggested to the designer that it might be a good idea to change this. Right now he’s busy working on ESP32 code.

So there it is – all working – hopefully I’ll soon get some clarification on the routine I mention above that I had to move. Time to start adding new features!!!

Facebooktwittergoogle_pluspinterestlinkedin

ESP12 and Memory

Anyone writing large projects with the ESP units in C (and probably in other languages without necessarily realising it) will have come across the problems of .TEXT memory. This is the RAM used to store functions.

Functions are stored in their own part of RAM called .TEXT memory and it is WAY too small. Now most of you will have made liberal use of the “ICACHE_FLASH_ATTR “ directive in your functions. This ensures that functions stay in FLASH until actually needed and this saves lots of .TEXT RAM – indeed without it, larger programs would be impossible.

Despite all of that I’ve been slowly running out, sitting at 7CFF  (you can only go up to 8000) recently and when the latest API came out – that was it – over the top – dead.

So for a while I stuck with the previous SDK and eventually got onto Espressif.

They asked me for a .DUMP file – which of course I neither had nor understood and they suggested that in the project directory I type:

xtensa-lx106-elf-objdump -x -s app_0.out > app_0.dump

WELL I was convinced this was another Linux tool that would not work in Windows – but it DID and produced the English-readable file app_0.dump which tells you where all your functions are sitting.

They suggested I add the "ICACHE_FLASH_ATTR"  to my EASYGPIO functions. Well of course as that was an existing library I’d not touched that – and sure enough – I got just enough memory back to be able to use the latest SDK!!   I was now sitting at 7B52 on the latest SDK – not good – but better… and at least I was using the latest SDK. They then suggested I use "os_memcpy, os_memset, os_strcpy" instead of "memcpy, memset, strcpy" so I systematically went through all the code and did this.

On recompiling – I now find the .TEXT RAM sitting at 79C2, the lowest it has been for AGES – another 512 bytes back just by using their functions instead of the standard ones!

I’m making no claims about reliability as I’ve not given this a thorough test – but that is for me a very worthwhile improvement in memory. Of course that means a tiny slow-down as functions have to be pulled out of FLASH to use… What speed would we have gotten out of these devices if they’d just done the RAM differently. Still – given the price of ESP devices I suppose we can’t complain.

Facebooktwittergoogle_pluspinterestlinkedin

EMW3265 ESP8266 KILLER

EMW3265Erm, no, though you might think so by this headline..  https://hackaday.com/2015/07/13/new-part-day-the-esp8266-killer

Read the comments – the $5 ESP8266 – where?  $2 ESP8266 more like it – less, even - http://goo.gl/VB0V9K

There’s even a claim that this new single-source board (Seeed Studio) has MORE memory – well at least one variation of ESP12 has 4Mbytes – and the EMW… according to the ad, 512K.

And what about the all important Windows and Linux support libraries? Erm, no.

Another comment – Chinese documentation? Well I don’t know about you guys but I have  raft of English documentation that until recently came with the Espressif SDK and which is now maintained online.

And the last – Chinese – guess where SEEED are!

Cheap, reliable, effective, more compatible variations of ESP and more suppliers than you can shake a stick at – and we want to move for what reason?

But – if you like spending money – there’s this – even more expensive - http://www.seeedstudio.com/depot/EMW3162-WiFi-Module-p-2122.html

Facebooktwittergoogle_pluspinterestlinkedin

Automatic Router Selection for ESP8266

I wonder if anyone who has done this can help.  As most of you know, I program my ESP8266 devices in C.  I've been having some issues with weak signal recently and it would be really nice to have a list of acceptable access points and their passwords - and have the ESP, if it loses the signal, check the list (including identical SSIDs) and reconnect.

Has anyone done this already and if so would they care to share code?

Facebooktwittergoogle_pluspinterestlinkedin

ESP8266 Remote Access

Notice I titled that remote access as against remote programming – but that’s part of it.

Years ago, having gotten sick and tired of dismantling my home control Arduinos just to upgrade the code, I purchased a couple of small units from China which I’ve just thrown in the bin.   One had a USB plug on it and plugged into the PC, the other had an FTDI output on it. Both had those black 2.4Ghz antenna.

So the purpose of these units was to wirelessly replace an FTDI – or serial programmer, the idea being that instead of bringing the Arduino project back to wherever one programs the devices, you plugged the unit into the project and over the air programmed it.

At the time I was using Arduinos and those CHEAP £1.50 radio modules – the NRF24L01s, the worst radio boards in the world who’s range could be so bad you’d be lucky to get through one wall. I had high hopes for them with a great mesh network called RADIOHEAD, but ultimately their range and lack of ability to read signal strength doomed them.   However, as you can imagine they were dotted all over the house and I thought this little radio pair would solve the issue of programming them without ripping everything up.

It turns out the “radio FTDI” had the same problem – it was based on the NRF24L01.

Fast forward to September 2015. Here I am in Spain with a new setup of ESP-12 modules controlling lights, watering system and a lot more.  I’ve made some improvements to our “home control 2015” software in the ESPs and once again I find myself having to dismantle kit.

Now, I know that is it possible to program over the air – a few guys have tackled this successfully and indeed much of the code is built into the Espressif SDK. Right now my friend Aidan is working on adding this to our code but there are two issues, he’s not had luck up to now and secondly as our code comes to something like 300k you’re never going to be able to add this to the ESP-01 modules (we use ESP-01 sometimes, ESP-12 other times…)  as you need as much spare storage as you have code for OTA programming!

It was with that in mind that last night I dragged out the old FTDI boards to see if I could save myself the effort of dismantling the Pergola lighting system to update it.  Suffice it to say that I managed to get a few bytes programmed before that all came to an end despite several attempts. With around 25 WIFI units of one description or another dotted around the place, the old wireless FDTI (same frequency range) simply could not handle more than a few feet.

Bin.

I figured it would not hurt to look around and see if someone has alternative wireless solutions – lots of time has passed since I bought that kit.

And so it was that I stumbled upon this.  https://github.com/jeelabs/esp-link – as it happens it is quite new – essentially what we are looking at is a single unit (yes, that had me going for a while) which will hook into the WIFI system and act like an FTDI – given that you can fire the serial out of your PC over TCP to a port – port 23.   I grabbed the code (the stable 1.04 version) and very excitedly blew it into an ESP-01 unit I had lying around.

Really, nicely done web interface (with remnants of Martin Hanzarov’s code in there) and a really good write-up – indeed there may be enough info in there to help us implement our own FOTO (over the air) updates to our code.

But this has far wider implications – a virtual serial link would also let you monitor the serial line of a remote unit for debugging info – and our units do indeed put out lots of serial debugging info. It would also let you remote program and monitor un-wirelessly enabled Arduino projects.

So the first problem was – I had this unit sitting as 192.168.1.27 on my network and I needed to blast info into port 23 to have it come out as 115K serial.  I have ECLIPSE sitting on the PC running my projects and there’s a line in the MAKE file that says “COM6”.  I have absolutely NO idea how to turn that into a TCP statement if indeed it can be done (FEEL FREE TO START TYPING IF YOU KNOW HOW).

So I turned to a 30 day trial of a PC program that promised to provide a “virtual com port” – and that it did. Serial/IP Redirector let me create a link to 192.168.1.27 port 23 and ship raw data back and forth to COM22 (which doesn’t exist on my PC)- LO AND BEHOLD I set it running, put PUTTY on the PC talking to COM22 and with a real FTDI attached to the little ESP-01 board on real COM6 I set that running on a terminal –at 115K. Utterly flawless raw serial communications – rock solid.

I was getting excited by now. I took my ESP-01 board and cross-connected the serial to an ESP-12 test board. I held GPIO0 to ground and turned the power on, putting the ESP12 into programming mode.

I punched COM22 into the ECLIPSE project and hit the FLASH button on the project. Lights flashed on both units. With a scope attached  to the serial in on the ESP-12 I could see clean 115k data going into my project and… nowt, zilch, nothing.

I was up until the early hours ensuring the baud rate was ok and that my level conversion wasn’t getting in the way – you see all my projects use a resistive divider on the serial input of ESP boards so the whole lot will work just fine with 5v logic on the FTDI and elsewhere.  I wondered if hooking two of these units together with serial cross connected might prove too much but the scope says no – it says I’ve a perfectly reasonable serial signal going into the ESP-12 – but it’s not having it – no responding output whatsoever. Eclipse tries sending out data over and over and eventually gives up just as if there’s nothing connected.

So near and yet so far.

I got up this morning and the unit was still rock-solidly connected to the WIFI and typing a few characters provided perfect results at the other end – but I cannot program ESP units – yet.

Very exciting, yet very frustrating.

Perhaps coffee and a new start to the day will help. BUT I had to bring this to you as the article is really good, the software seems to WORK and there’s a super-bumper version with it’s own OTA updating in the works for larger targets like an ESP-12 (more FLASH) than the ESP-01.

I’m sure this is going to be a winner – just need to get it to work. After all, the ability to add remote programming to any FTDI-enabled project for what, a couple of quid.. well, you just can’t ignore it.

Meanwhile if anyone reading this firstly knows how to set COMX on Eclipse (on Windows) to TCP – do let me know – and if you really think you know what I’m doing wrong here and want to jump in before I figure it out – by all means – you have the keyboard…

Facebooktwittergoogle_pluspinterestlinkedin