My thanks to Richard Burton for some updates. This entry which started out as guesswork (but with a practical aim) turns out to have been not that far off – I’ve amended the text and it’s now hopefully a decent guide to memory use in the ESP8266 etc… (bear in mind not ALL of these board have the same amount of FLASH – some of the ESP12s have way more than other boards (sadly, no more RAM but there you are).
As you’ll know from the previous blog I was having issues with the SDK which are now resolved thanks to some helpful types to whom I am eternally grateful. My code generally starts off using TuamPMs MQTT C code and expands on that (a lot), using MQTT as the base for communications. Well, as time goes on my code gets bigger and bigger – and I was starting to run out of both FLASH and RAM. The latter was easy to mitigate by using compiler directives to keep as much in Flash as possible – hence freeing up some RAM – but before long I was hitting limits. One kind chap made a mod to the MAKEFILE to move the FLASH around a bit but it’s only today that it has all really started to sink in – so here’s my attempt at explaining what I THINK is happening and how I THINK I’ve made a lot of improvement.
The ESP contains a 32 bit xtensa core processor which has a MASSIVE potential for address spaces. Hence, our programs which appear to start at 0x00000 i.e. 0, actually sit at 0x40200000. As you will know if you’ve been compiling code, you’ll see binary files at 0x00000 and 0x40000… and my code was heading up to the limit of 0x80000. Well, actually the limit is lower than that because it seems that Espressif have reserved some space up at 0x7c000 in our mapped area.
So the map on the LEFT shows how most programs seem to be laid out. In the case of TuanPMs code, some bits of the SDK, program data and any code not marked “icache_flash_attr” start at the bottom in the pink section – then there’s a big gap and TuanPM stores his non-vol variables at 3c000. He actually stores the lot in a struct and replicated it using 3 blocks – 3c000, 3d000 and 3e000. The first two are mirror images of each other and the third block has a byte that says which one was last updated. It’s a mechanism to ensure no loss of data if a block is being updated during a power failure… you just keep using the last one! And that’s fine but hey, I’m running out of room in the top half.
So I discovered by trial and error that you can’t use 7c000, well, you can but the program won’t work – so I moved his vars up to 78000. That left a honking great gap between the top of the SDK and the bottom of the area at 0x4000 which contains all the program material marked “icache_flash_attr AND the actual SDK (irom section) – as both your code and the SDK is in there – you want as much room for that block as poss. Well, it turns out that with 2 simple modifications, one to the Makefile and a corresponding change to the eagle.app.v6.ld file – you can move things about. That bit of dark blue at the top changes position depending on which chip you are using – but it is usually the top 4 sectors – so they are essentially verboten.
So now – see the second picture, my non-volatile info is up at the top, my program space is moved down – and there is a TON of FLASH space left to move into!
I may or may not have the description right – but I can tell you that it all works.I tried fitting the (red) non-vol storage just under 10000 and that survived power cycling but did not survive re-flashing the chip – where it is now DOES survive a re-flash.
And so there it is – I’m not confident enough to start telling people how to do this but I can say that (after backing EVERYTHING up) if you take a look at your makefile and find references to 0x40000 – and change to 0x10000 – and then look around line 8 in the eagle.app.v6.ld file and you’ll see there that you can change the base address and the length of the segment called irom0_0_seg.. if all of that is available in your installation – then you may just find you can move everything around.
Ok, those who know more than me – what did I get wrong? oh and I’m now running this no problem with version 1.11 of the SDK – AND am about to take out TuanPMs saving mechanism as they’ve built that into the SDK – no point in re-inventing the wheel.
Dear mr. Scargill,
For many times, i use your article in your post about esp8266 memory layout etc, to make apps on esp8266.
Lately, i got some problem on accessing (read or write) the flash region above 0x80000 (after 512 kbytes) on my esp12e (32mbit flash).
Everything that i wrote there (by erasing and then write to the flash), can not be read back (read result different from written data).
But if i wrote it below 0x80000, e.g. 0x7C000, i can read it back succesfully.
Was it caused by bugs on the sdk ? Already posted it on the bbs.espressif.com (reported bugs area) with no luck.
Please advice.
best regards,
Can’t help you there – because I have a mix of boards I tend to stick within the 512k space.
Hi Peter,
Just a few notes to hopefully clarify some of the bits of guesswork. The xtensa core at the heart of the ESP isn’t 64bit, it’s only 32bit.
On the left side of your picture, the pink block labelled “sdk etc.” is all the stuff that ends up in dram/iram, this contains a bit of code and data from the sdk but also your own data and any of your code not marked with icache_flash_attr. The light blue above it is most definitely wasted, no question about it. The light orange block labelled “program space” is the irom section, this is mostly made up of sdk code (at least 180k of it) and any of your code marked with icache_flash_attr. The position of the top darker blue block depends on the size of the flash chip – it will be the last 4 sectors of whatever size chip you have.
Richard.
Many thanks for that Richard, your help as always is appreciated. I will update the blog accordingly – well, I will that is when Windows decides not to close Live Writer on me mid-stream to do an update!!
Right Richard, I’ve updated the blog – please feel free to suggest any further clarifications or comment on any screw-ups I may have made. The better we understand this – the more we can do with it.
I am using this guys dev environment with a lot of success for my purposes, mainly for his OTA supporting Makefile and easy setup of new projects: https://github.com/nqd/esp8266-dev.
He also fiddled with the linker script, which worked like a charm for me and allows for bigger OTA binaries:
“To use OTA update firmware with 512KB, change linker file ld/eagle.app.v6.new.512.app1/2.ld, section irom0_0_seg to have more spaces.
irom0_0_seg : org = 0x40201010, len = 0x31000”
So I guess you should be fine fiddling around with the memory layout. As long as it works…
I shall try your layout as I am also running a bit low on space. Especially with OTA support which halves you available flash space 🙁
Interesting – – especially the location of the irom0_0_seg which seems to be in the boot area..
For simple projects mine is set to 0x40210000 and len 0x68000
The important difference between the address here and in yours is that this one is using a boot loader. This then loads a rom image of a different format, where the irom section is first. This is better because you don’t need that arbitrary split in the rom at 0x40000 (or 0x10000 as moved to in your case). You put the irom section first at a known address, and the iram/dram sections can follow immediately afterwards, eliminating the block of wasted space in the middle. this works because although the irom section needs to be placed at a known address on the flash, the iram/dram section do not (they will be copied by the boot loader their proper locations in memory). Somewhere in our long email thread there was some discussion of this, I think, and there is a lot more info on the workings of the bootloader on my blog http://richard.burtons.org/2015/05/17/decompiling-the-esp8266-boot-loader-v1-3b3/
You don’t get the wasted space when using a bootloader. So, as long as the section sizes have been set to the maximum they can be there is nothing else you can squeeze out of it.
If you’re back from hols Richard, I’d really like to have a go at using your bootloader with my projects on the Windows Eclipse setup here…. you up for a little hand-holding?
” find references to 0x40000 – and change to 0x100000″ in fact is 0x10000 not 0x100000
Well spotted… and now fixed.