The issue of running out of FLASH space on the ESP8266 is now resolved – if you are interested in how FLASH memory on the ESP8266 works – especially using the Unofficial Development Environment with the official SDK and RBOOT works – you may find this useful. I now have more FLASH than I know what to do with…
So the problem was this… if you’ve been reading the blog – you’ll know I’ve been adding icon fonts to my ESP8266 C code (not Arduino – I use Eclipse and the ESP8266 SDK 2.1.0) … well, I thought I’d run out of memory.
static const uint8_t IRA meteocons32x37[] = {
32,37,32,
0x11, 0x00, 0x00, 0x00, 0x00, etc
Above you see a typical declaration of a font FLASH array (well, the first few bytes)… simply enough – stored in FLASH –not in RAM.
If you’re wondering what that IRA is ..
#ifndef IFA
#define IFA ICACHE_FLASH_ATTR __attribute__((aligned(4)))
#endif
#ifndef IRA
#define IRA ICACHE_RODATA_ATTR __attribute__((aligned(4)))
#endif
II kept running out of IRAM space… that is the code would not fit into the Irom0_0:seg.
There is some confusion out there – you can easily run out of RAM space for program code (as against variables – which is completely different RAM) but this is resolved by putting program code into FLASH – and this is pulled into RAM by the processor as needed – I always seem to be NEAR the limit of this – but never actually run out of space – but here I was running out of FLASH space despite being nowhere NEAR the 1MB limit. So I was interested when someone on the web said all you have to do is I alter the default eagle.app.v6.ld file to make the irom segment larger… no matter what I did, it made no difference. Then I realised that this was happening as I was accessing fonts….
I have a 32 bit variable…. which I use to then pull out 8 bit variables from FLASH using a function that stops the processor from crashing (you HAVE to access on 32-bit boundaries) so all I need to do to access any data in FLASH is call the function – that works a treat.
case ‘5’ : ch32=&meteocons32x37[0]; break;
Except – this case statement -and variations such as “ch32=meteocons32x37” adds 17K to my irom0.text area and ultimately as I add several fonts – that area won’t fit into irom0.seg
So my question was – why? Clearly – adding in any reference to a font, made the compiler add that into the mix – but surely I had a full 1 MB to work with (the ESP12 in common with many modern ESP8266-based boards has 4MB of FLASH. You can only access 1MB at a time – but that means 1MB x2 (for OTA) should be easily achievable.
Going back in time, when the ESP-01 was all the rage with it’s 512KB of FLASH, we used to put user variables into 3c000. So all program code had to fit below that. When Richard Burton helped me get my boards working with 1MB for code and 1MB for OTA (swap) – for reasons I don’t recall he set user variable area to 79000. But what I didn’t realise is – his OTA code replaces the normal EAGLE file and puts control over who gets what into two files – in my case rom0.ld and rom1.ld – so no matter what I did with the original files – it was making no difference to the fact that I was not allowed to use more than 0.5MB of FLASH even though I have twice that much available.
Finally on a hunch after reading comments and sitting in the baking sun for a while (radiator ran dry this morning – long story) it slowly came to me that these two files may actually be the ones setting up how much IROM space is available for programs.
Having made an experimental version of my code to put the user area WAY up at F8000 – I tried this in ROM0.LD and ROM1.LD
MEMORY
{
dport0_0_seg : org = 0x3FF00000, len = 0x10
dram0_0_seg : org = 0x3FFE8000, len = 0x14000
iram1_0_seg : org = 0x40100000, len = 0x8000
irom0_0_seg : org = 0x40202010, len = 0xf0000
}
Note that the IROM section is now only a little short of a meg in size.
LO AND BEHOLD – I could now compile with all my fonts – and STILL use only half of the space available to me.
The future is bright.
CRACKED – not by intelligence but more the kind of brute force they use to win chess games…
Throughout all of this I could not get to the bottom of why the EAGLE file would not make a difference – well that is because Richard Burton – when designing the RBOOT system I use for OTA, decided to transfer the functionality to the files ROM0.ld and ROM1.ld – I changed both of those from his original settings – designed for older times with smaller FLASH – to allow me to use all the way up to F0000 (nearly a meg) because the only other thing in the first meg (or second – depending on OTA) is my user data which I’ve now set in an as yet unreleased version of the software (as it screws existing user settings) from 79000 to F8000… ie right at the top of the 1 meg window…
Quick compilation and sure enough – it was the largest font that was making me run out of FLASH – but as the compilation now ends at 0x75c00 – i.e.e just under 0.5 meg – I now have more FLASH than I could ever fill in a million years. MORE FONTS!!!
Wheeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee.
🙂 good to hear! Now you might be able to save even more flash with the read-back-from-flash-function mentioned above. Good luck.
What I’d REALLY like to do – is to take both the user config area and the fonts out of the first and seconds megs – and put them into the last – largely unused meg on a 4 meg system – still leaving a meg for any kind of file system – but for now – I’m happy!
Hey,
the following might not be related to your problem because the behaviour is quite different (seg fault while reading) but yours reminded me of this (Arduino buuuhh):
https://github.com/esp8266/Arduino/issues/3140#issuecomment-305711323
They ended up with these functions to retrieve bytes from flash:
https://github.com/esp8266/Arduino/commit/0b672668bf636ef650b8b700a0af14677c3fafe5
You might want to try them to get the bytes from flash instead of just dereferencing a pointer.
Worth a try I guess.
Hi Lny – that is actually very useful…. my 16-bit routine fails so I’m stuck using the 8-bit routine – If I can get that running on the compiler I’ll use it – but the problem I seem to have is no matter what the mechanism – as soon as I mention the font name – the storage goes up – so no matter what routine I’m calling – actually referencing the font at all puts it into the FLASH area..
Now that’s fine – and I guess you’d expect that because at the end of the day, you need that in the FLASH update – no point in referencing a font if you’ve not uploaded it. What increasingly seems to be the problem – is that I’m nowhere NEAR a meg and the compiler is griping… there are a couple of areas at the top used by Espressif but I should be able to access a meg of Flash – and it appears that, ignoring the settings in the eagle file – the compiler does not want to let me. But that info is definitely useful.
Another thing I just realized is, you are forcing the array to be 4byte aligned (as needed… also discussed in that git issue). Could it be that the size added to you flash file is exactly 4x sizeof(meteocons32x37) ? What, at least to me, would be reasonable because the alignment forces a unit8_t to occupy the space of a uint32_t.
Another thing, as you are talking about that 1meg thing, I also had to think about that:
https://github.com/raburton/rboot#limitations
…just brainstorming… not sure if that helps.
In the case of arrays I’m pretty confident it simply aligns the array on a 4-byte boundary – otherwise my simple maths for getting to the array (a mere 32 bit pointer) would fail as I incremented through – I have a routine for accessing any arbitrary 8 bits inside a 32-bit value… though it’s a tad wasteful – I have to say – it does not slow things down significantly.
How many case statements are in the switch? I have seen a limit of 255 in some compilers.
Maybe rather than using a switch statement, the fonts need to be an array of arrays, or an array of pointers?
Still brings them in – I’ve been thinking about this while out driving today… so you mention the font and it is brought into the flash – the only thing that can be wrong is that my setup is not allowing for a full 1 meg of FLASH and that’s a worry.
Here you’ll see my eagle file..
MEMORY
{
dport0_0_seg : org = 0x3FF00000, len = 0x10
dram0_0_seg : org = 0x3FFE8000, len = 0x18000
iram1_0_seg : org = 0x40100000, len = 0x8000
irom0_0_seg : org = 0x40240000, len = 0xE0000
}
And that has irom starting at 040240000
Problem is when the code compiles …. I see this…
4 .irom0.text 000663e4 40202010 40202010 0000b080 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
As you can see the irom start address is not quite the same as that defined in the eagle file… the length EOOOO is new – it used to say 3c0000 – well as I long ago moved my user info up to near the top of 1 meg, shifting that size to E0000 should have allowed me access to more Flash – but it apparently made no difference whatsoever – even though deliberately introducing an error causes the compilation to fail – so it must be the right file….
A successful FLASH finishes here…
Writing at 0x00072800… (99 %)
Writing at 0x00072c00… (99 %)
Writing at 0x00073000… (100 %)
As you can see it ends even below half a meg, never mind a meg – yet I add in 3 or 4 fonts which cannot take more than 100k or so between them – and the compilation says I’m out of room as above. I remain at this point – confused.
Doesn’t the linker omit the meteocons32x37 array if there are no references to it. By referencing it, it is now including that block of data.
Check the map file with and without the case statement to see if the array is included in both instances.
Glen.