Monthly Archives: September 2016

Sparkly new Pi

September 29, 2016Pixel, Raspberry Pi 2, Raspberry Pi 3, RaspbianNew Raspberry Pi Pixel interfacePete

Just a very quick one (and somewhere to park the conversation)– there’s a new Raspberry Pi update out promising a new smoother, simpler look, better video etc. Up to now it looks like the claim is correct.

I have several RPI installations…including TightVNC and so I didn’t want to lose that. I did however, want the Chromium browser. I’m not interested in PI-HATs and similar so here’s all I did to get the upgrade…

sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install -y rpi-chromium-mods

That’s it – nothing more –takes a few minutes all in - there is one Y for confirm and a reboot – and Bob’s your uncle. To test things out I went to the BBC website and of course got the usual message about Flash – things haven’t change there much – but with a signup for the BBCs “beta” HTML5 version (it’s a bit late for a beta, isn’t it? HTML5 has been out for longer than I can remember) all is well, videos no problem!

The new interface is to be given it’s own name “PIXEL” – that’ll make searching for information about it a whole lot of fun… they could have picked something unique like “PIXL” or something – however it all seems to work – more modern appearance – slick – file manager is nice and apparently if not faster it is at least no slower than it’s predecessor so really no reason not to go for it.

Here’s the link in case you need more info. Given this new look – with hardware graphics support – which likely means you could be using your Raspberry Pi 3 for a media centre… AND the best backup program out there which can replicate the system even on a different size SD -something as far as I’m aware NO-ONE else has – one has to ask – what are the other SBC board manufacturers going to do now!!!

Arduino STM32

September 28, 2016GeneralPete

Having decided I needed a more powerful generic peripheral board than the Arduino, I sent off for some STM32 boards to China – they got lost in the post. Meantime my friend Aidan sent me a nice little black board marked “JM ELECTRONIC STM32F10XC8 SYSTEM BOARD”.

It came complete with miniUSB connector so I plugged it in. A red light came on and stayed on – and a green light flashed on and off regularly. Always a good start.

I found this link… which suggested I install Arduino Zero into my existing Arduino 1.6.9 IDE setup. Life is never that simple – the nearest I could find in the Arduino boards manager was “Arduino SAMD Boards” – which mentioned Arduino/Genuino Zero. I installed that.

From here I grabbed the Arduino_STM32 directory from that link to put in \users\pete\my documents\arduino\hardware folder… I’m familiar with this idea as I have my mighty-1284p folder in there already to program the 1284p chips – a powerful more-or-less Arduino compatible chip which sadly never really took off.

I closed the Arduino IDE and re-opened… lovely – except – Windows 10 does not recognise the USB device. I loaded the drivers to no avail… it simply was not having the board yet I had no reason to suspect there was anything wrong with the board itself.

MrShark in here did his best to convince me that the USB connector was not for serial – some places suggested it was – anyway – I decided to assume he was correct on this and instead of using the USB connector, used one of the end connectors which has ground, 3v3, serial in and serial out – attaching them to my normal FTDI.

This PDF document got me part of the way because the board featured within is exactly the same as the board I have (courtesy of Aidan Ruff who bought it from AliExpress – mine is still floating about in the post, is slimmer and half the price – generally it looks like we are talking about 64k of FLASH and 20K of RAM and a BOATLOAD of peripherals). The connectors indicate that there are various ways to boot the board. Interestingly you can boot from FLASH, or boot into programming mode or boot from RAM (that’s interesting). I selected boot into programming mode and sure enough – reset the board, send the program instruction – voila – one programmed board – no warnings, no error messages.

Elsewhere I read that there are 15 pins that can be programmed as 16-bit PWM – a step up from the Arduino. Sure enough I tested this out in the examples – the pins were numbered 0,1,2,3 etc and these equate to A0, A1, A2 etc with 16 being B0 etc. I tested all 15 pins – and they all do PWM!!! (0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 14, 24, 27, and 28). The instructions in this link refer to the “Maple” – so it may be this is a good source to get started with. The only real difference up to now between this and the Arduino apart from superior PWM is the need to actually set the pin up for this – i.e.

pinMode(9, PWM);

This entry will evolve as I learn more – assuming I can get this board to be an I2c slave it could well be quiet useful.

A busy Week

September 23, 2016GeneralPete

It has been a busy week here in the cave and as well as reviewing the likes of the new Sonoff TH16 units I’ve been doing various improvements – not to mention getting to grips with the VT100 terminal commands, squeezing the best from the somewhat dwindling Spanish heatwave and getting annoyed because the tracking notice says that my highly anticipated STM32 board was delivered weeks ago – which it wasn’t!

So – to the HC2016 home control software, for example, I’ve added a new toggle function for the Sonoffs – and a general toggle function to any of the outputs. I’ve also made the button toggle for Sonoff permanent (i.e. if you toggle a light on – lose power, reconnect – the light will come back on).

On top of that I’ve been working with Aidan on the little NanoPi Neos which are a fantastically cheap little board but FriendlyArm don’t supply Debian for it – thankfully the fellows at Armbian do and we’ve been updating the script to take into account Armbian for the Neo – as of last night (as well as sorting an issue with Webmin that needed updating anyway in the script and adding in some other stuff) the updated script can produce a working NEO (from Armbian (Debian) Jessie base) complete with all the usual tools including Webmin, Node-Red with lots of nodes including my own BigTimer and others, SQLITE3, NPM, MQTT with websockets and a ton of other stuff – see the updated script on Bitbucket. (27/09/2016 both script and blog updated. I now have two working NEOs (by name – nanopineo and nanopineo2 – so I don’t have to worry about IP numbers).

I’ve also been resurrecting IOT Manager – now that the Russian author has eliminated the need for websockets – this Android app has promise and I’ll be blogging about it shortly – for now there’s a refresh problem and I spoke to the author about it at the weekend – he’s aware and has made some fixes to the BETA but it still has a couple of minor items that need fixing before it is ready for big time. It would help if I spoke Russian. Just back from a short break to Seville. For now, hopefully there are enough links in here to keep anyone new to the blog busy for a while.

I need to get my hands on an ESP-32 dev board – the limits of .TEXT RAM on the ESP8266 are getting on my nerves.

Have a nice weekend!

Sonoff TH10 and TH16

September 20, 2016Mains switch, sonoff, Sonoff TC16, Sonoff TH10, Temperature sensingESP8266, mains control, Scargill, SonoffPete

You will have seen me write about the excellent Sonoff ESP8266-controlled mains relay switching devices in the past and I went into great detail as to how to program them using the HC2016 software, the only caveat being they don’t have enough FLASH memory for OTA (well, not with my software). Oh yes and I HAVE added that “button control” that people asked for.

Well, here are two new boards – the Sonoff TH10 and TH16 – if you look on their website – full information including schematics etc. are freely available.

Before we start – here are my previous blog entries Sonoff

Slampher and Sonoff
More Sonoff
Even More Sonoff
Sonoff to Mains Block
32Mb ESP01 and Sonoff

Compared to other boards, several of us have discussed that the Sonoff boards are well made with good clearance on tracks for mains power etc. Some doubted whether their small relays would truly handle the loads claimed.

Well, it looks like ITEAD have taken this on board as I’ve just received both TH10 and TH16 modules. As far as I can tell the only difference is that the TH10 has a 10 amp relay, the TH16 has a 16 amp relay.

First impressions? Solid. They are bigger than the original Sonoff modules (which I use on a daily basis for controlling lamps etc) but also much more substantial looking – the programming button is much easier to access as it is intended to be used rather than looking like a reset button. I’ve made good use of this as an output toggle (which also works on the previous Sonoffs).

As usual I’ll leave it up to others to tell you what you can do with the Sonoff software – first thing I did was to reprogram the boards to handle my own software.

So – the output control is identical to the older models – but they’ve also added in a temperature port. They have very kindly made the input port compatible with my software (see settings below) and options are temperature only using the Dallas temperature sensor which they can provide on a waterproof lead complete with stainless tip – or the DHT22 which also handles humidity – I’m pleased to see they did not use the inferior DHT11 which isn’t that accurate.

So – see the original article – you’ll see there how to program up the board – this time – the ground and power connectors are blindingly obvious on the end of the board – that’s 3v3 not 5v – I got away with using an FTDI at 3v3 but you may want to consider a proper 3v3 supply for programming.

If you set wifi_button to 0 – sonoff to 1, temp_type to 0 (Dallas) or 1 (DHT) and temperature_port to 14 (these are non-volatile) you will be able to control the output on out12 (i.e. out12:0 or out12:1), you will be able to read the temperature AND you can toggle the output on and off with the external button on the Sonoff!

The CE and ROHS stickers should keep those who care about such things happy – the important thing for me is looking at the PCB around the mains it is clearly obvious that thought has been put into this. Well done.

Oh yes, the FLASH – like the unit before it – we’re looking at an 8Mb FLASH – that is 1MB – I don’t really support that for OTA though everything works except OTA. I am pretty sure that like other boards, this FLASH if you really must, can be replaced but be warned my soldering isn’t bad and I’ve done lots of Sonoffs, ESP-01s etc – but the little tracks came clean off when I tried it on one of these boards, immediately trashing it – so do so at your own risk.

As for the push connectors for mains in and out – not everyone’s cup of tea and I did have a chuckle that they’ve gone to the effort of avoiding using a screwdriver for the cables – but you have to use a screwdriver to get the cover off !!!!

VT100 Terminal for HC2016

September 15, 2016ESP12, ESP12E, ESP12F, ESP32, ESP8266, ESP8266 SDK, Raspberry Pi 2, Serial TerminalMQTT Terminal, Serial Terminal, VT100, VT100 TerminalPete

Well, I had to give it a go, didn’t I – porting the code for the cheap VT100-type serial terminal into the main ESP8266 home control software.

BOY was that difficult but… after 2 days of head-scratching – check out the home control manual in the source code repository for Home Control 2016 – this uses up four precious port bits (GPIO 13,14,15,16).

I have the terminal code up and running (minus baud rate controls… and the bottom line is now a general status line) and I have to say, fast doesn’t start to explain it.

As you can see in the image above, all we have here is the display with a slim ESP8266 board behind it – a WEMO D1 does the job superbly and you can use double sided tape to fasten the ESP8266 to the flat part of the board. I just used a board I had handy The FTDI you see attached to the back is merely there to provide 5v power and, erm, stand the unit up!!!

At this point I’ll add what I seemed to have missed out of the original article – the pins!

Connections: Connect VCC to 5v, ground to ground, light to 3v3. GPIO16 to D/C, GPIO15 to CS, GPIO14 to SCK and GPIO13 to SDI (MOSI). Connect RESET to RESET. Leave the SDO (MISO) on the display unconnected. The whole thing should light up.. and when you give the {ili_set:1} command and reboot the ESP, the display should clear and put up the header. That’s it.

I did some tests to see if how fast I could make it - I’ve already optimised the SPI and character routines – the board will not operate at all under ESP8266 at the fastest SPI speed so that is probably a dead-end, I tried caching the ROM reads (which are 8 bits – meaning you have to read 32 bits at a time and select the right 8 bits.

Caching that info actually very marginally slowed things down – I tried all sorts, writing 16 bits at a time to SPI – and finally after being able to obtain no more speed improvements, I stopped – not sure why I needed any of this as it was already blazingly fast. Now, writing an X to every location on the screen (that’s 1600 character writes) takes 330ms so that is 200us per character (5*7). I think that is fast enough for now. Clear screen is near enough instant and scrolling is instant.

See this demo video of the ESP8266 version – the 328 version isn’t quite THIS fast but it is still fast.

https://www.youtube.com/watch?v=YXqLVmoyKPE

So I’ve added some commands in the HC2016 project code

{ili_set:1}
The above will setup the display and make it available to accept data – once set the display will set itself up on power up of the unit. Setting that command to 0 stops any data and from there on the display will NOT initialise on powerup.
{ili_text:"\e[35;40mHello there \e[32;40mmate\r\n"}
{ili_status:"All working exceedingly well"}
{ili_title:"\e[37;40mESP8266 Status Monitor"}

In addition to the above, {ili_rssi} puts the time and date down in the status area AND puts a nice phone-like RSSI indicator on the bottom right, showing the current WIFI signal strength of the ESP board. {ili_reset} resets the display after clearing it – to show the header and the LED rectangles on the top.

If you want to experiment with lines – and remember this is going via MQTT so don’t plan on making complex objects… {ili_line:x1,y1,x2,x2,colour} but YOU WILL need to clear the screen first - you can do that with {ili_text:"\e[2J"} and when you’re done you can return the display to normal with {ili_reset}

{ili_pixel:x,y,colour} will draw a dot. {ili_rect:x1,y1,x2,y2,colour,background_colour} will draw a filled rectangle.
{ili_string:x1,y1,string} will position a string at an arbitrary location
{ili_fore:colour} will set the foreground colour for future strings
{ili_back:colour} will set the background colour for future strings

All of the above require that you clear the screen – you cannot do the scrolling terminal – and arbitrary text and lines at the same time but this does add a lot of flexibility (which I thought of much later than when I wrote this article originally).

The manual that comes with the bitbucket download is updated with new commands.

Hence by firing serial or MQTT commands at the board, I can get stuff onto the display. To monitor all MQTT traffic was easy – over to the Pi and MQTT..

In Node-Red – a simple MQTT node picks up all traffic, passes it to a function – which then passes the result to the board which in this case is called “thisone”.

tmpPayload=” “ + msg.payload;
tmpTopic="\\e[36;40m"+msg.topic+"\\e[32;40m\r\n";
if (msg.topic!="thisone/toesp")
{
msg.topic="thisone/toesp";
msg.payload="{ili_text:\"" + tmpTopic + tmpPayload + "\r\n\"}";
return msg;
}

Hence the board gets all traffic except for traffic destined for the board (which could end up in an infinite loop).

And now thanks to a conversation with reader Bob Green – a WIFI signal strength (RSSI) indicator for the board and the time in the bottom left. I deliberately left the seconds out as this would typically not be refreshed that often – maybe every couple of seconds...

Bob suggested that by plugging the board into a battery pack you have a simple range tester and he’s absolutely right of course. Now, how to teach it to make coffee…

On the subject of terminals

No need to talk about VT100-type terminals for PCs – they’re coming out of our ears – but…

Android

I’d put this all together and I thought…an 80 line or 132 line version of this would be nice – I’ll put one on my Android phone. Well, you may well write in and tell me I’m wrong but I cannot find a VT-100 compatible serial modem for Android anywhere (I did find one but it was not clear if scrolling regions worked and it had a limited range of serial settings). Surprising considering that it can be done on a relatively simple unit like the ESP8266

Linux

And that led me to Linux - or rather - I was thinking about the various single board computers I have lying around - a Pi would do but I have a rather nice FriendlyArm NanoPC T2 which was not quite fast enough for Android but runs a nice Debian. I started looking for fancy graphical terminals - not a lot - nothing to anywhere near TOUCH some of the stuff on Windows - however I had this T3 with a little 7" LCD touch screen lying around and was determined it would not go to waste.

It turns out that the humble LX terminal does at least some VT 100 - but that pulls up a command line and lets you interact with it -was there any way to get it to talk to serial instead - preferably one of it's own rather than a USB serial as it has FOUR serial ports on it.

Well, yes. I discovered this..

cu -l /dev/device -s baud-rate-speed

It looked as if this would let me use the terminal with serial instead of a keyboard- but of course when I tried it using:

cu -l /dev/ttyAMAT3-s 115200

I got zilch. The system didn't have a clue what CU is (neither did I for that matter).

Anyway, just for a laugh I tried SUDO APT-GET CU

And it worked. I tried again. THIS time all was well - but it could not contact the serial port - as it was "busy" - yeah right.

I added user FA (the user in control of that terminal session) to the relevant group - no difference - so as is often the case I just gave 777 permissions to the serial port and VOILA.

I tested some colour escape sequences from my PC Serial Terminal I wrote some time ago (and just recently updated to let me put ESC in there) and all worked well but for some visible representation of the ESCAPE sequences (which still worked). I continued experimenting and the UX terminal that comes with Debian LDXE does not suffer that particular issue – so it has the job!!!

Cheap Serial Terminal

September 13, 2016Arduino, Serial TerminalPete

How about a cheap serial terminal for less than a fiver all-in?

In a previous blog entry I’ve been working on more developments for the increasingly powerful home control system using an Arduino as a kind of universal I2c peripheral to the ESP8266. The reason for that is just the vast range of driver software our there and the fact that from China you can get an Arduino-Nano-type-board for under £2. There are drivers for just about everything so it seems daft to just ignore that. It ends up cheaper than most actual i2c peripheral boards.

First thing I had to do with the Arduino base code was increase the size of the buffer used for storing I2c and that is covered elsewhere – but finally I got that working. In the process that led me to the idea of a prototyping board with both the ESP8266, running the home control 2016 software AND a 328 to power some peripheral devices and give extra IO pins.

In the process of THAT I got the QDTECH boards running (128*160 LCD – cheap) – and that also is covered in a previous blog entry. These are great – and cheap (though not as cheap since Banggood jacked the price up but thankfully there is still AliExpress – thanks to Squonk for that particular link) but they are a little limited in terms of resolution as any kind of terminal. I even found a little bit of code to do vertical scrolling. Fast enough but then it is a low resolution display.

In the meantime however I pulled out another favourite display of mine, the ILI9340-driven displays at 320*240 – at £3.47 – mind you if you go to the Italian version of Aliexpress they seem to go down to just over £3. The increased resolution of these means you really COULD make a nice monitoring terminal but for one thing – speed. I checked out several implementations including the Adafruit ones and the drawing speed is atrocious – attempts to scroll the display are generally abysmal unless you use a fast processor.

And so it was that I stumbled on THIS article. The funny thing is that this is on an Arduino site – but points to an article that is anything but Arduino –the fellow has written some FAST code to chuck out small text in the form of a (subset of a) VT-100-compatible terminal. But the code needed you to use the C++ compiler on the command line, NOT in the Arduino environment – and that was no good for me as I have to add other stuff. But if you take a look, you’ll see my late hours have been FAR from wasted.

In the picture on the right (depending on your screen) you see this terminal running – it has 2 fixed areas and a scrolling area and it is fast.

The first problem was that this needed to be compiled outside of any environment and it simply would not work for me – the linker had issues and so on. I realised if this was to be useful – it needed to be inside the Arduino IDE where I could shove in all sorts of other code including my I2c connection.

I opened up a NEW Arduino project, copied all the files in there and renamed the primary file to be the same as the project. I then put the contents of his MAIN() code into “LOOP” – pulling out initialisation and putting that into “SETUP()”. Stunningly, it all SEEMED to work – I just don’t have that much luck normally. However any attempts to use the serial port failed miserably. So I threw away his serial code and put in the normal Arduino serial – that worked just fine – but when I actually tried to do anything with it other than “hello world” – I got garbage characters.

I was reminded of the work I’d done to expand the I2c buffer on the Arduino and that reminded my that the normal Arduino serial buffer is a staggeringly useless 64 bytes. I’m including hopefully helpful links here as you can fix this stuff. Despite the limits of the Arduino default software settings, I need way more than that so I increased the buffer to 1K, using up a significant amount of Arduino RAM. Voila – perfect.

As it happens I’ve been experimenting on a 1284 chip which while being Arduino-compatible kind of, it does have another serial port, so without disconnecting my FTDI – I hooked up the second serial port to the home control serial and fired the DEBUG command at the latter.

As you can see on the left, a perfectly usable terminal and now with programmable on-screen LEDS and baud rate control stored in EEPROM. At this point despite YEARS of VT-100 wilderness I’d mastered the colours and scrolling area controls – thanks to this handy VT-100-related link.

I’d even noticed that in the code, there was a fast horizontal line command. Putting the driver code side by side with the Adafruit code for the same chipset, there were similarities. I should be able to drop in a PIXEL command to get a dot up. YES, with slight mods it worked. For my next trick I would add in a fast VERTICAL line routine from Adafruit – and then – the general LINE drawing routine and the world would be my oyster.

Erm, not quite – the Adafruit SWAP routine caused me endless issues – but I replaced that and all is now well. What’s important here is that this is FAST. I’ve put a little character count on the bottom status line so we know that something is actually happening in the case of repeating commands… and this could form a great plug-in for our prototyping board – or indeed any kind of experimental setup where having a dedicated little serial monitor costing next to nothing would be handy.

The code is currently sitting on BitBucket. Oh - and if you use these displays, don’t make the mistake of thinking, as they run on 5v that the LED power should be 5v. It works for a while but I just burned a LED out on the larger display – and yes I’d noted that it was running a tad warm and didn’t twig. Now the LED supply is connected to 3v3 and it is just fine (apart from the one bust LED). I could probably do PWM brilliance control but that’s something for later. Right now I’m running the entire display on 3v3 and it is just fine. My tiny 3v3 regulator on my boards isn’t even breaking a sweat.

https://bitbucket.org/scargill/experimental-terminal

In the VT-100 command-set, they refer to a command to turn LEDs on and off, 4 of them in fact and I thought that might be fun to implement – now bearing in mind I’ve no prior experience of the switch statements used in the VT-100 code I managed to get this lot running this morning.

On the top right of the image here, you’ll see 4 simple rectangles in red – filled or not. The state of those LEDs can be controlled as follows…

ESC [ parm q

Where parm is one of 5 values 0=all off, 1=one on, 2=2 on, 3=3 on, 4=4 on.

That apparently is the standard though WHY it won’t let you turn an individual LED off I don’t understand so I’ve extended this so that 5 turns off LED1, 6 turns off LED2 etc.

That’s how I’m interpreting that particular VT-100 command.

I’ve added a non-standard command to set baud rate as I could not find a standard command to do it. This is stored in EEPROM.

ESC [ parm X

Where parm= 1,2,3,4,5 or 6 for 300,2400,9600,57500, 76800 or 115200 baud. Why that 76K? Because on power up that’s the speed the ESP8266 chucks out debug info.

Developments

I now have code running native on the ESP8266 to handle this display and the terminal – so as not to detract from this article I’ve split this into another article – but you can see the first attempts running here… https://www.youtube.com/watch?v=qBxXZcleKq0

HC 2016 Experiments

September 6, 2016Arduino, ATMEGA328, ESP12, ESP12E, ESP12F, ESP8266, Espressif, home controlPete

Throughout the development of the Home Control 2016 project I’ve constantly had the software tied a via a serial umbilical to my PC to monitor passing messages, make sure all is well etc. and to periodically check on RAM space to make sure nothing I’m doing is running away with precious memory.

You may have seen elsewhere that I’ve been working with a prototyping board which is basically an ESP12 with a prototyping area in which I’ve put rows of pins for ground, 3v3, 5v and the two lines I use for I2c as I add various peripherals culminating in the recent idea to use an Arduino 328 chip as an “ultimate peripheral”.

Well, we’ve taken that one a stage further and we’ve been discussing making our own prototyping board (Aidan’s a whizz at PCBs and of course we use the likes of Dirty PCBs in China so it is quite cheap to knock up a few boards) and the last few days I’ve been formulating a planas Aidan puts together the schematic. What I’ve realised is that I always run out of the power and ground lines no matter what I do. Add to that the FTDI so I can monitor serial Comms and it all gets a bit messy.

So right now we’ve been discussing a board with the ESP-12 module with lots of pins and additionally a 328 SMT chip with Xtal. Of course that means 2 serial monitors. I’ve been playing with the old QDTech 128*160 displays since I discovered that they work pretty well when driven from the proper SPI pins on Arduinos (don’t even think of using software SPI on them) and ramping up the SPI speed – and so it was that in conversation, we were saying “if only you could scroll them” – that led to thoughts of having a RAM buffer larger than the total available on the ESP8266 or Arduino and… well, I thought I’d use our friend Google to go take a look. It turns out – I was completely unaware of this and VERY pleasantly surprised to find – that the QDTECH chip has the ability to scroll up built in – all that is needed is to scroll up an area then paint a fast horizontal black line after each scroll.

There is a version of the driver for the board here and at some point in 2014, it turns out that an M J Sage added this scrolling ability. Well done. I wonder if he or she realised what a useful addition this could be.

So now we have a display, easily driven by the 328 which can scroll constantly (I’ve had it on several night’s test) and hence provide a boatload of status info. Top and bottom can if needed remain static.

We’re going to incorporate the display onto the prototyping board and it won’t take any additional space as it will sit neatly over the other components! The image above probably bears no relation to the final board but I’m just trying to convey an idea here. The prototyping area on the right of the board will be chock-full of PTH holes but there will be long runs of connected holes for power and I2c as these end up being needed the most with connector wires. All 0.1” of course.

With the QDTech display you can connect the reset line to the 328 reset and hence here are my first thoughts at the use of the 328 which will have access to A6 and A7. I’ve pretty much got this working – just need to add in the temperature handling code I already have for Dallas and DHT chips.

D0-7 – general purpose port extender – inputs or outputs or any mix
D8     D/C for QDTECH
D9     16- BIT PWM out
D10 16-BIT PWM out
D11   MOSI for QDTECH
D12   MISO
D13   SCLK for QDTECH
A0      CS for QDTECH
A1      Temperature
A2      Temperature
A3       Debounced input
A4      I2C to ESP
A5      I2C to ESP
A6      Analog in 1
A7      Analog in 2

Aside from losing GPIO2 and GPIO14 inputs (as they’ll run the I2c) the ESP use will remain the same. The only restriction being that we cannot run ESP PWM at the same time as the I2c due to timing interference– a pain but not something I can see away around. The I2c 16-channel PWM controller does seem a good way around this as it offloads all PWM work onto a cheap board – for single colour PWM lighting the new 16-bit PWM on the 328 works fabulously.

Given that the 328 can handle inputs for the main board we can do away with our normal GPIO2 and GPIO14 and they can have the job of I2c, leaving all normal outputs as for the existing system.

I guess the end-game here will be the smallest possible board that contains both the 328 and the ESP wired to this spec for general use. But that comes later after we see how well all of this works in practice. First the prototyping board. Watch this space.

Sadly in the link I’ve enclosed, the author refers to Banggood who used indeed to offer these displays for under £3 at a time when there were no software drivers for them – (I think we paid £2.60) but I’ve noticed recently that Banggood have been getting greedy and jacking their prices up – a cursory check of the link shows they now want £5.48 for these simple LCD displays – well good luck with that Banggood.

The only ones I’ve found up to now – and the price is good – as usual – is AliExpress.

Someone have another link? the board has an 8-way connector at one end and a larger connector at the other – with a full size SD socket on the underside. See the AliExpress link.

This blog entry could well change dramatically as we spot fatal flaws in the plan above Smile

Arduino 16 bit PWM Revelation

September 5, 2016UncategorizedPete

Here is a piece of code I found – quite old now. As most Arduino users will know, the PWM on the 328 chip is a bit… naff at only 8 bits. When attached to LEDS, at the low end, each step is painfully obvious – even value 1 is visible immediately – not then much use for smooth lighting!

However I discovered this..

void setupPWM16() {
    DDRB |= _BV(PB1) | _BV(PB2);        /* set pins as outputs */
    TCCR1A = _BV(COM1A1) | _BV(COM1B1) /* non-inverting PWM */
        | _BV(WGM11);                   /* mode 14: fast PWM, TOP=ICR1 */
    TCCR1B = _BV(WGM13) | _BV(WGM12)
        | _BV(CS10);                    /* no prescaling */
    ICR1 = 0xffff;                      /* TOP counter value */
}
/* 16-bit version of analogWrite(). Works only on pins 9 and 10. */
void analogWrite16(uint8_t pin, uint16_t val)
{
    switch (pin) {
        case 9: OCR1A = val; break;
        case 10: OCR1B = val; break;
    }
}

That lovely piece of code lets you put 16-bit PWM on ports 9 and 10 at the expense of TIMER 1 – MARVELOUS unless you want to do RGB lighting – as there are only two.

So I kept on looking. The 1284P chip – compatible but more powerful than the 328, has an extra 16 bit timer (ignore the blogs that say it does not – the guys are reading out of date info). It occurred to me that you should be able to use Timer 3 to get an extra 2 channels at 16 bits – wouldn’t that be marvellous.

Sadly there’s a problem – lack of info on Timer 3 meant that even after guessing the changes needed to that setup (ports 6 and 7 not 9 and 10) I managed to get the new PWM on port 7 – but not 6. And worse – they’ve gone and stuck this in the middle of the pins for SCK, MOSI and MISO so you can’t do both at the same time. Ah well. I simply changed all references to timer 1 to timer 3 to get port 7 working but that didn’t seem to help port 6.

So I was wondering – the 2560 boards – they have even MORE timers. Has anyone taken this basic code and managed to get more 16-bit PWM timers on the the MEGA2560 boards?

Espressif Save to FLASH–an alternative

September 3, 2016FlashAnother way to update FLASH ESP8266Pete

In this article I'll talk about making use of FLASH to store information in the ESP8266 wireless units (I refer here to those with 4Meg or more of FLASH – on the assumption that only the first 2 meg is in use plus a tiny bit at the end)…

To clarify - The ESP can make use of 1 meg of FLASH for program space, plus another meg if using OTA (online updating) - that is- the second meg is used to upload a new program while the original is still running. In addition Espressif use a tiny bit at the very top of whatever size FLASH you have. I'm going to talk here about those units with 4MB of FLASH - for example the very popular and cheap ESP-12 and variants. If you had a unit with MORE memory then this could be expanded. LESS and this isn't worth reading.

Espressif, in order to arrange for secure user data – use three 4k blocks…

The first two contain data up to 4096 bytes, the third says which block is the current one in use – accordingly that block is mostly wasted.

So when you want to update the data – you see which one or the two is NOT current – wipe it… and then update it, then update the third block with the pointer.

So you use 3, 4k blocks to get one good block of data. And I have to say it works, 100% – I’ve never lost data. Should the write procedure fail part way through due to power cut or whatever, all you lose is the latest update. The current block remains the previous one you were using.

In the SDK there is a routine to use this - I recall TUANPM had his own but then when Espressif incorporated this into the SDK I moved to their routines. They work perfectly.

The only issue with this is the waste of a block which if taken to extreme – using other spare blocks of FLASH would be quite wasteful. I thought of the various existing file systems out there and figured they were overkill for my needs and not necessarily as secure as this.

So I had this idea to take TWO 4k blocks and use the top 32-bits of each as a counter.

So to read a block – look to see which of the two has the highest number that is NOT FFFFFFFF - that is the block to read.

When writing, to whichever is NOT the current block – write the data including an incremented count (avoiding 0 and FFFFFFFF). As the count is the last thing to be written, any failure due to power loss will surely result in that 32-bit number NOT being updated – OR being reset to FFFFFFFF. In which case we stick with the current block and hence only the latest update is lost – just like the Espressif version but only using 2 blocks.

Reader Gary came in with a slightly better idea – to write the block – whatever that may be with the first 4 bytes set to FF. (you must write on 4-byte boundaries to avoid the ESP having a fit). Once the sector is written you can then go back and fill those 4 bytes in – because overwriting with 0 is always ok – what you can’t do is over-write with 1 as that is the function of the erase-block routine (4k). I’ve now tested this –checking adjacent 32-bit words – it all works a treat.

The next stage was to formulate this into something tangible…

To read into a structure in RAM

Check both FLASH blocks first 4 bytes.. if both FF – current block is first else if one block is FF – current block is other block else current block is highest.

If not new, Read struct size in from relevant offset into block. If new, fill struct with FF !! No point in reading all.

To write a structure to FLASH

Check both FLASH blocks first 4 bytes.. if both FF – this is NEW. Else if either is FF current block is other – else current block is highest.

If NEW – ERASE first block, write data into relevant part of FLASH – write number 1 into bottom 32 bits. Check – report any failure.

Otherwise read current block into 4K RAM buffer. FFFFFFFF into bottom 4 bytes. Erase OTHER block, write RAM structure with updates to new block. Take original counter – increment – write to bottom of new block. Check, report any failure.

Apart from that double read-write which might be small depending on your structure, this seems reasonably efficient.

And the result…

my_flash_read(sector, offset,buffer,buffer_size)

and

my_flash_write(sector,offset,buffer,buffer_size)

Assumptions for the above being that the sector would in fact be blocks of two sectors – so from the start of the third Meg to nearly the end of a 4 Meg FLASH (ESP-12 for example) you’d be able to use sectors 0-253 of protected data. The offset would be the offset from the start of the sector where your data is to go, the buffer the address of your buffer and the buffer_size to be no more than 4092 bytes. Ok, so any block can only be written to, maybe 100,000 times - but if you wanted to – you could keep track of that.

For those interested the working test code is here – not the most concise – but it works. Given it a good hammering this morning – yet to decide how best to use this – but it’s a great move forward from thinking of that extra space as a great black hole. I've just put in a second version of the WRITE routine that does NOT use a 4K buffer - initial testing suggests it is working fine.. mind you - the tests are small.

// Some assumptions-  your offset is on a 4-byte boundary...(size doesn't have to be) - your sector is 0-253 (for the top 2 meg)
// and your offset + buffer does not go over the 4096 (remember - first 4 bytes used up.

uint8_t IFA my_flash_read(uint32_t fsec,uint32_t foff, uint32_t *fbuf,uint32_t fsiz)
{ //READ
uint32_t first;
uint32_t second;
uint32_t current;
uint32_t structsize;
uint32_t flashbase;
uint32_t flashoffs;
uint32_t tmp;
uint8_t good;

if (fsec>253) { return 0; } // assuming starting at third meg and can't touch very top 4 secs (2 for each of these)
uint32_t flashStart=(fsec*2)+0x200000;

// size of struct must be rounded up - offset from base=arg1, ram buffer = arg2,
// size=arg3
// when writing if length not 32-bit rounded, will write to nearest boundary up

flashbase=foff&0xfffff000;
flashoffs=foff&0xfff;

structsize=fsiz; if (structsize&3) structsize+=(4-(structsize&3));
current=4096;
spi_flash_read(flashStart+flashbase,&first,4);
spi_flash_read(flashStart+flashbase+current,&second,4);

if ((first==0xffffffff) && (second==0xffffffff)) // ie all new
	{
	good=0;
	spi_flash_erase_sector(0x200+(flashbase/4096));
	tmp=1;
	spi_flash_write(flashbase+flashStart,&tmp,4);
	spi_flash_read(flashbase+flashStart,&tmp,4);
    current=0;
	if (tmp==1) good=1; else good=0;
	}
else if (first==0xffffffff) current=1;
else if (second==0xffffffff) current=0;
else if (second>first) current=1;
else current=0;

// can't read whole in once go if struct not on 4 byte boundary

current*=4096;
if (structsize==fsiz)
	{
    spi_flash_read(flashoffs+flashStart+4+current,&first,4);
	if (first!=0xffffffff) good=1; else good=0;
	spi_flash_read(flashoffs+flashStart+4+current,fbuf,fsiz);
	}
	else
	{
    spi_flash_read(flashoffs+flashStart+4+current,&first,4);
	if (first!=0xffffffff) good=1; else good=0;
	spi_flash_read(flashoffs+flashStart+4+current,fbuf,(fsiz&0xfffffffc));
	spi_flash_read(flashoffs+flashStart+4+current+(fsiz&0xfffffffc),&tmp,4);    //// CHECK
	memcpy(fbuf+(fsiz&0xfffffffc),&tmp,(fsiz&3)); // move those last 1,3 or 3 bytes
	}

return good;  // so you know if it is the first time or not.
} // done with READ operation - phew!!


// and now to tackle writing - same rules as reading - this version uses a 4k buffer

uint8_t IFA my_flash_write(uint32_t fsec,uint32_t foff, uint32_t *fbuf,uint32_t fsiz)
{ //WRITE
uint32_t first;
uint32_t second;
uint32_t current;
uint32_t structsize;
uint32_t counter;
uint32_t tmp;
uint32_t flashbase;
uint32_t flashoffs;
uint8_t good;
uint8_t bigbuf[4096];

if (fsec>253) { return 0; } // assuming starting at third meg and can't touch very top 4 secs (2 for each of these)
uint32_t flashStart=(fsec*2)+0x200000;

flashbase=foff&0xfffff000;
flashoffs=foff&0xfff;

// size of struct must be rounded up - offset from base=arg1, ram buffer = arg2,
// size=arg3
// when writing if length not 32-bit rounded, will write to nearest boundary up

structsize=fsiz; if (structsize&3) structsize+=(4-(structsize&3));
current=4096;
spi_flash_read(flashbase+flashStart,&first,4);
spi_flash_read(flashbase+flashStart+current,&second,4);

if ((first==0xffffffff) && (second==0xffffffff)) current=0;
else if (first==0xffffffff) current=1;
else if (second==0xffffffff) current=0;
else if (second>first) current=1;
else current=0;
	{
	good=0;

	spi_flash_erase_sector(0x200+(flashbase/4096)+(current^1)); // erase the OTHER one
	current *=4096;
	spi_flash_read(flashbase+flashStart+current,bigbuf,4096);
    memcpy(&counter,bigbuf,4); // copy counter
	memset(bigbuf,0xff,4);
	memcpy(bigbuf+4+flashoffs,fbuf,fsiz); // write the new data into the buffer
	if (current) current=0; else current=4096;
	spi_flash_write(flashbase+flashStart+current,bigbuf,4096);
	if (counter==0xffffffff) counter++;
    counter++;
	memcpy(bigbuf,&counter,4); // copy counter back
	spi_flash_write(flashbase+flashStart+current,bigbuf,4);
	spi_flash_read(flashbase+flashStart+current,&tmp,4);
    if (tmp==counter) good=1;
	}
return good;  // so you know if it is the first time or not.
} // done with WRITE operation 

// this version of writing does NOT use a 4k buffer - seems to work just as well.

uint8_t IFA my_flash_write_no_buffer(uint32_t fsec,uint32_t foff, uint32_t *fbuf,uint32_t fsiz)
{ //WRITE
uint32_t first;
uint32_t second;
uint32_t current;
uint32_t newcurrent;
uint32_t structsize;
uint32_t counter;
uint32_t tmp;
uint32_t flashbase;
uint32_t flashoffs;
uint8_t good;

if (fsec>253) { return 0; } // assuming starting at third meg and can't touch very top 4 secs (2 for each of these)
uint32_t flashStart=(fsec*2)+0x200000;

flashbase=foff&0xfffff000;
flashoffs=foff&0xfff;

// size of struct must be rounded up - offset from base=arg1, ram buffer = arg2,
// size=arg3
// when writing if length not 32-bit rounded, will write to nearest boundary up

structsize=fsiz; if (structsize&3) structsize+=(4-(structsize&3));
current=4096;
spi_flash_read(flashbase+flashStart,&first,4);
spi_flash_read(flashbase+flashStart+current,&second,4);

if ((first==0xffffffff) && (second==0xffffffff)) current=0;
else if (first==0xffffffff) current=1;
else if (second==0xffffffff) current=0;
else if (second>first) current=1;
else current=0;
	{
	good=0;
	spi_flash_erase_sector(0x200+(flashbase/4096)+(current^1)); // erase the OTHER one
	current *=4096;
	if (current) newcurrent=0; else newcurrent=4096;
	if (current) counter=second; else counter=first; if (counter==0xffffffff) counter++;

	tmp=0xffffffff;
	spi_flash_write(flashbase+flashStart+newcurrent,&tmp,4);	// write a blank counter

	uint32_t tstart,tstart2,tend;
	tstart=flashbase+flashStart+current+4;
	tstart2=flashbase+flashStart+newcurrent+4;

	tend=flashbase+flashStart+current+4+flashoffs;

	while (tstart<tend)
		{
			spi_flash_read(tstart,&tmp,4);
			spi_flash_write(tstart2,&tmp,4);
			tstart+=4; tstart2+=4;
		}

	spi_flash_write(tstart2,fbuf,structsize);
	tstart+=structsize; tstart2+=structsize;

	while (tstart<4096)
		{
			spi_flash_read(tstart,&tmp,4);
			spi_flash_write(tstart2,&tmp,4);
			tstart+=4; tstart2+=4;
		}


    counter++;
	spi_flash_write(flashbase+flashStart+newcurrent,&counter,4);
	spi_flash_read(flashbase+flashStart+newcurrent,&tmp,4);
    if (tmp==counter) good=1;
	}
return good;  // so you know if it is the first time or not.
} // done with WRITE operation

ADS1115 I2C A/D Convertor

September 1, 2016ESP8266Pete

Coming up next on HC2016 project - the ADS1115 I2C 4-channel A/D convertor.

This item came from AliExpress – a very nice purple board – very clean – and importantly – very cheap at £1.89.

I started off looking at some Adafruit code for a similar chip and immediately spotted something wrong and noticed some of the code comments seemed incorrect – one of our readers offered a solution but ultimately – as I plan only to use this as a straight-forward 4-channel A/D convertor, to study the Texas data sheet. Page 11 seemed pretty clear – I put ground to ground Vcc to 3v3, the “address” pin to ground (that pin can offer up to 4 addresses amazingly – grounded gives the base address of 0x48. Of course like other i2c 7-bit addresses this will be shifted left by one – but that’s already done in my routines.

As described on page 11, all you have to do is sent a 16-bit command (MSB first) to the config register, a zero to the pointer register then read 2 bytes (MSB first) from the read conversion register. What could be easier.

Having a need for a different number of parameters than before – in this case a receive where only the address is fired out – no parameters but just read 2 bytes back I made a new function which should make it easier to talk to devices in the future..

ic2_general(address,buffer,transmit_length,receive_length);

That way I get to need only one I2c function to do the lot – passing zero or more bytes – and receiving zero or more bytes. I could see me back-fitting older code with this.

And (cutting out lots of experimenting) it works – I didn’t need a library for simple reading – it’s easy.

If you look at page 19 of the data sheet there are all manner of options for the config reg – I chose 4.096v as the full scale input – and using an argument I can setup which input (1,2,3,4) or read (0).

Initially {ads1115:1} - that sets up input A0 and continuous conversion

then

{ads1115:0} – reads the value back.

Now an important point here – if you wire up the unit to 3v3 – you should NOT exceed that – so in my case I ended up with a reading around 26,385 when connecting the relevant input to 3v3 because of course this device can handle positive and negative inputs.

That value, given a top value of 32,767 for 4.096v full range indicates my supply voltage is 3.296. My meter suggests Vcc is 3.28v so one of them is out just a TAD (mind you as the meter only had 2 digits after the decimal point the real value could have been very near 3.29 – so I’m not complaining – the readings were pretty consistent, not varying more than 3 digits and zero volts was showing at value 3 which seems reasonable given my shoddy wiring on this prototype.

All in all – a nice addition for under £2 all in… so in one example I currently measure 12v battery voltage on my Pergola using the ESP built in A/D – now I can measure battery AND solar charger and figure out current and losses from there. That’ll make for some pretty graphs. I’m sure you can think of some much better uses.

Scargill's Tech Blog

Anything to do with gadgets – the stuff that gets me up in the morning full of enthusiasm