Landing on two Feet

feetSometimes I really do land on 2 feet.  So the ESP/MQTT rig now appears bulletproof and in a timely fashion the Raspberry Pi appears just in the nick of time. All I need is an easy to use Mosquitto/Node-Red setup and …. what should appear but a link to this … http://thethingbox.io/

I can see it’s going to be a busy weekend of discovery.

I was wondering how to make the thing programmable without having to blow the flash – and it hit me tonight… give the unit a default unit number to prefix any subscribes… and… use MQTT messages to rewrite the default unit number and any other info – and reboot!! Easy (well, of course, it won’t be).

That’s enough for one day.

Facebooktwittergoogle_pluspinterestlinkedin

ESP12 and More Pins

ESP-12I’m just about to have a go at one of the larger boards – the ESP12 (or 201)…   so can someone save me (and others) re-inventing the wheel.

Here’s what I do to initialise an ESP-01 port bit GPIO0 as an output.

Firstly the setup..

// For GPIO2 just change the 0 to 2 - there is one init line.
#define LED_GPIO 0
#define LED_GPIO_MUX PERIPHS_IO_MUX_GPIO0_U
#define LED_GPIO_FUNC FUNC_GPIO0

then initialisation

PIN_FUNC_SELECT(LED_GPIO_MUX, LED_GPIO_FUNC);

 

and here it is set to 1.

GPIO_OUTPUT_SET(LED_GPIO, OUT_ON);

 

As you can see, I’ve already figured out how to do the same with GPIO2 (though I’ve not used GPIO2 as an output yet)

So – what other pins can we use in exactly the same way and are the numbers as you would expect or different?? This general principle also seems to work for GPO4 and 5 but not for the higher numbers.

Facebooktwittergoogle_pluspinterestlinkedin

Rubbish plug-ins

Another rubbish spam plug-in - apologies to anyone who's been trying to comment this morning - I really do not understand some of the people that give stars to WordPress plug-ins. I used an anti-spam system that got high ratings - and it would not let people comment - when I went on the web, loads of people with blogs had the same issue. It's history now, I'll deal with spam by hand until I find something that actually works...

 

Facebooktwittergoogle_pluspinterestlinkedin

ESP8266 and MQTT Exciting Times

I don’t want to jump in prematurely here but those of you who have been reading the blog will know that I’ve had no end of problems with the MQTT stand-alone software on the ESP8266, but as each problem has emerged, the author (Minh Tuan) has been keeping in touch and working with me.  

MQTTLast week having apparently gotten everything working (that’s a real project continuously firing out temperature, responding to incoming time correction signals and accepting commands to turn GPIO0 on and off,  connected to a solid state relay controlling a lamp) when I realised that one of the real-life tests I’d missed out was … what happens if the broadband went off.

To my horror, it resulted in absolute failure.  The software would not reliably reconnect. I had some great suggestions from people – including one to think outside of the box, could it possibly be the MQTT broker not reconnecting properly. I worked on everyone’s suggestions to no avail.  I even updated my development environment (Windows 8 with Eclipse) to the latest SDK + patch (0.9.5) – no difference.  I can’t tell you now many evening hours I spent on this.

ESP-01Then this morning I had a contact from Minh to say he’d sorted the problem. I downloaded the code – and tested it – turned the WIFI off for a few seconds – turned it back on – HURRAY, reconnect no problem, re-subscribe no problem, queued outputs sent no problem. I was just about  to report success when I thought I’d see what happened if I left the WIFI off long enough to fill/overflow the queue (there is a RAM queue for messages and it includes incoming subscribes and outgoing messages, I had a hunch that a full queue might stop re-subscribes). Sure enough… the grim news on leaving the WIFI off for several minutes and reconnecting was that the software started up again – but all subscribes were lost – no room for them.

Thankfully Minh was around and immediately spotted this – we agreed that if the queue filled up completely and the WIFI restarted, the queue would be SCRAPPED.  Ok there are better ways and I’ve asked him to look at this – but for a quick fix he sent me a replacement mqtt.c file and… I put it to the test.

MQTT[6]3 hours later I cannot crash the software. No matter how long the WIFI is off it always reconnects properly. Some earlier strange issues on power up have disappeared (I feared they might be my own non-volatile variable storage additions.. but no).

The library has been updated accordingly just minutes ago and is here https://github.com/tuanpmt/esp_mqtt.

You really should take a look at this. For the first time we potentially have the means to make an ultra-reliable control system with the ESP8266 boards. Recent mods to the code free up RAM (an issue for example with the Lua software) and so I have no doubt there is lots of room in there to add your own bits.

To recap for those who’re looking in for the first time, anything I refer to here will be covered in other blog pages – but I’m using MOSQUITTO – in my case running on a Diskstation NAS Drive for reliability but it will also run on various platforms and there are servers out there for people to use as well. I’m testing this with the excellent MQTT-SPY and all of this without having learn a word of Linux Smile  thanks to the excellent feedback I’ve had on this blog and the work of others around the world who like me are so keen to get this working.  If all goes well I can now concentrate on mastering the likes of Openhab so that once I have boards in place controlling stuff, I can easily monitor and control the boards themselves from a mobile phone – and for the first time with a little security in place – the point of investigating MQTT in the first place!

phpIncidentally, the PHP page I use under CRON to push the time out to my MQTT broker – I had some fun with that as I realised it was pushing out rubbish (wrong time) and further to a PHP upgrade at my provider – was throwing in warning messages to boot. Here is the latest working version (been up there running 24/7 for a week now) with security information altered to protect the innocent. To keep things simply, the dawn and dusk information is only accurate to the nearest minute. That lowers the number of times it will be updated and as I keep this in FLASH, the number of times the FLASH is updated.

<?php
error_reporting(E_ALL ^ E_NOTICE);
require("../phpMQTT.php");

function varcheck($var, $default)
{
return isset($var) ? stripslashes($var) : $default;
}

$locn = varcheck($_GET[‘loc’], "Europe/London");
$lon = varcheck($_GET[‘lon’], 55);
$lat = varcheck($_GET[‘lat’], -2);

date_default_timezone_set($locn);   
$dateTimeZoneLocal = new DateTimeZone($locn);
$dateTimeLocal = new DateTime("now", $dateTimeZoneLocal);
$localDateTime = date("H:i:s d-m-Y", time());
$localDisplayDateTime = date("H:i l d-m-Y", time());
$localTime=strtotime($localDateTime);

$mqtt = new phpMQTT("your.mqtt.server", 1884, "whateverYouLike");
if ($mqtt->connect(TRUE,NULL,"yourwifilogin","yourwifipass")) {   
            $mqtt->publish("time",$localTime,0);
            $mqtt->publish("timestring",$localDisplayDateTime,0);
            $sun_info = date_sun_info($localTime, $lon, $lat);
            foreach ($sun_info as $key => $val) {
                if ($key=='civil_twilight_end') $mqtt->publish("dusk",(int)(($val %86400)/60),0);
                if ($key=='civil_twilight_begin') $mqtt->publish("dawn",(int)(($val %86400)/60),0);
            }
    $mqtt->close();
}
?>

Hopefully now I can stop worrying about reliability and start having some fun. I have suggestions for better ways to store things in FLASH and there is much to learn about Node-Red, OpenHab and much more so this is really just the beginning.

Facebooktwittergoogle_pluspinterestlinkedin

Raspberry Pi 2 Fast Enough?

The Raspberry Pi is a nice, cheap little computer but for me it has never been fast enough – I’m not interested in making another media server (I have a DISKSTATION which does that just fine) but as I’ve recently been working with MQTT in my home control setup, I did think it would be nice to have an MQTT broker and some ability to send commands to it in one little reliable box. 

Repeated messages out there indicate that the Raspberry Pi is pushing it’s luck to do all of that at high speed as it approaches the limits of speed (and possibly even memory).

What I’ve been looking for is the same thing with much more memory and much more speed. Well, how does (claimed) 6* more speed and twice the memory sound – for the same price?

I’m sure those who have recently purchased the original Raspberry Pi will have a million reasons why it’s not necessary – and I feel sorry for them but now it looks like the Raspberry Pi II (2) at long last has the power I need.

I wonder how long it will take before the price settles to what we would expect – I have NO doubt that early copies will be expensive – that tends to happen in Britain.

And as for Windows 10 – apparently Microsoft will make this available for free – that could be desperation or it could be EXTREMELY good marketing – for those who don’t really want to get into Linux – Windows development + Windows in the embedded code… has to be some mileage there.

More on this when my board turns up!

Facebooktwittergoogle_pluspinterestlinkedin

MQTT SPY

There’s a new MQTT-SPY out – here http://kamilfb.github.io/mqtt-spy/ This program has been invaluable to me throughout all the time I’ve been testing and learning about MQTT. It sits on my desk running constantly and never falls over. Properly set up it can send timed messages out and coming soon – logging graphs – the author is very helpful – and it’s free – what more could you ask for.

 

MQTT-Spy

Facebooktwittergoogle_pluspinterestlinkedin

Memory use ESP8266

Can someone explain this to me..

I’m using the ECLIPSE environment in a PC and only starting to get to grips with the MAKE files – I really still don’t understand most of what’s in there.

I’m looking at the MQTT installation which is one of the easier ones (I don’t understand why the likes of the LUA compilation throws out 18 million lines of un-decipherable gook while the MQTT files churns out very little – only showing you what you need to know about – i.e. errors).

So I understand how non-volatile storage is being done – at the segment 3C000  (I’ll probably get the number of zeros wrong  but that’s not important here).. So we blow some code at 0000, we wipe a little bit at 3C000 – which seems utterly pointless as the startup routines check a checksum at that area to see if it needs wiping/updating…  and then more code at 40000… now that last block seems to fill up rather quickly..

It not possible and indeed more sensible to start at 00000 and work all the way up in one block, leaving, say some blocks at the top free for non-volatile variables. I’d give this a shot myself but I need to understand why everyone splits up the code like this leaving a huge chunk of (unused?) memory down near the bottom.

Facebooktwittergoogle_pluspinterestlinkedin

Slight hitch

Apologies to anyone looking in this morning. I tried to get just a little too clever with the blog and it fell over - so anyone looking in while I was asleep (British time) would have seen a dead site - all working. As this blog is new could you please remember when commenting to tick the little box to say you want emails when someone replies to posts.

Facebooktwittergoogle_pluspinterestlinkedin

ESP8266 MQTT and LOSS OF WIFI

Update at 13:22:00 on 02/02/2015
This morning I had an email to say the WIFI loss was fixed in the MQTT code. I downloaded the latest MQTT software and put my code back into it. I turned the WIFI off for a few seconds then back on – that worked, sending out the message backlog and re-subscribing to messages. I then turned the WIFI off until the 2K queue buffer was full – that I guess is the ultimate test.  That took several minutes. I turned the WIFI back on and…no. BUT we understand what’s wrong. The queue is full – and hence there is no room to re-subscribe. This is now fixed here https://github.com/tuanpmt/esp_mqtt. Everything seems to operate perfectly. That is important for a number of reasons as it suggests that there is nothing wrong with the SDK.

Original blog:

This  is getting to a be a problem.  I don’t know if it is the MQTT software or ESPRESSIFs SDK but there is an issue if you lose WIFI connection.  I tested this on the road the week but now I’m back in the office I’m doing it a bit more scientifically.

So – I know that the WIFI here is perfect.  I also know that the SDK is case sensitive so in order to simulate loss of signal it’s simply a matter of renaming the “office” WIFI to something else or just change case (my computer is hardwired) to test.

Here’s the result of this mornings test.

So what we have is an ESP-01 – I’m using the latest SDK (0.9.5) on a PC using Eclipse and using Tuan’s MQTT software as a base (NOT the LUA version – so I’m programming in C).  The basic MQTT software has a number of subscriptions… including the time.  A separate system updates the time every minute (Linux timestamp – seconds since 1974).  The board has a temperature sensor which on a timer sends out the temperature every 5 seconds (using code that does not involve long delays).

Here is the log.. “in the beginning” the router has the WRONG name so that the board cannot see the WIFI.. It sits in a loop waiting….

 

no office found, reconnect after 1s
STATION_IDLE
STATION_IDLE
reconnect
STATION_IDLE
STATION_IDLE
STATION_IDLE
STATION_IDLE
scandone
STATION_IDLE
add 0
aid 1
pm open phy_2,type:2 0 0
cnt
STATION_IDLE

 

At this point I turn on the WIFI – and all is well, it is picked up – the board starts up and subscribes to the various topics.

After about a minute, it gets the time – and now starts transmitting the temperature (the sensor isn’t hooked up so it’s just sending dummy info)

 

connected with office, channel 13
dhcp client start...
STATION_IDLE
STATION_IDLE
ip:192.168.0.9,mask:255.255.255.0,gw:192.168.0.138
TCP: Connect to domain home.scargill.org:1884
DNS: found ip 109.170.132.114
TCP: connecting...
MQTT: Connected to broker home.scargill.org:1884
MQTT: Sending, type: 1, id: 0000
TCP: Sent
TCP: data received 4 bytes
MQTT: Connected to home.scargill.org:1884
MQTT: Connected
MQTT: queue subscribe, topic"output", id: 1
MQTT: queue subscribe, topic"mytime", id: 2
MQTT: queue subscribe, topic"time", id: 3
MQTT: queue subscribe, topic"dusk", id: 4
MQTT: queue subscribe, topic"dawn", id: 5
MQTT: queue subscribe, topic"peak", id: 6
MQTT: queue subscribe, topic"off_peak", id: 7
MQTT: queue subscribe, topic"frost", id: 8
MQTT: queue subscribe, topic"on_1", id: 9
MQTT: queue subscribe, topic"off_1", id: 10
MQTT: queue subscribe, topic"on_2", id: 11
MQTT: queue subscribe, topic"off_2", id: 12
MQTT: queue subscribe, topic"timestring", id: 13
MQTT: Sending, type: 8, id: 0001
TCP: Sent
TCP: data received 5 bytes
MQTT: Subscribe successful
MQTT: Sending, type: 8, id: 0002
TCP: Sent
TCP: data received 5 bytes
MQTT: Subscribe successful
MQTT: Sending, type: 8, id: 0003
TCP: Sent
TCP: data received 5 bytes
MQTT: Subscribe successful
MQTT: Sending, type: 8, id: 0004
TCP: Sent
TCP: data received 5 bytes
MQTT: Subscribe successful
MQTT: Sending, type: 8, id: 0005
TCP: Sent
TCP: data received 5 bytes
MQTT: Subscribe successful
MQTT: Sending, type: 8, id: 0006
TCP: Sent
TCP: data received 5 bytes
MQTT: Subscribe successful
MQTT: Sending, type: 8, id: 0007
TCP: Sent
TCP: data received 5 bytes
MQTT: Subscribe successful
MQTT: Sending, type: 8, id: 0008
TCP: Sent
TCP: data received 5 bytes
MQTT: Subscribe successful
MQTT: Sending, type: 8, id: 0009
TCP: Sent
TCP: data received 5 bytes
MQTT: Subscribe successful
MQTT: Sending, type: 8, id: 000A
TCP: Sent
TCP: data received 5 bytes
MQTT: Subscribe successful
MQTT: Sending, type: 8, id: 000B
TCP: Sent
TCP: data received 5 bytes
MQTT: Subscribe successful
MQTT: Sending, type: 8, id: 000C
TCP: Sent
TCP: data received 5 bytes
MQTT: Subscribe successful
MQTT: Sending, type: 8, id: 000D
TCP: Sent
TCP: data received 5 bytes
MQTT: Subscribe successful
TCP: data received 18 bytes
MQTT topic: time, data: {1422703981}
TCP: data received 62 bytes
MQTT topic: timestring, data: {11:33 Saturday 31-01-2015}
Get another published message
TCP: data received 23 bytes
MQTT topic: dawn, data: {439}
Get another published message
TCP: data received 12 bytes
MQTT topic: dusk, data: {1042}
MQTT: queuing publish, length: 17, queue size(0/2048)
MQTT: Sending, type: 3, id: 0000
TCP: Sent
MQTT: Published
MQTT: queuing publish, length: 17, queue size(0/2048)
MQTT: Sending, type: 3, id: 0000
TCP: Sent
MQTT: Published
TCP: data received 13 bytes
MQTT topic: mytime, data: {400}
MQTT: queuing publish, length: 66, queue size(0/2048)
Time: 11:33:09 31/01/2015 Dawn=439 Dusk=1042 Tick=86392MQTT: Sending, type: 3, id: 0000
TCP: Sent
MQTT: Published
MQTT: queuing publish, length: 17, queue size(0/2048)
MQTT: Sending, type: 3, id: 0000
TCP: Sent
MQTT: Published
MQTT: queuing publish, length: 17, queue size(0/2048)
MQTT: Sending, type: 3, id: 0000
TCP: Sent

 

As you can see above – the unit is sending out temperature and the outgoing queue is not building up – all is well. So now I disconnect the WIFI.

 

rm match
pm close 7 0 0/96959550
TCP: Reconnect to home.scargill.org:1884
reconnect
STATION_IDLE
scandone
no office found, reconnect after 1s
STATION_IDLE
STATION_IDLE
scandone
STATION_IDLE
STATION_IDLE
STATION_IDLE
STATION_IDLE
scandone
add 0
aid 1
pm open phy_2,type:2 0 0
cnt
STATION_IDLE

 

And that sits in the loop waiting for a reconnect.. NOW I reconnect (by renaming) the WIFI

 

connected with office, channel 13
dhcp client start...
MQTT: queuing publish, length: 17, queue size(57/2048)
STATION_IDLE
STATION_IDLE
STATION_IDLE
DNS: Found, but got no ip, try to reconnect
STATION_IDLE
STATION_IDLE
STATION_IDLE
ip:192.168.0.9,mask:255.255.255.0,gw:192.168.0.138
TCP: Connect to domain home.scargill.org:1884
DNS: found ip 109.170.132.114
TCP: connecting...
MQTT: queuing publish, length: 17, queue size(76/2048)
TCP: Reconnect to home.scargill.org:1884
TCP: Connect to domain home.scargill.org:1884
TCP:Reconect to: home.scargill.org:1884
DNS: found ip 109.170.132.114
TCP: connecting...
MQTT: queuing publish, length: 17, queue size(95/2048)
TCP: Reconnect to home.scargill.org:1884
MQTT: queuing publish, length: 17, queue size(114/2048)
TCP: Connect to domain home.scargill.org:1884
TCP:Reconect to: home.scargill.org:1884
DNS: found ip 109.170.132.114
TCP: connecting...
TCP: Reconnect to home.scargill.org:1884
MQTT: queuing publish, length: 17, queue size(133/2048)
TCP: Connect to domain home.scargill.org:1884
TCP:Reconect to: home.scargill.org:1884
DNS: found ip 109.170.132.114
TCP: connecting...
TCP: Reconnect to home.scargill.org:1884
MQTT: queuing publish, length: 17, queue size(152/2048)
TCP: Connect to domain home.scargill.org:1884
TCP:Reconect to: home.scargill.org:1884
DNS: found ip 109.170.132.114
TCP: connecting...
MQTT: queuing publish, length: 17, queue size(171/2048)
TCP: Reconnect to home.scargill.org:1884
MQTT: queuing publish, length: 17, queue size(190/2048)
TCP: Connect to domain home.scargill.org:1884
TCP:Reconect to: home.scargill.org:1884
DNS: found ip 109.170.132.114
TCP: connecting...
TCP: Reconnect to home.scargill.org:1884
MQTT: queuing publish, length: 17, queue size(209/2048)
TCP: Connect to domain home.scargill.org:1884
TCP:Reconect to: home.scargill.org:1884
DNS: found ip 109.170.132.114
TCP: connecting...
TCP: Reconnect to home.scargill.org:1884
MQTT: queuing publish, length: 17, queue size(228/2048)

 

Note the problem – it is connected – or so it says – but the queue is still building up – as if it is unable to send anything out.

And that, right now renders the code useless as there will always be problems with WIFI.   If anyone has any ideas do let me know – this has to apply to other setups to one extent or another. According to Espressif they have solved some issues with reconnects in the latest SDK – well, I’ve overwritten the old one and recompiled so unless I’m missing some trick in the compile process I’m definitely using the latest SDK.

Facebooktwittergoogle_pluspinterestlinkedin

The New ESP201 and Dev Board

All I have time for right now, here’s the new board, the ESP-01 (left) next to it should give you an idea of the size of both the development board and the ESP8266 board that goes into it – NOT SMALL.  And yes, those are 0.1” centres. No points for PCB design but it does look useful.

So you’re looking at a test board for around £10 complete with ESP board, wire antenna as well as the on-board antenna, relay, connector for DHT11/22, mini-USB and micro-USB, a 3-way connector for the relay (and by the look of it enough clearance to make it save for mains power use) 3-colour LED, a buzzer, interface components – pretty much all you need for experimenting, really…

New ESP8266 Board

Facebooktwittergoogle_pluspinterestlinkedin