This is kind of interesting – the promise of an ESP8266 MQTT broker (not client – BROKER) able to talk to up to 25 connections. Really? Let’s find out.
Go here to get an account (free) and download the .BIN file for your ESP8266. Use something like the NODEMCU flasher to put the single .BIN file into the ESP8266. All of this takes no time at all.
Update July 2019
Looks like this didn’t go very far – I’m sticking with Mosquitto broker on Raspberry Pi.
End of update
When you flash the chip – it will appear as an access point and you can connect to it with your mobile phone… password 12345678
Once connected, on a browser on your phone, go to 192.168.4.1
Well, that’s the theory – on my phone, though I could connect to the board, going to 192.168.4.1 did me no good at all, the browser went no-where. I had to turn off my mobile data first.
At the end of that a web page did indeed appear and wanted to know about my wifi details. I punched that in. There was no search for an access point which I found disappointing – but I put in my nearest access point and password.
I could not proceed here without creating an account – and the account registration makes no mention of how your details (email) will be used – that needs clarifying – I’d prefer a direct promise that my details will NOT be used for marketing purposes. After the normal “we’ll send you an email to confirm” sequence, I looked into the iotcentral.eu site and grabbed my details – I was given a unique number to use as part of my topic. The username and password that the little board was waiting for – was the password used with the account.
I entered my username and password into the phone and was immediately told that the page wasn’t working – of course that is because the ESP8266 had dropped out of being an access point and joined up to my network. Something a little more elegant would be nice however as it looks to the uninitiated like the whole thing has bombed out – and what if I’d made a mistake?
I reset the ESP8266 and hoped for the best. Nothing on the PC browser…. I hit refresh on the phone – I must’ve done something wrong as the phone was still connected to the ESP8266 and I was presented with an unfilled page of questions again. Eventually I twigged that I should not be using my username or topic number as user but my email address… I think a picture on the website showing this might help.
THIS time the phone came back with a sensible response and told me to reset the ESP8266.
The BROWSER on the PC then showed that I now had a broker logged in.
It showed the IP address of the ESP8266 (internal) as well.
Credentials above are blocked out to protect the innocent (me). All of this took very little time in practice.
Once set up – I opened up my local MQTT client, punched in the local machine (I really had no desire to go to iotcentral – there are plenty of online MQTT brokers out there – I wanted the local version. Sure enough I could connect to the local ESP8266 broker – and send and receive MQTT messages. HURRAY.
Given the above scenario…
1. When I connect to the local MQTT broker – does it access iotcentral? Does it NEED to access iotcentral?
To test this I shut off my router from the outside world – and reset the ESP8266. I could not, no matter what I did, connect to the local ESP8266 broker. My normal Raspberry Pi-based MQTT broker sprang back to life almost immediately and the browser connected to that. The ESP8266 broker however remained dead. I reconnected the outside world – but the ESP-8266-based broker remained unavailable. I reset the ESP8266-based broker – and nothing. When I looked on the IOTCENTRAL site – the router had changed IP addresses. I’ve had an email that said this
“If the IP address received is different that the one it has, update it to the settings.json from the SPIFFS” – that makes a lot of assumptions I guess because I have NO idea what SPIFFS has to do with this.
Start again… I used my router to map the MAC address and the IP address so this would not happen again! I disconnected my INTERNET access and the ESP8266 continued to act as a broker. I reset the board. Clearly it stopped being a broker for a moment… but after reset, maybe 10 seconds later – it sprang back to life.
For the sake of it (though I knew what the answer would be) I tried QOS levels other than zero and asked for retention of messages (not enough RAM in an ESP8266 to start doing that) – they were ignored.. but – the messages still worked!
So – my questions to the author:
1. When the ESP8266 broker connects – if it cannot get to the outside world – it makes a broker – lovely – but when it DOES connect – what information does it send to the iotcentral site? And while connected to the site and operating, what goes back and forth?
2. Is it possible to insist on username and password protection when using the broker locally? I use Mosquito and always insist on username and password. I’ve had many conversations with people over the years who say that even locally you should put a username and password in? Just asking!
3. If connection to the IOTCENTRAL site initially IS essential – and it probably is? What is this site sitting on? Will it still be working in a couple of year’s time for example?
APPARENTLY this little unit will talk to up to 25 devices all on it’s own – which, I have to say would make it very useful for smaller installations. How RELIABLE that is will be down to longer term testing. I tried turning the WIFI off and back on. I turned the WIFI off – the broker disconnected. I turned the WIFI back on – the broker reconnected immediately. That’s a good start because your MQTT broker MUST be reliable or you’re in deep trouble.
Well, there it is – something new! I have to say I really did not think the ESP8266 would get this far. I suggest if this interests you – you look at iocentral.eu -but let’s see what the author has to say about the above questions…
Funnily enough I’ve just done a video about basic MQTT access….
https://www.youtube.com/watch?v=NtyyfcYQxa4
Dear Mr. Scargill, I would be very pleased if you could clarify the ‘an ESP8266’. Any or only certain types.
I own an ESP8266-01. Would it be possible?
I dont think anyone is supoporting those any more – not enough IO, not enough Flash. Sorry, not used one for a long time. Check out ESP-12 – sub $3 from AliExpress. 4MB Flash…
Hi,
>> I had to turn off my mobile data first.
I have the same problem, did you find a way without turning off mobile data ?
Thanks.
phone keeps the outgoing traffic towards where it “feels” there’s external connectivity, so on the mobile part, as the wifi part has no gateway or routing towards internet…
Your mobile data interface is part of your provider network, your wifi interface is part of your local wifi network. If your main connectin is to your mobile, there is no way to access your internal LAN network without modifications in your firewall and knowing your external IP address that can be dinamically assign by your mobile provider. So to access your lan you need to be in the lan. Closing the mobile data will activate your wifi interface.
I tried a similar project, having to log on to an ESP8266, inputting my Wi-Fi credentials, and eventually was supposed to end up with a web site auto-loading at 192.168.4.1 — I could never get it to work with my Android phone, even manually inputting the 192.168.4.1 address, but it worked fine on my iPad.
I have problems with ESP8266 devices in general. I had one wi-fi router that simply would not work with them, while a couple others worked OK. But the device would go to sleep (or some other technical term), at random times, making it totally unreliable. The replacement software for the ESP266 in a Sonoff device works reliably.
Good to hear it works. With ESP8266 in my considerable experience problems are caused by inadequate power supplies or ropey software. My installations are generally rock solid.
let’s say you want a way more powerful device than ANY little SBC… let’s say you want it small form factor, anyway… and that does not cost too much (but sure a little bit more than a SBC…)… with 2gb ram and 32gb emmc, gigabit, many usb3 ports… well, one day yes and one no, these mini pc are on offer on Amazon, take a look:
this morning it’s just 75€, you just need to put an ubuntu on there and you’re done, a reliable little device…
Interresting lil device…
By the way: The italian page today shows 89€ while the german page says 99€.
I googled a bit and noticed problems to run Linux on it. Maybe old infomations. Do you have experiences with this Z83II running Ubuntu? Is WLAN / BT working now?
i’ve took it, when it was on offer yesterday, just 75€… have to try when it arrives… but one is like the other, they use a reference board and little customization… you see equivalent by beelink, just saying
and, btw, you can even setup all the IOT stuff in windows… you’ll loose the simplicity of Peter’s script, but all of the pieces are installable there, too: apache+php, mosquitto, nodejs, etc…
Sure i could, but i like to keep it simple and secure. So linux is always a better choice for me.
While the (great) script is simple to use and covers a lot (yea, i used it a few times), i prefer to install the needed software manually to better cover my personel needs.
The limited memory of ESP8266 could be a problem. I would prefer to have the broker on a ESP32.
In case of home automation with enough DC power, I would use two bridged Pi zeros for high performance and relability.
Me too but as far as I am aware, apart from the odd stalled attempt there aren’t any.
Also a Pi Zero WiFi does not cost much more than a 32 and you can easily put a full blown Mosquitto on there and much more….
My MQTT broker runs very reliably on my main wifi router/AP, which (obviously for me) has OpenWRT. It has the same reliability as mu LAN, no additional hardware nodes, cost = zero.
HI, I have an ASUS RT AC66U, with ASUS-WRT MERLIN firmware.
Do know if and how is it possible to include the MQTT BROKER inside?
thanks
ALex
Indeed, and will continue to be better supported and far more secure.
The broker is now on ESP32. The bridging is not yet working but I will try to fix it. It will be able to handle up to 32 clients. Need to rework the storage for configuration since ESP32 offers more options on that and the webserver that will be used for configuring the broker. iotcentral.eu has now ssl config for MQTT and for websocket on ports 8883 and 9883.
Cheers
@Catalin Batrinu: Good work, but no source and no guarantee, ! won’t touch it with a pole.
Unless you want to make money out of it, I don’t see the point in not publishing the sources along with binaries: for simple users, they will go with the later, for more experienced ones like the audience here, they will be able to customize/improve your code.
OTOH, if you want to make money out oi it, you will have to say clearly so, and what guarantees you will provide, terms of service, as well as comprehensive private information disclosure rules, etc.
And SECURITY!!! We need to understand that – there are enough IoT generated security issues in the world, we don’t need any more.
As an update, on https://github.com/bcatalin/demoapp there is a demo application, a plug that can be run together with the iotcentral.eu platform and Bondar MQTT broker to control a relay from anywhere in the world. If you have an Amazon Echo you can control it also by voice.
Binary file for app is also attached if you don’t want to mess with coding, compiling, libraries.
For the HW part you can use Wemos ESP8266 and and a relay shield.
Hi Pete,
I have made a little test regarding the performance of the uMQTT_broker using the tool sdkperf. With this load generator, you can send a lot of pubs/subs.
Here are the parameters:
-cip Broker-Host-Address:Portnr
-cu User name
-cp User Password
-ptl Topic for Pubs
-mn Total number messages
-mr Number messages/sec
-msa Payload length
-stl Topic for Subs
And the command for 1000 msgs, 100 per sec, payload 20 char:
D:\MQTT\Tools\Sdkperf>sdkperf_mqtt.bat -cip=192.168.4.1:1883 -ptl=mytop
-mn=1000 -mr=100 -msa=20 -stl=mytop -l -lat
Output:
> Getting ready to init clients
Sep 20, 2017 5:18:20 PM com.solacesystems.pubsub.sdkperf.core.AbstractClientCollection connect
Information: Connecting all clients.
Sep 20, 2017 5:18:20 PM com.solacesystems.pubsub.sdkperf.core.AbstractClientCollection connect
Information: Router capabilities: Unavailable due to: null
> Adding subscriptions if required
> Getting ready to start clients.
> Starting publish.
PUB MR(5s)= 0, SUB MR(5s)= 0, CPU=0
PUB MR(5s)= 101, SUB MR(5s)= 101, CPU=0
PUB MR(5s)= 101, SUB MR(5s)= 101, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 99, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 99, SUB MR(5s)= 99, CPU=0
PUB MR(5s)= 100, SUB MR(5s)= 100, CPU=0
PUB MR(5s)= 94, SUB MR(5s)= 95, CPU=0
======>Main thread finish
> Running sdkperf shutdown…
Pausing -ped time to allow clients to finish recv messages (2000 ms)
————————————————-
Aggregate Stats (Total # clients: 1):
————————————————-
Total Messages transmitted = 1000
Computed publish rate (msg/sec) = 99.0
————————————————-
Total Messages received across all subscribers = 1000
Messages received with discard indication = 0
Computed subscriber rate (msg/sec across all subscribers) = 100
You can see, that the broker can handle at least 100 pubs/subs per second. There is
no much degradation, when the payload has been increased to 100 chars.
Very impressive for such a small device!!! Good for small and medium sized projects!
For comparison, my 3Ghz PC with Mosquitto handles around 10.000 messages/sec.
Best regards
Walter
Absolutely Walter – GREAT – but how many clients? Anyone doing home control is likely to have 10-20 devices or more around the house….including a test client…
There is an alternative broker available including source.
Have a look on my project on:
https://www.elektormagazine.com/labs/mqtt-broker-on-esp8266-and-on-catamaran-1-1
regards Walter
Hi there Walter
It would be interesting to see your code expanded to handle 20 or more clients – I understand the one covered in this article can handle 20+ – so clearly if carefully used, there is enough RAM…
Also – any chance of adding username and password? That would be marvellous?
The esp_uMQTT_broker at https://github.com/martin-ger/esp_mqtt in the meantime is able to do u/p authentication, lwt, and retained topics. Currently it can work wih up to 8 clients, will work on reducing mem usage – perhaps 16 might be realistic for the near future…
My first question would be: what about the sources? This would perhaps bring some light into the other questions as well.
But obviously Catalin is working on a book project (“ESP8266 Home Automation Projects”). Perhaps he’s going to release the source when the book is published.
Well, I put a link in his ISSUES section – hoping he’d come in here and respond. Up to now this is what I’ve had back..
Interesting discussion, but it would be best to hold it at a community forum, as it is not an issue.
Closing due to off-topic for this Issue Tracker,
1. When the ESP8266 broker connects – if it cannot get to the outside world – it makes a broker – lovely – but when it DOES connect – what information does it send to the iotcentral site? And while connected to the site and operating, what goes back and forth?
When you connect the mosquitto from the Raspberry Pi to the cloudMQTT.com for example between these instances there is a bridging connection. This means that both are acting as on virtual broker. In Bondar’s case it have the same functionality but only if the topic has the form
/toker/topic1/subtopic1
otherwise the messages will stay in your WiFi LAN.
Example:
A thermostat is subscribing to the topic heating/main_heat and your device is publishing the temperature to the heating/main_heat the value will not leave you house.
If you want to have this temperature to your phone app when you are in Spain the temperature sensor need to publish it to the topic /34fr56ad/heating/main_heat on Bondar and your mobile app need to subscribe to iotcentral.eu:9004 on topic 34fr56ad/heating/main_heat where 34fr56ad is your base topic allocated by the system.
Every message that goes to/from iotcentral.eu is validated against user an password with a mosquitto plugin for the mysql database.
Conclusion:
Data is better protected then mosquitto to mosquitto and if you have messages that need to stay in your network will stay in there.
2. Is it possible to insist on username and password protection when using the broker locally? I use Mosquito and always insist on username and password. I’ve had many conversations with people over the years who say that even locally you should put a username and password in? Just asking!
Some explanation has been addressed on #1 answer, but yes it is possible to store user and password and check every message for u and p. Things get more complicated it I want to change the password but this can be done with some effort.
3. If connection to the IOTCENTRAL site initially IS essential – and it probably is? What is this site sitting on? Will it still be working in a couple of year’s time for example?
The connection to iotcentral.eu is necessary to be done only first time. If someone wants to keep the data private in its WiFi networks then don’t publish the data with /34fr56ad/ in front of the topic. In my mind staying isolated is not an option, but YES it is possible.
About the site. It is developed by me it is made with NodeJS/expressJS/passportJS as a backend and sits on digitalocean.com with the hosting paid in advanced. So the backend and mosquitto are connected to the same mysql database. I don’t keep any password in clear and I’ve used for password only proven solutions.If someone can take care of it and wants to increase the server power its fine with me.
4.I could not proceed here without creating an account – and the account registration makes no mention of how your details (email) will be used – that needs clarifying – I’d prefer a direct promise that my details will NOT be used for marketing purposes.
I’ve made the website with email confirmation just to be sure that I am not dealing with bots or spammers. I hate spammers. But as a funny fact I’ve made an account on your blog and I didn’t get any message or warranty that my email address will not be used for commercial purposes. I hope you don’t mind 🙂
5.Source code
Some people are asking about the source code. I can publish it but how many will used it for real, one or two ? It contains a lot of LWIP and FreeRTOS modified code. If Microsoft will not publish the code they will stop using Microsoft products. If the Apple is not publishing the code they will not write comments ? How many lines of codes they published. I will not publish the code that is part of a platform.
Thank you for that – you say you can add username and password for local use – but is this possible now or do you need to add that? I really would value a standalone setup like this for simpler installations. I’m sure others would too. I share your views on source – nice but how many would really go in and mess with it.
For U/P I need to modify the existing code.
Many thanks for the explanation. I would like to ask though, is the connection to cloudMQTT.com encrypted with TLS?
If not, then surely you are transmitting the uid/password in the clear over the wire along with all of the data?
This would really limit its usefulness and potentially opens a lot of people to hacking.
“I could not proceed here without creating an account – and the account registration makes no mention of how your details (email) will be used”
Yep..stopped reading right there!
My system is based around a mini-ITX dual Atom PC board (one of several picked up in a work lab clearout) and runs fine 24/7 on a shelf in the office where it hosts Mosquitto, Node-red and whatever I’m currently looking at (Openhab. Domoticz, Home Assistant efc.).
Keep up the good work Pete, and thanks for sharing your creations and thoughts.
I remember seeing this over at his blog and hoped he would release the code
https://myesp8266.blogspot.com/
This seems to be his github (maybe)
https://github.com/bcatalin/bondar
Well, it definitely seems to work and 25 is good for many installations – but for me I need answers about relying on a server which we know nothing about.. and I really would prefer to have to enter a username and password regardless… it would be nice to get info – so many times in the past I’ve committed to using something which relies on an external server then either the person has lost interest and shut it off – or the free server has become commercial etc… or it’s a home project relying on a server at home…. can’t have that… hence I have a Pi (and regular readers know the extend I go to make that reliable) with graphing software (Grafana) and MQTT etc, so the whole system UTTERLY works if the Internet goes down and even if the power goes off – comes back on but no Internet access – and no-one has access to my internal kit. Anyway, I’ve asked the questions in here… if others are interested maybe they’ll ask as well.
If this worked entirely stand-alone with username and password… yup, I could see several uses for it as long as it is reliable and it has already passed the failed WIFI test….. Of course things like last will and testament are out – as is QOS control (I’m fairly sure) but for a basic setup – half a dozen ESPs doing stuff and another acting as MQTT broker – I could see that…