Category Archives: Web Sockets

More Websockets

August 21, 2015ESP8266, MQTT, Web SocketsMobile Websocket page with MQTTPete

In a previous blog item here I covered the use of MQTT, jQuery Mobile and Websockets to make a controller web page for mobile phones/tablets etc. here https://tech.scargill.net/mqttjquerywebsockets-controller/

I showed just a basic and not too pretty page with buttons that would control real items, in my case via Websockets on a Raspberry Pi, firing off MQTT (also on the Pi) to ESP8266 modules (using our own code based around TuanPMs MQTT project).

Well, I’ve been doing a bit more playing around – here’s a better way to make the buttons – but all based on the previous article so you’ll need to read that first. I’ve also added a second page – merely to show you how that is done easily – and you could expand that ad-infinitum – but I’ve ALSO shown how to get information back in from the same web socket as I was asked about this aspect. It’s very easy (there was a time when I was terrified of the subject).

So first of all this time my Node-Red code is slightly different – I’ve separated off the incoming websocket node from the outgoing one – as long as they refer to the same item they don’t have to be joined together.

So in this version – the same code in “process commands” – as instructions from the web page go out to MQTT to control various gadgets. See other blog item.

Here, a Timestamp (standard Node-Red item) pushes out a timestamp every second – and another function converts that into a simple string as payload – which is then sent back, every second, to the web page. Normally you might send JSON data back for further parsing in the web page but in this case it’s a simple string.

Here’s the code inside timeConvert.

var dt = new Date(msg.payload);
var msg = { 'payload': dt.getHours()+":"+dt.getMinutes()+":"+dt.getSeconds() }
return msg;

And so to the web page itself: I’ve improved it some way as you’ll see as I figured out how to get 4 buttons wide (trivial – I just didn’t spot it in jQuery Mobile the last time around).

As you can see it is quite pretty, has a page button near the top to switch to another page (which is empty as yet so I won’t show it here but it will be obvious in the code how it works and how you can keep adding more. I’m not happy about the look of the paging button but it’ll do for now.

Then way down at the bottom left – you’ll see a timestamp – that is coming from Node-Red and is in this case updating every second. It COULD be status information – or anything – but for the purpose of demonstration (and it’s also nice to know that something is actually working by visual feedback like this) it is just a string as described above.

I wish I could figure out a simple additional class to colour these buttons individually instead of making themes for each button type – perhaps someone else could contribute here.

So now this is a quite reasonable control page with the option for more pages (not going to get any more buttons on this page). I did think of having Node-Red return a string with the states of the various outputs and changing the buttons accordingly – but that would not have made for easy reading. Anyone implementing this will soon figure out how to add such functionality. Instead of returning simple text, it would make sense to similarly send back jSon as there is a very easy to use function to split this up.

Recall too from the previous blog that I’ve implemented (in the .htaccess file) simple password protection not shown here and not of much use locally but pretty essential if you want to control from outside.

Here is the web page code for the above, note I’ve changed the name of the websocket to /myapp3 – could have called it anything, really. See the “onmessage” function – you don’t get any easier than this. There you go… https://bitbucket.org/snippets/scargill/AGMq5

Update: 23/08/2015 – there is a better way to do navigation in jQuery mobile (I’m still learning) but it seems incomplete. A slide-in-from-the-left menu system is easy to create – but if you try to make one menu system for all pages – there is no formatting – I guess this will be available in a future version – however, replicating a menu system for each page has it’s advantages as you can disable the current page link!

Here is a version with the awful button missing hence saving some space on the page. So this is a little like the swipe side menu in NETIO except I’ve implemented it by touching the top title bar because swiping doesn’t work too well on a PC and you might be testing this in a normal browser!!!

The title bar is sitting doing nothing anyway, so why not.

Actually I like this – on a typical phone you could easily get half a dozen pages and some explanatory text or imagery in that side menu… hell, you could even get some more buttons in there and a logo!

Here you go..

https://bitbucket.org/snippets/scargill/yREGd

MQTT/JQUERY/WEBSOCKETS Controller

August 18, 2015ESP8266, jquery, MQTT, Web SocketsESP8266 control by Mobile phonePete

I wrote a while ago about using web sockets as against something like NETIO for controlling the home – most folk liked the article but I think part of it was a little complicated – and at the time I’d not really thought it out to make it as simple as possible.

So, this morning I started again and now I have a decent and easy way to make control interfaces from the mobile phone – to ESP boards. I won’t go into MQTT in detail here – I’ll assume you have an MQTT interface of some description on your WIFI boards – if you need more info on that, look elsewhere on the blog.

I’ll also assume you have node-red running somewhere. In my case I have a Raspberry Pi2 sitting in a corner with battery backup, running node-red, MQTT and Apache web server.

So – in order to make a half-decent looking phone interface, I decided to go for jQuery Mobile – it is cheap, easy to run and there is a theme generator.

I went along to https://themeroller.jquerymobile.com/ and made myself a set of THEMES – that is some CSS. You can’t get any easier – you just drag the colours onto the various items – save the ZIP file – and then refer to the THEMES folder in your web page.

Here’s what I did – you might want less colours, more colours, different colours etc.… you can change all aspects of the jQuery Mobile interface which is basically a bunch of items that work well on mobile phones and tablets. Click on these images for larger versions.

So – what’s needed then is a web page…. that shows a series of buttons (jQuery Mobile), a means to send off button presses to Node-Red (web sockets) and some means of then controlling things (MQTT).

In my case I put the web page and dependencies in a folder within /var/www on my Raspberry Pi (could be anywhere – a Windows PC, whatever). I can access these pages externally and hence there is a password on the folder – that is done in my case with an.htaccess file but you might have other ideas.

The page should begin with a header for accessing jQuery, jQuery Mobile and the themes etc.… I figured as the phone has to have an Internet connection to work anyway, there was no point in filling up the Pi with libraries other than my custom theme..

    <head>
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <title>Simple Controller</title>
        <link rel="stylesheet" href="themes/jquery.mobile.icons.min.css" />
        <link rel="stylesheet" href="http://code.jquery.com/mobile/1.4.5/jquery.mobile-1.4.5.min.css" />
        <script src="http://code.jquery.com/jquery-1.11.1.min.js"></script>
        <script src="http://code.jquery.com/mobile/1.4.5/jquery.mobile-1.4.5.min.js"></script>
        <link rel="stylesheet" href="themes/petes.min.css" />

    </head>

And so above, in the HEAD section of the web page (which I simply edit with NodePad++) links to the local theme and remote jquery. That local stuff all comes in the zip file from the themeroller site – well it does if you make the title when saving “petes” – you might want to call it something else.

Here’s my test page which works equally well on a mobile phone as in a web browser - I’ve checked this out in Chrome and Edge and on my Android 5 phone – identical results.

As you can see I’ve gone for gross, using matching colours (more or less) for the colour controls I’m actually firing off.

The only difference between the buttons is a simple addition of data-theme="g" or whatever letter – if you look at the theme image at the top, as you add more themes, they start off as “a” then “b” etc. We’ll see this code later.

Rather than put a ton of code in here – here’s a snippet with the entire page.

https://bitbucket.org/snippets/scargill/jgBnK

You’ll see the header information followed by some setup for web sockets.

The basis of my controller as you’ll know if you’ve read Home Control 2015 in this blog, is to send MQTT data to my ESP8266 boards, so for example to control the RGB output on GPIO0 of a board called RED1, I’ll send the following…. the topic would be red1/toesp and the payload would be for example {rgb:0,255,0,0,70,1}

The above means – send a message to board red1 and tell it to fire out a WS2812b signal on GPIO0 to turn 70 LEDs on fully bright red.

I wanted the web page to be as simple a possible – adding to a button something like….

onClick='send("red1","rgb:0,255,0,0,70,1")'

And that is exactly what I’ve done. But firstly the mechanics of the Node-Red web sockets interface.

In Node-Red, I have this:

Simpler than it looks, I’ve dropped in a WS input module (standard node-red node unit) and out – and connected them together.

the input listens to “/myapp2” - could be /fred or anything you like to call it. Yes folks that is ALL there is to the setup – but…. assuming my web page connects to this – I need to do something with incoming data (in this blog I’m ignoring firing data back to the page but that is also easy – I referred to that in the last blog – the point of web sockets of course is that this is a continuous connection between the web page – and node-red web socket control.

So what I need is an incoming MQTT message and topic – and that needs firing out to MQTT to control things. The only bit of code here is in “process commands” – this is a function module (drag and drop) and this is what I put in it.

var jsondata=JSON.parse(msg.payload);

if (jsondata.topic)
{
    msg.topic=jsondata.topic+"/toesp";
    msg.payload=jsondata.payload;
    return msg;
}

Simple enough – any incoming information is assumed to be in json format and is “parsed” (built in function) so that I can extract a payload and a topic to simply fire off to MQTT and on to my boards.

As I’m always going to need “/toesp” on the end of the topic – I add it in here rather than needlessly replicate it in the web page.

The web page itself needs to fire out something like (in this case)….

{“topic”:”red1”,”payload”:”{rgb:0,255,0,0,70,1}”}

The inner braces are for my boards…

So – my board ends up with:

topic: red1/toesp

payload: {rgb:0,255,0,0,70,1}

I wanted to keep the actual buttons as simple as possible so there is a function in the web page to glue the topic and payload together.

function send(topic,payload)
{
ws.send('{"topic":"'+topic+'","payload":"{'+payload+'}"}');
}

and so all we need now is the connection between the web page and node-red web socket interface (simple as they’re both on the same server) hence if you look at the web page you’ll see a definition for wsUriC which is then used in wsConnect() which is called on page load. This makes the connection.

ws.send() sends a string to Node-Red and my function ensures the string is in the right format.

A typical button looks like this..

Note the very simple onClick function call to send a command off to a particular board…. and if you study the web page example, putting aside the formatting commands for jQuery Mobile (which you can learn all about here https://jquerymobile.com/) - there is WAY more than just buttons available to you - but I’ll leave that to the reader to explore (or not).

So I press a button on the phone – and the light comes on or changes colour or whatever. There is no feedback – that would be covered by a call-back function in the page and jQuery to alter the shape or colour of elements accordingly – really quite easy once you get started. I thought of putting that in here but I’ve probably given you enough to think about as it is.

Hope you find this useful. When the connection is working incidentally you get a little TICK at the top of the page – or a little cross if not.

Pete.

p.s. If you’re wondering what all this cave reference is about – head over to http://www.bedrock.es

Coming up soon: I hope to review the inexpensive NEXION touch display (one is on it’s way) to make a nice wall-mounted thermostatic control – which will talk via serial to an ESP-01 which in turn will control a relay and have a DHT-22 chip for temperature sensing – and Node-Red for background timing control – and of course I will have phone-based monitoring and over-ride based on code very similar to what you see here.

Web Sockets Web Page Part 3

April 23, 2015ESP12, MQTT, Node Red, Uncategorized, Web SocketsNode Red and Web Socket control of ESP8266 by MQTTPete

For this to make sense you should look at Websockets Web Page Part 1 and Websockets Web Page Part 2 first.

In this final section I add styling to the page and also change the code so that you can send a partial message i.e. {“my_input”:”Hello”} without affecting other parts of the page.

So in order to demonstrate this latest version you’ll need to go make yourself a theme (actually only one file needed for this – the css file) and put it in a directory called themes under the page shown here.

You’ll also need to change the Node-Red to this…

You’ll see here that I have to two experimental injects… and also I pull out the incoming web socket data – process the SWITCH and send a command off to one of my boards via MQTT to turn a light on or off, so NOW our simply page as well as looking pretty, actually DOES something. It does not take too much of a stretch of the imagine to ponder how to replace those injects with status info from, say an input or an MQTT input etc..

Here’s what my new page looks like…

The process commands box looks like this..

As you can see – I’m merely taking the input message – and parsing it – and I’m only interested in “my_switch”… I return in this case a message to turn my ESP8266-board relay output on or off.

Here is the code for the page – don’t forget to set the WS to your Node-Red websocket and don’t forget the theme data…

[github file="/scargill/various/blob/master/blog2.html"]

Note a link to a second page – as yet doing nothing but basically you can build a multi-page app all in one html file… check out Jquery Mobile site – or click the link in the page.

Web Sockets Web Page Part 2

April 23, 2015Web SocketsMore Websocket Information - a Simple Websocket PagePete

For this to make sense, you really should read the original websockets item first (which has just been updated)– as this is a follow up.

So now, having made my first web sockets web page I started to wonder about the mechanics of getting information out of the page – and into it. That is – how could I make that slider move programmatically – could I simply send information into the web socket and have it change the page dynamically?

Well… yes. Here’s the modified test in Node Red…

So we have our two web socket nodes – input attached to output… anything coming in from the page would go back out. But could I inject changes? As it happens the information I have going through is a simple block of JSON of the form:

{“my_input”:””, “my_slider”,””,”my_switch”,””}

That is, the input field and by one now slider and one flick switch on the page. Oh and an image just to test that too. So here is the page…

Just a little more ambitious than the first one (which is why you should look at the earlier blog first)…

So I have my WS in and out – and when a change occurs on the page – that JSON block will pass into “ws in” and out through “ws out” - so it is entirely reasonable to experiment with injecting that info into the middle… see my 2 grey inject nodes which pass the following information..

The first “Inject ON” passes a string {"my_input":"Hello","my_slider":"40","my_switch":"on"}

The second “Inject OFF” passes {"my_input":"Goodbye","my_slider":"60","my_switch":"off"}

Here is the page itself – only marginally more complex than before.. and in addition I have a couple of images called light0.png and light1.png stored in an icons directory just below where this page is stored i.e. (var/www/app/index.html)

[github file="/scargill/various/blob/master/blog1.html"]

The main page is in the bottom half – you’ll see my_input, my_slider and my_switch – the latter made possible thanks to the inclusion of JQuery mobile. If you can run this – you’ll see that moving the slider or entering text into the text box has no effect until you press update, whereas clicking the switch has an immediate effect. Note that as the switch changes state, the image (which does not appear at first as I’ve not put a default in there – trivial) changes as well. Once you’re happy playing with that – try pressing the two injector buttons in Node-Red – the page will instantly update itself. Look at the DEBUG window in Node-Red when doing any of this and you’ll see the simple JQuery string being passed around.

And that should be enough to start someone who like me, until a couple of days ago had never played with web sockets, thinking about what kind of wonderful control pages they could make. If you make this available externally via your router – do put in some security – and that’s another matter.

MEANWHILE I’ve been having a look at Martin Harizanov’s stand-alone 3 relay ESP8266 board, taking advice from Espressif as to a small modification to their development board, finally getting my head around an external drive for the Raspberry Pi and more – but that will wait for another time.

And now Websockets Web Page Part 3

A Node-Red Websockets Web Page

April 20, 2015JSON, Node, Node Red, Raspberry Pi, Raspberry Pi 2, Web SocketsWeb Socket Driven Web Page in Node-RedPete

Updated 26/04/2015

I’ve been after this ever since I discovered Node-Red.. and right up until this yesterday I was tearing my hair out – having settled for NETIO as my interface of choice and constantly being annoyed by the slow pace of progress with this interface.

So what is this all about? At the end of this, if you don’t know how already, you should be able to put together your own mobile friendly web page which will talk to Node-Red and from there to and from whatever other gadgets you wish.

So what is involved?

Well, you need Node-Red to listen to incoming websocket requests and to be able to respond… that and a web-browser for now is ALL you need. You do NOT need (though you could use) the whole web server structure). In my case, Node-Red on a Raspberry Pi – that’s it. The PURPOSE for me at least is to control stuff by MQTT but I’ll not cover that here as I’ve talked about it elsewhere and that too can be handled with ease in Node-Red.

So in my case this is running on my Raspberry Pi locally but it doesn’t have to be – that’s just what I’m using for testing. What you see here above are three standard node-red nodes.. a websocket input – a function (DIY info) node and a websocket output.

Aside from giving them meaningful (or otherwise) names the only thing you need on the websocket bits is the URL. Any URL within reason staring with /. I used /myapp but something more meaningful would be better. This is NOT the address of your web page – this is the address of the websocket connection. Read on.

When you hover over the Websocket input in Node-Red editor, there’s a mention in the INFO box about killing msg._session in the flow. That meant nothing to me until one of my readers set me straight.. but it’s very simple in the example above, any information coming in from say a browser will be sent back out (second block market “ws out” to th same browser.

That’s it. Anything coming in to the websocket input node goes out of the websocket output node to the same browser but not another browser looking at the same page.

This version however with the function in the middle with nothing more than…

msg._session="";
return msg;

The information above kills the session info needed to lock down a given session so that in this scenario one browser may send a button press causing an action – and that will also be sent to any other browser looking in… as you can imagine a one-to-one chat window becomes… trivial.

So all we need now is a web page that will connect to this websocket - one which can send information to Node Red and also receive information.

I wanted a simple test web page to have a field that I could fill in – and a submit button that would send information to the websocket without all that refresh stuff and which would return the information to the page. Clearly one can to things with that info as it passes through the middle orange function block if present (passed in msg.payload).

It is very convenient to have everything in Node-Red and it is perfectly sensible to have it serve up the web page AS WELL as handling web sockets….

Note the addition of the bottom three boxes (see above, “kill session is not necessary unless you want to broadcast any changes to all browsers looking in – but you may want to use that block for something else – like handling traffic passing through from the web page and back to it).

For the web page itself, these are http, template and http response blocks respectively. The left most simply needs a URL – I’ve called it /testing and a useful name - the rightmost needs nothing but a useful name.. the middle block contains a bog-standard webpage of your design – and within that the websocket connection.

Pretty basic but that’s the idea. No need to even construct a web page – the html and js can all sit in the block marked “html+js”.

My internal address is 192.168.0.15, unlike a normal web page, the pages you serve in here are on port 1880 by default so the internal address for this page would be http://192.168.0.15:1880/testing

Simples.

But herein lies the rub.. what if you want folders (css, images etc) – where would you put them? I chose instead as I already had Apache web server running on the Pi to put my PAGE there instead. There is also the point that basic page security is easy to implement on Apache.

So – there are examples out there of a simple web page, there are other examples of mobile web pages – and even more of simplifying normal Javascript using JQuery (which is wonderful) and I guess all I’ve done here is merge all of this into a useable if trivial page for the purposes of demonstration. I could have complicated things by requiring loads of libraries, filling it up with images etc but I wanted to keep is simple and as you have to be online for this to work I figured I’d simply link to the popular CDNs.

So – I wanted a responsive design that would work well on the mobile phone and look nice, be expandable with ease – and that comes from the combination of JQuery and JQuery mobile. In total we’re looking at no more than 4 lines of includes at the top of the web page to achieve all that.

As the web page and socket are on the same machine that simplifies the address link for the websocket – and so what you see in the code generates this page on first use. You are seeing this screenshot from a Google web browser but the phone image looks pretty similar without all that Google guff at the top.

Remember this is just for demonstration but I think you’ll agree it is quite clean. So what you’re looking at is default information in a web page. The page is connected to the websocket – just just does that – and it reconnects if you stop the socket for any reason and restart it. IT has a field that says “!empty!” – nothing special – just default text. .. and 0 – again just default text. Notice the input box, the slider and the button. I’ll put something in the input box and move the slider – which is just a test pretending I’m injecting temperature info. I decided for the purpose of this test to keep the traffic down and have the slider not return anything until you hi the button.

and when I hit the Update button –

As you can see the info I sent out has gone through the websocket connection (and hence you can do something with it…. I made the individual form items into a JSON string (simple – see the code) and sent it off – it came back in – I used JSON.parse function to split them apart and JQUERY to dump the values back into the page.

There are of course a million ways to do this – mine is just one – but it looks nice and is very responsive. My thanks to the various bits of code, to the people who wrote the JQUERY libraries and the help I’ve had from the Node-Red guys for helping me get this far.

At this point of course, expansion is so easy… the code can be tested in an html visual editor – or just a local web page and NotePad++ and developed ad-infinitum. For sliders, buttons and a whole lot more – take a look at JQuery Mobile. – the flip-switches look good too.

Colours? Try the JQuery Mobile Theme Site.

Colours?

I’ve not quite figured out how to add the colours in given that everything is running in a box…. but that’ll come soon enough.

Here is the code. By the time you read this it will likely have morphed into something altogether more elaborate so I figured I’d put it in here before I get carried away. I’ve deliberately expanded the vertical spacing to chop this into blocks for you. Go through it slowly – it really isn’t that hard. The links below show the latest JQuery Google CDNs but if you don’t like Google there are many others.

<!DOCTYPE HTML>
<html>

    <head>
        <title>Simple Display</title>
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <link rel="stylesheet" href="https://ajax.googleapis.com/ajax/libs/jquerymobile/1.4.5/jquery.mobile.min.css">
        <script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.3/jquery.min.js"></script>
        <script src="https://ajax.googleapis.com/ajax/libs/jquerymobile/1.4.5/jquery.mobile.min.js"></script>
    </head>

    function sendMessage(){
    // send message back to page in simple JSON format
    // example {“part1”:”Hello”,”part2”:”50”}
    var toPage='{"part1":"'+$("#txtMsg_1").val()+'","part2":"'+$("#slider_1").val()+'","part3":"3"}';
    ws.send(toPage);
} // end sendMessage

    </script>

<body onload="wsConnectC();" onunload="ws.disconnect;">
<div data-role="page" id="one">
        <div data-role="header">
            <h1>Websockets Test Page</h1>
        </div>
        <div role="main" class="ui-content">
            <h1>Temperature Display</h1>
            <div id="status">status unknown</div>
            <input id="txtMsg_1" />
            <div id="part1">!Empty!</div>
            <hr/>
            <div id="part2">0</div>
            <hr/>
            <label for="slider_1">Input slider:</label>
            <input type="range" id="slider_1" value="60" min="0" max="100" />
            <input type="button" value="Update" onClick="sendMessage()" />
        </div>
    </div>
    </body>

</html>

See next blog item– Websockets Web Page part 2

Mosquitto and Web Sockets

April 15, 2015ESP8266, mosquitto, MQTT, thingstudio, Web SocketsMosquitto and Web Sockets -, Thingstud.io for IOTPete

This entry is about ThinkStudio (thingstud.io), Web Sockets and Mosquitto in my quest for ESP nirvana.

So TODAY I was alerted to an early version of a new service which I think you might like… so check out http://www.thingstud.io/

Clarification: This is ALPHA software but already, provided they keep at it I think this has the potential to replace the rather slow-moving NETIO app which provides a nice user interface for IOT.

This software uses MQTT which is right up my street.

Well, always one for a challenge, I went to the site, set up an account and…. nothing – not a thing. I put in my MOSQUITTO (MQTT) credentials and got absolutely no-where apart from having a nice pretty interface. They have a nice pop up live help (pretty amazing for an alpha product) and before long I was happily chatting away to Michael Karliner. It turns out that the product needs MQTT to have Websocket support which my Mosquitto on the Raspberry Pi does not. I’ve been putting it off because every explanation I’ve seen up to now involves compilers and all sorts of horrible things.

I was pointed to this link… blog.thingstud.io which Michale had just completed - and in there is an article about installing Mosquitto with websockets. I was in one of those “devil may care” moods and so figured there was no reason why this could no upgrade my existing Mosquitto.

So I did – I followed the instructions from the start right up to but not including the bit about installing Node-Red as I’ve already done that.

WELL BLOW ME!!! it worked. Well… not quite. It over-wrote my mosquitto.conf file with a virgin one so now my security was bust and my local boards could not talk to anyone.

I opened the virgin mosquitto.conf which had everything commented out and added these lines as per the blog

listener 9001

protocol websockets

So now I could run a websocket on port 9001 (I tested that by redirecting the port on my external address then trying http://www.hivemq.com/demos/websocket-client/ – worked a treat)

That allowed the websockets to work – on port 9001. But… my normal port would not work any more – 1883.. so I added this.

listener 1883

protocol mqtt

That got both pipes listening – all that was needed was to restore my user access.

allow_anonymous false

password_file /etc/mosquitto/passwords.txt

And Bob’s your uncle – the latest Mosquitto (which is just excellent) and now websockets.. which means I can write my own web pages to access MQTT if I want to!!! And no doubt I will – but if ThingStud.io works out – I might not have to. Ensure you are registered to get updates in here or follow my usual Twitter and Facebook accounts to make sure you don’t miss out – I have a hunch about this one.

Update: Oh I found another link with instructions for installing websockets – and a little info on the mosquitto.conf file…

http://harizanov.com/wiki/wiki-home/raspberry-pi/how-to-rasbperry-pi-install-mosquitto-with-websockets-enable/

Scargill's Tech Blog

Anything to do with gadgets – the stuff that gets me up in the morning full of enthusiasm