Category Archives: node-red

Node Red Weather

One of the many things that Node-Red makes easy to capture – is the weather – and all credit is due to the people who put the DARKSKY API together.

Animated weather

What you are seeing above is a 6 day forecast in ANIMATED, COLOUR images (I appreciate it does not look animated here). It is LOVELY. This is running on my Raspberry Pi and of course the display, being a web page can be displayed on a tablet, a PC screen, a phone or an integrated touch-screen. Don’t go using old browsers though – this makes use of HTML5 and vector graphics so you should be reasonably up to date.

To get a nice pretty weather forecast for your home control display or whatever you plan to use it for, you need:

a. information

b. A display mechanism

In this case the information comes from DarkSky via a simple API

DarkSky

If you wish to “call” the DarkSky API you may do this up to 1000 times a day for free. Reader Jay and I felt that grabbing the information every 5 minutes might be a good start. Once the novelty wears off you might consider every 15 minutes or so. A timestamp node is simply used as a trigger – to get the information when Node-Red powers up and every 5 minutes from there on.

An HTTP request gets you a massive object with just about everything you could ever want to know about the weather and finally a function attempts to read that object with minimal error checking. I say this because in the past I’ve had the very odd bad package from DarkSky using the NODE that is available (I suggest you avoid that at least for now) which does not harm but looks bad in the logs!

The “http request” node in the middle requires little information – just your unique code which you get when you sign up (for free) with DarkSky – and your longitude and latitude.

DarkSky details

Here is the full URL I used with a fictional key – and my own coordinates.

https://api.darksky.net/forecast/xxxxxxxxxxxx/55.130575,-2.2534091?units=uk2

Now you COULD read all the information from the new object and make a whole boatload of MQTT messages – some of which you might never use. If you were to do this – your flow might look like this.. and my thanks to reader Jay for saving me all that work.

massive flow

i think you’ll agree that is slightly over the top – but it does give you an idea of how much info you can get out of this and that is NOT a complete list.

On the other hand it was good fun for a bit, checking out all the info

info from DarkSky

So, instead, what we’re going to do is simply pull out of the object what we need… See below. When this is called, the function I’ve dropped in after that simply contains…

try {
var weathertest=msg.payload.currently;
global.set("DarkSky",msg.payload);
node.send (msg);
}
catch(e) {}

"weathertest" is used purely to provoke an error if there is something wrong with the data and can be ignored from now on. So now with luck we have an object called “DarkSky” out of which we can pull all manner of things as needed – and of course being global we can access this from any flow at any time.

So – here’s a EXAMPLE function to get the current weather summary

try {
msg.payload=global.get("DarkSky").currently.summary;
node.send(msg);
}
catch(e) {}

What’s with the “try—catch”? Well, this is new for me – that comment I made above about accessing anything at any time - I got sick of seeing harmless complains about undefined objects in my logs due to using something before it has been defined – or bad data coming through etc.,  so my new thing is to actually check that a variable exists before using it. If you call this function BEFORE you’ve grabbed the weather for the very first time – nothing will output – simple!

So – how far can we go with this? VERY far – it’s a BIG object but for what we need here – it is all very simple.

global.get(“DarkSky”).when.what

where “when” is one of the following:

  • minutely
  • hourly
  • currently
  • daily.data[x] where x starts at 1 for today and goes to 7 days,

msg.payload=global.get("DarkSky").daily.data[2].humidity;  // humidity for tomorrow !!

“what” can be one of the following:

  • summary
  • nearestStormDistance
  • precipIntensity
  • precipProbability
  • temperature
  • apparentTemperature
  • humidity
  • windSpeed
  • visibility
  • cloudCover
  • pressure
  • icon

I’m sure you’ll  agree that’s a pretty damned comprehensive selection, nicely hidden away in an object for you to pick out what you need. cloudCover and humidity you might want to multiply by 100 to make them useful – and temperature is in degrees C though you can no doubt change that to anything you want – read the API docs.

For now what we’re interested in is the icon – the name of an icon to represent the weather for seven days. The icon can be one of the following:

  • clear-day
  • clear-night
  • partly-cloudy-day
  • partly-cloudy-night
  • cloudy
  • rain
  • sleet
  • snow
  • wind
  • fog

It JUST so happens that there is a set of animated icons called SKYCONS which match up with this exactly – and they are in technicolour – you have control over the colour of various bits of them including:

  • main
  • moon
  • fog
  • fogbank
  • snow
  • leaf
  • rain
  • sun

So putting this all together – you can make a full colour set of icons which adjust for each of several days. As I am using Node-Red Dashboard, 6 icons fit nicely onto my phone so I simply picked the next 6 days.

You might want to satisfy yourself that you can grab this data before moving onto the next stage. Get yourself a DarkSky account, punch in the details and maybe go and grab today’s weather. Of course without these icons you still have lots of info you can use for wall displays or maybe calculating what to do with your heating system – for example in cold weather, if it is damp I like to ramp the heating up just a little more.

And now, onto the imagery.

You will need to be comfortable with creating a folder to put JavaScript libraries for use with Node-Red. In the Node-Red settings file you can define such a directory – just make sure you don’t call it “js” as Node-Red already has one of those and you’ll end up in trouble. I call mine “myjs”.  Go and grab the skycons.js file and put it on your system. You’ll find a lot of information in that link I’ve just given you.

For 6 days your flow will look like this..

weather flow

So – every 5 minutes or however often you choose, grab the relevant icon from the global object you are populating and inject it into one of these templates.

Here is the code for template 1.

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

<script src="/myjs/skycons.js"></script>

<canvas id="icon1"  width="50" height="50"></canvas>

<script>
var skycons = new Skycons({"monochrome": false, 
                "colors": { 
                      "main": "#779966", 
                      "moon":"#666666",
                      "fog": "#555522",
                      "fogbank": "#884488",
                      "cloud": "#999999",
                      "snow": "#aaaaaa",
                      "leaf":"#00ff00",
                      "rain": "#0000ff",
                      "sun": "#ff3300"
                    } });
skycons.add("icon1", Skycons.PARTLY_CLOUDY_DAY);
skycons.play();

    (function(scope) {
        scope.$watch('msg.payload', function(data) {
           if (data=="clear-day") skycons.set("icon1", Skycons.CLEAR_DAY);
           if (data=="clear-night") skycons.set("icon1", Skycons.CLEAR_NIGHT);
           if (data=="partly-cloudy-day") skycons.set("icon1", Skycons.PARTLY_CLOUDY_DAY);
           if (data=="partly-cloudy-night") skycons.set("icon1", Skycons.PARTLY_CLOUDY_NIGHT);
           if (data=="cloudy") skycons.set("icon1", Skycons.CLOUDY);
           if (data=="rain") skycons.set("icon1", Skycons.RAIN);
           if (data=="sleet") skycons.set("icon1", Skycons.SLEET);
           if (data=="snow") skycons.set("icon1", Skycons.SNOW);
           if (data=="wind") skycons.set("icon1", Skycons.WIND);
           if (data=="fog") skycons.set("icon1", Skycons.FOG);
        });
    })(scope);

</script>

[/pcsh]

The other templates are identical EXCEPT that: they don’t need the include – and they don’t need the definition of skycons. Also – wherever you see “icon1” – change that to “icon2” etc.. up to “icon6” assuming you want 6 days. “icon7” if you want to go all the way to 7 days.

If this does not work smoothly – check out that you ARE pointing to the javascript library and that you’ve made no mistakes – check that the data is actually going into the template node and make sure you have an up to date Node-Red and Node-Red Dashboard.  That will serve well later as I’m no-where NEAR finished making use of templates… just the thorny matter of getting data into and out of them to get to grips with.

If you have problems formatting these images - I noted when moving them mine came out stretched - this is some js and css  I use for other purposes but I think you need the padding css in a template (make this the first template - auto size - it will not show up) - to remove padding around the edges .

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

<style>
  .filled { 
      height: 100% !important;

      padding: 0 !important;
      margin: 0 !important;
  }
  .nr-dashboard-template {
      padding: 0;
      margin: 0;
  }
  
  .rounded {
  border-radius: 12px 12px 12px 12px;
}
 
   .bigfont {
  font-size: 18px;
}

   .smallfont {
  font-size: 12px;
}
  
</style>

<script>
$('.vibrate').on('click', function() {
  navigator.vibrate(100);
});

function restore_bg(x) {
            $(this).css("background-color", x);
    };

$('.touched').on('mousedown', function() {
    var x= $(this).css("background-color");
    $(this).css("background-color", "yellow");
    
    setTimeout(restore_bg.bind(this,x),200);
    navigator.vibrate(80);
    });
    
</script>

[/pcsh]

Facebooktwittergoogle_pluspinterestlinkedin

More Node-Red Dashboard Success

Dashboard buttonsWell, it looks like my revelations with hiding the mouse has upset WordPress – the previous blog on the subject makes some people’s mice disappear!  Good job I wasn’t testing a self-destruct button.

It’s been a good day today – I discovered SLACK and I’ve been happily chatting to folk about my Node-Red-Dashboard issues and at the end of it all, not only have the issues gone away but I’m making progress.

So – there are a couple of things to notice and some you won’t.

So firstly – my button experience until now on Node-Red-Dashboard has been blighted by a SWIPE command which meant if you pressed a button near the right of the screen and in any way slid your finger  to the right – the page would change – in the unreleased update to Node-Red-Dashboard (needs just a tad more work and docs apparently) that has gone.  Excellent. I’ve seen this “feature” in other packages and it really does not work if you’re supposed to interact with buttons.

In my last blog on the subject you’ll see I can hide that top menu if needed (the blue menu is great on a phone – not much use on a thermostat)… and today I discovered that desktop TEMPLATES have a spacing round them that DID make it impossible to make your own buttons that would be size compatible with the UI built-in buttons.

That is all now history. So – the built in buttons – no text wrap which make it hard to make small, a visual effect on press that really is hardly noticeable, inability to change colour or indeed anything other than text colour – all problems now history.

The garish red, green and blue buttons you see above – note the rounded edges you control, there is an attractive and obvious colour change pm press (and return to original colour after press without you having to program it ), audible feedback on press (on phones with vibrators), wrap on text – indeed you can put anything in there - icons etc.

This is due to a combination on my insistence on having flexibility – my own experience and lots of help from several guys on the web today – thanks everyone for filling in the missing bits! The end result of all of this – I really DO think that Dashboard is able to take on the big boys and I’m currently backing away from the alternatives.

So – the enabling technology here is the node-red-dashboard template – this has LOTS of potential and it is now easy to accept incoming information – and send out a value on press or whatever. You’ll note elsewhere that I’ve used this to make whole pages with colour-changing icons etc., well here I’m using one template per button – but that’s just for the purpose of demonstration.

Things that needed considering – CSS for button sizing, JS to get the button colour and change it while pressing then change it back – I was not going to get into setting each and every button press colour etc. More CSS for styling. All described below and trivial to implement.

SO it turns out that if you put a template on a page – make sure it is the first one – you can put nothing but JS and CSS in it – and it will not do anything VISIBLE – but will supply your functions and CSS for the whole page – isn’t that HANDY!!!

tmp3513Let’s take a look at what is driving the display up there (again ignoring the text at the top – start with the buttons). To the left you see a template all on it’s own – then 3 templates – then 3 buttons – not connected to anything as this is merely a demo.

So – the FIRST template has info we need – CSS and JS to make everything work – you just need this once for any page. The light blue buttons are there just to show you what buttons look like in Dashboard (i.e. not as good as they could be) – and the three templates are the three new coloured buttons you see above right – which react to being pressed both visually and with haptic feedback.

Let’s take a look at the FIRST template which makes it all possible. It looks easy now but trust me it has taken some head scratching to get this far.

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

<style>
  .filled { 
      height: 100% !important;

      padding: 0 !important;
      margin: 0 !important;
  }
  .nr-dashboard-template {
      padding: 0;
      margin: 0;
  }
  
  .rounded {
  border-radius: 15px 15px 15px 15px;
  font-size: 18px;
}
  
</style>

<script>
$('.vibrate').on('click', function() {
  navigator.vibrate(100);
});

function restore_bg(x) {
            $(this).css("background-color", x);
    };

$('.touched').on('mousedown', function() {
    
    var x= $(this).css("background-color");
    $(this).css("background-color", "yellow");
    
    setTimeout(restore_bg.bind(this,x),200);
    navigator.vibrate(100);
    });
    
</script>

[/pcsh]

NOTE: At the time of writing this colour change works on Chrome etc on a PC and Android phone – it does NOT work on Firefox/Chromium on a Pi – this needs resolving.

The class FILLED is important - this is the one that gets rid of the padding that both buttons and template use by default - to make the button fill to the size of the template. nr-dashboard-template is also involved. None of this seems to affect other items on the screen.

ROUNDED is simply a means of forcing rounding on buttons - you can adjust that radius to suit or eliminate it if you like.

The scripts - well VIBRATE merely invokes the haptic feedback when you press a button - easy when you know how.

TOUCHED and it's associated restore function are variations on the ones I used with images. The class allows on press for a button to store it's colour - change it's colour - wait for a while - and change it back - all without having to have individual colour control on the actual button  - which simply has the default colour you select.

Now let's take a look at once of those templates. Oh and I got those subtle colours from here - https://flatuicolors.com/

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

<md-button class="vibrate filled touched rounded" style="background-color:#c0392b" ng-click="send({payload: 'Hello World'})"> 
    Red<br/>button
</md-button>

[/pcsh]

Simple? Yes because the previous code makes it so. A button in a template - auto stretched by the code above and all it has added is the background colour - which the above code can manipulate - and the classes. That's it. The ng_click is what happens when you press the button - the template outputs a payload of whatever you've set. No, really that is it.  so on that output, msg.payload will be whatever you set it to be.

With background colour control, RGB and related buttons become simple - a page full of colours for RGB controls becomes possible.  Note in the class ROUNDED I changed the font size - well you could separate that off to a separate class - and create classes (say) BIGTEXT and LITTLETEXT for example - suitable for different size buttons.

When you grasp this - if you're not there already, immediately all sorts of ideas should come to mind -it's a bit like opening a new door to see all the things behind it. Within days my demo screen will look utterly amateurish as I start adding icons etc but today was important as it was, for me a confirmation that there really is not a lot you can't do with node-red-dashboard – so many congratulations to the original developer and the guys at IBM and everyone contributing. 

If you go to the ANGULAR site and look at their examples - most of them work no problem - changing these rectangular buttons to round ones is a snap!

Oh, talking of improving things  - here’s a snapshot hours later as things progress…NICE!

new buttons

 

Oh – here… http://flows.nodered.org/flow/4bfad41290eda31588ff63a23815ac5c  - if you want a play…. and YES that template uses a little white space – but next Node-Red-Dashboard release it will NOT – css and scripts only will NOT use up screen real estate – all coming together VERY well right now.

Facebooktwittergoogle_pluspinterestlinkedin

NPM Update Status

If you are a Node-Red fan (and if not – why not) – you’ll know that the NODES – i.e. the modules that are provided with Node-Red or that people like myself create – get updated every now and then.  And it is a pain in the bum to keep checking to see if any of them need updating.

So – how about a weekly or monthly email detailing any updates!

It turns out that there is a handy command called “npm outdated” which returns a list of out of date devices!

So here’s the general sequence…

cd .node-red  // if you’re in the PI directory of your Pi or similar – this will take you to the Node Red directory

npm outdated // this will show you a list of out-dated nodes (or nothing).

You can then go off and do updates when you wish.

It occurred to me that this needed automating. One of the guys at the Node-Red groups (Mark Setrem) gave me this:

npm outdated --prefix ~/.node-red

That saves having to switch to the Node Red directory – and so from there – a simple flow…

tmpF1CD

The timestamp node can be set to output every week or month – the exec node contains what you see above – and the green email node contains email credentials.

The payload (email body) comes from the middle exec node, fill in the TOPIC in the timestamp and you have your SUBJECT line.

The output (which you can test by pressing the button on the timestamp – who’s payload is ignored)….

tmpE74B

The whole operation takes several seconds but because of the way Node-Red works – it’s not holding anything up. Ok, formatting could be better – but who cares – it does the job.

The original information that started me off is on the Node-Red site..

I’d updated NODE and NPN to the latest versions and that made some nodes fail (SQLITE)… also I realised when thinking about it – that some of those nodes have not been updated in a while – here’s how I gotaround the lot…

sudo npm cache clean

sudo npm update -g --unsafe-perm node-red

So that gets you the latest Node-Red after any updates of Node etc…

cd ~/.node-red

 

npm outdated

The above gives you a list of your nodes that may be out of date either through a node update – or simply – age!!

npm update              # to update all nodes, or

 

npm update foo          # to only update a node called foo

Just adding in a little extra here (10/dec/2016) as I cracked formatting.

HTML formatted email

Simple really…  in the Exec node itself I added a JSON flag

npm outdated -json --prefix ~/.node-red

and in the function I then simply used a table (the kind of clever CSS we often use on web pages is not too good in emails as there are many clients out there which still handle only the simplest of HTML – I learned that the hard way a while ago after developing a mass email system for a large organisation – my code was directly responsible for formatting and sending out many, many millions of emails running most hours of the day – you soon learn how many people are cheesed off at your attempts at styling)

So keeping it simple (ok, I should have used HEX for the colour – I’m being lazy here)..

[pcsh lang="css" tab_size="4" message="" hl_lines="" provider="manual"]

var m="<table style='font-family:arial; font-size:14px'><tr><td style='color:blue'><b>Node</b></td><td style='color:blue'><b>Current</b></td><td style='color:blue'><b>Wanted</b></td><td style='color:blue'><b>Latest</b></td></tr>";
msg.payload=JSON.parse(msg.payload);

for (var key in msg.payload)
{
m+="<tr><td style='color:green'>";
m+=key;
m+="</td><td>";
var mm=msg.payload[key];
m+=mm.current;
m+="</td><td>";
m+=mm.wanted;
m+="</td><td>";
m+=mm.latest;
m+="</td></tr>"
}
m+="</table>";
msg.payload=m;
return msg;

[/pcsh]

Now… I DID try Antonio’s version (see the comments below) and got the rather lovely email below… other than the missing first letter !! and I would have expected it to include the location contents as well – for some reason it didn’t… but nice and simple…

ansitohtml version

Facebooktwittergoogle_pluspinterestlinkedin

Fauxmo Alexa Delights

Amazon EchoI’m going to cut a very long story short here.  I’ve been looking at ESP8266 emulations of WEMOs because up to now this seems to be a popular device to emulate when using with your talking Amazon Echo. If anyone wants to come up with something that will take ANY word, not just ON or OFF – please do feel free to do the hard work. In another article, I described how, with the aid of a small SKILL at the Amazon end, an SSL certificate and some Node-Red code, we managed to make a generalised system to extract words out of Echo – and that is controlling my heating and all sorts right now -  marvellous. If you’re not up to speed with ECHO, MQTT or NODE-RED see earlier blogs in here.

Simple On Off

But what if you just want ON and OFF with no HTTPS, no messing at the Amazon end?  I checked out ESP8266 emulations. AWFUL. I’ll not mention names and I’m sure this was just done as a demo – but the one I tested was written in Arduino/ESP code – connected to WIFI and acted as an ON/OFF Wemo – MARVELOUS. Except that with the first sign of WIFI trouble – it fell over and died – absolutely no attempt at making it work regardless. Well as you know, WIFI dies from time to time. Bin.

So what next?

I found something others will have spotted called FAUXMO which is basically a single Python script –that emulates one or more WEMO type devices. At first it did not work – so half a day later having learned enough Python to get me by I realised that this was intended to call web pages – to turn things on and off. I understood there was an MQTT version of it – but that a friend of mine was having trouble getting that working – so  - I decided to go for broke and modify the original.

All you need here is the Pi or similar equipped with MQTT and with Python installed – Node-Red is not needed. Indeed once you see how this works – it would be very easy to take the cheapest of these little SBCs – the FriendlyArm NEO for example and make it look like several devices – instead of MQTT you could turn ports on and off in Python.

Cheapest multi-output Alexa-enabled device on the planet?

Orange Pi ZeroBefore I go further – you might have a simple requirement – far simpler than we usually cover in here – a single box that “does it all”, talking to Amazon Alexa with multiple outputs looking like multiple devices… well, the Orange Pi ZERO available at under £9 (inc shipping) from AliExpress could just be the ticket? Install Armbian, MQTT and Python – and… read on.

PAHO Mqtt

You need a client called Paho MQTT to bring MQTT to Python… so here’s what I did in order… we’re looking at a Pi or similar, running Node-Red and MQTT broker. In my case with username and password.

sudo pip install paho-mqtt

Seconds later, done – MQTT now available inside Python – that was easy.

I had to add a header into the code – to import mqtt – and I had to define username, password, mqtt address.

That all went into the top of the file and should be pretty obvious.

I then modified some code – and the definitions of two sample devices – “kitchen lights” and “bedroom lights”.

Here are the modifications.

Top:

[pcsh lang="python" tab_size="4" message="" hl_lines="" provider="manual"]

import paho.mqtt.client as paho

mqtt_host="192.168.aa.bb"
mqtt_user="xxxxxxx"
mqtt_pass="yyyyyyyyy"

[/pcsh]

Change to how recognised terms are actually used

[pcsh lang="python" tab_size="4" message="" hl_lines="" provider="manual"]

class rest_api_handler(object):
    def __init__(self, on_cmd, off_cmd):
        self.on_cmd = on_cmd
        self.off_cmd = off_cmd

    def on(self):
        dbg(self.on_cmd)
        topic,payload=self.on_cmd.split("|")
        client.publish(topic,payload,0,0)
        return 200

    def off(self):
        dbg(self.off_cmd)
        topic,payload=self.off_cmd.split("|")
        client.publish(topic,payload,0,0)
        return 200

[/pcsh]

And finally the bit you're interested in - device definitions... you only have to change the user settings at the top then this lot.

[pcsh lang="python" tab_size="4" message="" hl_lines="" provider="manual"]

FAUXMOS = [
    ['bedroom lights', rest_api_handler('toesp/bedlights|{out0:1}', 'toesp/bedlights|{out0:0}')],
    ['kitchen lights', rest_api_handler('toesp/kitchenlights|{out0:2}', 'toesp/kitchenlights|{out0:3}')],
	]

[/pcsh]

So what you’re looking at are the words that Alexa will accept (“kitchen lights” for example) followed by ON OR OFF. The two following strings are MQTT topic and PAYLOAD separated by | for firstly ON and secondly OFF.

As you can see it is easy to add new devices – you do with the MQTT what you will – in my case this would be talking directly to devices – more likely I’ll fire instructions off that Node-Red can handle for manual override of lighting etc.

To get this all to work you need to be running the program and say “Alexa find devices” to your ECHO – it will find the devices and you’re ready to go. If you make changes – restart the program – re-find devices.

And here  is the complete – modified Python code – complete with the original author info – after all I didn’t make THAT many changes even if it did end up taking me all day.

Don’t worry about it looking over-powering – you only need to know about the stuff above – the rest is just a case of pasting into into a file.

[pcsh lang="python" tab_size="4" message="" hl_lines="" provider="manual"]

#!/usr/bin/env python

"""
The MIT License (MIT)
Copyright (c) 2015 Maker Musings
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""

# For a complete discussion, see http://www.makermusings.com

import email.utils
import requests
import select
import socket
import struct
import sys
import time
import urllib
import uuid
import paho.mqtt.client as paho

mqtt_host="192.168.aa.bb"
mqtt_user="xxxxxxxx"
mqtt_pass="yyyyyyy"


# This XML is the minimum needed to define one of our virtual switches
# to the Amazon Echo

SETUP_XML = """<?xml version="1.0"?>
<root>
  <device>
    <deviceType>urn:MakerMusings:device:controllee:1</deviceType>
    <friendlyName>%(device_name)s</friendlyName>
    <manufacturer>Belkin International Inc.</manufacturer>
    <modelName>Emulated Socket</modelName>
    <modelNumber>3.1415</modelNumber>
    <UDN>uuid:Socket-1_0-%(device_serial)s</UDN>
  </device>
</root>
"""


DEBUG = False

def dbg(msg):
    global DEBUG
    if DEBUG:
        print msg
        sys.stdout.flush()


# A simple utility class to wait for incoming data to be
# ready on a socket.

class poller:
    def __init__(self):
        if 'poll' in dir(select):
            self.use_poll = True
            self.poller = select.poll()
        else:
            self.use_poll = False
        self.targets = {}

    def add(self, target, fileno = None):
        if not fileno:
            fileno = target.fileno()
        if self.use_poll:
            self.poller.register(fileno, select.POLLIN)
        self.targets[fileno] = target

    def remove(self, target, fileno = None):
        if not fileno:
            fileno = target.fileno()
        if self.use_poll:
            self.poller.unregister(fileno)
        del(self.targets[fileno])

    def poll(self, timeout = 0):
        if self.use_poll:
            ready = self.poller.poll(timeout)
        else:
            ready = []
            if len(self.targets) > 0:
                (rlist, wlist, xlist) = select.select(self.targets.keys(), [], [], timeout)
                ready = [(x, None) for x in rlist]
        for one_ready in ready:
            target = self.targets.get(one_ready[0], None)
            if target:
                target.do_read(one_ready[0])
 

# Base class for a generic UPnP device. This is far from complete
# but it supports either specified or automatic IP address and port
# selection.

class upnp_device(object):
    this_host_ip = None

    @staticmethod
    def local_ip_address():
        if not upnp_device.this_host_ip:
            temp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
            try:
                temp_socket.connect(('8.8.8.8', 53))
                upnp_device.this_host_ip = temp_socket.getsockname()[0]
            except:
                upnp_device.this_host_ip = '127.0.0.1'
            del(temp_socket)
            dbg("got local address of %s" % upnp_device.this_host_ip)
        return upnp_device.this_host_ip
        

    def __init__(self, listener, poller, port, root_url, server_version, persistent_uuid, other_headers = None, ip_address = None):
        self.listener = listener
        self.poller = poller
        self.port = port
        self.root_url = root_url
        self.server_version = server_version
        self.persistent_uuid = persistent_uuid
        self.uuid = uuid.uuid4()
        self.other_headers = other_headers

        if ip_address:
            self.ip_address = ip_address
        else:
            self.ip_address = upnp_device.local_ip_address()

        self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.socket.bind((self.ip_address, self.port))
        self.socket.listen(5)
        if self.port == 0:
            self.port = self.socket.getsockname()[1]
        self.poller.add(self)
        self.client_sockets = {}
        self.listener.add_device(self)

    def fileno(self):
        return self.socket.fileno()

    def do_read(self, fileno):
        if fileno == self.socket.fileno():
            (client_socket, client_address) = self.socket.accept()
            self.poller.add(self, client_socket.fileno())
            self.client_sockets[client_socket.fileno()] = client_socket
        else:
            data, sender = self.client_sockets[fileno].recvfrom(4096)
            if not data:
                self.poller.remove(self, fileno)
                del(self.client_sockets[fileno])
            else:
                self.handle_request(data, sender, self.client_sockets[fileno])

    def handle_request(self, data, sender, socket):
        pass

    def get_name(self):
        return "unknown"
        
    def respond_to_search(self, destination, search_target):
        dbg("Responding to search for %s" % self.get_name())
        date_str = email.utils.formatdate(timeval=None, localtime=False, usegmt=True)
        location_url = self.root_url % {'ip_address' : self.ip_address, 'port' : self.port}
        message = ("HTTP/1.1 200 OK\r\n"
                  "CACHE-CONTROL: max-age=86400\r\n"
                  "DATE: %s\r\n"
                  "EXT:\r\n"
                  "LOCATION: %s\r\n"
                  "OPT: \"http://schemas.upnp.org/upnp/1/0/\"; ns=01\r\n"
                  "01-NLS: %s\r\n"
                  "SERVER: %s\r\n"
                  "ST: %s\r\n"
                  "USN: uuid:%s::%s\r\n" % (date_str, location_url, self.uuid, self.server_version, search_target, self.persistent_uuid, search_target))
        if self.other_headers:
            for header in self.other_headers:
                message += "%s\r\n" % header
        message += "\r\n"
        temp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        temp_socket.sendto(message, destination)
 

# This subclass does the bulk of the work to mimic a WeMo switch on the network.

class fauxmo(upnp_device):
    @staticmethod
    def make_uuid(name):
        return ''.join(["%x" % sum([ord(c) for c in name])] + ["%x" % ord(c) for c in "%sfauxmo!" % name])[:14]

    def __init__(self, name, listener, poller, ip_address, port, action_handler = None):
        self.serial = self.make_uuid(name)
        self.name = name
        self.ip_address = ip_address
        persistent_uuid = "Socket-1_0-" + self.serial
        other_headers = ['X-User-Agent: redsonic']
        upnp_device.__init__(self, listener, poller, port, "http://%(ip_address)s:%(port)s/setup.xml", "Unspecified, UPnP/1.0, Unspecified", persistent_uuid, other_headers=other_headers, ip_address=ip_address)
        if action_handler:
            self.action_handler = action_handler
        else:
            self.action_handler = self
        dbg("FauxMo device '%s' ready on %s:%s" % (self.name, self.ip_address, self.port))

    def get_name(self):
        return self.name

    def handle_request(self, data, sender, socket):
        if data.find('GET /setup.xml HTTP/1.1') == 0:
            dbg("Responding to setup.xml for %s" % self.name)
            xml = SETUP_XML % {'device_name' : self.name, 'device_serial' : self.serial}
            date_str = email.utils.formatdate(timeval=None, localtime=False, usegmt=True)
            message = ("HTTP/1.1 200 OK\r\n"
                       "CONTENT-LENGTH: %d\r\n"
                       "CONTENT-TYPE: text/xml\r\n"
                       "DATE: %s\r\n"
                       "LAST-MODIFIED: Sat, 01 Jan 2000 00:01:15 GMT\r\n"
                       "SERVER: Unspecified, UPnP/1.0, Unspecified\r\n"
                       "X-User-Agent: redsonic\r\n"
                       "CONNECTION: close\r\n"
                       "\r\n"
                       "%s" % (len(xml), date_str, xml))
            socket.send(message)
        elif data.find('SOAPACTION: "urn:Belkin:service:basicevent:1#SetBinaryState"') != -1:
            success = False
            if data.find('<BinaryState>1</BinaryState>') != -1:
                # on
                dbg("Responding to ON for %s" % self.name)
                success = self.action_handler.on()
            elif data.find('<BinaryState>0</BinaryState>') != -1:
                # off
                dbg("Responding to OFF for %s" % self.name)
                success = self.action_handler.off()
            else:
                dbg("Unknown Binary State request:")
                dbg(data)
            if success:
                # The echo is happy with the 200 status code and doesn't
                # appear to care about the SOAP response body
                soap = ""
                date_str = email.utils.formatdate(timeval=None, localtime=False, usegmt=True)
                message = ("HTTP/1.1 200 OK\r\n"
                           "CONTENT-LENGTH: %d\r\n"
                           "CONTENT-TYPE: text/xml charset=\"utf-8\"\r\n"
                           "DATE: %s\r\n"
                           "EXT:\r\n"
                           "SERVER: Unspecified, UPnP/1.0, Unspecified\r\n"
                           "X-User-Agent: redsonic\r\n"
                           "CONNECTION: close\r\n"
                           "\r\n"
                           "%s" % (len(soap), date_str, soap))
                socket.send(message)
        else:
            dbg(data)

    def on(self):
        return False

    def off(self):
        return True


# Since we have a single process managing several virtual UPnP devices,
# we only need a single listener for UPnP broadcasts. When a matching
# search is received, it causes each device instance to respond.
#
# Note that this is currently hard-coded to recognize only the search
# from the Amazon Echo for WeMo devices. In particular, it does not
# support the more common root device general search. The Echo
# doesn't search for root devices.

class upnp_broadcast_responder(object):
    TIMEOUT = 0

    def __init__(self):
        self.devices = []

    def init_socket(self):
        ok = True
        self.ip = '239.255.255.250'
        self.port = 1900
        try:
            #This is needed to join a multicast group
            self.mreq = struct.pack("4sl",socket.inet_aton(self.ip),socket.INADDR_ANY)

            #Set up server socket
            self.ssock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM,socket.IPPROTO_UDP)
            self.ssock.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)

            try:
                self.ssock.bind(('',self.port))
            except Exception, e:
                dbg("WARNING: Failed to bind %s:%d: %s" , (self.ip,self.port,e))
                ok = False

            try:
                self.ssock.setsockopt(socket.IPPROTO_IP,socket.IP_ADD_MEMBERSHIP,self.mreq)
            except Exception, e:
                dbg('WARNING: Failed to join multicast group:',e)
                ok = False

        except Exception, e:
            dbg("Failed to initialize UPnP sockets:",e)
            return False
        if ok:
            dbg("Listening for UPnP broadcasts")

    def fileno(self):
        return self.ssock.fileno()

    def do_read(self, fileno):
        data, sender = self.recvfrom(1024)
        if data:
            if data.find('M-SEARCH') == 0 and data.find('urn:Belkin:device:**') != -1:
                for device in self.devices:
                    time.sleep(0.1)
                    device.respond_to_search(sender, 'urn:Belkin:device:**')
            else:
                pass

    #Receive network data
    def recvfrom(self,size):
        if self.TIMEOUT:
            self.ssock.setblocking(0)
            ready = select.select([self.ssock], [], [], self.TIMEOUT)[0]
        else:
            self.ssock.setblocking(1)
            ready = True

        try:
            if ready:
                return self.ssock.recvfrom(size)
            else:
                return False, False
        except Exception, e:
            dbg(e)
            return False, False

    def add_device(self, device):
        self.devices.append(device)
        dbg("UPnP broadcast listener: new device registered")


# This is an example handler class. The fauxmo class expects handlers to be
# instances of objects that have on() and off() methods that return True
# on success and False otherwise.
#
# This example class takes two full URLs that should be requested when an on
# and off command are invoked respectively. It ignores any return data.
# 
# Clearly somewhat modified by PS to take in a string - split in two and send out to MQTT
class rest_api_handler(object):
    def __init__(self, on_cmd, off_cmd):
        self.on_cmd = on_cmd
        self.off_cmd = off_cmd

    def on(self):
        dbg(self.on_cmd)
        topic,payload=self.on_cmd.split("|")
        client.publish(topic,payload,0,0)
        return 200

    def off(self):
        dbg(self.off_cmd)
        topic,payload=self.off_cmd.split("|")
        client.publish(topic,payload,0,0)
        return 200


# Each entry is a list with the following elements:
#
# name of the virtual switch
# object with 'on' and 'off' methods
# port # (optional; may be omitted)

# NOTE: As of 2015-08-17, the Echo appears to have a hard-coded limit of
# 16 switches it can control. Only the first 16 elements of the FAUXMOS
# list will be used.

# Clearly the above is changed. Here we have MQTT commands in the form topic|payload

FAUXMOS = [
    ['bedroom lights', rest_api_handler('toesp/bedlights|{out0:1}', 'toesp/bedlights|{out0:0}')],
    ['kitchen lights', rest_api_handler('toesp/kitchenlights|{out0:2}', 'toesp/kitchenlights|{out0:3}')],
	]


if len(sys.argv) > 1 and sys.argv[1] == '-d':
    DEBUG = True

# Set up our singleton for polling the sockets for data ready
p = poller()

# Set up our singleton listener for UPnP broadcasts
u = upnp_broadcast_responder()
u.init_socket()

# Add the UPnP broadcast listener to the poller so we can respond
# when a broadcast is received.
p.add(u)

# Create our FauxMo virtual switch devices
for one_faux in FAUXMOS:
    if len(one_faux) == 2:
        # a fixed port wasn't specified, use a dynamic one
        one_faux.append(0)
    switch = fauxmo(one_faux[0], u, p, None, one_faux[2], action_handler = one_faux[1])

dbg("Entering main loop\n")

client = paho.Client()
client.username_pw_set(mqtt_user, mqtt_pass)
client.connect(mqtt_host)

while True:
    try:
        # Allow time for a ctrl-c to stop the process
        p.poll(100)
        time.sleep(0.1)
    except Exception, e:
        dbg(e)
        break

client.disconnect()

[/pcsh]

(if this lot complains about spacings - get the original article and make my changes

So – put the file in a directory – I put it into /home/pi/fauxmo/fauxmo.py and gave the file execute permissions (no idea if that’s needed or not but as I do that with scripts I thought it best - 0754 if you're interested).

I ran the file in the /home/pi/fauxmo folder at a terminal as:

python fauxmo.py –d

The –d is for debug – you can scratch that once you are running – and then find a way to make it run reliably on power up. That is possibly described here:

https://github.com/makermusings/fauxmo  if someone has the working command line for this please do send it in – might save some experimenting.

And that’s it. Great fun and instantly useful without lots of hassle. With Node-Red you could fire the MQTT in to do with what you want – or fire straight to MQTT-accepting ESP8266s (what use are ESPs that DON’T support MQTT you might ask) – or if you look at the chunk of code handling MQTT – you could pull that apart and instead just control ports!!

So here’s a thought – one of our readers was kind enough to remind me that the Orange Pi Zero is now available and it is CHEAP. Normally I’d not touch Orange with a bargepole but the Armbian Jessie implementation seems to be now up to speed. SO - https://www.aliexpress.com/store/product/New-Orange-Pi-Zero-H2-Quad-Core-Open-source-development-board-beyond-Raspberry-Pi/1553371_32760774493.html

Under £9 for the cheapest, might not be that happy running Node-Red but it will certainly run PYTHON and MQTT (well, you don’t even need MQTT for that matter if you just want to control the odd output or send stuff through serial etc)… so for under £9 you could have several Alexa-enabled devices !!! That can’t be bad.

This will no doubt mutate as time goes on!

Facebooktwittergoogle_pluspinterestlinkedin

Node-Red Objects to File and Back

This is a major re-write of a year-old blog. At the time, I simply wanted to store a global object in a file and retrieve it. Things have changed, I realise the way we did this was awful – hence the re-write.

One way is to save the object as a persistent message in MQTT - another to a database, another to a file. I like the idea of a file as somehow it just seems that this will lead to the least WRITES – and that might be important for SD.

Why would you want to do this?

Continue reading Node-Red Objects to File and Back

Facebooktwittergoogle_pluspinterestlinkedin

Node Red Sunday

Node-RedIt is Sunday, I’ve done my 4,000 steps down to the village for coffee and back and I’m having a play.

I use Node-Red on a Raspberry Pi to control just about everything in two houses – as my confidence in the setup has grown, I’m currently keeping the Pi up 24/7 with no sign of any issues (well, it could ALWAYS be faster). Normally I control the likes of ESP8266 boards with this to do everything from monitoring temperature, pressure and humidity to controlling heating, fans, lights and more.

It occurred to me, as I’ve also had success with my script in getting Node-Red, Sqlite, MQTT and other software running on other SBCs including the really cheap new ones coming out of the woodwork that it would be nice to be able to talk to them with the same software especially those running Armbian as I’ve had a lot of success with that personally.

So I’ve made a start. I thought you might find this preliminary effort interesting.

In my controllers, MQTT is at the heart of everything and facilitates utterly reliable message sending from device to Pi and Pi to device (indeed there is no reason I could not do device to device really). I thought it might be nice to start off with a little bit of code that simulates what the ESP8266s do constantly – turn things on and off.

So typically, given a device called fred, I might turn a relay on or off using MQTT messages:

topic: fred/toesp
payload: {out0:1}

Let me first explain why I do it this way. So MQTT typically works by sending simple messages back and forth – they have a topic (address) and a payload (data).  My devices, for example fred, listen for two topics:

fred/toesp
esp

That way I can send a message to a SPECIFIC device – or to all of them at once (OTA for example)

If the device needs to send messages back they will always be of the form:

fred/fromesp/out0

in this case the payload returned would be the current state of out0.

So that represents the basics. Here I will show how a cheap controller running Node-Red can accept commands – I won’t go into the actual mechanics of turning a light on and off because for example the GPIO commands used in a Raspberry Pi won’t work on a NanoPi NEO – alternative methods are used. I’ll simply cover how to get to that point.

So here is the simplest possible setup on the Pi, to turn a couple of outputs on and off.

MQTT

This is nothing more than 4 INJECT nodes dropped onto a flow along with an MQTT output. Typically the content of one of these nodes might be:

INJECT

Simple enough. When pressed the node outputs a topic of nanopineo2/toesp and a payload of {out0:1}

This would work with one of my ESP8266 units to turn output 0 on. “nanopineo2” is just the name of one of my NEO units running Node-Red.

Here’s what’s at the other end to make this work.

Nanopi Neo

Two MQTT inputs feeding a function. So the function will work if either the generic “toesp” is called or a device-specific version. In that first function I check out the command – really simple – and will be dramatically improved soon.

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

// An attempt over time to emulate my code in ESP8266 - taking commands from MQTT
// and processing I/O
// for example message {out26:1}

var m=msg.payload;

if ((m.charAt(m.length-1)=="}") && (m.charAt(0)=="{")) // ignore if not in braces
{
    m = m.substring(1, m.length-1);
    var breakup=m.split(":");
    var getname=breakup[0];
    var sendto=breakup[1].split(",")
    
    switch (getname)
    {
    case 'out0' :  msg.topic='out0'; msg.payload=sendto[0]; node.send(msg); break;
    case 'out1' :  msg.topic='out1'; msg.payload=sendto[0]; node.send(msg); break;
    default: break;
    }
}

[/pcsh]

The output is reduced to the actual command “out0” for example – and a value. So the next item is a switch in this case selecting “out0” or “out1” and ignoring everything else. The outputs from that switch go to functions which simulate actually controlling something – like a LED for example and in my demo the colour of the black dot changes from black to red when you send a command to turn that output on.

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

if (msg.payload==1)
  { 
      msg.payload="lamp1.val=100"; node.send(msg); 
      node.status({fill:"red",shape:"dot"});      
  }
else
  {
      msg.payload="lamp1.val=0"; node.send(msg); 
      node.status({fill:"black",shape:"dot"});
  }
  

[/pcsh]

And that is it for now. As you can see, this can RAPIDLY be expanded to cover every port bit available on your micro, it could control PWM etc. The next step will be to take {command:arg1,arg2….arg6} and parse the arguments into an array. That will then allow PWM and other kind of controls. If you really don’t care about pricing you could do the whole lot with a bunch of Raspberry Pis… but with new Orange Pi boards coming out at under a tenner, the options are endless.

Facebooktwittergoogle_pluspinterestlinkedin

Node-Red on a Phone

I have to hand ALL the credit to Dave at the Node-Red Google Groups for this – and you might want to pop any questions in that direction because I know even less about running Android at the command prompt than I do Debian!

Running Node-Red on your Android phone or tablet!! Yes you CAN. it turns out that it is dead easy. Using a terminal program (App store) such as Termux.

So basically install Termux as you would any other app.. and run it – you get the usual black terminal screen. I did this just now on my HTC ONE-M8 – might work on yours – might not.

apt update
apt upgrade
apt install coreutils nano nodejs
npm install -g --unsafe-perm node-red
node-red

That’s it -  you’re up and running.

Well, so that is Node-Red running – of course if you close the terminal you stop Node-Red so the first thing to find out is how to press control-C in that terminal!!! Any ideas anyone?

Once I got back in – instead of starting Node-red I did the following..

cd .node-red

npm install node-red-dashboard

npm install node-red-contrib-admin

npm install node-red-contrib-bigtimer

cd

node-red

Once you run node-red – you can go into the browser on the phone and run 127.0.0.1:1880 and lo and behold- node-red  - probably more usefully would be to go to the phone’s WIFI setup and find out what the IP address is – and run Node-Red in a browser on your PC and that same port number. 

Now why on EARTH would you want to do this. Well, if you could get node-red running as a service on an old tablet – along with MQTT (not sure about that one) all as services – you have a home control server base – with several hours of reliable battery backup (you’d have to run a task to stop the thing turning off I guess.

Well, something for the weekend! As answers (hopefully) emerge for the following – also running my phone with SCP from the PC would be good… I’ve no idea what the user is or the password!!!! I’ll update this.

And the first update – control characters on that terminal… here.

Meanwhile – MQTT Broker Pro which will soon have it’s own basic authentication and which already comes up on phone boot – works a treat – here are the two of them running together as seen on my PC browser.

Node Red on a Phone

Be aware – that the MQTT server does not work if you use “localhost” – you need to use “127.0.0.1” or if you want to access the MQTT server externally – the IP address of the phone. And yes I’ve tested accessing the MQTT broker externally – which is another good reason for it to have a user name and password.

All good stuff.

Facebooktwittergoogle_pluspinterestlinkedin

Nextion Update

As you may know the home control software can work with Nextion displays and there is a  project on the blog on that subject – I note that one or two people are having issues here so rather than reply without pictures -  here’s a general  update.

Home Control 2016 startupCurrently with the home control software, version 1.531 (see right) and Nextion Editor version 0.36 (it just updated on my automatically)  and I’m doing this on Windows 10.

After powering up the home control board, you MUST for the first time issue the instruction {set_serial:2} to the ESP8266 at which point the board will reboot and part of the start up serial info will show as on the right – “Initialised software serial to 57600 baud” – this indicates that the unit is ready to work with the Nextion display. GPIO4 and 5 are used for this purpose and the Nextion board can be powered directly from ours. Ground, to_nextion, from_nextion, 5v – that is how our board is marked but of course you don’t need to use our board. Incidentally don’t be tempted to use your 5v FTDI to power any but the smallest of Nextion boards – I tried this on the large 7” model and killed the FTDI…you’ll need a separate (common ground) power supply for that and most likely anything over the 4.2” display. Don’t forget to change that set_serial command if you plan to use GPIO4 and 5 for something else as the setting is stored in FLASH.

In a simple test I have a plain Nextion panel with two buttons on it. In the background to the page – in the Nextion Editor – I have a “preinitialize event” which contains nothing more than “bauds=57600” without the quotes – this ensures the Nextion board powers up at 57600 baud for the purpose of sending data back and forth.

The two example buttons are marked “up” and “down” and the “touch press event” for each is as follows..

get "nodered~up"

and

get "nodered~down"

If you look at my documentation  - this is a notation I created that returns the above – for example the first one, to the ESP8266 and converts that into an MQTT message:

topic: nodered

payload: up

That’s it – press the up button and the message “up” is sent to topic “nodered” – could be any message you like – and any payload. You can use this directly to turn things on and off without even passing through Node-Red – OR you you have node-red intercept the incoming MQTT message and do something with it. So pressing a button might have an effect on the Nextion display for example. If you see the Nextion WIFI touch-display project you’ll see you can get WAY more ambitious than just a couple of buttons.

mqtt-spy

And lo – as I press buttons – you see the messages coming into MQTT-SPY (or whatever you use to test your MQTT).

That just leaves output to the Nextion itself from the board.

Again for the purposes of demonstration I add a text box in the Nextion editor – the default name for the first new text box would be “t0” so I rename this to “mytext”.

nextion text box

Not very pretty and normally I would NOT use their buttons etc. but make up my own imagery using PowerPoint or similar – but it’ll do for demo purposes.

Nextion displayNow I have to say, the worst part of this is the Nextion editor – progress is moving slowly with this – when you come to program it – make sure you have the right port address because it is hopeless at searching – and if it gets stuck you end up using task manager to reset it – comes up as PRJ or something – I’d so like to get my hands on the source code to give it a good shake-up.

Anyway, there it is- you end up with a window in this case with default text “newtxt”.

nextion display[5]Fire out the command {to_nextion:"mytext.txt=\"Hello\""}

You can do this serially or via MQTT- as the example from MQTT_spy shows.

This results in “Hello” without quotes appearing in the text box. Why the backslashes – that is to “escape” the quotes so they don’t end up terminating the string above – in other words so the quotes are passed through.

The WORD manual for my software describes the “to_nextion” and “set_serial” commands.

Have fun.

mqtt-spy[7]

Facebooktwittergoogle_pluspinterestlinkedin

Node Red Dashboard

node-red-dashboardAs regular readers and fans of node-red will know, I’ve been a fan of node-red-contrib-ui for some time – I wrote about adding security to it some time ago and until Imperihome came to my attention I was developing pages based on this.  But, sadly the fellow developing it ran out of time I guess. One day he was working hard on it – the next – nothing.

Some time ago I became aware that the Node-Red guys were developing this and today they announced that Node-Red-Dashboard was available on NPN.

If you have node-red-contrib-ui on your Node-Red installation – and are not using it – simple uninstall.

I stopped Node-Red, in a terminal went to my .node-red directory and..

npm remove node-red-contrib-ui

npm install node-red-dashboard

node-red-dashboard[8]node-red-dashboard[6]That simple really, at least it was for me – unless you’ve heavily invested in the former there is no point in keeping it any more as development has stopped. So – what’s new – well, it is early days, if you don’t like simple light or dark – well, that’s unfortunate because right now you have two  theme options – but that’s a start and the gauge and graphs shown above right took seconds to set up with this simple set of nodes – a couple of injectors firing out 2 numbers – and the gauge and graph nodes – that’s it.

Dashboard looks remarkably similar to node-red-contrib-ui and time will tell how compatible it is but the gauge alone has been markedly improved and is lovely – you can control size, there are four different types of gauge – it  really is easier to try it out than to discuss.  There is now a simple dashboard on the right of the Node-Red web page to allow you to re-arrange items by drag and drop – this is a great improvement.

You’ll be making simple pages of imagery in no time – as for actually making worthwhile dashboards – well I suspect that will be down to how good the documentation is.

node-red-dashboard

Let us know your experiences with this.

Facebooktwittergoogle_pluspinterestlinkedin

Speech … the Final Frontier

Ivona speechUpdated 02/01/2017

Buffered, SD-cached high-quality human-like speech working alongside sound effects for your IOT project.

Requirements: Raspberry Pi or similar with audio out, Node-Red and a little of your time.

But you don’t need this blog – there’s a WAY better way to do this – in my latest entry about the Ivona Node.

As you may know I’ve done a couple of blog items on Speech on the Raspberry Pi over time. I was quite happy with the Google Translate API until recently when they stopped making it available for free,  so I was forced to go off in search of alternatives and settled on local synthesizers like eSpeak – the problem being – they generally sound AWFUL. eSpeak sounds like someone’s dog being strangled.

And so it was that one of our readers (thank you for that) contacted me and suggested I take a look at Ivona. https://www.ivona.com/us/

This is new: I’ve just gutted the code from the original blog to produce a much better version, better thought out with cacheing for off-line use, unifying speech and sound effects into an auto-created library. In the event of an external comms failure so that you cannot access the outside world, your Ivona is going to fail and this happened here at Bedrock – SO the idea hit us to CACHE all mp3 files (assuming you’re not going to do unique messages every time). That way the SECOND time you ask for a message you will already have a file with that name – the total file space for dozens or even hundreds of these MP3 files is a tiny fraction of a typical SD storage capability and not even worth taking into consideration in most projects. Currently we are working on modifying the Ivona node to handly dynamically changing voices. More on that soon.

I originally did a video to accompany this blog - https://www.youtube.com/watch?v=qoxPVa48qRw

If you use the link I’ve provided above (to Ivona) and select a suitable voice on their web page then enter some text in the box, you’ll find it does a pretty good job. Ivona is free to developers – just grab a free account and get an API key and secret key. I spent the entire evening playing with the code and when I looked at the percentage of my “free use” – nothing  - so you’re unlikely to run out of free use and with the mods here even less so.

Take a tip when you copy and paste that API code information (which you should immediately on getting an account as you won’t be able to get the same key later) and pass them via NOTEPAD (paste then copy again) to get rid of any hidden characters.

So now you have an API key and secret key to be used in the Node-Red node Ivona (node-red-contrib-ivona) https://www.npmjs.com/package/node-red-contrib-ivona

Your API key for Node-Red is the ACCESS code and the PASSWORD is the secret key !!!!! I used my email address for the username – this is all a bit non-intuitive so beware.

Over to Node-Red. So what this node does is take in your TEXT, send it off to Ivona which returns an MP3 file with your chosen speech. You should also have MPG123 installed on your end computer (I’m using a Raspberry Pi2 for all of this). http://www.mpg123.de/

In the simplest case you would send off your text, get the MP3 file, send that to mpg123 for playback. But then you are stuck with a file… and what if you send 2 in quick succession – they will overlap each other as Node-Red runs asynchronously.

Here’s the solution and it’s a lot better than I had in the past. You can fire off several speech requests including requests for other .mp3 files.  for special effects I have a bunch of MP3 files already stored – such as “alert” and “hailing frequencies open”.

Ivona speech

In the example above (that red block is NOT the Ivona node – it is a subflow I wrote – more in a minute)… let me show you those two INJECTS on the left..

Ivona speech

Ivona speech

The first has “alert” in the topic and some text “Red 1 logged in” – the second simply text.

I can click one, wait for it to speak and then click the second – or I can chose not to wait, clicking wildly – and they will still play in order.  So if you specify speech for the TOPIC AND THE PAYLOAD, simply both will go into the queue in order.

How do I do that – so looking at the red Node-Red SUBFLOW…

flow

The yellow’ish blocks are user functions, the red block is a Node-Red EXEC functions – the purple item is a simple 1 second delay for good measure. The purple item is the node-red-contrib-ivona node.

I take in text… if the topic has the word “alert” in it – I put that on the queue BEFORE the main text – other than that there is NO difference between the two.

If there is no text, just a blank message coming in, I check the queue and if not empty, try to use the items on the queue (first in, first out) one at a time.

The INJECT function is needed to start the ball rolling for the first item in the queue. Once I find text in the queue, it is send to the Ivona node IF such a named file does not exist - and then on to the mpg123 player – either way setting a BUSY flag so that those one-second ticks can’t pull another item off the queue until I’m done.

When done – I  send empty messages back into the input to trigger off any further items in the queue.

Here is the main function:

var frompush=0;
if (typeof context.arr == "undefined" || !(context.arr instanceof Array)) context.arr = [];
if (typeof context.global.speech_busy == "undefined") context.global.speech_busy=0;
if ((msg.payload==="")&&(context.global.speech_busy===0))
if (context.arr.length)
{
frompush=1;
msg.payload=context.arr.shift();
}

if (msg.payload!=="")
{
// just push but not recursively
if (frompush===0)
{
if (msg.topic!=="")  context.arr.push(msg.topic);
context.arr.push(msg.payload);
return;
}

context.global.speech_busy=1;
msg.fname=msg.payload.replace(/ /g,'_');
msg.fname=msg.fname.replace(/\./g,'');
msg.fname=msg.fname.replace(/\,/g,'');
msg.fname=msg.fname.toLowerCase();
msg.fname="/home/pi/recordings/"+msg.fname+".mp3";
return msg;
}

 

Note the busy flag and the use of PUSH for the queue.

The “copy file to payload” is trivial – Ivona returns the filename in msg.file which is not where I want it.

msg.payload=msg.file;
return msg;

The reset flag function simply clears the busy flag and returns a blank message.

context.global.speech_busy=0;
msg.payload=""; return msg;

The trigger is the Node-Red DELAY function simply set to delay for one second and then pass the message on.

Ivona speech

 

The MPG123 EXEC node calls mpg123 and passes the file name as parameter. The DELETE node simply deletes the file that Ivona creates… Here’s the Ivona setup. Put your credentials in the top box.

moustache

Note the triple moustache {{{}}} -  Ivona examples use a double – but that then interprets slashes and we don’t want to do that because we have a file path in there.

And that is about it – works a treat and produces high quality buffered speech – for free – for your IOT endeavours.

Pick your own file directory (note that I used /home/pi/recordings but that isn’t in any way special) and any words or phrases you want SOUNDS for instead of voice – simply replace with files of the same name (not that spaces are replaced in files names by underscores).  So “alert 2” as a file name would be “alert_2.mp3”

Of course I'm always on the lookout for alternatives, especially those which profess to need no connection.

https://mimic.mycroft.ai/

This looked great - and the short  example in their videosounded like a human. Sadly, on installing this on the Pi (it can't play without drivers which don't come on the Pi as standard - but it can generate files) I gave it my favourite micktake of a certain politician "I'm a little tea-pot short and stout, open my mouth and shite comes out"...  and I have to say it made an UTTER and complete mess of it, sounding like a quite ill Dalek. Think I'll stick with Ivona.

Facebooktwittergoogle_pluspinterestlinkedin

To Weather or not to Weather

That’s it – I’m giving up with OpenWeatherMap. I’ve been having trouble recently with it accepting Hexham,UK some days and not others and it is getting worse – when you look at it’s search list it says Hexham,GB (despite the fact that I have to put in UK and not GB for it to work at all) – then sometimes it can’t find it at all. Someone is having a laugh. Secondly for 2 days it was MISERABLE here in Bellingham just a few miles North of Hexham – and OpenWeatherMap said it was “clear skies” – I’ve not seen blue sky since Thursday until today.

Meanwhile, forecast.io is spot on – see above. See this blog item for how to extract data - https://tech.scargill.net/the-weather/

So you will have noted that I’ve been messing with weather for some time and one of the issues I’ve had is debugging! The standard debug tool that comes with Node-Red is fine but has limited capacity and has no processing capability – for example if you fire a bunch of JSON at it, it looks HORRIBLE (for those not in the know, JSON is a standard, like XML but in most cases very simple, for sending data – simple text).

And it was with that in mind, when mindlessly trawling the web that I stumbled on a reference to a debugging FLOW for Node-Red which would treat text as text and would format JSON – but more’s the point – in a separate browser window – THAT caught my attention.

So – here is the article that started it all off for me this evening…  http://flows.nodered.org/flow/b0fcb7b72fc05a30e55b

Take PARTICULAR notice of Julian’s “Configuration” notes which came about as I completely messed things up and Julian very kindly helped me and then updated his documentation accordingly.

So what we’re doing here is sending debug info to a Node-Red web page – a great idea – but it involves having access to a static library…. /js/mqttws31.js and Julian in his debug notes now tells you how to ensure that Node-Red knows where to look – I would imagine that this info is also awfully useful for node-red-config-ui if you’re planning on adding libraries!!! Certainly added to my knowledge.

And so NOW when I fire out debug info from forecast.IO…..

 

Using the debugger

“Debugger” is the sub-flow that Julian has created (I just renamed it and stored it all in one sub-flow) and “readable debug” is this…

msg.topic="weather";
return msg;

The timestamp just triggers an action every 5 minutes and at power up – sensibly every 15 minutes is actually enough. Then there is the grabbing of data from forecast.io where xxxx  is your API key – freely supplied – and note I output as a “parsed JSON object”.

HTTP node setup

 

Then I bang the stuff into a global variable for later use.

global.set("weather",msg.payload);

and finally I add a topic for the new debug

msg.topic="weather";
return msg;

And so now instead of garbage coming out of the regular debug when I’m testing the weather I get….. wait for it – in the new debugger…. perfectly readable English!!!

Debugger output

Isn’t that wonderful – and as you have the source you could imagine all sorts of variations on this. I am SO chuffed about this – a genuine reason to buy MORE MONITORS!!

Here’s what the debugger sub-flow looks like…

Debugger subflow

Facebooktwittergoogle_pluspinterestlinkedin

The Weather

I had a chat with my pal Jonathan today about the weather – and more specifically getting it into Node-Red.  If you are building your home control with Node-Red at the centre then it is always nice to have information about the local weather to send off to your various displays (such as the one described earlier today) and forecast.io is a dead easy way to get it!

Go to http://forecast.io/ and sign up - that will give you an API number. No, really it is that easy and as long as you don’t use it 1,000 times a day, it is free. Simples.

Here’s what you do:

forecast.io setup

Using an HTTP node (the yellow one) and fill it in as below substituting XXXXXXX for your API key – and the long/lat coordinates for your own (I got mine off the Google Maps command line when pointing at my village). If you’re not in the UK then you’ll have to try something else. US is probably a good start for our friends in the USA.

forecast.io setup

In the function (orange) you need this..

var weather = JSON.parse(msg.payload);

And that is that. NOW you need to know what is available in your new json object and remember – don’t use that timestamp inject (it is irrelevant what it is injecting – it’s just a trigger) more than 999 times a day Smile

So if you look at the output from the incoming XML it is MASSIVE – but put into a JSON online viewer like jsonviewer.stack.hu once converted it starts to look more reasonable.

forecast.io json

For example if we expand currently – we get…

tmpD60D

And how do we access that? Simples – having called the object “weather” we get the temperature as weather.currently.temperature

and so on. In HOURLY we see timestamps and things like chances of precipitation… a little clever loop and… remember in an earlier blog we looked at showing graphs of history in Imperihome? It would not be beyond the bounds of reason to feed this lot into said graphs!!!

Well, even without digging any further, hopefully there are a few items of interest for you.  weather.currently.summary is good as are weather.hourly.summary and weather.daily.summary.

If you want to check the weather you could look at this – but with your coordinates. . http://forecast.io/#/f/51.6571,-0.0295

And on the subject of weather but otherwise unrelated – this looks like a fun site… http://www.wunderground.com/

Oh, nearly forgot – and you’ll be needing some fancy icons..  http://www.wunderground.com/weather/api/d/docs?d=resources/icon-sets

Have fun. And if you know of something BETTER – leave a link in the comments.

Facebooktwittergoogle_pluspinterestlinkedin