Monthly Archives: October 2016

Last Monday of the Month

Closing this one off - there are solutions in here and thanks to all.

I thought I’d kluged this function together correctly but apparently not. I’m looking to be able to enter the 4th Monday of the month (ie today) as an example – and get true – getting false every other time – well, I’m getting false every time and I’ve tried a couple of variations.

Anyone feeling clever this morning (once it works – it is going into BigTimer which has just seen some other improvements).

function dayinmonth(weekday,n)
{
var date=new Date();
if (n > 0)
{
var first = new Date(date.Year, date.Month, 1);
return ((date.Day - first.Day)/ 7 == n - 1) && (date.DayOfWeek == weekday);

}
else
{
var last = new Date(date.Year, date.Month, 1).AddMonths(1).AddDays(-1);
return ((last.Day - date.Day) / 7 == (Math.Abs(n) - 1)) && (date.DayOfWeek == weekday);
}
}

msg.payload=dayinmonth(2,3); // assuming Monday day two – week 3 (0-3) – I’ve tried variations – I get FALSE every time
return msg;

Facebooktwittergoogle_pluspinterestlinkedin

Alexa Enlightenment

Last updated November 16, 2016: In a recent blog entry I told you how we’d been playing with Amazon Echo (Alexa) and got it working. Well, nearly. It turns out that what SEEMS obvious in the system is not all it seems - but read on as we've had success.

What is Alexa?

Actually the device is called an Amazon Echo – or for the cheap version and Amazon Dot. It is a box requiring power and WIFI that you can talk to and which will control things. It can automatically discover certain (and often expensive) commercial devices, it can play music, it can answer questions, set diary appointments, make shopping lists – all by voice. And with a little effort can talk to Node-Red to open up a whole new world of DIY possibilities.

Listening for specific words

For example in the Amazon setup pages, you can define variables that will only accept a certain amount of options – so for example – you might say “Alexa, tell computer to turn kitchen lights on” where “lights” could be, say, light, lights or heater.

And that’s fine – but you can’t TRUST that. the word “flights” works equally well and Amazon makes it clear in their documentation that you must do your own tests – any similar valid word in the dictionary could have been accepted (which seems utter madness to me but there you go). Indeed when we tested this – we found all sorts of irregularities – there is no way to say “accept ONLY these words”. And, if you say something that is not at all acceptable, the system will wait to time out – and that takes several seconds – which would annoy the hell out of the spouse. So – if you’re going to have to do your own checks anyway, we reasoned you may as well just try to get the basic words out of the units into Node-Red and “do your own thing”. To be clear then, the Amazon unit is no-where near perfect at word recognition and you have to consider that when designing a solution. I wonder if Google will be better.

Wemo Emulator

A slight diversion here: If ALL you want to do is turn things on and off with Alexa and Node-Red – you COULD just ignore the rest of this article and go grab node-red-contrib-wemo-emulator or read other posts here describing my modifications to FAUXMO.

tmpDE0A

No – REALLY – drop it in – give it a name – like Orange – and define topics and payloads (which might go off to, say, MQTT or may be used to trigger direct port control on a Raspberry Pi or similar) for each of the two commands ON and OFF. Tell your Alexa to go looking for devices – it will find orange – and THAT IS IT – no HTTPS, no NOTHING. It really doesn’t get any easier.  But only ON and OFF which is a bit restrictive - and for heaven's sake don't use the word "shed".

How about controlling your TV. Well, the Anymote Smart controller for Android will talk to Alexa and then hence control stuff by infra-red  – except I can’t get it to talk to Alexa. I’ve written off to the guys at Anymote and will report back. But that takes care of all sorts of Infra-Red stuff as long as you leave your phone on!  Not ideal really.

The Bigger Picture

And now – back to the bigger picture!

Before Alexa, all Aidan and I wanted was something that would listen to requests and then fire speech out so we could handle it ourselves in Node-Red – and that is exactly what we have now achieved. Some will shrink in horror at this having already done it another way – but it looks to us like a great general way forward provided you’re happy with writing your own simple,  high level code to handle a sentence like “kitchen lights on” which could be as simple as a string comparison.

tmp4405

What we’ve done here – is to simple make Alexa collect up to 15 words (arbitrary number) after “ask computer to” – where “computer” is whatever word you want to be (the “skill”) and put them in an array, that function above, called “Alexa” does all of this and strips out words like “eh” “and”, “please” etc. – and you can add more exclusions.  If you forget to actually say anything – the second output of the Alexa function handles it all for you automatically.

So in USER FUNCTION – you get an array – msg.word – with up to 15 lower case words – with the rubbish stripped out. You also get msg.combined, a simple string with everything in one string. If your requirements are simple – the latter might be the way to go.

If you merely want to have Alexa repeat the salient points of what you said so that “Alexa tell the computer to turn the lights on” – the output will be “you said: lights on”

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

msg.payload = "You said. " + msg.combined;
return msg;

[/pcsh]

Alexa tell the computer to turn the lights on”

So above, Alexa reads the words in purple and passes on the rest. The simple Node-Red Alexa function strips out the words in red – and passes on the remainder in both an array of words and a simple string.

So clearly you might want that USER function to have outputs for MQTT and EMAIL and whatever other stuff to control your kit – that, of course, is easy and up to you. If you have 2 outputs from the function (set the number of outputs at the bottom) you can write to both at once – or separately by returning an array of messages -  or just one and a null etc. When you send a null – nothing goes out.

Now – I don’t want to go TOO heavily into this because simply string comparisons is probably not the way to go for any kind of sizeable setup… but here is something I’ve used as a test and it absolutely works a treat.

tmp27C1

The code in the user function (note the extra output) is trivial to say the least.

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

var msg2 = {};
switch (msg.combined)
{
    case "office lights on" : 
            msg2.topic="mains3/toesp"; 
            msg2.payload="{out12:1}"; 
            node.send([null,msg2]);
            msg.payload="The office lights are now active";
            break;
    case "office lights off" : 
            msg2.topic="mains3/toesp"; 
            msg2.payload="{out12:0}"; 
            node.send([null,msg2]);
            msg.payload="The office lights are now off";
            break;
    case "thank you" : msg.payload="No problem, to be sure."; break;
    default : msg.payload="You said: " + msg.combined;break;
}
return [msg,null];

[/pcsh]

In the example above, I have the function react to “Alexa tell the computer to turn office lights on” by sending an ON command via MQTT to one of my ESP8266 boards. Off has a similar effect, different parameter. The final command it just a bit of fun.

Really – so to get all of this fun, you have to set up your SSL (which I found incredibly painful but in retrospect it was dead easy – another of life’s hurdles out of the way) so that Amazon’s servers can talk to a secure connection (your Raspberry Pi or whatever you are using – FriendlyArm Neo or similar would be just as good running Debian), you need to update some stuff into Amazon’s site – and from there on you never touch the Amazon site again –which is of course the point really -  you only have to concern yourself with your USER FUNCTION in the cosy and quick-to-alter Node-Red environment. What you do with that is up to you.  In my case I’ll add another output to go to MQTT and whatever comes in is processed and maybe does something – any message in msg.payload (you HAVE to send something back) is spoken by Alexa when you’re done. It really doesn’t get a lot easier.

In a really trivial case you might, with TWO outputs,  (the first should be used to return a message;

say

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

if (msg.combined==”lights on”)
{
msg.payload=”Ok, light is on”; node.send([msg,null]);
msg.topic=”kitchen/toesp”; msg.payload=”{out0:1}”; node.send([null,msg]);
}

[/pcsh]

Ok, I know, create another object and send them both out at once – I’m trying to keep this simple.

But of course, in reality, you’ll be likely to do FAR more than this and hence having the separate words in an array is useful.

So - on Amazon - you need an intent!

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

{
  "intents": [
    {
      "intent":"inputIntent",
      "slots": 
        [
          {"name": "wa", "type": "LITERAL"},
          {"name": "wb", "type": "LITERAL"},
          {"name": "wc", "type": "LITERAL"},
          {"name": "wd", "type": "LITERAL"},
          {"name": "we", "type": "LITERAL"},
          {"name": "wf", "type": "LITERAL"},
          {"name": "wg", "type": "LITERAL"},
          {"name": "wh", "type": "LITERAL"},
          {"name": "wi", "type": "LITERAL"},
          {"name": "wj", "type": "LITERAL"},
          {"name": "wk", "type": "LITERAL"},
          {"name": "wl", "type": "LITERAL"},
          {"name": "wm", "type": "LITERAL"},
          {"name": "wn", "type": "LITERAL"},
          {"name": "wo", "type": "LITERAL"}
        ]
    }    
  ]
}

[/pcsh]

and a "sample utterance"

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

inputIntent  {LITERAL|wa} {LITERAL|wb} {LITERAL|wc} {LITERAL|wd} {LITERAL|we} {LITERAL|wf} {LITERAL|wg} {LITERAL|wh} {LITERAL|wi} {LITERAL|wj} {LITERAL|wk} {LITERAL|wl} {LITERAL|wm} {LITERAL|wn} {LITERAL|wo}

[/pcsh]

All we’re doing here is telling Amazon, for the “intent” “computer” – to collect up words and fire them off. And that's it for the Amazon end apart from the usual details they need about you and the https address of your website (the node-red exposed point).

As for Node-Red itself - the example above:

We have an http node which has nothing more than POST and “/echo” as the URL - then our ALEXA function contains the following - it may well be far from perfect but for now it works.

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

var doStuff = {payload: msg.payload.length};

switch (msg.payload.request.type)
    {
    case "IntentRequest":
    if (msg.payload.request.intent.name   === "inputIntent")
        {
        var word = [];
        
        word[0] = msg.payload.request.intent.slots.wa.value;
        word[1] = msg.payload.request.intent.slots.wb.value;
        word[2] = msg.payload.request.intent.slots.wc.value;
        word[3] = msg.payload.request.intent.slots.wd.value;
        word[4] = msg.payload.request.intent.slots.we.value;
        word[5] = msg.payload.request.intent.slots.wf.value;
        word[6] = msg.payload.request.intent.slots.wg.value;
        word[7] = msg.payload.request.intent.slots.wh.value;
        word[8] = msg.payload.request.intent.slots.wi.value;
        word[9] = msg.payload.request.intent.slots.wj.value;
        word[10] = msg.payload.request.intent.slots.wk.value;
        word[11] = msg.payload.request.intent.slots.wl.value;
        word[12] = msg.payload.request.intent.slots.wm.value;
        word[13] = msg.payload.request.intent.slots.wn.value;
        word[14] = msg.payload.request.intent.slots.wo.value;
        
        var thisone =0, processed = 0, total = word.length;
        
        for (;;)
            {
            var nxt = "";
        
            switch (word[thisone])
                {
                case "cancel" :
                        msg.payload = "";
                        return [null, msg];
                        break;
                case undefined:
                case "the":    
                case "to":
                case "thanks":
                case "thank":
                case "and":
                case "turn":
                case "a":
                case "please":
                case "you":
                case "er":
                case "erm":
                word.splice(thisone,1);
                break;
                
                default:
                ++thisone;
                break;
                }
                
            if (++processed >= total)
                break;
            }
            
        msg.topic = "";
        msg.payload = "OK";
        doStuff.word = word;
        msg.word = word;
        msg.combined="";
        for (a = 0; a < word.length; a++)
            {
            msg.combined += word[a] + " ";    
            }
        }
    return [msg, null];

    case "LaunchRequest":
    msg.payload = "You need help";
    return [null, msg];
    
    case "SessionEndedRequest":
    msg.payload = "Session Ended";
    return [null, msg];
    
        
    default:    
    msg.payload = "Unrecognised Intent";
    return [null, msg];
    }

[/pcsh]

Note I've added a check for the word "cancel" which simply returns nothing, immediately. This is a vital addition as sometimes you just talk gibberish and want to get out of it!

The format response function looks like this…

tmp9E6A

The other blocks don’t have anything in them – other than the user function in which you can make use of msg.combined and msg.word.  When you are done be sure to send the message out – payload containing text of your choice – as Amazon does need a return value even if your text is blank.

So:

Set up an Amazon account – get PRIME if you want their music – if you’re rich get a Spotify account as the Amazon music selection is naff.

Get HTTPS so the end point of your Node-Red page (no real page exists) in my case http://whatever.com/url

That domain or subdomain needs to point to your building – and your router should redirect port 443 (ssl) traffic to port 1880 on your PI or whatever you are using. Why 443 - isn't that a pain for routers which have SSL management and VPN - yes it is - you have to move those to another port. WHY Amazon absolutely insist on port 443 for SSL I don't know - because any port will work but they won't accept them - please by all means do take it up with them.

If you have passwords set up on your Node-Red and your Node-Red UI (which I do) you’ll have to pass them in the  Amazon website string – and I recommend you do this as clearly you don’t want any old Tom, Dick or Harry logging into your Node-Red

Once they are set up is a matter of going to the Amazon developer site with your account – the Alexa tab, setting up s skill including letting it know about your URL – filling in the bits above – and soon thereafter your DOT or ECHO or whatever you are using should be sending text over to you – to do whatever you want with.

Clearly, this will develop – splitting text up into words like this is useless for email addresses – so “Alexa tell the computer to send an email to pete@scargill.org” is simply not going to work – and as for “Aidan” – it makes a complete mess of that Smile

For reference – we made up some notes from our efforts to get an SSL certificate – if you have proven, better ways that cost less and don’t involve re-signing on every now and then as some freebies do – and which you know Amazon will accept as genuine – by all means give us blow by blow info. If you have a better, more complete, simpler description – please do let us know.

From our notes on setting up SSL – https – which may or may not be useful…

You need a trusted certificate to work with Alexa if you want it to talk to your Node-Red setup. Aidan has emailed the Amazon developer guys and eventually got a reply to say they had an issue with self-generated certificates, now resolved - sadly my router setup appears ot have an issue with them and so I can't test this (the router setup in Spain simply WILL not pass through port 443 to an internal unit).

With Node-Red typically you would have separate username and password for external access - this can be re-used with https: and the https: url can bypass the username and password requirement which means you can continue to use UI (dashboard externally).

Obtain SSL from – in our case, https://www.ssls.com

So, the procedure is to pay your money and then use openssl to generate a certificate request (.csr) file.

openssl req -out my.domain.name.csr -new -newkey rsa:2048 -nodes -keyout my.domain.name.key

Enter all the details that are asked for, but when it comes to the ‘Common Name (e.g.server FQDN or YOUR name)’ bit, you must put in the domain name of your server (without the ‘www’ as you get that anyway), so enter ‘my.domain.name’ into the ‘common name’ section and you will be verified for my.domain.name and www.my.domain.name

Before you start the above process, make sure that you can access your Pi (or whatever) externally using your domain name to avoid unnecessary delays.

When you have your .csr, then edit it and cut and paste the full certificate request into the ssls.com request box when it asks for a certificate request.

It will then generate a small text file for you to place in the www/html root of your Pi. Make sure that you can access it externally by cutting and pasting the file name into a web browser, such as http://my.domain.name/AB5678DEF.txt (or whatever the file name is). When you enter that into a browser, you should see the contents of the file appear.

When you’re happy that works, then click continue on ssls.com and it will verify your website and generate your certificate and bundle which you can then get by clicking ‘download’ to get a zip file.

Store this all safely (including the private key file  that you generated with the .csr file) – i.e. make a backup!

You can use a subdomain - so that if you use www.fred.com for something else you can have https://mysub.fred.com for Alexa and Node-Red (verified – I am using a “things” subdomain without issue while my basic root www address is going elsewhere).

Using advanced DNS or similar function with your provider, point mysub.fred.com to your site. In your router, take 443 and point to the 1880 port of your node-red Pi.

We checked and you definitely can’t use anything other than port 443.

Temporarily route port 80 (website in our case) to Pi - as you'll be asked to dump a file in the root folder for verification..... once complete this can be restored to whatever you were using it for in the first place.

Some files need to go into a folder i.e. /home/pi/ssl (our choice) and your info set up in node-red settings.js needs to know where they are...

Need to setup certificate in settings.js (https://... And uncomment ‘fs=’ …etc.

NOTE: I’ve just had to move the lot from one Pi to another – because I messed up the installation of the Pi and had no ports to play with. I can confirm – ALL that is needed is to move the certificate files (in any old directory) to the new Pi and make changes to the Node-Red settings file – uncommenting FS and adding two lines for the certificates - that’s it – all up and running – this is no-where near as complicated as I initially thought it would be.

Also, if your certificate isn’t from a root CA then you need to copy your base certificate into the certificate chain, mainly because node-red doesn’t give you any option to use a certificate bundle file. i.e. edit your .crt file, copy the contents and paste to the top of the ‘my.domain.name.ca-bundle’ file -  this will add your certificate to the chain file.

Use SSL CHECKER  https://www.sslchecker.com/sslchecker  to check validity

Make sure that the certificate is verified all the way up to the issuing root Certificate Authority or Amazon will reject it

When all working - put your port 80 to where it was. After everything works - reboot your router to be sure.

And if I have not said it before – Amazon’s music library is SHITE!

Facebooktwittergoogle_pluspinterestlinkedin

Alexa Node Red and SSL

The DOTIt’s running. I’m very chuffed. Yesterday I picked up my Amazon Dot – which is basically the big ECHO unit without the big speaker – it has a little one instead.

What is Alexa?

Before I go any further – let me backtrack for those who’ve been in – well, a cave or something – what the hell is Alexa? Ok, you’ve seen SIRI and OK GOOGLE etc. and probably talked to one or the other. Well, this is better. A LOT better. This is the fore-runner of the companion you’ll be talking to when you get old!

So the Amazon Echo is a tall, round black box with a speaker that plugs into the wall, hooks into your WIFI and you talk to it and it talks back. The Dot is the same thing, third the price – without the big speaker. It has a small speaker and you can plug another speaker into it or hook to a Bluetooth speaker. For my money, the Dot is the best buy.

How much?

So far so good. But why would you spend £45 on that?

The ECHOThe Dot unit has a microphone “array” – all you need to know is that when it is blasting music – it can still hear you! If you happen to have an Amazon Prime account or a full Spotify account – you can say things like “Alexa, play some Pink Floyd”. Now that might not seem like much – but trust me – this is going to be a party trick for MONTHS. No you can’t play your own stuff – well, that’s probably  not strictly true – depends how clever you are.

Ok, so we have a talking, listening music centre? Is that it? HELL NO!!! Alexa can handle everything including  “what is the meaning of life”. But also – “Alexa – add beans to my shopping list” – and variations (you can get the list of on the somewhat lacklustre APP) – alarm clock, timer – the list goes on.

But where it gets really exciting and frustrating at the same time is when you realise you can attach it to NODE-RED and have it control your house – securely. Where’s the frustrating bit? The HTTPS: bit… it won’t work without it (don’t even DREAM of mentioning certain sites that only work in America or those who are still talking about deprecated code). Well, the good news is that today we tackled this and cracked it and we have this now working in Node-Red without any special libraries and when you see it you’ll immediately realise the fantastic potential – heating, lighting, security, kettle boiling and other important items – all within grasp. Take a Raspberry Pi and a speaker amp and a microphone array and a power supply and a case and spend £60+ or just buy the DOT at £45 !!! The alternatives simply are not worth it as far as I can see.

No – not easy at first – which is why Aidan and I plan to sit down to document the lot. In reality – it should be possible to reduce the whole procedure to half an hour and the cost to a few quid a year. Right now I have everything in front of me and it is merely down to some trivial coding to get my various heating systems, lighting systems and the rest on voice control. My next job after writing this preliminary blog – is to order another DOT.

But I can do this for free?

There are countless stories of alternatives out there. I'll just ignore any hardware solutions that don't use special microphones – and when I asked SEED about the apparent fact that their solution plus microphone board come to MORE than the Amazon product – I didn’t get as much as a reply. As for free SSL? Show me a free SSL that does not require constant renewal, that works with subdomains (xxxx.xxxx.com), which has been proven to work with Amazon's system (please DO) and which doesn’t rely specifically on one machine like a Pi - until then - a REALLY cheap SSL seems a reasonable way forward and despite the sweat at the very thought of it, obtaining and using the certificates was actually quite easy.

Is code needed?

I've seen solutions requiring java and other languages - I wanted to do everything in Node-Red - i.e. generally visually - and this is what we have done. No compiling, no special requirements - just Node-Red and some nodes and functions. There is absolutely no requirement to be using a Raspberry Pi here.

We spent a lot of time yesterday getting to grips with terms like “intent” – I don’t know why people keep coming up with new terms… and initially we were looking at various “intents” but I’m half-convinced you only need one or two and the rest can be done in Node-Red.  I won’t go into the image below – it won’t look like this when we’re done – but if you ignore that “status report” you pretty much have the basics – I can turn a light on an off already via MQTT (but the actual mechanism is irrelevant – I just happy to use MQTT for everything – all my little ESP8266 boards run MQTT ultra-reliably.tmpE39A

So the basic idea is simple enough – in order not to confuse your home control with other stuff, you can hand control over to your stuff using what I would call a keyword… so  “Alexa – ask XXX” or “Alexa – tell XXX” – which seem to do the same thing…  where XXX is your chosen word or phrase. I’m going to call my controller “Charlie” because “controller” sounds too geeky and if you use “house” then when video Skyping your friends with the same keyword you’ll chance to mess up their heating Smile

So (and you can add irrelevant words like please and thanks and to etc…. but I won’t) – I’m thinking – “Alexa ask Charlie location device action”  - for example “kitchen heating up”, “bathroom lights off”, “office heating status” etc…. others may do things a different way but that works for me with the minimum number of words.

When implemented – I’ll be back! But to be clear – this is working NOW, completely reliably and there is no chance the idea above is not going to work.  However today the BT man is coming over to fix our rotten broadband and I’m sure he’ll want to turn it on and off.

About the only down side up to now – is the ATROCIOUS Amazon music library which is as much use as an Ashtray on a motorbike. No AC/DC – WHAT!!!!?!?!?!?!?!? Clearly if not on a budget – Spotify must be worth the investment.

Alexa – play “I’m not in love”

Magic.

Update Monday: We’ve spent the entire day on this – and that flow up there is out of the window – it turns out that all is not perfect in the standard way to handle Alexa – for example you would thing that by defining a range of words that are acceptable, no other words could creep in there – but in fact if you read the Amazon docs – they can. So for “light” you could end up with “flight” – so for all that range checking you STILL need to do your own checks.

We’ve taken a different approach. I’ll soon present some Node-Red code… that will do this. “Alexa tell the computer to turn the kitchen lights on” – the relevant bits of this are “turn the kitchen lights on” – and by the time you remove irrelevant words you are down to “kitchen lights on” – out block for Node-Red will do exactly that, return a cleaned up array of up to 15 words. What you do with them is entirely up to you. No chance of getting words wrong and have Alexa sit and time out. This is all working now – but it is also very late at night. The beauty of doing your own thing is that you can do things Alexa might balk at..

“Alexa tell the computer to turn the kitchen lights off and heating down”

 

So you end up with “kitchen lights off heating down”

Look through a series of acceptable locations, if none, bomb out otherwise look at a series of units – got it…. next parameter is what you want to do. Next parameter is location – but if not found – use the last one – then a device – then status.  In one instruction you might be able to control 3 or 4 items!

I can’t wait to get started – no matter what we add – more messing with the Amazon site!

Facebooktwittergoogle_pluspinterestlinkedin

A Quickie on Itead and CE

I recall a couple of questions in earlier blogs about Itead (Sonoff) and CE. Well, they’ve just written to me to confirm that “CE marking on Sonoff products does mean Conformity European, not China Export.” which should set some minds at ease – they also sent me the test reports.

I’m sitting on a Sonoff Dual, a Sonoff POW and a Sonoff TH16 here. Right now my Amazon DOT has turned up – I’ve just got it hooked into their music system and having great fun – next step is to figure out how to get a cheap SSL certificate running on the Pi so I can use the Node-Red-Contrib-Alexa – that will likely happen tomorrow – THEN I’ll get around to dismantling these Sonoff units and let you know a little more about them.

Facebooktwittergoogle_pluspinterestlinkedin

Another Weekend of Experiments

I’ve just been up to Northern Spain with a friend and our wives looking at some property up there  and when we got to our hotel and got the wine open, Aidan showed me his new toy – Amazon Echo.

The Echo Moment

EchoWell, I was so excited I’ve ordered the new Dot which should turn up next week – after all my failed experiments with Google Now and Microsoft’s  Cortana trying to fire text at Node-Red so I could control things, this might just do the job – but that is something for next week.

FabriqOf course – you could always hang around and wait for the forthcoming  fluffy version called FABRIQ – which appears to be rechargeable – but no mention of microphone arrays so you take pot luck on how good the audio capture is. You could even take the kit route with Seeed’s ReSpeaker – but by the time you add up the cost of ReSpeaker and the microphone array… well, I’m wondering if they’ve lost the plot. They sent me a ReSpeaker to play with minus the microphones – I nearly choked when I saw the additional cost. If you have an Amazon account you can play with Alexa here.

I’ve not had a lot of luck with the Raspberry Pi versions, one only works in the USA, others are out of date and require button presses… anyway I’ve ordered a USA mic any my DOT turns up next week so maybe in a couple of weeks this will start to fly at this end.

We came back to the cave on Friday and yesterday  my friend Jonathan rang me up full of new stuff he’s been playing with.

Glasswire and Divvy

So first things first – for Windows enthusiasts – check out GLASSWIRE. I have it sitting here making pretty graphs of my network – I’m not saying it is any good – I’m saying it is extremely pretty and possibly deserves a corner of your desktop. Something else you might like to play with is DIVVY from MIZAGE. I’ll leave you to look those up. I found them interesting.

Meanwhile as some of you know I’ve been experimenting with Armbian on my little Nanopi NEO. Lovely operating system and I managed to get my script working on it no problem – but for one thing – WIFI  - the NEO doesn’t have WIFI and so you have to stick one of those cheap Chinese WIFI dongles in – and on most of the SBC boards I’ve tried – no problem. However after jumping through some hoops to get the WIFI working on this little Neo board – I was very disappointed – WIFI performance was so poor you could see delays when doing the likes of apt-get update etc. I wrote off to the Armbian forum and the response was along the lines of “you should try an Orange Pi Lite if you want WIFI” – well that’s all very nice but I already have two NEOs.

Well, I was sitting getting depressed  about this when my friend Jonathan rang me up with some new things to try.

WinDiskManager and Etcher

EtcherSo first things first – you know that Win32DiskImager that everyone uses to FLASH Linux SDs – he’s spotted something not only a little faster but a little nicer looking called Etcher – so the first useful takeaway from this blog entry is Etcher. I installed it on my Windows 10 64-bit system no problem – none of your “administrator-only” stuff – and Etcher works  - point it to an image file – and it flashes it to SD!! Mind you that’s ALL it does.  If you need to READ images onto your PC you still need Win32DiskManager.

The DietPi Moment

He then reminded me that before I was playing with Armbian I’d had a go at DietPi for the Raspberry Pi.  It only worked on the Pi itself -  it was a nice, lean installation of Debian – but hey – the normal Pi setup was fine – so I put Diet-Pi to one side.

WELL – take a look at DietPi today – it is no longer for the Pi only – and it can install lots of goodies – kind of complimentary to my script (with a nicer interface).  And in the process of this conversation I noted it has a setup for the NEO and other FriendlyArm boards.

NanoPi NEO using DietPiWell, wouldn’t you know it – there’s a menu driven setup for WIFI – and it WORKS!!! And the funny thing is – the designer credits Armbian – WELL!!  Ok, so there’s a LOT more to this and I’m only at the beginning – I noted that this Diet-Pi on the NEO seems to run even faster than the hardwired version on Armbian – and I’m only at the surface of finding out why – but one reason it seems nippy is you have full control over logging – which by default is running in RAM and at a minimal – a very sensible starting point. 

In the process of setting things up I discovered some sillies in my script (which takes a basic Debian setup and installs a while raft of useful goodies centred around Node-Red) and I’ve fixed those.

I have now run my updated script on DietPi on the NEO and M2 and up to now everything seems to work  – and it’s a small installation for a small board – and nippy… I need to go way more deeply into this –  there is SO much in Diet-Pi now but I’ve no idea how much of it actually works – I DO know that not everything works on all boards but as I’ve only installed this on the FriendlyArm M2 and NEO I’ll keep my mouth shut except to say I’m mildly enthusiastic – my little NEOs are now useful wireless devices – which is a great step forward.

Node-Red on NEO

Experiments above with Node-Red, MQTT and serial – all working.  And below – with a simple mod – a nice colour terminal… all on my NEO soon to be replicated on M2 and T2.

Terminal

Now there DO seem to be some issues with DietPi – I’ve reported them – on a real Pi the serial port needs permissions altering to work (added to script just for belt and braces) – and on all installations, Mosquitto fails to run at power up – MAYBE starting too early I don’t know but taking a line out of the RESTART bit of the start-up script and putting it in /etc/rc.local does the job. 

Up to now everything seems ok and I’m working with MrShark  to looking toward the lightweight LIGHTTPD..  we’ve already improved my script – and you know how it goes – once someone triggers you off…

More of that soon – for now – I have my script (as of today) running (with the caveats above) on a Pi2, M2, NEO and BananaPiM2 – and that’s only due to restricted time… DietPi has the advantage of making a MUCH smaller footprint at under 2GB compared to nearly twice that on, say a standard Pi – as there is only in there what’s needed.

Oh while I think on, here’s the link to my original Neo article.

Expect to hear more about this in the coming week – I’m off now travelling with my wife and 3 cats ALL the way up Spain – 24 hour ferry – ALL the way up England… you won’t hear anything from me in here as I’ve not mastered blogging on a phone Smile

.By now Aidan has his Alexa turning his lights on and off via Node-Red and so that will be the next job – integrating this whole lot together.

Facebooktwittergoogle_pluspinterestlinkedin

Variable Persistence in Node-Red

It is my understanding that Node-Red will eventually have persistent variables - but while we're waiting - this is how I do it. Using the global.get and global.set mechanisms, I store a single object.

So - when Node-Red pops up it looks to the SD for that object - if not there it creates and initialises it. If there, it reads it into the struct. There is a countdown timer in the object and the function controlling all of this is called every 5 seconds. If the counter is zero, nothing happens. If the counter is true, it is decremented. If it gets to zero, the object is saved to disk over-writing the original.  It is that simple and it has worked for me for a long time now.

So imagine a Node-Red user function - with a 5 second timer feeding it (the INJECT NODE) - the payload of the latter is irrelevant.

Here's the function - the object and it's internal bits are mine - you'd alter them for whatever you need.

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

/ The purpose of this code is to update an object on change. At power up check for file and create if necessary
// var is "intel" in this case
var fs = global.get("fs");
var fl="/home/pi/intel.data";

if (global.get("intel")!==undefined) // Remember - don't use !== with null
    {
        if (global.get("intel.Counter")!==0) // if the var exists - now check counter
        {
         global.set("intel.Counter",global.get("intel.Counter")-1); // if counter is non-zero - decrement it
         if (global.get("intel.Counter")===0)  // if counter drops to zero update file
            {
                   fs.writeFile(fl, JSON.stringify(global.get("intel")), function(err) {
                    if(err) {
                        return console.log(err);
                        }
                    }); 
            }
        }
    }
    else 
    {   // If no var (powerup scenario) ..does the file exist?
        try {
                fs.accessSync(fl, fs.F_OK);
                // If file exists create the VAR by reading the file into it
                fs.readFile(fl, 'utf8', function (err,data) {
                  if (err) {
                    return console.log(err);
                        }
                  global.set("intel",JSON.parse(data));
                });
            } catch (e) 
            {
                // Otherwise create both var AND file -  ensuring counter is zero. New bits can be added dynamically
                intel = {
                        ShelfWhiteLight:0,
                        BoilerState:0,
                        ShelfRgbState:0,
                        ShelfRgbLevel:0,
                        ShelfRgbColour:"AAAAAAAA",
                        OfficeRgbState:0,
                        gOfficeRgbLevel:0,
                        OfficeRgbColour:"AAAAAAAA",
                        PergolaRgbState:0,
                        PergolaRgbLevel:0,
                        PergolaRgbColour:"AAAAAAAA",
                        Counter : 0
                    };
                    fs.writeFile(fl, JSON.stringify(intel), function(err) {
                    if(err) {
                        return console.log(err);
                        }
                    }); 
            }
    }

[/pcsh]

 

 

Facebooktwittergoogle_pluspinterestlinkedin

Node Red Sunday

Node-RedIt is Sunday, I’ve done my 4,000 steps down to the village for coffee and back and I’m having a play.

I use Node-Red on a Raspberry Pi to control just about everything in two houses – as my confidence in the setup has grown, I’m currently keeping the Pi up 24/7 with no sign of any issues (well, it could ALWAYS be faster). Normally I control the likes of ESP8266 boards with this to do everything from monitoring temperature, pressure and humidity to controlling heating, fans, lights and more.

It occurred to me, as I’ve also had success with my script in getting Node-Red, Sqlite, MQTT and other software running on other SBCs including the really cheap new ones coming out of the woodwork that it would be nice to be able to talk to them with the same software especially those running Armbian as I’ve had a lot of success with that personally.

So I’ve made a start. I thought you might find this preliminary effort interesting.

In my controllers, MQTT is at the heart of everything and facilitates utterly reliable message sending from device to Pi and Pi to device (indeed there is no reason I could not do device to device really). I thought it might be nice to start off with a little bit of code that simulates what the ESP8266s do constantly – turn things on and off.

So typically, given a device called fred, I might turn a relay on or off using MQTT messages:

topic: fred/toesp
payload: {out0:1}

Let me first explain why I do it this way. So MQTT typically works by sending simple messages back and forth – they have a topic (address) and a payload (data).  My devices, for example fred, listen for two topics:

fred/toesp
esp

That way I can send a message to a SPECIFIC device – or to all of them at once (OTA for example)

If the device needs to send messages back they will always be of the form:

fred/fromesp/out0

in this case the payload returned would be the current state of out0.

So that represents the basics. Here I will show how a cheap controller running Node-Red can accept commands – I won’t go into the actual mechanics of turning a light on and off because for example the GPIO commands used in a Raspberry Pi won’t work on a NanoPi NEO – alternative methods are used. I’ll simply cover how to get to that point.

So here is the simplest possible setup on the Pi, to turn a couple of outputs on and off.

MQTT

This is nothing more than 4 INJECT nodes dropped onto a flow along with an MQTT output. Typically the content of one of these nodes might be:

INJECT

Simple enough. When pressed the node outputs a topic of nanopineo2/toesp and a payload of {out0:1}

This would work with one of my ESP8266 units to turn output 0 on. “nanopineo2” is just the name of one of my NEO units running Node-Red.

Here’s what’s at the other end to make this work.

Nanopi Neo

Two MQTT inputs feeding a function. So the function will work if either the generic “toesp” is called or a device-specific version. In that first function I check out the command – really simple – and will be dramatically improved soon.

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

// An attempt over time to emulate my code in ESP8266 - taking commands from MQTT
// and processing I/O
// for example message {out26:1}

var m=msg.payload;

if ((m.charAt(m.length-1)=="}") && (m.charAt(0)=="{")) // ignore if not in braces
{
    m = m.substring(1, m.length-1);
    var breakup=m.split(":");
    var getname=breakup[0];
    var sendto=breakup[1].split(",")
    
    switch (getname)
    {
    case 'out0' :  msg.topic='out0'; msg.payload=sendto[0]; node.send(msg); break;
    case 'out1' :  msg.topic='out1'; msg.payload=sendto[0]; node.send(msg); break;
    default: break;
    }
}

[/pcsh]

The output is reduced to the actual command “out0” for example – and a value. So the next item is a switch in this case selecting “out0” or “out1” and ignoring everything else. The outputs from that switch go to functions which simulate actually controlling something – like a LED for example and in my demo the colour of the black dot changes from black to red when you send a command to turn that output on.

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

if (msg.payload==1)
  { 
      msg.payload="lamp1.val=100"; node.send(msg); 
      node.status({fill:"red",shape:"dot"});      
  }
else
  {
      msg.payload="lamp1.val=0"; node.send(msg); 
      node.status({fill:"black",shape:"dot"});
  }
  

[/pcsh]

And that is it for now. As you can see, this can RAPIDLY be expanded to cover every port bit available on your micro, it could control PWM etc. The next step will be to take {command:arg1,arg2….arg6} and parse the arguments into an array. That will then allow PWM and other kind of controls. If you really don’t care about pricing you could do the whole lot with a bunch of Raspberry Pis… but with new Orange Pi boards coming out at under a tenner, the options are endless.

Facebooktwittergoogle_pluspinterestlinkedin

IOT Manager Revisited

This started off as a very short blog some time ago – you’ll see  comments below but I’ve re-written it. I occasionally like to revisit stuff I have reviewed or played with ages ago because, well, things change. Had I not persevered with ImperiHome I would not now have a perfectly good IOT controller on my phone as originally it did not support Node-Red (Peter Oakes and I soon fixed that - documented elsewhere) and that package is now my preferred central control tool (the fact that it is now bog-standard fitment on the world’s most popular SBC – the Raspberry Pi – is another good reason to stick with it).

(Some regular readers may ask – what happened to BLYNK? – well,  it is still running back in the UK reliably – has been doing so for nearly 6 months – I just happen to prefer Imperihome).

IOT ManagerSo, IOT Manager is a weird one – the programming is different to many of the systems out there which try to be all things to all people – supporting endless devices many of which will no doubt bite the dust in time and it is not yet as advanced as either Blynk or Imperihome, but the author is keen – has done a LOT of changes for me and is rapidly approaching the point where it will compete with the likes of Imperihome.

http://esp8266.ru/iotmanager/

Let me say that at the time of writing you need the BETA version of IOT Manager to get the best out of it. Joinging the beta program is trivial.

So on first revisiting IOT Manager, could I HELL remember how to use it.  One of it’s original “features” was support for websockets for MQTT – fine – but it ran rather slowly and I think that’s one reason I abandoned it – so when I noted that it now also supports MQTT directly – with username and password – and figured I’d give it another shot.

So I loaded up the Android App and filled in my MQTT broker details and… nothing. Try as I might I could not figure out how to add a widget. All I had was a screen that said “no data” and a green indicator showing that it was indeed connected to my MQTT server (public address). It didn’t help that the website went on about Arduinos and stuff and really didn’t seem to get to the bottom of how to quickly get started if you didn’t have one of those. The documentation has still not quite caught up with developments.

ChartHaving loaded up IOT Manager there are no settings on the phone to create widgets  – instead, MQTT messages tell it what to do and YOU send those from, for example Node-Red! It’s really easy once you get past that hurdle.

So how the package NOW works is to initially send out a request for updates on the number and details of pages needed – and to ask for details of widgets required.

From then on the phone will request updates on specific page widgets as and when a page is selected or focus is lost. This minimises the information that needs to be sent.

When a button is pressed the info is sent back from the App by MQTT. I’ve chosen to process all of this with Node-Red. Plain NodeJS would no doubt do just as well.

Here is a list of the current widgets which includes sliders and power buttons as well as toggles…

  •       anydata
  •       simple-btn
  •       display-value
  •       small-badge
  •       power-button
  •       range
  •       toggle
  •       SteelSeries
  •       fillgauge
  •       chart

The CHART widget is new and I’ve not yet had time to test it but here’s an example on the right above.

In the example below I have three pages (named page1, page2 and page3). On the first page are two switches and a slider (range7). On the second are 3 level gauges and on page 3 are a couple of thermometer-type displays.

IOT Manager - my Node-Red code - Peter Scargill

This is what they look like…

1[4]  2[4]  3

What we have here is an APP that connects to your MQTT server (or a public one) and at that point sits and does nothing.

Essentially once the phone is connected it is listening to an MQTT subscribe. Your controller – say, Node-Red on a Raspberry Pi or even a simple ESP8266 (but you still need an MQTT broker either locally or a publically-available broker)  when it turns on and connects to the same MQTT broker, sends some init code and magically a bunch of controls appear on the phone – you can then use the controls and they send out info back to Node-Red or whatever you want to use as a central controller. What is great about this is that it could be dynamic – as this is set at the control end, not the App end.

I centralise everything via Node-Red which of course can publish and subscribe to MQTT and then control my devices from there. I decided to write the code for Node-Red to give others a starting point.

So you can see in the diagram above the two SUBSCRIBES. That info is passed to a couple of functions – here they are…

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

var topic=msg.topic.split("/")[3];
global.set("IoTmanager_" + topic,msg.payload);

[/pcsh]

And now the second function...

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

var prefix = "/IoTmanager";
var deviceId = "bedrock";
var config = [];

var pagesList = {
    pages: [
        { pageId: 10, pageName: "page1" },
        { pageId: 20, pageName: "page2" },
        { pageId: 30, pageName: "page3" }
    ]
};

var id = "0";
var widget="toggle";
  config[0] = { 
  id : id,
  page: pagesList.pages[0].pageName, 
  pageId: pagesList.pages[0].pageId,
  descr : "pump 1",
  widget : widget,
  itemstyle : "",
  itemclass : "",
  topic: prefix + "/" + deviceId + "/" + widget + id,
  color : "orange",
  style : ""
};


  id = "1";
  widget="toggle";
  config[1] = {
  id : id,
  page: pagesList.pages[0].pageName,
  pageId: pagesList.pages[0].pageId,
  descr : "pump 2",
  widget : widget,
  style1 : "",
  class1 : "",
  topic: prefix + "/" + deviceId + "/" + widget + id,
  color : "blue",
  style : ""
};

  id = "7";
  widget="range";
  config[7] = {
  id : id,
  page: pagesList.pages[0].pageName,
  pageId: pagesList.pages[0].pageId,
  descr : "range 1",
  widget : widget,
  topic: prefix + "/" + deviceId + "/" + widget + id,
  color : "blue",
  style1 : "\"style\":\"range-balanced\"",   
  badge1 : "\"badge\":\"badge-balanced\""
};


  id = "2";
  widget="fillgauge";
  config[2] = {
  id : id,
  page: pagesList.pages[1].pageName,
  pageId: pagesList.pages[1].pageId,
  widget : widget,
  class1 : "item no-border",
  style1 : "",
  descr  : "Water level",
  class2 : "light text-center",
  style2 : "font-size:24px;font-weight:bold;padding-bottom:5px;",
  topic: prefix + "/" + deviceId + "/" + widget + id,
  width  : "150",
  height : "150",
  class3 : "text-center",
  style3 : "",
  widgetConfig : {
  }
};

  id = "3";
  widget="fillgauge";
  config[3] = {
  id : id,
  page: pagesList.pages[1].pageName,
  pageId: pagesList.pages[1].pageId,
  widget : widget,
  class1 : "no-border text-center col-xs-4",
  style1 : "",
  descr  : "Fuel level",
  class2 : "assertive text-center",
  style2 : "font-size:14px;padding-bottom:5px;",
  topic: prefix + "/" + deviceId + "/" + widget + id,
  width  : "70px",
  height : "70px",
  class3 : "text-center",
  style3 : "",
  widgetConfig : {
    circleColor : "#FF7777",
    textColor : "#FF4444",
    waveTextColor : "#FFAAAA",
    waveColor : "#FFDDDD",
    circleThickness : 0.2,
    textVertPosition : 0.2,
    waveAnimateTime : 1000
  }
};

  id = "4";
  widget="fillgauge";
  config[4] = {
  id        : id,
  page: pagesList.pages[1].pageName,
  pageId: pagesList.pages[1].pageId, 
  widget    : widget,
  class1    : "no-border text-center col-xs-4",
  style1    : "",
  descr     : "Water level",
  class2    : "energized",
  style2    : "font-size:14px;padding-bottom:5px;",
  topic: prefix + "/" + deviceId + "/" + widget + id,
  width     : "70px",
  height    : "70px",
  class3    : "text-center",
  style3    : "",
  widgetConfig : {
    circleColor : "#D4AB6A",
    textColor : "#553300",
    waveTextColor : "#805615",
    waveColor : "#AA7D39",
    circleThickness : 0.1,
    circleFillGap : 0.2,
    textVertPosition : 0.8,
    waveAnimateTime : 2000,
    waveHeight : 0.3,
    waveCount : 1
  }
};

    id = "5";
    widget="steel";
    config[5] = {
    id     : id,
    page: pagesList.pages[2].pageName,
    pageId: pagesList.pages[2].pageId,
    descr  :"Steel 1",
    widget : widget,
    topic: prefix + "/" + deviceId + "/" + widget + id,
    widgetConfig : {
    width  : "auto",
    height : 100,
    type   : "Linear",
    titleString: "Thermometer 1",
    unitString : "temp C",
    threshold: 90
}
};

    id = "6";
    widget="steel";
    config[6] = {
    id     : id,
    page: pagesList.pages[2].pageName,
    pageId: pagesList.pages[2].pageId,
    descr  :"Steel 2",
    widget : widget,
    topic: prefix + "/" + deviceId + "/" + widget + id,
    widgetConfig : {
    width  : "auto",
    height : 100,
    type   : "Linear",
    titleString: "Humidity 1",
    unitString : "% Hum",
    threshold: 90
}
};


// extract command and param
var msg2 = JSON.parse(msg.payload.toString());

if (msg2.command==="getPages")
    {
     msg.topic=prefix + "/" + deviceId + "/response";
     msg.payload=JSON.stringify(pagesList);
     node.send(msg);   
    }

else if (msg2.command === "getPageById" && msg2.param > 0) 
    {
                 pagesList.pages.forEach(function(item, i, arr) {
                // it is one of our pages?
                if (item.pageId === msg2.param) {
                    // node.warn('Request is for existing page "' + item.pageName + '", pageId="' + msg2.param + '"');
                    pubPage(msg2.param);
                }
            });
    }

else if (msg2.command === "getPageById" && msg2.param === 0) 
        {
            //node.warn('Request is for All pages.');
            pubPage(0);
        }






function pubPage(page) {

    // check all widgets and pub widgets config from requested page
    config.forEach(function(item, i, arr) {
        if (item.pageId === page || page === 0) {

            // pub config for one widget
            msg.topic=prefix + "/" + deviceId + "/config";
            msg.payload=JSON.stringify(item);
            node.send(msg);
            //node.warn("Page " + item.pageId + " id " + i);
            // pub status for one widget
            pubStatus(i);
        }
    });
}


function pubStatus(i) {
var status;
switch (i)
    {
        case 0 : status=global.get("IoTmanager_toggle0"); break;
        case 1 : status=global.get("IoTmanager_toggle1"); break;
        case 2 : status=global.get("IoTmanager_fillgauge2"); break;
        case 3 : status=global.get("IoTmanager_fillgauge3"); break;
        case 4 : status=global.get("IoTmanager_fillgauge4"); break;
        case 5 : status=global.get("IoTmanager_steel5"); break;
        case 6 : status=global.get("IoTmanager_steel6"); break;
        case 7 : status=global.get("IoTmanager_range7"); break;

    }
msg.topic=config[i].topic + "/status";
msg.payload=JSON.stringify({ status: status });
node.send(msg);
}

[/pcsh]

As for individual items - well, to turn on or off a switch is as easy as...

[pcsh lang="js" tab_size="4" message="" hl_lines="" provider="manual"]

var fred=msg.payload;
msg.topic="/IoTmanager/bedrock/toggle0/status";
msg.payload="{\"status\":\""+fred+"\"}";
global.set("IoTmanager_toggle0",fred);
return msg;

[/pcsh]

Depending on whether you fire the string "0" or "1" at that function it will turn on or off the switch.

The main function above is comprised mainly of definitions for the various items and you can find out more about them at the author's site - indeed that IS the place to look in case he changes anything and expect him to add more in future.

So - do we have an up and coming competitor for Blynk and Imperihome? I'll leave that for you to decide.

Facebooktwittergoogle_pluspinterestlinkedin