Category Archives: javascript

Node Red Initialisation

There have been various conversations about Node-Red initialisation of variables recently. Here’s a solution.

So firstly – what’s the problem? Well, because of the asynchronous nature of Node-Red, it seems to be that sometimes you find yourself using a global variable before it has been defined. No lectures on using global variables please. I can see in my own systems, in the log, incidences of uninitialized variables.

My own solution to the issue of non-volatile storage has been to save global variables as a single object, into a file – and restore said on power up (thermostat settings etc.) – you could store them in MQTT, in a database, anywhere – doesn’t matter. The issue is restoring them.

On power up one assumes the left most tab in Node-Red will run first and that is generally true – but where anything asynchronous is happening – like getting a file for example (hint) there could be delays and other code could run using one of your variables before you’ve actually set it up. This usually does NOT result in Node-Red crashing but you can get rubbish results. Of course you can check every time you use a variable if it exists but that can get messy REAL quick.

A conversation started in the Node-Red discussion groups recently – one of many - and I just latched onto an idea and ran with it.  Of course my first few attempts resulted in suicidal feelings as they failed time after time – but just mid- morning it all started to come together.

So, first things first – the Node-Red settings.js (created when Node-Red is installed and could be for example at /home/pi/.node-red/settings.js)  file runs BEFORE Node-Red comes online – so you know that if you initialise stuff in there, it will be set up in time for you to use – but messing with that file from within Node-Red could be dangerous and you could kill the whole installation. It also  defeats the object of having everything in one nice visual interface.

Here’s another way

Within settings.js there is “functionGlobalContext” which sets up any globals you might need for example – fs object for using files. Here is that section (typically) with one line added into settings.js

functionGlobalContext: {

os:require('os'),

moment:require('moment'),

fs:require('fs'),

mySettings:require("/home/pi/.node-red/redvars.js")

    },

See that last line with “mySettings” -  you can call that what you like.

In (for example) your /home/pi directory, create a simple file called redvars.js (or anything you like really). In there you could put this..

and thanks to Nick O’Leary for the specifics…

module.exports = {

    "fred": 1,

    "brian": 2

}

However, what happens if the redvars.js file is corrupt for any reason – answer? Node-Red crashes..

Here then is a better way:  

At the VERY START of setting.js add this..

var mySettings;

try {

mySettings = require("/home/pi/.node-red/redvars.js");

    } catch(err) {

mySettings = {};

    }

So mySettings is either populated or made empty. No crashing. Now you only need to add ONE line to the functionGlobalContext – here it is in my case… yours might be different…

functionGlobalContext: {

os:require('os'),

moment:require('moment'),

fs:require('fs'),

mySettings:mySettings

    },

As you can see, with an additional comma then one line, a foolproof version of mySettings has been added.

Run Node-Red and you will now find that (for example)  global.get(“mySettings.fred”)  returns 1.  SO – now all you have to do is to put what you need in that file redvars.js – and if you need to update them you can do that within Node-Red. 

In my case I have a variable called counter in there.   Every time I update something, I update that counter to 10;   I have a little timer job running that checks that var and if not zero, decrements it – if it is now zero, it updates the file… see how I did that below.

If you look at my original article on this subject – all seemed well, simply stringify the object and save it to disk – and that’s really how I’d like it to be - but my mechanism for RETRIEVING the object on power-up failed miserably because of the async nature of the file read – often resulting in use of the variables before they existed. This way, that won’t happen and all it needs is to stringify the object and add “module exports-“ at the start.

In the unlikely event that the redvars.file is corrupt, on power-up, mySettings will be undefined – easy to check and if necessary overwrite.

So one simple change on initial setup of Node-Red – and from there on you can create or amend non-volatile global variables from within Node-Red.  You could expand on this of course – that is up to you.

Update 26/03/2017 - This modification is now in the script!

Facebooktwittergoogle_pluspinterestlinkedin

The LCD Object

lcdIt has been a very busy week for us here in Spain – what with partying and what-not, so I’ve not had as much time as I would like to develop my JavaScript object skills, however, I’ve managed to get much further with a project I’ve been working on in the background – a little LCD display.

I wanted to simulate a real LCD and you’ll have seen in an earlier blog entry that I managed some JavaScript code to do this.  Well, I’ve had all SORTS of unexpected issues, now resolved and I’ll detail here what I’ve done.

colour changeSo first off, I was having some interaction with other objects on the page – resolved – but an issue that came to light when I got here to Spain – with my little Raspberry Pi that is running this stuff left in England – was that of loading external items – for example, images and fonts.  Designing items for Node-Red Dashboard is slightly different to working on an empty web page as you’re basically working inside a DIV – you don’t have the luxury of ONLOAD and HEADERS etc. and I found myself getting annoyed with my web seven-segment font not working when first loaded  - i.e. initial text would show up in a default font because of the time it takes to load the web font from England. I could not understand why this would always happen – surely there’s some caching going on? But if you read the comments in previous blogs – that apparently is not the case as the templates in Node-Red Desktop add a timestamp to URL calls which kills the cache. Well, not ALL of it but certainly makes life difficult when trying to load up fonts and images. 

LCD in redAnyway, the first idea that came out of various comments was to URL encode the fonts – and of course, wanting to keep with good practice I put all of that into a separate style sheet. NO! The page has to load the style sheet, so the problem doesn’t go away. The answer lay in having the font inline – now that SOUNDS messy but actually it’s not that big a deal especially if you don’t have wrapping turned on – it’s just a one liner – albeit a very long line. The breakthrough came when I read a suggested link which took me to Fontsquirrel – having initially used this 14-segment font for my LCD only to find that the capital S was a bit naff and the spacing was NOT identical on all characters, I was then quite excited when I found an excellent WOFF file called segment14 here.  Mono spaced and with a decent S.

Lovely, but of course this still had the same issue with loading – not a big issue at home but as soon as myself and the source were separated by a plane trip – the loading issue became apparent. Anyway, Joe Lippa and Antonio (Mr Shark) have been feeding me links and it turns out that FontSquirrel can load up the font and return a CSS file with the code all inline (they don’t appear to have 7 or 14 segment fonts so it is as well you can upload fonts you have found -  and have them converted) – so no external loading.  In the data that came back was a WOFF data block and a WOFF2 data block – I ditched the former as the latter works just fine.  I’ve put links to fonts here – I’ve modified them for my own use but you should know I’m not claiming any ownership of the fonts – others have done the excellent work on designing the fonts.

Purple LCDWith the font issue resolved I set about making a basic object for a display (which will no doubt mutate – but right now can be dropped into a template in Node-Red to show lines of simulated 7-segment display in a variety of colours. Methods can be invoked to updated the two lines and to change the colours – both background and foreground. I managed my colours by trial and error but you can of course change them and add more.

Simple testing of the LCD objectThis, then, is merely the first stage – everything works – you can change colours and text… so much more can be done – choice of how many lines, how many characters, what font size etc. but all of that is easy to add once the basic idea is in place. This is going to be added to my armoury of gauges and shapes available in Node-Red Dashboard – I’m playing with the GITHUB beta right now and it is coming along VERY nicely – now that the actual template width and height can be tweaked.

Anyway, there it is… if you want to play with this – grab the code and paste into a template via the “import clipboard” option (top right – 3 bars)– yes, it’s big because of the font – but very do-able. You can replicate this and make two or more displays but more likely the object and font should be split off into another template if you plan to do that. 

JSON

Note that in my test inject nodes I am now injecting JSON, not text (see image above and note the {} which means JSON is selected – then I’ve filled in some standard JSON text to adjust two variables) – just as easy but more powerful as you’ll see in the text injectors.

When this information comes into the node it is processed as such…

(function(scope){
        scope.$watch('msg', function(msg) {
            if ((msg.payload.lcdm1!==undefined)&&(msg.payload.lcdm1!==undefined)) lcd2.update(msg.payload.lcdm1,msg.payload.lcdm2);
            if (msg.payload.colour!==undefined) lcd2.updateCol(msg.payload.colour);
        });
})(scope);

Right now I don’t allow on or the other line to be processed but only both at once – but that would be easy to change.

If you look into the code there are two methods (functions) of relevance once the object is defined – this.update and this.updateCol – which are merely setting internal variables then calling the update function.  The text update function could easily be split into two.

At the very end, the LCD is created, needing only a DIV to work in..

var lcd2 = new myLcd("newDiv1");

I called it lcd2 as I also tried putting several on a page to ensure there was no interaction. They need unique names and unique div names and several will co-exist no problem (though it would make sense to have only one copy of the object definition if you want to do the job properly).

See previous blog entries for related info on LCD simulation (alpha blending of the all-segments on character under the actual text).

Specific questions are invited in the comments area (not 2 years later please).

Facebooktwittergoogle_pluspinterestlinkedin

A Deep Learning Weekend

Gauge by Peter ScargillWell, this has been an interesting journey (a deep learning WEEK more like it) as I’ve been working on making my own widgets for Node-Red. I thought I’d share some of the things I’ve picked up. Experienced JavaScript object users please look away now. Those not interested in JavaScript look away now.

Note: The original video for this is out of date as I have learned more – the new video has some extra code, too – here it is - https://youtu.be/mTAPrvB1EeI

C Programmer: As a life-long C programmer, I came into JavaScript with some trepidation. Like alternative universes, JavaScript may look similar to C – but it is not the same – and that led to all manner of silly mistakes over time. However as I see a tiny light at the end of the tunnel, I have to say, it was worth the effort. Despite being a client-side interpreted language, it really is great fun to use.

The gauge: Well, as long as you keep it simple that is. When designing my first attempt at a gauge for Node-Red, I realised that a typical gauge operation goes like this:

Some features such as the background are typically set in an init function once only (and as such it would be better if the user is not involved in this. Other features such as needle positions are set dynamically and will change during operation.

This is generally handled one of two ways – either by setting a value and then calling an update function, or by running something in the background which checks for value changes so all you have to do is change a variable. On many of the gauges you’ll see flashing lights and smooth movement of gauge needles – that implies something running constantly or at least checking for changes and running when change is needed.

Concerns: This worries me as there can be quite some overhead in updating an entire gauge, umpteen times a second. So I decided that what was needed was a layering system as you might find in Photoshop – such that I could lay down the basic design and background of a gauge once only then dynamically clear and write to a top layer whenever changes are required. Such changes might include the state of a flashing light – or a needle not being where you want it, requiring successive incremental updates until the needle or needles do get to be where you want.

I then started thinking about variables and functions (methods). Did I want to update variables such as the needle pointer directly (direct variable manipulation) – or did I want to manipulate the input value before passing it through (that needs  a method/function).

That led me to create, as you will have seen earlier, a large blob of JavaScript for my gauge, including initial setup – then a timer-driven function to update the gauge regularly. That was stage one.

HTML5 Canvas: In the process I discovered (by looking at what others had done) that HTML5 Canvas (which really is easy once you get past the initial learning curve) Is a great choice. What did not come so easily was the realisation that the canvasses themselves are indeed transparent and you can layer one directly on top of another – just like Gimp or Photoshop layers. That led me to split the job into two – the overall backdrop which remains fixed throughout operation – and the top layer with my LEDs, needles and set points on it.

Adding a bunch of variables to this for setup and defaults brought me out into a sweat as I struggled to imagine how two or more gauges would interact. And THAT led me onto objects – something I’d never really touched in JavaScript before  – specifically to wrap up my entire code into a definition – which meant I could then simply create instances for EACH gauge without worrying about all those variables interacting. That was the point I last few days in deep learning.

Having wrapped everything up in a bulletproof package I was then faced with the question of how to access required functions and data externally. The obvious way (and the one I’ve adopted) is a series of callable functions for example:  mygauge1.needle1(50);

In the process I had to learn how to hide functions within the definition and make functions accessible externally. That was easy enough.

Here is a simple example – the assumption is that you have a jQuery link (to update a div) and a DIV called “fred”

1.  function myobj(){
2.  this.v1=5;
3.  init=function(passThis) { setInterval(every25ms, 25,passThis); }
4.  this.doubleit=function(r) { this.v1=r*2; }
5.  every25ms=function (passThis) { $("#fred").html(passThis.v1); }
6.  init(this);
7.  }

8.  brian=new myobj;
9.  //brian.doubleit(12);
10. //brian.v1=7;

There is a lot to this – and I’ll be using the word “this” a lot – so for clarify I’ll use italics for the special meaning of this.  It may look strange at first but if you follow me it soon won’t.

Definition: So the point of this was to create a test object definition – an object that would simply put a value in a DIV (I know, slightly overkill but I was trying to prove a point).  We start off with the definition in line 1.

Take a look at the code above – there is a lot in here despite their being only 9 lines. I figured any more and people might run away.

Firstly on line 1,  we define a type of object. A “myobj” type.  I’m not making anything here – just defining it (ends in line 7).

On line 2 – we define an externally available variable – v1. “this” makes it accessible externally (try removing “this”).

In line 3, we are defining  a function called init.  When the instance is created on line 8, init will be called internally (by line 6) and will start off a timer which will run every 25ms and which will call an internal function called “every25ms” (line 5), passing this to it. Do not get hung up on this yet.

Line 4 defines a function (again made available externally by “this” and called “doubleit”,  will double whatever is passed to it.

The  “every25ms”  function on line 5 which is to be called every 25ms by init… simply prints out the local variable.

Line 6 calls init (which is NOT accessible externally) when the instance is created in line 8 - to initialise the instance and start off the timer – this saves running an init externally.  From line 8 onwards,  the function “every25ms” is called every 25ms to put the value of this.v1 onto the DIV.

You’ll see “5” appear in your DIV if you test this somewhere – that’s the initial value. Note "passThis" – (arbitrary name). Because the function init is private - I don't prefix it with "this" – when it sets the interval timer it has pass to the timer function the outer “this”… - so I pass "this" in the init call.  This is new as a result of feedback in here suggesting a private “init” function that was no accessible externally. In the original version I used this.init() - and passing this worked. Removing that resulted in the timer callback function using the wrong "this".  Trace it through and it makes sense. The callback function needs to know about the outer "this".

Line 8 is where things start to happen. At that point, the instance (brian) is created and the timer starts all on it’s own -  thanks to the internal init function.

So far so good. How do I now interact with that variable v1? I wanted the choice of updating a variable or calling a function. The two commented lines are two ways to do this – their operation should be self-explanatory.

Line 9 simply won’t interact with local functions inside the definition and we don’t want them to – we want interaction with specific functions in each instance of the object – like “brian” for example. And that’s where “this” comes in. Add “this” to a variable name or a function name – and it becomes accessible to the outside world. For example you CANNOT do brian.init() or worse, brian.every25ms() and that’s a good thing!

Scope: So using this technique allowed me to keep all variables and functions private and only “expose” those I needed. LOVELY.  If I only wanted to use function and not expose any variables – I would not use for example “this” as a prefix to v1.

These ideas then form the basis of my current gauge. I can built a massive machine with lots of variables and functions inside and have no worries about interactions with other gauges because I’m only “exposing” stuff I want…. and gauge1.v1 has nothing to do with gauge2.v1

Of course I’m assuming that was as clear as mud -  but if you are interested, go have a play in this CodePen where I’ve left the code. See the number that comes up in the working area – try un-commenting those functions (methods) in lines 9 and 10. See what happens if you replace passThis.v1 with this.v1 or just v1.  See what happens if you remove references to this altogether. CodePen is great as changes run immediately and safely.

Merely experimenting with the code in that CodePen should do what it did for me – clear a massive amount of mental fog.

Simply creating v1 – and removing this and passThis in referring to v1 – leads to disaster – v1 is accessible externally on it’s own –so kiss goodbye to multiple instances.  But the same code using “var v1” leads to a save internal v1 NOT accessible as “brian.v1”. Your choice.

My new gauge object is coming along nicely – and a minimal installation involves little more than a link and a couple of lines of code – stunning how these simple ideas change the whole game.

Facebooktwittergoogle_pluspinterestlinkedin

Some thoughts on Node Red variables

In learning how to use Node-Red I’ve changed my coding countless times as I’ve found better ways to do things.

Please note that in 2017, the way I use global variables here is out of date. It still works but for example context.global.fred=1 is now written as global.set("fred",1). Read this later article on using globals and context variables.

I’m currently working on my thermostat code using a Nextion touch-sensitive display, Raspberry Pi2 as controller and ESP8266s for relay and temperature/humidity sensing. This was a slow-running project until my old Arduino-based heating controller packed in yesterday without warning and so now the fate of the household heating is in the hands of an on-going project.

So the general idea is easy enough – lots of settings for weekdays and weekends, read sensor, control relay. That’s pretty much it in a nutshell except that on my first attempt I didn’t take into account power cuts – and when the power was restored to the Pi, hardcoded defaults took over instead of the settings I’d put in using the display.

So there’s a need for lots of variables and a means to store them permanently. This is what I’ve come up with and the logic behind it:

In order to pass variables around in Node-Red, you need to use “context.global” variables as these are the only ones that will work from page to page (i.e. from powering up node-red to powering down across all pages of your project). This can make for messy variable names for those of us who like to keep things short because we’re lazy or developing typists finger.

context.global.stat_inside_temperature

etc..

I’m a C person and so a struct is ideal for me – keeps everything together.  The nearest I get to that in JavaScript is an object and that’s fine.

A global object containing everything to do with my stat.

   var stat = {
display_temperature : 22,
display_humidity : 45,
heating : 0,
display_external_temperature : 14,
display_external_humidity : 54,
display_status : "Normal",
display_target_temperature : 20,
display_enabled : 0,
display_last_page : "home",
display_last_colour : 0,
pass : "",
display_dim : 60,
peak : 21,
night : 16,
frost : 12,
hold : 0,
weekon : 480,
weekoff : 720,
weon : 480,
weoff : 720,
week2on : 780,
week2off : 1380,
we2on : 780,
we2off : 1380,
temperature_manual_override : 0,
update : 0
};

That stat var of course is only valid while the relevant function is in use and is invisible outside of that particular page hence…

context.global.stat=stat;

So now we have a nice but somewhat longwinded way to access the variables..

context.global.stat.display_temperature;

(I know I should probably use displayTemperature but there’s no global replace in the Node-Red editor (hint)).

Because JavaScript objects are passed by reference – you can use a reference to this in other pages which makes things a little more readable.

var stat=context.global.stat

Hence

stat.display_temperature;

That’s more like it and you can write to that new variable and it will update the original as they are one and the same.

The obvious solution for storing variables permanently, is a database, but I’m trying to minimise that to keep writes down for the time when I run all of this and much more on an SD (currently using a little 2.5” drive with the Pi2).

The file node lets you read and write text files and so the next challenge was how to get that object into a text file – easy:

msg.payload=JSON.stringify(context.global.stat);

That JSON string can be saved directly to disk and the inverse is true of course…

context.global.stat=JSON.parse(msg.payload);

The good part of that is that the string saved to disk is readable and if need be directly writeable.

Thankfully when the file node reads a file if the file is not there, it returns “undefined” so in my function I check for undefined and if so – create a new one with defaults in it. I added a timeout variable in there and whenever I update any item in the object I also set a counter to 10.  That counter is decremented every second and when it reaches zero, the object is updated on disk. That means I can do all sorts of adjusting of controls on the Nextion display and only when I’m done will there be a single block write to the disk. The READ is only done once when Node-Red powers up.

Seems like a reasonable solution and it’s nice and warm in here.

In an earlier blog you’ll see my Nextion display solution which is quite pretty ( created my own imagery because built-in Nextion options are currently not that good) – here just for reference is the current Node-Red block controlling the display and the stat itself including disk reading and writing.

tmp7866

Facebooktwittergoogle_pluspinterestlinkedin

Drilling into Blynk

zeRGBaYesterday I referred to using DELAY in Node-Red to slow down one of the Blynk controls. To backtrack a little…  We’re talking here about controlling devices such as ESP8266 boards using MQTTT – which I maintain is the best way to have a boatload of devices around the place and talk to them. I do this from a Raspberry Pi 2, using Node-Red which is a visual system, relatively easily fitted to the Pi (see my articles about my script). Once you have this in place, you can use simple (but excellent tools such as MQTT-Spy to send out MQTT messages to your devices and get input from them.

But in the real world who wants to sit there programming a tool just to turn a light on. So there are various options including the rather expensive Raspberry Pi touch display. A better option IMHO for local operation are the Nextion touch displays which work serially. I’ve covered these elsewhere -  a project that is still in it’s early stages but with a little work is usable – so much so that I plan to use them in my place here in Spain and back in the UK as thermostat wall mounts – I’ve covered that elsewhere under “Nextion”.

And so to Blynk. For many months I’ve been trying to get a decent control/display combo for the mobile phone. I’ve written my own in jQuery mobile and used the likes of NETIO and THINGSTUD.IO – all of these are viable methods – another that is coming up is Blynk – again very preliminary and I’ve reported lots of issues with this – but it works.

There are several useful controls – you see in the image above some simple LEDs (which can be set to various intensities though you can’t programmatically set the colour, sadly), some simple status outputs and two buttons (which can be set to ON/OFF or push to operate). They also have a controller called the zeRGBa which is a visual controller for setting RGB lighting. You see 4 of them in use top right of this article. You can basically move your finger over the control – which will then fire out RFGB values. Thankfully you can set the maximums as not everyone works in the 0-255 space. My PWM controls for ESP8266 use 0-99 for example.  This widget falls down currently in 2 areas, firstly you cannot SET it’s position – you can only use it as an output control. Secondly and arguably more importantly it sends out rather too many commands one after another when you move the unit around. I prefer courser control, smoothing the transition in my ESP boards.

So, given a control that fires out RGB sets rather too rapidly, how can Node-Red help. Thanks to the developers who I’m fortunate enough to get lots of feedback from, it’s actually quite easy.

RGB Control

Here we see 3 inputs from the App coming in via the Blynk Node. The first thing I need is a function to convert that data (a simple array of 3 values) into an MQTT statement to fire out.

msg.topic="sidewall/toesp";
msg.payload="{rgb:4," + msg.payload[0]+","+ msg.payload[1]+","+ msg.payload[2]+",70,1}";
return msg;

Easy enough – I set the topic for MQTT to my wall RGB ESP8266 board (the format is my own, don’t get hung up on that) and I use a message payload of say “{rgb:4,255,0,0,70,1}” – what the hell does that mean? Again my own format… the braces are there for technical reasons – my ESP boards accept MQTT or serial – and the ESP’s put out some crap at the beginning and debug info – so I wrap my messages in “{}” just as delimiters. So the actual message is “rgb:4,255,0,0,70,1” – which means – using output 4 of the ESP, send out RGB 255,0,0 (i.e. RED), smooth the transition from one colour to another over 70ms – and I can’t remember what the last parameter is – irrelevant here.

If you look at the code, what you’re getting from the Blynk node is an array (msg.payload[0-2]) so a simple substitution.

That one string goes out RATHER TOO QUICKLY on message after another for my little boards and I was wanting to slow them down just a touch. the DELAY node can be used for this..

delay

Absolutely brilliant – so it will not allow more than 16 message a second – and if it gets more – it will drop some intermediates. Works perfectly – the actual rate of course depends on your project. I had utterly missed this – thinking that node was nothing more than a simple delay as suggested by the name – but dig in and it has so much more to offer.

So right now I can control several RGB devices around the house, limited only by the somewhat large size of the zeRGBa widget and the inability of BLYNK right now to handle several pages per project (though as of the other day you can seamlessly move between projects which helps).

In  case you’re wondering – the language used in the functions is JavaScript  - who’s ability to make life easy when it comes to handling different types of data such as arrays and JSON, is a wonder.  I look forward to the day we have a PROPER, fully working interpreter on the ESPs themselves.

Oh – and to run the Blynk server on your own kit – you need Java 8

apt-get install oracle-java8-jdk

And I made my blynk directory and those below it full read/write/execute.

Facebooktwittergoogle_pluspinterestlinkedin