Tag Archives: Node Red saving variables

Node-Red Objects to File and Back

This is a major re-write of a year-old blog. At the time, I simply wanted to store a global object in a file and retrieve it. Things have changed, I realise the way we did this was awful – hence the re-write.

One way is to save the object as a persistent message in MQTT - another to a database, another to a file. I like the idea of a file as somehow it just seems that this will lead to the least WRITES – and that might be important for SD.

Why would you want to do this? Well, specifically my own requirement is to save the state of a home control object which has all sorts of info about a heading system. Let's call the object "thermostat" and it may have all sorts of innards -"thermostat.temperature", "thermostat.set", "thermostat.fallback" etc. In my case about 20 items. Not something you want to split up into individual variables.  I used this as I have 2 parts of the house with separate heating and so I could duplicate all the code simply using two separate objects. I need to store them on disk when changes are made - and retrieve them on power up.

And this is where the original article went wrong...

The easy and obvious way is to use an inject node which runs once only on startup - to feed a file function - which gets the contents of the file - and stores them into an object. Now - you need to do this in the LEFT-MOST tab in Node-Red as this is executed first - before any other tabs - so you get to define the object BEFORE it gets used.

That's not the end of it of course - what happens if this is a new installation and the file does not exist? Well, in that case, you check for the "undefined" object coming out of the file node - create the object with defaults - and store the result in the (new) file. Easy. And that's what this article was originally about.

Except that on close inspection - it didn't work. Originally, Node-Red didn't guarantee that the left-most tab would work first - now it does run first. But that still didn't get it - when I started looking - there were (non-fatal) error messages showing that clearly the object was being used before it had been defined!

Now, why would that be? The answer lies in the fact that (just about) everything in Node-Red is asynchronous - i.e. it does not hang around to wait. So here I was using a file input node assuming the file would be opened - and my function would be executed before anything else. Nooooo.  The file node accepts the command to open a file - but then while it is waiting for this - other things can happen - that is, other flows can run which is EXACTLY what I did NOT want to happen. My thanks to the guys at the Node-Red Google forums for that insight.

The solution is simple once you now what the problem is. Firstly we need a function (which I made into a sub-flow) which takes in a file path, opens up the file and returns either the contents of the file in the payload or returns "undefined" if the file is not there - just as the normal file-in node would do... but synchronously.

There is also the matter of SAVING an updated object – I’ve used a second timer for that – one which checks every second – but NOT at the start. It simply outputs the text “check. So – at the start, the file is read, if it is not there, “undefined” is returned.

If “undefined” then create the object and store it. If defined, read it. If “check” – check if object.counter is true and if so decrement it – if now zero, update the object to disk. By doing this all in the same place, a lof of code duplication is avoided. Indeed as I have TWO thermostats I can use the same code with a different object and re-use one of the timers.

update objects to and from disk

Read file sync is a sub-flow – could just as easily be a function – but I put it in a sub-flow as the content is the same every time – it is the initial inject that is passing the file name.  The two (orange functions are the same other than the actual object name (thermostat and office_thermostat).

Here’s the content of the function at the heart of Read File Sync

var fs=global.get("fs");
try {
msg.payload = fs.readFileSync(msg.payload).toString();
}
catch (ex)
{
msg.payload=undefined;    
}
return msg;

And here is my code in the orange top function – the brown item on the right is just a file write node. Clearly this is for my particular application but you should get the general idea..

if (msg.payload===undefined) // if the file was not valid - create the object, predefine, store
    {
       var stat = {
        max_humidity : 100,
        display_temperature : 22,
        desired : 22,
        display_humidity : 45,
        heating : 0,
        display_external_temperature : 14,
        display_external_humidity : 54,
        display_status : "Normal",
        display_enabled : 0,
        display_last_page : "home",
        display_last_colour : 0,
        pass : "",
        display_dim : 60,
        
        frost : 14,
        hold : 0,
    
        p1 : 180,
        p2 : 480,
        p3 : 600,
        p4 : 1080,
        p5 : 1380,

        pw1 : 180,
        pw2 : 480,
        pw3 : 600,
        pw4 : 1080,
        pw5 : 1380,
        
        t1 : 16,
        t2 : 21,
        t3 : 19,
        t4 : 21,
        t5 : 18,
    
        tw1 : 16,
        tw2 : 21,
        tw3 : 19,
        tw4 : 21,
        tw5 : 18, 
    
        temperature_offset : 0,
        last_state : -1,
        current_state : 0,
        update : 0,
        daynow: 1
    };
    global.set("thermostat",stat);
    msg.payload=JSON.stringify(stat);
    return msg; 
    }
else if (msg.payload=="check") // this is the every second check - assuming file exists
    {
    if (global.get("thermostat.update")!==0)
        {
        global.set("thermostat.update",global.get("thermostat.update")-1);
        if (global.get("thermostat.update")===0)
            {
            msg.payload=JSON.stringify(global.get("thermostat"));
            return msg;
            }
        }
    }
else // read in the file if it was valid - don't output anything
    {
    global.set("thermostat",JSON.parse(msg.payload));
    global.set("thermostat.laststate",-1); // detecting change of timezone
    global.set("thermostat.temperature_offset",0);
    }


Not sure how useful you feel this is for your projects compared to a database – but it is simple and fast. The point of the global object, of course, is that you can use it on any page in your node-red setup. Certainly, by careful use of the timer when updating, you can vastly decrease the number of WRITES compared to a normal database setup. Handy for those of us on a Raspberry Pi with limited SD writes.

There are those who put their hands up in horror at the idea of globals - I do not. I like the idea of having the ability to view or update things from any tab (flow) with ease.  Real world apps? I've been using this (bearing in mind the initial issue with start up values just resolved) in a holiday rental since April - with people in all the time - running the heating and some lighting - not one issue.

Facebooktwittergoogle_pluspinterestlinkedin