Variable Persistence in Node-Red

It is my understanding that Node-Red will eventually have persistent variables - but while we're waiting - this is how I do it. Using the global.get and global.set mechanisms, I store a single object.

So - when Node-Red pops up it looks to the SD for that object - if not there it creates and initialises it. If there, it reads it into the struct. There is a countdown timer in the object and the function controlling all of this is called every 5 seconds. If the counter is zero, nothing happens. If the counter is true, it is decremented. If it gets to zero, the object is saved to disk over-writing the original.  It is that simple and it has worked for me for a long time now.

So imagine a Node-Red user function - with a 5 second timer feeding it (the INJECT NODE) - the payload of the latter is irrelevant.

Here's the function - the object and it's internal bits are mine - you'd alter them for whatever you need.

/ The purpose of this code is to update an object on change. At power up check for file and create if necessary
// var is "intel" in this case
var fs = global.get("fs");
var fl="/home/pi/intel.data";

if (global.get("intel")!==undefined) // Remember - don't use !== with null
    {
        if (global.get("intel.Counter")!==0) // if the var exists - now check counter
        {
         global.set("intel.Counter",global.get("intel.Counter")-1); // if counter is non-zero - decrement it
         if (global.get("intel.Counter")===0)  // if counter drops to zero update file
            {
                   fs.writeFile(fl, JSON.stringify(global.get("intel")), function(err) {
                    if(err) {
                        return console.log(err);
                        }
                    }); 
            }
        }
    }
    else 
    {   // If no var (powerup scenario) ..does the file exist?
        try {
                fs.accessSync(fl, fs.F_OK);
                // If file exists create the VAR by reading the file into it
                fs.readFile(fl, 'utf8', function (err,data) {
                  if (err) {
                    return console.log(err);
                        }
                  global.set("intel",JSON.parse(data));
                });
            } catch (e) 
            {
                // Otherwise create both var AND file -  ensuring counter is zero. New bits can be added dynamically
                intel = {
                        ShelfWhiteLight:0,
                        BoilerState:0,
                        ShelfRgbState:0,
                        ShelfRgbLevel:0,
                        ShelfRgbColour:"AAAAAAAA",
                        OfficeRgbState:0,
                        gOfficeRgbLevel:0,
                        OfficeRgbColour:"AAAAAAAA",
                        PergolaRgbState:0,
                        PergolaRgbLevel:0,
                        PergolaRgbColour:"AAAAAAAA",
                        Counter : 0
                    };
                    fs.writeFile(fl, JSON.stringify(intel), function(err) {
                    if(err) {
                        return console.log(err);
                        }
                    }); 
            }
    }

 

 

Facebooktwittergoogle_pluspinterestlinkedin

11 thoughts on “Variable Persistence in Node-Red

  1. with a lot of writing to the SD card how do you protect the card from wearing out to fast?

    I've had problems with SD cards going down after a period of time 3 to 6 months which is i think caused by writing to them to often. its to easy to treat them like a hard disk.

    any thoughts

    1. There are lots of ways to minimise writing - don't use MYSQL for example - I use SQLITE3. Turn off logging etc.

      One method I'm playing with now is to us DIETPI - which is set by default to write

      Probably the most important thing is to use quality branded SDs which (so it is said) can do wear levelling.

      It does appear to be something of a black art picking the right SDs, but up to now I have not had an issue with SDs.

      On The subject of this blog entry - the idea of using a timer is to ensure one write no matter how many variables are changed in quick succession. I could of course store variables in a database - but that would involve more writing, indexing (writing) etc. So it seems to be saving the lot as a block by a simple file write is likely to be the least destructive way.

      We really need the industry to get their finger out and come up with SDs that don't wear out.

    1. Thanks for that Scropion86 (should that be Scorpion86?) yes, aware of using SD - indeed for some time I used an external hard drive.

      But here's my question - if one uses a standard USB stick - why should that last longer than an SD - same technology? Now one could use an external solid state hard drive but then they are a LOT more expensive.

      Open to all suggestions and knowledge on this subject as I'm sure others will be.

      Hard disks go wrong too - so my favourite idea is to minimise writes and that's what I've tried to do. I like what they've done with DietPi in reducing logs to 30 minutes clearout from RAM - and not writing to SD for logs - that also seems to speed things up a bit.

    2. Oh yes, I'm learning about diet-Pi right now (and reporting bugs and issues) and one of it's features is - part of the menu setup lets you set everything to external drive or USB if you want - hence this can be done with no knowledge. I think the Pi3 ALSO has the ability to skip the SD altogether but I'm not sure I entirely understand how that would work.

    1. Hi Mr Shark.... I could have used a SQLITE DB yes, but again that is likely to involve background index writing etc.....

      If you look at my code, I simply read and write the global struct - it doesn't get any easier other than the need to update a counter. Now, If I was clever, I'd make a function totally compatible with global.set - that just happened to set that counter - but as you can use global.set in a number of ways - I'm not sure I'm up to doing that yet.

      And in any case - I missed the main point here. The node-red guys fully expect to make global variables non-volatile in the future - this is just a stop-gap measure.

      🙂

  2. Wearing out SD cards is exaggerated. All SD card controllers have wear leveling (or they wouldn't last long at all) [OK, maybe some horrible no-name or counterfeit SD cards have crap controllers]. I've had a Pi running my sprinkler controller for 5 years with no problems and I've done nothing to reduce writes (turn off logging, etc.). With heavy write applications, SD wear could be an issue, but for most uses it's not. I think Pete is doing enough to keep writes down, and I like what DietPi has done, but I don't think you need to go to extremes.

    Get a good quality SD card, get at least 8GB (more unused space = more room for wear leveling), take reasonable steps to reduce writes, make occasional backups of your SD card and you should be fine.

    1. FIVE YEARS!!!!!! Good grief!! Well, that's encouraging - I'm going along the route you suggest... with an 8GB card (the smaller ones are either unavailable in modern form or not much cheaper) and the DietPi setup I just made uses less than 2GB of FLASH - so assuming you're right about wear levelling - that should last a LONG time. I've had a golden rule since day one - NEVER use cheap SD cards. Not had a failure yet.

      Erm as the Pi A came out in 2012 did you mean 4 years???

      Pete.

  3. Hi Pete,
    I tried the code you provided, but I get the error:
    Cannot read property 'writeFile' of undefined.
    Can you help me?
    Best regards,
    Peter

  4. Hi again,
    I fix was able to it . It's necessary to add:
    functionGlobalContext: {
    fs : require('fs')};
    }
    into settings.js file, located in ./node-red directory.
    Best regards,
    Peter

Leave a Reply

Your email address will not be published. Required fields are marked *