Saturday Node-Red Sermon

Peter ScargillNode Red18 Comments on Saturday Node-Red Sermon

Node-Red NodesI just learned something new!

ONLY of interest if you are using Node-Red and will lead to a description of the new node-red-contrib-diode node.

If you are into Node-Red – you really want to see this one…

There I was messing about with a function in Node-Red… and I needed to send some data out – and then send it out again – don’t ask why…

msg.payload=”fred”

node.send(msg);

return (msg);

This simple function sends out the message out of the output – then when it finishes – does the same thing again. I know, useless but I’m making a point here.

In the receiving function… I altered msg.payload… and why not – you’ve passed a message and can do what you like with it can’t you? Well, no.

As was pointed out to me by Dave of Node-Red fame when I asked,  Javascript passes messages by reference, not by value.  And so as the message is sent out and processed by a second function – if msg.payload is altered, the ORIGINAL function is affected… and so the second time the message goes out (the return) – the ALTERED version goes out.

As was pointed out to me the solution is simple – to do any messing about – make a COPY of the message. But as they are passed by reference, if you do that you STILL end up in trouble.

The answer is something like this..

var newMsg = RED.util.cloneMessage(oldMessage);

Well, that’s all very sensible – but WHERE THE HELL did that RED.util.cloneMessage function come from??  I’d never heard of it?  Thankfully my friend Google was at hand.  Check this lot out.

http://nodered.org/docs/api/runtime/api

Update:

So here is an example fail…

fail

What you see above is an injector (content irrelevant) – and two functions – the output of both are monitored.

The first node simply sends the same message to 2 outputs.

msg.payload=”hello”;
node.send([msg,null]);
node.send([null,msg]);

Simple enough – first to the first output and then to the second output.  You might expect to see “hello” on the green debug node and most of the time you will.

However, because the message is passed by reference and not value, see that “edit msg” function? It merely contains this.

msg.payload=”goodbye”;
return msg;

So when pressing that timestamp node you might expect to see “goodbye” followed by “hello” on the debug – but you WON’T – you’ll see “goodbye followed by “goodbye” because the actual sending message ITSELF has been altered. I can tell you – that caught me out the first time until it sunk in that all that is being passed is the address of the message – not the message itself.

Ok, as referred to in the comments, the above is something of an “edge” case – and now you understand that it is a REFERENCE that is passed you might avoid this –  but one I think I’ve created this situation more than once and only now do I fully understand it thanks to readers.

So here is a simple solution if you can’t get your head around “reference” or you need the message to be isolated for some other reason…

diode

The  new “diode” node can be simply dropped in – and will effectively isolate the output  by creating a copy of the message.  If you find yourself in this situation, you will find this new node in the usual place. http://flows.nodered.org/ – just search for “diode”. As I write this it has not yet filtered down into the flows library but no worries,  it is also here… https://www.npmjs.com/package/node-red-contrib-diode

18 thoughts on “Saturday Node-Red Sermon”

  1. Pete,

    Diode works just as described. I was pulling my hair out trying to debug some inconsistencies.

    Question: Is there any memory concern with creating all of these new objects?

    Joe

  3. What is the voltage drop across the diode node? I input 4.12v and add 2 diodes. The result is still the same?

    1. Brian – the diode node isolates input from output so that changes to the output cannot affect the input. It is not a real diode! Read the help.

  4. Most of the times when the input is really different from the output it mentally does not make a lot of sense to clone the message and then edit it. Quite often I simply create a new object and populate it. This is exactly what a clone does, I find it a bit more intuitive. Example:

    var MqttTable = {
    “Lamp/Sonoff01”: “cmnd/sonoff/power”
    };

    var sCode = MqttTable [msg.topic];
    if (sCode === null) return null;

    var newMsg = {};
    newMsg [“topic”] = sCode;
    newMsg [“payload”] = msg.payload == “On” ? “ON” : “OFF”;
    return newMsg;

    1. Yes, that is what I do. In the majority of cases you either want the whole thing and your modifications don’t matter because upstream is creating the data or you need something significantly different and so just create a new object.

      But as Pete has pointed out, it’s those edge cases that catch you out.

  5. Call By Reference is what programming languages usually do, for structures. Call By Value is usually only for single primitive values.
    C, Java, C++, JavaScript, C# etc does this, as some popular languages. Old pascal could do either one, you just added var in front of the parameter, and you got CallByValue. And you couldn’t then call the function with a constant to that parameter…

    So, this is why most modern system treat some structures as non modifiable, and even can declare structures (and classes) as such. In Java String are not modifiable. Each time you do modify a String, you get a new one. Also have a look at Erlang, and you see you are only allowed to assign a variable once, a great feature, as you have seen why. 😉

    Yes, I would have done the same bug as you did, too.

    1. Hi Anders – yes I’m aware of the various languages and how they generally handle variables – but somehow for reasons that are no longer obvious I’d expected Node-Red to make copies… however now that I’m AWARE of this – it isn’t a problem – and I hope that others reading this will also save themselves some pain by taking this into account.

      I’ll be it accounts for some previous almost-brilliant ideas of mine that I gave up on, unable to spot the deliberate mistake.

      Now we know.

  6. I think that there are some events that DO clone a msg too so that can be confusing.

    Thankfully, this very rarely bites. In general, NR behaves as you would expect. But the more advanced you get, the more you need to understand how JavaScript and Node.JS work.

    JavaScript’s focus on passing by reference is efficient for most use but is a nightmare when you get it wrong. In NR’s case, passing by reference means that the msg is not really “moved” through the flow at all, only the reference has to be passed which is phenomenally efficient. Which is why we can get so much done, even on a Pi Zero 🙂

    1. Julian – yes, on thinking about it, it makes sense, passing an address rather than a potentially large object… but it still feels strange that something inside a black box can affect the black box that called it….

      I often use other people’s “black box” nodes – thinking I can fire anything at them and it will just do it’s own thing internally – never in a moment did it occur to me that the black box might just affect the values I’m sending to it – because QUITE A LOT I use “node.send” to send off more than one message. It has certainly made me think… given the knowledge that the receiving box just might be messing with MY data.

      1. “never in a moment did it occur to me that the black box might just affect the values I’m sending to it”
        put a DIODE-NODE in the middle 😀
        sorry, i had to comment 🙂

        1. Good idea – but as far as I can tell there is no such thing as a diode node…

          However……

          A function called ISOLATE might go something like this…

          var newMsg = RED.util.cloneMessage(msg);
          return newMsg;

          Now, if someone were then to make a node-red-contrib-diode……

            1. Ok, I made the node – just giving it a hammering before putting it up as node-red-contrib-diode to see what folk think. Simple but probably worth having.

  7. Does that mean that the execution context is shared across the entire flow? Just starting out with Node-Red and I had assumed context was isolated on a per node basis. 😵

    1. I think I’m safe in saying the msg is created in the flow that starts it off. So 2 injects feeding 2 functions – you’re not going to get crossover – but if you have a chain of functions passing the flow from one to the other – it is the same block of data by reference that you are passing. Personally I’d much rather have passed the data than the reference but there you go – probably saves resources.

Comments are closed.