A Deep Learning Weekend

Gauge by Peter ScargillWell, this has been an interesting journey (a deep learning WEEK more like it) as I’ve been working on making my own widgets for Node-Red. I thought I’d share some of the things I’ve picked up. Experienced JavaScript object users please look away now. Those not interested in JavaScript look away now.

Note: The original video for this is out of date as I have learned more – the new video has some extra code, too – here it is - https://youtu.be/mTAPrvB1EeI

C Programmer: As a life-long C programmer, I came into JavaScript with some trepidation. Like alternative universes, JavaScript may look similar to C – but it is not the same – and that led to all manner of silly mistakes over time. However as I see a tiny light at the end of the tunnel, I have to say, it was worth the effort. Despite being a client-side interpreted language, it really is great fun to use.

The gauge: Well, as long as you keep it simple that is. When designing my first attempt at a gauge for Node-Red, I realised that a typical gauge operation goes like this:

Some features such as the background are typically set in an init function once only (and as such it would be better if the user is not involved in this. Other features such as needle positions are set dynamically and will change during operation.

This is generally handled one of two ways – either by setting a value and then calling an update function, or by running something in the background which checks for value changes so all you have to do is change a variable. On many of the gauges you’ll see flashing lights and smooth movement of gauge needles – that implies something running constantly or at least checking for changes and running when change is needed.

Concerns: This worries me as there can be quite some overhead in updating an entire gauge, umpteen times a second. So I decided that what was needed was a layering system as you might find in Photoshop – such that I could lay down the basic design and background of a gauge once only then dynamically clear and write to a top layer whenever changes are required. Such changes might include the state of a flashing light – or a needle not being where you want it, requiring successive incremental updates until the needle or needles do get to be where you want.

I then started thinking about variables and functions (methods). Did I want to update variables such as the needle pointer directly (direct variable manipulation) – or did I want to manipulate the input value before passing it through (that needs  a method/function).

That led me to create, as you will have seen earlier, a large blob of JavaScript for my gauge, including initial setup – then a timer-driven function to update the gauge regularly. That was stage one.

HTML5 Canvas: In the process I discovered (by looking at what others had done) that HTML5 Canvas (which really is easy once you get past the initial learning curve) Is a great choice. What did not come so easily was the realisation that the canvasses themselves are indeed transparent and you can layer one directly on top of another – just like Gimp or Photoshop layers. That led me to split the job into two – the overall backdrop which remains fixed throughout operation – and the top layer with my LEDs, needles and set points on it.

Adding a bunch of variables to this for setup and defaults brought me out into a sweat as I struggled to imagine how two or more gauges would interact. And THAT led me onto objects – something I’d never really touched in JavaScript before  – specifically to wrap up my entire code into a definition – which meant I could then simply create instances for EACH gauge without worrying about all those variables interacting. That was the point I last few days in deep learning.

Having wrapped everything up in a bulletproof package I was then faced with the question of how to access required functions and data externally. The obvious way (and the one I’ve adopted) is a series of callable functions for example:  mygauge1.needle1(50);

In the process I had to learn how to hide functions within the definition and make functions accessible externally. That was easy enough.

Here is a simple example – the assumption is that you have a jQuery link (to update a div) and a DIV called “fred”

1.  function myobj(){
2.  this.v1=5;
3.  init=function(passThis) { setInterval(every25ms, 25,passThis); }
4.  this.doubleit=function(r) { this.v1=r*2; }
5.  every25ms=function (passThis) { $("#fred").html(passThis.v1); }
6.  init(this);
7.  }

8.  brian=new myobj;
9.  //brian.doubleit(12);
10. //brian.v1=7;

There is a lot to this – and I’ll be using the word “this” a lot – so for clarify I’ll use italics for the special meaning of this.  It may look strange at first but if you follow me it soon won’t.

Definition: So the point of this was to create a test object definition – an object that would simply put a value in a DIV (I know, slightly overkill but I was trying to prove a point).  We start off with the definition in line 1.

Take a look at the code above – there is a lot in here despite their being only 9 lines. I figured any more and people might run away.

Firstly on line 1,  we define a type of object. A “myobj” type.  I’m not making anything here – just defining it (ends in line 7).

On line 2 – we define an externally available variable – v1. “this” makes it accessible externally (try removing “this”).

In line 3, we are defining  a function called init.  When the instance is created on line 8, init will be called internally (by line 6) and will start off a timer which will run every 25ms and which will call an internal function called “every25ms” (line 5), passing this to it. Do not get hung up on this yet.

Line 4 defines a function (again made available externally by “this” and called “doubleit”,  will double whatever is passed to it.

The  “every25ms”  function on line 5 which is to be called every 25ms by init… simply prints out the local variable.

Line 6 calls init (which is NOT accessible externally) when the instance is created in line 8 - to initialise the instance and start off the timer – this saves running an init externally.  From line 8 onwards,  the function “every25ms” is called every 25ms to put the value of this.v1 onto the DIV.

You’ll see “5” appear in your DIV if you test this somewhere – that’s the initial value. Note "passThis" – (arbitrary name). Because the function init is private - I don't prefix it with "this" – when it sets the interval timer it has pass to the timer function the outer “this”… - so I pass "this" in the init call.  This is new as a result of feedback in here suggesting a private “init” function that was no accessible externally. In the original version I used this.init() - and passing this worked. Removing that resulted in the timer callback function using the wrong "this".  Trace it through and it makes sense. The callback function needs to know about the outer "this".

Line 8 is where things start to happen. At that point, the instance (brian) is created and the timer starts all on it’s own -  thanks to the internal init function.

So far so good. How do I now interact with that variable v1? I wanted the choice of updating a variable or calling a function. The two commented lines are two ways to do this – their operation should be self-explanatory.

Line 9 simply won’t interact with local functions inside the definition and we don’t want them to – we want interaction with specific functions in each instance of the object – like “brian” for example. And that’s where “this” comes in. Add “this” to a variable name or a function name – and it becomes accessible to the outside world. For example you CANNOT do brian.init() or worse, brian.every25ms() and that’s a good thing!

Scope: So using this technique allowed me to keep all variables and functions private and only “expose” those I needed. LOVELY.  If I only wanted to use function and not expose any variables – I would not use for example “this” as a prefix to v1.

These ideas then form the basis of my current gauge. I can built a massive machine with lots of variables and functions inside and have no worries about interactions with other gauges because I’m only “exposing” stuff I want…. and gauge1.v1 has nothing to do with gauge2.v1

Of course I’m assuming that was as clear as mud -  but if you are interested, go have a play in this CodePen where I’ve left the code. See the number that comes up in the working area – try un-commenting those functions (methods) in lines 9 and 10. See what happens if you replace passThis.v1 with this.v1 or just v1.  See what happens if you remove references to this altogether. CodePen is great as changes run immediately and safely.

Merely experimenting with the code in that CodePen should do what it did for me – clear a massive amount of mental fog.

Simply creating v1 – and removing this and passThis in referring to v1 – leads to disaster – v1 is accessible externally on it’s own –so kiss goodbye to multiple instances.  But the same code using “var v1” leads to a save internal v1 NOT accessible as “brian.v1”. Your choice.

My new gauge object is coming along nicely – and a minimal installation involves little more than a link and a couple of lines of code – stunning how these simple ideas change the whole game.


18 thoughts on “A Deep Learning Weekend

  1. Hi Peter, really useful to be taken through the learning process rather than the finished product alone. One possible addition is that in most practical situations you might normally initialise the product during the object creation process.
    In your javascript that would involve adding "this.init();" between lines 4 and 5. You would then no longer require line 7 for every object you instantiate and couldn't accidentally create objects that were improperly formed.

    1. Hi Steve - I'm in an NXP Bluetooth talk all day today, hence brief reply. you are absolutely right... I was thinking of some kind of constructor but that will do just fine....

      I have now implemented this (I have no idea why I didn't think of that in the first place) and tested it in CodePen for anyone wondering why the demo has changed - I've also updated the blog but scrapped the video link as that is now out of date. I'll re-do the video later. Also, to now make init() private I had to ensure I was passing THIS through to the callback function - (TTHIS). All works a treat.

      Keep on firing good advice to perfect this as a base. I'm already half way through an LCD display... an automatic but delayed callback init is needed to give web fonts time to load... all that in a future blog.

  2. I'm not sure if anyone else has noticed, if it will specifically apply to this node/widget implementation or if this is just strange local browser behaviour that I'm seeing here - when I've included a JS resource within a template, I'm seeing that a browser request for this resource always includes a timestamp query string parameter tacked onto the end of the url

    i.e. including a script resource in a template like this:

    results in a request for the resource that looks like this:

    In my case [MY_DOMAIN] is a real publicly accessible domain name running over TLS on a specific port number and the browser in use is chromium 53.0.2785.143 on linux mint.

    The presence of this timestamp is circumventing browser caching completely here

      1. I agree this does look like some sort of default jQuery behaviour and node-RED dashboard does embed jQuery within it's app.min.js so you could be onto something there. This isn't ajax as described in that SO article though and I've tested that doing something like $.ajaxSetup({'cache':true}); has no effect on this phenomenon. This is just a regular script include within a template node (my attempt to demo/show it got stripped out of my last post so I won't try again :))

        I'll ask the node-RED guys on slack to see if they know what's going on with this

          1. The results of the investigation are here:

            This flow demonstrates the original problem where a script tag is included within a template resulting in a timestamp query string cache busting parameter being appended:


            And this flow demonstrates a JS script tag injection solution that at the end of day will result in a 304 not modified for requests made to the included JS resource:


              1. I guess it's not going to be a problem for everyone but it could be a problem on low bandwidth networks.

                When I compare the requests/responses to skycons.js in the example, I see the full cache busted 200 response is a 19.2kB response that takes ~230ms and the 304 not modified response is a 243byte response that takes ~200ms.

                So overall it's not a huge difference in time interval but it's worth pointing out that the user experience is completely different (imo better) in the new version - most of the page is able to load early with the script tag injection method because it's a non-blocking mechanism whereas the jQuery ajax request for the script resource blocks until done.

                I suppose another alternative would be to completely inline all JS directly within a template node removing all requests to JS resources on the server and as a result avoiding this issue completely 🙂

  3. Hello Pete,
    With all the advancements you've made with Node-red dashboards, would you still implement a Nextion based interface?

    I ask as I'm finally in a place to implement the user interface part of my home automation system and now not sure if to invest the time into developing the Nextion screens or Node-red and JS.... Seems to me that once you have got your head around JS it would be far more flexible and faster to develop than a Nextion screen. I'm guessing the downsides are the 'non-dedicated' nature of a Android tablet hanging on a wall.

    Thanks again for sharing your learnings and insight.



    1. That's a very good question Alex. Two different things entirely. To do what I'm doing here - and what I think we'll be doing more of in future, requires a Raspberry Pi or similar and a touch screen (or a tablet) - or at least a screen able to act like a proper hdmi screen no matter what the resolution.

      On the other hand, a Nextion display can run on a tiny ESP8266 such as the projects you'll see on the right menu in here - accessible over WIFI. The costs are wildly different.

      What I WOULD like in the Nextions - is capacitive touch - and I think the new ones are capacitive. The older ones are resistive and very nice with it - but you can forget a touch-sensitive rotary dial for example in the resistive version.

      My living room for example has a Nextion running on 4 thin wires from the Pi serial last year when we were away in Spain for several months, holiday renting the cottage, it performed flawlessly for various visitors. Wall mounting is easy and the cost if replacement maybe £20. That would not be the case with a wall mounted Raspberry Pi and screen.

      There is another issue I've not entirely resolved with Pi and touch screens. I run the screen, powered by Node-Red Dashboard - in a browser. With some Javascript I've been able to hide the Dashboard menu - and arrange for the browser to come up automatically on powerup. I have NOT yet figured out how to protect it from reboots and signal loss. I still get STUPID browser messages on occasion along the lines of "oops" or "would you like to debug"... The Nextions on the other hand are rock-solid. If there are any wizards at keeping a browser page running no matter what - do feel free to let us know how you do this.

      I think there is room for both.

      1. You appear to be using wordpress so if you want to control how the feed works you'll likely need to understand how RSS works in wordpress so you can get control of it. I think there are some plugins (yay add more WP plugins) that can be used to manage feeds and how they work. Or you could go in there with a hatchet hacking about with the php in your theme / setup to fix the problem 🙂

  4. I really like what you're doing here. I've been messing around the past couple weeks experimenting through different ways to display OBD2 data from a car. I ended up making the most progress on the raspi 3 through Node-Red. I have a Bluetooth Elm327 OBD adapter plugged in and Python pulling whatever I ask for. I have high hopes of streaming engine data from my 24hoursofLemons racecar. If I can figure out how to make this all work together the next race will be even more fun :D. I'm not much of a programmer though.

    I hope to learn from you. Please keep posting:)

Leave a Reply

Your email address will not be published. Required fields are marked *