Regular reader will know about the script that Aidan Ruff and I originally developed to put Node-Red and several other packages onto the Raspberry Pi for our own home control purposes.
WELL - there is a new version of the script, modified to include logging and work to handle Raspbian Stretch (tested on Raspberry Pi 2 and 3). The script also runs well with several other boards running Debian, Ubuntu and more. See right hand side of the above image for what it does. Read on…
I’ve just spent time working on my hallway wall display, the amount of which would be considered commercial suicide. Good job it isn’t commercial.
Of course I’ve not just been working on the pretty colours – I’ve revamped the control codes for the ESP8266-driven controller twice, discovered and fixed an OTA flaw in the code, re-hashed the Node-Red driving code, found and fixed countless other bits and pieces… and in the process taken pause for thought as to why I ever used the Dark Skies Node… you know – in other words – pretty much revamped everything just to improve a simple display.
Continue reading The Wall Display
Big Timer is probably the best timing node for Node-Red, providing a general purpose timer as well as handling summer/winter correctly as well as (importantly) lighting up time (for which it needs longitude and latitude). After all you probably don’t turn the outside lights on at 6pm!! You turn them on when it gets DARK. Now with new seconds timer mode AND updated for the latest Node-Red 0.17.0+ including new help formatting and tips on input and output.
Updated to v 1.7.5 –August 30, 2017 - Recommend updating. New features include a seconds timer and even/odd day of the month selection and fixes for injected time setting! See this video!
I am constantly asked about using Bigtimer (node-red-contrib-bigtimer) and so here’s an example of how I am using it in conjunction with both Amazon’s Alexa (voice control) – and the Node-Red Dashboard.
Continue reading Node-Red BigTimer Example
I just learned something new!
ONLY of interest if you are using Node-Red and will lead to a description of the new node-red-contrib-diode node.
If you are into Node-Red – you really want to see this one…
Continue reading Saturday Node-Red Sermon
This entry is all about InfluxDB, Grafana and a cute little Bluetooth garden sensor!
Some time ago a pal of mine and I did some swaps of spare kit and I ended up with a Mi Flora sensor. A rather pleasing looking device, the MI Flora has a 3v lithium button cell and can transmit by Bluetooth LE, the light level, temperature, battery level and moisture.
Battery is supposed to last for months. I’ve provided an AliExpress link above which puts the units at under £8 Inc. postage. No doubt that price will vary dramatically depending on where you chose to spend your money. Amazon charge more like £13.
So having pulled off the protective tab which meant the battery would work, I put the unit to one side as I had no idea how to read it.
A week or so later in Spain, with poor weather my pal Jay came on to ask me if I’d used it. I had not – and so we decided to give it a go. As it turned out I had some cheap Bluetooth 4 dongles I’d bought on a whim last year. I plugged one into my Raspberry Pi 2. The program “hcitool” lets you scan for Bluetooth devices and as it happens it was already installed on the Pi. From the command prompt:
sudo hcitool lescan
Instant success as the Mi Flora device with it’s MAC address was revealed. I made a note of the MAC address.
I would need Python 3 to experiment – but this was already installed on the Pi and some code to get the information from the unit. This code turned out to be ok with a little modification. I ended up creating a folder called /home/pi/myflora and dumping the lot in there.
Jay had modified the program to send MQTT. That of course needed the PAHO client. In order to do that I needed something called PIP.
sudo apt-get install python3-pip
sudo pip3 install paho-mqtt
Note NOT pip (which works for Python 2.7) but pip3.
I then added to the MI flora directory a test program which Jay sent me called bathroom.py – presumably because he’d put the sensor in his bathroom – I adjusted the output slightly as I was putting my unit in the garden
import sys
import paho.mqtt.client as mqttfrom miflora.miflora_poller import MiFloraPoller, \
MI_CONDUCTIVITY, MI_MOISTURE, MI_LIGHT, MI_TEMPERATURE, MI_BATTERYpoller = MiFloraPoller("C4:7C:8D:32:08:63")
print("Getting data from Mi Flora")
print("FW: {}".format(poller.firmware_version()))
print("Name: {}".format(poller.name()))
print("Temperature: {}".format(poller.parameter_value(MI_TEMPERATURE)))
print("Moisture: {}".format(poller.parameter_value(MI_MOISTURE)))
print("Light: {}".format(poller.parameter_value(MI_LIGHT)))
print("Conductivity: {}".format(poller.parameter_value(MI_CONDUCTIVITY)))
print("Battery: {}".format(poller.parameter_value(MI_BATTERY)))# Publishing the results to MQTT
mqttc = mqtt.Client("miflorabathroom")
mqttc.username_pw_set("admin","xxxx")
mqttc.connect("192.168.1.19", 1883)
mqttc.publish("miflora/garden", "{ \"battery\" : " + str(poller.parameter_value(MI_BATTERY)) + ", \"version\" : \"" + str(poller.firmware_version()) + "\", \"sunlight\" : " + str(poller.parameter_value(MI_LIGHT)) + ", \"temperature\" : " + str(poller.parameter_value(MI_TEMPERATURE)) + ", \"moisture\" : " + str(poller.parameter_value(MI_MOISTURE)) + ", \"fertility\" : " + str(poller.parameter_value(MI_CONDUCTIVITY)) + " }")
mqttc.loop(2)
# End of MQTT section
So the idea was that running “python3 garden.py” would send some MQTT to my Mosquitto MQTT broker – and indeed it did (change MAC address for the MI Flora and MQTT details in the above). Subscribing to “miflora/garden” did the trick – I could fire off the program and with in seconds I’d see the resulting JSON package coming out into my MQTT-Spy client.
Next job was to get Node-Red talking to the MI Flora – I used an INJECT node, set to retrigger every 15 minutes – firing off to an EXEC node which contained nothing more than “python3 /home/pi/miflora/bathroom.py”
That worked – so now I had a regular supply of data in the form of an easy to use JSON package.
I split the package up – and fed it into a Node-Red Dashboard graph – and that was my first disappointment. If you fire more than 4 items into those graphs – the popup fails to show all of them – and the graphs are not very good anyway. I think a lot more work needs to go into that.
I do use GROVESTREAMS on-line (which is REALLY flexible) to store data remotely but there’s a limit as to how far back you can read for free – that was not the case when I first started using them and wrote the Node-Red-Contrib-Grove node to make it all easy.
So now I was on a search for a decent graphics package that would run on the Raspberry Pi (or similar). NOT as easy as it sound, unless you’re very easily pleased when it comes to graphics.
After a lot of experimenting I decided to give Grafana a shot. I think if I started writing this blog again, I would would have Grafana in the title because this really, as it turns out, is the star of the show.
Having seen a few screenshots, I had visions of populating a SQLITE 3 database and firing that data into Grafana.
Erm, no. Despite (wait for it) using SQLITE internally to store users and dashboards, the package does not actually support SQLITE for data !!! I've explained that now, to save you some confusion later. When I did the install, it kept referring to SQLITE and could I HELL figure out how to set up a database for my data – that’s why.
As a Pi user (that means SD use so I’m not keen on MySQL as SD has limited WRITE capability and I worry about databases shortening the life of the SD - see the later article on SD life - very informative) the options for data sources did not look appealing to me at all, the least offensive being INFLUX and that is the one I settled on – partly as there is a node for it available for Node-Red. Despite the pretty website this is still very preliminary at version 1.02. Influx doesn’t use terms like tables and fields which kept me going for a couple of hours – but it is good at storing time-stamped data – indeed you don’t even do the time-stamping. You create an empty database and start firing data at the DB in pairs – stream name – and data value – it really is very simple once you get started. I guess you could think of it as a single table with pairs of data-name and data-value. I went down a few dead ends and discovered if you got your stream names wrong – good luck renaming them as INFLUX does not support renaming!
Anyway, it ‘s all very easy once you get started. I did things backwards and installed Grafana first. Here’s what I did – no guarantee it will work for you.
You might want to consider reading this before using InfluxDB on the Pi for long-term use - there are very few build options for small SBCs and this article gives the impression it likes more RAM than we would normally have on the likes of the Pi. I do have to say though that it is working just fine on mine after several days of operation. Comments welcome.
I grabbed the Grafana file for my Pi from here. https://github.com/fg2it/grafana-on-raspberry
sudo dpkg -i grafana_4.1.2-1487023783_armhf.deb
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable grafana-server
That was Grafana up and running on port 3000 – user name admin, password admin to actually DO anything – and quickly change that password!!! I did that, went into the ini file and disabled user registration as the admin can do that and opened up port 3000 redirect in my router so I could access this from the outside world. That all worked well.
wget http://ftp.us.debian.org/debian/pool/main/i/influxdb/influxdb_1.0.2+dfsg1-1_armhf.deb
sudo dpkg -i influxdb_1.0.2+dfsg1-1_armhf.deb
That set up influx and that ran without further ado at port 8083. I did NOT open that up to the outside world. In there I created a new empty database called “logger”
In Grafana – which supports influx without any changes, I followed Engineer John’s instructions to set up the data-source and a new Dashboard.
So the next thing – would be to get some data. I played with this for some time – should I keep incoming data source names simple – as you can’t rename them? Then Jay pointed out that in Grafana you can ALIAS names. That did it. So in influxDB node for Node-Red, you fire in a payload – and depending where you want to put your data – you use msg.measurement.. Yes, exactly, what’s wrong with msg.topic you might ask. So here is my little MI Flora MQTT node firing values into InfluxDB
![MiFlora[6]](https://tech.scargill.net/wp-content/uploads/2017/02/MiFlora6.jpg "MiFlora[6]")
And that yellow function block…
node.status({fill:"blue",shape:"dot",text: msg.payload});
tmp=JSON.parse(msg.payload);
msg.measurement="battery"; msg.payload=tmp.battery; node.send(msg);
msg.measurement="sunlight"; msg.payload=tmp.sunlight; node.send(msg);
msg.measurement="temperature"; msg.payload=tmp.temperature; node.send(msg);
msg.measurement="moisture"; msg.payload=tmp.moisture; node.send(msg);
msg.measurement="fertility"; msg.payload=tmp.fertility; node.send(msg);
Yes, I know – far better ways to do this – but it was my first stab at it and it worked. For general use however I added an incoming function that merely copies msg.topic into msg.measurement and that means any single data-source coming in via MQTT can be blasted straight into InFluxDB.
Yes, that’s write – you don’t create tables or fields – you just send in field names and data and that’s it!
I also fed in some data from my general sensors – with horrible topics like pergola/lighting but of course thanks to the Grafana alias ability – names like that would not show up in Dashboards.
Having stored up some data it was time to get it into graphs – I took the VERY simplest approach of showing lines as I have at this point no idea what some of the Grafana functions do – I DO know that smoothing the data might be fun as Grafana relies on the data-source to do that and Influx doesn’t do that. What IS nice is that you don’t have to worry about things like “I want the last 5 minute’s worth” or “show me the last month’s worth” as Grafana does all of that for you – from here it gets easy.
New dashboard and under metrics ADD ROW – takes an hour or so to get to grips with this..
I’ve expanded the top one here – I have 3 lines in that one graph – see the blank ALIAS BY - if you want to change the source name (in this case battery) put something in there.
And that is how I got to this point…
So top left you see battery status from my deep discharge battery and the two MI Flora units, then moisture levels from the two units – light levels – and on the right centre – a whole host of temperature readings, overall humidity and finally just to play with – the UK temperature split off and displayed differently.
There’s a lot to take in – and I am as you can see NOT an expert at this yet so before firing questions back – you might want to spend some time looking at the links on this blog entry and using Google. If you think another database is better by all means write in to say why – and if you think there’s a better easy-to-fit-in-Pi graphing system, preferable one that works with SQLITE – again do let us know. I started all of this with zero information late afternoon yesterday and had it all working by the end of last night – so it’s not THAT big a deal even if my clumsy writing makes it look that way.
Not entirely sure what’s going on the this battery level on my first day – but it did occur to me that the unit is intended for putting into plant-pots – not sitting outside in the sopping rain so I’ve covered it in cling-film!! As we’re off back to the UK for a few weeks, I don’t want to come back to a rusted heap. Note the Mi Flora battery at the top – seems to be self-recovering!!
Update: the little Bluetooth sensors have now been running for months and the battery levels remain fine – the Bluetooth range however is crap – best to have a Bluetooth dongle nearby to talk to them – don’t even think about two thicknesses of wall!
Update 27/03/2017: I've been hitting issues with out of bounds temperature data – i.e. a couple of duff readings crept in - with values like 2000 and -64 which of course flattened the data completely. I realised quickly that InfluxDB has no DELETE function and as far as I could tell - things like comparing data with a value made the graph line disappear.
I put something up on the web and someone came back and asked if I was using InfluxDB 1.2 which apparently has a DELETE function. Well, I wasn't - I was way back at 1.01 - worse - the same folder where that came from only went up to 1.1
However, thanks to THIS helpful chap - I easily updated to Influx 1.2 without the issue he described. I STILL can't add something like "where value < 100" but maybe I can delete ??? Anyone with more info on this please by all means chip in...
Update 18/04/2017: I've just fitted the second of these units and also fixed a minor error in the code above. All that is needed for a second unit is a copy of the program with different MAC address and different MQTT publish address. Sure you could use the same one twice with command line parameters but I took the easy option.
Bluetooth RANGE is becoming an issue. The Pi is in my office - breezeblock wall then the garden - the second unit I tried maybe 15ft or so from the office wall - not a chance. Clearly BT4 is good for power, not so good for range.
So - there's a nice job for an ESP32... BT in - WIFI out. A relay!
Update 30/04/2017: At the time of writing, InfluxDB is up to version 1.2.2 and if you follow the link I gave above but instead of 1.2.0 use 1.2.1 in the two relevant links - you will get the latest version of InfluxDB. Similarly I found Grafana 4.20 - same place as before but called https://bintray.com/fg2it/deb/download_file?file_path=main%2Fg%2Fgrafana_4.2.0_armhf.deb
I just repeated the Grafana install routine to update it using the new simpler name grafana_4.2.0_armhf.deb - all worked perfectly.
I'm pretty happy with Grafana and I now know how to delete data points (by time) but if you really want to use SQLITE and do your own thing - take a look at the excellent work that Csongor Varga has done - https://www.youtube.com/watch?v=nkKf26oKzhQ
If you like this post – please share a link to it by social media, by email with friends or on your website.
More readers means more feedback means more answers for all of us. Thank you!
A couple of weeks ago I went to stay with my pal Jonathan and he had some Byron doorbell pushes to play with.
This article updated 22/03/2017
He showed me a universal RF 433Mhz receiver board he purchased which could look at the signals from the various kids of 433Mhz standard transmitting units.
This was interesting as I have an Acurite weather station with the most appalling interface which requires a PC to be on constantly in order to remotely access the information.
I don’t know what planet the designers were on – but it was a gift from my wife, solar powered and including rain level, wind speed and direction etc.. a nice job other than the software. It has been sitting outside the wall on my office for months doing nothing. I thought it might be interesting to get that going.
The unit my friend was using was rather expensive but he suggested I try the RFLink software along with a DIY unit comprising an Arduino Mega2560, a little board called an RFLink V1.1.4 (now updated) and an antenna – you see the lot here.
The kit arrived days ago and I put it together with a soldering iron easily enough but had to wait for a Mega board to arrive which it did this week. I downloaded the software – very simply install program for the PC – couple of button presses really – and that was that. I plugged the little RF board into the MEGA, plugged the Mega into my PC and… out of the blue, information from my Acurite board appeared out of no-where, as did more one-liners from my Byron button presses and even our doorbell. It took no time at all with help from a fellow enthusiast who’s used this stuff before to figure out how to send a signal back to the doorbell to make it work.
All very nice but I needed this into Node-Red.
I took my latest Raspberry Pi using DietPi and after adjusting comms permissions, simply plugged the USB device into the Pi and set up a serial node for both transmit and receive.
From there on it was easy. Far from complete - but thought you might be interested – the combination of the two boards and that software seems to work really well up to now for receiving from 433Mhz sensors and for sending out commands to 433Mhz boards.
I had a slight concern about how long you have to leave the Byron SX35 pushbuttons before pressing again (3 seconds). But I mentioned this to the author and within an hour he came back with an update which made the delay much more practical.
Valid input instructions from the various sensors is in this link along with the Arduino software download….
http://www.nemcon.nl/blog2/protref
I bought a bog-standard Arduino 2560 (cheap Chinese version)
I bought this board… the RFLink 433 kit – requires a little soldering…
https://www.nodo-shop.nl/en/21-rflink-gateway
10 minute soldering (take note of version numbers – important), 10 minutes max to blow the software. Test the board (56k baud) to ensure when nearby sensors are sending results – they are coming in and then I wrote this test… it isn’t very elegant yet.
My SERIAL node is set to split input on character “\n” so the code has to get rid of return characters as you’ll see in the “replace” line below. Note also that when you send out serial (to actually control something) it should be followed by both “\r\n”.
Put that in a Node-Red template – attach a Node-Red serial Node set to take serial input from USB0 at 56k – character /r as a separator and deliver ascii strings…and that – is just the beginning… note also that the designer of this free software has also added GPIO control both input and output – on several pins (recently expanded so check his docs).
Tests: Right now for my tests - I have the K10000 phone acting as a server running Debian and running Tasker and the MQTT client Tasker plugin with the same phone running as a resource…. and I can now fire an MQTT message at the latter to get a doorbell message out! Meanwhile a Raspberry Pi is running that RFLink unit and when one of the Byron doorbell pushes is pressed – a message is sent out to the phone to play the doorbell – yes, I know, somewhat over the top – but I’m just experimenting for now… and sure enough – press the button and pretty much in real time the doorbell sound appears.
In the comments below you’ll see reference to an ESP8266 to MQTT Gateway – and this would be ideal as it would be all in one little box – whereas I need to stick something like a Pi on the end to generate a wireless MQTT signal…. so – I grabbed the software and (disregarding several wasted hours due to a duff FTDI) put together one of these – as you’ll see in the photo on the right- the antenna is due to kind feedback below – and as you can see, it is a precision job (it is accurate however).
Well, I have a 4-way remote control for a cheap Chinese remote and indeed this little system does pick it up and sends a unique number for each key off as MQTT – lovely – however – even with a decent little aerial the unit does not pick up (or recognise) my weather station of any of my BYRON pushbuttons – and the data coming back is crude compared to the RFLink software so at first glance, not impressed.
Costs: Ok of course the hardware for the little ESP board is DIRT CHEAP compared to what I’ve put together – which in turn is cheap compared to one of these all in one boxes – but you pay your money – I’m sticking with https://www.nodo-shop.nl/en/21-rflink-gateway – at under £20 plus £9 for the aerial plus a Mega2560 (cheap from China) I think it is worth it (no I don’t know the company and no I didn’t get samples etc).
The transceiver they supplied costs £16.51 on Ebay so the board with connectors and the transceiver really is a good deal. You can of course use cheaper receivers – but the software writer suggests these might be naff. I’d like to hear back from someone who had had GOOD results just using the Mega board (they’re only a fiver from China) and other boards. There is information here on that subject. RXB6 board seems cheap but don’t buy from the UK as they seem to be a rip here – on guy wants nearly £8 inc postage – China for same board – under £2 all in. I’m kind of stuck for testing one of these as I’m off to Spain in a few weeks and chances are it won’t get to the UK in time for me leaving!! Shame as I’d like to have given one a shot on my spare MEGA board.
Summary: Already the RFLINK setup has made my day by turning my otherwise useless weather station into another working part of my home control – and I’ll soon have buttons all over the place. It also has good range though I think aerial design and positioning could be improved. I can’t do a range comparison with the ESP project as it only recognises one my my 433Mhz devices and even then gives out a number, no proper ID and name… still – worth keeping an eye on for future developments.
If you like this post – please share a link to it by social media, by email with friends or on your website.
More readers means more feedback means more answers for all of us. Thank you!
Why am I showing you a picture of a cracked-screen phone?
Well because this particular phone is a bust Oukitel K10000, the phone with the world’s longest-lasting battery and an excellent workhorse. My friend Aidan gave me this (rather than chucking it away) some time ago and it has been sitting doing nothing. All that is wrong with it is a cracked (and exceedingly dangerous on the fingers) screen. I’ll bet I’m not the only one with such a phone lying around wasting space.
Well, as of yesterday, it is a Debian server with all my usual stuff going on quietly in the background – with the screen normally off – running freezing cold and hopefully being super reliable.
This is an experiment only – beware – if your phone or tablet dies it is your problem…. oh and your Android phone/tablet needs to be ROOTED.
Imagine turning your old, dust-covered phone into a sleek, battery backed-up server with unfeasibly long backup time, immunity to any mains spikes, a silent, fast Debian base for all the stuff in my script – which includes Node-Red, Apache/PHP, SQLITE (+ PHPLiteAdmin), MQTT, MC, Ha-Bridge and more! If you’ve been following this blog you’ll know about the script.
So this article applies to ROOTED Android phones and we’ve only tested around Android 5.0 onwards. In my case I rooted the phone with KingRoot (note, NOT the one in the PlayStore which is a con – avoid it - but the one at kingroot.net ) – please note that rooting phones is beyond the scope of this article and if you are not confortable with this you should not do it. There are a lot of links out there on the subject and many of them are fraudulent.
There is an APP in the Play Store called Linux Deploy. It is also on GitHub. Beware that this is essentially undocumented unless you are Russian – so please don’t ask how you can use your phone’s GPS or Sound from Debian – not a clue!
You should have a modern-ish WiFi enabled (or hardwired if you like) Android phone or tablet with at least 16GB of internal FLASH (which should give you 10GB working space). If you only have 8GB that will leave you only 2GB which - really – isn’t enough.
Getting Debian 8 on the phone: After ensuring your phone/tablet is rooted, Install the App.
In the app, on the top LEFT menu – find REPOSITORIES and select Debian 8 for ARM.
On the bottom right is a drop down – you should go in there and select INSTALLATION TYPE – directory (OR FILE with a reasonable file size limit – say 5GB – the 2GB default will NOT work). Change the user to “pi” with suitable password in the same menu. TICK for allowing initialisation – and use of SSH server. Also where you see INIT SYSTEM change that to sysv.
Then TOP RIGHT menu - INSTALL – that might take some time – top right again CONFIGURE – then bottom menu START – and you should have a working Linux you can get into with for example WINSCP for that “pi” user. The IP address is clearly shown in the App.
I suggest not going further until you are comfortable with the above – took me several attempts because I didn’t follow the above exactly (well, and Antonio and I were learning at the same time).
Running the script: Via WinSCP or similar, put the script into the pi directory – changing permissions as normal - run the script – and ensure PHONE is selected – it isn’t by default. Come back after lunch. The script will no doubt suggest a reboot. Instead, hit the STOP button on the bottom of the phone screen – wait for it complete, hit the START button – wait – and you should now have everything in the script running!
Now – I’m using this to run my script – but I’m thinking you COULD use it to serve files etc. – I’ve not tried this but I’m guessing it would be easy to access an SD card…. and make that a folder…. hmmm.
Anyway, it is now the day after I installed all this – the phone is sitting there “off” and unlike my FriendlyArm board with it’s whirling fan, is sitting silently and freezing cold yet ran the script much faster than any of my SBCs – around 40 minutes.
No guarantees as there just isn’t enough info about Linux Deploy out there (unless you find/translate some) – but it does seem to work well now that we’ve made sufficient alterations to the script to take this particular setup into account. A fun project, to be sure. Now, I know this is a not a fair comparison and tomorrow we might come back and say … no good (for whatever reason).. but at £107 for that particular phone… compare – how much would it costs for a Raspberry Pi 3, uninterruptable power supply able to keep the phone going for something like a couple of days with no power, a high-def touch screen, a solid case for the whole thing.. indeed ,it might be possible to use a cheap tablet… I was looking on Ebay – 16GB Android tablet – perfectly working except for missing buttons and cracked digitiser – around £10
One thing to note – I’ve turned rsyslog messages off – it was spitting out all sorts of unwanted helpful messages when you changed brilliance on the phone or disconnected the USB etc –REALLY annoying.. If you feel you need logging on -
sudo service rsyslog start
That will stay on until the next stop-start…
Such a shame it isn’t clear how to access some of the peripherals like audio. But who knows – someone reading this article may already have solved such challenges.
Please note: the pretty colours on the right do not come as standard. Click images to see larger versions.
This is really starting to look GOOD!!!!
Revelation: I’ve now taken on-board ideas from others and thanks for that – both people in here and on Google+ – most of the other solutions are longwinded and need other packaged so up to now Linux Deploy – I’m now installing on my old Nexus 7 2012 UTTERLY SUCCESSFULLY (though not the fastest kid on the block) ( after rooting it with the Nexus toolkit ) - using Linux Deploy (which is completely self-contained and offers full root access – is looking the best). The ONLY thing you can’t do is use the Android peripherals – because of lack of UK info but this morning I figured it all out.
We’ve also tested this one OnePlus One (model BACON) and a Xiaomi Redmi 3 model IDO). The K10000 has now been up for several days.
Ok, bear with me – you have Node-Red on Linux – and MQTT. So, you run Tasker on the phone (in the Android section) with MQTT – and now you have access to and control of all of the Android facilities that TASKER can handle (i.e. just about the lot) from within the Debian environment. Doddle.. now all I need is some time!!
There’s a mouthful. This short article is about the various kinds of variables used in Node-Red and more importantly how to initialise them and how to view them.
So in Node-Red when you want to use variables – there are three basic types as well as local variables.
So for example – you’ve opened up a FUNCTION block and you want to use some variables. VAR is always a good start.
var fred=1;
A way to get a variable to PERSIST behind that simply function invocation to the next time around, use this;
context.set(“fred”,1);
If you’d like that var to be available to any other function in that flow (or page if you like) - use:
flow.set(“fred”,1);
and if you need a global variable that can be accessed anywhere within Node-Red including other flows – I use them all the time for things like thermostat objects etc… use global.
global.set(“fred”,1);
Of course you can recall said variables as global.get(“fred”); etc. Don't use the old method (example global.context.fred) - there's a reason the Node-Red guys are steering us in this direction.
This is not meant as a course on programming – you can read all about this on the Node-Red site – just look up variables.
So three useful types of variable.
But there’s a problem. Node-Red runs asynchronously and in my first efforts I would have an INIT tab and initialise my global variables there. Once in a blue moon I would see a non-destructive failure with the logs showing that I’d tried to use a variable before it had been initialised (created, even). So in a platform where everything is running asynchronously how do you ensure (without checking every single time you use it) that a variable has been initialised…. especially important for globals.
One way which I don’t really like is to ensure that the variable is initialised in the left most tab. as that runs first… ok… kind of….
Well it turns out “there’s a node for that”. Node-Red-Contrib-Config does that – you can set up a variable – which could be a whole object – an no matter where you put this node – it will run BEFORE the main flows start up – I tested it, it works. My only niggle might be that the VALUE field needs to be expandable – a single line is not much use for large objects.
npm install node-red-contrib-config
So that’s one side of it.
The other side is – would it not be nice if you could have a tab to show you the current status, names and values of all of your global variables – and more. Well, now you can.
This needs a SLIGHT mod to Node-Red so if you’re new you might want to back stuff up.
In the directory you normally (hopefully locally) npm install stuff –
https://www.npmjs.com/package/node-red-contrib-contextbrowser
So the install is:
npm install node-red-contrib-contextbrowser
Stop Node-Red and in (in my case) and make a slight edit
nano /usr/lib/node_modules/node-red/red/runtime/nodes/context.js
This is the function you need to alter - very carefully add one line as I have in bold. That’s it – don’t make any other changes.….and this IS case-sensitive.
function createContext(id,seed) {
var data = seed || {};
var obj = seed || {};
obj.get = function get(key) {
return util.getMessageProperty(data,key);
};
obj.set = function set(key, value) {
util.setMessageProperty(data,key,value);
}
obj.getKeys = function() { return Object.getOwnPropertyNames(data); }
return obj;
}
If you're something that does not show bold, the line starting "obj.getKeys" is the addition.
(NODE RED GUYS – any chance of making the above standard???)
Start up Node-Red (this works even if you use HTTPS and have password on Node-Red)
And lo – you should have a new tab on the right… click in the editor on a node or function you know has variables initialised…
and then use the refresh in that new tab..
My test..
The contents of that orange function
And..
As you can see, you must select the particular node or function with the mouse first – then hit the little refresh in the contextbrowser tab - you then see above freddy and the value in the tab – along with any others you’ve defined in that node or function. Ok it’s not quite debugging but better than nothing.
Now moving to the Global tab is another matter as I have several globals including some big objects – if they are very big – you might have to hit the little … indicator to expand them.
So for me this is marvellous – I can now set up global vars with the confidence that they will be initialised properly at the start – and I can also get a nice overall view of the globals I’ve used – something I could not easily do before.
If you like this post – please share a link to it by social media, by email with friends or on your website.
More readers means more feedback means more answers for all of us. Thank you!
First things first – HighCharts are not free for commercial use. However, for your own website, a school site or a non-profit organisation they are free to use. You are also OK to make modifications. So – that’s the “cover my back” bit out of the way – now, how to use them in Node-Red.
I’ve looked at this before, because their charts are very, very nice. As some of you know, I’m now sold on Grafana but I also really like the idea of using charts with my favourite database – SQLITE and this is not supported as far as I can see with Grafana. Indeed I’m glad there is a Node-Red Node for InfluxDB as I had trouble getting my head around inputting data into InfluxDB otherwise (the node makes it REALLY easy).
And so it was that I came back to HighCharts. Now one of the demos of theirs that I really like, involves plotting variable numbers of time-based lines in a graph (temperature, humidity etc)– with potentially missing values. I found when using other chart systems that if you had the odd missing value due to power failure etc, you had to fix that – with the HighCharts demo it just draws a smooth curve between the two.
Another of their demos features zooming and it is only in the last couple of days when taking another look – that I realised you could combine the two – zoomable multi-line graphs.
What you see on the right is little more than a tweeked version of their basic demo – with a few bits taken out to make the whole thing work in the limited space we typically have in Node-Red Desktop – and zooming built in.
I tested everything in JSFiddle and then tried dumping it into Node-Red – nothing – zilch. I found that using the libraries remotely just would not work. Out of desperation I grabbed the libraries, one of which is big and put them into one of my usual “myjs” directories on the Raspberry Pi and lo and behold, it all worked.
Here you see above my test – now bear in mind this is very preliminary. In the brown FUNCTION nodes I have 2 different sets of data just to test things… and these feed an object in msg.payload through to the Dashboard UI. I’ve made two of these – using two different DIVs. One is called “mine1” and the other is called “mine2” – they fill DIV ids “container1” and “container2" respectively. Everything else is the same.
So – starting off with their demo here http://www.highcharts.com/demo/spline-irregular-time
I took that demo and wrapped the lot in a function (object) – so as to hide everything from the outside world in case I wanted two or more of these on a page. I then wrapped the code inside a local, externally accessible function. This could be way more efficient – but I wanted something up and running and as you will see I have two completely independent graphs here.
Here is the code inside the blue template. The only thing that changes is the reference to the DIV (container1/container2) and the references to the new object (mine1/mine2) – and of course near the end I’ve fed unique titles and subtitles when creating the object but the latter do not have to be unique.
and here is the code inside one of those orange functions – the only thing that changes is the data.
With more work, this could be really useful – and don’t forget to check the BIG range of graphs and other widgets in the HiCharts demos – there is some wonderful stuff in there.
Back to databases - as you can see it would not be hard at all to pull data like this out of just about any database! That's my next job after I tidy this up a bit.
When testing Node-Red, the common INJECT node is invaluable. Indeed it is just about the only way to instantly inject information into another node. Here is a typical example.
It doesn’t get any simpler than this. Here I have merely dropped an “Inject” node onto the Node-Red working surface along with a “Debug” node. I’ve made no changes to either of them. Note that “inject” and “debug” nodes come as standard with Node-Red.
Correctly set up, on the right of the Node-Red screen, you can see (or make available via the top-right three horizontal bars – view – “debug messages”) a debug window and simply by pressing that button-like item to the left of the inject node, you can put a timestamp into the debug window as such:
Simple enough. Note that this says (in red) “msg.payload”. In homage I guess to MQTT, the inject node outputs a simple object which deals primarily with msg.topic and msg.payload. You can use these for whatever messages you want. The debug node by default shows only msg.payload but you can of course change this to show the entire message object.
Note that here (image above) we also see a blank topic (because the unmodified inject node has that as empty – you could put “Merry Christmas” in there if you like. Also included is _msgid which is of no interest to us for now unless you have a specific need for a unit number.
In general then, you can fire out msg.topic and msg.payload from the inject node. Instead of putting out a timestamp, you can put out a string (for example “Hello world”) and this is output in msg.payload.
Expanding the INJECT output
For many purposes, we are done – you know know how to output a value into msg.payload and you can easily add msg.topic to that. An inject node then can directly output data suitable for, say an MQTT node which needs both msg.topic and msg.payload.
For some purposes however, this is not enough.
In the above image, I need to inject one of two messages into a template. Easy enough, “message 1” as string into one injector, “message 2” into another. But what if we have a string of messages for different purposes going into that template?
On way to differentiate them would be to use topic.
if (msg.topic==”needle1”) { /*use msg.payload to influence a gauge needle */ }
if (msg.topic==”lcd”) { /* use msg.payload to influence a display */ }
Another way to do this is by putting a function node after the inject node…
msg.needle=msg.payload;
The output of this then, is your message sitting inside msg.needle instead of msg.payload.
Do-able but two things about that. Firstly you still can only output one message at a time and now you have the extra work and additional screen real estate used up for the additional function node. Not very elegant. If that works for you – stop here.
So what if we want a simpler way that may include sending multiple messages out at once.
In the inject.node you can’t change the name of “msg.payload” – but you can change what it outputs – a number, a string etc.. and in this case of particular interest, JSON.
Let’s take an example. Let’s say that by pressing the test button we want to set a needle value to 10 AND we want to send the message “hello” to an LCD display in our gauge.
This LOOKS like you are sending a string of text but look what comes out – when you view the debug payload:
Not a string, but an object - but how do you make use of this? Simple. In your template – or function or whatever you are firing this message into:
if (msg.payload.needle!==undefined) { /*make use of msg.payload.needle */ }
if (msg.payload.lcd!==undefined) { /* make use of msg.payload.lcd */ }
This way, directly from a single inject node you can send either one, or several messages at once. The check for undefined means you don’t always have to send all of them.
Hence, for example, in testing my gauge, I can have a number of buttons testing different parts of the gauge and one of them sending messages to both lines of the internal LCD at once without making a big mess of the screen. In the example below, the bottom two injects each send 2 different commands to the gauge when pressed.
But what about existing objects
Another way to send multiple messages through the injector allows you to make use of existing global objects…
In this example, I happen to have an existing object called “thermostat” – which can be referred to as “context.global.thermostat” though the more modern way to do that is by use of global.get and global.put.
Armed with the above, which contains in this case lots of information about the state of my house… I can send that – either manually or of course in a timed fashion, to the debug node.
A little confusing at first until you notice the three dots at the bottom… which mean “I’m only showing you some of the result. Click on them and..
Far too much information to display here – but I think you get the point – though msg.payload looks restrictive – in fact you can send a whole load of information thanks to the ability of the INJECT node to carry an object in msg.payload.
In this case, if you set DEBUG to look for msg.payload.desired you will indeed see a result of 23.
Can I create an object? Sure:
In this case – the content of the injector is irrelevant. Just drag it onto the screen. Here we have (for the sake of it) two debug nodes – one is looking for msg.payload.actual and the other is looking for msg.payload.desired.
And here are the contents of that simple function node in the middle.
msg.payload={
desired:25,
actual:23
}
return msg;
Debugging: Keith Howell wrote in to remind me about node.warn and node.error – two very handy debugging functions. Here’s an example of use…
Again out of sheer laziness I’ve used the inject node on the left straight out of the box as it were. On the right the orange “test” function – again – drag from the left… double-click to open and enter the following:
node.warn("just info, really!");
node.error("ooh errr!");
and the result in the debug area when you press the timestamp button…
Isn’t that handy – the only difference here being the colour – yellow for warning (I’d have used blue but hey ho) and red for error. Using this instead of a debug node could save a little real estate on the screen in some cases.
Three thermometers in a row for Node-Red (or as many or few as you like really) for Node-Red Dashboard. Another fine example of simple gauges and unlike some it is easy to make multiple gauges on one page – I will demonstrate three.
Here is the library source, I grabbed the larger one and saved as gauges-min.js in my /myjs folder – see previous blogs for setting this up with Node-Red.
Here is the code for one – and a flow example with three independent units.
You can of course alter the colours to whatever you want.
The code for one template:
The sample flow above
