Graphing System Info in Node Red and Grafana

We were just talking about getting system info into graphs in the comments in a previous blog entry – and I realised I already had some code for getting that info in Python – and using a library that can provide a LOT more.

So – as it was there already  - I thought I’d put this quick entry together for anyone interested.

There are a couple of nodes available for Node-Red but one of the things that I noticed was absent – someone is SURE to prove me wrong – was processor temperature. One use for that might be "let's graph this intensive scrip with and without using a fan"

But I already have that, I use it in a little Python script in a macro called “cls” – which when typed at the command line, gives me a clear screen with some useful info.

So – 15 minutes later, a VERY shoddily modified script – the thing that took the most time was using the PRINT statement while bearing in mind changes to PRINT from Python 2 to 3 as I’ve no idea which version you are using. Hence the long ugly line – but hey, it works – only tested in Python 2.7

So – here is the Node-Red…

tmp54FD

The blue INJECT could inject anything in the payload – how about “hello” at timed intervals – say every minute or every 5 minutes… ok the EXEC node has to call the Python script and that takes a tiny fraction of a second and what a waste.. but if you’re only doing that once a minute – who cares.

The brown EXEC node has this…

tmp529F

 

The yellow (ish) FUNCTION node has this…

msg.payload=JSON.parse(msg.payload);
return msg;

and for now the output is an object going to a debug node (green) – but of course it could just as easily be going to… say, Grafana.

And that’s it – here’s the Python script… I’ve left some stuff in there from my INFO.PY but typically anyone using this is going to want to add their own stuff so feel free to add in or strip out anything you like.  You can of course test the script on the command line and it will put out something like this..

pi@pihd1:/home/pi> sudo python info2.py
{"cpu1":0.1,"cpu2":0.1,"cpu3":0.1,"temperature":42,"ram":274108416,"disk": 4377710592,"cpu_speed":1200}

If you’re not familiar with objects… to get the temperature would be something like….

fred=msg.payload.temperature

If you wanted to sent the temperature to the influxdb node for graphing, you might use something like this  - sending the output straight into the influxdb node.

var fred=JSON.parse(msg.payload);
msg.payload=fred.temperature;  msg.measurement="temperature";
node.send(msg);
msg.payload=fred.ram; msg.measurement="ram";
node.send(msg);

Hopefully you get the general idea. Here’s my script – works for me – change/add/delete for what you might want to see and don’t forget, python is indent-sensitive (reminder - there's a lot of stuff in there you can strip out).

import time
import os
import psutil
import platform
import socket
from datetime import datetime
 
def get_ip_address(ifname):
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    return socket.inet_ntoa(fcntl.ioctl(
        s.fileno(),
        0x8915,  # SIOCGIFADDR
        struct.pack('256s', ifname[:15])
    )[20:24])
  
byteunits = ('B', 'KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB')
def filesizeformat(value):
    exponent = int(log(value, 1024))
    return "%.1f %s" % (float(value) / pow(1024, exponent), byteunits[exponent])
  
def bytes2human(n):
    """
    >>> bytes2human(10000)
    '9K'
    >>> bytes2human(100001221)
    '95M'
    """
    symbols = ('K', 'M', 'G', 'T', 'P', 'E', 'Z', 'Y')
    prefix = {}
    for i, s in enumerate(symbols):
        prefix[s] = 1 << (i + 1) * 10
    for s in reversed(symbols):
        if n >= prefix[s]:
            value = int(float(n) / prefix[s])
            return '%s%s' % (value, s)
    return "%sB" % n
   

def cpu_usage():
    # load average, uptime
    av1, av2, av3 = os.getloadavg()
    return "%.1f %.1f %.1f" \
        % (av1, av2, av3)
  
def cpu_temperature():
    tempC = ((int(open('/sys/class/thermal/thermal_zone0/temp').read()) / 1000))
    return "%s" \
        % (str(tempC))
  
def mem_usage():
    usage = psutil.virtual_memory()
    return "%s" \
        % (usage.used)
   
def disk_usage():
    usage = psutil.disk_usage("/")
    return " %s" \
        % (usage.used)
   
def network(iface):
    stat = psutil.net_io_counters(pernic=True)[iface]
    return "%s: Tx%s, Rx%s" % \
           (iface, bytes2human(stat.bytes_sent), bytes2human(stat.bytes_recv))
   
def lan_ip():
    #f = os.popen('ifconfig eth0 | grep "inet\ addr" | cut -c 21-33')
    f = os.popen("ip route get 1 | awk '{print $NF;exit}'")
    ip = str(f.read())
    # strip out trailing chars for cleaner output
    return "IP: %s" % ip.rstrip('\r\n').rstrip(' ')
 
cpu_sp=psutil.cpu_freq(0)
av1, av2, av3 = os.getloadavg()
 
print ("{\"cpu1\":" + "%.1f" % av1 + ",\"cpu2\":" + "%.1f" % av2 + ",\"cpu3\":" + "%.1f" % av3 + ",\"temperature\":" + cpu_temperature() + ",\"ram\":" + mem_usage() + ",\"disk\":" + disk_usage() + ",\"cpu_speed\":" + "%d" % int(cpu_sp.current) + "}")

Facebooktwittergoogle_pluspinterestlinkedin

23 thoughts on “Graphing System Info in Node Red and Grafana

    1. GOOD - but only 7 out of 10 marks 🙂

      Not all boards by any means use that... not sure but might've been the Orange Pi... uses zone1.... and then another one the value needed to be divided by 1000.... etc...and if you look at the Python code, not sure but that might be how I did it anyway... it is of course trivial to check - if the temperature is 40,000 - divide by 1000 - if it is zero - you're looking at the wrong register or your Pi just turned into a peltier device.

    1. Interesting that there is a significant difference between the two readings on my Pi. With the /sys version being around 14 deg C higher.

      /sys/class/thermal/thermal_zone0 actually points to /sys/devices/virtual/thermal/thermal_zone0/ and there is a fair bit more information hidden in there if you can work out how to read it.

      1. Be really nice if someone came out with a simpleton-readable general guide to all of these useful locations. So much info, so well hidden.

        A bunch of aliases like /info/cpu_temperature would be nice 🙂

          1. Yes, it is a bit weird. Oddly, this is something that Windows does much better than Linux.

            Again though, running a utility to capture all this stuff is much better. Telegraf or something similar isn't a great overhead even on an SBC.

    1. But it's being bothered... the code I just put up works - I use it on all my machines and the only thing I've ever had to change is the location of the temperature sensor and MAYBE do a multiplier - but generally just works.

    1. If you want, you can configure Telegraf to output to MQTT.

      For Node-RED, you could also output to UDP (via the Socket_Writer).

      Or output to file, InfluxDB, Apache Kafka, elastic search, some cloud monitoring services, ...

      1. Fun! I just installed and configured Telegraf on my Windows PC with default inputs and output to UDP. I set up my test Node-RED instance to listen on the UDP port and bingo!

        Loads of data such as:

        {"payload":"{"fields":{"Percent_Disk_Time":1.1143429279327393},"name":"win_diskio","tags":{"host":"DESKTOP-.......","instance":"0 C:","objectname":"PhysicalDisk"},"timestamp":1510580620}","fromip":"127.0.0.1:57987","ip":"127.0.0.1","port":57987,"_msgid":"5b97c47e.f1a33c"}

  1. Here is a simple flow that will get CPU temperature, CPU load, free memory and free hard drive space for an ARM.

    [{"id":"1c9d19c9.8a913e","type":"exec","z":"d8b4747f.73fb58","command":"cat /sys/class/thermal/thermal_zone0/temp","addpay":false,"append":"","useSpawn":"","timer":"","name":"CPU Temp.","x":432,"y":106,"wires":[["cf6d0532.df47d8"],[],[]]},{"id":"567c9074.b5128","type":"inject","z":"d8b4747f.73fb58","name":"10 seconds","topic":"","payload":"","payloadType":"date","repeat":"10","crontab":"","once":false,"x":209.5,"y":183.25,"wires":[["1c9d19c9.8a913e","d283ed48.5dc4","78aea6f8.1cd8"]]},{"id":"d283ed48.5dc4","type":"exec","z":"d8b4747f.73fb58","command":"top -d 0.5 -b -n2 | grep \"Cpu(s)\"|tail -n 1 | awk '{print $2 + $4}'","addpay":false,"append":"","useSpawn":"","timer":"","name":"CPU Load","x":432,"y":186,"wires":[["2be28cc6.2b667c"],[],[]]},{"id":"78aea6f8.1cd8","type":"exec","z":"d8b4747f.73fb58","command":"free | grep Mem | awk '{print 100*($4+$6+$7)/$2}'","addpay":false,"append":"","useSpawn":"","timer":"","name":"Free Memory","x":432,"y":266,"wires":[["2572d691.7c055a"],[],[]]},{"id":"d08bcea3.6c4ef","type":"exec","z":"d8b4747f.73fb58","command":"df -h","addpay":false,"append":"","useSpawn":"","timer":"","name":"Disk Usage","x":432,"y":346,"wires":[["dc5bcdc5.fedcc8"],[],[]]},{"id":"dc5bcdc5.fedcc8","type":"function","z":"d8b4747f.73fb58","name":"extract data","func":"var re = /([0-9]{2})%/\nvar idx = msg.payload.search(re);\nvar str = msg.payload;\nif (idx >=0) {\n str = msg.payload.substring(idx, idx + 2);\n}\nmsg.payload = str;\nreturn msg;","outputs":1,"noerr":0,"x":632,"y":346,"wires":[["bc5a1426.0e5b58"]]},{"id":"c0a69251.44a42","type":"inject","z":"d8b4747f.73fb58","name":"1 minute","topic":"","payload":"","payloadType":"date","repeat":"60","crontab":"","once":false,"x":222,"y":346,"wires":[["d08bcea3.6c4ef"]]},{"id":"cf6d0532.df47d8","type":"debug","z":"d8b4747f.73fb58","name":"","active":false,"console":"false","complete":"false","x":847,"y":97,"wires":[]},{"id":"6979f666.012968","type":"debug","z":"d8b4747f.73fb58","name":"","active":true,"console":"false","complete":"false","x":837,"y":166,"wires":[]},{"id":"2be28cc6.2b667c","type":"debug","z":"d8b4747f.73fb58","name":"","active":true,"console":"false","complete":"false","x":837,"y":166,"wires":[]},{"id":"b889c277.c74d68","type":"debug","z":"d8b4747f.73fb58","name":"","active":true,"console":"false","complete":"false","x":839,"y":249,"wires":[]},{"id":"2572d691.7c055a","type":"debug","z":"d8b4747f.73fb58","name":"","active":false,"console":"false","complete":"false","x":839,"y":249,"wires":[]},{"id":"bc5a1426.0e5b58","type":"debug","z":"d8b4747f.73fb58","name":"","active":false,"console":"false","complete":"false","x":833,"y":344,"wires":[]}]

    1. Erm, just out of interest, I tried that flow (the temperature mechanism isn't much different to the Python one and may need adjusting on different SBCs)...

      I get "32" for disk usage... what does that mean?

      and for cpu temp/cpu load and free memory... I get only two answers "47236" and "57.058"

      I know what the 57 is - clearly processor temperature - but the other one and the missing one.....

  2. The disk and memory are the % remaining.
    Are you running it on a pi? I totally understand what you are saying about having one flow that works on every device in existence (ie, your house), but I only have Pi's (at my house). So I was just trying to help on that front.
    Attached is the quick Node-RED Dashboard that I built for showing that flow data.

  3. Pete, Try this flow, it is directly out of my Pi3.

    [{"id":"5c9283f1.63d1ec","type":"exec","z":"4f751a6c.09aaa4","command":"vcgencmd measure_temp","addpay":false,"append":"","useSpawn":"","timer":"","name":"RPi Temp.","x":370,"y":160,"wires":[["58bbbcb9.27ab74"],[],[]]},{"id":"61f52cfc.a1ec94","type":"inject","z":"4f751a6c.09aaa4","name":"10 seconds","topic":"","payload":"","payloadType":"date","repeat":"10","crontab":"","once":false,"x":164.5,"y":168.25,"wires":[["5c9283f1.63d1ec","ceda7b32.8763e8","39a7cbfa.8aa854"]]},{"id":"58bbbcb9.27ab74","type":"function","z":"4f751a6c.09aaa4","name":"","func":"str = msg.payload\nmsg.payload = str.substring(5,9);\nreturn msg;","outputs":1,"noerr":0,"x":550,"y":160,"wires":[["7ccd8a7e.b7efa4","911bc84a.386348","29e073e4.a77cec"]]},{"id":"ceda7b32.8763e8","type":"exec","z":"4f751a6c.09aaa4","command":"top -d 0.5 -b -n2 | grep \"Cpu(s)\"|tail -n 1 | awk '{print $2 + $4}'","addpay":false,"append":"","useSpawn":"","timer":"","name":"CPU Load","x":370,"y":240,"wires":[["8545fcc2.f858c","8c05f5b6.e39e68"],[],[]]},{"id":"39a7cbfa.8aa854","type":"exec","z":"4f751a6c.09aaa4","command":"free | grep Mem | awk '{print 100*($4+$6+$7)/$2}'","addpay":false,"append":"","useSpawn":"","timer":"","name":"Free Memory","x":370,"y":320,"wires":[["3a4727ea.f42288","93c5f5dd.2cd4f8"],[],[]]},{"id":"42cc10e7.3b76e","type":"exec","z":"4f751a6c.09aaa4","command":"df -h","addpay":false,"append":"","useSpawn":"","timer":"","name":"Disk Usage","x":370,"y":400,"wires":[["63477c68.7bf214"],[],[]]},{"id":"63477c68.7bf214","type":"function","z":"4f751a6c.09aaa4","name":"","func":"var re = /([0-9]{2})%/\nvar idx = msg.payload.search(re);\nvar str = msg.payload;\nif (idx >=0) {\n str = msg.payload.substring(idx, idx + 2);\n}\nmsg.payload = str;\nreturn msg;","outputs":1,"noerr":0,"x":550,"y":400,"wires":[["f31aee1f.d7827","438bd797.1b5198"]]},{"id":"7fcadbe0.57c984","type":"inject","z":"4f751a6c.09aaa4","name":"1 minute","topic":"","payload":"","payloadType":"date","repeat":"60","crontab":"","once":false,"x":160,"y":400,"wires":[["42cc10e7.3b76e"]]}]

    1. That does work - and thanks for taking the time to bring that here - for those desperate to avoid moving outside of Node-Red this is a good way to do it. However it involves making 4 EXEC calls as against one calling a simple bit of Python which you don't really have to understand to use as it's already done.... not sure which is the better method 🙂

      Now - what would be nice would be feedback on some other snippets from folk with OTHER useful pieces of information.. we have temperature, CPU usage, free memory and disk space, I know from messing with the python code that you can easily access up-time..... what about power voltage - can we get that.... there is some kind of status showing if the voltage goes below a minimum, could be useful to log how many times that happens.

      1. Glad that worked, but yeah, its rather hard coded. Not a fan, but it is pretty clean in Node-RED.
        Honestly, since getting Monitorix running, I have not been using Node-RED to do that function any more, but just wanted to share what we had working (sorry for the wrong flow in the first post, it was for a different ARM7 processor I think).

        For example, Monitorix shows voltage and clock frequency, so it can be helpful to see how 'hard' your Pi is spinning up its CPU to do intense tasks and how often it is thermal limiting the clock.

        I really need to spend more time in Python - been trying to chip at it for Open CV to read some old analog gas meter dials and get that data into Node-RED.....

        1. Don't get me wrong, I'm not a "fan" of Python- I absolutely hate the indent-sensitivity almost as much as I hate case-sensitivity in other languages...

          However, it does seem awfully widely available and the amount of drop-in libraries is, well, staggering, really. So sometimes as a "black box" callable from Node-Red it comes in very handy and it is also rather handy that the same output comes out to the console when testing - which makes iterative testing (or as I do it, suck it and see) very easy.

          But now this blog entry + comments contains ideas for both methods - and that's a good thing.

      2. Sorry to keep banging on about it but you can feed Telegraf directly into Node-RED via UDP (or even UNIX Sockets if you fancy). Several different output formats to choose from including JSON. Works fine on my Windows PC too. Or, of course, send it to MQTT directly.

Leave a Reply

Your email address will not be published. Required fields are marked *