Category Archives: imperihome

Background Beavering

You could be forgiven for thinking I’ve gone off the boil as I’ve not written much in here this week… far from it in fact. This week I’ve been working on my ImperiHome setup and the new Raspberry Pi software.

ImperiHomeImperiHome: In case you don’t know, ImperiHome is an app for Android and IOS that enables remote IOT control. It does this by trying to appeal to just about every system out there, none of which I use.  I’ve taken the approach of having IBM’s Node-Red control everything – and thankfully ImperiHome has a generic API to allow control if you don’t have ANY of the devices it supports.

I’ve written more than one item on this subject and I have to say despite a rather lacklustre approach to answering people’s questions by their team and something of a lack of generic controls (such as a momentary push-button), the product actually works and works quite well. I have RGB lights, relays, sensors and a host of gadgets attached to Node-Red. Those which are required to just come on and off at varying times of the day are controlled by my BigTimer (which, recently updated, works a treat – or rather, it does if you remember to set the time zone when setting up the Pi as I found out last night when all the lights were still on at 2am, defying all explanation until my wife said “did you ever set the clock on that thing?”).

Those gadgets which require interaction are controlled by ImperiHome on my HTC Smartphone.

On the subject of Node-Red, we are promised non-volatile global variables in future by the Node-Red team and I for one can’t wait – but for now I’ve just completed a function to save a global object (buffered in time) when any change is made. That global object forms the variables used in my ImperiHome project – so now at last, the phone is always aware of the state of my lights – a handy thing. Essentially the object contains a counter which is set whenever any other variable in the object is altered. This is monitored by a timer which also kicks in on power-up.  If the counter is true, it is decremented – if it hits zero I update the file on SD which contains that object.  I also check to see if the file exists – and if not, I create it. If the file DOES exist and the object doesn’t – I populate the object.  So now, my lighting and other settings can survive power-cycling.

Raspberry PiRaspberry Pi: On 10/05/2016 the Raspberry Pi foundation put out a new release of Jessie (Debian) and with it, items you might expect like better support for Bluetooth etc., but then something I wasn’t expecting – a decent backup program. After (what seems like) years of struggling with arcane Linux commands some of which I never really got to grips with and hence never achieved a satisfying backup solution, at last, a simple push-a-button backup program that not only allows for live backups, but also allows for the use of varying size SDs both larger and smaller.

WHY did this take so long…  I can’t tell you how this has changed things for me – backup time is reduced, no need to shut the house controller down when I’m doing backups, the use of smaller SDs, running multiple copies of the same software on different boards – the list goes on.

But for days I’ve been struggling with upgrades – the backups just did not want to accept upgrades (apt-get-upgrade) and when they did, invariably Node-Red would fall over with missing nodes etc. I didn’t know if it was the backup program or what…  (I knew it was not the SDs as I use only good ones). Well, it turns out that my script was in need of bringing up to date. I still scrap node, npn and node-red and re-install to get the very latest but I’ve done it a different way, more by trial and error, but I got there in the end and now my backups and restores are working perfectly. I’ll publish the updated script soon, replacing the older one and I’ve added some tricks I’ve picked up along the way from readers like you.

Tiny part of my controls

I started this blog entry before our BBQ yesterday and here I am up early on a Monday morning finishing off the job. The lights worked a treat last night now the clock is set correctly – thanks to NTP it will now stay that way.

Coming up this week hopefully – a couple of new boards from Friendly Arm, the new Nextion display and more. But first things first, some supply shopping,  I’ve lost a filling and Maureen’s done her foot in – this should be good – dentist and hospital – in Spain (and not in the tourist areas where they speak English).

Facebooktwittergoogle_pluspinterestlinkedin

Imperihome Database

I’ve been asked to go into a little more detail about the database side of Node-Red with Imperihome

So for example, let’s say you have a temperature sensor. My last blog on the subject shows you how to put the sensor up and show the current temperature. However, if you want to store readings and be able to click on the icon and see a graph, you have to make the data available to Imperihome in a form it understands.

Some time ago I explained my reasoning for using SQLITE on the Raspberry Pi and other SD based systems and so this explanation applies to SQLite 3 and the PHPLITEADMIN tool as implemented in my script.  I use this setup along with Mosquitto (with websockets) in all my installations.

So before we get into this – let’s define tables in a database. You can use any database you have handy or a new one – but you’ll need 3 tables. One defines the device, another defines an attribute of that device (so you might have a device which is a thermometer and the attribute might be the temperature – but there are combination devices with more than one attribute such as the combined thermometer/hygrometer so there are 2 attributes – temperature and humidity – and finally you need a third table – the data. This will contain a timestamp, the actual value, the device and an attribute so that Imperihome can request data – and it might do this for the last day, last week, last month etc.  Clearly you don’t want to send thousands of values out so for longer periods you want a reasonable number of averaged values.

Here are the three tables.

Device;

CREATE TABLE 'Device' ('DeviceID' INTEGER PRIMARY KEY,'DeviceName' TEXT NOT NULL)

DeviceData

CREATE TABLE 'DeviceData' ( 'DataID' INTEGER PRIMARY KEY, DeviceID INTEGER,'DataName' TEXT, FOREIGN KEY(DeviceID ) REFERENCES Device(DeviceID ) )

Data

CREATE TABLE 'Data' ( SequenceID INTEGER PRIMARY KEY, 'DeviceID' INTEGER NOT NULL, 'DataID' INTEGER NOT NULL, 'DataValue' NUMERIC NOT NULL, 'epoch' NUMERIC NOT NULL, 'timestamp' DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP , FOREIGN KEY(DataID, DeviceID ) REFERENCES DeviceData(DAtaID, DeviceID ) )

Imperihome uses epoch but as humans have trouble with that we also include a human-readable timestamp.

Typical devices – deviceID is created automatically as you add devices – yes I could have used way more meaningful device names.

tmp5C8D

DeviceData

tmp6F46

As you can see above – one device (number 3) with two values (temp and hygro) – the rest are just called value (don’t blame me – that’s what Imperihome is asking for).

And finally some typical data – the lower values are temperature in degrees C and the higher ones are percentage humidity.

tmp33AC

All simple stuff. You need a flow which will work without modification for all data

tmp1EE7

The incoming html GET on the left is this

tmpD9BC

Then the function which says above “Request History from SQL Lite”

// This will handle any device and any attribute as long as it is in the DB
var Month = 5184000000 ; //60 Days
var Week  = 1209600000 ; //14 Days
var Day   =  172800000 ; // 2 Days
var Hour  =    3600000 ; // used to aggregate data by the hour
var Minute  =   600000 ; // used to agrgegate data by the 10 Minute
var period = msg.req.params.enddate - msg.req.params.startdate;
var groupby = Day ; // used for group by clause below

// The select will provide the starting Date for the period and the average value for the period
//The History Ui on ImperiHome expects the current period and last, that’s why double the rang than expected
if       (period <= Day)  {groupby =Minute;}// Ask for a Day and get by the minute
else if  (period <= Week) {groupby =Hour;}// Ask for a week and get it by the hour
else if  (period <= Month){groupby =Day/2;}// Ask for a Month and get it by the Day
else                      {groupby =Week/2;}// Ask for a Year and get it by the Week

msg.topic =
"SELECT MIN(epoch) as date, printf('%.2f', AVG(DataValue)) as value FROM Data" +
" WHERE DeviceID = (SELECT DeviceID from Device WHERE DeviceName = '" + msg.req.params.deviceID + "')" +
" AND   DataID = (SELECT DataID from DeviceData WHERE DataName ='" + msg.req.params.paramKey + "'" +
" AND DeviceID = (SELECT DeviceID from Device WHERE DeviceName = '" + msg.req.params.deviceID + "'))"  +
" AND   epoch >= " + msg.req.params.startdate +
" AND   epoch <= " + msg.req.params.enddate +
" GROUP BY epoch/" + groupby

return msg;

A friend of mine Peter Oakes wrote this as I’m crap at databases. It works.

The orange History Archive is a SQLITE node – just point it to your chosen database remembering that a database in SQLITE is just a file – I keep mine in a folder under /user/pi

The next function could not be simpler..

//Really complex conversion so be careful if you change this 🙂
var results = msg.payload ;
msg.payload = {"values": results };
return msg;

and you feed that to the outgoing html node.

That’s it. Imperihome will take care of the rest – all you have to do now is make sure that whenever you take, say, a temperature reading, you fire it into the database – example below.

tmp6D4F

So I save my values in global variables and here, every 5 minutes I inject the values into the database – you might do it another way.

So each of the 3 functions above (temperature, humidity, office temperature) looks like this:

msg.deviceName="dev05";
msg.dataName="value";
msg.payload=context.global.thermostat.display_temperature;
return msg;

and the function that shoves it into the database is simply….

// Set the next two variables to suit your device
// make sure there in the DB as well

var d = new Date();
var epoch = d.getTime();
msg.topic = "INSERT INTO Data(epoch, DeviceID, DataID, DataValue)" +
            "Values(" +
            epoch + "," +
            "(SELECT DeviceID from Device WHERE DeviceName = '" + msg.deviceName + "')," +
            "(SELECT DataID from DeviceData WHERE DataName ='"  + msg.dataName   + "' AND" +
            " DeviceID =(SELECT DeviceID from Device WHERE DeviceName = '" + msg.deviceName + "'))," +
            msg.payload +
            ")"
return msg;

That’s pretty much it.

Facebooktwittergoogle_pluspinterestlinkedin

Putting it all together in Imperihome

This blog entry is about using the Imperihome APP with Node-Red to control IOT. This is looking like the best APP out there.. Read on..

Please note: 14/03/2016 Update thanks to vastly improved understanding of Imperihome – part of this is  thanks to Peter Oakes and his homework (check him out on YouTube). See here for a new video on the setup… more up to date than this blog entry. https://www.youtube.com/watch?v=1p3OwICqQbc

I’m writing this for three reasons:

1. To clarify how Node-Red and Imperihome can work together to allow remote access to IOT

2. To form a base from which to expand

3. To get the conversation away from the original Imperihome article (previous blog) which is now WAY overcrowded.

So here is a basic setup for an RGB LED with colour, saturation and intensity control. Output is to debug but you could of course send that off to any RGB light or to MQTT (which is what I will do). Here is the API information for various devices..

Here is the APP itself in the Google Play Store.

The basic setup for using the Imperihome API (documented here – suggest printing this out it you are interested) is as follows:

You need to define:

  1. The system
  2. The rooms
  3. The devices

Rooms are merely a mechanism for separating stuff up into pages – use them as you will.

You set this up on Node-Red – then merely create a new SYSTEM using IMPERIHOME STANDARD SYSTEM in the App. If you don’t have password setting on your Node-Red pages then you don’t need a username and password. That simple. The local URL  in the APP will be http://192.168.0.x:1880/test1 – if you want to use this outside of the home at the very minimal you want username and pass and preferably setup https (the two security items being outside of the scope of this article).

Imperihome[6]

Let’s go through this step by step – the cream things are standard http input and output nodes. The latter need no setup at all, the input nodes need and address – I’ve called my base address “test1” – don’t worry you don’t have to create any test pages – you need do nothing more than set up this tab in Node-Red and setup the APP as I’ve described above.

The URLS for the first 3 http nodes are self explanatory – the last on (rest actions) is as follows:

/test1/devices/:deviceID/action/:actionName/:actionParameter

So the first 3 are incoming requests from the APP (by default every 5 seconds and at startup) – I change my default (in settings for the APP SYSTEM) to 2 seconds.

The system request and room requests are just setup – in this case I have one “room”  - I can have as many as I like and these are essentially tabs in the APP.

The orange units are functions you write yourself. The first function is merely setup.

msg.payload = { "id": "ISS:Test:01", "apiversion": 1};
return msg;

The third one – the rooms is also simple:

msg.payload = {"rooms": [{ "id": "roomID1", "name": "Test Room" }]};
return msg;

If you want more rooms you simply add them as such:

msg.payload = {"rooms": [{ "id": "roomID1", "name": "Test Room" },[{ "id": "roomID2", "name": "Other Room" }]};
return msg;

Ok, this takes us to the devices function. This simultaneously tells the APP what devices you have (you don’t need to set anything up on the APP at all other than your node-red page address the name of which (test1) is arbitrary.

var rgbState = global.get('rgbState') || 0;
var rgbLevel = global.get('rgbLevel') || 0;
var rgbColour = global.get('rgbColour') || "AAAAAAAA";

msg.payload = {"devices": [
{ "id": "dev01", "name": "RGB Light", "type": "DevRGBLight", "room": "roomID1", "params": [{"key": "dimmable", "value": "1"}, {"key": "whitechannel", "value": "0"}, {"key": "Status", "value": rgbState}, {"key": "Level", "value": rgbLevel}, {"key": "color", "value": rgbColour}]}
]};
return msg;

If you look at the Imperihome API instructions you’ll see all about parameters and types.

So – I pull in three global variables (which used to be of the form context.global.rgbstate but which are now accessed with .get and .put commands – looks odd if you’re not used to them but it’s simple enough.. the “!! 0” simply means return 0 or “AAAAAAAA”  if the variable has yet to be initialised.

Incidentally AAAAAAAA for a colour – yes that caught me out – it is HEX and the first 2 digits are irrelevant unless you are using WRGB LEDS.  I’m using RGB (the W is for WHITE).

So this sets up the RGB light device in the APP and prefills values – but also it is called regularly to update the APP display – such as the sliders and RGB control. rgbSTATE incidentally is for the APP RGB on-off button – and will be 1 or 0. Level will be 0-100 and the rgbColour will be any WRGB value (first 2 digits ignore).

What we are saying in that mess above is that this is an RGB light control in the main room, it is dimmable, we’re not using WHITE and here are the settings.

All that remains to do now is take action when the actual APP controls are changed… and that means storing updates to the global variables.

That is this bit extracted from above.

Imperihome[8]

So we have the http input and I described that URL above – and a simple switch – ie. which device are we talking about? I only have one device here…

Imperihome switch

See “dev01” – that is the name of the one and only device – really one should give the devices more meaningful names but that’ll do here.

There is one output from this switch node – there would be more if you added more (the +rule).

The output goes to a simple standard function (top)

msg.payload = { "success": true, "errormsg": "ok" };
return msg;

That then goes back to the APP via the http node. This is common to all devices…

So here it gets more interesting. You are taking in commands from the APP and doing something – firstly storing the results in global variables – but then taking RGB and brightness data and creating an output to go off to your RGB light – in this case just a debug output.

This is a bodge – does not take into account proper colour adjustment at various brightnesses – and right now people (just got your email Chris) are firing ideas at me so this version will go in the bin – but it will do for the purpose of experimenting.

 

function colorLuminance(hex, lum) {
    // Validate hex string
    hex = String(hex).replace(/[^0-9a-f]/gi, "");
    if (hex.length < 6) {
        hex = hex.replace(/(.)/g, '$1$1');
    }
    lum = lum || 0;
    // Convert to decimal and change luminosity
    var rgb = "#",
        c;
    for (var i = 0; i < 3; ++i) {
        c = parseInt(hex.substr(i * 2, 2), 16);
        c = Math.round(Math.min(Math.max(0, c + (c * lum)), 255)).toString(16);
        rgb += ("00" + c).substr(c.length);
    }
    return rgb;
}

switch (msg.req.params.actionName){
case "setStatus": global.set('rgbState', msg.req.params.actionParameter);  break; 
case "setColor":  global.set('rgbColour', msg.req.params.actionParameter);  break;
case "setLevel":  global.set('rgbLevel', msg.req.params.actionParameter);  break;
default: break;
}
if (global.get('rgbState') == '0' || global.get('rgbLevel') == '0') {
    msg.payload = 'RGB = 0,0,0';
} else {

    hexColor = colorLuminance(global.get('rgbColour').slice(2), -1+(parseInt(global.get('rgbLevel'))*0.01));

    msg.payload = 'RGB = ' + parseInt(hexColor.substr(1,2), 16) + ',' + parseInt(hexColor.substr(3,2), 16) + ',' + parseInt(hexColor.substr(5,2), 16);
    }
return msg;

 

The function above merely takes RGB and brightness and produces an RGB value adjusted by brightness.

The main code has a switch to check which command came in – to adjust status (1,0), level (brightness 0-100) and RGB value. These are merely stored in the global variables.

All that remains to do is output something suitable to the debug – it is crude but it works.

 rgb

 

So whenever you make a change to the on-off state of the control in the APP, or change colour, brilliance or saturation – you will see a new RGB value come out of the debug.

When first setup this works. I then go into the APP and adjust the refresh time to every 2 seconds and something strange happens… the level control “flickers”. It did this in my more complex set up and now does it in this simplified and “clean” version.

https://bitbucket.org/snippets/scargill/prn7L

Here are APP screenshots – ignore my username and pass.

Imperihome[10]

Imperihome[12]

 

Imperihome[14]

 

All very simple – and can be expanded on with multiple rooms, IP cameras and all SORTS of widgets – a BARGAIN at £4 for the APP.

Update 09/03/2016

It turns out that the brilliance slider thing was a bug – fixed. Meanwhile here’s a picture of my current setup with a few controls – note that the http responses are all tied together where the response is identical – makes for a much simpler layout.

imperihome

Facebooktwittergoogle_pluspinterestlinkedin

Imperihome

 

ImpirihomeImperihome is a well-developed App (Android and IOS) for controlling gadgets (there are some IOS limits). It supports a wide range of systems and has a good range of controls and graphs.

Problem is… I don’t want to control any of these – Node-Red is doing a fine job of that right now.  I’m using Blynk and node-red-contrib-UI for visuals along, of course with Nextion displays (more on the latter soon).

The Blynk is fine but it has some really dumb restrictions (one page per project – and as some of the widgets are very large that makes it useless for some applications) – also it remains to be seen how quickly we’ll get new widgets when they go commercial this month and I’m not really that happy about an on-going payment. Blynk could do well or flop due to lack of support when they ask for money – and how quickly new controls emerge – who knows.

The node-red-contrib-UI program development for now at least APPEARS to have stopped – hoping that will change. Both of these incidentally work very reliably in their latest incarnation but Blynk has an inexplicable “ON” or “OFF” message on buttons which just seems rather limiting to me – what if you want to use the buttons for “up” and “down”. I’m told this will be fixed… but no timescale for when.  I’ve also been pressing for multiple pages per projects for some time – that also has still not arrived.

ImperihomeAnd so we have this lovely looking Imperihome App which reader Gregg brought to my attention. Now I have to say that I am able to write this now only because of some sterling weekend work by others who’s comments you should read. AT FIRST GLANCE this would appear to blow the pants off either of the previously mentioned UI systems.

A quick browse through the web showed NOTHING on the subject of Imperihome and Node-Red – but we’re fixing that right here – this APP is meant to work with commercial off the shelf units and I’m sure it will do that well – look at the comments on their site – mostly positive or even glowing – but I’m not interested in any of that – I have a working setup to control lots of devices via Node-Red and I’m happy with that  - what I want is a pretty and well-functional display for my phone – and it might just be that this is it. Now that Node-Red is standard on one of the world’s most popular SBCs (the Raspberry Pi – just recently given a major upgrade to RPi3) – I see Node-Red really going places hence sticking with it as my main control hub.

So – Imperihome - multiple pages – SURE – now, not later.  Decent RGB controls – YES – and they don’t take up half the screen.  Loads of devices including heating systems, RGB lights etc, yes. Multiple screen layouts – yes – again – now not later.

And now – Node-RED. Adding new devices is a matter of adding a couple of lines to the code that Steven Daglish has provided in the comments below (note his example mentions a lightswitch – just not implemented so don’t panic when you can’t see the light.

Note the pictgures here – this is just a TINY glimpse at what you can do with this app - you really need to go look at the site – the App is £4 but there’s a free demo – I lasted 20 seconds before I had to go buy the full App – it is WELL worth it.

You can see a bunch of controls – or have widgets with options for each widget – you can have variable numbers of widgets on a page so you might have large widgets on a phone or loads of little ones on a tablet – this is just WAY more advanced than anything else out there.

Oh, oh!! And it has speech recognition “turn off boiler” works no problem – sadly you seem to have to press a button to start it off – “ok google” takes you to the normal search.

This just keeps getting better – I just added one of my EASY-N cameras in Spain – and it works.

Note this blog entry is a re-write of the original article from earlier this week in the light of new knowledge. Expect to see LOTS more about this in here soon as I get my head around this and thanks to the guys who’ve contributed – keep it coming  - we’re onto a winner here I think. See the comments.

Facebooktwittergoogle_pluspinterestlinkedin