Waiting for Peter, closing out 2017

Peter ScargillNode Red, node-red34 Comments on Waiting for Peter, closing out 2017

MrShark at the console, while we’re all waiting for Peter to come back, let’s try to add some new content to the blog 🙂

I’ve a couple of new boards to test, unfortunately they’ve to wait I receive their PSUs… material for an other time 🙂

So, I’ll add some valuable resources found on the net in recent times, about Node-Red development…

I wish a Happy New Year to Peter and his wife, and to all of you readers… please keep visiting this blog and comment, let’s leave some new material for him when he’s back 🙂

Coursera

Coursera published 2 introductionary courses about IoT and Node-Red: both are completely free (while you can pay a fee to have a certificate at the end of it, if you want/need it) and very well done…

1st course is about IoT and basic Node-Red: A developer’s guide to the Internet of Things (IoT). From the overview on their site:

Quickly create applications that leverage connectivity and analytics as part of an integrated IoT platform. Use Node-RED, an open-source visual application development environment, on both the device and the cloud. Create a basic IoT solution by leveraging pre-built blocks of code that abstracts and speeds the development process. Use APIs to access the platform and explore the different connectivity options for various devices, gateways and applications. Explore options to ensure your solution makes best use of the captured data. The programming assignments require you to have a Raspberry Pi device – any model of Raspberry Pi with a 40 pin header. The course also uses the SenseHAT extension board, whilst it is preferable to have a SenseHat there is a simulator provided if you can’t get a SenseHAT.

2nd course is about Node-Red itself: A Developer’s guide to Node-RED. From the overview on their site:

The course will improve your use of Node-RED. It will introduce some more advanced features available in key nodes, show you how to visualise data using dashboard nodes. It shows you how to create web APIs using Node-RED and how to consume web services and how to make use of different storage technologies within a Node-RED flow. The last section of the course shows how you can extend Node-RED by creating your own nodes.

Advanced Videos by Csongor Varga

Reader Csongor Varga makes excellent and thorough videos about Node-Red, very detailed so you can learn good practices, new nodes and how to use Node-Red while you can even don’t apply all his flows but take useful pieces of them to suit to your needs 🙂

Here’s some of the longest ones, you can find many more on his YouTube channel.

34 thoughts on “Waiting for Peter, closing out 2017”

  1. If anyone is interested, I’ve done a few new PCB designs during Pete’s recovery – all ESP8266 based.

    So, the most useful one for me at the moment is an Infre Red repeater board. This picks up the IR from – currently – a Sky+ remote control and transmits it over either a direct HTTP web access or viq MQTT or both at the same time. So, you could control 2 different devices at the same time.

    Why, I hear you ask? Well, like Pete I’m building a house in Spain (offgrid), plus the wife and I travel a lot. I have a Slinbox tied to one of my SKy receivers, so I can now change channels using a standard Sky remote control from anywhere in the world. She isn’t keen on using the on-screen control menu in the Slingbox android app and frankly neither am I as it’s a bit clunky.

    The software so far decodes the Sky IR remote signals and then transmits only debugged codes. So, I’m not getting any false triggering at all.

    There’s no reason why this couldn’t be used to control any IR device with standard 38KHz modulation. I’ve even done a simple interface using the html GET method. So, from a web browser you can type:-

    http://slinbox-ip-url/command?irCommand=45FC67
    for example, to send a random IR code.

    Or, you can send:-
    http://slinbox-ip-url/command?irSendCommand=SELECT

    I’ve implemented all of the SKY+ remote buttons as text commands.

    I’ve used a small plastic case that Pete found on Aliexpress which costs just a few pence each.

    I’ve also done another PCB using the same case which has 2 relays and a DS18B20 temperature sensor as a cut down version of the ESP-DEV board that we developed a couple of years ago.

    Pete’s recovering well and as soon as he gets back up to speed fully, I’ll ask him to put the files up on the blog if anyone shows interest.

  6. Maybe not relevant here but…
    I have 20 Sonoffs around my house all using Pete’s firmware and all up to date. A couple of them keep dropping out and will only recover with a re-power. Any guidance anyone?
    thanks
    John

  7. Adding new content, well, I did discover a few things since the comments I have made around autumn time. Hence, I’ll elaborate.

    ESP8266 will execute directly from flash. evidenced by:
    SPI_write of raw xtensa instructions
    a typedef void (*pfn)(void *cxt);
    then a pfn Jumper = (pfn)0xADDRESS_I_SPI_WROTE;
    then a Jumper(myData);
    and a safe execution and safe return.
    ESP8266 will have unsafe function calls.
    by unsafe I mean if you have string literals the printf style functions will do a l8ui and crash
    therefore, single byte operations from flash are not permitted
    furthermore, unaligned access is a potential problem with their library
    by unaligned I mean 32bit alignment is violated ptr & 0x3 != 0
    In order to execute code loaded to flash, there were some headaches
    had to hard code the flash address for the linker script
    had to ensure I flashed the code at that address on the esp8266
    had to make a choice of either using RAM to copy parameters of std functions when addr was > 0x4021000 or to rewrite the std functions to do bit shifting and avoid unaligned or byte level access
    had to modify the makefile examples heavily in order to generate an elf that is reliably workable for the next step (immediately below) and post-proc the bin
    had to write an elf parser to strip out all the needless junk that gcc adds in and strip down to the string literal tables and code area (bss? IIRC)
    finally, wanted to add a lot of analysis tools in the makefile, such as objdump, running thru C pre-processor only, and generating C mixed with ASM instruction files

  8. Thanks for the Node-red links, esp the Coursera courses. Completed the first one today (almost – discovered I really need to learn more Javascript) and progressing through the second one. It’s very IBM cloud-centric – but you can get a six-month trial whilst progressing through the courses

  9. I have seen on some internet post where people have ‘bridged’ MQTT brokers, I am not certain ‘bridging’ brokers provides any back up ability because I don’t really understand what ‘bridging’ does. I have seen it mentioned on Element 14. I guess another option might be to use HiveMQTT.
    A decent UPS would also be an option but as with all things its a matter of cost and how critical your system / data is.

    1. If something being automated is so critical, that no downtime, or data loss can be tolerated, then I’d question a PI/ESP solution.
      We are relying on Wifi networks which can/do drop out, and client devices such as the ESP which have no queing ability of their own. So even if you have the most robust mqtt server in the world, there,s still no guarantee that all messages will make it onto the Q.
      I have implemented Fail Safe (actually Marriage Safe) mode for anything critical. Thats to say, when the system isn’t working, my wife can still turn the lights on and off, and change the temp on the cooling system. This for some reason tends to coincide with when Im doing some minor tweaks to node red or the like. (just a coincidence- mind you)

  10. All the best for a happy, healthy 2018 to all, and a rapid recovery to Pete! And thanks for the NodeRed pointers – having only recently discovered this blog, I’ve become addicted to NodeRed but could do with some ideas on how best to approach its configuration.

  11. Thanks for the excellent links, wishing a Happy New year to all on here, hope it brings lots of good things. Best wishes for a speedy recovery to Pete here’s to him bringing us many more hours of Blogging fun.

  13. Any chance to integrate NRMetric in HC2017? I suppose very far away!
    What have you done now with these links to Csongor Varga! He has published so many interesting videos about iot. I get all new ideas for my home automation.

    Any update on the UPS project for Pi? I now run on this(goo.gl/Ndxj1B) extra shield which has a terrible weak 5V regulator, continuously giving signals of insufficient 5V stability, it keeps going but just…..

    Any suggestions making my MQTT broker(on the Pi locally) more safe? Fall back on a mirror to make my home automation really reliable?

    So much to do…..

    1. follow Csongor directions on how to produce services that his firmware can manage, and you’re done, is well explained in his video, waiting for the next part about the esp8266 code…

      about securing mqtt, mosquitto does not seem very well suited to clustering, AFAIK, so you have to switch it to other brokers or manage yourself some sort of double sending messages to 2 different brokers, taking care of reacting to only the 1st message arrived (the 2nd, if any, should be its clone…)

      1. Thanks very much, Mr Shark. First, to wish Peter and all his and your friends Health, Wealth and Happiness in this New Year. I’m so busy now reading up about the new pointers I’m glad I’m retired, as I’d never have time to go to work 🙂

      2. This link expains how you could cluster mqtt
        https://medium.com/@lelylan/how-to-build-an-high-availability-mqtt-cluster-for-the-internet-of-things-8011a06bd000

        Theoretically you could get a cluster running using multiple PI’s but you still end up with a single point of failure, the load balancer. In my view the weakest link is always going to be a PI, not that Im saying there’s anything wrong with them, I luv the little buggers, just that if you want High Availability, then dedicated hardware designed for the purpose is the way to go.
        Another option may be to use RabbitMQ, with this
        https://www.rabbitmq.com/mqtt.html. You can then explore the clustering capabilities of Rabbit.

        1. Thanks for these links. The rabbitMQ is way over my head. The other link is promising.
          The chance of failure of a Pi has been proven quite high without a rock solid supply backup! I had a few incidents where my system SD card broke without notice before I installed the UPS HAT. So I need a quick backup when you are running your house tech on it.
          I only had a quick look at it and I have to read about it a bit more. But without the support in The Script I think it is not possible for me to realise.

    2. You might want to look at secure cloud services if you want more resilience and backup of MQTT. I think that Azure, AWS and IBM Cloud all have robust services. You would need a resilient Internet link then as well of course.

      As to the reliability of the Pi. I’ve only a couple but they seem absolutely rock solid as long as you feed them enough power, don’t overheat them and give them storage more reliable than just the SD card. And, of course, cheap enough to run a couple of them anyway.

      1. problem i’ve found with “secure” mqtt, is that not any firmware allows to use it, as the ssl/tls part is quite hard on the cpu of these little devices like esp8266… i’ve a public broker, had to revert to standard not secure because of this…

        1. Yes, that is still a major issue. If you need an external broker, I would set up a local broker with non-secured connections internally and a secured connection to the cloud broker.

          In general it is always best to keep IoT devices of every kind away from your normal traffic and certainly away from the Internet. Even if you don’t care about them, they cause issues for everyone else as they provide potential platforms for the bad actors out there.

  15. Lots of interesting information on Noe-RED, thanks for the post.

    Happy New Year to all of the viewers/readers and a speedy recovery to Pete.

  18. Our summer intern made these videos on Node-Red.
    They have a flow from getting started to more advanced.



    This one is more stand alone, on how to make a RESTful call and control outputs.

    This one covers email.

    This one covers MSSQL.

  19. Happy New Year.

    Thanks for picking up the reins during Peter’s recovery.

    Peter, Get better. Prayers to you and your family during your recovery.

  21. Excellent idea Mr Shark, may I wish you and all Peter’s readers a very happy New Year. Some of the video stuff you posted looks really interesting. I’ll be following that up.
    If someone had some one had some interesting stuff related to what Peter has looked at or some further updates, how would we post it? Any ideas? Perhaps just as a reply. Any suggestions would be helpful.

Comments are closed.