The Script

If you are running a Raspberry PI and the script gives problems with Node-Red, note that 22/09/2018 I've updated script to use Node6.x and not Node 8.x as that currently causes issues, even the IBM script uses Node 6.x

Regular reader will know about the script that Aidan Ruff and I originally developed to put Node-Red and several other packages onto the Raspberry Pi for our own home control purposes. This has been developed with help from several people and in particular my friend Antonio.

The Script, ESP-GO and Nano Peripheral

WELL – here is the script which is intended to help set up certain Raspbian, Debian or similarly-based SBCs which now includes logging and handling Raspbian Stretch (tested on Raspberry Pi  2 and 3, 3B+). As well as it's original purpose of setting up a Raspberry Pi, the script also runs well with several other boards.  See right hand side of the above image for what the script does, given a basic operating system install. Read on…

Originally designed to run under Debian – reader Antonio and I have also made the script compatible with Xenial in some cases. So it now handles Raspbian, DietPi (Debian) and the Armbian versions of Debian and Xenial (Ubuntu) as well as the DietPi VM.

One issue I have occasionally with new boards is lack of remote root access to install the script. For example on a Banana Pi 3 I had this.. there was PI access for the likes of WinSCP but not root access. This was easily solved as user Pi - after using "sudo passwd root" to ensure I had a known root password.

sudo sed -i 's/prohibit-password/yes/' /etc/ssh/sshd_config
sudo systemctl restart sshd

That won’t work in all cases but I’m parking that here for reference.

The easiest way to get the script – as ROOT user, in your ROOT directory is:

wget --no-check-certificate  https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh

This can now be run easily: bash script.sh – run as user root, it will start up, copy itself into PI user area, creating that if it does not exist then will give instructions.

Before I begin... no, we’re not a support shop – you may or may not end up having to tweak things for your system but please don’t write in to say “I tried this on a Mac and it doesn’t work”.

The level of interaction after the beginning is now almost zero thanks to a new menu-driven approach and many improvements. We continue to add/modify features as appropriate.

What is this about?

What we have here is an old explanation of how to get Xenial running in a VM – the purpose of this is merely as a base to test the script which includes installs and set-ups as below:

  • Node-Red at port 1880 and lots of useful NODES
  • Mosquitto (MQTT) with Websockets
  • NODEJS/NPM
  • Webmin at port 10000
  • Apache at port 80
  • HA-Bridge at port 82 (suitable for Amazon Alexa)
  • SQLite and PHPLiteAdmin
  • MC File manager and editor
  • Grafana with InfluxDB (only tested on a few setups – so OFF by default)
  • Lots more – see the menu screen

Typically this setup would be used as the basis of a nerve-centre to control and monitor home control gadgets such as the ESP8266 units we discuss in this blog, Sonoff controllers and much more. Armed with this software, controlling the home is easy – and more importantly can be done YOUR WAY.

The Virtual Machine

Let’s take a look at what we did to get this running on a VM – you can of course duplicate all of this as you go along. We used the excellent VMware of VirtualBox on Windows PCs. This explanation covers VMware.

So – first things first – as I’ve done a video of the VirtualBox installation complete with sample use of Node-Red, here we’ll discuss the VMWARE version.  Both of these were done as test mechanisms before using the script on real devices such as the Orange Pi Zero, Raspberry Pi, Odroid and other devices discussed at length in the blog.

We loaded up a copy of VMware (I used VirtualBox but the procedure is similar – VMware player is free of you can use the full commercial workstation) and made sure it works. That much is up to you. There is a wizard for creating new virtual machines -  we chose a CUSTOM machine. Steps from that point on were:

NEXT - NEXT – “I will install the operating system later” – NEXT
Linux / Ubuntu 64-bit – NEXT – d:\test  - NEXT
Number of processors – 1  - number of cores – 1 – NEXT
Memory for this machine – 1024MB – NEXT
Use Bridged Connection - NEXT
I/O Controller type LSI Logic - NEXT
Disk Type SCSI - NEXT
Create a new virtual Disk - NEXT
12GB (I used 16GB – 8GB would have done) – Store virtual disk as a single file - NEXT
At this point we gave the file a name – in our case d:\test\Ubuntu 640bit.vmdk -  NEXT – FINISH

Under settings – hardware – CD/DVD we ticked “Use ISO image file” and selected the file we loaded from the Ubuntu website – “Ubuntu-16.04.1-server-amd64.iso” – OK.

At this point you’ll note reference to AMD64 – that had me as well but it works a treat on INTEL hardware.

Start off the VM – you get asked which language to use (ENGLISH) and various options will come up – the first being “Install Ubuntu Server” – that’s the one.

Once again you’ll get a language question – English. Location – in my case “United Kingdom” – if you don’t fit into the rules use “Other”.

Time Zone in my case – EUROPE  and then country. Locale – in my case United Kingdom.

Hostname – we went with the default “Ubuntu” – but call it whatever you want.

At this point you get asked to create a user – it is important you make a user “pi”. Enter password also (though the new script will create user PI if it does not exist).

tmp6049Home encryption – we said no. When asked if the time zone was correct we said yes.

Partitioning – “use entire disk” (remember this is a virtual disk – not your real disk). You are then asked to confirm erasing disk – go with that. You then get asked to confirm AGAIN. YES.

HTTP Proxy just leave blank – CONTINUE.

Install security updates automatically.

Add OpenSSH server and CONTINUE (important for remote access)

Install GRUB bootloader – YES.

Installation is complete – CONTINUE. Let it run.

And voilà – one working copy of Ubuntu.

When run, the script will generate a Pi user if it does not exist - will make that user part of the correct groups and will ask you to restart the script as user Pi.

The BASH Script

This part is about running the script and the easiest way is to start in a terminal as user ROOT in the root directory.

do the WGET shown above and you will end up with a script file called “script.sh” in the root directory.

At this point BACK EVERYTHING UP to avoid disappointment – no, REALLY.

Execute as:

bash script.sh

The script will if necessary create the Pi user but either way will now ask you to log in as user pi and repeat the procedure in the /home/pi directory - i.e. bash script.sh.

The point of this is to avoid another way which would be to create a file, load in the script contents ensuring you’re using Linux LF only format – then run the script after setting up permissions – this way you don’t have to do any of that.

In the end you will see a menu which will vary slightly depending on which system you are using – you won’t be asked about GPIO on a Raspberry Pi for example.

tmpE673

For the VM we left everything as you see here and tabbed to OK.

At this point you will be asked for a USER and ADMIN names (which can be user and admin if you like) and corresponding passwords. From there on – everything should happen with no user interaction.

After maybe 15 minutes – or an hour - ore a lot longer on a Raspberry Pi Zero WIFI   – you’ll see an invite to REBOOT – so REBOOT. When the board/VM boots up you will find hopefully a working system.

Assuming you've installed everything:

  • Node-Red runs at port 1880
  • Webmin at port 10000 (Webmin needs Pi or ROOT username and pass until we figure out how to add in the ADMIN user).
  • Apache runs at port 80
  • HA-BRIDGE will run at port 82 – It seems that Amazon Echo is quite happy with this but the Google product wants port 80!

Node-Red should include MQTT which will need admin username and password as will Node-Red itself. USER name and password is used by Node-Red Dashboard.

There is also the excellent MC – a file manager and editor.

MC and MC EDIT

I hope this brief introduction sets you off in the right direction and of course no doubt there will be discussing in the notes below.

Update 05 Jan 2017: At the time of writing I'm having issues with PIXEL in VirtualBox and DEBIAN on VirtualBox in terms of Apache not loading.. I'm sure we'll sort that - meantime some useful links.   Debian may not come with SUDO depending on which installation - here's a fix link that worked for me in my VirtualBox image.. If SSH setup won't allow remote access using passwords, here's a link that helped me.  Pixel of course will run in VirtualBox but by default it runs of a virtual DVD which is about as much use as an ashtray on a motorbike as nothing gets saved. Once running however it is a breeze to move over to the Virtual hard disk - you'll find that part way down the comments in this link.  So right now I have VirtualBox running Xenial, Pixel and Debian and the only issues I have are with Pixel and Debian - Apache - watch this space.  I’ve also improved the input for user and admin.

Important: Whatever you are using should have the SUDO command built in – it may not be obvious but Debian often does NOT have this installed by default – sort this FIRST. You should also make sure you have Pi user with SUDO group privilege – PI already has this of course and you may have added a user Pi – if not..

wget https://bitbucket.org/api/2.0/snippets/scargill/E5rG7/master/files/makepi
su -c "bash makepi"

Or, Here’s a useful link – takes seconds and is easy…  basically – as root you can do - apt-get install –y sudo….. and you should make sure that your PI user is part of the sudo group.

Update March 31, 2017:  the new version of the script "the script" works with the brand new NanoPi NEO2 and the NanoPi M1+ and features temperature monitoring and other improvements.

If doing this as a non-Pi user – i.e. ROOT on a machine with no Pi user – follow instructions above OR a slightly easier way…

Having uploaded the script and given permissions… as ROOT

./script.sh

as before – but when it has completed and wants you to log in as PI – stay where you are and:

su - pi -c /home/pi/script.sh

That might save you a little hassle.

Update 19 Aug 2017: Script updated to handle Raspbian Stretch on Raspberry Pi – tested on RPi 3. Also added code to help with non-root access to GPIO using ONOFF library. See blog entry on GPIO.

Facebooktwittergoogle_pluspinterestlinkedin

533 thoughts on “The Script

  1. I've been doing lots with running Ubuntu(Kubuntu,Lubuntu,Xubuntu,etc) from liveCD ISO files on my HD and with a partition or USB thumb drive labeled casper-rw there's an overlay(persistent) filesystem capability too.

    So my thought is, instead of installing Ubuntu in a VM, why not download the Ubuntu ISO file and setup a VM with the CDROM pointing to the file. Then create a small 8GB hard disk image and boot the vm into the live mode. partition and change the label of the 8GB partition to casper-rw and reboot. Now you have a quickly booting Ubuntu system to experiment loading up your home automation stuff(docker, node red, etc).

    For the noobs it gets them through the installation process fast and lets them play in Ubuntu land for a bit. If they like it, they can then generate another VM but this time go for the installation of Ubuntu so it's more able to handle updates.

  2. Hi Pete / Antonio,
    I've used Pete's script several time in the older format and had few problems. I ran the new one this morning with the menu system and found this problem on my Raspberry pi 3 loaded with Raspbian:-

    The script does not detect my Raspbian OS correctly. It thinks that it's not a genuine Raspbian OS. Here's my OS:-

    pi@raspberrypi:~ $ cat /etc/os-release
    PRETTY_NAME="Raspbian GNU/Linux 8 (jessie)"
    NAME="Raspbian GNU/Linux"
    VERSION_ID="8"
    VERSION="8 (jessie)"
    ID=raspbian
    ID_LIKE=debian

    I changed this line in your script:-
    if [[ $OPSYS == *"RASPBIAN"* ]];then
    to
    if [[ $OPSYS == *"raspbian"* ]];then
    and it detects my OS correctly.

    Many thanks to both of you for your great work.
    Keith

    1. Correct, Keith - I've now put a case converter in to make all results upper-case - so that should not happen again - please report back any other issues as this is all new. I've also just put a handler in for the CANCEL option which now works.

  3. Hi Pete,
    Here's another one:-

    Line 255
    MYMENU=$(whiptail --title "Main Raspberry Pi Selection Checklist" --radiolist \
    This only allows you to check ONE option.

    I think it should be :-
    MYMENU=$(whiptail --title "Main Raspberry Pi Selection Checklist" --checklist \

    Thanks for the 'CANCEL' handler.

    Keith

  4. Pete,

    There's a typo that prevents NPM from updating:-

    Line 268
    "upgradednpm" "Upgrade NPM to latest version " ON \
    I think that this should read:-
    "upgradenpm" "Upgrade NPM to latest version " ON \

    to match line 715
    if [[ $MYMENU == *"upgradenpm"* ]]; then

    1. Also, lines 721 and 722 should be swapped?

      720 # Add CU to enable serial VT100 mode for terminals
      721 printf "${ICyan}============================ Installing CU...etc
      722 if [[ $MYMENU == *"installcu"* ]]; then

  5. New update - should now check to ensure it is user PI that has logged in! If root and there is no user PI it should create one and ask you to log out and try again. !! Thanks to Mr Shark for this one.

  6. Hi Pete,

    Nice video. Always good to see things in action. Can I suggest maybe a video on messing with Node Red Dashboard, be nice to see its capabilities.

      1. Hi John - did you use the latest script as of today... I just installed on an Ubuntu real machine - and Debian and Ubuntu VMs without issue... but not on a Pi3. The issue with NPM was resolved ages ago - perhaps on a Pi the existing version is causing a problem?

        As for X terminal if the terminal is too small when you start the script it is no good - that is a "feature" of the menu script... just make the terminal a little larger.

          1. Hi Pete,

            I'll help with the testing on the Pi 3. I'm flashing a fresh Jessie image to a sd card now. Does it matter if I run apt-get upgrade before I run the script or should I just start with the default image?

          2. Pete, I'm happy to help test with Raspberry Pi if you want. I have a lot of experience with shell scripting too.

            "The issue with NPM was resolved ages ago"
            On a pi with the latest Jessie, NPM is not installed (or if it is, it's not on $PATH)

            1. I THINK that the Pi issue is now resolved and 2 changes made today to SSH and some helpful text for menus - please report only issues with software from this evening's date ONWARDS - and thanks to EVERYONE for their help - shell scripting experience always comes in handy - as you can see!

  7. I've been trying this script on a Raspberry Pi 3 with Jessie downloaded today.
    A couple of points:
    1 I see the whole menu if in command line mode, but on an X terminal a few characters are missing on the left hand side - so you can't select the options.
    2 I get a lot of errors eg Failed to fetch http://packages.dotdeb.org/dists/jessie/InRelease Unable to find expected entry 'all/binary-armhf/Packages' in Release file (Wrong sources.list entry or malformed file)
    3 As with the old script, it attempts to use NPM to install things and fails because NPM is not installed. "sudo: npm: command not found" Presumably NPM is already installed with other operating systems? Could it be added to the prerequisites?
    4 A logfile is useful to trawl through for error messages. You can get one with the command ./script.sh 2>&1 | tee script.log

    Regards,
    jb

    1. both right... just sent patches to Peter, i did tests on dietpi and ubuntu, all right, missing raspbian, if someone want to test, in latest script change this (from line 367):

      if [[ $OPSYS != *"UBUNTU"* ]]; then
      #enable php7 on debian jessie - no need for xenial
      echo "deb http://packages.dotdeb.org jessie all" |sudo tee /etc/apt/sources.list.d/dotdeb-php7.list
      echo "deb-src http://packages.dotdeb.org jessie all" |sudo tee -a /etc/apt/sources.list.d/dotdeb-php7.list
      wget https://www.dotdeb.org/dotdeb.gpg -O - | sudo apt-key add -
      sudo apt-get update
      fi

      with this:

      if [[ $OPSYS == *"DEBIAN"* ]]; then
      #enable php7 on debian jessie DIETPI
      echo "deb http://packages.dotdeb.org jessie all" |sudo tee /etc/apt/sources.list.d/dotdeb-php7.list
      echo "#deb-src http://packages.dotdeb.org jessie all" |sudo tee -a /etc/apt/sources.list.d/dotdeb-php7.list
      wget https://www.dotdeb.org/dotdeb.gpg -O - | sudo apt-key add -
      sudo apt-get update
      fi
      if [[ $OPSYS == *"RASPBIAN"* ]]; then
      #enable php7 on debian jessie RASPBIAN
      echo "deb http://repozytorium.mati75.eu/raspbian jessie-backports main contrib non-free" |sudo tee /etc/apt/sources.list.d/mati75-php7.list
      echo "#deb-src http://repozytorium.mati75.eu/raspbian jessie-backports main contrib non-free" |sudo tee -a /etc/apt/sources.list.d/mati75-php7.list
      sudo apt-key adv --keyserver pgpkeys.mit.edu --recv CCD91D6111A06851
      sudo apt-get update
      fi

      unfortunately, dotdeb repository has not arm packages, had to find them elsewhere...

      1. Right everyone - ... menu fixed - narrowed in for smaller screens and simplified.

        I've implemented the changes above suggested by Mr Shark. I have to go out - I will not be back until tomorrow mid-day - here's hoping these changes sort the Pi out.

        1. i think in the raspbian section, if wolfram and openoffice are checked to be removed, this should happen BEFORE initial general update/upgrade in prerequisites... no need to update something and then remove it after 😉

    2. 1 Peter reduced menu size

      2 my patch in comments here, and already included in latest online script, solved php7 on raspbian

      3 no npm problems, just installed latest raspbian (november 2016), launched script, everything is up and running...

      i made install in 2 batch, just to check everything working...
      1st run: modpass prerequisites mosquitto apache screen mpg123 phpsysinfo addindex installcu installmc wolfram office
      2nd run: modpass (to reinsert password for node-red and webmin) nodejs nodered webmin java upgradenpm
      not a single problem...

    3. pi@raspberrypi:~/.node-red $ npm -v
      4.0.5
      pi@raspberrypi:~/.node-red $ node -v
      v4.7.0
      pi@raspberrypi:~/.node-red $ npm view node-red version
      0.15.2
      pi@raspberrypi:~/.node-red $ npm list --depth=0
      /home/pi/.node-red
      ├── bcryptjs@2.4.0
      ├── node-red-contrib-bigtimer@1.4.86
      ├── node-red-contrib-blynk-websockets@0.0.3
      ├── node-red-contrib-boolean-logic@0.0.3
      ├── node-red-contrib-config@1.0.1
      ├── node-red-contrib-esplogin@1.1.9
      ├── node-red-contrib-file-function@1.1.2
      ├── node-red-contrib-gpio@0.9.1
      ├── node-red-contrib-graphs@0.3.5
      ├── node-red-contrib-grove@1.0.7
      ├── node-red-contrib-isonline@1.1.15
      ├── node-red-contrib-ivona@0.0.2
      ├── node-red-contrib-moment@2.0.0
      ├── node-red-contrib-npm@1.0.0
      ├── node-red-contrib-particle@0.0.4
      ├── node-red-contrib-timeout@1.0.4
      ├── node-red-contrib-web-worldmap@1.0.22
      ├── node-red-dashboard@2.2.1
      ├── node-red-node-arduino@0.0.12
      ├── node-red-node-darksky@0.1.14
      ├── node-red-node-emoncms@0.0.11
      ├── node-red-node-geofence@0.0.16
      ├── node-red-node-google@0.1.0
      ├── node-red-node-ledborg@0.0.17
      ├── node-red-node-openweathermap@0.1.14
      ├── node-red-node-ping@0.0.13
      ├── node-red-node-pushbullet@0.0.10
      ├── node-red-node-random@0.0.8
      ├── node-red-node-smooth@0.0.9
      ├── node-red-node-sqlite@0.1.2
      └── raspi-io@7.1.0

  8. Very interesting stuff! As I have an issue with the loss of mouse events in Chrome on Jessie I will try another installation on an 8Gb SD with the proposed Ubuntu image.
    I am a bit confused about all usernames and passwords needed. Should I use PI or pi as a user name? An overview of all needed un&pw's where to use what and when to create would be appreciated.
    Another item I have not touched is access from outside of home through internet, what to do on router(s) to reach my home server from outside?
    Keep on going the good work!

      1. Will give it a try with VMWare and that takes some time. The description is very extensive so I might be able to come all the way through. So no big hurry anyhow, but thx for the swift attention to this.

        1. Does not show the same options as described although the same Ubuntu image.
          There comes a question BIOS or EFI and default is BIOS, but I have no idea whether this is right. I can try EFI for a better result, but that is a shott in the dark. No question about adding SSH or GRUB.
          Display settings are not usable. No display is detected. It uses host settings and does not completely fit in window of VMware.
          How to change to ROOT user? I am a Windows user. This will take some googling.

          1. full album of the VM creation in Vmware: http://imgur.com/a/X8xLi
            you could use Vmware Player, too, as it's free and despite the name, allows to create VMs, too...
            in vm, no real difference between bios and efi, choose what you want...
            this setup uses UBUNTU 16.04.1 LTS SERVER, and so, no desktop environment... if you use a desktop iso, be sure to install vmware tools at the end, to enable graphic support (it's in the vmware player/workstation menus)...
            root is useless in ubuntu, it has no password, when you need to do something ubuntu asks for YOUR password, as first user created has SUDO rights, so can do administrative tasks without ever know root password or become root... but, many windows users used to be "administrator", tend to enable it... bad security habit...

            in a system with root enabled, an attacker only need its password to be harmful... in a system with a regular user, attacker has to know user name, user password, and THEN root password, if used has no sudo rights... but, your choice...

            1. Thx.
              I installed on W10-64 the desktop version. You are showing install on linux. Diff is menu with SSH and you did not cross Samba server, probably needed for W10 PC's on the network.
              Pete is advising some work in ROOT. Should I remove Root user?
              I will start from scratch again in EFI mode with one of the premade VM's you mention hereafter.

    1. the username and password that are asked by Peter's script are used for:
      user: only for node-red dashboard
      admin: mosquitto, phpliteadmin, node-red main interface
      webmin uses it's own access, which is root or pi, and their respective password

          1. ALWAYS use latest script, NEVER ask for help using old ones, as this could bring problems, to you and to us not understanding what's causing what else... in case, just say I'M USING VERSION FROM XYZ (DATE)... but better using latest, as said...

    2. for the internet access... without discussing security issues which will take way too much time... you need to create "virtual servers" or "nat" or "port forwarding" or whatever they are called in your internet router... if you have access to its admin interface, or you need to ask your ISP to do it... and open, for example:
      external port 1880 --> yourdeviceIP port 1880 (for node-red)
      external port 1883 --> yourdeviceIP port 1883 (for mosquitto)
      external port 80 --> yourdeviceIP port 80 (for apache)
      leave webmin and ssh away, for your good, if you don't know what you're doing... and use strong passwords...

  9. I've done another install on my Pi3 with a clean image of Jessie. I used the defaults on the script and I got through to the end without any errors. I did get a warning of an unused variable 'status' when the sqlite was installing though.

    Comments:-
    I though my Pi had crashed to start with until I saw the -qq switches on the update and upgrade. I'll remove the -qq next time I run it.

    I did like the elapsed time feature of the older scripts (now gone).

    Some packages recommend 'apt-get autoremove'. Maybe put this at the end of the script?

    Otherwise - awesome!! Well done Peter & Antonio.

    1. Hi Keith

      Can you clarify - Pi3 - you say Jessie - was this a generic Jessie or are you talking about Raspbian - very important to clarify.

      The qq options are to try to quieten down the "noise" coming off the installs - after all a BOATLOAD of messages telling us what has been successful are actually of no use - it would be nice indeed to figure out a reliable way of ONLY showing error messages - as they often get lost in the "noise". The old elapsed feature - yes I agree with you but that was based on a notion of interaction - incidentally the old script is still there for those who want to use it.

      Apt-Get autoremove - I see no reason why we should NOT do a cleanup at the end - Antonio - your thoughts?

      1. yes, as said on skype, when you install "something", it brings together an other lot of packages... when you remove "something", those addons packages are left behind... to remove these packages not needed anymore (as the packages using them was already removed), you do the "autoremove"... to remove the downloaded (let's say the setup ones, in windows style) and already installed packages, you do the "clean", too

      2. Hi Peter,

        Sorry, yes it was Raspbian straight from the raspberrypi.org website.

        I liked the elapsed time as it gave some idea of performance. You often mentioned how many minutes it took to install on your blog.

        I'm a ardent console watcher, so like to see the 'noise' 🙂 It gives me a warm feeling that things are still moving along as opposed to being stuck.

        1. in me, too 😀

          we added a (no)quiet menu option yesterday, you can use it now... but time depends on your menu selection, so compare is valid only on default selections... and can change from microsd to microsd on the same platform...

  10. Hi Peter,
    Great video & script. I've installed Ubuntu Xenial Desktop as a VM in Virtualbox and followed your instructions. However, the script has stalled at the point below. Tried twice but getting the same result.
    ================= Installing Node-Red ====================
    make: Entering directory '/usr/lib/node_modules/node-red/node_modules/bcrypt/build'
    CXX(target) Release/obj.target/bcrypt_lib/src/blowfish.o
    CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt.o
    CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt_node.o
    SOLINK_MODULE(target) Release/obj.target/bcrypt_lib.node
    COPY Release/bcrypt_lib.node
    make: Leaving directory '/usr/lib/node_modules/node-red/node_modules/bcrypt/build'
    make: Entering directory '/usr/lib/node_modules/node-red/node_modules/ws/node_modules/utf-8-validate/build'
    CXX(target) Release/obj.target/validation/src/validation.o
    SOLINK_MODULE(target) Release/obj.target/validation.node
    COPY Release/validation.node
    make: Leaving directory '/usr/lib/node_modules/node-red/node_modules/ws/node_modules/utf-8-validate/build'
    make: Entering directory '/usr/lib/node_modules/node-red/node_modules/ws/node_modules/bufferutil/build'
    CXX(target) Release/obj.target/bufferutil/src/bufferutil.o
    SOLINK_MODULE(target) Release/obj.target/bufferutil.node
    COPY Release/bufferutil.node
    make: Leaving directory '/usr/lib/node_modules/node-red/node_modules/ws/node_modules/bufferutil/build'

  11. Just tried it on a fresh install of Armbian (Ubuntu 16.04.1 LTS) on my Olimex A20-Olinuxino-Micro, and it seemed to work flawlessly (meaning I didn't notice any major errors, and all the components seem to be functioning), in case anyone has the same platform and is interested in trying it.

  12. I just registered to share a comment about your script. A few days ago I booted the Raspbian "Pixel for PC" from a USB flash drive onto an old Toshiba Satellite laptop (M115-S3094). This afternoon I tried running your script on that machine. It worked! I rebooted and now have node-red running at 127.0.0.1:1880 with your large array of nodes to try out. Webmin is also working. I haven't tried anything else yet. Just wanted to let you know that the script works even with "Pixel for PC". I have a very limited knowledge of Linux so to me this seems quite a trick! Thank you and Happy New Year!

      1. as far as i could see in the few minutes i used it, it has no ssh server... so
        sudo apt-get install ssh # (to install ssh server+client)
        sudo passwd # to give root a password
        then it should work, as always

      2. I struggled with scp on the Pixel OS - always timed out when trying to send a text file to my Windows 10 machine. I was able to setup and use a PuTTY connection. I had to turn on the SSH server (OpenSSH_6.7, already there, shown in Webmin) and allow authentication by password. Finally, I installed WinSCP and was able to drag and drop a text file from the Pixel machine into a directory on the Windows 10 computer.

          1. Thank you for your help. I located the ssh block in the script, then I viewed the sshd_config file and saw the appended lines about UseDNS and ClientAlive... I have learned that Pixel for PC version 1.1 has the ssh server off by default. I don't understand Linux command syntax well enough to know if that would have any effect on the script. I do see the last line in the block restarts the ssh server, but the Webmin interface said the ssh server was off. Thanks again, no response necessary. The problem is most likely operator error. Right now I have a working system and plenty to learn!

            1. use raspi-config: https://www.raspberrypi.org/blog/a-security-update-for-raspbian-pixel/
              or, as per this link https://www.raspberrypi.org/documentation/remote-access/ssh/ :
              "For headless setup, SSH can be enabled by placing a file named 'ssh', without any extension, onto the boot partition of the SD card."

              these are the lines i added/modified to sshd_config:

              PermitRootLogin yes # to allow root login directly... i would not do this, but some wants...
              UseDNS no # to reduce the time between inserting user and the following password prompt
              TCPKeepAlive no # these 3 lines are for disable ssh timeouts
              ClientAliveInterval 30
              ClientAliveCountMax 100

  13. Just completed a rather long script install on a Raspberry Pi 2 having added a timestamp to the beginning of each section - I seem to recall some folk liking to keep track - also added (thanks Antonio) a progress bar. The screen is still WAY too noisy for me but we're working on that.

    The script despite taking ages, worked perfectly - bog standard Raspberry Pi2 with the November Raspbian.

    The FriendlyArm S2 also works but I'm just kitting that out with a new eMMC-based Debian to try again, leaving it to run overnight. I'll do a writeup on this board soon, probably tomorrow.

  14. Thanks Peter,

    Trying to run the script on a Virtualbox running MX-Linux (Jessie) and cannot get pst the npm installation part. Attached image shows the errors. I've tried a couple of times with no luck.

    Any ideas?

    1. Clearly, NPM was not installed. So you need to look back in that installation and see the earlier section to find out why not.

      Interestingly the only issue I'm currently having on VirtualBox is with Apache - in both the PIXEL and DEBIAN 8.6 images, Apache won't load and so of course PHPLITE INFO won't load (I'm sure we'll fix that today and that it is to do with getting the right installation image) - but NPN on VirtualPC - Pixel, Debian and Ununtu are all fine

      If anyone wants to jump in on this and help - by all means.. I'm thinking on the Apache thing we're going to need a menu option so the system knows this is a VM on a PC... but that still leaves the question of which apt-get etc.

        1. I've purged the virtualbox OS and started over. Get the same problem. npm seems not to be insstalling. The text below is what comes prior to the image II posted just above here.

          ====
          ==== Installing Installing Node-Red - Total Time: 00h:10m:02s
          ====
          sudo: npm: command not found
          --2017-01-05 12:04:48-- https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/nodered.service
          Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.20.133
          Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.20.133|:443... connected.
          HTTP request sent, awaiting response... 200 OK
          Length: 888 [text/plain]
          Saving to: ‘/lib/systemd/system/nodered.service’

          /lib/systemd/system/node 100%[===================================>] 888 --.-KB/s in 0s

          2017-01-05 12:04:48 (14.6 MB/s) - ‘/lib/systemd/system/nodered.service’ saved [888/888]

          --2017-01-05 12:04:48-- https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/node-red-start
          Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.20.133
          Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.20.133|:443... connected.
          HTTP request sent, awaiting response... 200 OK
          Length: 2111 (2.1K) [text/plain]
          Saving to: ‘/usr/bin/node-red-start’

          /usr/bin/node-red-start 100%[===================================>] 2.06K --.-KB/s in 0s

          2017-01-05 12:04:49 (9.68 MB/s) - ‘/usr/bin/node-red-start’ saved [2111/2111]

          --2017-01-05 12:04:49-- https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/node-red-stop
          Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.20.133
          Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.20.133|:443... connected.
          HTTP request sent, awaiting response... 200 OK
          Length: 226 [text/plain]
          Saving to: ‘/usr/bin/node-red-stop’

          /usr/bin/node-red-stop 100%[===================================>] 226 --.-KB/s in 0s

          2017-01-05 12:04:49 (2.63 MB/s) - ‘/usr/bin/node-red-stop’ saved [226/226]

          Failed to get D-Bus connection: Unknown error -1

            1. This Debian Jessie, distro is MX-Linux https://mxlinux.org/ . Tried both 32 and 64 bit versions.

              The computer is a Panasonic Toughpad with the same distro as the Host, using Virtualbox.

              I'm willing to test!

              Let me download the Ubuntu. Never been a fan of Ubuntu, but I will give it a go.

              1. please, while reporting bugs, EXPLICIT what script version you are using (and try to use ALWAYS the latest...), and which distro... if it's not 1 of those I listed, JUST SAY IT, we have no crystal balls to guess, nor we can make script work on EVERY single version of distros out there...

                1. Repaired this part of the problem, but editing the /etc/ls-release file. Changed the Metamorphosis to jessie, that way nodesetup recognizes the distro and proceeds.

                  Still can get past the installation of the nodes tho. Appears to a systemd problem.

                  Fun playing.

  15. apache+php7 install on standard debian+pixel sorted out... they report 32bit arch as i686... patch sent to Pete, which is trying pixel now, i tried debian, everything fine now...

    npm, still don't know, as it's not affected by php part of the script...

  16. I just finished running your script in Virtualbox 5.1.12 and everything appears to have worked flawlessly. I have used Virtualbox many times over several years for various reasons so that part was easy. The script itself took less that 15 minutes but I did not time it to get an exact time (Win 7 i5 2500K host). My "real" server is running Ubuntu 14.04 desktop and I'm looking to upgrade it to Xenial. I have been using MySQL vs SQLlite but your script has everything else I use plus more.

    Thanks for a great way to test things before I try upgrading my real server.

    John

    1. Hi john

      Well, the purpose of using SQLITE is to keep writing down to a minimum for SD memory on little SBC boards such as the pi - for simple logging, MYSQL though it would be my preferred DB is well over the top. Still - no doubt you can install that separately on your VM - and if you look at the script itself you'll see it isn't that hard to add sections - just be aware if you add anything to the menu - you need to change 2 values at the top of the menu to make it larger (because it isn't intelligent enough to autosize).

      1. Just getting back to the virtualbox installation of the script on Xenial and I can't seem to find Bigtimer in node-red. It is in .node-red/node_modules and I was able to update it to 1.4.88 using npm. However it still does not appear as an available node on the node-red web page. esplogin, grove and timeout appear under the scargill heading but no bigtimer. I don't understand enough about node-red to diagnose such issues. Can anyone point me in the right direction?

  17. PLEASE NOTE ALL - a BUG crept into NODE-RED-CONTRIB-BIGTIMER - I have updated this and fixed this bug which would have stopped bigtimer running.

    To fix on existing setups, go to your .node-red directory (probably /home/pi/.node-red
    and as PI user, from there merely...

    npm install node-red-contrib-bigtimer

    Update already but please wait until 9am today (Friday 6/1/2017) or later. The version you should end up with is 1.4.88 or later.

  18. Has anyone successfully run the script on a Pi 1 loaded with DietPi. I'm having no luck.
    "npm command not found"
    And yes I have a PI3 running well, but I have a couple of early PIs which I would like to give to my son.

  19. Hello

    Thanks for your work.

    Two comments :

    For real Raspberry, I would recommend to add to your script the log2ram tool. This is helping to save SD card and avoid corruption (logs are stored in RAM and dump to SD only once an hour per default). -> https://github.com/azlux/log2ram

    For your Debian stuff, you should have a look to the new X86 Raspberry/Pixel official image (if you've not had the info yet). This is made by raspberry org themself to run on PC or MAC and this should be pretty easy to have it on VM with persistence. This will be as close as possible to the regular raspbian ... -> https://www.raspberrypi.org/blog/pixel-pc-mac/

    Happy new year.

    Sebastien

    1. Hi there Sebastien

      Thanks for that and happy new year. Raspberry Pi - do you want to work up the exact instructions to add Logram and put in here why it's a great idea - if it is reliable we can always addt it to the Raspberry Pi menu!!! I'm all for offering to ditch logs actually - ie run in RAM but don't save. I only ever seem to use logs to check why something went wrong - not really interested weeks later and it would lengthen life of SD ??

      Debian - I have PIXEL running in VIRTUALBOX - works a treat - and the script works perfectly on it - it's a good quick way to get Debian and of course includes SUDO which Debian often doesn't . Don't get me wrong however, it is NOT a Raspberry Pi emulator... sound controls are missing - there is no RASPI-CONFIG and no hardware - like ports or anything... Shame - had they thought it through - a little usb board with ports on, made to LOOK like you have ports - would have been nice. Still - it is free and it works. Shame they're not ready to make it work on other boards because some of the alternatives to Pi, while having great hardware have really rubbish software.

      1. pixel raspbian was not meant for virtual machine (so the extra steps needed to have it working in a vm - why not doing as ubuntu, an icon on the desktop of the live cd, INSTALL, done???), and having it on actual physical hardware can bring audio and other stuff working... for ports, you can just passthrough them to the hypervisor of choice, not always easy but doable... clearly no gpio or other hw, only boards you can already have working via usb, like an arduino via firmata, or other boards Pete already tested in his blog entries in the past...

    2. frankly, the only advantage raspbian has is that's official... i don't care about the gui, i use ssh into my system every time, so pixel is useless for me... and, to be honest, it seems windows 95 with its icons... better than the former ones, but quite oldstyle anyway... but everybody says LOOK'MA, DAS ICONS! (there are TONS of better looking icon themes out there, from DECADES...)

      oh, and pixel has no clear instructions to put it in a vm, you have to dig into comments in official blog page to know how to do...

      raspbian has nodered and other stuff in it, but usually is old itself, too...

      ubuntu has newer packages already in its standard repositories, while dietpi has the best starting point for SBC, with low memory, low space needed, logs in ram, and many other optimizations...

      once setup, are all the same, you don't bother about what's underneath, your interest is in the service you use above the OS... which are all the same...

      dietpi is sure my distro of choice, right now...

      if nodered is your sole purpose, http://thethingbox.io is a good choice, too...

    3. Sebastien, this looks interesting.

      Along these lines I've been using CIFS to access disk space on my network shares (CIFS was easier to explain to friends than NFS or AFS). For when I must have local storage I dropped on a cheap 120G SSD drive and cross link it to the drive. I need to sit down and figure out how to mount the SSD as the writable part and the SD as just readable.

  20. Hello Peter

    I have to admit that I am stumped. I have been using Mosquitto for about 6 months but recently I had a crash, lost my installation on my Pi 3, and reinstalled all. Now I cannot get Mosquitto to work. I get this message when I enter this:

    sudo mosquitto_sub -d -t Humidity

    the error I get is:

    Connection Refused: not authorised.

    I used the script to install Mosquitto and Node-Red. Can you point me in the right direction please? I have searched the web but have not made any progress for the past several days. As I said, prior to the crash I had collected 100's of megabytes of data and controlled lighting in my home. Now, I cannot get Mosquitto to work at the simplest level.

    Regards,
    Bill Miller

  21. just to report, the script worked on Orange pi zero with Armbian_5.24_Orangepizero_Ubuntu_xenial_default_3.4.113.7z
    reported execution time 46 minutes.
    Thankyou 🙂

    1. Let me know how your WIFI works - particularly recovery from access point failure (just unplug) and general reliability... that is my one reservation about the Zero and I had no luck at access point failure recovery using Armbian (fine on hardwired ethernet).

  22. Ping times are awful (anywhere from 3ms to 2000ms; mostly 200-300ms).

    Wifi comes back up on restarting my bthhv2b. But, my wireless router is NOT my dhcp server, so it may not be an equivalent test..... Saw your topic on the armbian forums...

    ahhh.. just looked at logs. It did not reconnect to the SAME router (I have 2, and it knows about both...), so not a suitable test.... Interesting that it's happy to migrate though.

  23. looking through available source & firmware, looks like it's going to be wired only for me for the moment; no time to port wifi drivers this week.....

  24. Maybe this is not your field, but I will try this question to you.

    I reinstalled my system on a pi3 a few days ago when an npm update miserably failed and I lost all my nodes. As I am a Linux illiterate this was the only thing I could do.
    Now I found SD Card Copier and made a backup SD card with success, I swapped the written card with the original. As I did not like to do this job by hand every time I searched for another method and found this: https://github.com/aweijnitz/pi_backup.

    As I have an old 2,5 inch laptop hard drive I wanted to use for automatic backupping. Then the problems started as I found out I have a read-only file system and this HD cannot be written to with the above mentioned script. Really nothing goes!

    I suspect this behaviour could have been setup by the Christmas script, but that is strange. I know you are not a linux specialist yourself I cannot understand why this is the normal behaviour of the pi3.

    I have read some articles about corrupt SD cards caused by power outages. I now think I know why you are so keen on having this backup board with a lipo on it. My experience until now is that during all my resetting the pi3 by flipping the power switch myself when I thought things were screwed up I never had a SD card failure. You also have this experience in your house in Spain, where the power is not that reliable too. I found this article https://hallard.me/raspberry-pi-read-only/ that tells it is a good thing to have a read-only file system, but I will not fiddle with my system as described there.

    On the side of all this I noticed a thing about the pi user being a member of the sudo group. In the blog is is written you have to be sure pi is a member of it. What I see when opening the file manager then pi is listed as a member of group pi in the permissions box of the file manager. I have no idea if this is wrong and will have influence on the problems I experience!

    Can any one guide me out of this situation and explain how to be able to use the backup script on an external drive?

    1. So for other people having same problems:
      I installed gnome-schedule, gparted and I could work without learning Linux deeply. I edited the backup script to my needs and now I have a working backup automatically going every night. I formatted the external disk to NTFS so I could detach it from my "dead" RPI just in case and I can write a new image from the latest backup with Win32 DiskImager on my PC to the "dead" SD-card. Occasionally I make a copy with SD Card Copier, as extra safety. All with very little attention needed.
      Why not integrate this in the startup script to make live easier for Linux illiterates.

  25. Just a suggestion,

    How about adding the following to the install list?

    jquery, vnc (remote viewing) and the ftp node.

    Thanks for the good work!!

      1. What I meant by JQUERY was... if you're doing anything with Node-Red Dashboard - like using a template - you have full access to jQuery in there without having to insert a link

  26. Yes, to the Node-Red ftp node (node-red-contrib-ftp). I coundn't find the directory where jquery was located, but I will look again. Will do the same using a wildcard to see which version of the vnc server might already be installed with Raspbian. Might just need to do a configure for automatic startup at boot.

    Thanks.

  27. Ok, just got a couple of new RPi3's and downloaded the latest Rasbian and wanted to install the system "headless". After using the trick to have SSH enabled at boot up (which by the way don't work for Ubuntu even if that is the directions) 🙁 Then after following you direction plus a few more to get a working password for root and pi added to the sudo user group I loaded you script to /home/pi then chmod to 744. When I tried to run it with just ./script.sh I got a (bash: ./script.sh: /bin/bash^M: bad interpreter: No such file or directory) error. When I did it (bash ./script.sh) it just bailed out back to command prompt. No menu, nothing. Can someone tell me what I am missing??? Thanks

    1. Sounds to me like the file is saved in PC format.... needs to be saved in Linux format - that is - with only Linefeeds (I think that's right) and not both CRLF. I've never understood the message when that is wrong.

      1. the first line (called SHEBANG or HASHBANG, and starting with #! ) in a unix script tells the shell which interpreter to use to execute the following lines... the error is just saying that there's no interpreter called:
        /bin/bash^M
        as it's actually called
        /bin/bash

        the ^M is exactly what Peter said, the carriage return char, which should be stripped out...
        you can do it in linux itself:

        sudo apt-get install -y dos2unix
        dos2unix script.sh

        (the same package adds the unix2dos conversion tool, too, just in case...)

        info: https://en.wikipedia.org/wiki/Shebang_(Unix)

      2. Yes, that's what I had done.. I read your note but choose the wrong EOL. I saved it as Mac. It is running now, wo hoo!! Now to see what else I can foul up.

        Thanks.

        1. This new Microsoft Visual Code editor - whereas in Notepad++ it is as easy as selecting EDIT then the format - I'm not finding a similar menu item in the Microsoft editor - which is a bit of a pain - it would be nice if the editors would flash up somewhere what format you're using...

          1. found... honestly, this is done way better and easier in n++... anyway:
            open vs code
            ctrl-p
            ext install beautify
            on the left, install the one by hookyqr (has this icon: https://marketplace.visualstudio.com/items?itemName=HookyQR.beautify )
            reload vs code

            then open a file, and press F1 (or ctrl-p, then > )
            write: eol (and press enter)
            choose the desired eol...

            the extension itself does MUCH more than this, try to write beutify on a js, or css, or html file...

    1. Oh hell - is it THAT easy.... will you look at that - all those other things on the bottom do things as well.... I'm liking this more by the minute.

      Thanks for the feedback on this guys.Oh - and an update, too.

  28. Hi Pete,
    I used your script in my newest video and wanted to thank you for this great job. From the comments, my viewers like it, too!
    One question: Do you have any plans to include InfoxDB and Grafana in your script? It looks like these two are quite famous (and I was not able to get Grafana running)...

    1. Hi Andreas

      Thank you for the kind words... Grafana I'm more than happy with - InfluxDB - not so... I really would like to see someone do a SQLITE interface... I've had a couple of days of disconnects (power cuts in Spain) and some time ago a glitch -and there are values not necessarily out of range but wrong - InFlux simply won't let you do something as simple as pick a record or records and delete them. So if you don't mind I'm going to hang back on that a little in case a better, equally simple solution comes along - and of course - whatever it is has to work on everything from a Pi Zero, through various boards and phones to VMS - as does the rest of the script.

      1. Hi Pete
        I have some trouble with the credentials in node-red dashboard. Admin admin does not work for me...
        Can you guide me to looking up the user and change the password without re-installing everything?

          1. Thanks for your fast answer. Unfortunately, I am a complete Linux beginner. How do I "run" this script? I tried by just pasting it into the terminal and creating a .sh file. Both, unfortunately were not successful...

            1. Don't forget when running a script - it has to be in Linux format - that is line feeds only - you can ensure that in NotePad++ under EDIT - EOL CONVERSION - LINUX - and the script once created (it IS just a text file) needs EXECUTE permissions.

              If you get the format wrong - the resulting error message is not helpful.

  29. I changed to the pi home director and started the two commands (Wget and bash). Unfortunately, I always get the error message "line 5: $'\r': command not found
    ".
    Then, I created a file using sudo nano and copy-paste raw. Same thing...

  30. I was not able to install dos2unix with your command (it did not accept the -y). So, I went the way via notepad++. Now, I get no error message, but the script does not return anymore... (it also does not react to ^C. I have to logout...
    This seems to be really hard... (for me, at least)

    1. Andreas - as the script clearly works - would it not make life easier here to simply reload the operating system image and run again. It doesn't take any interaction once it starts as you know.

      1. You might be right... Did not think, it will be so complicated. But I always learn. So, I will start over. Thanks for your support!

  31. The script was looping endlessly, because of a lacking settings.js in my directory. Now, I am one step further. I get an error message at node-red start. But I have to run...

  32. The second problem was, that, everytime the script runs, it adds a section, which is already there, if node-red was installed by your script...
    Now, it works (I followed the advice of Peter).

    httpStatic: '/home/pi/.node-red/public',
    functionGlobalContext: {
    os:require('os'),
    moment:require('moment'),
    fs:require('fs')
    },

    adminAuth: {
    type: "credentials",
    users: [{
    username: "admin",
    password: "$2a$08$o9n62Pq6s5YVfD2C.inYKucfIOp26eYart4s0XeUB3Cbf7h.GIsrC",
    permissions: "*"
    }]
    },

    httpNodeAuth: {user:"user", pass:"$2a$08$z2m9DthU3ZrfFtDx7ceD..HlFOuJvdX2Plmdkpvg//YNfCYbJ2BpW"}
    }

  33. You are right. I did it then, but ran into my first problem (infinite loop) and forgot about your comment. Sorry about that.
    In the meantime, I found out, why the loop was infinite: If node-red still runs when I start your script it does not create a "virgin" settings.js. So, I added the line:

    sudo service nodered stop;
    at the begin of the script and now, it runs perfectly. Thank you very much for your patience!

  34. WOW! This is great!
    Saved me a lot of work!
    Thank you so much!

    And - don't try this on orange pi official rasbian distro - that is total crap and I wasted 3 days trying to install everything on it. On Ambrian it did everything for less than an hour.

  35. I ran The Script last night on a NanoPi M1+ and it took longer than I could wait so I left it overnight. In the morning it had stopped from a connection timeout at the "change hostname" dialog box.

    Maybe a better option would be to move all user input requirements to the top of the script so it can run to completion unattended after a few user input items at the beginning?

      1. I reran it and sure enough, it's waiting for the hostname and finished. I could have looked at the script but wanted to see what it would do re-running it. 🙂

        Wish List: move hostname prompt to the top and add a menu item to reboot when complete.

          1. oh, the nodejs install from xz package, is less noisy too, but we tested only in vm, so you didn't realize that... needs testing on a arm64 board (you know which 🙂 ), and/or a rpi0 or rpi0w... i added "pv" to have it show a nice progress bar

      2. mmm, we can do that... ask at beginning, and leave the current ip and hostname at the end, where they are... 🙂

        needs testing, as changing hostname while script running already brings some little delays at the end and some warnings, which i suppressed as we do a reboot soon after, and after reboot all is fine...

  36. I ran the script on NanoPi M1, left it over night and hostname dialog or connection timed out. I reran the script and came back in 1 hour and changed the hostname and rebooted. Logged in via ssh as pi user. Browsed to device http://hostname.local and saw the "Pete's Utilities" page. WebMin worked, Node Red didn't, OPI Mon worked, SQLite Admin and Sys Info worked too. Only Node Red didn't work because of the 2nd run of the script and double entries in settings.json.

    Note Red installation had multiple entries for admin and user Auth's so once those were cleaned up in pi/.node-red/settings.js and rebooted it worked.

    In The Script, line #6 of "IMPORTANT" states to not access(run?) the script as SUDO. But my understanding is that on systems without a pi user it has to be run first with sudo and then run as pi user.

    1. The hostname dialog has now been moved to the front of the script.

      If the Node-Red installation has mutliple entied for admin - the script or part of it has been run twice by some means or another.

      If the system has no Pi user, running it the first time as ROOT will create the Pi user and will now move the script - with permissions modified, to the Pi directory - you then log in as Pi (password password unless you change it which is a good idea) and you run again and that completes the operation.

    2. in the version just released of the script, if you try to rerun the node-red setup it will skip the step and go on, preventing double lines... if you want to rerun, at lease rename settings.js, but best rename or move elsewhere the .node-red folder

  37. I am having problems with running the script on the latest release of raspbian jessie lite,
    The problem starts when installing Node-Red,

    --2017-04-21 05:00:23-- https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/nodered.service
    Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 62.252.172.241
    Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|62.252.172.241|:443... connected.
    GnuTLS: A TLS fatal alert has been received.
    GnuTLS: received alert [0]: Close notify
    Unable to establish SSL connection.

    I have done a few google searches but cant seem to find any information on this.
    Any help would be appreciated.

    1. Hi Craig

      The script has not been tested on Raspbian Lite, just on the full Raspbian - and amongst the things you can do is remove office and that maths program which collectively take up a lot of room.

      1. no problem on jessie lite, i used that, i don't like wasting space for a gui (plus the tons of gui programs) i will never run... their base is the same, only difference is the presence of the Pixel gui in the full version...

        the problem of craig is the same of that guy we solved in chat a while ago: some of the github round-robin served servers have wrong ssl config... solution: add this line to /etc/hosts and retry:
        151.101.128.133 raw.githubusercontent.com

      2. solved, to put in script... it first tests if the actual github server returns a good certificate, in case does nothing, while in case of a wrong one, it adds a good host/ip association to /etc/hosts... Peter, add these 2 lines before the one at line 754, but after the ELSE on 753

        echo | openssl s_client -showcerts -servername raw.githubusercontent.com -connect raw.githubusercontent.com:443 2>/dev/null | openssl x509 -inform pem -noout -text > /dev/null 2>&1
        [[ $? -ne 0 ]] && echo 151.101.128.133 raw.githubusercontent.com | sudo tee -a /etc/hosts

      1. I've just tried with the recently updated script and still get the same error.
        The first response to this issue i tried and that worked O.K.

        I now have my Pi up and running but if you need me to test any more changes to the script regarding this issue just let me know.

        1. i tried those lines, now in script, and here worked well... did you try before or after that mod i suggested? Which does nothing but the same thing, as 1st line in script now tests if ssl certificate is correct, and 2nd one adds the line you added manually in case 1st line returns an error... so, it should have worked...

          can you remove that line from your /etc/hosts file, and run these 2 lines and report if they say OK or KO? or better, run these 2 lines first WITH and THEN without that line in hosts, and report both results? Thanks

          echo | openssl s_client -showcerts -servername raw.githubusercontent.com -connect raw.githubusercontent.com:443 2>/dev/null | openssl x509 -inform pem -noout -text > /dev/null 2>&1

          [[ $? -eq 0 ]] && echo OK || echo KO

  38. Hi Pete/all,

    just working with my Orangepizero, restoring it after an SD card corruption. Whilst looking for regular writes, I found that WebMin has folders on the SD card which it both logs to and uses for temporary files for serving up info.
    'The script' installs log2ram? This effectively makes '/var/log' a ram disk, but persists it at startup and shutdown, and once per hour?

    As a test, in '/etc/webmin/miniserv.conf' I'm changing
    logfile=/var/webmin/miniserv.log
    errorlog=/var/webmin/miniserv.error
    pidfile=/var/webmin/miniserv.pid
    env_WEBMIN_VAR=/var/webmin

    to
    logfile=/var/log/webmin/miniserv.log
    errorlog=/var/log/webmin/miniserv.error
    pidfile=/var/log/webmin/miniserv.pid
    env_WEBMIN_VAR=/var/log/webmin

    and
    sudo cp -r /var/webmin /var/log/
    and
    sudo nano /etc/webmin/var-path
    (change the path in there too).
    sudo service webmin stop
    then
    sudo service webmin start

    and now webmin is again running, but with no regular SD card writes (except once per hour when log2ram will store everything in var/log).

    webmin probably does a LOT of individual small writes, which could do quite some damage to the SD card over time (e.g. the sessions db is in there....).
    This may be a useful addition to the way the script installs webmin for those of us with fragile media?

    Simon

    p.s. My home-auto node-red flows also persist data to disk (graphs, etc.), so I've moved them to a folder within the /var/logs folder; far better a once-per-hour write than a once per minute - and no need to design SD protection into the flow... but you do risk up to an hours' worth of data if you have an unexpected failure. I decided better lose an hour of data than 4 hours of my time restoring the whole setup with a risk of wife-rage because the hot water did not turn on!
    mosquitto is still writing to SD occasionally.
    I'm using 'sudo iotop -a' to observe disk writes - just leave it on for a while, and offending processes float to the top.

    1. ohhh.. reading above, I don't know of log2ram was a script install, or if it was part of the armbian OS image I started with. If it never got into the script, I highly recommend it :).

    2. after a reboot, webmin did not start; seems log2ram did NOT copy /var/log/webmin/ to /var/log.hdd/ on a sudo reboot.
      Since I believe the folder is only temporary files effectively, I've done a manual copy, then the reboot works. Also from brief reading, log2ram should copy it....

        1. i remember there were references to apache, which should be explicitely added as to start AFTER log2ram start, otherwise it would fail... maybe it's the same for webmin... but on that github there are no more references to apache...

        2. Note: armbian has
          https://github.com/armbian/build/tree/master/scripts/log2ram
          ('updated' - mainly seems to be about config and script location)
          log2ram script appears to be disabled at first look, but in armbian version, actually loads config from /etc/default/log2ram which overrides the enabled=false in the script.
          Mine appears to be set for 50M, of which 28% is in use (including webmin folder + 7M of my own data).
          Rsync was OFF; I just turned it on and it seems to be doing 'reasonable' things.
          also found:
          https://github.com/Fourdee/DietPi/issues/781
          which may be relevant to some rpi users.
          Maybe if we're to use (as I am) /var/log as ram based perm store for NR, we should add the NR service to the 'before' line too....
          (my apache2 was failing to start - but because /var/log/apache2 did not exist).

  39. Nodejs in the /dist/latest-v6.x/ folder is now 10.2 instead of 10.1, which makes the installation fail, then nodered fails too.

    Please note there is also nodejs 7.

      1. Fully agree on not using 7 or 10.2 for that matter. It is just that script now hard fails the nodejs installation. So my suggestion is to either change line 712 (and 711) to use 10.2 instead of 10.1, or use /v6.10.1/ instead of /latest-v6.x/

        Should have been clearer about that.

      2. just checked, and he's right, nodejs 6.10.1 was deleted from node servers and now there's only the newer 6.10.2... every mention to
        node-v6.10.1
        should be changed to
        node-v6.10.2
        in script, now...

        he's NOT right about nodejs7, as it's not STABLE and we use only stable versions...

  40. Is there any reason why the script doesn't install fail2ban to deter hackers?
    And a related question, why do you choose to enable the root login on a Raspberry Pi?

    I do occasionally see strangers attempting to login as pi but nowhere near the number who try as root.

    1. because in general we use this setup locally... i don't enable root on my systems, but others prefer using it, maybe just "habits"... i use vpn to connect to my services from outside... anyway, this is just an easy setup of a bunch of services, suited for most but not everybody, and whomever can modify their setup and make them suited to their needs... feel free to close everything, disable ssh root login, add ssh keypair login instead of user and pass, add fail2ban, etc

    2. The script does not do a lot of things. I've never heard of fail2ban and I don't add anything I'm not familiar with and feel comfortable with. Root? Because here are some things that are a pain in the backside to deal with, without Root. Generally speaking I run everything as user Pi.

      1. Fail2ban:
        If you open SSH to the internet on the default port, you very soon attract login attempts, lots of them.
        grep "Failed password" /var/log/auth.log to see how popular this is with the hackers.
        All fail2ban does is ban an IP address for 10 minutes after 6 attempts. Of course it's configurable, that's the default.
        (On the Pi) sudo apt-get install and it starts working, no setup needed.

        I'd also suggest that logins and passwords for node red, webmin etc should be defined at the top of the script rather than buried in the code.
        Best regards,

          1. Passwords deined at the beginning...
            Lines 357 - 363 in the script:

            username="user"
            userpass="password123"

            adminname="admin"
            adminpass="password123"

            newhostname="myboard"

            Only useful if you...
            Yes that's true. It's just a suggestion for a simple enhancement.

            1. question isn't where password are (they're at qute the beginning of the MAIN SECTION OF THE SCRIPT, the first part are general functions never touched and that a while ago were in a separate file, now included just for easier management), question is if you ever run the script or are questioning just for the sake of it... if you ever run it, you should know that there's nothing you need to edit yourself, and that running it there's an option in menu that uses THOSE defaults and asks for new ones, JUST AT THE BEGINNING OF THE INSTALL process... after those first seconds, you can go to take a coffee or what else, and return after about 15-30 minutes, depending from your board speed... and now, enough...

              1. I was asking because i have on occasion recommended your script on the Raspberry Pi forums and I wondered why you did/did not do a few things that slightly worry me - including enabling root login and not installing fail2ban. But please don't think I'm criticising your work, just asking questions.

                Yes I do run it sometimes. I have never previously done so without modifications to suit my preferences.

                However, as an experiment I just tried running the script unedited on a headless Raspberry Pi 3 with a fresh copy of Raspbian 2017-04-10 to which i added /boot/ssh and /boot/wpa_supplicant.conf.
                It fell over at this point:
                ++ whiptail --title 'Main Raspberry Pi Selection' --checklist '\nSelect items for your Pi as required then hit OK' 30 73 23 quiet 'Quiet(er) install - untick for lots of info ' ON prereq 'Install general pre-requisites ' ON phone 'Install on Android Smartphone - see blog' OFF mosquitto 'Install Mosquitto' ON apache 'Install Apache/PHP/SQLITE + PHPLITEADMIN ' ON nodejs 'Install NodeJS' ON nodered 'Install Node-Red' ON webmin 'Install Webmin' ON screen 'Install Screen' ON java 'Update Java' ON wiringpi 'Wiring Pi for the GPIO utility' OFF mpg123 'Install MPG123' ON modpass 'Mod USER and ADMIN passwords (password123)' ON phpsysinfo 'Install PHPSYSYINFO' ON upgradenpm 'Upgrade NPM to latest version ' ON addindex 'Add an index page and some CSS' ON passwords 'Update ROOT and PI user passwords' OFF installcu 'Install CU for serial VT100 Terminal' ON installmc 'Install MC+MCEDIT file manager + editor ' ON installjed 'Install JED file editor' OFF habridge 'Install HA-bridge on port 82' OFF log2ram 'Install Log2RAM default 40Meg' OFF ' '
                Box options:
                --msgbox
                --yesno
                --infobox
                --inputbox [init]
                --passwordbox [init]
                --textbox
                --menu [tag item] ...
                --checklist [tag item status]...
                --radiolist [tag item status]...
                --gauge
                Options: (depend on box-option)
                --clear clear screen on exit
                --defaultno default no button
                --default-item set default string
                --fb, --fullbuttons use full buttons
                --nocancel no cancel button
                --yes-button set text of yes button
                --no-button set text of no button
                --ok-button set text of ok button
                --cancel-button set text of cancel button
                --noitem don't display items
                --notags don't display tags
                --separate-output output one line at a time
                --output-fd output to fd, not stdout
                --title display title
                --backtitle display backtitle
                --scrolltext force vertical scrollbars
                --topleft put window in top-left corner
                -h, --help print this message
                -v, --version print version information

                ++ wolfram 'Remove Wolfram on a PI to save space' OFF office 'Remove LibreOffice on PI to save space' OFF
                Wolfram Language 11.0.1 Engine for Linux ARM (32-bit)
                Copyright 1988-2017 Wolfram Research, Inc.

                In[1]:=

                (Yes, I did edit the script and insert set -x to obtain the above)

                Not complaining, just letting you know.

                1. Thank you - I am going away for a few days so can't progress this now but I just had someone use the script successfully barring one item which we fixed (nodeJS) today. I have root because I want it - it is not necessary for this script... and I've no interest right now in fail2ban - others can add these things if they want along with anything else. The script will work with a basic Raspbian - I cannot really comment on it's operation outside of that basic scenario or if it is modified.

  41. Ref 'The Script' & mosquitto:
    I think mosquitto needs to have /var/log/mosquitto created and 'chown'ed to mosquitto:mosquitto in order to log...
    In my quest for minimum writes, I set it's persistance path there, and then wondered 'why no logs? why no db?'. only when I created the folder and 'sudo chown mosquitto:mosquitto /var/log/mosquitto' did a restart cause logs + db to be present.
    Can someone check on another system to see if you have mosquitto logs, and if not, we may want to add this to 'The Script'?

    1. must have been something about the state the Armbian OS was in when I installed - maybe *my* (Armbian pre-installed) log2ram did/does not correctly copy to log.hdd on shutdown, and after The Script install, I rebooted before the hourly copy..... mosquitto always worked, it just did not log.... Feels like the same non-copy bug was responsible for Apache not running too. Something to highlight in the main post maybe.
      'In at least one installation where log2ram is pre-installed (e.g. Armbian), it has been observed that folders created in /var/log/ by the installations may not get copied to the /var/log.hdd/ folder at reboot, resulting in services failing or misbehaving. This can be avoided by a manual copy after installation or by waiting an hour and checking /var/log.hdd/ matches /var/log/'.
      Or we could add a 'sudo log2ram write' to the end of the script, which will (quietly?) fail if log2ram is not present, but solve the issue if it is?
      It may just have been some one-off.....

      1. Open an issue on the log2ram github if you have problems (and the last version of course).
        Check you starting log, you will have error from systemd for the log2ram service. And there are the file /var/log.hdd/log2ram.log file to watch too.

  42. Hi Pete!

    Until now, I have used only parts from your script. But now I tried to run it complete, and there was no menu displayed.
    The buggy line was
    "log2ram" "Install Log2RAM default 40Meg" OFF \
    Behind the \ there was a TAB and then LF. I had to remove the tab to get it to work.

    1. It is highly likely your diagnosis is correct as this was a brand new addition to the script. I have loaded up the script -ensured that it is a space before the backslash and saved - so hopefully others will not come across the problem.

      Thanks for bringing this to my attention!!!

  43. Hi Pete,
    As I found out that "G P" is right:

    "The buggy line was "log2ram":

    "Install Log2RAM default 40Meg" OFF \ "
    Behind the \ there was a TAB and then LF.

    I also removed the tab to get it to work.

    Thx.
    Nadav

  44. Hi Pete,

    Love your work.
    The script worked like a charm.
    It took over 10 hours to install on a Pi Zero...

    I started on a similar project a couple of years ago with nodejs,
    never got very far with it because of time issues.
    Recently i wanted to start it up again and then found node-red.
    Love it.. has a huge potential. Now i'll start over with node-red.
    Also like what you're doing with the esp's too.
    Did a small test project with them, and to keep it small i use
    the 128x64 i2c oled display's for status and debuging. I use them on almost anything, they' re so cheap these days.

    I have question about your system:

    Why do you use imperiHome on your phone?
    You could do al that with the dasboard in node-red.

    keep it going,
    may thanks,

    Kiki

    1. Hi there Kiki - thanks for the kind words. Re: Imperihome - I think it depends on what order you read the blog - I am indeed now favouring the Dashboard over Imperihome. However..... there is one little niggle with Dashboard that as far as I know has never been resolved... if you only use Dashboard then the inputs and outputs stay in sync with your phone but imagine you have a timer running in the background - changing the state of things - it is sometimes necessary to refresh the display to make sure it is up to date. In other words I push changes regularly when the phone which is running Dashboard is operating.... and therein lies the issue - the password system - and I've mentioned this in the forums as have others... there is something not quite right about the security/password setup in that it regularly asks for the username and password when you go to the dashboard (only on Android phones, never on my Windows 10 PC) - and that's a pain but not TOO bad.. BUT if you have any kind of refresh going as I do - you get half way through entering the password and the screen refreshes - indeed to use the dashboard reliably I've had to stop refreshing the screen as such.... the thing is - it should not be asking for username and password once you've stored it - but it does from time to time. That is currently unresolved... But generally speaking yes I'm finding it better to work with than Imperihome - which STILL is missing some generic controls - they seem intent more on supporting every device on the planet than providing generic controls that we can use in Node-Red - hence you'll see I spent some time working with templates in Dashboard and various gauges etc..

    2. Pi Zero - yes, very nice but WAY too slow and the script only serves to remind us how slow it can be. I must say - even with the outstanding issues (ease of use of GPIO) I'm favouring the FriendlyArm boards such as NEO etc... slightly more expensive but WAY more powerful and they are a helpful lot.

  45. My pi3 nodered just suddenly died. It worked for months and now I cannot start node-red anymore, all kind of error messages with no clue what is wrong. I did an upgrade of Raspbian and it only gave worse error messages with Syntax error!
    Is it possible to reinstall node-red and then run part of the Script to reinstall all extra nodes or... is a complete reinstall necessary?
    I am now trying to reinstall from backup image from about 5 days back, but that is a separate excercise.

    1. i think you can still recover your flows from the .node-red folder
      about reinstalling, you can just rename .node-red folder and redo just all the npm part, without actually reinstalling node-red itself

      1. I have done a new install ran the Script. I backed up the flows files in .nodered directory. I copied them back to their original location and nodered does not "see" them, stopped and started nodered. Any idea how I can activate my flows? There is a function import, but I only see a map example and cannot freely choose the backed up file!

  46. Now I only have to copy back the database with my gathered data into SQLite, I have a file in the same directory as where the iot.db is. How to import that file into SQLite?

  47. Thanks for your swift answer, but I do not understand the process you suggest. I have a filled otherthaniot.db file in the same directory. I do not have a database in SQLite yet. I like to have a database with the same name as the file, restoring the old situation.

    1. My database can receive new data, but phpliteadmin is not showing the records in the database! When I create the database in phpliteadmin with the name of the file in ~/dbs then I get an unknown error. That can be expected, but how to synchronize phpliteadmin to the file in ~/dbs??? B.t.w new records are created in the database from my restored flow, I see that on my dashboard graphics! So SQLite is functioning.

          1. All kind of wrong permissions needed to change. Now database is accessible again! Other problems coming up like MQTT broker does not start anymore after reboot????????????
            How to start the broker and how to set that it starts automatically at reboot?

            1. honestly... you're having too much problems with an evidently corrupted installation... save what you need and start from scratch... easier... or use a different sdcard in case forgetting something back...

            2. I agree with Antonio - this is sounding like a corrupt installation - maybe SD issues, maybe power - but I would not be continuing - you'll spend more time trying to fix than you would starting from scratch - check power - check that the SD card is a good one.

  48. Thanks Peter, This is good information for beginner like me.
    I just run the script from fresh install dietpi to my old raspberry Pi.

    It seems Nodejs should be change to the latest node-v6.10.3

  49. Thanks for this great script Peter!
    I just installed an Orange Pi PC+ with it and it worked like a charm!

    1. Excellent - enjoy - well it's not just me - I've had lots of help from Antonio - and originally when I was REALLY rubbish at scripting, my friend Aidan Ruff was involved and we've had input and feedback from all sorts of people. I think I probably hold the record for testing hours 🙂

  50. I am again having a PI3-system up and running. As i got the feeling I 'll have to make a better supply,instgnated by your remarks and your newest blog entry, I gathered some components together and made my own UPS. I already had a lipo charger, a 1500mAh 3.7V, a buck converter up to 5V with those little V-meters on the lipo and the output. That whole thing was triggered by the misery I contracted for some reason and a bad supply could have been the reason. I only miss the elegant switching off when the battery goes really low by a power outage that stays out too long. I only have to really test how long it will last without mains, but this stabilizes the voltage much better. I found out the difference between the voltage a the micro USB differs about 0.2 V compared to the pin header value. I can reduce that when I connect my little 2,5 inch HDD on the USB which has extra power from another charger externally.
    I estimate this kind of supply is not a gadget for unstable mains, but also when using a standard charger a necessity.
    When I was running the script I have seen quite a number of warnings. Those might be ignored, but I am wondering if there is a log of all those messages running on the screen or did I miss a check somewhere in the starting menu?
    I did something wrong when asked for the HOST name. There another name is suggested "myboard" and it says something like if you do not want a change just press OK. However the HOST name is changed to myboard from raspberrypi. I once renamed my HOST also and that gave all kind of permission problems so I will not touch that.
    For some reason it took about 1 3/4 hour to finish, but the system is functioning at the same slow pace it did before I forget about that.
    I will start working on a better back-up strategy. At the moment I will start with a copy of the SD card with the SD card copier which is standard in the raspbian jessie firmware. That should bring my up and running again much much faster than doing it from scratch.

    1. There are warnings Leo but that's what they are - warnings - of issues in other people's scripts - over which of course we have no control - usually they are to do with unused variables - why on earth those are not turned off is beyond me.

      On power supply switch off - see my latest blog - I've just hooked up a cheap OLED to my NANO and this can now be added to a supply to make it shut down the Pi or whatever gracefully and recover when the power recovers. All working.

      No there is no log - but by all means make one...

      If you don't change myboard it will be myboard - change it to raspberrypi if you want to call your board raspberrypi.

      The script takes anywhere from 7 minutes on a fast PC with Virtual Linux - to several hours on the lamentably slow Raspberry Pi Zero WIFI.. on the NEO 2here I think about half an hour or so - I'm quite getting to like this little board, I'll like it better when Friendlyarm get around to releasing proper GPIO code and the ability to use more than one of the four UARTS.

      1. I have read your latest blogs about uninterruptable supplies and GPIO, but that is , at least for me, shooting with a canon on a mosquitto(lol!). A whole new field of knowledge. I understand that if not running down the PI3 gracefully it might wrecken the SD card software in a lot of cases. I am surprised why your unit in Spain has never died when you say there is a horrible mains(solar supply?).

  51. The script took approximately 12 hours to run on a Raspberry Pi model B1, but in the end it did finish!

    I found two things that I would like to note:

    1- the rc.local file gets mangled up somewhat, there are two lines on the script with a sed command to change "exit 0" to something else, and one of the two changes "exit 0" to "chmod 777 /dev/ttyAMA0 exit 0" in the comments in the top part of the script, and the line break gets mangled too and thus the rc.local file fails on startup.

    I believe this line is the culprit:
    sudo sed -i -e 's#exit 0#chmod 777 /dev/ttyAMA0\nexit 0#g' /etc/rc.local

    2- I appreciate that IPv6 is still not very much used, but I do not understand why you are shutting it down for good in the script, both in /etc/sysctl.conf and in /etc/avahi/avahi-daemon.conf

    I use IPv6 and had to remove those lines to regain IPv6 access to my Pi. I noticed as I did start the script from an IPv6 SSH connection to my Pi, and as soon as I started the script, logging stopped as I killed my own SSH connection! 😀

    A great script, which helps immensely on starting up with a more IoT-oriented software load on the Pi.

    Many thanks, Peter!

    Luca Bertagnolio

    1. Hi there Luca

      Yes the old Pi boards really are not up to it - I think I threw my last one in the bin. Of interest, comparing your 12 hours - Ununtu running in emulation on a PC takes about 15 minutes - and I think the Pi3 takes way less than an hour. Much depends on broadband speed of course but basically the original Pis are now showing their age.

      That AMAO mod I'll ask Antonio to take a look at - that's a recent change - as for IPV6 -my take on this is that recently it has started to cause issues with people's routers being unable to resolve the addresses so the simplest solution was just to disable it - in the future we'll take another look at that. I am assuming those who use IPV6 specifically remain for now a minority - sorry it cave you problems.

      1. Hi - (and thanks for the work you've put into this and for publishing...)
        I've just run the script on a fresh installation (stretch lite) on a raspberry pi B +
        and the only problem was that rc.local is still being mangled. Simple fix to edit but thought I'd let you know.
        BTW - script took 2hrs 17mins.

  52. This works on the CHIP! I like that I don't have to worry about it too much with a 2500 mAh battery connected. The only challenge I had was that node-red won't accept my login credentials from Safari. I kept thinking I had entered the password wrong and re-ran the script a few times till I figured it out. Chrome worked fine.

    Any cautions with using the CHIP as it has no removable (i.e. easily replaceable) storage?

    1. Excellent - well, the latest version uses log2ram so while it will use up some of your RAM it may improve long term life of the FLASH... As that has a battery you might be able to change what I think we set as an hourly cron job to a daily cron job and keep the writing down even further.

      1. yes, i tested on the chip a while ago, i think about christmas, during the script rewrite we did that time... veeeeery slow, i think as the rpi1 and the rpi0/w... the good, it has an emmc... the bad, it's only 4gb and slow, that, too... but sure for a device of 9$ with embedded storage, is quite good, have to test it's gpio part, it has a good support and documentation on its site... what you did on the nas can sure be replicated on this little monster...

        p.s.: actually the original log2ram script flushes its caches every hour, you changed to every day for yourself 🙂

      2. I read somewhere you said your houses are connected, are your pi servers clustered? I'm not sure if that's the right word though, but basically that they are redundant and that mqtt nodes can failover and publish to any of them?

        Also, thank you for this resource. It's amazing the amount of work you've put into this, and the result is outstanding. I'm working on a design for an open source water treatment controller, something that could handle a fish tank, a hydroponic system, or wastewater, and all of this is a big help with prototyping the control and monitoring side.

        1. They are connected - each has it's own MQTT broker and they share information. It is also a handy way to know if something has gone wrong because either can email me if the other is not responding. They are not clustered however. Right now my ESP code can handle two WIFI access points but only one MQTT server. That could get quite tricky - if one of them went off on a tangent and others didn't.

          AS Robin Williams said .... glad one can be of service 🙂

  53. Housekeeping for The Script

    Peter, Just downloaded the script - in the initial Menu once it is running - there is a typo in the option to Update NPM to latest - it says NPN

    Otherwise looks good.

    In your instructions in relation to VMware - you might also want to give some info on how to install VMware Tools into the VM

    1. Hi - thanks for that - will fix it - be VERY wary about updating NPM to the latest version - I could not get Node-Red to start with it. I saw a comment in the Node-Red Google forums that it DOES - but isn't the recommended option. That why I un-ticked that option by default.

      Ok, script is fixed 🙂

      1. Yep tried that with the NPM upgrade - took a snapshot first just to be safe !!

        I love Virtual Machines !

        Craig

  54. Peter,

    OK weird one here - did a Virtual Machine (Vmware Workstation 12 Pro) deployment of The Script - using the Ubuntu Server ISO as per instructions.

    All worked fine - no issues whatsoever. Assigned a static IP address to the Virtual Machine and can login through SSH with no issues.

    Node-Red comes up fine - however whenever i try and connect nodes to it i am gettting errors. Basically on the canvas all the MQTT nodes have a disconnected underneath them.

    I exported some nodes from another Virtual machine i have been playing with (using Centos and my own manual install of most things) imported them into the canvas and the same thing happens.

    if i change my node (it is an Arduino Mega with ethernet shield) to point from my Test-VM (where it is sending status messages and receiving Time Updates) to the new VM where the Script is i have all the nodes as disconnected.

    I then tried MQQT-SPY and tried to connect to the topics - no good (but works fine for my other test VM)

    I thought maybe a problems with IPTables/Firewall so i shutdown the UFW on Ubuntu but still the same issue - any ideas - something weird with non-standard port numbers, authentication etc ?

  55. Guys ( Mr. Scargill and all those that helped) Thank you all for this effort. And DrFragle and MrShark - thank you for the mention of DietPi - to this point my only choices were Raspbian (nothing wrong with it) or more efforts and NOOB..

    I've just put DietPi on a Raspberry Pi 2 B with minimal additional software, then ran "The Script" as it was four days ago. And tada! working system.
    The note about dos2unix above was a big help, too. Glad I learned to read a bit before jumping in and asking questions.

    I've always gotten to a point (Pi loaded and ready to go) and then wondered where do I go from here? Well, I've got a better idea, now. And a couple Arduinos, a couple of NodeMCUs, and a box of sensors, so I think I might get somewhere, finally.

    One, incompletely informed question - I see WEMO boards mentioned here...would those be any relation to the WEMO switches? Which means I could control the outdoor floods from my Pi?

    Now, where did I put my log in credentials......

    1. Wemo - Wemo D1 - check them out on Ebay or aliExpress - just a cheap, very small form factor ESP-12-based board, I bought a couple - there are also some header boards available like a relay board... so nothing special except they are a nice small size and inexpensive.

      1. maybe it was just a problem with actual naming, as the esp8266 boards are called wemoS, not as plural of wemo but by themselves 🙂
        the full name of the small board is Wemos D1 Mini, as there are also the d1, the d1 r2 (both arduino uno form factor) and the d1 pro and the d1 lite, same as the mini but with more memory the former, and a different chip the latter

        1. Found/recreated those pesky login credentials. The WeMo Switch is actually a Belkin product - and apparently an American thing - according to Amazon, not available for shipment over seas. So I will have to do bit more digging on this side of the "pond". I've had the switch in place for a couple of years for those evenings when I get home after dark. Use the phone to turn on the outside lights - get inside. Use phone to turn off lights.

          There are a few how-tos that mention using an ESP8266 (or similar) and a relay to recreate the same functionality.

          Again, thanks to all y'all that have contributed.

          1. Oh WEMO - as in working with ALEXA - easy - there are a couple of software packages that do that - and if I am not mistaken the new software that ITEAD are putting in their SONOFFS does the same thing now.

            This is probably the best replacement software (tested by me)

            https://github.com/arendst/Sonoff-Tasmota

            then there is this (untested by me)

            https://community.home-assistant.io/t/sonoff-wemo-emulator/11001

            And the ITEAD link where they talk about their product working with Amazon

            https://www.itead.cc/sonoff-work-with-amazon-alexa

  56. Raspbian server... orange pi zero (ssh login)... error:
    [ ok ] Restarting ssh (via systemctl): ssh.service.
    ./script.sh: line 457: 1577 Killed whiptail --title "Installation Aborted" --msgbox "Cancelled as requested." 8 78
    pi@OrangePI:~$ /script.sh: line 457: 1577 Killed

  57. More details:

    + echo 'Dpkg::Progress-Fancy "1";'
    + sudo tee /etc/apt/apt.conf.d/99progressbar
    + echo 'APT::Color "1";'
    + sudo tee -a /etc/apt/apt.conf.d/99progressbar
    + username=user
    + userpass=password123
    + adminname=admin
    + adminpass=password123
    + newhostname=myboard
    + SECONDS=0
    + [[ RASPBIAN == *\R\A\S\P\B\I\A\N* ]]
    ++ whiptail --title 'Main Raspberry Pi Selection' --checklist '\nSelect items fo r your Pi as required then hit OK' 30 73 23 quiet 'Quiet(er) install - untick fo r lots of info ' ON prereq 'Install general pre-requisites ' ON phone 'Install o n Android Smartphone - see blog' OFF mosquitto 'Install Mosquitto' ON apache 'In stall Apache/PHP/SQLITE + PHPLITEADMIN ' ON nodejs 'Install NodeJS' ON nodered ' Install Node-Red' ON webmin 'Install Webmin' ON screen 'Install Screen' ON java 'Update Java' ON wiringpi 'Wiring Pi for the GPIO utility' OFF mpg123 'Install M PG123' ON modpass 'Mod USER and ADMIN passwords (password123)' ON phpsysinfo 'In stall PHPSYSYINFO' ON upgradenpm 'Upgrade NPM to latest version ' OFF addindex ' Add an index page and some CSS' ON passwords 'Update ROOT and PI user passwords' OFF installcu 'Install CU for serial VT100 Terminal' ON installmc 'Install MC+M CEDIT file manager + editor ' ON installjed 'Install JED file editor' OFF habri dge 'Install HA-bridge on port 82' OFF log2ram 'Install Log2RAM default 40Meg' O FF wolfram 'Remove Wolfram on a PI to save space' OFF office 'Remove LibreOffice on PI to save space' OFF
    + MYMENU=
    + [[ '' != *\q\u\i\e\t* ]]
    + AQUIET=
    + NQUIET=
    + [[ '' == '' ]]
    + whiptail --title 'Installation Aborted' --msgbox 'Cancelled as requested.' 8 7 8
    ./script.sh: line 457: 2052 Killed whiptail --title "Installat ion Aborted" --msgbox "Cancelled as requested." 8 78
    + exit

    1. don't have the smallest idea of what you did... lot of what you posted is not even in the script, sure not in latest... nowhere in script are comparison towards strings with all those asterisks... and raspbian is for original raspberry, not orange...

      1. The first post is the normal output, second one is debug output of bash script... The slashes are added by debug in the output.
        Raspbian is Raspbian server version downloaded from orange pi zero pages.

  58. Problems with the script!

    I downloaded the latest Raspbian and latest script and run into a big number of problems!

    First I was wrong with the answer to give the PI3 the name of myboard. There is something wrong with the name setting. I did not want to change the name to myboard. I ended up with an empty name field in the preferences. OK I wanted the name to be raspberrypi and changed that after installing Raspbian. Also chosing the options in the menu is confusing there is no mentioning how to choose options other than the hint on top of the menu to choose(how???) I made the mistake by pushing a return after the namesetting to and the script started running, which was not the intention. Choose with arrow keys and Tab or space , unclear especially when you are not doing this regularly.

    I get a lot of messages about email system not being alright, probably nothing because I do not use this but annoying!

    I was a bit attending the flow of the script and I have seen an error when installing Webmin. As a result I cannot login to Webmin. Can I install it afterwards?

    I backed up the files in .node-red from my flows and the database from my MySQLite data, but no flows appear in node-red. Then i tried stopping and starting node-red by node-red stop and node-red start but this gives errors because of a faulty package.json. I did that by overwriting it with the backup files . I only need to set back floss_*.json or also flows_*_cred.json?

    I cannot login to my Root account although I have given exact answers when asking for my username and passwords in the script.

    It does not seem to be so easy as the first time I ran the script on my Pi3.

    Have there been introduced problems because of all adaptions for other than Pi3(/2?)

    Are there any logs of the script to see if there were errors, I have no idea?

    Repeating the procedure is the best option I think!

    1. Situation update: 2nd time I did it alright and updated the ROOT an pi passwords which I unintentionally skipped the first time. Importing flows through clipboard by copying the JSON text from the flow* file. Only fill in username and password for the broker and we are going again.
      odd and even are there of course.

    2. space to select/deselect options, enter to confirm, easy peasy, you're the first to complain about this... webmin complained about missing user and password, probably... if you screw up things, don't try to patch a dead system, just restart flashing a clean distro... and if using latest testing version, you'll find logs in the same folder of the script...

  59. Ran the new and improved, experimental script last night and it ran without a hitch. Ran it over Real VNC, 2 hours and 19 minutes later, it was done.

    Nicely put together.

      1. unfortunately that distro is a mix of dietpi on top of armbian based on jessie, a bit of a mess... mosquitto is on version 1.3.4 so the problems in it not starting were related to websockets, in the end... other problem is that the mosquitto_passwd file works only interactively, so no scripting for that part... i could add a switch to check this specific version on this specific board, but is it worth it? If someone wants to go this route and use this dietpi, wait for the updated version Peter will publish and then you have to comment out the websockets part completely, so you end up with mosquitto passwordless but at least starting... too much of a hassle for now...

    1. Because 7 gave some other boards issues - and because I had no particular use for 7. Previous versions work just fine for the likes of PHPLiteAdmin etc... nothing to stop anyone upgrading...

            1. Hi
              Well it's probably very nice - but twice I tried - the first time nothing - the second time looked promising but no data... the point of this was? We know all of those work together? Or were you just proud of the site?

              1. Yes, they all work together.
                But if I had to do it now, I would develop it in another way.
                Probably with NGIX and skipping all the graphics.
                You know, putting an ssh file in the boot before booting and doing it from the beginning via ssh.
                Surely it is how you end up doing it.
                By the way, I saw on your profile that you spent some time in Spain.
                Do you understand Spanish?
                I develop in English with difficulties.

                1. I am indeed in Spain right now and for the rest of the summer - that is if we don't all die from the heatwave. Sorry can't help you with Spanish as my Spanish language is atrocious.

    2. because raspbian is just... OLD... still based on Jessie, and as far as debian is done (and raspbian on it), they have a release time of about 3 years... Stretch is out, let's hope in soon adoption, and support for the 64 bits specific of the rpi3... if you want php7 (which is faster than 5, but in our scenario is used only for phpliteadmin and little more, nothing vital to a iot setup), then go for an ubuntu 16.04 distro like armbian or dietpi, i think

        1. problems with github, not script... unfortunately their files are served in a round-robin fashion, or the like, and sometimes you end up in a server that's not aligned with the others and have missing files... that's what happens with that bcrypt file in your logs... i tried about 10 times to go to that url, 2 times i downloaded the file, 8 time i got a 404 page (not existing url)... so, just cross finger and retry...

          1. I thank you Dr....

            How do I then "reset" to start flawlessly from the top? eg, does not complain about directories, etc?

            Is there a way to know I have d/l the GOOD script?

            I will keep trying.
            Again, thank you.

  60. I see you made an update for The Script.
    As I think I am behind in updating my existing Pi3 and Raspbian Jessie I thought let's try that VM method to create a new image with the latest version.
    Then the question what OS to use! I got lost. You sort of advertised Raspbian Stretch, but when I read the comments of your masked friend that is not latest and best technology to use ?????????????? A little advice would be appreciated here.

    1. No advertising - simply mentioned that it exists (and hence WILL be with us) and so we modified the script accordingly. Which one you use is up to you. I'm upgrading my PIs. I just did the install on a Pi3 - had to modify a separate Grafana setup to get it to work but most other stuff seems just fine. Backed up the SD and now I have a Pi2 working on the same setup.

      1. So at last xenial is a little advice! So I take the desktop version as I am a linux illiterate. So I got this one: ubuntu-16.04.3-desktop-amd64.iso

      2. Sorry to say. In VMWare Workstation 12 I am stuck at the globe asking for timezone. The flatted globe does not fit into my little window "Where are you". I only see a part of a unactivated button and that's where I am no further actions possible ESC ctrl-c Tab nothing! What is the trick to get out of that situation? see also png
        There is a message which asks for Install Tools and that seems the only way out circumventing all following settings.

          1. Cannot resize window. Something to do with display adapters not yet initialised. I must update Tools. I will start from scratch again. I took away 3D feature.

  61. I followed creation of pi. HOwever there is a problem!
    >>>

    root@Ubuntu-VM:/home/hoekbrwr# adduser --quiet --disabled-password --shell /bin/bash --home /home/pi --gecos "User" pi echo "pi:password" | chpasswd usermod pi -g sudo echo "pi ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/pi chmod 0440 /etc/sudoers.d/pi
    chpasswd: invalid option -- 'g'

    <<<

    What is going on?
    I have created pi user as administrator through menu, but cannot let it login automatically. Slow boat to China....

    1. honestly, don't know what the h... are you doing... why create the pi user from menu, if script creates it itself? and why not just creating it during install... and, repeat, customizing the graphic part of ubuntu is beyond this blog, there are TONS of tutorials online on how to do... why are you running those lines by hand and not by script? Restart from scratch, do a textual install, install the script, then do whatever you want with gui, not relevant for the script functionalities

  62. Took away the invalid option -g., next problem

    >>>

    root@Ubuntu-VM:/home/hoekbrwr# adduser --quiet --disabled-password --shell /bin/bash --home /home/pi --gecos "User" pi echo "pi:password" | chpasswd usermod pi sudo echo "pi ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/pi chmod 0440 /etc/sudoers.d/pi
    adduser: Only one or two names allowed.

    <<<

    I think I already had to define the pi user during installation of the OS!

    That's it for today... Tomorrow more tries.
    It is not as easy as it looks. All the details in the text are useless because with every other OS installed things look different to the blog text.

    1. last comment, then i surrender: WHY ARE YOU RUNNING THOSE LINES BY HAND??? Instructions are CLEAR: login with whatever user you want, create a file, paste the script, save, give permissions, run it... if logged in user it's not pi it will be created, if it's pi it will be updated with needed permissions, then you have to login as pi and rerun the script and answer some VERY basic questions, PERIOD! DON'T OPEN THE SCRIPT, LEAVE IT THERE AS IF IT IS A BLACK BOX! RUN IT, NOTHING ELSE!

      1. Sorry but this is still in the blog with a lot of other stuff not useful because this was for one specific system accurate. I will run The Script and then it should be OK.

        1. You've lost me Leo - the script works well for a bunch of different systems - most of which are noted in the comments at the start. Don't deviate from simply running the script - or we can't help.

          1. NOTHING BIG WRONG WITH THE SCRIPT! Rest assured it is almost flawless. unattended in the VM in 18 minutes on my 6 year old PC(a bit pimped SSD 8GB RAM)
            I was following the instructions in the blog and it is not in sync with Ubuntu UI messages. When I start installing Ubuntu I am asked to make a user account and there I used my standard username. I think one should use pi as a user and set the pw at that point. Then the script will start without the extra round because there is a pi user. I tried this big command(found in your text) to create a pi user and I got all kind of strange errors. I am not a Linux guy and I tried to create the user with the User menu in the settings and that was not clear to me. Then Antonio got irritated by my complaints and I realised I had seen this earlier with another try that the script creates the pi user in the first round and then can be started and run to the end.
            The text just above Bash script just did not fit for Ubuntu. I think it was written with another linux version when you first made the blog for VM installation. All things very confusing root, pi user admin(later on in the script)
            Now I have run The Script and have a VM with Ubuntu.
            How should I produce an image for my pi3 from that? Convert .vmdk to .iso? Can I boot it then?

              1. I am lost. What is the purpose of running on a VM? I made a wrong conclusion it can be used to flashed to use as home controller on the tested hardware?

  63. Peter and Mr Shark, Thank you for the 'script' and the set up of the VM, I used virtual box. I had one issue and that was when I went to install Ubuntu I could only see 32 bit files despite me having a 64 bit Windows 10 machine. This problem is caused by hyper-V being disabled in the BIOS. There's quite a lot of information about that on 'Google' and once I found the correct location in the bios and enabled that functionality I could see the 64 bit files in the virtual box set. I think where Leo is getting confused, as was I, is that Peter's excellent video describes the Virtual box install whereas the blog text describes the VMWARE version, which of course it states. In my case once I had the issue of the missing 64 bit files solved I followed the video and everything installed perfectly and after installation everything runs perfectly. I put a BigTimer flow on the virtual machine and used MQTT to send the data to MQTT-spy running on my Windows 10 desktop PC, in fact the same physical machine that also has the virtual machine. Windows and Linux in harmony!
    Incidentally it 16 minutes to run the 'script' and load everything onto the virtual machine. A super job in getting the script to be so easy to use and so versatile. I am very new to Linux so it is a big help to install so much software easily.

      1. Now I can see how many wrong turns I took in my run on the VM installation. It is very nice and helpful. You have that so fast great work!

  64. Hi , I could not install the script in ubuntu-16.04.3-server VM.
    This is the error:
    E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable)
    E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it? .Thank your help.

    Marcelo

  65. Hi Peter,

    Firstly, Heartfelt gratitude for sharing this wonderful piece of script. Works smoothly for everything. However, I am facing trouble accessing the MQTT broker. I am trying the access the MQTT broker running on my Raspberry pi (raspbian stretch) from my Node MCU. The connection can't be established. The Node MCU connects to the Wifi but returns an error (rc=5) connecting to the MQTT server. It says connection denied and I am not authorized to connect.

    I need some help. May be this is not the right blog but a little pointer in the right direction would be great.

    Thanks,
    Aritra

  66. Testing on a Raspberry Pi 3 this script breaks /etc/rc.local:

    Before running it has
    #!/bin/sh -e
    #
    # rc.local
    #
    # This script is executed at the end of each multiuser runlevel.
    # Make sure that the script will "exit 0" on success or any other
    # value on error.
    #
    # In order to enable or disable this script just change the execution
    # bits.
    #
    # By default this script does nothing.

    # Print the IP address
    _IP=$(hostname -I) || true
    if [ "$_IP" ]; then
    printf "My IP address is %s\n" "$_IP"
    fi

    exit 0

    After it has
    #!/bin/sh -e
    #
    # rc.local
    #
    # This script is executed at the end of each multiuser runlevel.
    # Make sure that the script will "chmod 777 /dev/ttyAMA0
    exit 0" on success or any other
    # value on error.
    #
    # In order to enable or disable this script just change the execution
    # bits.
    #
    # By default this script does nothing.

    # Print the IP address
    _IP=$(hostname -I) || true
    if [ "$_IP" ]; then
    printf "My IP address is %s\n" "$_IP"
    fi

    chmod 777 /dev/ttyAMA0
    exit 0

    Your command on line 866
    sudo sed -i -e 's#exit 0#chmod 777 /dev/ttyAMA0\nexit 0#g' /etc/rc.local
    matches two occurrences of exit 0, including one in the comments at the start.
    Inserting a new line there results in a Bash error.

    I suggest this instead to only match a line containing only "exit 0"
    sudo sed -i -e 's#^exit 0$#chmod 777 /dev/ttyAMA0\nexit 0#g' /etc/rc.local

    By the way, why do you chmod 777 /dev/ttyAMAO?

  67. It works perfectly - unless you try to start programs from /etc/rc.local!

    Try running rc.local from the command line and you see the error.

    Actually, since it errors, I don't think it can be executing the chmod command anyway, so maybe this line could be deleted. (I think it's only run for Raspbian)

    1. Quote: " Author: Antonio Fragola
      Comment:
      well, just don't do it... rc.local is supposed to be the LAST started up script during boot up, not a script that you run every now and then... you want to run it as you do? Then know what you're doing and how to deal with your issues..."

      Thanks for that Antonio. It's always a good idea to know what you are doing and how to deal with your issues.

      I don't recommend running /etc/rc.local from the command line EXCEPT in the circumstances where it is broken and you want to find the error.

  68. I have tried The Script on a new Stretch install (not upgrade) and it kept choking. After some investigation I found that Mosquitto (MQTT server) will not install because of messed up dependencies. 🙁

      1. I was mistaken. I thought it was Mosquitto but it seems to be sqlite. I had tried the-script a couple of few and it alway failed. So I started with a fresh image (NOOBS) and manually removed and installed packages. I got stuck on Mosquitto.

        From internet searches I found that the install of Mosquito fails with Stretch, so I assumed my problem with the-script was Mosquitto, but looking through the log it did install with an older version.

        My complete log is at https://gist.github.com/RudyFiero/e390a5b840177c27d527ab1910e52491

        The last bit where it failed is this,

        > sqlite3@3.1.13 install /home/pi/.node-red/node_modules/sqlite3
        > node-pre-gyp install --fallback-to-build

        node-pre-gyp ERR! Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v3.1.13/node-v48-linux-arm.tar.gz
        node-pre-gyp ERR! Pre-built binaries not found for sqlite3@3.1.13 and node@6.12.2 (node-v48 ABI) (falling back to source compile with node-gyp)
        make: Entering directory '/home/pi/.node-red/node_modules/sqlite3/build'
        ACTION deps_sqlite3_gyp_action_before_build_target_unpack_sqlite_dep Release/obj/gen/sqlite-autoconf-3150000/sqlite3.c
        TOUCH Release/obj.target/deps/action_before_build.stamp
        CC(target) Release/obj.target/sqlite3/gen/sqlite-autoconf-3150000/sqlite3.o

        I am not very familiar with Linux. (but with every problem learning more) The NOOBS image has node-red included under applications. After running the-script it is missing but it does work when run from a terminal window. It was the main reason I tried the-script. I have a older model B running Mosquitto on a DietPi install.

        I hope you can get me past this bump. If not, I still appreciate any thoughts.

        1. seems just a temporary error, try redoing this in a few days, or report directly to the sqlite3 node author, as there's nothing wrong in script... it does "npm install" of the node, if the binary is not available it downloads sources and try to compile, if sources have problems script can't fix them...

  69. From email - "Don't use script with. Noobs. Not tested. Use full raspbian clean install"

    So I did a new Raspbian install, no NOOBS, ran the script. And it all worked 🙂

    Thank you.

  70. Hey, I just wanted to point out a potential issue I stumbled across.
    I wanted to install the script on a Raspberry Pi Zero W (running DietPi) and ran into an issue with NodeJS. The NodeJS installation failed for me as the wget resulted in a Permission denied error. After utilizing the command again manually with sudo it worked. (I'm referring to line 798 of the script)

    wget --no-check-certificate https://nodejs.org/dist/$LATESTNODE/node-$LATESTNODE-linux-armv6l.tar.xz -a $LOGFILE

    Also I was curious, why you hardcoded the version 6.11.3 in the script when there are newer releases. on nodejs.org? I'm new with NodeJS so forgive me if this is a stupid question I'm asking here.

  71. Hey.. I found some more issues on the RPi Zero W (DietPi), e.g. with node-red I have one error when it tries to create the node-red directory
    the-script: line 849: cd: .node-red: No such file or directory
    and then later it tries to cd into that directory, which of course it can't
    the-script: line 872: cd: /home/pi/.node-red/: No such file or directory

    I'm not complaining, more like a little wondering, whether it is as fool proof as I thought (read that), and yes, it is kind of fool proof, I just found it irritation that I run into these errors where in fact sudo is missing in front of the commands from time to time and whether this is a script issue, or an issue with my installation.

    1. as always, publish somewhere full logs you have in your home directory

      if you see script you'll find, at line 847:

      cd
      mkdir .node-red
      cd .node-red

      so, it goes in your home folder, creates that new one and enters it
      Why on earth that should fail... Are you sure your sd card is good, or is it mounted readonly? Did you enlarge the FS on first boot, and rebooted after?

      1. Hey,
        I tested this further, now on a VM, also for speed reasons, and there it worked fine. So seems there was something completely wrong with me user setup. Sorry for "wrong alarm". I will check further

  72. Hello
    I have a problem after running the script on a rasppi 3.
    The os is Raspian stretch and I have it set up booting from a hardisk 360GB
    All appeared ok. I had a clean unused OS booting ok.
    I ran the script removing wolfram, changing the user and pw for dashboard, and
    the pw for nodered & MTTQ
    I enabled Webmin
    I saw no errors the script finished ok. Booted ok
    Mqtt running ok
    Nodered flow ok
    put small flow in nodered, connected to mqqt ok
    Opened dashboard prompted for user and pw entered both which I had noted
    reaction was no autherisation.
    I tried various combinations of pw etc
    I have rerun the script 4 times noting exactly what I changed each time-
    I have the same result, I can open nodered flows, mqqt, but not nodered dashboard
    Anyone know of this issue or have a pointer to possible error I have made
    I have in the simple test flow an inject timestamp to Text(ui) and debug. Debug gives out a valid timestamp
    I am a newbie to nodered and Linux

  73. put my harddisk to pc ran Paragon harddisk manager
    I did a new build of raspian stretch on harddisk using Etcher
    booted rpi3
    ran the script again no errors
    nodered dashboard now works
    can only think was a corruption in a file or the harddisk
    The script worked perfectly

  74. First:
    THANKS a million for "The Script" it was the only method that worked for me to get Mosquitto running. HOWEVER, nothing I seem to try gets Node-Red to connect to the Mosquitto broker.

    I would not be bugging you except I have now spend 3 days trying to get Node-Red to connect to Mosquitto and am hopelessly lost. Given my 0.0001 level of expertise in Linux this is no surprise.

    RPi 3: NOOBS_v2_4_5, loaded Stretch, updated, loaded "The Script" and a cup of coffee later, seemed everything is running. NO Mosquitto load errors [that's a first].

    I have 1 Sonoff flashed with ESPEasy, and used the information from videos by Csongor Varga to configure the Sonoff as "Sonoff_1".

    I used his supplied library @ https://flows.nodered.org/flow/5c8a19f0e862fd59457d3b8c2c906acc and only changed all references from Sonoff1 to Sonoff_1 and of course the IP address.

    If I follow his instructions to the letter I always get a "disconnected" message in any MQTT node in Node-Red.

    However, if I put something for an IP address that I know is wrong in "Edit MQTT-broker-node", I get a "connecting" message but never connected.

    My Sonoff [IP 192.168.1.101] "Controller IP" is set to 192.168.1.9 [RPi IP addr] Port 1883

    In node-red I have tried "localhost:1883", 127.0.0.1:1883, node-red@192.168.1.9:1883 and none of those work.

    Putty:SSH into the RPi as I am clueless in Linux as to determining if anything is runnning I tried this.

    pi@homeauto:~ $ mosquitto
    1515598144: mosquitto version 1.4.10 (build date Mon, 29 May 2017 13:43:29 +0100) starting
    1515598144: Using default config.
    1515598144: Opening ipv4 listen socket on port 1883.
    1515598144: Error: Address already in use

    So that told me, I THINK, that Mosquitto is running.

    During the script process anytime I was prompted for a login/password I used "homeauto" so hopefully that is what Mosquitto is expecting.

      1. Thanks DrFragle, I have tried a manual load of Raspian/Stretch-->Node-Red & Mosquito, a NOOB --> Raspian/Stretch plus "The Script", a Raspian/Jessie--> plus "The Script", a Raspian/Stretch--> plus "The Script" and I still cannot get any indication that Node-Red is even aware that Mosquitto is running.

        Even using a known running node configuration from Csongor Varga and only changing "Sonoff1" to "Sonoff_1" and changing the IP address to reflect the IP address of my RPi3 renders the same result "Disconnected"

        After now 4 days, I am admitting defeat and will try to find another solution where I am not working for the software instead of the software working for me.

        I can still greatly appreciate the huge effort and brilliance in creating "The Script" but it is obvious my skills as an 8 core assembly programmer are insufficient to this task.

        Thanks again for your efforts.

        1. don't know what's happening... is port 1883 available?
          i usually put an inject node in an mqtt publish node, with some test topic, then a mqtt subscribe node to that topic, redirected to a debug node... always worked...

          1. This would tend to tell me Mosquitto is running and listening on port 1883 but then again, I obviously know zero about Mosquitto.

            pi@homeauto:~ $ mosquitto
            1515598144: mosquitto version 1.4.10 (build date Mon, 29 May 2017 13:43:29 +0100) starting
            1515598144: Using default config.
            1515598144: Opening ipv4 listen socket on port 1883.
            1515598144: Error: Address already in use

            When it shows "disconnected" in Node-Red, does that mean Node-Read is not connected to Mosquitto or that Mosquitto is not connected to my device, in this case my single Sonoff which was flashed with ESPEasy?

            These are my Sonoff settings.

            Name: Sonoff_1
            Admin Password:
            SSID: xxxxxx
            WPA Key:xxxxxxxxx
            WPA AP Mode Key: configesp
            Unit nr: 1
            Protocol: OpenHab MQTT [have tried others]
            Locate Controller:
            Controller IP: 192.168.1.9
            Controller Port: 1883
            Controller User: homeauto
            Controller Password: homeauto
            Sensor Delay: 60
            Sleep Mode: [not checked]
            Optional Settings
            ESP IP: 192.168.1.101
            ESP GW: 192.168.1.1
            ESP Subnet: 255.255.255.0
            ESP DNS:
            8.8.8.8

            1. try using whatever mqtt client you know (for example: mqttlens or mqttbox, both are google chrome extensions), or via command line to check if it connects:
              mosquitto_sub -h localhost -t "#" -u YOURUSER -P YOURPASSWORD

              1. This is what was returned in Putty/SSH

                mosquitto_sub -h localhost -t "#" -u admin -P homeauto
                -56.00
                -56.00
                -55.00
                -55.00
                -55.00
                -55.00
                57.00
                -58.00
                -57.00

                This looks like the WiFi signal strength from the Sonoff

                1. Looks like the command you suggested is returning all the expected values to Mosquitto even though nothing appears on Node-Red. I added the device nomenclature in [xx]

                  -58.00 [Wifi]
                  0 [button]
                  0 [button]
                  -58.00 [Wifi]
                  0 [button]
                  0 [button]
                  21.00 [UpTime]
                  0 [button]
                  -63.00 [Wifi]
                  0 [button]
                  1 [button]
                  1 [button]
                  -63.00 [Wifi]
                  1 [button]
                  1 [button]
                  -59.00 [WiFi]

            2. if you used the script, then the mqtt user will be admin, this user is created by the script, and the script prompts for the password right at the start.
              Secondly you're getting an adress already in use error when mqtt starts.Im not sure why this would be, seems strange that port 1883 is already in use. Unless there are 2 instances running.
              you could try changing the port number to something else, and reconfigure node-red and the sonoff to use the different port.

                1. Yeah that login was my typo, it is using admin.

                  Changing the port# in node-red gave me a lost connection error. Restoring it to 1883 removed the error but still disconnected in node-red.

                  Does, "disconnected" in node-red mean node-red is not connected to the server OR does it mean it cannot see the Sonoff?

                  1. It means disconnected from mqtt. Node -red only ever sees mqtt, as its a queue, It knows nothing of the sonoff, it just subscribes and publishes messages to the mqtt queue.

                    can you supply your mqtt connection and ssecutity settings, as they appear in node red

  75. A few days ago I posted about how the latest DietPi somehow broke the-script but it did not make it past the moderators to show up on here.
    I have since been messing with the system and discovered the problem is the clean DietPi install contains a /home/pi folder but no pi user. I found that deleting the /home/pi folder and then running the-script fixed the problems
    Hope this is helpful to other new users (assuming it ever gets displayed.

    1. Thank you, i was puzzled by this and threw the towel expecting a user/permissions nightmare without investigating. Serves me right that it was that easy to fix 🙂

  76. Sorry guys,
    it was a messy installation due to the poor internet connection.
    I’ve done it again form the beginning and everything works now.
    Marvellous!
    Thank you for everything!

  77. what am i missing when i get the web page Pete's utilities up at 192.169.0.202 all the links work but not the node red two second and third in the list.

    by the way i am not a coder and find all of this really hard going

  78. Hello Peter,

    I have follow the instructions to install the different programs in the script and they all seem to be ok. However, I don't have any packages on Node-Red. I had node-red installed previously so I have uninstalled it and installed it again from your script. Still no new packages in node-red. Am I missing something?

    Thanks

  79. Hey Pete how are you, Im happy that you are back!
    I was able to use your amazing script on jessy, and whezzy distros on a RP3, now Im doing a clean install on the latest stretch. I was going crazy because putty was ending the connection, and I found out that the raspy restarts suddently at the stage of installing nodes
    I re run the script, without the extra nodes, all the rest of the things seems to work fine
    I re run the install node red alone and same problem at the stage of installing nodes
    Not a big problem, Node red works, I just needed to set ip up as auto start
    but im not able to install the ui nodes

    pi@raspberrypi:~ $ npm i node-red-dashboard
    npm ERR! Linux 4.9.59-v7+
    npm ERR! argv "/usr/bin/node" "/usr/bin/npm" "i" "node-red-dashboard"
    npm ERR! node v6.13.1
    npm ERR! npm v3.10.10
    npm ERR! file /home/pi/.npm/encodeurl/1.0.2/package/package.json
    npm ERR! code EJSONPARSE

    npm ERR! Failed to parse json
    npm ERR! No data, empty input at 1:1
    npm ERR!
    npm ERR! ^
    npm ERR! File: /home/pi/.npm/encodeurl/1.0.2/package/package.json
    npm ERR! Failed to parse package.json data.
    npm ERR! package.json must be actual JSON, not just JavaScript.
    npm ERR!
    npm ERR! This is not a bug in npm.
    npm ERR! Tell the package author to fix their package.json file. JSON.parse

    npm ERR! Please include the following file with any support request:
    npm ERR! /home/pi/npm-debug.log

    Let me know If i'm doing something wrong here

  80. Howdy! Great work!

    On a recent install of your script I ran into an issue where NodeRed wouldn't start. i traced it down to a commented close brace in the settings.js file at line 244 after the install. I don't know if I am on the only one running into the this issue but it was something I had to look at it since I use your script just about everywhere. Thank you for the amazing work and contributions.

    Neal

    1. I did an install a few days ago and the script worked just fine, I do hope you sort this out. Do let me know what happens. I can't immediately see anything wrong with the script.

    2. Hi - I had exactly the same issue on an install done on Saturday - latest version of raspbian stretch lite on a pi3b and latest copy of the script - ended up with superfluous closing brace and node red would not start

        1. i stopped nodered on my board, saved all the content of the .node-red folder, emptied it, ran lines 872-919 of the script (the ones that create the settings.js and update it adding encrypted password and some other stuff): no problems at all... sure you didn't run the script twice, even if there are some protection in script to prevent that?

          In case you can, can you start over, and send me the 3 settings.js (1 as such, 1 with .bak-pre-crypt and 1 with today's date in its filename) files you'll find into your .node-red folder, BEFORE removing any damn bracket, to see WHERE it's put so to find WHAT put it there...

          you can strip your encrypted password if you want from the actual settings.js file (it's in 2 lines towards the end of the file)

  81. I'm not by my machine but I will send the log file - not that I could see anything in it when I checked earlier. If it helps the extra brace and comma appears after the logging section and just before the editor theme section ( which being new may be the culprit). One other clue may be in that the script appears to have skipped the two lines which set up nrlog and the systemctl enabling of node red as I had to start it manually after fixing the problem and rebooting and nrlog is not there. Actually while typing this I managed to access the settings file and I note that the httpStatic section in my settings file which is the last section is missing its closing brace - I hadn't noticed because this is not throwing an error.
    In summary the last two sections of my settings file are editorTheme and httpStatic and the closing brace that should come after httpStatic appears before editorTheme and this may be linked to the fact that the script then skipped the nrlog and systemctl lines (although the intermediate lines relating to flowfile and require fs were carried out. As to why I cannot see.

    Just to be clear this was a completely fresh first time install on a new install of the OS which had been upgraded to latest versions.

    1. i think i've found the problem... now i've read your comment and i realized just a few days ago there was a NEW nodered version, 0.18.6...

      take a look at last line of the changelog:
      https://github.com/node-red/node-red/releases

      "Add editorTheme.projects.enabled to default settings.js"

      they enabled the project feature by default now... so, settings.js file is now different from before... and that's the error, not the script itself, or not just it...

      if i fix script now, it will work ONLY if you install it NOW and so having nodered 0.18.6... but, btw, what we advert is to run on fresh systems, so if you install it now, you'll end up having that anyway...

      let me update nodered and fix it...

      to be continued

      1. sorted out... Peter is going to update the script... the mod is to add this line after line 919 of ACTUAL script, to restore the correct number of curly brackets:

        sed -i -e 's#\/\/ Customising#},\n\n \/\/ Customising#' /home/pi/.node-red/settings.js

        as said, problem introduced by nodered 0.18.6 which is out for 3 days, now...

          1. Thank you guys very much! Amazing work so quickly.

            And to be clear, I got it working last night. Y'all need anything else from me?

  82. Peter,

    Wonder if you investigated turning the script into a docker image ? Would be much easier for you to manage longer term and fairly easy from an install point of view (with adequate documentation)

    1. guys, docker is f***ing awesome... working on it, there's A LOT of stuff out there... but we can reduce memory footprint of the various services A LOT, without loosing functionalities and having a setup easier to deploy, update and backup... stay tuned!

      Peter, do you mind creating a topic to discuss of this, so i can have help from others and have a quicker setup soon? Something like: "WIP: Docker", or the like, you decide... Thanks in advance

  83. Peter, What is the recommended way to upgrade Node-Red once installed with your script ?

    I have built a Virtual Machine about a year ago from the script and it has been running fine - i have kept all of its modules upto date with Webmin.

    Now i want to upgrade Node-Red - is the official script from Nick Oleary OK to run on your deployed system ?

    https://nodered.org/docs/hardware/raspberrypi

    regards

    Craig

  84. Thanks Guys,

    Using the supplied update script from the Node-Red guys worked perfectly to perform the upgrade.

    I then rolled back the snapshot and tried the script supplied by Antonio - again this worked just fine - so either option will work for you it appears on a VM built from the Original Script as supplied by Peter.

    Craig

  85. I did several installs and all failed installing node-red and HA bridge. Everything else worked fine. Im installing on a orangepi zero. I read the comments about update of NR. I assume the script is fixed now? Any ideas why the NR install and HR bridge failed? Each install was to a clean flash drive.

    What should I do to ensure a good install?
    Thanks

    1. You've not stated which operating system, not all combinations have been tested. Orange Pi Zero was some time ago. If you are on same version of operatinmg system it could be changes to Node Red?

    1. i see both installed in logs... can you run "nrlog" to see what's there?
      for habridge, sorry, never used, don't know what to check there... i think it's useless now, as newer nodes for node-red are out and can manage same thing easier...

  86. nrlog....

    Started Node-RED graphical event wiring tool.
    Error loading settings file: /home/pi/.node-red/settings.js
    { Error: Could not locate the bindings file. Tried:

    The above is exactly what nrlog reported. I do not see a .node-red directory in the home/pi directory. The only files there are...
    dbs, habridge, nodesetup.sh and the script log file.

  87. Do to my limited linux knowledge...I didnt know that was a hidden folder. I used winscp and I do see the .node-red folder. The settings.js file is in there too! Yes I am 100% sure I did a clean install. I reformat the sd before every install. I tried to start NR manually(node-red-start) and I get all the info on how to use NR etc, then see NR started as service, starting graphical event wiring tool then this...

    Started Node-RED graphical event wiring tool.
    Error loading settings file: /home/pi/.node-red/settings.js
    { Error: Could not locate the bindings file. Tried:

    Same info from nrlog.

    The settings.js file in the /home/pi/.node-red folder and its full of NR stuff...ie not empty. Seems like NR cant see that folder or rights to the file/folder are bad? The settings.js file has group sudo R properties too.

    any new ideas?

    1. as said, i just did a fresh install using latest script on top of DietPi on a NanoPi Neo... standard menu selection + checked "grafana and influx"... well, "flawless victory"... all services up and running... so maybe something in raspbian? Don't know and don't have a spare sd/rpi to try...

      there are some problems with those packages Peter added at the end of the script a while ago (bottle and glances), but absolutely not his fault, it's just that those installs sometimes work and some don't, i've already seen them going in error... just retrying a few days after and they're fixed... nothing essential, BTW...

  88. I was just now looking at doing a manual install of all the stuff, looks pretty daunting. After your comment...I'm thinking maybe best to use the script to install everything EXCEPT NR. Then manually install NR. Good idea?

    Maybe you can help with the command lines to do a manual install of NR? Also a list of all the script boxes to check to get all the stuff required. If I get a good install...at least that would isolate the area thats causing the problem.

    I tried doing a manual install of NR on top of existing install...but it was a fail. However there was more diagnostic messages but I didnt save those messages :-(. At this point I am at a clean install of armbian and trying to decide what to do next. Your post was very timely...however I was hoping you would find the problem and I could run a new script and celebrate...haha.

    If I do a manual install of NR and get those error message again...I will save them. I'm thinking its something in the settings.js file thats goofy.

    1. yes, try armbian or dietpi (which is based on armbian and more lightweight, i like it a lot... even thought probably tkaiser will say it's not good 🙂 )

      or AFTER Peter's script, try running the script suggested by Peter himself a few days ago: https://github.com/node-red/raspbian-deb-package/blob/master/resources/update-nodejs-and-nodered

      it was just updated to latest nodered... it's specific to raspbian, even though it "should" work on other distros, too

  89. I am currently running Armbian 5.38 stable Debian GNU/Linux (stretch) now. I just did a clean install.

    I want to be sure I understand your suggestions....

    1. Run Peter's script (should I select NR install or not?)
    2. Run the NR update script

    1. i suggested to run second script after peter's one when you where on raspbian... now that you already changed distro, go for default setup in peter's script and let's see what happens... don't run second one, report back only on peter's script, we offer help on that one only...

  90. I have never been on raspbian....I have always been on Armbian 5.38 stable Debian on an Orangepi Zero. I downloaded the latest script and see no changes to the one I have been using...so I would expect the same results...ie bad NR install.

    OK...I will run the same script again and expect the bad NR install....then I will run that update script and hope for the best.

    Thanks

  91. I will give dietpi a try as Im still getting bad NR installs. The dietpi website doesnt have any images for the orange pi zero...but there seems to be some floating around. I'll try to find one and give it a shot. Crossing fingers!

    fyi....after the latest script run...NR still bad. I ran the second script mrshark suggested...and it coughed up an error and never ran.

  92. Hi Peter,

    I did use Andrea's Spiess image before based on a outdated Dietpi image now which cannot be update anymore.

    Because of this i wanted to install it again on a new version of DietPi (v6.8)
    I ran into some issues:

    * PIP did not seem to be installed (sudo: pip: command not found)
    can be resolved by installing before (sudo apt-get install python-pip)
    * dropbear SSH server is being used by default (did nout cause issues as far as i am aware of, but configuration might nt be altered by the script)

    Did anyone ran into these issues as well?

    1. 1st question: strange, python-pip is in line 585 of script from ages, now...
      https://bitbucket.org/snippets/scargill/ekon6o/the-script

      for the latter, yes, it's standard dietpi... you can switch to ssh via dietpi-config after 1st boot, or just by editing the config.txt and/or dietpi.txt in the fat32 partition of the sd card, even before putting it into your device, just after flashing the sd...

      take a look at these 2 files, you can setup a lot of things before even doing the 1st boot... example, set wifi credentials, fixed ip, change dropbear for openssh, etc...

      ah, you can use dropbear as it's a valid ssh daemon, but it does NOT allow file transfers... so if you don't need to use winscp or similar, don't bother...

  93. I think I finally discovered the NR install problem. I tried a manual NR install again and this time I noted the errors in the nrlog. All the errors relate to I2C. Seems the opi0 has i2c ability but its not enabled. I dug through some info about overlays etc to enable ic2, but didnt succeed. I also read your blog about i2c on the opi0 but you were using dietpi. I am using armbian debian. Does dietpi somehow pre-enable i2c?

    Here are a few of the lines of the NR error log....

    Error loading settings file: /home/pi/.node-red/settings.js
    { Error: Could not locate the bindings file. Tried:
    → /home/pi/.node-red/node_modules/i2c-bus/build/i2c.node
    → /home/pi/.node-red/node_modules/i2c-bus/build/Debug/i2c.node
    → /home/pi/.node-red/node_modules/i2c-bus/build/Release/i2c.node
    → /home/pi/.node-red/node_modules/i2c-bus/out/Debug/i2c.node
    → /home/pi/.node-red/node_modules/i2c-bus/Debug/i2c.node
    → /home/pi/.node-red/node_modules/i2c-bus/out/Release/i2c.node
    → /home/pi/.node-red/node_modules/i2c-bus/Release/i2c.node
    → /home/pi/.node-red/node_modules/i2c-bus/build/default/i2c.node
    → /home/pi/.node-red/node_modules/i2c-bus/compiled/6.14.2/linux/arm/i2c.node

    So now what is your suggested next move? ...
    * Go to dietpi (thats no longer supported on the opi0),
    * Help with enabling i2c on the opi0,
    * Buy a different sbc (looks like the rasberrypi w is a good replacement)
    * or ?

    I'm leaning toward dumping the opi0 and going to the raspberrypi.

  94. I deleted all lines in the settings.js file that related to i2c. Manual install NR again and got no errors! Connected to opi0 via web and now the NR links open ok from Peter's webpage.

    I'm going to do a clean install again later and edit the settings file to remove those i2c lines and run the script again. I suspect all will be well. I will update again.

    Im thinking I will have to run the script once to get the settings.js file..then edit it and run the script again?

    1. no, you can just remove those 2 lines before rebooting after running the script, or you can comment these 2 lines (which are the ones that add those 2 lines to settings.js) before running Peter's script:

      886: sudo echo " i2c:require('i2c-bus'), " >> tmpfile

      906: sed -i -e 's#\/\/\s*var fs = require("fs");#var fs = require("fs");\nvar i2c = require("i2c-bus");#' /home/pi/.node-red/settings.js

    2. Peter, what about adding a menu item to add or not the hardware related stuff? I2C, for example, is useless if you are installing in virtual machine... and have caused problems to me, too, with some boards... even standard raspbian does not enable serial, i2c, spi, in default install...

      1. Yes, but the only thing we have to do with i2c surely is a node and that should not cause issue till used? The mosca mqtt broker looks to be broken but does not fail and kill nr until I attach an mqtt input to it?

        Pete

        1. i had problems, in various occasions, with the serial node and the i2c one for sure... sometimes it happens that npm complains about missing binaries and have to compile them, and it takes AGES on these ones...

          problem is that i couldn't address the problem, as rerunning script a few days after is totally fine... so, is something related to their maintainers and their updates to their packages, don't know what else to say...

          if someone wants an SBC (or a VM) to run just as a controller, demanding the actual sensor/actuator stuff to mqtt controlled nodes, these HW related stuff is useless...

          and in any case, we will NOT remove them, for sure: i think the best option is to add an item to menu, unchecked by default, to allow everybody who needs them to add, on his discretion...

  95. !!!!! SUCCESS !!!!!

    The script ran fine and all the utilities/programs installed OK and run fine from the web page interface.

    I wanted to get back here to thank Antonio for hanging in there to get the script working for me. Your expertise and patience are world class. Your help is very much appreciated.

    I wanted to also summarize what happened and point out a few things that could have shortened the time required to diagnose problems.

    My first run of the script resulted in a bad node-red install. I knew this because the web page interface would not launch NR. I was ask to upload the script log. Nothing in the log pointed to the problem. Then I was ask to run nrlog. The log showed a truncated error list and wasn't helpful. Turns out a complete NR log would have pointed to the problem. It was many days later that full NR install errors showed the problem was related to I2C. I discovered the problem after attempting a manual install of NR. The manual install of NR streamed many errors to the terminal indicating I2C was the issue. I did another clean op sys install, ran the script, edited the settings.js file to remove I2C, then rebooted. All was well. So the problem could have been resolved much quicker IF those NR errors would have been logged in the NR log. I'm not sure if that is a NR or script issue. I also think it would be a good idea to allow the user to disable I2C in the script, so an edit of the settings.js file is not required.

    Hardware and op system:
    Orangepi Zero with 512mb
    Armbian_5.38_Orangepizero_Debian_stretch_next_4.14.14.img

    Thanks again! Now I am off to understand and use all stuff I have installed! haha

    Cheers
    Bob

    1. my bad, i thought the error was recent just after boot, so nrlog (which is an alias for a longer command and that colors its output) was enough, if the error was in latest lines... instead, it would have been better asking for full node-red log file, indeed...

  96. just installed this on a pi 3b+ it took 35 mins to install and works great
    love all the new programs which it installed. another deep learning curve now

  97. I tried this last night on an old B+ with DietPi. It seemed to run OK but had to leave overnight to complete. Tried to access the Web page this morning with success but everything else seems to have failed. No time this morning to check more but will update later. Anyone else had success running the script on a B+?

    1. B+ is that the original pi? Ie not one of the version 2 variations? I don't test that far back as they are really not powerful enough. Maybe someone else has.Even if everything works you will be disappointed with performance.

      Pete

    2. never tested on such an old device, but it should be like an RPi Zero or the like... you have a log file in your home folder, take a look at there to see what happened... a night is TOO MUCH, to me you had network issues and it couln't download some stuff...

    3. Success! How bizarre. I came back from work. Started the Pi up and now everything is working. Just created a test flow for MQTT and it's spot on. Thanks very much for all your work on this! This Pi does have limited resources but will be fine for my tinkering.

  98. Hi Pete / Antonio,

    Great website / script.

    I did a fresh install last night of Raspbian Stretch Lite + the script on a RPI3B

    I'm also getting the corrupt /etc/rc.local file like Luca mentioned previously.

    from pi home directory as user pi did:
    wget --no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh
    bash ./script.sh

    Only defaults where used.

    On line 7 in /etc/rc.local, get an unterminated string:
    exit 0 " on success or any other

    Easy to fix, but not sure if others getting same error.

    Link to log of install.
    https://www.dropbox.com/s/ohxtszr6ktiq5gr/script.sh-2018-07-01_01h00m.log?dl=0

    Cheers

    Nigel

    1. no problems in log file, except that java repositories is changed again so no java installed... i'll have to take a look, hoping java8 is still there, as java10 has some problems... but we use it only for habridge AFAIK, so no big issues...

      about rc.local, please share the bugged one so i can take a look to what causes the corruption...

  99. FYI...

    Some new Debian updates (that included nodejs) broke node-red. After many hours of trying to determine what caused the break...I discovered it was the addition of the I2C stuff Peter added to the NR settings.js file. I removed the additions (I dont need I2C) and NR launched OK. There is probably a more elegant fix, but wanted to post this here in case someone runs into this same issue. Here is some details about the error message...

    Error loading settings file: /home/pi/.node-red/settings.js
    The module '/home/pi/.node-red/node_modules/i2c-bus/build/Release/i2c.node'
    was compiled against a different Node.js version using
    NODE_MODULE_VERSION 48. This version of Node.js requires
    NODE_MODULE_VERSION 57.

    1. if you have time and you want to, can you try this version of the script, restarting from scratch? The only difference is that i swapped nodejs from v6 to v8, which is the new stable and anyway we should move to it soon...

      https://pastebin.com/Gsy5EWQx

      i think it should work flawlessly, but if not we need to:
      1) move to v8 anyway, looking for eventual problems (i've none on my boards...)
      2) move HW related stuff on a new menu item (default UNCHECKED), so those who want just a bare IOT controller system or virtual machine and NO physical sensors/actuators should be just fine...

      if you look at other comments, mine too, you'll see recent problems with i2c, and with serial in the past...

  100. Yes it was the update to node.js LTS 8 that broke things. node -v shows v8.11.3 now. I also see some warning errors in the syslog file related to serial, etc, but they don't break NR. At this stage I don't want to do another clean install. Its been a struggle to get everything running. One of the issues was the OrangePi. It just wasnt stable. I dumped it and got a RaspberryPi 3 B+ and things have been MUCH better.

    I did a fresh reboot now and copied the NR startup lines from the syslog file. Here is a link to those NR errors...

    https://www.dropbox.com/s/6eshobtlgd5nwbg/NRerrors.txt?dl=0

    Hopefully this info will be useful to fix things.

    The log show there are some remaining warning errors due to epoll.node, sqlite3.node, serialport.node. It would be nice to clean up those problems...so any help on how to fix those would be appreciated.

    Cheers
    Bob

    1. Well, I just took Antonio's mod and installed the modified script on a fresh Raspberry Pi 2, with latest Stretch. After the usual RPI nonsense due to ssh not being set up by default along with root access, meaning I had to stick a monitor on the PI initially, Node-Red appears fine. Of course I had to copy my password from a previous installation. I could have sworn we put the code in there for Node-Red and ui password creation... anyway it works, no sign of any issues yet.

      1. ssh, just the usual: create an empty "ssh" file in the folder which you have access in windows itself, just after flashing the sdcard...

        indeed the node-red password creation is in script, from ages now... have to test in virtual machine, then... work for tomorrow, friends on the way now... 🙂

      2. but in the end, if you run my modded script, you end up with the recent nodejs version, from official site, nothing more or else... it's a part of what you do with that other script you run to update to version 8, which is specific to raspberry and made a mess on my rock64 when run...

    2. as i said, i've serial port and i2c problems very often, on various boards... and now that you named it, i had problems with sqlite, from time to time... unfortunately nothing we can do in script: the way all of these 3 packages are installed is just NPM, so it does what it should and install process is beyond our intervention... we should check on their site or github issue pages to see what they report...

      epoll: don't even know what it is or what does install it...

      about your "Yes it was the update to node.js LTS 8 that broke things"... what update? previous script installed version 6 and we never implemented an update script, beside the one found by Peter for raspberry...

    1. Peter, i see interesting stuff here: https://github.com/node-serialport/node-serialport#raspberry-pi-linux

      "If you're going to use sudo or root to install Node-Serialport, npm will require you to use the unsafe parameters flag.

      sudo npm install serialport --unsafe-perm --build-from-source
      Failure to use the flag results in an error like..."

      i think it's best to use this version for these problematic nodes (i2c, serial, sqlite are ALL, from time to time, error prone...), and remove them from the long npm install line we have right now... this way is SURE they're compiled for the specific board and right in time...

      1. so, splitting this line:

        npm $NQUIET install moment node-red-contrib-config node-red-contrib-grove node-red-contrib-diode node-red-contrib-bigtimer \
        node-red-contrib-esplogin node-red-contrib-timeout node-red-node-openweathermap node-red-node-google node-red-node-sqlite \
        node-red-node-emoncms node-red-node-geofence node-red-contrib-moment node-red-contrib-particle \
        node-red-contrib-web-worldmap node-red-contrib-sqldbs node-red-contrib-ramp-thermostat node-red-contrib-graphs i2c-bus \
        node-red-contrib-isonline node-red-node-ping node-red-node-random node-red-node-smooth node-red-contrib-npm node-red-node-arduino \
        node-red-contrib-file-function node-red-contrib-boolean-logic node-red-contrib-blynk-ws node-red-contrib-chatbot \
        node-red-dashboard node-red-node-darksky node-red-node-serialport node-red-contrib-owntracks node-red-contrib-opi-gpio node-red-contrib-alexa-local 2>&1 | tee -a $LOGFILE

        into these 2 lines:

        npm $NQUIET install moment node-red-contrib-config node-red-contrib-grove node-red-contrib-diode node-red-contrib-bigtimer \
        node-red-contrib-esplogin node-red-contrib-timeout node-red-node-openweathermap node-red-node-google \
        node-red-node-emoncms node-red-node-geofence node-red-contrib-moment node-red-contrib-particle \
        node-red-contrib-web-worldmap node-red-contrib-sqldbs node-red-contrib-ramp-thermostat node-red-contrib-graphs \
        node-red-contrib-isonline node-red-node-ping node-red-node-random node-red-node-smooth node-red-contrib-npm node-red-node-arduino \
        node-red-contrib-file-function node-red-contrib-boolean-logic node-red-contrib-blynk-ws node-red-contrib-chatbot \
        node-red-dashboard node-red-node-darksky node-red-contrib-owntracks node-red-contrib-opi-gpio node-red-contrib-alexa-local 2>&1 | tee -a $LOGFILE
        sudo npm $NQUIET install --unsafe-perm --build-from-source serialport node-red-node-sqlite i2c-bus node-red-node-serialport 2>&1 | tee -a $LOGFILE

  101. What if I execute the following on my existing system?

    sudo npm $NQUIET install --unsafe-perm --build-from-source serialport node-red-node-sqlite i2c-bus node-red-node-serialport 2>&1 | tee -a $LOGFILE

    You think the warning will be fixed and wont cause any NEW problems? Not that the warnings are a big problem...its just a clean startup might make me sleep better 🙂

    I will do a backup before...just in case.

    Bob

    1. you can remove those nodes, before, if you want... so you're going to install them fresh... and after that, run the command but as follows (those vars work only inside the script...)

      cd .node-red

      npm remove serialport node-red-node-sqlite i2c-bus node-red-node-serialport

      sudo npm -s install --unsafe-perm --build-from-source serialport node-red-node-sqlite i2c-bus node-red-node-serialport

      1. Well after much success with an RPI2, my RPI3 B+ installation was looking grim with no Node-red.. So....

        sudo npm install --unsafe-perm i2c-bus

        that worked but no serial..

        sudo npm install --unsafe-perm --build-from-source serialport node-red-node-serialport

        That worked..

        1. i'm testing in vm, where of course there's no hw, but i think we can safely add this mod to script... when we see nodes with warnings, compile from source does the trick...

          let me test the problems with nodejs v8 and i'll release the updated script ASAP

          please send me your logs to see what happened in your v8 install yesterday

          1. not a good day to test anything... tried in vm using ubuntu 16.04, italian mirror is corrupted and i had tons of errors 404... reverted snapshot and retried, switching to german repository, same errors... i'll test again in weekend... or have to test using debian...

  102. The following did not work.... (I did have to cd to the cd node_modules after cd .node-red)...

    cd .node-red

    npm remove serialport node-red-node-sqlite i2c-bus node-red-node-serialport

    sudo npm -s install --unsafe-perm --build-from-source serialport node-red-node-sqlite i2c-bus node-red-node-serialport

    All sorts of mystery messages spewed out. Looks like a lot of syntax errors? I let the it all run until complete, rebooted. NR did start but a lot of the flow nodes were screwed up.

    I restored the backup I had and all is back to what it was before the "fix". That is the good news. Backups indeed do work! Yeepie.

    I will live with the few warning when NR starts up. Maybe I should have coped all the install messages...but I didnt.

    Hope some of this helps you get things working.

    Cheers
    Bob

      1. Many of those messages are warnings not script related... but yes my confidence level in the change is not high yet hence leaving the original pre-node-8 version in place.

        1. On my way out for the morning but for the record, this morning I took the unmodified script, rpi3 b+ and made a single change to reference latest node. all worked except passwords not in the nr settings file - added those from an older installation and it looks like all is ok up to now.

          1. seems like some nodes are not ready to be used with node v8, so...
            about password, take a look at logs, recently i had issues with bcrypt (which is used to generate the passwords) even on v6, so that could be the problem...

            both, anyway (v8 and bcrypt) are not script related but due to the respective maintainers...

        2. sure, better stick with v6 for the moment, as it's an LTS version anyway... it works and there's no hurry to go to a newer version... in the end, we use the nodes (which use node undenneath...), not node itself 🙂

  103. Hi guys,

    Thanks for taking the time to make/update the script and help people to get it working.

    I've been trying for days trying to get it to work for me. I have a Raspberry pi 3B and I've tried the script with Dietpi and Rasbian Stretch lite and I've been having the same problem with both.

    script.sh: line 847: npm: command not found
    sudo: npm: command not found

    Then the script hangs at...
    Unpacking python3-rpi.gpio (0.6.3~stretch-1) ...
    Setting up python3-rpi.gpio (0.6.3~stretch-1) ...

    I would appreciate any help you could provide.

    Here is a copy of my log file. https://www.dropbox.com/s/h4pzllwvn7u7r99/script.sh-2018-07-13_05h06m.log?dl=0

      1. https://nodered.org/docs/hardware/raspberrypi

        and

        dceejay Leader
        July 11
        If that fails then delete
        /etc/apt/sources.list.d/nodesource.list
        And then re-run
        (The issue is that you are Nide 6, and pi have now released node8 and it starts to install that and while it should work... Obviously it isn't. Deleting the file above and rerunning will make my script think its not installed and will update to 8 "our" way.)

        1. Peter,

          Thanks for all of your hard work, even on the weekend I see.

          Your recommendation fixed my node-red issue but I ran in to a problem with settings.js. I did have to delete the nodesource.list file and rerun their install/upgrade command and then your script again. It worked but when attempting to load node-red I would get an error:

          $ node-red
          Error loading settings file: /home/pi/.node-red/settings.js
          /home/pi/.node-red/settings.js:251
          editorTheme: {
          ^

          SyntaxError: Unexpected token :
          at createScript (vm.js:80:10)
          at Object.runInThisContext (vm.js:139:10)
          at Module._compile (module.js:616:28)
          at Object.Module._extensions..js (module.js:663:10)
          at Module.load (module.js:565:32)
          at tryModuleLoad (module.js:505:12)
          at Function.Module._load (module.js:497:3)
          at Module.require (module.js:596:17)
          at require (internal/module.js:11:18)
          at Object. (/usr/lib/node_modules/node-red/red.js:115:20)

          I removed the extra }, just above editTheme (there are 2 sets back to back. That then gave:
          Error: Cannot find the module 'i2c-bus'

          So I commented the variable at the beginning and the call in httpStatic. That got me an error on 'moment':

          Error: Cannot find module 'moment'

          So commented that too and node-red would then start. I am sure I have broken more than I fixed but wanted to report my findings.

          Have a good weekend
          Glenn
          Alaska

  104. Hello peter, hello guys,

    I have problems using the script script.sh on a new Raspberry Pi 3B , OS stretch.

    script.sh: line 847: npm: command not found
    sudo: npm: command not found

    Then the script hangs at...
    Unpacking python3-rpi.gpio (0.6.3~stretch-1) ...
    Setting up python3-rpi.gpio (0.6.3~stretch-1) ...

    Please help to get it to work
    Thanks a lot.

    1. I don't see this being the script - I just set up 2 RPI3 B+, an RPI3 and an RPI2. There was an issue with Node the other day but that seems to be history - again it was not the script.

  105. I tried this yesterday as I managed to fry my install trying to update Node.js.
    First try, the script didn't work (I now think it was connection issues while installing).
    Second try, I used Dr Fraggle's script from his post on July 8 as I wanted Node V8 and that worked BUT none of my passwords that I set up during the script worked. I can get into Node-Red admin panel if I uncomment the login details in the settings.js config file (I tried to add my passwords there but that didn't work) but I cannot access the dashboard.
    Is there a way I can update the passwords and make them work? Or should I start from scratch and try the standard script again?

    1. as said, there were (or still are) problems with some remote repositories from where the script downloads the needed packages... if repos are offline or broken, errors occur... but it's not a problem of the script itself... take a look at logs in your home folder and search for "404"... if you find them, then some packages were not downloaded/installed... nothing we can do about it, unfortunately...

      1. I stepped through your script and found that bcryptjs didn't install so the password hashes weren't where they should be. I was able install it and do the steps required to create the hashes and copy them into the settings.js file.

        I'll be sure to back up my SD card this time so I can recover more easily next time I screw something up.

        Thanks for your work.

        Note: For some reason the log was owned by root and therefore hadn't been written to for the part of the install done by user pi.

        1. that package, together with others like i2c, sqlite, serial, is one of those that for some reason are failing recently to be downloaded... retrying will often solve, but you should know what to retry, as you cannot rerun script as it is...

  106. Sorry for the long post.

    @Steve - I was running the script on a DietPI distro and I saw the same problem where the log file in the home/pi dir was owned by root and had not been updated when the pi user ran the script.

    This happens when the root user runs the script (creating the pi user and copying the script log file to home/pi) and within the same minute the pi user tries to run the script again.

    There are 3 line in the script which I changed to avoid this.
    Line #288 - Added seconds to the temp filename so the pi user gets a unique name.

    Lines #365 and #375 - It looks like the intention was to have the copied log file owned by pi (because of the "chown pi.pi") but because the "-p" option was not used, the result of the "cp $LOGFILE /home/pi" command is that the /home/pi version is owned by root. I decided to reverse the commands do the "chown pi.pi" on the /home/pi version after the cp command.

    Here is my version in case anyone needs the details.

    Line#288
    LOGFILE=$HOME/$0-`date +%Y-%m-%d_%Hh%Mm%Ss`.log

    Line#365, 375
    cp $LOGFILE /home/pi && chown pi.pi /home/pi/$LOGFILE && chmod 644 /home/pi/$LOGFILE

    1. Sorry but there was a bug with my change to the script on line 365, 375.

      It is better to just use the "-p" option to the cp command and leave the rest of the line the same.

      Line#365, 375
      chown pi.pi $LOGFILE && chmod 644 $LOGFILE && cp -p $LOGFILE /home/pi

  107. I tried several times with DrFragle's script because I wanted to try out home assistant and it appears the nodes for that need node V8. Every time I wanted to add some nodes it would delete some others, so I went back to the official script. It failed a few times and I suspected it was due to flakey internet, but on inspection of the logs, it wasn't installing npm so would fail with unknown command when npm was called.
    I did an apt-get update and apt-get install npm before the script and it went through fine. I couldn't see a line to install npm in the script so I'm wondering if it used to be installed as a dependant, but now needs to be explicitly installed.

  108. I suspect the problem is in the script has been broken by the OS.

    I suspect that previously, npm was installed as a prerequisite of node.js and now that is not happening so the script will need to be modified to install npm.

    To test this, I just tried 2 installs on Raspberry Pi 3's at the same time to get around issues being caused by my dodgy internet connection - the only difference between them was that one was on wifi and the other cat5.
    Both started with a fresh SD card with the 6/18 version of raspbian. On both systems I ran apt-get update and apt-get upgrade then on one I installed npm.
    From there the script ran identically up until it got to installing nod-red when the system I hadn't installed npm on had an error.

  109. I ran the updated script 8-1-18 and everything installed.
    my password for node-red and dashboard are not working.
    any ideas on a workaround so I can log into node-red ect.

    Thanks for your hard work!
    Brad.

    1. No idea as I;ve no idea why your Node Red login failed.... unless.... You don't log in with PI or root - you log in with the admin name and pass you put in when running the script. Node-Red-Dasyhboard login is the user name and password you put in when running the script

          1. i see this at the end of node install, before nodered:
            E: Unable to correct problems, you have held broken packages.

            unfortunately no other info in logs... so, try using the v6 version script, not the test version as node8 still causes problems, as far as i can see...

            and... java repository seems broken, too:
            E: Package 'oracle-java8-installer' has no installation candidate

            both are not script related, but repository related... nothing we can do, beside stick with previous versions...

  110. just sent to Peter an updated script version, changelog:
    removed menu options for MC, CU, SCREEN, MPG123, as i've added them per default to the requirements apt-get line (who uses them will have it, who don't will not be harmed by their presence...)
    fixed java install on latest raspbian...
    added menu option HWSUPPORT (default UNCHECKED), that will enable i2c, gpio, serial, only for who needs it, reducing install time for those who just use script in a virtual machine or on an SBC just as centralized controller with no hw connected...
    and of course the mods of yesterday to fix settings.js problems with latest nodered

  111. Antonio/Peter,

    What is you recommended path once this is in an running. I have created a Ubuntu 16.04 VM about 18 months ago from the script and it has been running with no problems. Was thinking about upgrading to the latest Node.js - is it best to just handle all of those things by ourselves once the system is in place - or is there some magic in the script to handle in place upgrades ??

    1. script is NOT meant to be used for upgrades... it's only for 1st run, initially... other than that, most of the installed software is added via repositories, so you can just update your system via apt to have it upgraded... but nodejs will not be upgraded this way, and on purpose: every node should be recompiled, if you upgrade node...

      you can use my script to upgrade everything, not node, as said:
      https://gist.github.com/fragolinux/412aa50683eb32bb879a120c73110938

      as said, do backups...

  112. Yep already manage it all myself (snapshot before doing an upgrade) and all good. I was just interested if that use case had been handled or not - then everyone would stay pretty much in lockstep when asking you questions etc - obviously docker will make all of this a LOT easier

    Craig

  113. Just a heads up guys - not sure where it is coming from - I have a Ubuntu VM built from 16.04 from THE SCRIPT.

    I have been managing my own updates etc through webmin no problems.

    Today i went to install contrib-moment through the palette manager and it deleted a heap of the nodes that had been installed via the script - the only reason i picked it up - was the Darksky node i was using refused to deploy after i restarted Node-red.

    Apparently this could come down to the nodes that you provide as part of the script are not registered in package.json and there are changes between NPM5 and NPM6

    I recently upgraded to Node-red 19 and also NodeJS 10 so this may have triggered it.

    I have a snapshot of the VM from prior to the upgrade - will try and fire it up and see if installing the updated moment node causes the same issue and report back

  114. Dear Pete,

    Thank you for this excellent script which worked like a charm for me (Debian 8 running on Synology VMM).
    I am only having a small issue with NodeRed, it seems the created Admin account does not have the rights to install further nodes through the Palette Manager.
    So I tried to install the nodes from the root account via SSH but it completely messed things up and had to start over again (I love VM...).

    Log below. Could be obvious but I am (still) fairly illiterate on Linux, thanks for any help!

    Loic

    npm WARN checkPermissions Missing write access to /home/pi/.node-red/node_modules/bcryptjs
    npm ERR! path /home/pi/.node-red/node_modules/bcryptjs
    npm ERR! code EACCES
    npm ERR! errno -13
    npm ERR! syscall access
    npm ERR! Error: EACCES: permission denied, access '/home/pi/.node-red/node_modules/bcryptjs'
    npm ERR! at Error (native)
    npm ERR! { Error: EACCES: permission denied, access '/home/pi/.node-red/node_modules/bcryptjs'
    npm ERR! at Error (native)
    npm ERR! stack: 'Error: EACCES: permission denied, access \'/home/pi/.node-red/node_modules/bcryptjs\'\n at Error (native)',
    npm ERR! errno: -13,
    npm ERR! code: 'EACCES',
    npm ERR! syscall: 'access',
    npm ERR! path: '/home/pi/.node-red/node_modules/bcryptjs' }
    npm ERR!
    npm ERR! The operation was rejected by your operating system.
    npm ERR! It is likely you do not have the permissions to access this file as the current user
    npm ERR!
    npm ERR! If you believe this might be a permissions issue, please double-check the
    npm ERR!...

  115. the script has now a new option, default unchecked...
    you can use Caddy (https://caddyserver.com), a very lightweight http server, INSTEAD of Apache for the services that the script is meant for: so, you can have phpliteadmin, phpsysinfo and the basic services access page Peter created on this http server, using less resources...
    BARE IN MIND the capital "INSTEAD" i just wrote... you have to be SURE to UNCHECK apache and CHECK caddy if you want this, as they do the same stuff and use the same tcp ports and local folders, you can't have them both... well, technically you can if you know what you're doing and install caddy yourself, but that's an other story...

    1. unfortunately, i screwed up my gists, wrongly deleting some of them, so the caddyfile is now lost... if any of you has installed caddy from the script in the days between september 16 and 18, please share this file again:
      /etc/caddy/Caddyfile

  116. just found a nice, modern web gui for sqlite: sqlite-web (https://github.com/coleifer/sqlite-web)
    very easy to install:

    sudo pip install --upgrade pip
    sudo pip install sqlite-web
    sqlite_web -H ip.ip.ip.ip -p 81 -x -P dbs/iot.db

    change the ip.ip.ip.ip to the ip of your board, otherwise the web will be available only on 127.0.0.1, so only from the board itself, locally...

    -x is to avoid it opening a browser automatically

    -P is to ask for a password to protect the web interface (you can delete the -P option if you don't need a password)

    -p is to change the http server port (default if -p is missing is 8080)

    last parameter is the db to open, in this case Peter's default one

    that's it 🙂

  117. Dear Pete,

    Thank you very much for this very useful script. I ran it on a VM Debian 8 and it worked like a charm. I have a small issue however, hope you can help:

    Somehow in NodeRed it seems I can only login with the admin account (not the User account although the User account logs in fine through SSH to the VM) and the admin account will not let me install new nodes. See log below.

    Maybe I missed something?

    Thanks in advance

    Loic
    __________________________________________________
    npm WARN checkPermissions Missing write access to /home/pi/.node-red/node_modules/bcryptjs
    npm ERR! path /home/pi/.node-red/node_modules/bcryptjs
    npm ERR! code EACCES
    npm ERR! errno -13
    npm ERR! syscall access
    npm ERR! Error: EACCES: permission denied, access '/home/pi/.node-red/node_modules/bcryptjs'
    npm ERR! at Error (native)
    npm ERR! { Error: EACCES: permission denied, access '/home/pi/.node-red/node_modules/bcryptjs'
    npm ERR! at Error (native)
    npm ERR! stack: 'Error: EACCES: permission denied, access \'/home/pi/.node-red/node_modules/bcryptjs\'\n at Error (native)',
    npm ERR! errno: -13,
    npm ERR! code: 'EACCES',
    npm ERR! syscall: 'access',
    npm ERR! path: '/home/pi/.node-red/node_modules/bcryptjs' }
    npm ERR!
    npm ERR! The operation was rejected by your operating system.
    npm ERR! It is likely you do not have the permissions to access this file as the current user
    npm ERR!
    npm ERR! If you believe this might be a permissions issue, please double-check the
    npm ERR!...

        1. use script as it is right now, the recent addition was a lighter http server (caddy), which was optional and alternative to apache... unfortunately, while cleaning my github gists i wrongly deleted its configuration file, so i've to redo it... but, NO ETA, as it's not an essential service and apache worked and works just fine... if i've spare time in weekend, i'll try to have it back, but priority is my docker setup, right now, and i can work on it only in weekends as i've busy days at work and in the evening all i want is everything but a linux terminal...

  118. There may be an issue with the script. This is with a fresh install of 2018-04-18-raspbian-stretch an a Raspi 3.

    I get this error when issuing this command node-red-start

    Node-RED is not yet fully installed
    /usr/bin/node-red-start: line 91: update-nodejs-and-nodered: command not found

    I've tried multiple times with the same result.

  119. I'm going to get to the bottom of this... I have a Rasperry Pi 3 Model B here and this morning am doing a fresh install including the latest full Raspian (26/7/2018) which I'm currently downloading from https://www.raspberrypi.org/downloads/raspbian/

    I now use the free ETCHER program on Windows to grab Raspbian Stretch (full version zip download) in this case. This does NOT need to be run as Windows administrator and does a better job of checking SDs than Win23DiskImager. I'll take this opportunity to recommend using only genuine Sandisk Utra or Samsung Evo/Evo+ SDs. I generally use the 16GB variety and that leaves plenty of room for expansion/wear. As I have a few 8GB SDs lying around, this morningI'll be using one of thosea a Sandisk. Etcher incidentally does the copy then checks the SD, without user interaction. All this on my not-stunningly-fast Windows 10 PC takes several minutes to download then several minutes to install on the SD. This is not something I relish doing frequently. Windows takes great delight in suggesting formatting the SD just after you've waied ages to program it up - ignore that - if Etcher says "success" - you're done.

    The script for clarification is today best grabbed directly onto the PI as root user using:

    wget --no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh

    i.e. after grabbing in this case Raspbian operating system (a massive 1.6GB) and getting Raspian running, I then (using a screen and keyboard on the PI graphical desktop) enable remote root access and pass control to a PC (using Mobaxterm on the PC) at which point user ROOT on the PI can wget the script and run it. That takes a short length of time after which I switch to user pi and run the script fully. More on this later.

  120. Following on from my comment earlier this morning, this is the bit about setting up Raspberry Pi that I hate - plugging in a keyboard and monitor.

    This is a good time to set the timezone and password and of course connection. I use hardwired ethernet, not WIFI.

    In the RPI menu - go to configuration - I enable ssh, I2c and serial and disable serial console - others may have different requirements. This is a good time to point out that the Pi has built-in pullup resistors for I2c, other boards often do not.

    First reboot coming up.

    At THIS point the PI wants me to change password to something more secure.

    Done...

    Now I pop up a terminal on the Pi - and key in this mess...

    sudo nano /etc/ssh/sshd_config

    I need this info:

    https://raspberrypi.stackexchange.com/questions/48056/login-as-root-not-possible

    This will let me log in remotely as rooot after another operation. In the mentioned file, uncomment #PermitRootLogin and change the rest of the line to yes

    Save with control-o (Enter) then control-x

    Back at the terminal I change the ROOT password using sudo passwd root

    I then make a note of the IP address (I could at this point have made this fixed), sudo reboot - that's it, no more messing with Pi keyboard and screen.

    I turn off the Pi, disconnect keyboard, screen and mouse. From now on I can do the lot from the comfort of my PC.

    Using (in my case) Mobaxterm on the PC (I used to use WinSCP which is also great) I'm now talking to the Pi from the comfort of my desk as root user for a brief time.

    On the Pi (as user root) we need to key in:

    wget --no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh

    and then bash script.sh

    You will note that quite quickly, some preliminary software will load on the Pi. Within a minute or so you'll be asked to log out as root and run the script as pi. I find it easy to simply switch to another session in Mobaxterm - logging in as user Pi.

    Repeat:

    bash script.sh

    This time will take longer but most of it is just waiting (or more sensibly, having a coffee break). First however, in the menu that pops up on the screen, select any changes we want from defaults. using cursor and space bar. I enable hardware support, enable habridge and grafana, remove wolfram and office as I have no plans to use the Pi graphical interface from now on. When done use TAB, Ok

    We then get to (optionally change the hostname and) update USER and ADMIN names and passwords at the end of which the script runs with no further interaction and could take a while.

    Indeed it will - which is why I'm stopping this comment here.

  121. i think swapping comments on line 789-790 and 811-812 should solve, as this way it's restored node V6 that we used for months... what i don't understand is why v8, which both used flawlessly various times, without issues, last weekend, now fails...

  122. It turns out - and Antonio is right here - that while I tried to use the latest NODE 8.x, Node-Red does not yet always seem to work correctly with this... indeed the IBM upgrade script uses Node 6.odd - so for now I've modified the script back to Node 6.x - if you've installed the lot on a PI and Node-Red isn't having it, you can firstly use

    sudo rm -r /home/pi/.node-red

    then run the script again as PI user but this time with ONLY node + node-red ticked - and the update admin and user passwords options ticked.

    Just done it myself on a new setup. The script by now refers to NPM 6.x, not 8.x

    All worked but no extra nodes and no serial port so in /home/pi/.node-red I opened (in a text editor) package.json and replaced it's (almost non-existent) content with:

    {
    "name": "node-red-project",
    "description": "A Node-RED Project",
    "version": "0.0.1",
    "dependencies": {
    "i2c-bus": "^4.0.1",
    "node-red-contrib-bigtimer": "~2.0.2",
    "node-red-contrib-blynk-ws": "~0.7.1",
    "node-red-contrib-chatbot": "~0.15.0",
    "node-red-contrib-diode": "^1.0.1",
    "node-red-contrib-esplogin": "~1.3.7",
    "node-red-contrib-geofence": "^0.2.4",
    "node-red-contrib-i2c": "~0.5.2",
    "node-red-contrib-ibm-watson-iot": "^0.2.8",
    "node-red-contrib-influxdb": "~0.2.1",
    "node-red-contrib-owntracks": "0.0.2",
    "node-red-contrib-play-audio": "^2.3.2",
    "node-red-contrib-sqldbs": "^1.1.8",
    "node-red-contrib-timeout": "~1.1.0",
    "node-red-contrib-web-worldmap": "~1.3.8",
    "node-red-dashboard": "~2.9.8",
    "node-red-node-arduino": "0.0.18",
    "node-red-node-email": "~1.0.4",
    "node-red-node-ping": "0.0.15",
    "node-red-node-random": "^0.1.0",
    "node-red-node-serialport": "^0.6.8",
    "node-red-node-smooth": "^0.1.0",
    "node-red-node-sqlite": "~0.3.5",
    "sqlitedb": "^1.0.0"
    }
    }

    Remember I'm doing this as user pi. If you don't need serial and sqlite - leave them out as they take ages to update - no binaries out there apparently so a lot of "rebuilding" goes on.

    I then made sure Node-Red was stopped (node-red-stop) - and that I was in the /home/pi/.node-re directory before (as pi user) typing:

    npm rebuild
    and
    npm upgrade

    This takes ages to run non-interactively. sqlite and the serial port are a pain here.

    After some time - that did the trick (don't comment about warnings, out of our hands). REBOOT.

    All working. npn 6.41

    I have it running successfully here on a PI3 and we'll do a much neater job of this - but I move from Spain to the UK this coming weekend then have work to do on the house (including wiring in broadnand and computers - and a floor etc - so consider this a stop-gap.

    Don't ask what "npm audit" does, I didn't run that.

    1. a BIG advice: that command will remove .node-red and should be done ONLY on freshly installed boards, and only if you had problems with node v8 and want to go to v6 without rerunning full script from scratch... otherwise you'll loose all your nodes and configs...

      ah, for the missing hw support, you must check the relative menu option to have it, of course...

      and for package.json, instead of all those nodes listed in script, we can have a package.json which installs them as you just did... tip: you can strip out versions and replace them with * to have a more generic version, like this i'm using on my docker setup: https://github.com/fragolinux/DockerIOT/blob/master/nodered/package.json

      the npm audit will check all nodes if they have security issues and correct them, it's a new addition in npm 6 i think:
      https://docs.npmjs.com/getting-started/running-a-security-audit

      https://blog.npmjs.org/post/173719309445/npm-audit-identify-and-fix-insecure

    2. nodered guys have proper, official, nodered images, and they too stick to v6 (compare size of different v6 and latest tags...)... https://hub.docker.com/r/nodered/node-red-docker/tags/

      i'm customizing these official images for my setup adding your nodes, and here, too, it took ages to install serial and sqlite... BUT... we could pack everything once done and release customized images on dockerhub, at the end...

      this way reinstalling will just be downloading a new image and deploy a container, but this implies that every time some new node is released, we should re-release the customized image (the new layers of it, really)...

      i think that the time needed to install could be just acceptable, as it's the same if you run classic setup or dockerized one... which, in the end, is just the same process, just running on different places (host or container), always on some linux...

      better stick to stock image and add a custom layer with our usual nodes, less work for me, and a little more time for who uses it, yup 😀

      1. this is the line in latest script that downloads the official nodejs setup script, and as you can see, it points to v6 (v8 of course has an 8 in the url):
        curl -sL https://deb.nodesource.com/setup_6.x > nodesetup.sh

        we're not reinventing the wheel, it's the official way to install it, taken from nodejs site... but we noticed in last times that this, too, brings to a v8 setup once finished installing, but this time v8 works, on the contrary of the v8 script that seems to work very seldom...

  123. Hi Peter, I have a little issue that I wonder if you might have an answer to?

    I have been using ESPLOGIN and your software on a few devices for a few years now, and they how worked pretty much flawlessly. I recently started converting a few older switches (Sonoff S20) onto your software, where they used to run TASMOTA. The ROMS where erased first, and the switches work as expected now. However, I have noticed that two switches (from eight in total) seem to be running an Access Point and show a port 80 service on 192.168.4.1. I haven't been able to access the server. When initially burnt with your software I was unable to access the web config on these two, so setup was done via serial connection.

    Have you any idea why this may be? It seems odd that they have an AP but are also active on my own network too...

    Thank you for your (and all other contributors!) efforts with the entire ESP-GO suite.

  124. Hello Pete

    Thanks for all the info and work you do.
    Hope sunny Spain is that.

    After doing a new sd card on raspian stretch and using script from Sept
    I copied file flows.json from my previous setup to .node-red in new
    I have the same problem as Loic74 September 19, 2018 at 7:32 pm
    post to which I believe you gave him a solution

    I cant seem to find your reply
    Can you enlighten me please?

    Not certain if its a problem credential password or permissions.

  125. God that was quick, all working now

    thanks a lot.

    I have spent all day setting up and reading the comments etc
    for latest script- This was the only problem I encountered by installation

    regards from sunny Germany

    1. for unknown reasons, some nodes change owner from time to time, or you used sudo when it was not supposed to use it, so the nodes were then owned by root instead of you... as it happens to me, too, i've added that line to my full system update script a lot of time ago, so i've not to think to it every time... it updates system and nodes, automatically:

      https://gist.github.com/fragolinux/412aa50683eb32bb879a120c73110938

      (remove last 4 lines if you don't use homeassistant)

  126. Hi folks.

    I have been told to get my system as Peter's so I can run his flows and see what they are like on MY machine.

    To do this I believe I have to run the script mentioned way up near the top of this.

    (for clarity)
    wget --no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh

    Ok.
    There are things in there about if it is running on a PI. Though I have a lot of Pi's, for now I want to get it working on a bigger machine.
    (x86 NUC)

    So I have the script.
    Reading it.... yeah, ok. A bit above me.
    I am running/installing the stuff on a VM running Ubuntu. Not Raspbian, and there is no PI user.

    Reading in the script I see:
    IMPORTANT:-
    ## 1. Run ONLY as ROOT (or PI user on Raspberry Pi). If ROOT, PI user will be checked/created for you as the main
    ## script must be run as pi.
    ## User PI must be in the SUDO group. Other groups are added by the script.

    Then a few lines down I see:
    6. Do not access this script as SUDO.

    I'm lost.

    And if I am not installing it on a PI, I am at a loss how to do it.

    I am trying to get Peter's "base line" of what he has installed, so I can see what the stuff looks like, but am just not "getting it" with the instructions as they are.

    Thanks in advance.

    1. when i use an ubuntu vm for this stuff, i always create a "pi" user just during setup, so it's already there when i run the script... as script was originally meant for raspberry, default user "pi", it works ONLY with that user, unfortunately...

      otherwise, do as such if you already installed and have a different user:

      sudo -s -H
      # give YOUR ACTUAL USER password at step above... after that, you ARE root

      # start from here running the usual setup
      cd && wget --no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh
      bash script.sh

      # after it created the user pi, logout and login with that user, or do:
      su - pi
      bash script.sh

      or just reinstall ubuntu creating a pi user during setup, as i said... DON'T use ubuntu 18.04, only 16.04 is supported...

      1. Thanks.

        16.04..............

        Drats.

        I started with 17.01 when I got the NUC.

        I know it isn't too difficult to get the older version.

        Ok, that is a whole new can of worms with which to deal.

        I don't get why 18.04 isn't "supported"? You mean for the build on which it is built?

        1. it isn't supported by Peter's script, as none of the sbc has something different from debian stretch or ubuntu xenial, so why adapting the script to something that very few will use?

          You can use it on newer ubuntu, but you're on your own with all the problems you'll find, don't ask for help... i don't plan to port it on 18.04 in the near future, or ever... because i see docker as the future of this...

          and when my docker setup will be usable, there will be no differences as containers ignore the underlying OS, so you'll be able to run docker on 18.04, too...