A Christmas Script

tmp81FFOver the weekend and beyond, reader Antonio Fragola (Mr Shark) and I have been working on a couple of things together (no we’re not in the same country – Skype). Regular reader will know about the script (original blog material now ditched though the old script is still on BitBucket) that a couple of us originally developed to put Node-Red and several other packages onto the Raspberry Pi for home control purposes.

Well, that original script mutated to run on several different boards and my one big gripe has been the amount of interaction once the script starts.  THIS one is simple – and after reading some comments about how good Xenial (Ubuntu) is – we’ve also made the script compatible with this. So Raspbian, DietPi (Debian) and the Armbian versions of Debian and Xenial (Ubuntu).

Before I begin – no, we’re not a support shop – you may or may not end up having to tweak things for your system but please don’t write in to say “I tried this on a Mac and it doesn’t work”).

The level of interaction is now zero thanks to a new menu-driven approach and many improvements including using a VM (virtual machine) to test the whole thing at accelerated speeds. We’ve also put in some extras.

What is this about?

What we have here is an explanation of how to get Xenial running in a VM – the purpose of this is merely as a base to test the new script mentioned below which will install somewhat more than the original script (designed originally for Raspberry Pi but now working on several Debian-based devices and now Xenial) – namely:

  • Node-Red at port 1880 and lots of useful NODES
  • Mosquitto with Websockets
  • NODEJS/NPM
  • Webmin at port 10000
  • Apache at port 80
  • HA-Bridge at port 82 (suitable for Amazon Alexa)
  • SQLite and PHPLiteAdmin
  • MC File manager and editor
  • Lots more – see the menu screen

Typically this setup would be used as the basis of a nerve-centre to control and monitor home control gadgets such as the ESP8266 units we discuss in this blog, Sonoff controllers and much more. Armed with this software, controlling the home is easy – and more importantly can be done YOUR WAY.

The Virtual Machine

Let’s take a look at what we did – you can of course duplicate all of this as you go along. We used the excellent VMware of VirtualBox on Windows PCs. This explanation covers VMware.

So – first things first – as I’ve done a video of the Virtualbox installation complete with sample use of Node-Red, here we’ll discuss the VMWARE version.  Both of these were done as test mechanisms before using the script on real devices such as the Orange Pi Zero, Raspberry Pi, Odroid and other devices discussed at length in the blog.

We loaded up a copy of VMware (I used VirtualBox but the procedure is similar – VMware player is free of you can use the full commercial workstation) and made sure it works. That much is up to you. There is a wizard for creating new virtual machines -  we chose a CUSTOM machine. Steps from that point on were:

NEXT - NEXT – “I will install the operating system later” – NEXT
Linux / Ubuntu 64-bit – NEXT – d:\test  - NEXT
Number of processors – 1  - number of cores – 1 – NEXT
Memory for this machine – 1024MB – NEXT
Use Bridged Connection - NEXT
I/O Controller type LSI Logic - NEXT
Disk Type SCSI - NEXT
Create a new virtual Disk - NEXT
12GB (I used 16GB – 8GB would have done) – Store virtual disk as a single file - NEXT
At this point we gave the file a name – in our case d:\test\Ubuntu 640bit.vmdk -  NEXT – FINISH

Under settings – hardware – CD/DVD we ticked “Use ISO image file” and selected the file we loaded from the Ubuntu website – “Ubuntu-16.04.1-server-amd64.iso” – OK.

At this point you’ll note reference to AMD64 – that had me as well but it works a treat on INTEL hardware.

Start off the VM – you get asked which language to use (ENGLISH) and various options will come up – the first being “Install Ubuntu Server” – that’s the one.

Once again you’ll get a language question – English. Location – in my case “United Kingdom” – if you don’t fit into the rules use “Other”.

Time Zone in my case – EUROPE  and then country. Locale – in my case United Kingdom.

Hostname – we went with the default “Ubuntu” – but call it whatever you want.

At this point you get asked to create a user – it is important you make a user “pi”. Enter password also (though the new script will create user PI if it does not exist).

tmp6049Home encryption – we said no. When asked if the time zone was correct we said yes.

Partitioning – “use entire disk” (remember this is a virtual disk – not your real disk). You are then asked to confirm erasing disk – go with that. You then get asked to confirm AGAIN. YES.

HTTP Proxy just leave blank – CONTINUE.

Install security updates automatically.

Add OpenSSH server and CONTINUE (important for remote access)

Install GRUB bootloader – YES.

Installation is complete – CONTINUE. Let it run.

And voilà – one working copy of Ubuntu. You have a pi user – at this point you’ll want to give ROOT user a password (sudo passwd root).

You need ROOT user for the next step.

No matter what machine you are using – you need a “pi” user and to make life simple – and compatible with that of the original Raspberry Pi user, a little command line work is needed. The rest is handled in the script.

So as ROOT user…

adduser --quiet --disabled-password --shell /bin/bash --home /home/pi --gecos "User" pi echo "pi:password" | chpasswd usermod pi -g sudo echo "pi ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/pi chmod 0440 /etc/sudoers.d/pi

The FIRST and SECOND lines are not needed if you followed the instructions above as you already a “pi” user.

tmpB7DFSo now we have a pi user able to do SUDO commands.

The BASH Script

This part is about running the script and assumes a “pi” user with SUDO capability.

Create a file called script.sh in the pi directory and populate it from the script. https://bitbucket.org/snippets/scargill/94yxL

Ensure it has the right permissions.

and that is that. as PI user, in the /home/pi directory you should now have this script. DO NOT run this as ROOT user.

At this point BACK EVERYTHING UP to avoid disappointment – no, REALLY.

Execute as:

./script.sh

You will see a menu which will vary slightly depending on which system you are using – you won’t be asked about GPIO on a Raspberry Pi for example.

tmpE673

For the VM we left everything as you see here and tabbed to OK.

At this point you will be asked for a USER and ADMIN names (which can be user and admin if you like) and corresponding passwords. From there on – everything should happen with no user interaction.

After maybe 15 minutes – or an hour – depending on your system – you’ll see an invite to REBOOT – so REBOOT. When the board/VM boots up you will find hopefully a working system. Node-Red runs at port 1880, Webmin at port 10000 (Webmin needs Pi or ROOT username and pass until we figure out how to add in the ADMIN user).

You should have Apache at port 80  HA-BRIDGE (if installed) will run at port 82 – It seems that Amazon Echo is quite happy with this but the Google product wants port 80!

Node-Red should include MQTT which will need admin username and password as will Node-Red itself. USER name and password is used by Node-Red Dashboard.

There is also the excellent MC – a file manager and editor.

MC and MC EDIT

I hope this brief introduction sets you off in the right direction and of course no doubt there will be discussing in the notes below.

Update Dec 29: Early start this morning - just for the sake of it I’ve just taken the Ubuntu desktop image, installed it on a laptop (a very old laptop) and ran the script – perfect – everything installed. My thanks to Mr Shark for filling in the blanks in my Linux knowledge.

Update 05 Jan 2017: At the time of writing I'm having issues with PIXEL in VirtualBox and DEBIAN on VirtualBox in terms of Apache not loading.. I'm sure we'll sort that - meantime some useful links.   Debian  may not some with SUDO depending on which installation - here's a fix link that worked for me in my VirtualBox image.. If SSH setup won't allow remote access using passwords, here's a link that helped me.  Pixel of course will run in Virtualbox but by default it runs of a virtual DVD which is about as much use as an ashtray on a motorbike as nothing gets saved. Once running however it is a breeze to move over to the Virtual hard disk - you'll find that part way down the comments in this link.  So right now I have VirtualBox running Xenial, Pixel and Debian and the only issues I have are with Pixel and Debian - Apache - watch this space.  I’ve also improved the input for user and admin.

Important: Whatever you are using should have the SUDO command built in – it may not be obvious but Debian often does NOT have this installed by default – sort this FIRST. You should also make sure you have Pi user with SUDO group privilage – PI already has this of course and you may have added a user Pi – if not..

wget https://bitbucket.org/api/2.0/snippets/scargill/E5rG7/master/files/makepi
su -c "bash makepi"

Or, Here’s a useful link – takes seconds and is easy…  basically – as root you can do - apt-get install –y sudo….. and you should make sure that your PI user is part of the sudo group.

Facebooktwittergoogle_pluspinterestlinkedin

106 thoughts on “A Christmas Script

  1. I've been doing lots with running Ubuntu(Kubuntu,Lubuntu,Xubuntu,etc) from liveCD ISO files on my HD and with a partition or USB thumb drive labeled casper-rw there's an overlay(persistent) filesystem capability too.

    So my thought is, instead of installing Ubuntu in a VM, why not download the Ubuntu ISO file and setup a VM with the CDROM pointing to the file. Then create a small 8GB hard disk image and boot the vm into the live mode. partition and change the label of the 8GB partition to casper-rw and reboot. Now you have a quickly booting Ubuntu system to experiment loading up your home automation stuff(docker, node red, etc).

    For the noobs it gets them through the installation process fast and lets them play in Ubuntu land for a bit. If they like it, they can then generate another VM but this time go for the installation of Ubuntu so it's more able to handle updates.

  2. Hi Pete / Antonio,
    I've used Pete's script several time in the older format and had few problems. I ran the new one this morning with the menu system and found this problem on my Raspberry pi 3 loaded with Raspbian:-

    The script does not detect my Raspbian OS correctly. It thinks that it's not a genuine Raspbian OS. Here's my OS:-

    pi@raspberrypi:~ $ cat /etc/os-release
    PRETTY_NAME="Raspbian GNU/Linux 8 (jessie)"
    NAME="Raspbian GNU/Linux"
    VERSION_ID="8"
    VERSION="8 (jessie)"
    ID=raspbian
    ID_LIKE=debian

    I changed this line in your script:-
    if [[ $OPSYS == *"RASPBIAN"* ]];then
    to
    if [[ $OPSYS == *"raspbian"* ]];then
    and it detects my OS correctly.

    Many thanks to both of you for your great work.
    Keith

    1. Correct, Keith - I've now put a case converter in to make all results upper-case - so that should not happen again - please report back any other issues as this is all new. I've also just put a handler in for the CANCEL option which now works.

  3. Hi Pete,
    Here's another one:-

    Line 255
    MYMENU=$(whiptail --title "Main Raspberry Pi Selection Checklist" --radiolist \
    This only allows you to check ONE option.

    I think it should be :-
    MYMENU=$(whiptail --title "Main Raspberry Pi Selection Checklist" --checklist \

    Thanks for the 'CANCEL' handler.

    Keith

  4. Pete,

    There's a typo that prevents NPM from updating:-

    Line 268
    "upgradednpm" "Upgrade NPM to latest version " ON \
    I think that this should read:-
    "upgradenpm" "Upgrade NPM to latest version " ON \

    to match line 715
    if [[ $MYMENU == *"upgradenpm"* ]]; then

    1. Also, lines 721 and 722 should be swapped?

      720 # Add CU to enable serial VT100 mode for terminals
      721 printf "${ICyan}============================ Installing CU...etc
      722 if [[ $MYMENU == *"installcu"* ]]; then

  5. New update - should now check to ensure it is user PI that has logged in! If root and there is no user PI it should create one and ask you to log out and try again. !! Thanks to Mr Shark for this one.

  6. Hi Pete,

    Nice video. Always good to see things in action. Can I suggest maybe a video on messing with Node Red Dashboard, be nice to see its capabilities.

      1. Hi John - did you use the latest script as of today... I just installed on an Ubuntu real machine - and Debian and Ubuntu VMs without issue... but not on a Pi3. The issue with NPM was resolved ages ago - perhaps on a Pi the existing version is causing a problem?

        As for X terminal if the terminal is too small when you start the script it is no good - that is a "feature" of the menu script... just make the terminal a little larger.

          1. Hi Pete,

            I'll help with the testing on the Pi 3. I'm flashing a fresh Jessie image to a sd card now. Does it matter if I run apt-get upgrade before I run the script or should I just start with the default image?

          2. Pete, I'm happy to help test with Raspberry Pi if you want. I have a lot of experience with shell scripting too.

            "The issue with NPM was resolved ages ago"
            On a pi with the latest Jessie, NPM is not installed (or if it is, it's not on $PATH)

            1. I THINK that the Pi issue is now resolved and 2 changes made today to SSH and some helpful text for menus - please report only issues with software from this evening's date ONWARDS - and thanks to EVERYONE for their help - shell scripting experience always comes in handy - as you can see!

  7. I've been trying this script on a Raspberry Pi 3 with Jessie downloaded today.
    A couple of points:
    1 I see the whole menu if in command line mode, but on an X terminal a few characters are missing on the left hand side - so you can't select the options.
    2 I get a lot of errors eg Failed to fetch http://packages.dotdeb.org/dists/jessie/InRelease Unable to find expected entry 'all/binary-armhf/Packages' in Release file (Wrong sources.list entry or malformed file)
    3 As with the old script, it attempts to use NPM to install things and fails because NPM is not installed. "sudo: npm: command not found" Presumably NPM is already installed with other operating systems? Could it be added to the prerequisites?
    4 A logfile is useful to trawl through for error messages. You can get one with the command ./script.sh 2>&1 | tee script.log

    Regards,
    jb

    1. both right... just sent patches to Peter, i did tests on dietpi and ubuntu, all right, missing raspbian, if someone want to test, in latest script change this (from line 367):

      if [[ $OPSYS != *"UBUNTU"* ]]; then
      #enable php7 on debian jessie - no need for xenial
      echo "deb http://packages.dotdeb.org jessie all" |sudo tee /etc/apt/sources.list.d/dotdeb-php7.list
      echo "deb-src http://packages.dotdeb.org jessie all" |sudo tee -a /etc/apt/sources.list.d/dotdeb-php7.list
      wget https://www.dotdeb.org/dotdeb.gpg -O - | sudo apt-key add -
      sudo apt-get update
      fi

      with this:

      if [[ $OPSYS == *"DEBIAN"* ]]; then
      #enable php7 on debian jessie DIETPI
      echo "deb http://packages.dotdeb.org jessie all" |sudo tee /etc/apt/sources.list.d/dotdeb-php7.list
      echo "#deb-src http://packages.dotdeb.org jessie all" |sudo tee -a /etc/apt/sources.list.d/dotdeb-php7.list
      wget https://www.dotdeb.org/dotdeb.gpg -O - | sudo apt-key add -
      sudo apt-get update
      fi
      if [[ $OPSYS == *"RASPBIAN"* ]]; then
      #enable php7 on debian jessie RASPBIAN
      echo "deb http://repozytorium.mati75.eu/raspbian jessie-backports main contrib non-free" |sudo tee /etc/apt/sources.list.d/mati75-php7.list
      echo "#deb-src http://repozytorium.mati75.eu/raspbian jessie-backports main contrib non-free" |sudo tee -a /etc/apt/sources.list.d/mati75-php7.list
      sudo apt-key adv --keyserver pgpkeys.mit.edu --recv CCD91D6111A06851
      sudo apt-get update
      fi

      unfortunately, dotdeb repository has not arm packages, had to find them elsewhere...

      1. Right everyone - ... menu fixed - narrowed in for smaller screens and simplified.

        I've implemented the changes above suggested by Mr Shark. I have to go out - I will not be back until tomorrow mid-day - here's hoping these changes sort the Pi out.

        1. i think in the raspbian section, if wolfram and openoffice are checked to be removed, this should happen BEFORE initial general update/upgrade in prerequisites... no need to update something and then remove it after 😉

    2. 1 Peter reduced menu size

      2 my patch in comments here, and already included in latest online script, solved php7 on raspbian

      3 no npm problems, just installed latest raspbian (november 2016), launched script, everything is up and running...

      i made install in 2 batch, just to check everything working...
      1st run: modpass prerequisites mosquitto apache screen mpg123 phpsysinfo addindex installcu installmc wolfram office
      2nd run: modpass (to reinsert password for node-red and webmin) nodejs nodered webmin java upgradenpm
      not a single problem...

    3. pi@raspberrypi:~/.node-red $ npm -v
      4.0.5
      pi@raspberrypi:~/.node-red $ node -v
      v4.7.0
      pi@raspberrypi:~/.node-red $ npm view node-red version
      0.15.2
      pi@raspberrypi:~/.node-red $ npm list --depth=0
      /home/pi/.node-red
      ├── bcryptjs@2.4.0
      ├── node-red-contrib-bigtimer@1.4.86
      ├── node-red-contrib-blynk-websockets@0.0.3
      ├── node-red-contrib-boolean-logic@0.0.3
      ├── node-red-contrib-config@1.0.1
      ├── node-red-contrib-esplogin@1.1.9
      ├── node-red-contrib-file-function@1.1.2
      ├── node-red-contrib-gpio@0.9.1
      ├── node-red-contrib-graphs@0.3.5
      ├── node-red-contrib-grove@1.0.7
      ├── node-red-contrib-isonline@1.1.15
      ├── node-red-contrib-ivona@0.0.2
      ├── node-red-contrib-moment@2.0.0
      ├── node-red-contrib-npm@1.0.0
      ├── node-red-contrib-particle@0.0.4
      ├── node-red-contrib-timeout@1.0.4
      ├── node-red-contrib-web-worldmap@1.0.22
      ├── node-red-dashboard@2.2.1
      ├── node-red-node-arduino@0.0.12
      ├── node-red-node-darksky@0.1.14
      ├── node-red-node-emoncms@0.0.11
      ├── node-red-node-geofence@0.0.16
      ├── node-red-node-google@0.1.0
      ├── node-red-node-ledborg@0.0.17
      ├── node-red-node-openweathermap@0.1.14
      ├── node-red-node-ping@0.0.13
      ├── node-red-node-pushbullet@0.0.10
      ├── node-red-node-random@0.0.8
      ├── node-red-node-smooth@0.0.9
      ├── node-red-node-sqlite@0.1.2
      └── raspi-io@7.1.0

  8. Very interesting stuff! As I have an issue with the loss of mouse events in Chrome on Jessie I will try another installation on an 8Gb SD with the proposed Ubuntu image.
    I am a bit confused about all usernames and passwords needed. Should I use PI or pi as a user name? An overview of all needed un&pw's where to use what and when to create would be appreciated.
    Another item I have not touched is access from outside of home through internet, what to do on router(s) to reach my home server from outside?
    Keep on going the good work!

      1. Will give it a try with VMWare and that takes some time. The description is very extensive so I might be able to come all the way through. So no big hurry anyhow, but thx for the swift attention to this.

        1. Does not show the same options as described although the same Ubuntu image.
          There comes a question BIOS or EFI and default is BIOS, but I have no idea whether this is right. I can try EFI for a better result, but that is a shott in the dark. No question about adding SSH or GRUB.
          Display settings are not usable. No display is detected. It uses host settings and does not completely fit in window of VMware.
          How to change to ROOT user? I am a Windows user. This will take some googling.

          1. full album of the VM creation in Vmware: http://imgur.com/a/X8xLi
            you could use Vmware Player, too, as it's free and despite the name, allows to create VMs, too...
            in vm, no real difference between bios and efi, choose what you want...
            this setup uses UBUNTU 16.04.1 LTS SERVER, and so, no desktop environment... if you use a desktop iso, be sure to install vmware tools at the end, to enable graphic support (it's in the vmware player/workstation menus)...
            root is useless in ubuntu, it has no password, when you need to do something ubuntu asks for YOUR password, as first user created has SUDO rights, so can do administrative tasks without ever know root password or become root... but, many windows users used to be "administrator", tend to enable it... bad security habit...

            in a system with root enabled, an attacker only need its password to be harmful... in a system with a regular user, attacker has to know user name, user password, and THEN root password, if used has no sudo rights... but, your choice...

            1. Thx.
              I installed on W10-64 the desktop version. You are showing install on linux. Diff is menu with SSH and you did not cross Samba server, probably needed for W10 PC's on the network.
              Pete is advising some work in ROOT. Should I remove Root user?
              I will start from scratch again in EFI mode with one of the premade VM's you mention hereafter.

    1. the username and password that are asked by Peter's script are used for:
      user: only for node-red dashboard
      admin: mosquitto, phpliteadmin, node-red main interface
      webmin uses it's own access, which is root or pi, and their respective password

          1. ALWAYS use latest script, NEVER ask for help using old ones, as this could bring problems, to you and to us not understanding what's causing what else... in case, just say I'M USING VERSION FROM XYZ (DATE)... but better using latest, as said...

    2. for the internet access... without discussing security issues which will take way too much time... you need to create "virtual servers" or "nat" or "port forwarding" or whatever they are called in your internet router... if you have access to its admin interface, or you need to ask your ISP to do it... and open, for example:
      external port 1880 --> yourdeviceIP port 1880 (for node-red)
      external port 1883 --> yourdeviceIP port 1883 (for mosquitto)
      external port 80 --> yourdeviceIP port 80 (for apache)
      leave webmin and ssh away, for your good, if you don't know what you're doing... and use strong passwords...

  9. I've done another install on my Pi3 with a clean image of Jessie. I used the defaults on the script and I got through to the end without any errors. I did get a warning of an unused variable 'status' when the sqlite was installing though.

    Comments:-
    I though my Pi had crashed to start with until I saw the -qq switches on the update and upgrade. I'll remove the -qq next time I run it.

    I did like the elapsed time feature of the older scripts (now gone).

    Some packages recommend 'apt-get autoremove'. Maybe put this at the end of the script?

    Otherwise - awesome!! Well done Peter & Antonio.

    1. Hi Keith

      Can you clarify - Pi3 - you say Jessie - was this a generic Jessie or are you talking about Raspbian - very important to clarify.

      The qq options are to try to quieten down the "noise" coming off the installs - after all a BOATLOAD of messages telling us what has been successful are actually of no use - it would be nice indeed to figure out a reliable way of ONLY showing error messages - as they often get lost in the "noise". The old elapsed feature - yes I agree with you but that was based on a notion of interaction - incidentally the old script is still there for those who want to use it.

      Apt-Get autoremove - I see no reason why we should NOT do a cleanup at the end - Antonio - your thoughts?

      1. yes, as said on skype, when you install "something", it brings together an other lot of packages... when you remove "something", those addons packages are left behind... to remove these packages not needed anymore (as the packages using them was already removed), you do the "autoremove"... to remove the downloaded (let's say the setup ones, in windows style) and already installed packages, you do the "clean", too

      2. Hi Peter,

        Sorry, yes it was Raspbian straight from the raspberrypi.org website.

        I liked the elapsed time as it gave some idea of performance. You often mentioned how many minutes it took to install on your blog.

        I'm a ardent console watcher, so like to see the 'noise' 🙂 It gives me a warm feeling that things are still moving along as opposed to being stuck.

        1. in me, too 😀

          we added a (no)quiet menu option yesterday, you can use it now... but time depends on your menu selection, so compare is valid only on default selections... and can change from microsd to microsd on the same platform...

  10. Hi Peter,
    Great video & script. I've installed Ubuntu Xenial Desktop as a VM in Virtualbox and followed your instructions. However, the script has stalled at the point below. Tried twice but getting the same result.
    ================= Installing Node-Red ====================
    make: Entering directory '/usr/lib/node_modules/node-red/node_modules/bcrypt/build'
    CXX(target) Release/obj.target/bcrypt_lib/src/blowfish.o
    CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt.o
    CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt_node.o
    SOLINK_MODULE(target) Release/obj.target/bcrypt_lib.node
    COPY Release/bcrypt_lib.node
    make: Leaving directory '/usr/lib/node_modules/node-red/node_modules/bcrypt/build'
    make: Entering directory '/usr/lib/node_modules/node-red/node_modules/ws/node_modules/utf-8-validate/build'
    CXX(target) Release/obj.target/validation/src/validation.o
    SOLINK_MODULE(target) Release/obj.target/validation.node
    COPY Release/validation.node
    make: Leaving directory '/usr/lib/node_modules/node-red/node_modules/ws/node_modules/utf-8-validate/build'
    make: Entering directory '/usr/lib/node_modules/node-red/node_modules/ws/node_modules/bufferutil/build'
    CXX(target) Release/obj.target/bufferutil/src/bufferutil.o
    SOLINK_MODULE(target) Release/obj.target/bufferutil.node
    COPY Release/bufferutil.node
    make: Leaving directory '/usr/lib/node_modules/node-red/node_modules/ws/node_modules/bufferutil/build'

  11. Just tried it on a fresh install of Armbian (Ubuntu 16.04.1 LTS) on my Olimex A20-Olinuxino-Micro, and it seemed to work flawlessly (meaning I didn't notice any major errors, and all the components seem to be functioning), in case anyone has the same platform and is interested in trying it.

  12. I just registered to share a comment about your script. A few days ago I booted the Raspbian "Pixel for PC" from a USB flash drive onto an old Toshiba Satellite laptop (M115-S3094). This afternoon I tried running your script on that machine. It worked! I rebooted and now have node-red running at 127.0.0.1:1880 with your large array of nodes to try out. Webmin is also working. I haven't tried anything else yet. Just wanted to let you know that the script works even with "Pixel for PC". I have a very limited knowledge of Linux so to me this seems quite a trick! Thank you and Happy New Year!

      1. as far as i could see in the few minutes i used it, it has no ssh server... so
        sudo apt-get install ssh # (to install ssh server+client)
        sudo passwd # to give root a password
        then it should work, as always

      2. I struggled with scp on the Pixel OS - always timed out when trying to send a text file to my Windows 10 machine. I was able to setup and use a PuTTY connection. I had to turn on the SSH server (OpenSSH_6.7, already there, shown in Webmin) and allow authentication by password. Finally, I installed WinSCP and was able to drag and drop a text file from the Pixel machine into a directory on the Windows 10 computer.

          1. Thank you for your help. I located the ssh block in the script, then I viewed the sshd_config file and saw the appended lines about UseDNS and ClientAlive... I have learned that Pixel for PC version 1.1 has the ssh server off by default. I don't understand Linux command syntax well enough to know if that would have any effect on the script. I do see the last line in the block restarts the ssh server, but the Webmin interface said the ssh server was off. Thanks again, no response necessary. The problem is most likely operator error. Right now I have a working system and plenty to learn!

            1. use raspi-config: https://www.raspberrypi.org/blog/a-security-update-for-raspbian-pixel/
              or, as per this link https://www.raspberrypi.org/documentation/remote-access/ssh/ :
              "For headless setup, SSH can be enabled by placing a file named 'ssh', without any extension, onto the boot partition of the SD card."

              these are the lines i added/modified to sshd_config:

              PermitRootLogin yes # to allow root login directly... i would not do this, but some wants...
              UseDNS no # to reduce the time between inserting user and the following password prompt
              TCPKeepAlive no # these 3 lines are for disable ssh timeouts
              ClientAliveInterval 30
              ClientAliveCountMax 100

  13. Just completed a rather long script install on a Raspberry Pi 2 having added a timestamp to the beginning of each section - I seem to recall some folk liking to keep track - also added (thanks Antonio) a progress bar. The screen is still WAY too noisy for me but we're working on that.

    The script despite taking ages, worked perfectly - bog standard Raspberry Pi2 with the November Raspbian.

    The FriendlyArm S2 also works but I'm just kitting that out with a new eMMC-based Debian to try again, leaving it to run overnight. I'll do a writeup on this board soon, probably tomorrow.

  14. Thanks Peter,

    Trying to run the script on a Virtualbox running MX-Linux (Jessie) and cannot get pst the npm installation part. Attached image shows the errors. I've tried a couple of times with no luck.

    Any ideas?

    1. Clearly, NPM was not installed. So you need to look back in that installation and see the earlier section to find out why not.

      Interestingly the only issue I'm currently having on VirtualBox is with Apache - in both the PIXEL and DEBIAN 8.6 images, Apache won't load and so of course PHPLITE INFO won't load (I'm sure we'll fix that today and that it is to do with getting the right installation image) - but NPN on VirtualPC - Pixel, Debian and Ununtu are all fine

      If anyone wants to jump in on this and help - by all means.. I'm thinking on the Apache thing we're going to need a menu option so the system knows this is a VM on a PC... but that still leaves the question of which apt-get etc.

        1. I've purged the virtualbox OS and started over. Get the same problem. npm seems not to be insstalling. The text below is what comes prior to the image II posted just above here.

          ====
          ==== Installing Installing Node-Red - Total Time: 00h:10m:02s
          ====
          sudo: npm: command not found
          --2017-01-05 12:04:48-- https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/nodered.service
          Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.20.133
          Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.20.133|:443... connected.
          HTTP request sent, awaiting response... 200 OK
          Length: 888 [text/plain]
          Saving to: ‘/lib/systemd/system/nodered.service’

          /lib/systemd/system/node 100%[===================================>] 888 --.-KB/s in 0s

          2017-01-05 12:04:48 (14.6 MB/s) - ‘/lib/systemd/system/nodered.service’ saved [888/888]

          --2017-01-05 12:04:48-- https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/node-red-start
          Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.20.133
          Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.20.133|:443... connected.
          HTTP request sent, awaiting response... 200 OK
          Length: 2111 (2.1K) [text/plain]
          Saving to: ‘/usr/bin/node-red-start’

          /usr/bin/node-red-start 100%[===================================>] 2.06K --.-KB/s in 0s

          2017-01-05 12:04:49 (9.68 MB/s) - ‘/usr/bin/node-red-start’ saved [2111/2111]

          --2017-01-05 12:04:49-- https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/node-red-stop
          Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.20.133
          Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.20.133|:443... connected.
          HTTP request sent, awaiting response... 200 OK
          Length: 226 [text/plain]
          Saving to: ‘/usr/bin/node-red-stop’

          /usr/bin/node-red-stop 100%[===================================>] 226 --.-KB/s in 0s

          2017-01-05 12:04:49 (2.63 MB/s) - ‘/usr/bin/node-red-stop’ saved [226/226]

          Failed to get D-Bus connection: Unknown error -1

            1. This Debian Jessie, distro is MX-Linux https://mxlinux.org/ . Tried both 32 and 64 bit versions.

              The computer is a Panasonic Toughpad with the same distro as the Host, using Virtualbox.

              I'm willing to test!

              Let me download the Ubuntu. Never been a fan of Ubuntu, but I will give it a go.

              1. please, while reporting bugs, EXPLICIT what script version you are using (and try to use ALWAYS the latest...), and which distro... if it's not 1 of those I listed, JUST SAY IT, we have no crystal balls to guess, nor we can make script work on EVERY single version of distros out there...

                1. Repaired this part of the problem, but editing the /etc/ls-release file. Changed the Metamorphosis to jessie, that way nodesetup recognizes the distro and proceeds.

                  Still can get past the installation of the nodes tho. Appears to a systemd problem.

                  Fun playing.

  15. apache+php7 install on standard debian+pixel sorted out... they report 32bit arch as i686... patch sent to Pete, which is trying pixel now, i tried debian, everything fine now...

    npm, still don't know, as it's not affected by php part of the script...

  16. I just finished running your script in Virtualbox 5.1.12 and everything appears to have worked flawlessly. I have used Virtualbox many times over several years for various reasons so that part was easy. The script itself took less that 15 minutes but I did not time it to get an exact time (Win 7 i5 2500K host). My "real" server is running Ubuntu 14.04 desktop and I'm looking to upgrade it to Xenial. I have been using MySQL vs SQLlite but your script has everything else I use plus more.

    Thanks for a great way to test things before I try upgrading my real server.

    John

    1. Hi john

      Well, the purpose of using SQLITE is to keep writing down to a minimum for SD memory on little SBC boards such as the pi - for simple logging, MYSQL though it would be my preferred DB is well over the top. Still - no doubt you can install that separately on your VM - and if you look at the script itself you'll see it isn't that hard to add sections - just be aware if you add anything to the menu - you need to change 2 values at the top of the menu to make it larger (because it isn't intelligent enough to autosize).

      1. Just getting back to the virtualbox installation of the script on Xenial and I can't seem to find Bigtimer in node-red. It is in .node-red/node_modules and I was able to update it to 1.4.88 using npm. However it still does not appear as an available node on the node-red web page. esplogin, grove and timeout appear under the scargill heading but no bigtimer. I don't understand enough about node-red to diagnose such issues. Can anyone point me in the right direction?

  17. PLEASE NOTE ALL - a BUG crept into NODE-RED-CONTRIB-BIGTIMER - I have updated this and fixed this bug which would have stopped bigtimer running.

    To fix on existing setups, go to your .node-red directory (probably /home/pi/.node-red
    and as PI user, from there merely...

    npm install node-red-contrib-bigtimer

    Update already but please wait until 9am today (Friday 6/1/2017) or later. The version you should end up with is 1.4.88 or later.

  18. Has anyone successfully run the script on a Pi 1 loaded with DietPi. I'm having no luck.
    "npm command not found"
    And yes I have a PI3 running well, but I have a couple of early PIs which I would like to give to my son.

  19. Hello

    Thanks for your work.

    Two comments :

    For real Raspberry, I would recommend to add to your script the log2ram tool. This is helping to save SD card and avoid corruption (logs are stored in RAM and dump to SD only once an hour per default). -> https://github.com/azlux/log2ram

    For your Debian stuff, you should have a look to the new X86 Raspberry/Pixel official image (if you've not had the info yet). This is made by raspberry org themself to run on PC or MAC and this should be pretty easy to have it on VM with persistence. This will be as close as possible to the regular raspbian ... -> https://www.raspberrypi.org/blog/pixel-pc-mac/

    Happy new year.

    Sebastien

    1. Hi there Sebastien

      Thanks for that and happy new year. Raspberry Pi - do you want to work up the exact instructions to add Logram and put in here why it's a great idea - if it is reliable we can always addt it to the Raspberry Pi menu!!! I'm all for offering to ditch logs actually - ie run in RAM but don't save. I only ever seem to use logs to check why something went wrong - not really interested weeks later and it would lengthen life of SD ??

      Debian - I have PIXEL running in VIRTUALBOX - works a treat - and the script works perfectly on it - it's a good quick way to get Debian and of course includes SUDO which Debian often doesn't . Don't get me wrong however, it is NOT a Raspberry Pi emulator... sound controls are missing - there is no RASPI-CONFIG and no hardware - like ports or anything... Shame - had they thought it through - a little usb board with ports on, made to LOOK like you have ports - would have been nice. Still - it is free and it works. Shame they're not ready to make it work on other boards because some of the alternatives to Pi, while having great hardware have really rubbish software.

      1. pixel raspbian was not meant for virtual machine (so the extra steps needed to have it working in a vm - why not doing as ubuntu, an icon on the desktop of the live cd, INSTALL, done???), and having it on actual physical hardware can bring audio and other stuff working... for ports, you can just passthrough them to the hypervisor of choice, not always easy but doable... clearly no gpio or other hw, only boards you can already have working via usb, like an arduino via firmata, or other boards Pete already tested in his blog entries in the past...

    2. frankly, the only advantage raspbian has is that's official... i don't care about the gui, i use ssh into my system every time, so pixel is useless for me... and, to be honest, it seems windows 95 with its icons... better than the former ones, but quite oldstyle anyway... but everybody says LOOK'MA, DAS ICONS! (there are TONS of better looking icon themes out there, from DECADES...)

      oh, and pixel has no clear instructions to put it in a vm, you have to dig into comments in official blog page to know how to do...

      raspbian has nodered and other stuff in it, but usually is old itself, too...

      ubuntu has newer packages already in its standard repositories, while dietpi has the best starting point for SBC, with low memory, low space needed, logs in ram, and many other optimizations...

      once setup, are all the same, you don't bother about what's underneath, your interest is in the service you use above the OS... which are all the same...

      dietpi is sure my distro of choice, right now...

      if nodered is your sole purpose, http://thethingbox.io is a good choice, too...

    3. Sebastien, this looks interesting.

      Along these lines I've been using CIFS to access disk space on my network shares (CIFS was easier to explain to friends than NFS or AFS). For when I must have local storage I dropped on a cheap 120G SSD drive and cross link it to the drive. I need to sit down and figure out how to mount the SSD as the writable part and the SD as just readable.

  20. Hello Peter

    I have to admit that I am stumped. I have been using Mosquitto for about 6 months but recently I had a crash, lost my installation on my Pi 3, and reinstalled all. Now I cannot get Mosquitto to work. I get this message when I enter this:

    sudo mosquitto_sub -d -t Humidity

    the error I get is:

    Connection Refused: not authorised.

    I used the script to install Mosquitto and Node-Red. Can you point me in the right direction please? I have searched the web but have not made any progress for the past several days. As I said, prior to the crash I had collected 100's of megabytes of data and controlled lighting in my home. Now, I cannot get Mosquitto to work at the simplest level.

    Regards,
    Bill Miller

  21. just to report, the script worked on Orange pi zero with Armbian_5.24_Orangepizero_Ubuntu_xenial_default_3.4.113.7z
    reported execution time 46 minutes.
    Thankyou 🙂

    1. Let me know how your WIFI works - particularly recovery from access point failure (just unplug) and general reliability... that is my one reservation about the Zero and I had no luck at access point failure recovery using Armbian (fine on hardwired ethernet).

  22. Ping times are awful (anywhere from 3ms to 2000ms; mostly 200-300ms).

    Wifi comes back up on restarting my bthhv2b. But, my wireless router is NOT my dhcp server, so it may not be an equivalent test..... Saw your topic on the armbian forums...

    ahhh.. just looked at logs. It did not reconnect to the SAME router (I have 2, and it knows about both...), so not a suitable test.... Interesting that it's happy to migrate though.

  23. looking through available source & firmware, looks like it's going to be wired only for me for the moment; no time to port wifi drivers this week.....

Leave a Reply

Your email address will not be published. Required fields are marked *