The Script and RPI 3 Update

Peter ScargillESP-12, ESP-12E, ESP-12F, ESP12, ESP12E, ESP12F, Raspberry Pi 3, The Script53 Comments on The Script and RPI 3 Update

Having started putting together “the script” a long time ago, long before the Raspberry Pi 3 came out – and having developed it to handle a wide range of boards and scenarios including of course the Pi3, I found it interesting to return today to installing “the script” on the Raspberry Pi 3 – and of course I wanted to ensure that nothing had changed recently which might stop others doing the same.

Oh! While I’m on – the ESP8266 code – just updated it – got LOTS of RAM back, added in a message on power up to indicate why the board is starting… and re-thought a power up message that wrote to FLASH (and hence could have worn the FLASH out after long periods of using deep-sleep restarts. All updated and OTA available.

Right – where was I… yes… Raspberry Pi 3

From a convenience perspective, most of the small boards I’ve dealt with lately have SSH enabled at start as well as root access. The Pi3 setup (Raspbian) currently has neither. So in order to install “the script” easily,  I found myself having to plug in a monitor and keyboard and use the desktop to enable I2c and enable SSH. Simple enough.

I then used Mobaxterm (could have used WinSCP) to access the board from my PC – I find that easier than messing around with a small keyboard on the actual PI and swapping screens over etc. At that point I realised that remote ROOT access was disabled. I know there are those who will say this is a security issue but personally I’d be happier with (as happens elsewhere) initial access and THEN an offer on first setup – to disable it unless instructed otherwise.

Anyway, off I went to sudo nano /etc/ssh/sshd_config to place/update an entry “PermitRootLogin yes” then reboot to ensure I could remote in as root and then change the password for both pi and root.

I then found “apt-get update” failing miserably because the guys at Raspberry Pi seem to have decided to use IPV6 without a fall-back position.  Adding “ipv6.disable=1” to the first line of /boot/cmdline.txt file seemed to fix that. I took the opportunity to remove the serial debug entry while I was on.

I know, IPV6 etc is long overdue etc. etc. but I’m certainly not about to spend hours adjusting my router and talking to my WIFI provider just for one project that insists on IPV6. Life is short and I simply wanted to install some software and not get side-tracked – no other board I know of currently presents this challenge.

So two reboots later I was on my way to installing “the script” which has operated flawlessly on just about every other setup for some time now.

Now, the script started up just fine (installing as root, only to be quickly told to run again as user Pi – this is deliberate as this is handy for those systems which initially don’t have a PI user which needs to be set up with a range of group access permissions and we want to keep things consistent).

While installing Node-Red I noted a whole boatload of warnings (not errors) to do with the serialport – you would think that on the most popular of boards on the latest operating system setup, this would have been resolved a while ago – not that deprecation warnings do a lot of good except look awful. Unless you want to get into obtaining the source and fixing it (hence turning an evening’s project into a day’s project) for not much apparent benefit, all these warnings seem to serve little purpose but to confuse the beginner.

Facebooktwitterpinterestlinkedin

53 thoughts on “The Script and RPI 3 Update”

  1. Thanks Antonio! V6 is working as supposed! 72hours lost but at least it works now! Thanks to pointing out! Would love to donate you some Rpi 3B+ for you just in case … 😉 PM me your address if you agree

  2. Interesting – I am messing with a pair of Raspberry Pi 3 B+ and they have the very latest Stretch installed, the script with the latest Node is on there – 8.12 and npm version 6.4.1 – Node-Red version 0.19.2 and all is working well.

  3. More info:

    pi@raspberrypi:~ $ cat /etc/os-release
    PRETTY_NAME=”Raspbian GNU/Linux 9 (stretch)”
    NAME=”Raspbian GNU/Linux”
    VERSION_ID=”9″
    VERSION=”9 (stretch)”
    ID=raspbian
    ID_LIKE=debian

  4. I am pretty sure you know, but in case you do not – The script does not install Node-red any more. Was using it few weeks earlier without an issue – did you changed a lot in it or it’s due to new Node-red or due to Stretch updates or…? Node-red is not installed by default in Stretch anymore as well – maybe it causes all the issues?

    I am using Raspberry Pi 3B+ and 2018-06-27-raspbian-stretch. Getting the same Error all the time:
    script.sh: line 1153: npm: command not found

    If any other info can help – would love to provide

      1. and this is not related to you board, as you said in previous comment that you’re using a raspberry 3b+ so the lines that should be triggered by script are the ones between 809 and 820 of actual version… BUT, there are chances that as you have the 3b+ version, something in it could be changed and so script tries to execute wrong install…

        post output of these lines, run it on a rpi3b+

        . /etc/os-release
        OPSYS=${ID^^}
        DISTRO=$(/usr/bin/lsb_release -rs)
        CHECK64=$(uname -m)
        echo -e OPSYS: $OPSYS
        echo -e DISTRO: $DISTRO
        echo -e CHECK64: $CHECK64

        1. pi@raspberrypi:~ $ echo -e OPSYS: $OPSYS
          OPSYS: RASPBIAN
          pi@raspberrypi:~ $ echo -e DISTRO: $DISTRO
          DISTRO: 9.4
          pi@raspberrypi:~ $ echo -e CHECK64: $CHECK64
          CHECK64: armv7l

    2. use latest version of script, don’t know which one you’re using, as line 1153 has NOT npm commands in it… and those lines are ok and works, just tried in a linux shell… they have to download the correct binary version for the given architecture

          1. I guess you are looking for these:?

            ############################################################
            Installing NodeJS

            ## Installing the NodeSource Node.js 8.x LTS Carbon repo…

            ## Populating apt-get cache…

            + apt-get update
            Hit:1 http://raspbian.raspberrypi.org/raspbian stretch InRelease
            Hit:2 http://archive.raspberrypi.org/debian stretch InRelease
            Reading package lists…

            ## Confirming “stretch” is supported…

            + curl -sLf -o /dev/null ‘https://deb.nodesource.com/node_8.x/dists/stretch/Release’

            ## Adding the NodeSource signing key to your keyring…

            + curl -s https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add –
            Warning: apt-key output should not be parsed (stdout is not a terminal)
            OK

            ## Creating apt sources list file for the NodeSource Node.js 8.x LTS Carbon repo…

            + echo ‘deb https://deb.nodesource.com/node_8.x stretch main’ > /etc/apt/sources.list.d/nodesource.list
            + echo ‘deb-src https://deb.nodesource.com/node_8.x stretch main’ >> /etc/apt/sources.list.d/nodesource.list

            ## Running `apt-get update` for you…

            + apt-get update
            Hit:1 http://archive.raspberrypi.org/debian stretch InRelease
            Hit:2 http://raspbian.raspberrypi.org/raspbian stretch InRelease
            Get:3 https://deb.nodesource.com/node_8.x stretch InRelease [4,647 B]
            Get:4 https://deb.nodesource.com/node_8.x stretch/main Sources [762 B]
            Get:5 https://deb.nodesource.com/node_8.x stretch/main armhf Packages [1,002 B]
            Fetched 6,411 B in 1s (3,364 B/s)
            Reading package lists…

            ## Run `sudo apt-get install -y nodejs` to install Node.js 8.x LTS Carbon and npm
            ## You may also need development tools to build native addons:
            sudo apt-get install gcc g++ make
            ## To install the Yarn package manager, run:
            curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add –
            echo “deb https://dl.yarnpkg.com/debian/ stable main” | sudo tee /etc/apt/sources.list.d/yarn.list
            sudo apt-get update && sudo apt-get install yarn

            Selecting previously unselected package nodejs.
            (Reading database … ^M(Reading database … 5%^M(Reading database … 10%^M(Reading database … 15%^M(Reading database … 20%^M(Reading database$
            Preparing to unpack …/nodejs_8.12.0-1nodesource1_armhf.deb …
            Unpacking nodejs (8.12.0-1nodesource1) …
            Setting up nodejs (8.12.0-1nodesource1) …
            Processing triggers for man-db (2.7.6.1-2) …
            E: Unable to correct problems, you have held broken packages.
            /usr/bin/npm -> /usr/lib/node_modules/npm/bin/npm-cli.js
            /usr/bin/npx -> /usr/lib/node_modules/npm/bin/npx-cli.js
            + npm@6.4.1
            updated 1 package in 29.73s
            ############################################################
            Installing Node-Red
            sudo: npm: command not found
            –2018-09-21 04:43:15– https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/nodered.service
            Resolving raw.githubusercontent.com (raw.githubusercontent.com)… 151.101.84.133
            Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.84.133|:443… connected.
            HTTP request sent, awaiting response… 200 OK
            Length: 917 [text/plain]
            Saving to: ‘/lib/systemd/system/nodered.service’

            0K 100% 6.76M=0s

            2018-09-21 04:43:16 (6.76 MB/s) – ‘/lib/systemd/system/nodered.service’ saved [917/917]

            –2018-09-21 04:43:16– https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/node-red-start
            Resolving raw.githubusercontent.com (raw.githubusercontent.com)… 151.101.84.133
            Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.84.133|:443… connected.
            HTTP request sent, awaiting response… 200 OK
            Length: 4608 (4.5K) [text/plain]
            Saving to: ‘/usr/bin/node-red-start’

            0K …. 100% 1.57M=0.003s

            2018-09-21 04:43:17 (1.57 MB/s) – ‘/usr/bin/node-red-start’ saved [4608/4608]

            –2018-09-21 04:43:17– https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/node-red-stop
            Resolving raw.githubusercontent.com (raw.githubusercontent.com)… 151.101.84.133
            Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.84.133|:443… connected.
            HTTP request sent, awaiting response… 200 OK
            Length: 226 [text/plain]
            Saving to: ‘/usr/bin/node-red-stop’

            0K 100% 2.89M=0s

            2018-09-21 04:43:17 (2.89 MB/s) – ‘/usr/bin/node-red-stop’ saved [226/226]

            ############################################################
            Installing Nodes (could take some time)
            script.sh: line 855: npm: command not found
            script.sh: line 863: npm: command not found
            sudo: npm: command not found
            Reading package lists…
            Building dependency tree…
            Reading state information…
            python-rpi.gpio is already the newest version (0.6.3~stretch-1).
            python3-rpi.gpio is already the newest version (0.6.3~stretch-1).
            0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
            Created symlink /etc/systemd/system/multi-user.target.wants/nodered.service → /lib/systemd/system/nodered.service.
            ############################################################

            1. something failed in installing nodejs or npm, so everything else related failed, too… someone else please, on a raspberry, possibly a 3b+, try latest script and let me know… just a few days ago we tested same script on some boards with Peter, with no issues…

              Preparing to unpack …/nodejs_8.12.0-1nodesource1_armhf.deb …
              Unpacking nodejs (8.12.0-1nodesource1) …
              Setting up nodejs (8.12.0-1nodesource1) …
              Processing triggers for man-db (2.7.6.1-2) …
              E: Unable to correct problems, you have held broken packages.
              /usr/bin/npm -> /usr/lib/node_modules/npm/bin/npm-cli.js
              /usr/bin/npx -> /usr/lib/node_modules/npm/bin/npx-cli.js

                1. try if you can, i don’t know, as said we tested everything last days with Peter, only problems where with caddy because me deleting the wrong file on my github, we didn’t touch nodejs and peter didn’t have issues…

                    1. start again, same script, but BEFORE RUNNING, change on line 812 that 8 to a 6, so to install previous nodejs version… this v8 is causing too much problems, but in weekend with peter we have NONE on this version… unbelievable… let us know if with v6 is fine…

  6. Hi Everyone, I ran the script on my new RaspberryPi3 and all seems OK except for the Mosquitto installation. I cant get the broker and client parts to work together. Is there any issues with installing mosquitto on to Raspbian stretch desktop?

    Sorry Im a noob with all this so you would have to ease me in to it 🙂

    1. OK scrap that……i had to enter the admin and password in to the MQTT nodes then it all works! thank you.

      1. Hi again,

        I’m a slow learner here and keep picking up and putting this thing down after I get a head ache. I’m trying to test the MQTT out by opening two separate Putty windows and typing the following in to each, my problem is that I don’t receive the “message” but see an error:

        I assume due to user and password issues………..?

        Putty #1 window:
        mosquitto_sub -d -t armtronix_mqtt

        Client mosqsub/1394-raspberryp sending CONNECT
        Client mosqsub/1394-raspberryp received CONNACK
        Connection Refused: not authorised.

        Putty #2 Window:
        mosquitto_pub -d -t armtronix_mqtt -m “Hello World”

        Client mosqpub/1398-raspberryp sending CONNECT
        Client mosqpub/1398-raspberryp received CONNACK
        Connection Refused: not authorised.
        Error: The connection was refused.

        Both windows have the above errors but I don’t know how to get past this point…..any help please 🙂

          1. Hi Pete,

            Great to hear from you how things are on the mend now.

            Pete, I’m a complete newbee at all this so I don’t know how to structure the MQTT command to include the user and password……..any help would be great.

          2. Hi again Pete,

            Would the command be like this:

            mosquitto_pub -t ‘#’ -u ‘username’ -P ‘password’ -d -t armtronix_mqtt -m “Hello World”

            and

            mosquitto_sub -t ‘#’ -u ‘username’ -P ‘password’ -d -t armtronix_mqtt

            I’m currently at work so will test this tonight……

  8. Peter thank you for all you do to help those just starting out. I’m having a little operator headspace issue. Installed the latest Raspbrian on a RPi 3 which creates a pi user, droped the script.sh in pi’s home directory gave it exacute and tried to run it and this is what I get. Any help would be greatly appreciated.

    root@rwpi:/home/pi# ./script.sh
    bash: ./script.sh: /bin/bash^M: bad interpreter: No such file or directory

  9. Happy Christmas all,

    Thanks very much to Peter and his collaborators for their work in this arena. The
    information, inspiration, leadership and helps you provide are invaluable to those of us with limited time, resources and knowledge. Enabling us to undertake projects otherwise beyond our scope. Your work is greatly appreciated!

    Sorry that this is rather long. It is so because I tried not to leave anything relevant out. My problem is that I am having trouble getting the script to install properly on emmc of Nano PI NEO 2 Plus.

    Background
    I have downloaded from FriendlyArm the current Ubuntu from here: https://www.mediafire.com/folder/vknb4g7udw103/NanoPi-NEO-Plus2

    and installed without errors. Console capture here:

    After re-boot on eMMC I put the script in root and wile logged in as PI ran script https://drive.google.com/open?id=1Xt2l6vNHbSV-3246maiTPZxub7rv84vo

    the script fails consistently at this point: CC(target) Release/obj.target/sqlite3/gen/sqlite-autoconf-3150000/sqlite3.o

    When it fails the heat sink (oem) gets rather warm and stays that way (hours later) but console is unresponsive. Normally the unit is cool to touch. I even tried applying fan to processor during install in that case, the sink stayed cool but lockup occurred in same place. For last fail, I turned fan off after lockup and as expected temp on sink rose quickly to the ‘warm’ state. If memory starvation was the problem I would expect that after say, 8 hours, the process would fail but no. When I come home from work the core is still screaming away doing something to generate that heat! Also from an 8g transflash install, I am seeing 2.3G or so of available storage space which should be far more than enough – even with the ‘extras’ I selected.

    When I began my attempts, I had added a few options (webmin, log2ram, HA-bridge), removing one at a time and ended up with only webmin (still fails). Each time I would get a fail I would re-flash eMMC and then copy over script and run again.

    I have been able to get the transflash release of Ubuntu (same FriendlyARM release date) to run script to completion (log file). I am using a 16G transflash card for development and it is showing 12G out of 14.5G free so the 8G eMMC should not be an issue.

    But wait – that’s not all!

    This occurs once during script this occurs and I have no clue what best response is:

    Installing new version of config file /etc/issue …

    Configuration file ‘/etc/issue.net’
    ==> Modified (by you or by a script) since installation.
    ==> Package distributor has shipped an updated version.
    What would you like to do about it ? Your options are:
    Y or I : install the package maintainer’s version
    …

    Remember, in every case the script is run immediately after flash and reboot of the os – no updates or other changes. The clip above ( /etc/issue )from script log / boot is from 16G transflash with same Nov 3,2017 release by Friendly ARM (not the eMMC version) but same occurs on eMMC install.

    Any insights please?

    P

    If memory serves, doing a D seemed to break the script. It may be that I don’t know the secret things every ‘user’ – read that – “sysadmin” should know. [The concept of ‘users’ in Linux is a fiction from my experience.]

  10. Pete, A couple of comments on your script. First and foremost — thanks a lot for this work.

    Second a couple of questions about some of your choices. Most of the traffic to the server is MQTT and Node Red (which presents its own web service). The LAMP and fileshare demands are minimal — a home page and a few PHP services. So why use Apache2 which is resource hungry for a RPi — especially with the out-of-box Prefork MPM defaults and use of the mod_php5 option. For this usecase wouldn’t lighttpd + php5-cgi be a better fit?

    Secondly Samba. I know it makes Win$ PC access easier but again it is pretty resource hungry and SFTP is absolutely fine for most usecases, (and seamless if you use Linux). OK, this needs open-ssh-server rather than Dropbear, but the resource hike for this is a *lot* less than running Samba.

    1. i had a setup of lighttpd+php5 for a while, now lost somewhere as we did not embed it in the script, ever… maybe a better choice today, with Stretch and Xenial out there, is to move to nginx+php7… which would be even easier and more useful, as you can use it even as reverse proxy so to have the different web services all on port 80… maybe a work for the christmas holidays 🙂

      btw nobody halts you from disabling useless services for you 🙂

    2. Hi Terry

      Well, everyone has their favourites – so I chose Apache.

      As for the second, I do use openSSH, don’t like Dropbear at all. I used Samba as I wanted to be able to handle the PI’s by name for the likes of WinSCP and Samba does a good job of that. I’ve never understood why that isn’t standard – it’s not as if the PC is a minority player or anything.

      See the comments by Mr Shark – he likes nginx, you’d like lighttd. Incidentally – I’ve been using the same service provider for the blog for years without issue and the moment they moved to lightpd I started having problems 🙂

      You can disable any of the options and add in your own – indeed if anyone wants to ponder changes to the script – perhaps offering Apache OR lighttpd OR NGINX might be nice AS LONG AS such an installation works with all of the boards this script works with (it is not by any means just for the PI, Antonio and I have put a lot of work into making it work with FriendlyArm, Orange Pi, Roseapple Pi etc etc using a variety of different operating systems (see the list at the top of the script in the comments) – so any changes need to be considered very carefully for compatibility – and that of course demands that someone test all of the boards, some of which I for example don’t have any more. Therein lies the problem.

      1. Peter, perhaps I didn’t express my point well.

        Apache or Apache + Varnish is also my preferred engine choice for the LAMP servers / VMs that I administer, though I’ve also used nginx. In fact this is the first time that I’ve used Lighttpd. I was a PHP engine contributor for a couple of years working on Opcache and the VM internals so I also understand the strengths and weakness of PHP5 vs PHP7 and the use of Opcache, etc.

        My main point is that the RPi and equivalent SBCs have limited resources — particularly RAM and the primary job of the SBC here is to act as an MQTT broker and Node-Red server, so there are few benefits of having fat LAMP components taking up a third of the free RAM resources just to serve up the odd admin page when alternatives will do this with significantly less resources.

        The same argument applies for WinSCP and Samba: heavy on resources, and for little gain in _this_ usecase. Modern routers offer DCHP and will add any DCHP clients to their local DNS cache and so you should have no resolution issues. Using MS-specific protocols is also questionable when most home browser clients run iOS or Android these days.

        You are using SQLite as your DB engine. This is probably fine for this application, but it really needs a decent amount of RAM allocated to VFS buffer cache; replacing the Apache, WinSCP and Samba services frees up a lot of RAM for this purpose at no material performance hit on the small admin page use.

  11. Peter your script works well. The only thing I had to do was “sudo apt-get install ntpdate” to get the webmin to get the time from a time server.

    Thanks for sharing the script.

  13. Hi there,

    I used the script now first time on Pi 3 with an image maybe older because of the touch screen implemented (aliexpress). For the tool i select all except the password change. It works great!
    All what I can in the moment not regonize are the login credentials for the node red ui desktop. The try of pi, root or admin doesn’t bring me forward.
    Some suggestions from your side?

    Great job, Jb

  14. If you put an empty file in the boot partition with the name “ssh” it will start ssh automatically. And if you create a valid wpa_supplicant.conf file and put it in the root of the /boot partition it too will get your rPi onto the local WiFi network.

    I wrote a script which runs in a bootable Ubuntu ISO which handles all that and even punches in a changed hostname so it isn’t just another “raspberrypi” host on your network.

    The hostname stuff needs to have access to the EXT filesystem and therefore it’s easiest to do a bootable Ubuntu ISO to handle all that. I did this for the DIYRoboCar project.

  15. Hello Peter,

    I’m just a new user of your script and I used it the first time today, didn’t notice any errors when installing BUT when using it afterwards I didn’t have Webmin (It’s in the list when you browse to the ip adress of the Raspberry Pi 3, but it didn’t work).
    I used an image of Raspbian Stretch

    Second thing I noticed was when I tried to follow a video of Andreas Spiess (I believe 126), I tried to use his flow.txt(found it on github) and when importing it in node red (I am also new to this things so maybe I do something wrong) Node Red complained that it missed 2 nodes, both with sqlite, I then removed the ‘node-red-node-sqlite’ node via manage palette and re-installed it, this gave me a new node in the sidebar called ‘sqlite’ (I attached a screenshot). I then used node red for several hours but then node red complained again that it was missing the sqlite node.

    Regards,
    Dirk

    1. Webmin is turned off by default as it takes WAY too long to load and is of limited value, so you have to TICK if you want Webmin. Just did Raspberry Pi 3 yesterday – see blog you HAVE to disable IPV6 – everything installed for me…

      1. i remember who had problems with sqlite, was updating NPM… so, or he did it himself, even if option is not in script menu anymore, or he used an old script version, and checked to update…

        1. MrShark, that was me that updated NPM (via the checkbox at the beginning of the script) but you and Peter set me straight on that one. And in that case, it wasn’t SQLite that was having problems, it was that NR would crash after trying to load any new module after initial boot.

          And one additional side rant:
          The IPV6 thing sounds peculiar that there would be no fallback. Surely i’m not that out of touch to have missed some tipping point where the technical world has now accepted IPV6 as the default (and certainly not for a consumer targeted device like an RPI)…

          1. i know, but we have to see why all of the sudden the raspberries (usually with the original raspbian) seem to decide to go for ipv6 for naming resolution: which SHOULD work, but instead we had many problems with and miserably fails almost everytime it goes that path… could even and probably be an ISP issue, of course… maybe their dns cannot resolve in ipv6, but for now, the safest path to have all working is just disabling ipv6, with no appreciable consequences…

  16. just for reference: no need for keyboard and monitor… just open the config.txt in the partition you see from windows just after flashing the sdcard, and add these lines:

    dtparam=i2c_arm=on
    dtparam=spi=on
    dtparam=i2s=on

    you can even add multiple parameters in a single line:

    dtparam=i2c_arm=on,spi=on,i2s=on

    while for ssh, just create an empty “ssh” file in that same partition, when you boot you’ll have ssh activated

    and about root login, the script will activate it when you run it first time, so you just need to do
    sudo -s -H to become root, giving PI password, then
    bash script.sh

    1. Oh good man – I was going to say, could add that to the script but you can’t get to it until you’ve made those changes. Right – well, that’s now in here for future reference.

      1. non addable to script as they’re needed before running it, but you can make them before even starting, with a notepad 🙂
        just remember to enclose “ssh” in quotes when saving, otherwise notepad will add .txt to the name…
        which maybe works anyway, but i never tested this way…

Comments are closed.