Page 2 – the Pre-2019 Script
The easiest way to get the pre-2019 script – as ROOT user, in your ROOT directory is:
wget –no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh
This can now be run easily: bash script.sh – run as user root, it will start up, copy itself into PI user area, creating that if it does not exist then will give instructions.
Before I begin… no, we’re not a support shop – you may or may not end up having to tweak things for your system but please don’t write in to say “I tried this on a Mac and it doesn’t work”.
The level of interaction after the beginning is now almost zero thanks to a new menu-driven approach and many improvements. We continue to add/modify features as appropriate.
The Virtual Machine (historical information)
Let’s take a look at what we did to get this running on a VM – you can of course duplicate all of this as you go along. We used the excellent VMware of VirtualBox on Windows PCs. This explanation covers VMware.
So – first things first – as I’ve done a video of the VirtualBox installation complete with sample use of Node-Red, here we’ll discuss the VMWARE version. Both of these were done as test mechanisms before using the script on real devices such as the Orange Pi Zero, Raspberry Pi, Odroid and other devices discussed at length in the blog.
We loaded up a copy of VMware (I used VirtualBox but the procedure is similar – VMware player is free of you can use the full commercial workstation) and made sure it works. That much is up to you. There is a wizard for creating new virtual machines – we chose a CUSTOM machine. Steps from that point on were:
NEXT - NEXT – “I will install the operating system later” – NEXT
Linux / Ubuntu 64-bit – NEXT – d:\test - NEXT
Number of processors – 1 - number of cores – 1 – NEXT
Memory for this machine – 1024MB – NEXT
Use Bridged Connection - NEXT
I/O Controller type LSI Logic - NEXT
Disk Type SCSI - NEXT
Create a new virtual Disk - NEXT
12GB (I used 16GB – 8GB would have done) – Store virtual disk as a single file - NEXT
At this point we gave the file a name – in our case d:\test\Ubuntu 640bit.vmdk - NEXT – FINISH
Under settings – hardware – CD/DVD we ticked “Use ISO image file” and selected the file we loaded from the Ubuntu website – “Ubuntu-16.04.1-server-amd64.iso” – OK.
At this point you’ll note reference to AMD64 – that had me as well but it works a treat on INTEL hardware.
Start off the VM – you get asked which language to use (ENGLISH) and various options will come up – the first being “Install Ubuntu Server” – that’s the one.
Once again you’ll get a language question – English. Location – in my case “United Kingdom” – if you don’t fit into the rules use “Other”.
Time Zone in my case – EUROPE and then country. Locale – in my case United Kingdom.
Hostname – we went with the default “Ubuntu” – but call it whatever you want.
At this point you get asked to create a user – it is important you make a user “pi”. Enter password also (though the new script will create user PI if it does not exist).
Home encryption – we said no. When asked if the time zone was correct we said yes.
Partitioning – “use entire disk” (remember this is a virtual disk – not your real disk). You are then asked to confirm erasing disk – go with that. You then get asked to confirm AGAIN. YES.
HTTP Proxy just leave blank – CONTINUE.
Install security updates automatically.
Add OpenSSH server and CONTINUE (important for remote access)
Install GRUB bootloader – YES.
Installation is complete – CONTINUE. Let it run.
And voilà – one working copy of Ubuntu.
When run, the script will generate a Pi user if it does not exist – will make that user part of the correct groups and will ask you to restart the script as user Pi.
The BASH Script
This part is about running the script and the easiest way is to start in a terminal as user ROOT in the root directory (in non-Raspberry Pi installations).
Do the WGET shown above and you will end up with a script file called “script.sh” in the root directory.
At this point BACK EVERYTHING UP to avoid disappointment – no, REALLY.
Execute as:
bash script.sh
The script will if necessary create the Pi user but either way will now ask you to log in as user pi and repeat the procedure in the /home/pi directory – i.e. bash script.sh.
The point of this is to avoid another way which would be to create a file, load in the script contents ensuring you’re using Linux LF only format – then run the script after setting up permissions – this way you don’t have to do any of that.
In the end you will see a menu which will vary slightly depending on which system you are using – you won’t be asked about GPIO on a Raspberry Pi for example.
For the VM we left everything as you see here and tabbed to OK.
At this point you will be asked for a USER and ADMIN names (which can be user and admin if you like) and corresponding passwords. From there on – everything should happen with no user interaction.
After maybe 15 minutes to an hour – or a LOT longer on a Raspberry Pi Zero WIFI – you’ll see an invite to REBOOT – so REBOOT. When the board/VM boots up you will find hopefully a working system.
Assuming you’ve installed everything:
- Node-Red runs at port 1880
- Webmin at port 10000 (Webmin needs Pi or ROOT username and pass until we figure out how to add in the ADMIN user).
- Apache runs at port 80
- HA-BRIDGE will run at port 82 – It seems that Amazon Echo is quite happy with this but the Google product wants port 80!
Node-Red should include MQTT which will need admin username and password as will Node-Red itself. USER name and password is used by Node-Red Dashboard.
There is also the excellent MC – a file manager and editor.
I hope this brief introduction sets you off in the right direction and of course no doubt there will be discussing in the notes below.
Update 05 Jan 2017
At the time of writing I’m having issues with PIXEL in VirtualBox and DEBIAN on VirtualBox in terms of Apache not loading.. I’m sure we’ll sort that – meantime some useful links. Debian may not come with SUDO depending on which installation – here’s a fix link that worked for me in my VirtualBox image.. If SSH setup won’t allow remote access using passwords, here’s a link that helped me. Pixel of course will run in VirtualBox but by default it runs of a virtual DVD which is about as much use as an ashtray on a motorbike as nothing gets saved. Once running however it is a breeze to move over to the Virtual hard disk – you’ll find that part way down the comments in this link. So right now I have VirtualBox running Xenial, Pixel and Debian and the only issues I have are with Pixel and Debian – Apache – watch this space. I’ve also improved the input for user and admin.
Important: Whatever you are using should have the SUDO command built in – it may not be obvious but Debian often does NOT have this installed by default – sort this FIRST. You should also make sure you have Pi user with SUDO group privilege – PI already has this of course and you may have added a user Pi – if not..
wget https://bitbucket.org/api/2.0/snippets/scargill/E5rG7/master/files/makepi
su -c “bash makepi”
Or, Here’s a useful link – takes seconds and is easy… basically – as root you can do – apt-get install –y sudo….. and you should make sure that your PI user is part of the sudo group.
The script works with both NanoPi NEO2 and the NanoPi M1+ and features temperature monitoring and other improvements.
If doing this as a non-Pi user – i.e. ROOT on a machine with no Pi user – follow instructions above OR a slightly easier way…
Having uploaded the script and given permissions… as ROOT
./script.sh
as before – but when it has completed and wants you to log in as PI – stay where you are and:
su – pi -c /home/pi/script.sh
That might save you a little hassle.
For Mike who commented today that he has Grafana working – my RPi 4 came back with repository errors on Grafana when I tried a straight install..
“`
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://repos.influxdata.com/debian buster InRelease: The following signatures couldn’t be verified because the public key is not available: NO_PUBKEY D8FF8E1F7DF8B07E
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://packages.grafana.com/oss/deb stable InRelease: The following signatures couldn’t be verified because the public key is not available: NO_PUBKEY 963FA27710458545
W: Failed to fetch https://packages.grafana.com/oss/deb/dists/stable/InRelease The following signatures couldn’t be verified because the public key is not available: NO_PUBKEY 963FA27710458545
W: Failed to fetch https://repos.influxdata.com/debian/dists/buster/InRelease The following signatures couldn’t be verified because the public key is not available: NO_PUBKEY D8FF8E1F7DF8B07E
W: Some index files failed to download. They have been ignored, or old ones used instead.
“`
and on the actual install…
`grafana is already the newest version (9.3.2).
As you know, I’d followed the instructions on the webpage Update: Linux Package Signing Key Rotation | InfluxData indicated in your notes and that updated the keys etc. This was on a clean ‘virgin’ copy of the OS (Bullseye), and it loaded v10.1.0 of Grafana – the faulty one. When the new version – 10.1.1 – was announced (yesterday 1 Sep, I think), I tried to update but it told me that ‘Grafana had been held back’. So, I used the ‘sudo apt-get install’ instruction, and it did – overwrote the 10.1.0 version with 10.1.1 and that allowed me to login with and set a new (secure) password – just as expected. I have a Pi running v9 of Grafana – I might try an update on that myself and see what happens. What a nightmare this all is!!. Hope this helps – and sorry I can’t be of more assistance.
Pete – small error in your commands. It should be:
gpg – -with-fingerprint –show-keys ./influxdata-archive_compat.key (2 dashes before the with-fingerprint)
and NOT
gpg -with-fingerprint –show-keys ./influxdata-archive_compat.key
That missing dash before ‘with-fingerprint’ is important! I think it’s the web text conversion that does it!)
Hi Mike – I could not get this to work – as we seem to be having formatting issues here – can you resend this with the line in a text file attachment so there is no chance of getting it wrong – I’ll test it myself. Grafana is the only thing I now don’t have totally up to date.
Or just plain-text email me – once I have something working I’ll make sure I end up with a perfect set of instructions in the blog entry..
Regards
Pete
Hi Pete and Antonio,
I want to express my gratitude for your dedication and the time you’ve invested in developing the script and making it accessible to the world. Throughout the years, I’ve used many iterations of the script without any issues. However, recently I’ve encountered a problem.
I’m unable to successfully install additional node-RED nodes. Determining the root cause of this problem has been a bit elusive for me, whether it’s related to hardware, software, or user error.
I’m running the installation on a Raspberry Pi 3B using the pi lite OS Bullseye release dated 23-05-03. I’m using the most up-to-date version of the script, as of today’s date, August 29th, 2023.
I’ve attempted to install various nodes. On examination of the event log associated with these installation attempts, the error messages seem to be the same. To troubleshoot, I’ve tried different SD cards, tested on two separate Raspberry Pis, and re-installed both the OS and the script multiple times.
I’m including a screenshot of the event log that captures the error message when node installation fails. I’m hoping you might be able to provide some suggestions to help me solve my problem. Any assistance and guidance would be greatly appreciated.
Thanks,
Dan
As I now only install on the full Raspberry Pi OS and not had an RPI3 to test for quite some time, I’ve no idea at this time. I’m using RPi4 2GB, next upgrade will be to 4GB or 8GB as I keep adding stuff. I imaginhe 4GB will do it. 2GB works but occasionally responds slowly.
Pete,
Upon closer examination of the Event log, I noticed it states that particle-api-js@9.4.1 is “Not compatible with Your version of node/npm:”.
Evidently, the particle api group of nodes requires node.js to be >= to v12.x and npm to be exactly v8.x.
The installed version of node.js (v18.17.1) seems to meet the particle’s requirements, whereas the installed npm version (v9.6.7) is a later version and does not meet the requirments.
Once I realized the issue, I removed the particle nodes within Node-RED. This enabled me to successfully install contrib nodes without encountering any errors.
I’m curious if anyone else has encountered the same problem . I’m puzzled as to why a error wasn’t triggered during the addition of the particle node group to the Node-RED. Anyway I hope my experiance helps someone else with the same issue in the future.
Thanks again for all your great work.
Dan
I too removed the particle stuff, Dan – same problem – and I’ve not used them in ages anyway – all works now EXCEPT the eWeLink nodes which will no longer connect… all this to use the MATTER node which it SEEMS doesn’t really work with Google Home yet. But I guess I would have had to do this updating st some time if not now….
Oh and Grafana works but I CANNOT update it – griping about bad or wrong keys.
Hi Peter – Well, it starts to look like there are increasing problems with the script! I’m trying on a brand new Pi400 with a new copy of the 32bit full OS. It’s still not installing Apache (although as I mentioned last year, I can install it from Webmin – which does get installed). In addition, the index file isn’t installed, although, again, re-running the scrip a second time does install it. Neither Grafana nor InfluxDB are installed – in both cases, error messages appear about the keys being deprecated. Cockpit is installed, though, but not shown in the index file? In addition, I ran it as user pi from a SSH instance and it didn’t request username or passwords for Node-Red. That’s installed – I assume – but of course, I can’t login as I don’t know what user name or passwords have been set-up by default. It’s a huge shame as I’ve used it many time in the past without most of these issues. Is it an OS version thing (mine is ‘Bullseye’)? Has the script been updated recently (it would appear not?). Thanks again for all your hard work!
Hi Mike
I only now test The Script on Raspberry Pi having initially tested it on many different boards as detailed in the script itself.
I currently use the latest Raspberry Pi OS. I installed an RPi4 from scratch only a month or so back.
As for the default passwords:
username=”user”
userpass=”password123″
adminname=”admin”
adminpass=”password123″
If you’ve any specific changes you think are worth putting in, I’m happy to do that.
Regards
Pete
Npw you remind me I’ve just done an install on RPi4 (2GB) and Grafana DIDN’T install – I call something about keys – I can’t remember where I found the solution last time but I’ll look.
Ok apart from that.. see my notes from this morning..
Aug 27, 2023
I grabbed the latest version of Raspberry Pi Imager…(v1.73 in my case) from here https://www.raspberrypi.com/software/
Having installed the imager on my Windows 11 PC:
I selected a handy, brand new 32GB microSD in a USB adaptor and installed the OS – takes a few minutes to write the image then another few minutes to verify the image, depending on the machine (all without interaction).
I selected Raspberry Pi OS with Desktop (Debian version 11 – Bullseye)
After inserting the microSD into the RPi I applied power and an Ethernet lead to the RPi. Using “Advanced IP Scanner” on the PC I noted after a couple of minutes that the RPihad appeared on my network at address 192.168.1.108 (this will vary from system to system).
I stared a new SSH session in Mobaxterm – address 192.168.1.108 – username pi
Nothing happened – so I turned off the RPi, removed the microSD and put it back into the PC and added a blank file called ssh onto the SD (which appeared as a small drive H (bootfs) on my computer) and then inserted the microSD back into the RPi and re-applied power.
Another couple of minutes and this time the RPI appeared with a note to accept thw new server hostkry and carry on connecting.
I was then asked for a password for pi@192.168.1.108
I entered the default raspberry
This didn’t work as some bright spark has come up with the idea of changing the years-old, trusted method of starting up a new RPI.
So all of the above was wasted time due to recnt changes..
I went back to RPI Imager – before I started WRITE, I went to the COG – I was offered the chance to set hostname (or leave the default – then enable SSH with a new user – you can set a new username and password – I left them as pi and raspberry. Other options include setting up the WiFi – as I had no intention of using WiFi I skipped that… set my timezone to Madrid and keyboard layout to gb. Save. More minutes of my life gone.
Now I had an RPI 4 (2GB) working as user pi – same password as original default
As user pi in SSH, I grabbed and ran the script on the RPi itself.
wget –no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/kAR5qG/master/files/script.sh
then
bash script.sh
at which point the script started. I was invited to look at a menu as usual – nginx is the default, not apache – so I left that along – changes I made were to add webmin, add hwsupport and wiring pi, add grafana (not selected by default)
I was asked to select a user name and password – I selected user and raspberry – same again for admin
The script went off to do its thing as it does – several minutes later…
“You sure you want to do this?”
So began the Node-Red automated install…
“Would you like to customise the settings now?” – NO.
Now for all the extra nodes which took quite a while… (some node installs are slow – especially SQLITE)…
Webmin install is also slow…
sudo reboot
Relcome to Node-Red 3.0
AND THEN
At the command line after running this fix for grafana and influx
https://www.influxdata.com/blog/linux-package-signing-key-rotation/
section: Configuring Linux hosts to use the new signing key
Subsection: DEB Based Systems
i.e.
wget -q https://repos.influxdata.com/influxdata-archive_compat.key
gpg –with-fingerprint –show-keys ./influxdata-archive_compat.key
cat influxdata-archive_compat.key | gpg –dearmor | sudo tee /etc/apt/trusted.gpg.d/influxdata-archive_compat.gpg > /dev/null
echo ‘deb [signed-by=/etc/apt/trusted.gpg.d/influxdata-archive_compat.gpg] https://repos.influxdata.com/debian stable main’ | sudo tee /etc/apt/sources.list.d/influxdata.list
sudo rm -f /etc/apt/trusted.gpg.d/influxdb.gpg
sudo apt-get install $AQUIET -y influxdb grafana chronograf
sudo systemctl daemon-reload
sudo systemctl enable influxdb
sudo systemctl start influxdb
sudo systemctl enable grafana-server
sudo systemctl start grafana-server
sudo systemctl enable chronograf
sudo systemctl start chronograf
and now Grafana comes up – as said, the script installs nginx by default – not apache. Not used Apache in a long time.. It MAY work if you change the tick boxes…
Password issue on Grafana and the install info on their site, updated this month, doesn’t help.
Well, thanks so much Peter! That is definitely extremely useful! I did exactly what you suggested with a new full desktop version of Pi OS. I knew the ssh file ‘trick’ so that was OK. I didn’t bother with either Apache or Nginx, as I know I can (easily) install Apache from Webmin (and did). Webmin, Node-red and Cockpit all installed but – of course – Grafana didn’t. I followed the instructions on your link – they were easier to follow as the commands in your reply were difficult due to fitting in the reply box. Note that SQLite didn’t install, nor SysInfo. However, I’ve now got Grafana and Chronograf showing (as well as the index page after a second run of the script once Apache was installed) but I CANNOT login to Grafana! I know the initial user:password should be admin:admin but I constantly get ‘Login Failed – Unknown error occurred’. You mention a login problem with Grafana but I cannot find a solution that works (and I’ve tried a lot!!). Any tips here will be gratefully received! (BTW, the Grafana instance is showing as v10.1.0 (838218ba20)). So, things are looking up! Thank you again
Mike – I have NO idea how to fix the password login issue – the problem totally contradicts the latest update on the Grafana site – if you or anyone else DOES get around this please DO feed back..
The reply box indenting is something I’d like to sort.
Thanks so much, Peter. This issue of not being able to login to Grafana, now that it appears to be successfully ‘active’, is quite ridiculous. I’ve set up an account with the Grafana community; and asked for assistance but not a word. There are many instances (of course!) of what to do on the ‘net – but as you’d imagine, most of them are useless. I’ve tried resetting the admin password in the Grafana SQLite database – nope! I’ve tried checking the ini file. All good (admin:admin) but still cannot login. What’s the **** point of this if you cannot even start to do anything with it? And there’s a deafening silence as to what the problem is! Sorry – I just get SO frustrated with the incredible waste of time and effort!
Me too – I guess I’ll have to put a note in the script to say it won’t log in until someone comes up with a solution. Thats’ a bummer. I’m just so happy I have endless clones of my current setup but was looking forward to a complete refresh at some point…
We might be getting somewhere. It appears to be an obvious error in v10.1.0. I had a good look at my Grafana log this morning and it says ‘level=warn msg=”Plugin missing module.js” pluginID=input warning=”Missing module.js’. And there is a lot on the Grafana community site https://community.grafana.com/t/grafana-not-starting-after-update-to-10-1-0/101993/15 about this error. Even a recognition it’s a bug!! I wonder if this is part of the problem?
Hi Peter – good news!! Grafana has been upgraded to v10.1.1 and it now works! They’ve obviously been stung into action by the multitude of people complaining about this problem, and fixed it. I did have a slight problem – the standard ‘sudo apt-get update|upgrade’ sequence didn’t work – it kept telling me that ‘The following packages have been kept back:
grafana’ so I used ‘sudo apt-get install grafana’ and it upgraded perfectly. And allowed me to login with and asked me for a more secure password!! So it looks like our problems are fixed. Thanks for all your help!
Good hunting. Thanks for the comment. I’ll update mine too.
Peter, i/m stuck. I’m using your fantastic IoTstack on my RB Pi. The issue i can’t walk around is that i need to install some additional nodes in Node Red which is running in container and i can’t access internet from inside of the docker container. Is there and trick for that without the need of rebuilnding the whole thing? Thank you and keep doing your great work! Appreciate it.
Good morning. I’m no whizz on using Docker I’m afraid.
Pete
Thanks for reply, Pete. Sorry to hear that:)
Thanks for such great work!
Since I’m using docker engine to run some of the script apps, wondering if it would be also more convenient and flexible way to use as part of the original script:
install docker engine
install the script apps suitable to run on docker
install docker management agent (like portainer)
this will allow users to start and stop services (like web min – to get more secure environment)
install additional containers without affecting the original base image structure
Hi
I’m not an expert on using or installing Docker…
Regards
Pete
Now that I postetd a comment I find out… dow
With
pi@timbuktu:~ $ sudo testparm -s
Load smb config files from /etc/samba/smb.conf
handle_name_resolve_order: WARNING: Ignoring invalid list value ‘hosts’ for parameter ‘name resolve order’
Error loading services.
I get this and after commenting the the line
#name resolve order = hosts wins bcast
all services start correctly… might be something to ajust in the script? I don’t know…
Well done Roger.
Hi Peter
Great job with the script! I’ve been using it for a couple of years. Now I’d like to migrate to a new raspi4 with M2 SSD and therefore installed new from scratch with the latest script. Install went smooth but left the system with 3 services not working. They are nmbd, smbd and winbind.
09:07 Failed to start Samba NMB Daemon.
09:07 nmbd.service: Failed with result ‘exit-code’.
09:07 nmbd.service: Main process exited, code=exited, status=1/FAILURE
09:07 Starting Samba NMB Daemon…
unfortuantely I was not able to find out what caused the issue with those services. I would be grateful for any hints what could resolve the issues.
Many thanks
Roger
Great job Peter. I really love this kind of script!!
I found two small issues:
– In the requirements statements I guess the ‘pi’ user MUST be in ‘sudo’ group, otherwise the command “bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)" is in a loop!
– I didn't find php-sqllite4 package
Thanks a lot
Thanks for the feedback. The user Pi in Raspbian is always in the SUDO group – and on other systems, the script when creating user pi should make them part of the sudo grou- but we’ve not tested systems other than RPI and Raspbian for some time now. At the start of the script you can see what systems it has been tested with.
Well congratulations !!
I’ve been running my-off grid system for the past 4 years, using a pi and your excellent script.
4 years and never had to re-boot….
I decided to update the pi, so I installed Bullseye, ran your script, and I’m back up and running.
Thank you and all your supporters for a GREAT, STABLE and EXCELLENT script…
DennisB
Glad it works for you, Dennis – and of course my thanks to Antonio (Mr Shark) for his help with the latest updates.
Me neither – I’ve not used Apache for a couple of years now (at least) and never had compatibility issues using NGINX. Still it seems that Mike has a way around it for those who want to use Apache.
Hi Pete – have just run the 19 Feb updated script on a brand new instance of Bullseye on a RPi4 in an Argon case with a M2 SATA SSD. I prefer Apache rather than Nginx (personal preference) and I ‘checked’ it (and unchecked Nginx) accordingly in setup. Trouble is, it didn’t load! Other web based items did – Grafana, Webmin, Node Red etc, but NOT Apache. I can’t say I saw any error messages – there may have been but they slipped past! I’ve loaded it manually (sudo apt install apache2 -y) and it has created a www directory in /var which wasn’t there before. Of course, your ‘default’ index file isn’t showing; just the Apache default, and I need to investigate that. I thought you might want to know – is this a Bullseye error? Thanks again for everything.
Oooh dear… right – will check that but in the process of packing up for our long trip to Spain. Made a note. Not used Apache in a long time….
Thanks Pete. I can understand the wish to leave the UK winter. Mind you, weeks of 35 plus in Perth is just as tiresome!! After loading Apache manually, I ran script again and it now seems to be OK. Your index file is there and shows as default. Really weird but all’s well that ends well!
the trick is: ignore personal preferences and use nginx, as i didn’t test apache nor updated that part, not using it anymore 🙂
Me neither – I’ve not used Apache for a couple of years now (at least) and never had compatibility issues using NGINX. Still it seems that Mike has a way around it for thoe who want to use Apache.
I understand your rationale. I personally find nginx more confusing than Apache and the latter suits my personal requirements. If it “needn’t” be used, perhaps the best solution is to remove it as an option from the Script? After all, if it doesn’t work as designed, and there’s no plan to support it, why not just leave nginx as the only option? Just my two-pennyworth…….
not a point, it’s just that i’ve not so much time and to test that i need an rpi4, mine is now at my home while now i’m at my parents, so i need Pete to give me access via ssh to one of his… we’ll fix it once he settles in warmy rural spain 🙂
Thanks Mike
Once I get to Spain (departing UK in 2.5 hours – arrive late Saturday or Sunday) and I’m unpacked/settled, I believe Antonio will take alook at making it compatible with the latest Pi OS variant and I’ll check on a fresh RPi4.
P.
‘The script runs in ‘raspiOS bullseye of 2022-01-28’ with a lot errors and warnings.’
The script is not broken, so don’t fix it.
There is obviously a problem with bullseye, so do a separate script and fix the issues cumulatively. Don’t risk compromising something that works and has taken a lot of effort to develop.
Not a bad idea Acme – and thank you – I will leave the script as is – and add a second one as soon as I get more feedback as to what’s actually wrong and how to fix it – all feedback welcome, folks.
Being a complete RPI noob, I have tried following directions and using the script. As such, I installed RPI 11 bullseye, not realizing the script was only updated for buster. I found it worked, mostly, but left out Influxdb and Grafana. So I added this line, and reran the script with only the Grafana option.
1246 test $VERSION_ID = “11” && echo “deb https://repos.influxdata.com/debian bullseye stable” | sudo tee /etc/apt/sources.list.d/influxdb.list
This seemed to work and the only problem I see right now is that I have doubled the entries in sources.list.d.
Thanks for that……
John, sorry to be slow – what’s that “1246 test” bit? How do I change the script to accomodate? I’ve also found an error at line 861 but I suspect it isn’t important? See the new entry which covers this – I spent the entire night on this – new OS – ran the script etc, documented…
https://tech.scargill.net/script-end-of-days-or-just-a-hiccup/
If we can sort this so it remains backward compatible – that new entry can go….
Also – NR runs – but I can’t seem to get past the admin password…
Hmm, the default web page at port 80 also seems not to be functioning – missed that entirely.
Morning john
Can you remember where in the script you made the change? And is it exactly as you say above – I’m not following the “1246” at the start of your change.
I’ll get back to you asap. Line 1246 was the change, though (buster script). Getting hospital discharge today.
Peter,
Firstly, thankyou so much for this great script! What a huge boon for use newbies!
The change as to add one line after existing line number 1245, which is the first line here (the second line is my addition):
test $VERSION_ID = “10” && echo “deb https://repos.influxdata.com/debian buster stable” | sudo tee /etc/apt/sources.list.d/influxdb.list
test $VERSION_ID = “11” && echo “deb https://repos.influxdata.com/debian bullseye stable” | sudo tee /etc/apt/sources.list.d/influxdb.list
Thanks John, I think we beat you to it – script is upgraded to be backward compatible while working with Pi OS 11 now. we also checked some other items and I improved the text in the script file. Enjoy.
On the latest Raspberry Pi image I used update/upgrade etc and run your script a few times, uninstall nodered and npm, but the following error keeps popping up:
“npm does not support node.js v10.24.0 raspberry”,
Many errors to follow, this is on 27 Dec 2021. Thus the script fails. Running on a Pi3B.
Hi Gerrie
For issues with Node-Red and NPM you will need to chat to the Node-Red guys on Discord. The last time I used the script was a matter of weeks ago on an RPi4, to do a fresh install on the latest Pi Operating system (i.e. Raspbian) – for Node-Red, the script merely calls the official Node-Red/NPM install/upgrade script so if there is an issue the problem will be there.
Regards
Pete
Thanks for the prompt reply.
Hi Peter,like many others, I have made use of “The Script” many times to greatly speed up the task of building an IOT App Stack on a RPi…but recently I’ve encountered problems installing Influxdb and Grafana (a single selection) under the Main Menu as you know. From what I can see by the screen reports, the repository key for Influxdb is depricated, probably because Influxdata have moved to a 64-Bit build…and I need my App stack on a RPi 4 now running Bulleye. Is there any easy way round this ?…
Yes, there is an issue of Influx focussing on 64 bit. Erm, but I recently had to use the script to reinstall an RPI after one went inexplicably dead (not something that has ever happened before) and Grafana and Influx went in without issue – if you spot anything in the script that needs changing now, please do let me know.
Hi everyone!!
I used lot of times this wonderfull script to test several devices, as I’m an IoT entusiast. Thanks for this excellent work!
My only concerns is the missing uninstall option. Or at least, I couldnt find it. My workflow in these cases is simply reinstall whole OS, but sometimes I wan’t uninstall some of the features. Using apt remove do somethimes work, but leaving traces everywhere.
Are there a best way to remove some of the features?
Lot of thanks!
Bruno.
Hi Bruno
Never tried uninstalling….
Pete
O.k.
So I started again, first I loaded a full version on Rasbian OS latest desktop. then I set the Password and SSH.
Did update and upgrade
Downloaded your bitbucket of “The Script”.
Found I had to sudo chmod +x Script.sh
Then tried the script sudo ./script.sh
Came up with this?
./script.sh:173 ./script.sh: function: not found
./script.sh:174 local: not in a function
Not I used Kate to find line I think it is this?
function user_input {
local VARIABLE_NAME=${1}
Anyone help please.
Raspberry pi 4
Raspberry Pi 4… install as user PI – NOT root – and you should have only to do the install as documented in the script itself. What’s Kate? I used the script only a short while ago on a new RPi4.
Thinking about it I’ve never used the desktop version as I always use the RPi via SSH.
Kate is KDE like Nano, but with more GUI stuff mate. Will try again. I did set the sudo password first, before doing the script mate. Hope the holiday is going good.
Hi Peter,
I just installed three days ago , “The Script”, the latest one. I am having a problem with Node-red, and I am not the only one. Node-red asks for a user and password. No I have used a common password, but t doesn’t work. I have looked at both the Settings.Js file for Node-Red and turn on the
adminAuth: { type: "credentials", users: [{ username: "admin", password: "$2a$08$zZWtXTja0fB1pzD4sHCMyOCMYz2Z6dNbM6tl8sJogENOMcxWV9DN.", permissions: "*" }] },Plus even change the password for both no joy with it. I used the Bcrypt hashed for the password too.
I need help mate, first time I have used this in three or four years.
All the best to you and Mr Sharky.
Stay safe, hopefully we can work this out please.
Sorry bad timing, just left for the week, trip up North. The admin pass you enter when running the script should work.
No Peter,
It didn’t mate, anyway have a good holiday. I have time to work it out in lockdown here in Victoria Australia. May start again. By the way the rpi clone does not cp or rm the files. Got around it by putting card in Linux machine and sudo Nano the two files to right place.
I tried to run the script today on a Raspberry Pi 3B+ with the “Raspberry Pi OS with desktop” I downloaded and installed today from raspberrypi.org. The script cannot install influxdb or grafana. I don’t think the influxdb link is good anymore and I don’t know what I should do now.
== Installing Grafana and Influxdb
== Total: 00h:47m:09s Cores: 4 Temperature: 53’c
==
Selecting previously unselected package apt-transport-https.
(Reading database … 127790 files and directories currently installed.)
Preparing to unpack …/apt-transport-https_1.8.2.3_all.deb …
Unpacking apt-transport-https (1.8.2.3) …
Setting up apt-transport-https (1.8.2.3) …
OK
deb https://repos.influxdata.com/debian buster stable
gpg: no valid OpenPGP data found.
OK
deb https://packages.grafana.com/oss/deb stable main
E: Unable to locate package grafana-data
E: Failed to fetch https://packages.grafana.com/oss/deb/dists/stable/main/binary-armhf/Packages.bz2 File has unexpected size (27537 != 27465). Mirror sync in progress? [IP: 151.101.198.217 443]
Hashes of expected file:
– Filesize:27465 [weak]
– SHA512:dc94d4fbb856ff2d5441416f27024b0ca263f9cbc91edc3f1e44517843e\
2217d8114062cd6039a2ef95f3ffa736ac4e0413add503e1756f9679f4157b2fc5f62
– SHA256:4a44601f7e28d7fb6fd9f6886953a9d69972f6e91baeb0f0612d686c938e3957
– SHA1:e5c908c673466a462156d7b5aeed0fb8bdb0e824 [weak]
– MD5Sum:b97a895ffdea473d6141ecbfad4a58a9 [weak]
Release file created at: Thu, 29 Jul 2021 10:11:04 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.
E: Package ‘grafana’ has no installation candidate
Failed to enable unit: Unit file influxdb.service does not exist.
Failed to start influxdb.service: Unit influxdb.service not found.
Failed to enable unit: Unit file grafana-server.service does not exist.
Failed to start grafana-server.service: Unit grafana-server.service not found.
Failed to enable unit: Unit file chronograf.service does not exist.
Failed to start chronograf.service: Unit chronograf.service not found.
Detected platform: linux-arm
Hi
A real email would be nice if you want to ask questions… anyway it seems that the InfluxDB people are no longer interested in 32-bits – so it’s out of my hands. I’ve already seen comments about no plans to make Influxdb2 available for 32-bit (ie most) Raspberry Pis. I might be able to upgrade the script to the new location (nice of them) for the repo for Grafana – but I’m in the same boat for new installs as I’ve no wish to go through the hassle of upgrading the RPI to the 64 bit version, cost me endless hours a while ago in a failed upgrade.
I followed Andreas Spiess here because this seems to be exactly what I am looking for. The script runs OK but it won’t install grafana. It seems that bintray has been ‘sunset.’ I’m a bit of noob and I’m not sure what it does but I think that is my problem.
https://dl.bintray.com/fg2it/deb-rpi-1b/dists/jessie/InRelease
The above is forbidden.
Any idea how to proceed?
Script is on Bitbucket…. Bintray scrapped? Thats bad I used to like that….. RPI – script should install that…
that link points to the raspberry pi 1/0/0w version, pretty much NOBODY runs ANYTHING anymore on such underpowered devices… in case you still need grafana, click the ARM button here and follow instructions for ARMv6: https://grafana.com/grafana/download/8.0.0-beta2?platform=arm
Thanks for that antonio, I’d not twigged that he was using an RPI ZERO..
i don’t know what he’s using, but that line he mentioned is triggered only if detected architecture is armv6, which is the one of rpi1/0/0w… time to move to more advanced devices 🙂
i can’t even image to wait half a day to having everything “working” on such low power devices…
Me neither – life is too short – I can’t even manage to keep up with today’s emails 🙂
Hi! I have Banana Pi M2+ and i want to use GPIO in node red. its compatible this script?
Thnanks!
The script merely installs Node Red, it does not control what Node Red does.Also not tested the script on M2+, indeed in the last year or so I’m only aware of testing on Rpi3 and RPi4.
Regards
Pete
When running the script, it fails….
The message I receive is:
!!!! No internet connection available, aborting !
The internet connection shows to be OK via the web browser.
The demos of the end result in use looks great, videos by Andreas..
Please advise.
Thanks!
I can only imagine something up with your connection.
Dear all,
I was using Peter’s script for several months. Node RED, Mosquitto, SQLite, etc. everything was running fine. Then I thought I want to try IOTstack. I used another SD card. I tried to use MariaDB with Adminer as a replacement of a good combo SQLite/phpLiteAdmin. I failed so I run Peter’s script to get SQLite/phpLiteAdmin. I moved my old iot.db file into ~IOTstack/volumes/nodered/data folder. in phpLiteAdmin I see the content of my database but I get the information that the file is not writable. Could anyone advise how to fix the problem?
A quick thought from helpful Antonio. That folder ~IOTstack/volumes/nodered/data – change permissions to 666 just to be sure…
I changed the permissions to rwx for the file and the whole dbs folder and it works fine. Thank you Peter
Hi Everyone,
I remember reading one article about a script to install HASS, node-red, mqtt broker and everything else needed for home automation, but for some reason I can’t find it anymore.
I think it was something similar to “The Script”, but it used docker containers instead.
Can you please let me know?
Thanks in advance.
Somewhere else I think. I’ve never done a complete article on HASS.
Thanks for confirming this Peter, I have find it now 🙂
It was mentioned in Andreas Spiess video #295 Raspberry Pi Server based on Docker, with VPN, Dropbox backup, Influx, Grafana, etc: IOTstack https://youtu.be/a6mjt8tWUws
It’s about IOTstack https://github.com/SensorsIot/IOTstack
My prefered route is Node-Red, Mosquitto, NR Dashboard, Grafana… I’m playing with HASS in Docker but I’m half convinced its putting a heavy load on my 2GB Pi4 (with SSD).
Me again with new information. It appears there was a change between my older version of the system installed with “The Script” from a year or so ago and the new current one. If I supply a username and password I can publish and subscribe. The old system was allowing anonymous connections. I have a dozen or so Wemos D1′ that are not able to publish and would be hard to reconfigure. How can change mosquitto to allow anonymous connection and not use passwords? I tried a couple online tutorials and they failed. I must be missing something.
Can’t really advise there as I ALWAYS use a username and password with Mosquitto. The Script also has always assumed a username and pass for Mosquitto.
Thanks much for The Script. Went through the exercise of setting up node-red, influx and grafana manually some time ago and recently found your script with lots of nice extras so thought I’d do another install using your script. Went awesome on a new pi4 but encountered the same problem as Rich above.
Since I have a lot of IOT sensors built without sending authentication to an MQTT broker, didn’t want to recode them all so discovered all you have to do is go to the /etc/mosquitto folder and edit the mosquitto.conf file to include allow_anonymous true. My network is closed and locked down from the outside world so not too worried about security of my iot devices. My goal is to keep everything off the cloud as I VPN in to all my home automation stuff.
I’m impressed with Cockpit….a very nice utility. Appreciate all your great work consolidating installation of all these great applications.
Would be nice to see PiVPN on here too but right now it’s beyond my 74 year old mind and skill set to make that happen and happy to keep it on a separate pi0w
Warm regards
Bill
Honestly I thought VPN setup was beyond me but PIVPN really IS easy. You are told to set up a certificate – but you don’t actually need to understand what it is. The PIVPN install does it all for you – and at the other end – maybe your phone – you run the free OPENVPN and simply have to transfer the .ovpn file created by the PIVPN installer to OPENVPN on the phone. Once I got over THAT fear I’ve since tackled other VPN setups….
I think I am missing something. I did the rpi-clone from a 32Gig uSD card in the PI to a 32Gig uSD card in a USB to uSD adapter. I was under the impression that if I did the “sudo rpi-clone -f -U sda” operation I could remove the uSD card and put it in a Pi and it would run as it was when cloned.
It should. I use Sandisk and Samsung uSDs and they work. Looks to me like you did it exactly right. Assuming you have enough room on the destination you can also use a larger, same size or smaller destination disk normally without issue. I do this all the tim (but since discovering SSDs I’ve been doing the same with those – same but sometimes more power sensitive – uSDs generally just work).
After trying another fresh clone uSD both were same size as original and SanDisk brand. Would not boot in new Pi. I installed NEW Pi image and ran the script. Once it completed I created a couple nodes and could not get connected to MqTT. Tried command line mosquitto_sub -t test -v and I get connection refused. mosquitto is running, I checked to make sure. I am now lost!
Not sure what’s happened there – it normally works.
Hi Peter I appreciate ‘The Script’ the most, discovered it thanks to Andreas Spiess. A big THANK YOU for that!! I installed a new RPi4 and it didn’t go further on the blue config screen… waited for hours… until I discovered to handle it correctly with tab&Space… thanks to the hint of one of the last comments. So, please, for non-English natives the sequence might be not that clear. Maybe you could modify the instruction line on that screen or detail it at least in the ‘newbies section’ above. Best regards, Stephan
Hello,
my apologies, tried to execute “The Script” on a Raspberry Pi 2B with
DietPi v6.31.2 HW_MODEL:1 HW_ARCH:1 DISTRO 5 (Raspberry PI OS (32-bit) Lite)
Trying to install SSH after complete Execution of “The Script”
showed an ERROR:
GPG error: http://ppa.launchpad.net/webupd8team/java/ubuntu xenial InRelease: The following signatures couldn’t be verified because the public key is not available: NO_PUBKEY C2518248EEA14886
E: The repository ‘http://ppa.launchpad.net/webupd8team/java/ubuntu xenial InRelease’ is not signed.
Tried mitigation measure listed under this c2518248EEA14886 in the thread, but did not work.
Thanks& Regards
Michael
C2518248EEA14886
Hi
I’ve installed the latest version of the Script (14/5/2020) on top of a fresh install of Raspberry Pi OS (27/05/2020) on my RPi 3B. Nearly everything works except for Mosquitto MQTT broker does not start.
In the boot up screen I get this error: Failed to start Mosquitto MQTT v3.1/v3.1.1 Broker. The output from “systemctl status mosquitto.service” is as follows:
systemctl status mosquitto.service
● mosquitto.service – Mosquitto MQTT v3.1/v3.1.1 Broker
Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2020-07-13 12:07:42 BST; 59min ago
Docs: man:mosquitto.conf(5)
man:mosquitto(8)
Main PID: 2298 (code=exited, status=3)
Jul 13 12:07:42 HomeServer systemd[1]: mosquitto.service: Service RestartSec=100ms expired, scheduling rest
Jul 13 12:07:42 HomeServer systemd[1]: mosquitto.service: Scheduled restart job, restart counter is at 8.
Jul 13 12:07:42 HomeServer systemd[1]: Stopped Mosquitto MQTT v3.1/v3.1.1 Broker.
Jul 13 12:07:42 HomeServer systemd[1]: mosquitto.service: Start request repeated too quickly.
Jul 13 12:07:42 HomeServer systemd[1]: mosquitto.service: Failed with result ‘exit-code’.
Jul 13 12:07:42 HomeServer systemd[1]: Failed to start Mosquitto MQTT v3.1/v3.1.1 Broker.
Hi Kim
Sorry to hear you had issues installing MQTT. In the process of moving for the summer, I’ve made a brand new installation with the script and updated several others…all to the latest Raspbian as I need SSD booting (blogged). I’ve certainly had no issues with Mosquitto so I’ve no idea why you have. On the other hand I just sold my last RPi3 in favour of RPi4 (for the fast USB boot) so slightly different scenario but of all the things to go wrong, Mosquitto is the last one I’d expect to give grief.
Thanks for your response. I’ve fixed it now. It was a combination of a permissions issue and using the include mechanism in the mosquitto.conf file which ended up with me having multiple password files. Trouble is, there’s so many conflicting tutorials on setting up mosquitto, it’s very confusing for a Linux newbie like me.
I’m using my RPi3 as it was spare after upgrading my NAS to a RPi4. It seems capable enough to run my DIY home automation system.
WELL, I’m very pleased you resolved the issue – great… and yes, much of the time even the old RPi2 is good enough for home automation. If only they’d included USB3 in the RPi3, we might not have needed the RPi4 but to make real use of an SSD drive, you really do need USB3, hence my logic for moving to the RPi4.
Thanks for the feedback, much appreciated.
Hi, all
I am new in unix and still playing with ioBroker on RasperryPi 4B. As I was unable to install MQTT due to issues about dependencies I tried The script but was not successful as it did not install anything. Assumin it may be the previous broken installs. Today I started with a new system. The script started until the screen where to select the options. I only deselected quiet then Tab for OK or Cancel but then nothing. I expected to be asked for somthing, but no
The Log shows the following:
############################################################
Welcome to ‘The Script’
Vormals nicht ausgewähltes Paket libncurses5:armhf wird gewählt.
(Lese Datenbank …
(Lese Datenbank … 5%
(Lese Datenbank … 10%
(Lese Datenbank … 15%
(Lese Datenbank … 20%
(Lese Datenbank … 25%
(Lese Datenbank … 30%
(Lese Datenbank … 35%
(Lese Datenbank … 40%
(Lese Datenbank … 45%
(Lese Datenbank … 50%
(Lese Datenbank … 55%
(Lese Datenbank … 60%
(Lese Datenbank … 65%
(Lese Datenbank … 70%
(Lese Datenbank … 75%
(Lese Datenbank … 80%
(Lese Datenbank … 85%
(Lese Datenbank … 90%
(Lese Datenbank … 95%
(Lese Datenbank … 100%
(Lese Datenbank … 46057 Dateien und Verzeichnisse sind derzeit installiert.)
Vorbereitung zum Entpacken von …/libncurses5_6.1+20181013-2+deb10u2_armhf.deb …
Entpacken von libncurses5:armhf (6.1+20181013-2+deb10u2) …
Vormals nicht ausgewähltes Paket ccze wird gewählt.
Vorbereitung zum Entpacken von …/ccze_0.2.1-4_armhf.deb …
Entpacken von ccze (0.2.1-4) …
libncurses5:armhf (6.1+20181013-2+deb10u2) wird eingerichtet …
ccze (0.2.1-4) wird eingerichtet …
Trigger für man-db (2.8.5-2) werden verarbeitet …
Trigger für libc-bin (2.28-10+rpi1) werden verarbeitet …
OPSYS: RASPBIAN
############################################################
Grabbing some preliminaries…
DISTRO: 10
CHECK64: armv7l
USER: pi
############################################################
Stopping IPV6 access
############################################################
Allow remote root login and speed up SSH
Restarting ssh (via systemctl): ssh.service.
update-alternatives: Fehler: keine Alternativen für newt-palette
Paketlisten werden gelesen…
Abhängigkeitsbaum wird aufgebaut….
Statusinformationen werden eingelesen….
avahi-daemon ist schon die neueste Version (0.7-4+b1).
Das folgende Paket wurde automatisch installiert und wird nicht mehr benötigt:
rpi-eeprom-images
Verwenden Sie »sudo apt autoremove«, um es zu entfernen.
Die folgenden NEUEN Pakete werden installiert:
avahi-utils
0 aktualisiert, 1 neu installiert, 0 zu entfernen und 6 nicht aktualisiert.
Es müssen 54,6 kB an Archiven heruntergeladen werden.
Nach dieser Operation werden 128 kB Plattenplatz zusätzlich benutzt.
Holen:1 http://ftp.cica.es/mirrors/Linux/raspbian/raspbian buster/main armhf avahi-utils armhf 0.7-4+b1 [54,6 kB]
Es wurden 54,6 kB in 1 s geholt (79,5 kB/s).
Vormals nicht ausgewähltes Paket avahi-utils wird gewählt.
(Lese Datenbank …
(Lese Datenbank … 5%
(Lese Datenbank … 10%
(Lese Datenbank … 15%
(Lese Datenbank … 20%
(Lese Datenbank … 25%
(Lese Datenbank … 30%
(Lese Datenbank … 35%
(Lese Datenbank … 40%
(Lese Datenbank … 45%
(Lese Datenbank … 50%
(Lese Datenbank … 55%
(Lese Datenbank … 60%
(Lese Datenbank … 65%
(Lese Datenbank … 70%
(Lese Datenbank … 75%
(Lese Datenbank … 80%
(Lese Datenbank … 85%
(Lese Datenbank … 90%
(Lese Datenbank … 95%
(Lese Datenbank … 100%
(Lese Datenbank … 46102 Dateien und Verzeichnisse sind derzeit installiert.)
Vorbereitung zum Entpacken von …/avahi-utils_0.7-4+b1_armhf.deb …
Entpacken von avahi-utils (0.7-4+b1) …
avahi-utils (0.7-4+b1) wird eingerichtet …
Trigger für man-db (2.8.5-2) werden verarbeitet …
After waiting for more than one hour still same blue screen in console. Am I too unpatient?
Regards Gerhard
Anyone any ideas? I don’t speak German which doesn’t help. “Stopping IPV6” is right as that is turned off in the script – but from there… an hour is excessive – even with quiet turned on, some of the 3rd party scripts called by “the script” are noisy and may show warnings only (lots of them in once case) – I would not expect silence for more than a few minutes at any time during the install. But then it is stated somewhere on the blog that this should be installed on a clean device, and NOT the NOOBS version of Raspbian – not a machine with previous broken installs on it, in which case it could be anything. I suggest Raspbian Lite straight from the Raspberry Pi store. I just installed the latest script version on two such machines (I’d already installed rpi-clone as well but that won’t do any harm). The script pulls in all necessary dependencies first so MQTT should install just fine.
Hi Peter,
thankyou for your quick respond.
Obviously I was too impatient. After several hours later as this blue sceen still not changed and I was sure that the installation failed again, I pressed Tab and got asked about these passwords. Then everything went until to the end. I rebooted and can see in FTP client that installation sucseeded. The only drawback is I cannot open the ioBroker on port 8081 on Port 80 I see your default Webpage. process iobroker is running. Any idea what is wrong.
Regards Gerhard
I cannot comment on Iobroker – on their website it says (righly) that the English instructions are for JESSIE and out of date – it then recommends we go to the German instructions and as I don’t speak German that’s not easy for me to progress. Even running TRANSLATE on the recommended link, they refer to an outdated version of Raspbian called STRETCH so it seems that keeping up to date with Raspbian is not a priority. Until recently the Pi operating system was referred to as BUSTER (the underlying Debian version BUSTER which appeared mid-2019 and will be supported iunti 2024) and now referred to as “Raspberry Pi OS”
IOBroker looks to be very popular but mainly in Germany and that is where the developers are. I suggest aasking them questions on their GitHub pages (which appear to be in English and have been updated recently). https://github.com/ioBroker/ioBroker/issues unless someone in here steps in.
Pete
Hi Peter,
As ioBroker was fine in Buster before I ran The Script I guessed that something was changed during this process. If you think this is not the case I shall try to reinstall iObroker over the existing installation. I shall keep you informed.
As The Script is concerned I wonder if it may be possible to implement a message indicating to the user to be patient until the process will continue at least to include this information into the manual. Otherwise it appears, especially for non english people, that the process has stopped after pressing Tab for the first time when expecting a immediate confirmation OK/Esc as written in the first line in the blue.
window. Or change the text of this line into something like “Press Tab to continue and wait for pressing Tab again for continue or Esc for exit”
Regards Gerhard
Hi
The delays in the script are caused by third party scripts and installation programs, the content of which is out of control of the script itself. There is no manual for the script, it is self-documenting. In the blog entry “the script” I have now added a paragraph to suggest expecting periods of no output.
Hi Peter,
Firstly, thanks to you and team for producing the script. It’s taken me a long time to get here. but I’m trying it for the first time today.
Installing (3 attempts to eliminate user error) on a Raspberry 3 with fresh Buster Lite (2020-02-13-raspbian-buster-lite.img) I was getting the following errors:
[snip snip]
avahi-utils is already the newest version (0.7-4+b1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Box options:
–msgbox
–yesno
–infobox
–inputbox [init]
–passwordbox [init]
–textbox
–menu [tag item] …
–checklist [tag item status]…
–radiolist [tag item status]…
–gauge
Options: (depend on box-option)
[snip snip]
I’ve narrowed down the problem to extraneous two tabs at the end of line 494
“cockpit” “Install Cockpit” ON \[tab][tab]
removing these cleared the problem.
Forgive me if this is my error or a known problem.
Regards,
Dominic
Hi there Dominic – the “team” is largely Antonio (Mr Shark) and I with helpful folk like yourself occasionally pointing out issues and providing solutions.
Assuming you are right – and I’m pretty sure you are – but feel free to test – I’ve already put out an update with those two tabs removed.
That’s what I get for putting out “trivial” changes without testing. I won’t do that again in a hurry – but we had so much on this weekend…
Anyway thanks for your input – I’ve noted it in the updates section in “The Script”. Busy working on ESP32 today… and some new solar lights from Bangggood – and… bored, me? Never.
Of course you have to wonder – who would design a scripting language that cared about extra tabs!!! Maybe the same folk who make languages case-sensitive. When I say in English “Hi there” – it means EXACTLY the same as “HI THERE”…. 🙂
the backslash is used in bash to ESCAPE next char’s special meaning… put at the end of a line, REMOVES the special “carriage return” meaning of it, and makes next line like it’s the continuation of the previous one… this just to allow you to have a better formatting instead of a looooooooong single line… it’s then evident that adding something AFTER the final backslash will escape THAT character and NOT the EOL, causing the error the commenter had 🙂
and that’s why every good text editor allows hidden chars to be shown someway, like spaces, tabs, eol, etc 🙂
it’s like pretending C composed statements not to be enclosed in curly brackets, or python ones not be perfectly indented 🙂
Hah, well done Antonio, I was just thinking about that.
Thanks both. FYI I’ve just re-run the script and can verify it’s fixed.
Dominic.
Hello Peter, I downloaded the script and installed it, just great! 🙂
Now I’ve been working on rode ned for a long time and it makes me very happy. I have already carried out some projects that worked great.
Since I am unfortunately not a programmer and I am not so familiar with the subject, I would have had 2 suggestions for the script. I have built an alarm message for my grandma that if you have problems I will receive an email message. The problem is that you always need internet. What would be really nice would be a kind of SMS server on the Raspberry Pi, which sends an alarm SMS to several recipients via Node Red. This could possibly be implemented using a Shield Sim800L gsm or similar. If you would like to be in a mood, you could add something so that the SMS server is simple and local, but is also addressed directly with Node Red. Greetings from the Black Forest, Bernd
Hello Peter, I downloaded the script and installed it, just great! 🙂
Now I’ve been working on rode ned for a long time and it makes me very happy. I have already carried out some projects that worked great.
Since I am unfortunately not a programmer and I am not so familiar with the subject, I would have had 2 suggestions for the script. I have built an alarm message for my grandma that if you have problems I will receive an email message. The problem is that you always need internet. What would be really nice would be a kind of SMS server on the Raspberry Pi, which sends an alarm SMS to several recipients via Node Red. This could possibly be implemented using a Shield Sim800L gsm or similar. If you would like to be in a mood, you could add something so that the SMS server is simple and local, but is also addressed directly with Node Red. Greetings from the Black Forest, Bernd
it’s not that complicated, these type of modules all you some pretty much standard serial AT protocol, so once interfaced to raspberry (beware to logic levels…), all you need is to send something to a serial port… take a look: https://www.youtube.com/watch?v=XqoeqG2j-ik
Hi all
I have a pi3b connected connected via rj45 to my local lan.
I have uSD loaded with the busterlite all updated per the blog.
When I attempt to load rpi-clone using a pasted copy as per the github site I get an error message command does not exist.
Any advice greatly appreciated.
You didnt say if you are logged in as pi or root. The rpi-clone install – as user pi – in the /home/pi folder to start with – is easy
https://github.com/billw2/rpi-clone
INSTALL section
If I use the full ver of Buster it works ok!
Each time it is a new download of either the full ver or the lite ver just plug the sd in power up run the apt update then upgrade then try the rpi-clone github site this is when it fails with the lite ver.
I am happy using the full version just puzzled.
thanks
Well, personally I use the full version but with the default set to terminal as against desktop – as I have no need for the graphical environment. Mr Shark however (see his many comments) tends to use th elite version Lite as downloaded here – and in one installaation I have used that too – and had no problem running up rpi-clone – so I’m not sure what to say..
Hi Pete.
Newbie to linux, thanks again for the script, it’s an easy way to install the programs.
Is it possible to group all the data and config files from Node-Red, InfluxDB, Grafana,… in a unique folder for an easier backup?
Hi
It is probably possible – but then time travel is probably possible:) – if you need that for backup, I suggest you may be doing backup the hard way. The totaal effort in the way I do backup is to type “clone” and wait a few minutes. All described in the blog.
that’s why i moved to docker, which allows to keep all you data in some dedicated volumes, you just need to backup them and that’s it 🙂
in my case i use home assistant with its addons, which makes life even easier, as you can just create a snapshot of your system and restore on a completely different system: i moved my setup from an rpi3+ to an x86 pc with no issues… all my data trasferred 🙂
Hi Pete, Got a surprise when I saw “micro installed” at the end of the installation log today. Yes, I have been running “the script” a number of times on a RPi 3B over the last few days. Then I read your blog and it was clear.
Okay, here’s the bug I think you have introduced:
The lines added to include micro are after the attempt to change the hostname. I actually did change the hostname and the sudo commands fail after the script makes the hostname change viz
Micro has been downloaded to the current directory. You can run it with:^M
^M
./micro^M
^M
^M
sudo: unable to resolve host raspberrypi: Name or service not known^M
So the hostname change must be the very last thing done!
BTW the hostname change commands run a not existing file and it goes unnoticed because of the re-directions:
sudo /etc/init.d/hostname.sh > /dev/null 2>&1
I do appreciate the work that has gone into this script. Thanks.
Hi there NigelTwo
I’ve updated the script again and put the “micro” entry slighty further up. However, there remains a message as there has for some time, about “hostname not known” – this persists until the reboot only – if you or anyone can think what I got wrong there and let me know, I’ll fix this minor but annoying item – still – better to get it right.. yes, do let me know what I got wrong there. That would be helpful.
The relocated “micro” entry looks (and runs) okay to me (RPi 3B Raspbian Buster Lite and changing the hostname). But while I have the microscope out… about those 3 lines you added.
1. The first command is cd – did you mean “pwd” or “cd $HOME”? Probably you were just re-assuring yourself that the install was going into the current user’s homedir. That’s good, and on my install the script is still in $HOME at this point.
2. The third command (apt) doesn’t seem to be acting on the -qq option. At quiet level 2 it is supposed to suppress the Y/n user confirmation. The man pages show apt-get has this option, but I am no expert in apt versus apt-get. Get more advice and proceed with caution here.
Regarding the “hostname not known” messages. A Linux/Unix that cannot “find itself” is a worrying beast. After this script alters the two hostname related files, there are two sudo commands:
1. The first has an error that is re-directed and is lost (on Raspbian Buster Lite there is no such /etc/init.d/hostname.sh file, and on Ubuntu the command file expects a stop|start parameter and therefore just prints a help message). I note that raspi-config wants to reboot after it changes the files. Desktop Ubuntu seems to able to survive with the help of hostname.sh.
2. The second sudo is at the very, very end and this is probably the message you are seeing. Maybe print a message to the log that it is an expected response if the hostname has been changed, or use the effective redirection that suppressed the first sudo warning. I do agree this is minor as we don’t work for a bank or the military.
All the above can wait until you next do some maintenance (you know, that maintenance team developing this script, winks), or you can send it all to /dev/null. Testing is time consuming, I know.
Cheers for now, Nigel
cd on its own is just fine – it takes you to your home directory. Always best to be sure. that qq option does not seem to be working. I hwill be putting up a new script.sh soon – minor issue to sort out and check first.. apt-get to my knowledge is no longer required – I simply use apt and have done for some time. Hostname not found – solved. Will update the live script file soon. I love it “”the team”….. 🙂
Pete, I stand corrected. The default behaviour of the “cd” command is just what we need.
Hi Mike, really pleased you got it working. I have learnt a couple of things too from your’s and Peter’s comments. Thanks Peter too, I might update my 3 year old version of the ‘script’ to the new version.
Please do and let me know if you spot any mistakes or anything out of date. There’s nothing worse than out of date material on the web (apart from Covid that is).
Yes Bob lots of things have changed – on the other hand if it works and you are happy – fair enough. But I do not have a maintenance team so don’t ask me about early versions 🙂 Antonio and I don’t even now HAVE some of the kit we tested 3 years ago.
OK, well despite my reservations on Mother’s Day!! I have managed to do a quick rebuild with 32GB sd card; Rasbian Buster lite; Etcher etc and followed your instructions to the letter, Pete. Couple of minor exceptions – I didn’t change the pi password, and I ‘unchecked’ the quiet option on the menu (like to see what’s going on!). But I do have a mea culpa! I’ve been using ./script.sh as the command to run the script. So, today I did exactly as you suggest (bash script.sh) from the pi account and – hooray!! – everything worked!!! 🙂 Apache is running (checking via a networked win10 machine browser – I have the MAC address of the Pi locked to a specific IP address in my DHCP server so I know exactly where to look) and even the “The Script” index page looks good! Node-Red and Grafana both work as expected; sysinfo page is good….in short, exactly as you have said. And if all my problems have been caused by running the script in a slightly different way – well, deepest apologies, and maybe this, too, needs to be emphasised. And, yes – the sun is out here, too! But I guess for Perth that’s hardly news……. 🙂
Whee. tell EVERYONE – the more the merrier. Obviously things improve as time goes on yet I constantly get people using old versiond of software and wondering why they don’t get success. I’ve yet to find a flaw in the last script update (I did some more tidying up on May 8) but it is always possible I could be using newer, better links.
For now, I’m glad you got it running.
Hi Mike
You left another nice comment entry here which I’d managed to lose in the blog queue so I’ll paste in here from my email screen. You were slightly out in that the install procedure was not only buried in comments – it was also just a few paragraphs down – but it is now right at the TOP thanks to your prompt.
I’m still hanging on waiting for the RPi guys to give us true USB3 boot. Here’s you last comment entry minus the now defunct section on finding the install info:
Well, even better news! I believe I mentioned that my ultimate intention was to have the root f/s on a (spare 240GB) USB3 SSD, leaving the boot partition on the – somewhat smaller! – SD card. This is, as I understand it, the recommended way to have the system essentially running from a larger drive on a RPi4 – and believe me, a USB3 SSD is considerably quicker than a SD card!
I hadn’t had a lot of success running the script with this configuration, and hence some of my earlier comments. However, after the success yesterday with a ‘vanilla’ SD configuration, I decided to try again this morning.
This time, I used the ‘mid-sized’ Raspbian image. There’s a certain amount of configuration to be done – the latest versions of Raspbian set the Partition UUID identifier to the same value every time! This means an obligatory change of PUUID on the SSD to differentiate it from the SD card in the FStable. But we got there, and after it was all running successfully – and updated as per Pete’s comments – I crossed fingers and ran the script.
Oh, one issue – the ‘unwanted’ partitions on the SD card (rootfs) and the SSD (/boot) seem to mount automatically when the device boots, so I first unmount them to make sure there’s no confusion by the script as to what f/s is to be changed. (To run via USB, you first flash both SSD and SD card with the same Rasbian version). And it worked!! Again!! It MUST be that stupidly small error of not using ‘bash script.sh’ that has been causing all my heartache! Everything is installed just as I’d expect – and as I have always found in the past.
Thanks again – and onwards and upwards….. Oh, and stay safe!
Thanks, Peter and Bob for your very informative and helpful replies. It’s early morning (and a sunny Mother’s Day!) here in Perth, so I probably won’t get a chance to retry today. One comment – the Pi4 uses an ‘official’ Pi4 power supply, so I don’t think current draw is a problem. Bob, I will say that on my early efforts, I had let the initial boot-up process run to full completion, upgrading locale, time, date, passwords, files, setting ssh, wifi connection (I normally connect through ethernet) etc etc and it was because of the spectacular failures with that option that I tried without going through any of that process to make sure that one of those changes wasn’t causing my problem. And Peter – yes, the script most certainly was the most recently modified (07 May) version. I double-checked that before running it! Two things that I didn’t do that you recommend. I didn’t (obviously) run an update / upgrade sequence after booting. I also didn’t create the ssh file. I normally do, but as I wanted to run this entirely on an attached local console monitor, it didn’t seem necessary. Also, of course, I chose a few more options than you appeared to do. I will try again – I’m lucky as this is a brand-new Pi4 that doesn’t have any information on it, so I’m not in any fear of losing anything. In fact, I was attempting to – very successfully – operate it from a combined sd card / USB3-SSD combination (boot on sd and rootfs / on ssd) for speed and storage capabilities, but have reverted to just a sd for script test purposes after the early failures. Small steps first – and I’ll certainly report my observations.
Mike, I do not have either Peter’s or Antonio’s expertise on the script but reading your post I wondered if you should maybe let the full set up run on the Pi before trying to install the script. The set up after the initial successful install of ‘Buster’. I installed ‘Buster’ on 3 Pi’s recently a 3B+, 3b and 4 with 4 Mb. I was surprised how my files were updated given I used the latest ‘Buster’ release. Just a thought or two!
Hi Bob – I’ve now hopefully made it abundantly clear in this blog entry that Buster install should be followed by adding the blank ssh file for remote access, sudo apt update and sudo apt upgrade as well as installing rpi-clone (even though it is added in the script) to avoid un-necessary heartache if a mistake is made and backtracking is required at any point. I learned these steps the HARD way.
Anyone else reading this – don’t take any or changes or shortcuts… e all think we have a better way.
To avoid confusion, if I messed anything up today in the blog entry, please contact me directly by the contact form and I’ll fix – there may be a gap as the sun is out and my garden is waiting. Antonio woulds normally be passing me private comments by now but this is not a good time for him.
Regards
Pete
Thanks Pete for checking. I really must be doing something terribly wrong at this end? Perhaps the script doesn’t like running upside down….. 🙂 Yesterday, I flashed a brand new 32GB sd card with the latest Rasbian Buster full version and booted my RPi 4 with 4GB RAM. I cancelled – immediately – any form of set-up and ran the latest modified script. And, I’m sorry to say, it was a disaster. Nothing seemed to work after rebooting and testing. So, in view of your comments about Rasbian Light, I did the same again. Reflashed the sd card with the latest Lite version, and ran (via wget etc) the script immediately after a successful boot. No mods at all – straight into the script. And the output doesn’t look good. Node-red is running, but that’s about all. After rebooting, the info tells me mosquitto can’t run; I know Apache isn’t running as I cannot connect (“192.168.100.155 refused to connect.” and I can see that it refuses to start); Grafana is ‘stopped’ etc etc. So something is really amiss. Is it the version of Rasbian? (My full version is dated 10 Jul 2019 and Lite is 13 Feb 2020). Is it the Pi model itself – we know the model 4 has some significant changes from earlier models? And as I say, this has been run on an absolutely ‘raw’ version of the OS in both cases. As the script has been so successful and useful in the past on other model RPi, it seems to a great pity that I’m having such spectacular un-sucess now! Any comments gratefully received!!
Hi Mike there is probably something we’re not seeing in your message above – or you are running the qrong script – because I took the latest Raspbian Buster Lite from the web, put it on a brand new (several weeks old) Raspberry Pi 4 using BalenaEtcher on my Windows 10 64 bit PC,adding the empty text file “ssh” which the uSD was still on the PC, took the uSD and plugged it into the Pi (remember you need a supply able to deliver 3 amps max).
At that point I turned on the Pi while plugged into my RJ4 network at which point I logged onto it from my PC, using the default password with MobaxTerm on the PC.
Before installing the script I keyed in:
sudo apt update
sudo apt upgrade
wget https://bitbucket.org/api/2.0/snippets/scargill/kAR5qG/master/files/script.sh
See script update of May 7, 2020 – that’s where that link came from. P unnched that into the Pi and then typed:
bash script.sh
into the Pi – at which point I was asked if I wished to update passwords etc which I did – and selected the first Node-Red option as well as adding in Grafana from the menu (I did not slect Webmin as that takes ages)…. a while later, as instructed, I rebooted the Pi (sudo reboot) and a minute later – a working Pi.
I’m going to further improve the blog this afternoon to make all of that crystal clear. If you did anything DIFFERENT to that, start utterly from scratch, don;t try and bodge it.
Now, having backed up the original Raspbian (with on addition necessary to back it up), I have one uSD with only Raspbian and rpi-clone on it – and another with a fully functioning Node-Red + MQTT + Grafana to experiment with.
Look for blog updates I just completed (UK lunchtime May 9, 2020). Just FORGET the idea that the RPI4 being different or Buster being different is causing your problem as I hope I’ve just demonstrated.
Good luck.
I’ve just had a couple of spectacularly unsuccessful attempts to run the script on a 4GB RaspberryPi4 with latest Rasbian Buster desktop version. In the first, the script ran but nothing, apart from Webmin, appeared to work – for example, I could not connect to the webserver locally or via my local network. I uninstalled apache2 and reinstalled it and then I could see the ‘Pete’s Utilities’ (index) page. However, NO css – and I find that if I attempt to look at ‘typecast.com’, it immediately relocates to ‘monotype.com’. However, even without the bells and whistles of the css file, I can still now click on the links. Except nothing appears to work. Both Node Red links are ‘Unable to Connect’. Same with Grafana and SQLite. So, in the spirit of ‘what have I done wrong?’, I started from scratch. Reinstalled Raspbain Buster again, and went through a complete start-up process. Used wget to get script via ‘sudo wget https://bitbucket.org/api/2.0/snippets/scargill/kAR5qG/files/script.sh‘ and ran it again. And exactly the same! Again, reinstalled apache2 and, again can only see the index file details without the associated css. And, again, nothing works except Webmin. Now, like many here, I have run this script many – many! – times on older (RaspPi3) with hardly any problems at all, and I’m eternally grateful for the work you have put into it. But now, it just seems to be a disaster! So, any advice will be very useful! Is it something to do with Buster? The RaspPi model 4? Looks like I’d better reinstall Rasbian – again! – to get a ‘clean system, and then will wait this time for any helpful comments…..
Seeing no reference to typecast in the script – – without pulling it apart the reference must be inside some third party script used by the script. Node-Red – well, that uses the official upgrade/install script for Node-Red and I know that works better now than ever before. Get the updated script as of this morning, I simply changed references to https://tech.scargill.net/iot/…. into https://tech.scargill.net/iot/…. as that might help with your css.
it’s used in your css file to download the fonts, i think, the site is now gone, if you search in comments someone said it could help in changing that, but don’t know how it went after those comments…
for script, please try using raspbian-lite, not desktop, see if that helps, i can’t test anything right now as all my stuff is at my home and i’m at my parents for 3 months now, locked out…
I am currently outfitting an /RPI4 with Raspbian Lite after doing the full update and upgrade – howevever if it all works, I WILL remove the DARK SKY node as it is now defunct and replace it with an alternative. I see the SQLITE script hasn’t improved any in recent months – still throwing out a sheload of un-necessary warnings – If I knew more about it I’d write my own version. I’m also taking out th CSS JS in the sample web page as clearly the relevant site is now dead. I need an alternative to make the sample page look pretty.
no, YOU can! 😀
really, flash a raspbian lite and run the script, it’s been a while we don’t test it completely, and in case of problems send me the logs you’ll get… for css, it’s up to you, change that file to something else not using online fonts, or just using google ones, always available…
Well guys I have just spent the afternoon running the Script on a Raspberry Pi 4 using a brand new “Raspbian Lite” – the only thing wrong was that dead external website for the basic web page includes – I have now GUTTED that page so it no longer relies on external resources – I’ve updated the script accordingly – also removed the DARK SKIES node and in general it all just WORKS. I’ve tested Node-Red, Mosquitto, Grafana SQLite Admin and SysInfo to ensure they all come up without issue. All looks fine to me. Over the weekend I’ll likely go through the script looking for other potential updates. For now, all is well.
I have zigbee2mqtt running on a separate RPi to the one I use for the Script (excellent – thanks).
I tried today to combine the Script installation (a fresh one) with zigbee2mqtt. I had node-red and Mosquitto mqtt running before I attempted to add zigbee2mqtt. The flashed dongle was found OK. I got stuck with installing –
sudo apt-get install -y nodejs git make g++ gcc
When I tested the node –version there was no response.
I got a positive result with npm –version
I woud be interested to know how you get on.
All of those are already preinstalled by the script, so you can just go on with zigbee2mqtt install, skipping that command…
from official install: https://www.zigbee2mqtt.io/getting_started/running_zigbee2mqtt.html
You can just go to the “Clone zigbee2mqtt repository” part of point 2… you already have nodejs 10 I think, I haven’t run the script for the last few months but v10 was there for sure, maybe now is even v12 (we just run the official nodered install script from inside Pete’s script, which takes care of downloading the latest nodejs version)…
I think that running that 1st command could have broken something, maybe going to overwrite the node version preinstalled by the script… restart fresh, and do as I said, it should work 🙂
Result (nearly). I have managed to install zigbee2mqtt along side the Script installations (node-red, mqtt, +=).
I followed your instructions and it was straight forward. Getting some cross-talk between RF signals and tasmota devices but that may be due to the reinstall of flows. S now running a single RPI to do all the work.
Thanks again.
Really helpful install script.
Can I ask if you think zigbee2mqtt could install and run along side the script installations.
never tried but i don’t think it can do any harm 🙂
i think Peter will be very interested in your findings 😀
I have to get zigbee2mqtt running first…
running it is not that complicated, but you need that usb dongle connected to your Pi 🙂
Hey Peter,
Thanks for your response. I managed to solve the issue by: installing Buster, and changing “wget –no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh” to “wget –no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/kAR5qG/master/files/script.sh“. Now I’ve run into another issue which I don’t understand how to fix.
Running the most updated version of Buster on a 3B+ now. It seems to choke on the Node.js LTS Install, then doesn’t install node-red after that failure.
Drops a lot of the following error codes:
script.sh: line 823: cd: /home/pi/.node-red/: No such file or directory
script.sh: line 1278: npm: command not found
This was based off a the Raspbian Buster for Desktop, I will try again with Buster with Desktop and Recommended Software, but don’t expect this to fix it… Any ideas? Thanks again for taking the time to help
That wget issue was probably caused by copying the wget line from the blog – not from the scdipt itself. If you point me to the relevnt blog andline I’ll fix that…. node – that;s for the node people to fix…. install exactly as recommended.
I got the working wget line from “The Script” from https://bitbucket.org/snippets/scargill/kAR5qG/the-script on line 46. On this blog you have /ekon6o/ instead of above. This blog post also requires a fix, as it also says /ekon6o/. https://tech.scargill.net/the-script-and-esp8266-code-for-beginners/.
I managed to fix the problem: Turns out my issue was in using the plain Desktop version of Raspbian Buster. When I used the Raspbian Buster with Desktop and Recommended Software, it worked out!
Big thank you to Peter for putting this awesome package together and providing it to the public. Excited to get using it now
Hi All,
I’m relatively new to using the RPi, and am trying to install the script without success. I’ve installed Stretch on my Pi 3+, updated and upgraded it, yet when I try to run the wget, I get the error in the image. I’ve tried Googling but can’t seem to find an answer to my issue. I’d really appreciate some help if possible. Please forgive my lack of knowledge around these things, I’m learning…
Don’t know if this helps, but this is the guide I’ve been following to install everything. https://www.youtube.com/watch?v=2gvRVaHxhW4&feature=youtu.be
The current version of Raspbian is BUSTER, not STRETCH… and no hanges o the script have been made in the last couple of months that would affect the wget or the instructions in thw script itself. Much as I’m grateful to the writer of the video you sent for mentioning my work I cannot comment on the advice of others. The two people who know the most about using that script would be myself and Mr Shark and we keep the script updated as best as possible in the face of changes elsewhere. I suggest loading a clean BUSTER and you should set everything up as user PI, not ROOT.
Thanks for all the work on the Script. It’s my goto for easily setting up a new Pi with most of the stuff I want. This week I used the Pi foundation’s new imager to flash an SD card with their latest version of Raspbian and then went through the steps to install the script. Has anyone else noticed that the CSS, reset.css, no longer works correctly on the Apache home screen?? I think I traced the failure to the call being made to typecast.com which is now redirected to another website. When I get some time this week I’ll slip in some of my own CSS to see if this is the culprit.
Tom, if you manage a fix, please DO comment back – I’ll fix script… I’ve not used for a while as I keep successfully cloning Pis.
Hi All,
What is the best way to remove all the apps that the script has installed, or to revert the script process all together.
sure, just format the sdcard and flash a brand new raspbian distro…
I assume you made a backup before using the script – using rpi-clone?
Dear Pete,
I am now trying to get a Buster system up and running on a rpi3+.
I took a copy from the image and ran The Script. copied all my flows and databases to it and no big problems.
I now found out that you did not copy the setup script of rpi-clone to usr/local/sbin! So with the script you cannot use -s option. Is this done with a reason?
I have major problems running rpi-clone on this fresh Buster system, but I must dig in a bit deeper to find out what is going on. Lots of fault system crashes grrrrrrr….
Not changed anything recently and the script work with Buster do not sure what is wrong there.
In your blog about RPi-4 you give an example for rpi-clone with the -s option for a new hostname. As you do not copy the rpi-clone-setup file in the SCRIPT this cannot work. If I compare the Install instructions in Bill’s Github page he copies the setup script to /usr/local/sbin. (https://github.com/billw2/rpi-clone).
I use rpi-clone separately to script. You should copy rpi-clone and the setup file to your pc. I am in the USA on holiday so cannot get to my installation to take a look.
new toy! InfluxDB Studio, a (windows) GUI program to manage all your InfluxDB instances/databases/data in general: https://cmikavac.net/2017/01/02/influxdb-studio/
downloads: https://github.com/CymaticLabs/InfluxDBStudio/releases
main github: https://github.com/CymaticLabs/InfluxDBStudio
p.s.: Sorry Pete, i had to add this here just to have a future reference, as all the other influxdb blog articles have comments disabled 🙂
That’s Ok Antonio, if it is any good I’ll create a blog entry on the subject.
I saw this in latest Andreas Spiess’ video: https://youtu.be/noUROhtf0E0
I would love to thank you for your script. It has helped me immensly in the past. However now on the latest Buster the package selection tool doesn’t pop up after the initial Bash command. I’ve tried twice on new installs and the same thing.
pi@N-red:~ $ bash script.sh
==
== Welcome to THE SCRIPT!
== Total: 00h:00m:00s Cores: 4 Temperature: 46’c
==
Selecting previously unselected package libncurses5:armhf.
(Reading database … 39968 files and directories currently installed.)
Preparing to unpack …/libncurses5_6.1+20181013-2+deb10u2_armhf.deb …
Unpacking libncurses5:armhf (6.1+20181013-2+deb10u2) …
Selecting previously unselected package ccze.
Preparing to unpack …/ccze_0.2.1-4_armhf.deb …
Unpacking ccze (0.2.1-4) …
Setting up libncurses5:armhf (6.1+20181013-2+deb10u2) …
Setting up ccze (0.2.1-4) …
Processing triggers for man-db (2.8.5-2) …
Processing triggers for libc-bin (2.28-10+rpi1) …
==
== Grabbing some preliminaries…
== Total: 00h:00m:25s Cores: 4 Temperature: 47’c
==
!!!!pi@N-red:~ $
Am I doing something wrong?
I don’t see what went wrong here but then you’ve not provided any clues as to what hardware and software you are using this on – I’m no even sure you are on the latest script – maybe I need to put the last update date on that title. I can tell you that the current script (which has not seen any mods in weeks) works with Buster on RPI3 and 4 (not tried RPI2 recently but I’ve no reason to doubt it will still work).. only this week I installed the script on Debian – not even on a Pi but on a PC running the Windows Linux SubSystem – the script was fine apart from some Apache issues – but after getting nearly everyging going I abandoned the lot because it turns out that the Windows Linux subsystem (WSL) and the Windows “store” “Debian” (Buster) are not a direction worth pursuing – can’t do this, can’t do that, however, by the time I came to that conclusion the script had run through from one end to the other – and as I say, that is on a PC, not a Pi. On Pi3/4 with Buster it is fine. Just for confirmation: when I refer to Buster on the Pi I’m generally referring to the non-Noobs version, either with or without the graphical desktop.
While I’m here griping about Windows – I also had a go at installing Debian in a virtual PC on my Windows 10 PC. It ran like a DOG – at the end of the day I got more speed out of a Raspberry Pi 4 than any of the attempts to emulate it. You have to ask, given the cost of an RPI3 2GB+PSU, is it even worth trying to emulate… better to get a small monitor (I have an HD 10″ on the way) and just stick the Pi + PSU on the back of it, perhaps.
I apologise for not giving more info, i got ahead of myself.
I’m using RPi3 on Buster lite.(no NOOBS)
Im using the script with version of “This version of “The Script” October 26 2019″.
I shall give it a few more goes. maybe the SD card is corrupt. Thank for a quick respons
Since both of you agreed upon that the RPi Zero is under-powered I went out and got me some toys.
Bought an RPi4, and two cheap compatibles – just to try and learn.
– The RPi4 is easy enough
– Banana Pi M2+ – it was available and cheap enough
– Orange Pi zero – so outrageously cheap, i could not say no.
Installed Armbian Buster on the BPiM2+ and “the script” just breezed thru, no problems that I could see, and a working system, with the basics, MQTT, Nodered, Webmin. Performance just as good as the RPi3B+ with quick boot thanks to the onboard 8GB emmc.
The OPi0 was more of a challenge, only because the Armbian Buster Image seems broken, wasted hours getting it to boot – but nothing.
With an Armbian Bionic Image it booted up and your script also went thru without notable problems.
Amazing, what kind of performance is available from these little SBCs.
The RPi4 is overkill for my home automation so it is now running LibreElec with the latest version of Kodi. Good picture quality, great sound.
Excellent Mike – glad it all worked for you. As for RPI4 being overkill – maybe now but wait until you get more ambitious later – it should have plenty of reserve – and when the RPI guys get their act together, there should soon be USB 3 boot for it which will make it even better 🙂
Pete
Hi Peter
I have been experimenting Chromebook’s Linux beta and it is very exiting environment to run Node-red and etc. I tried to run the script without success. Got following:
==
== Leaving script as you are neither PI nor ROOT user.
== Total: 00h:00m:22s Cores: 4 Temperature: Unknown
==
Regards,
Jaakko from Finland
Hi there Jaakko
I don;t think anyone ha ever suggested this would work on a Chromebook… certainly if you are not on a Raspberry Pi you need to start off as user “root” and that is available on every Linux machine I’ve tried.
Regards
Pete
Okay, tried this cockpit – looks nice – BUT:
After installing it my Zero started acting up, every reboot yielded a new IP address, and my router showed it with ever changing mac addresses.:-(
Took me a while to figure out, that the cockpit installation also installed a “network-manager”.
“sudo apt-get remove network-manager” fixed that annoying “feature”.
Cockpit still works without it.
First a big THANK You !
Great work that is really appreciated.
Andreas Spiess got an old fart (60+) started in home automation with your script and now a year later I am still learning and still awed at the amount of work you guys put into this and the perfection level archived.
I have my “productive environment” on a pi 3B+ and until now no test & deployment environment – so today I just tried a new install on a RPI0W: Buster lite and your script –> It just plain works!
Just the basics Mosquitto, Nodered and Webmin – no problems and even performance is more than acceptable.
Thanks!
Mike in Germany
Hi Mike – thanks for the kind words – and while I’m here, I don’t use Webmin these days but it is there for anyone who wants it… check this out… it’s LOVELY…
sudo apt install cockpit
sudo systemctl enable cockpit.socket
Then go into a browser and look at your Pi on port 9090 (user name and password can be pi and your own password.. tick the little box underneath). I have this installed on two machines (one Pi4, one AcePC) and can view them both from either.. hope that’s interesting, I should add it to the script. Oh and 60+ is not old – I’m 65 and it took a stroke in December 2017 to slow me down but I’m picking up where I left off… hence the recent interest in WLED, Tasmota and similar – oh and Antonio may have me interested in RpiEasy – looking at that tomorrow to see if the port can be shifted from 80 as I use that with Node-Red Alexa nodes (81 would be nice as it is sitting doing nothing)… but I digress (how unusual).
Good! but on a such a low memory device (and single core cpu…), i suggest pulling down Webmin, leaving it installed but not auto-started, and starting it just when you really need it… you can use Webmin itself to do its “harakiri” 😀
Well,
I have it running on my new Raspberry pi4, 4GB.
Installed the lastest buster from raspi website, installed pihole (with their curl script), the script.
Installed the latest mosquitto according to https://mosquitto.org/blog/2013/01/mosquitto-debian-repository/
with sudo wget http://repo.mosquitto.org/debian/mosquitto-buster.list
Then I updated nodejs after watching https://www.youtube.com/watch?v=z9a_ztJqaII
Please do the rebuild in the .nodered subdirectory
starting node-red:
Welcome to Node-RED
===================
27 Oct 12:10:58 – [info] Node-RED version: v1.0.2
27 Oct 12:10:58 – [info] Node.js version: v12.13.0
27 Oct 12:10:58 – [info] Linux 4.19.80-v7l+ arm LE
27 Oct 12:10:58 – [info] Loading palette nodes
27 Oct 12:11:02 – [info] Worldmap version 2.1.4
27 Oct 12:11:02 – [info] Dashboard version 2.17.1 started at /ui
27 Oct 12:11:02 – [info] Settings file : /home/pi/.node-red/settings.js
27 Oct 12:11:02 – [info] HTTP Static : /home/pi/.node-red/public
27 Oct 12:11:02 – [info] Context store : ‘default’ [module=memory]
27 Oct 12:11:02 – [info] User directory : /home/pi/.node-red
27 Oct 12:11:02 – [warn] Projects disabled : editorTheme.projects.enabled=false
27 Oct 12:11:02 – [info] Flows file : /home/pi/.node-red/flows.json
27 Oct 12:11:02 – [info] Server now running at http://127.0.0.1:1880/
27 Oct 12:11:02 – [warn]
versions:
mosquitto version 1.6.7
Hope this helps someone.
Hi Squirrel
I guess I’m a little jealous as my RPi4s are only 2G (not that I NEED more than that).. yes, “the Script” – been updating that over the last couple of days and will soon add in the optional switcheover to NGINX instead of Apache. Not used Apache in a year and no missed it.. Also I have some bits I put into /etc/bash.bashrc – some aliases and functions – will add them in as iptions as I find them invaluable along with my CLS command though that DOES make some assumptions – it clears the screen and shows among other things processor temperature. How you do the latter varies wildly for different boards…
Here for example is my CLS (cls.py and an alias in /etc/bash.bashrc) on my newly re-hashed ACEPC T9, set up using the script.
And yes, Node-Red 1.02, Mosquitto. PHP etc all ran out of the box using the script on this PC.
Guessing not quite ready for ubuntu 19.10 aarch64 on pi4 yet…
For anyone not yet attempted 19.10 aarch64, my experience thus far was painless. HTOP shows the 2 megs of memory on my 2meg pi4.
Pete (and his supporters), I do love all your hard work and reviews!!
a little contribute so i can have an RPi4 and start testing 64 bit version of the script, would be appreciated 🙂
Personally I’m very happy with Buster… thanks for the kind words and continuing to look in and comment on the blog – it all helps…
Pete
I use a Raspberry pi4 4GB as a PI-Hole.
After running the script I found out that there are some things missing:
Mosquito repository not correct. It installs a version of more than a year ago.
I have no idea where all my http ports are. Is there a central file with that information? Pihole can be accessed by going to http://192.168.1.7/admin/index.php but not to http://192.168.1.7.
Is is possible to enhance the script to include pihole?
Cheers,
Are you using the latest script? Msquitto worked fine last time I tried – sure, jst send the instructions for PiHoe and PiVPN and I’ll add them in as options…
hello Pete
just some info
Trying a little experiment to add my own nodes in your script. All loaded ok, but!! at the moment bcrypt version is 404 anyone with problem on rollback and fallout of script IT IS NOT A SCRIPT PROBLEM
FROM NODE_RED INSTALL LOG
> bcrypt@3.0.6 install /usr/lib/node_modules/node-red/node_modules/bcrypt
> node-pre-gyp install –fallback-to-build
node-pre-gyp WARN Using request for node-pre-gyp https download
node-pre-gyp WARN Tried to download(404): https://github.com/kelektiv/node.bcrypt.js/releases/download/v3.0.6/bcrypt_lib-v3.0.6-node-v64-linux-arm-glibc.tar.gz
node-pre-gyp WARN Pre-built binaries not found for bcrypt@3.0.6 and node@10.16.3 (node-v64 ABI, glibc) (falling back to source compile with node-gyp)
SCRIPT LOG last 2 entries then stops
############################################################
Installing node “node-red-node-sqlite”
+ node-red-node-sqlite@0.4.0
updated 1 package and audited 13221 packages in 12.761s
found 1 high severity vulnerability
run `npm audit fix` to fix them, or `npm audit` for details
############################################################
Installing node “bcryptjs”
+ bcryptjs@2.4.3
updated 1 package and audited 13221 packages in 11.983s
found 1 high severity vulnerability
run `npm audit fix` to fix them, or `npm audit` for details
regards Brian
Hi
I think the new version of nodered has caused issues with some of the package.
The following packages fail completely on my system:
node-red-node-sqlite
node-red-contrib-sqldbs
node-red-contrib-graphs
node-red-node-arduino
node-red-contrib-chatbot
I get a few warnings on some other packages but nothing serious
Nowhere NEAR enough info – your system being? Hardware? Full upgrade or starting from scratch etc ? You’ll need to check with the Node-Red guys…
from scratch they should work, but upgrading causes issues with some packages, I’ve blogged the issues I had.
I had tried on an old install and had the issues, could have been due to messed up rights and permissions.
I redid the install on a fresh Raspberry PI 3B last night and got different results.
“Linux raspberrypi 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux” “Raspbian GNU/Linux 10”
node-red-node-sqlite (with a lot of error/warning in the output) and node-red-node-arduino installed. I have not validated their function, and I’m not going to.
however node-red-contrib-sqldbs,node-red-contrib-graphs and node-red-contrib-chatbot were missing from the palette after install. A manual install from inside nodered passes on node-red-contrib-graphs and node-red-contrib-chatbot but fails on node-red-contrib-sqldbs
I don’t know why the script would be skipping packages. Thats all
Sqlite always does tons of warnings. Bot the script. Sqlite install is awful.
maybe npm reaches some or other limit (ram/swap/log level) and decides its had enough. I think node-red-contrib-sqldbs depends and sqlite, could contribute to the issue. If i get the time i will have a look on the nodered forums to see if there are compatibility issues worth interest
the nodes Pete decided to include in his script are not A MUST… you can add other yourself, or remove other not used… if some causes issues, remove them, especially if you don’t even use them… you can always readd them in the future, if needed… sqlite is a f…ing mess, always, it takes LOT of the script install time…
Hey, just a quick question, I installed Grafana with the script some time ago, and when I connect via ip it ask for a username and a password.
I dont remember setting special password and username and normal ones dont work. Any ideas?
you should watch on grafana website for info on default credentials AND how to recover a lost one… AFAIK, we never set up a password for grafana, it’s on my todolist but forgot about that, so never implemented in script AT ALL…
Thanks, I found the credentials and looks like It was my first login, so it asked to set a password after the login.
Great script so many thanks to both!
Hi Pete – Just a minor point that might assist others who are getting strange results! If you ‘wget’ the Script and use the URL in your 3 Sep note (above), it will fail because of the ‘!’ in front of the ‘api’ component. However, if you put single quotes around the entire URL, it works perfectly! That is:
wget ‘https://bitbucket.org/!api/2.0/snippets/scargill/kAR5qG/files/script.sh’
OK, the Linux gurus probably knew this – but if you don’t, it’s a pain to keep getting a fail! Thanks, as always, for everything…..
Thank you Mike, I’ve just removed that exclamation.
Re my last – I can see that Python3 is installed, and from what I read, the advice is *not* to delete Python2. So, all is good! Thanks again, Pete.
Glad all is well.
Thanks Pete. That 23 Aug script worked exactly as I’d expect. I can access the ‘Pete’s Utilities’ web page and Webmin is working. Running the script still threw a few warnings – the npm warnings specifically. I note that it said ‘run npm audit fix’ to correct these, but after rebooting and trying this, I get an error ‘No package.json found: Cannot audit a project without a package.json’. Should I be concerned – or is it one of those things that can be safely ignored? I also noticed a warning about ‘Python 2.7 reaching end of life on 1 Jan 2020’. Again, something to consider, or wait for a while to try and fix? Please accept that these are in no way a criticism of the invaluable work you do and the advice you so generously give. I think I speak for many, many people around the world when I say how much we all appreciate your zeal! If I may offer a small piece of advice myself – could you make the various dates at the top of this page into clickable links to the specific script? I’m sure that would help those of us that find it a bit difficult to determine exactly what script should be run – and, more importantly, where it is located. Your text says “..here is the script..” but the first link is some way down the page under the February 2019 date. You appear to have to go a long way down into the comments to find a link to a later script! Anyway, again, all appears to be well and, as I have in the past, I’ll look forward to using the various tools with the newest version of Rasbian. Thanks again!
Python: DON’T remove 2.7, it’s VITAL…
npm: they’re useless junk we could not get rid of, even after asking nodejs people…
package.json: did you run your commands from inside the .node-red folder, right? I think not…
The warnings are not from the script itsself but from installations within the sciprt and hence out of control of the script. The NPM Audit FIX is just a suggestion – I ignore it, I’ve never had that “no package.json notice – I can only imagine you ran some node install(s) outside of the NR filder as MrShark suggests. And yes, do not remove Python 2.7
Hi Pete – Looks like the Script is increasingly broken on Buster RPi3B+ rev 1.3. I’ve had great success in the past, but trying to run the latest script (well, I assume the latest – can I gently say that you don’t make it easy to get the ‘latest’. You really do have to search for it!) is giving heaps of errors. For example, Node-Red constantly tells me that there is a ‘npm WARN’ and ‘no repository field’ nor ‘no licence field’. Sqlite has so many warnings and errors that they are too many to document. Then, at the end, I get messages that it is ‘unable to locate chronograf’ amongst others. After rebooting, and trying my Pi IP address in a browser, it tells me it ‘refuses to connect’ although looking at the same address with port 10000, Webmin works prefectly. Webmin actually tells me Apache is ‘unused’ – and I’m pretty certain there’s no nginx server there? Now, all this is very quick – I haven’t done any real research yet, although I can say I have tried the script three times now, re-flashing a new OS each time, and always the same issues. Buster?? Any comments gratefully received!! Thanks for all your efforts
Hi
Wrong- The script latest version is called “the script” – and is currently 28/08/2019 and is the highest on in my list of scripts at BitBucket. Just used it recently. Once you make sure you are using the right one (and on Raspbian Buster preferably) do some back with any mistakes.
Thanks to Mr Shark for the correction, I originally said 28/02.
Pete
Thanks for the reply, Pete. However, I believe you mean 23/08/2019? The one that starts with “This version of “The Script” August 21 2019 includes changes to handle Raspbian Buster, RPi4 and other recent changes”? I can’t see anything later. But, apologies – I was downloading the ‘The Script Early July 2019’ because it is clearly headed “Created by Peter Scargill last modified 2019-08-21”. I can now see where the later version is (although I still believe the dating leads to some confusion, as I said before). What a difference 2 days makes! Haven’t had an opportunity to try it yet – I’ll need to start from scratch and it’s late at night here in Oz. I will, though, and report back how it runs.
Let me know how you get on Mike, I do try to keep the entries in order but don’t always get it perfect. This is not my job, I do it for my own entertainment in the hope that others may benefit.
Great script Peter,
but on the July 2019 Edition, once I see the select items to install screen, how is it possible to select or deselect an item?
If I go with arrow keys over the item, it is not possible to select and deselect it.
If I just hit enter over an item, it starts with the process without letting me selecting the items. So I choose abort.
Thank you very much for your help.
Best regards
I’m another happy successful user of the Script. You may want to beware of Webmin. A backdoor was discovered in the App which has been there for a year.. Version 1.930 is the fix to shut down the backdoor.
yup: https://thehackernews.com/2019/08/webmin-vulnerability-hacking.html
Thanks folks, I had to do a re-install today because I broke something yesterday, ermmmm!
Anyway I think I spotted a bug during install – either grafana or influx, I think it was influx. It appeared like a typo in a repo list; “infux” instead of “influx” which threw an error like “no file” or “no path” exists…
Can these be manually added with apt-get if it is the case that one didn’t install?
Thanks
G
i see “infux” indeed, but not in the apt part, it’s in the node list (twice in script) that will be installed… Pete, please correct, and Garry please share the log files you have in your home folder
I just checked the last couple of updates to the script on Bitbucket – can’t get past FIND “infu” – doesn’t exist apparently
“infux” was not in this (old) edition of the script:
https://bitbucket.org/snippets/scargill/ekon6o/the-script
while it’s in this one and all the following versions:
https://bitbucket.org/snippets/scargill/qexexb/the-script-2019
but, as said, it’s only a NODE, not the apt part…
So I am not an IT wiz… and I am in the office all day today.
I will see what sort of install log I can find and report back. The “apt” question was to ask; can I retrospectively add uninstalled components which failed due to the typo with the apt-get utility just to avoid running the script again?
Thx, G
sure you can, but again, the only errors i’ve found for your report is in the nodes install section, 2 occurrances of “node-red-contrib-infuxdb” instead of “node-red-contrib-influxdb” (“L” missing), nothing else… to install any missing piece, use npm if they’re nodes, or apt if they’re packages, no problem in this…
log is in your /home/pi directory, named something like script+datetime.log
Get you… thanks. Looks like I just manually search and install any missing Node-red nodes on my Pallette…
Using my new VPN (thanks for this too Pete!) I can see the log on my Pi at home. Will see if I can SCP to my phone or something, will email to Pete if I can.
G
log: share it on pastebin.com
nodes: you can just stay as you are and eventually install missing nodes if you ever use them, Pete makes a (very) good selection, but you may not need all of them…
https://pastebin.com/h91k7K3D
look in there, search for “infu”, and then for “influ”… the former, you’ll see only the error about the node mispelled and therefore not installed, as already stated and corrected in today’s script thanks to your report; the latter, influxdb regularly installed by apt, and service activated…
Not sure why the issues with locale settings – what did you install that on?
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
Influx sorted…Sqlite always throws out that crap. The installation procedure for SQLITE is AWFUl, but works.
I think you can ignore that Pete, I am sure I broke and fixed that all on my own! I recall being asked to confirm default locale which I pointed to IE UTF8 etc but think it threw an error. Thanks for the follow up and sorry for the false alarm, my fault!
Locale is set, all OK, no more errors…
Thanks
G
Magic – thanks for the feedback.
yes, those errors come out as soon as you change default locale… that’s why i just change keyboard settings, and leave that untouched
All the influx errors fixed along with unrelated spelling issues… and files renamed do that “The Script” is the current version (as it should be)- date reference (today) is inside the file itself.
Loving the script… however a couple of fixes. wiringpi has been pulled from the github by the author very recently, see the explanation at their site. however it is currently available as a .deb package
https://project-downloads.drogon.net/wiringpi-latest.deb
that installs on most recent machines.
and the sections for influxdb and grafana should probably not use tee -a for the apt sources, just tee like the mosquitto section does.
Thank you for that.
all done and sent to Peter, he’ll update bitbucket
about wiringpi, read this sad rant from author…
http://wiringpi.com/wiringpi-deprecated/
I take it all back, fixed lists properly and it appears to have intalled fine.
Clone of Pi model B from original production to Model 3B+ with Pete’s latest recommended additions working fine, I am toggling stuff with MQTT.
Many thanks Pete and your supporters!
G
Are there issues with mosquitto at present? I have tried to manually fix package lists for to install mosquitto but having dependency issues:
The following packages have unmet dependencies:
mosquitto : Depends: libssl1.0.0 (>= 1.0.0) but it is not installable
Depends: libwebsockets3 (>= 1.2) but it is not installable
FYI it seems I managed to get Pi-Hole and PIVPN installed OK and cloned my NodeRed and the static IP to the new Pi, should drop in to replace the old one if I can get Mosquitto to run!
Thx G
Thanks again for the hard work chaps! I have had a “script-equipped” Pi running for some years, it is actually a Pi Model B from the original wait list batch from the product launch. It is physically embedded in my home alarm control box and takes 12Volts from the backup battery/charge circuit via a dc-dc converter and uses GPIO to monitor the alarm.
I have used all the script goodies to build home automation including WiFi connectivity to deep sleeping ESP8266 sensors on my garden shed to sympathetically alarm the shed with my house alarm and of course lots more including intensive use of Bigtimer.
Time is cruel however and (thanks Pete!) I am never one to leave well alone so I am now cooking up a spare Pi Model 3B (buster) with the script and will add PIVPN and PI-HOLE. I am hoping this is transparent to all the automation devices but in any case thanks for the help!
Good luck, Gary,
Short update. Running on latest Buster light now and all works perfect! 😉
Thanks for your help!
@Peter: Light version of raspbian starts a lot faster than the full blown version and usually also runs more stable and with better performance. As long as you are running headless this is golden.
You have me wondering what it is about the lite version that makes it run faster. I wonder if merely turning off the graphical desktop on the full blown version sould have similar results…
it’s the exact same thing, if you disable gui… it’s faster just because you don’t have to wait for it to initialize and start the gui, nothing more… you can install gui on lite, or remove from the full one and have it lite
Thomas – nothing I’ve done has ever claimed to work with the LITE version of Strech or any other variation. For all I know there could be important command line tools missing from Stretch Lite. Antonio, what do you think?
lite version are fine, they’re just the headless/server editions, with no gui… i always use them as i’m not interested in ever connecting a monitor to my rpi, so no need to waste sdcard space or bandwidth when updating… that’s the only difference, the base is the same…
of the gui variant, there are 2 editions: with basic tools (a browser and no much more), and full blown (with office, graphic apps, scratch and all the other stuff)…
i prefer lite editions, as said… of course, whatever you choose, select Buster as the base, no more stretch… whatever tool we need is added by script in the requirements stage (i added wget and ssh recently, for example)…
Alright… will migrate to buster then! Is https://bitbucket.org/!api/2.0/snippets/scargill/kARod4/master/files/snippet.txt still the most recent version for the script? Thanks!
Hell no.. https://bitbucket.org/snippets/scargill/kARod4/script-july-20-2019-inc-buster-and-rpi4
Go to Bitbucket – my snippets – in order – latest at the top. Good decision to go for Buster. It gave me grief upgrading, but in the end worth it and things will only move on so may as well be up to date.
they’re the same link, one is in the bitbucket page, the other is in raw format, ready to be copypasted or downloaded 🙂
Thanks for this awesome script!
However I cant install any additional nodes (neither via Node-RED GUI, nor via CLI npm install …). Regardless what node I try to install I always get this error message (see screenshot)
I completely reinstalled multiple times already to no avail. Setup is RPi3 with latest Stretch
You’re not using NOOBS are you? Also, if you are starting from scratch you might want to look at recent items in here, stretch is now old.. BUSTER is the current version of Raspbian.
sudo chown -R pi.pi ~/.node-red/node_modules
anyway, as said by Peter, use Buster and NOT the noobs version
Unless you have existing setup there really is little point in NOT using Buster… however, read the discussions in here surrounding the BUSTER entry. I’ve now succeeded in totally upgrading old installations to Buster. JUST upgraded the last of my SDs to the larger Buster BOOT size. I had to use GPARTED to make a fresh SD with Buster-compatible partitioning and my original Stretch-or-earlier-created code and programs – then used that as the master for subsequent clones, which require a boot and clone with -f mode the first time they are used – subsequently simple cloning as normal.
Thanks for the timely reply. I already tried the “chown” trick and indeed I can install a new node after that using the GUI. But bad things happen: Once the new node is installed all other nodes are all gone after a reboot and Nodered GUI won’t even start anymore! The directory /.node-red/node_modules/ is literally empty after that.
@Peter: I did not start from NOOBS, never used it. I downloaded the last version of “stretch lite” from the raspbian repo. Everything else is really working like a charm. I think this is not a linux distribution or kernel issue. Seems to be something is wrong about file permissions about bcrypt.
Thank you very much! Your help is much appreciated.
Cheers!
it’s something happened in the past, we couldn’t address the issue, something related to npm… suggest moving to buster, btw, no real reason to stay on stretch…
Hello Pete and Mr Shark
confirm script worked by me today on RPI4 Buster Light.
sorry a bit late was delayed in hospital,
regards Brian
Hope you’re ok, Brian, been there myself today – EEG tests. Knew nothing about the subject this morning, do now. Checked out methods and kit…
I like to understand things so when we got back from Granada I checked what they did to me – looked for abnormal brain patterns under severe/varied strobe lighting compared to none. Seems I’m good. Not sure why the guys in local area couldn’t do that. Countless thousands of Euros? Not necessarily, it would seem.
Apply probes and gel electrodes etc, ensure all working properly Repeat same tests with and without strobes. All over in half an hour. Not complaining,, a nice day out – just wondering. Tech types might find this marginally interesting. https://shop.openbci.com/collections/frontpage. Kit can be cheap, open-source software free, specialists expensive I guess.
hope you’re both fine… thanks for the report 😉
Hay can someone help me i am trying to follow this and install it on my Pi 3 but i got this error while installing “iptables-persistent”:
IPv4: Unable to save (table filter isn’t available or module not loadable)
ip6tables/1.8.2 Failed to initialize nft: Protocol not supported
IPv6: Unable to save (table filter isn’t available or module not loadable)
so i carried on with this command
sudo iptables -I INPUT 1 -p tcp –dport 80 -j ACCEPT
and now i get this error
iptables/1.8.2 Failed to initialize nft: Protocol not supported
I am using Buster on a Raspberry Pi3
don’t know/not script related, btw, look at last comment here: https://bbs.archlinux.org/viewtopic.php?id=241668
Gents,
Firstly, thank you for all your hard work. The install went flawlessly on my Rpi 2.
With Buster now being the latest release, PLEASE put the link to the Buster install up first on the page. I tried and failed to install with the link on the front page over the period of a full day. I almost walked away until I dug through this thread.
However, nice work and it is at 100% now.
The command to download the Buster version of the script is:
“`
wget –no-check-certificate -O script.sh https://bitbucket.org/api/2.0/snippets/scargill/kARod4/master/files/snippet.txt
“`
Yup
test script for buster here: https://pastebin.com/DmnMtcEE
all important stuff tested and working, what’s still out:
samba: it asks in early stages for a dhcp mod, just press enter when asked, leaving its defaults… have to see how to bypass this as it blocks full unattended install
java: even using official info, it says java not found or the likes… BUT, as it’s needed only for habridge, and nowadays who cares about habridge, i’ll remove it alltogether… oh, and java is attempted to install even if no habridge selected…
for Pete: add grafana to your index html page, and i’ll remove that OPI Monitor, as its same 8888 port is used by chronograf, installed if you select grafana, which brings back influxdb, too, of course…
p.s.: tested webmin, too… for me, 95% of the work is done, only minimal tweaks needed… not bad for 3 hours i’ve finally found to work on this…
latest script: https://pastebin.com/6eWvbJan
all working (nodered, mosquitto, grafana, influx, chronograf, apache+php, phpliteadmin, phpsysinfo, habridge)
changes:
-using openjdk for buster, as oracle package is broken (it tries to download a file which points now to a 404 error page)
-latest phpliteadmin as previous one does not work on php 7.3, which is default in buster
-java is not installed by default anymore, you need to check it explicitely, but it’s used only for habridge right now… you may need it for blynk local server, if you use it…
-removed opi-monitor to avoid conflict with chronograf tcp port 8888 (installed by grafana)
known bugs: samba asks for that dhcp question, i don’t know how to avoid it… the package causing it is SAMBA-COMMON, feel free to find a solution…
i’ve an ftp server installed on port 21 by default, don’t know why (pure-ftp) of if it’s default on buster…
for me, work done…
Pete: modify your index page to:
remove opi monitor
add grafana on port 3000
add chronograf on port 8888
add habridge on port 82
Dear Sir, I am 72 years old retired Telecom engineer and my hobby is IoT. I am really fascinated by your script and my hearty thanks for sharing this.
My node-red works fine, but the Drop down list of “tts voice” is empty and hence no audio output.
Please suggest me any tips to resolve this issue.
With very Great Regards.
I don’t do anything with TTS voice so I’m afraid I can be of no help here. My Node-Red is talking to Amazon via an EXEC node and another to talk to the MP3 player which handles Amazon Amy output, and another EXEC node handles the MP3player. All of this works a treat.
So i am triying to install the script on a raspberry PI Zero W and ive run into some problems, if i leave the defaults setting and add influxdb and grafana, it will only install the node-red but not the mosquitto, or influxdb or grafana, also the web browser doesnt work i am using the raspbian buster lite any suggestions of what i could do different to make it work?
simple: change distro and board, as none of them is supported by script (well, we didn’t test RPi0W in 2 years, now, i think, because it has only 1cpu and 512kb ram, sure you can’t run all that stuff on this board… web browser? You installed a gui, too? Really?)
at the moment i am testing with raspbian stretch as i saw this video https://www.youtube.com/watch?v=2gvRVaHxhW4&feature=youtu.be and realized he is using stretch will let you know if all works, and he did use a raspberry zero w which has 512 MB of ram not KB.
oh, nice video! Pete, you should add it to article, i think 🙂
yes, of course it’s MB and not KB, but simply RPi0W is not powerful enough to do anything good using the script, sure not all the stuff you selected… last time i ran the script, it took about 6 hours to complete… just madness, there are better options…
So just if you are interested I was able to install the script on stretch which took over 2 hours apparently everything is working as you can see in the picture, so i am using the RPI0W because thats all I have at the moment and its hard to get them in Mexico for a good price, besides this is for testing my home automation with MQTT and ESP32, do you have any other advice for me or link to manuals or tutorials that might be useful for me it would be great, you guys are awesome and do a great job, also i did the same several times with the RPI0W and Buster and the script doesn’t work have you tested on RPI3B+? or what hardware do you advice to use? thanks again for your time
understood, good! The worst part of when we tried last time was sqlite and webmin, which both brought install time to 6 hours…
forget buster for now, nowhere is advertises as it working, Pete did some testing UPGRADING from stretch and this seems to work, while fresh install from buster will not…
i don’t understand this rush to jump on the running train… stretch works? yes… script on stretch works? yes… period! What services of buster do you REALLY need? None…
use what’s suggested, or contribute patches to help it works on unsupported hw or sw platforms… i don’t have time right now to work on this, nor it’s a need, for anybody…
what you all use are nodered, sqlite, mosquitto, which are the same if running on stretch or on buster, so, cool down everybody, i need my holidays as anybody else…
Thank you so much, I don’t have a need to use buster is just that I am really new to raspberries and I get confused a lot, as I used the links provided in the video I thought I was using the same version but later found my mistake, I didn’t install webmin as in my previous tests I found it crashes the PI0W, even with the installed software my PI0W only has like 32 MB free of ram, so yeah I’m gonna take out the unnecessary software, and then get a 3B+.
I would not dream of installing Webmin on the PI0W, the latter being too under powered. Indeed I don’t install Webmin at all these days but left it there for others who might want it.
With the 4 being the current model (looks like mine was dead on arrival unfortunately, not sure what to do about that at this moment) the 3B+ will no doubt soon come down in price. Even the 2 is fine really… but the lack of speed and RAM on the PI0W is why Antonio and I stopped work on it, I sold mine off. See my comments clarifying Pi3B+ as it’s clock speed is definitely higher than the original model 3. 1400Mhz as against 1200Mhz… but of course if you want to use USB for a drive, the 4 is a good idea as USB on the PI3 is slow.
Marvellous, nice to see the script is working for you using Stretch and RPI0W, personally now I use RPI2 as a minimum though given the relative cost of 2 and 3, it seems to make more sense to go for the latter and thinking about it I closed down the last RPI2 a matter of months ago for no other reason than “need for speed” – I COULD have stayed where I was of course. I STILL can’t comment on RPI4 as mine has still not turned up from the UK, posted to me last Monday. I tried RPI3 B+ and a clean Buster install and the Script (or rather some of the dependancies as used in the script) failed, so I went back to my fully working Stretch setup and upgraded THAT to Buster (3 liner) – and THAT works a treat. Hopefully at some point as clearly all the programs DO work in Buster, we’ll get to the bottom of why the difference between an upgrade and a clean install. Since the upgrade I’ve done further minor updates and they TOO work perfectly.
hello Pete
Raspian Buster STATIC IP eth0 editing DHCPCD.conf does not work you need to edit
/etc/network/interfaces Here link to guy whos done it
https://linuxconfig.org/how-to-set-a-static-ip-address-on-debian-10-buster
regards brian
oh, back to the origin…
if running script on buster you get stopped while running prelimeries
at !!!!pi@raspberrypi4
try updating if you cant you could be on the trial repository
Buster is now stable
run once!! apt-get update –allow-releaseinfo-change
this worked for me
giving N: Respository ‘http://raspian.raspberrypi.org/raspian buster InRelease’ changed its ‘Suite’ value from ‘testing to ‘stable’
Very useful, thanks for that Brian. I upgraded my existing setup to Buster so no issues but that might be useful to other readers.
Thanks Brian, good tip.
as i am installing the script on new install of Debian buster under VM, so i didn’t got this problem.
Did that too before running the script – still had some issues: see post above.
HI,
i tried to update the script on Debian buster under a VM, but its showing few problems, especially
E: Unable to locate package grafana
E: Unable to locate package grafana-data
E: Unable to locate package chronograf
and mosquitto,
with mosquitto its showing
“E: Unable to correct problems, you have held broken packages.”
i can live without grafana, as i use separate VM for that, but mosquitto is a real issue. even log don’t show much information.
but here is the log file link
https://pastebin.com/qtazAey7
any idea?
As I’ve only updated my script based system on a Pi 3 to Buster I can’t be of much help at this time. The update worked without a single hitch. Elsewhere I have detailed most of the packages and exactly what I did (very little) to update/upgrade.
ideas? Just don’t do that… personally never tested script on buster, too new, nor i plan to do it this summer as i’ve other plans and things to do, eventually it’s stuff for september and going on… if you have a working setup, keep it or try the adventure at your own risk, as it was never advertised that script can be used to update anything, nor that it works on a fresh install of buster… want to try? Do as Peter did, coming from a previous Stretch install…
Thanks
and i understand, script is for raspberry pi, I just wanted to try new things and perhaps help a bit with testing.
I spend few hours and the script is working now, and here are the changes I made.
* I am not a expert or skilled programmer, and I know you guys can write it much better.
For mosquito.
debain buster by default come with 1.5.7.1 (20190213) the latest version at mosquitto.org is 1.6.3 (20190618), so I option to go with buster default.
Original Code:
if [[ $OPSYS == *”UBUNTU”* ]]; then
sudo apt-get $AQUIET -y install mosquitto mosquitto-clients 2>&1 | tee -a $LOGFILE
[ $? -eq 0 ] && mosquitto=1 # mosquitto installed
New Code:
if [[ $OPSYS == *”DEBIAN”* ]] && [[ $DISTRO == *”10″* ]]; then
sudo apt-get $AQUIET -y install mosquitto mosquitto-clients 2>&1 | tee -a $LOGFILE
[ $? -eq 0 ] && mosquitto=1 # mosquitto installed
import thing was $DISTRO == *”10″* must be “10” and not “10.”
note: I also try to add the mosquito repo for Debian buster, but its gpg key is not signed and failing
PHP:
In debian buster php7.3 is the default version of php, and php7.0 is not recommended as per security reasons, so I tweak the code like this
Original Code:
if [[ $OPSYS != *”UBUNTU”* ]]; then
if [[ $OPSYS == *”BIAN”* ]] && [[ $DISTRO == *”9.”* ]]; then
sudo apt-get $AQUIET -y install apache2 libapache2-mod-php7.0 sqlite3 php-sqlite3 php-xml php-mbstring 2>&1 | tee -a $LOGFILE
else
sudo apt-get $AQUIET -y install apache2 libapache2-mod-php5 sqlite3 php5-sqlite 2>&1 | tee -a $LOGFILE
fi
[ $? -eq 0 ] && apache=1 # apache installed
else
sudo apt-get $AQUIET -y install apache2 libapache2-mod-php7.0 sqlite3 php-sqlite3 php-xml php-mbstring 2>&1 | tee -a $LOGFILE
[ $? -eq 0 ] && apache=1 # apache installed
fi
New Code:
if [[ $OPSYS != *”UBUNTU”* ]]; then
if [[ $OPSYS == *”BIAN”* ]] && [[ $DISTRO == *”9.”* ]]; then
sudo apt-get $AQUIET -y install apache2 libapache2-mod-php7.0 sqlite3 php-sqlite3 php-xml php-mbstring 2>&1 | tee -a $LOGFILE
else
sudo apt-get $AQUIET -y install apache2 libapache2-mod-php7.3 sqlite3 php7.3-sqlite php7.3-xml php7.3-mbstring 2>&1 | tee -a $LOGFILE
fi
[ $? -eq 0 ] && apache=1 # apache installed
else
sudo apt-get $AQUIET -y install apache2 libapache2-mod-php7.0 sqlite3 php-sqlite3 php-xml php-mbstring 2>&1 | tee -a $LOGFILE
[ $? -eq 0 ] && apache=1 # apache installed
fi
Here when [[ $OPSYS == *”BIAN”* ]] && [[ $DISTRO == *”9.”* ]] is failed, and loop goes to else statement , I assume that now we are in Debian 10 and change the php5 to php7.3.
As I said, it may not be the beautiful way to do.
For Influxdb:
it was the easy one, I added the new repo for Debian buster
test $VERSION_ID = “10” && echo “deb https://repos.influxdata.com/debian buster stable” | sudo tee -a /etc/apt/sources.list.d/influxdb.list
I don’t use Webmin or have any experience with it, so there I am not sure if its working or not.
I hope this will help someone.
thanks, i’ll try as soon as i’ve a spare rpi (3+, not having 4 till september i think), and let you all know
Hi there!
Fist up: Thx for the great script.
Got to know about it through Andreas Spiess.
Here a small heads up:
Got the same error message as Azee on a Pi Zero W
running “Raspbian Buster with desktop and recommended software”:
script log: https://pastebin.com/09bRsCvQ
During install also java 8 was not able to install.
Found that the new Raspian “Buster” (2019-06-20) does not support java 8:
http://downloads.raspberrypi.org/raspbian/release_notes.txt
* Oracle Java 7 and 8 replaced with OpenJDK 11
So retried with Raspian “Strech Light” (2019-04-08) which did work.
script log: https://pastebin.com/J6jmznsE
node-red install log: https://pastebin.com/Mn34rRRc
The installation ran through far cleaner.
On first glance everything seems to work.
There were some hickups though:
– There seems to be an vulnerability issue with “node-red-node-google”.
– “node-red-node-sqlite” popped up multiple issues.
– java showed up following error:
E: Package ‘oracle-java8-installer’ has no installation candidate
Greetings,
Jonathan
You are right Jonathan, Java 8 is no longer supported in Buster. I didn’t Select the Java update in the menu, so i missed that.
and Debian Buster comes with Open JDK 11, so easiest way will be to install that, but it will install complete JDK and not just JRE (at least that what i am aware of).
On the other hand its good to go away from Oracle JRE/JDK, as recently Oracle changes it License agreement, and its no longer free ( free is a broad word, simply you must be on some Oracle agreement ).
i will try to install Open JDK 11 tomorrow and let you know how it went.
Used The Script on a RPi3 and everything worked great.
I would like to install on a Rock64 as well. Any tips?
Which OS to start with?
Will everything work the same as with RPi?
Thanks
I’ve not yet used the Rock64 so no comment from me.
armbian xenial (NOT bionic) or stretch: https://wiki.pine64.org/index.php/ROCK64_Main_Page
should work pretty much everything, but of course you can have problems with gpio…
Recommended program for installation script
zigbee2mqtt
What credentials do you mean? I entered a new password for pi, for root, and for node-red user and admin. I didn’t do anything (that I know of) about mosquitto.
(some time later)
I did a search about passwords on mosquitto, then found the password file. There was an entry with the administrator name I entered for Node Red. There was no indication that this would also be used for mosquitto.
Thank you for pointing me in the right direction.
every service installed by the script (except grafana…) share the same user and password you put in on script run, in case of mosquitto it’s the admin user and password… use that, or search the net on how to remove the password or change it…
I just did a new install onto a Pi3 with Raspbian lite. I am trying to test mosquitto but I am not having much success. I am very green when it comes to Linux.
I don’t know what to expect from this script. Should Mosquitto be configured to run as is or is there additional setup that I must do first? I tried an initial test with ssh with the following result. (and on the Pi with lower response)
pi@Broker:~ $ sudo /etc/init.d/mosquitto start
[ ok ] Starting mosquitto (via systemctl): mosquitto.service.
pi@Broker:~ $ mosquitto_sub -d -t hello/world
Client mosqsub/850-Broker sending CONNECT
Client mosqsub/850-Broker received CONNACK
Connection Refused: not authorised.
(repeating)
(on Pi)
Client mosqsub/1153-Broker sending CONNECT
Client mosqsub/1153-Broker received CONNACK
Connection Refused: not authorised.
…
I made Stretch Lite image on a new sd card. I did an install of Mosquitto and it works as I expected. I am able to use mosquitto_sub, mqtt-spy, and an ESP8266 board with mqtt running on it. All good. Unfortunately it does not give me all the programs your fantastic script provides. But the critical part I needed was the mqtt. I’m sure I will get the rest in time.
if you entered credentials during script running, then same credentials are used by mosquitto to authenticate with it, so just use them 🙂
the ones used are the “admin user” ones
As Antonio says (he beat me to it) you need the admin credentials
**Thanks** indeed DrFragle… That got me going.
The “Serial” input and output nodes were missing but unlike before I have been able to add them via the manage palette.
I guess there is a minor bug due to some old i2c code which is currently problematic with the current script. I don’t need i2c so no worries.
My previous inability to add the serial nodes was probably due to using an old script.
I still have much to learn. Now for making an image of my SD Card. Life is too short.
I seem to be the unlucky one. My previous installation with “the script” from a while ago had no serial node in node-red… the rest seemed OK.
So today I rebuilt my HomePi from scratch – 3B.
– New light Raspbian with desktop downloaded.
– New Script with wget.
– Ran perfectly I believe.
– Reboot —
– No NodeRed. Won’t start from VNC either. I get
———————
pi@HomePi3:~ $ node-red-start
Start Node-RED
Once Node-RED has started, point a browser at http://192.168.1.21:1880
On Pi Node-RED works better with the Firefox or Chrome browser
Use node-red-stop to stop Node-RED
Use node-red-start to start Node-RED again
Use node-red-log to view the recent log output
Use sudo systemctl enable nodered.service to autostart Node-RED at every boot
Use sudo systemctl disable nodered.service to disable autostart on boot
To find more nodes and example flows – go to http://flows.nodered.org
Starting as a systemd service.
Started Node-RED graphical event wiring tool.
Error loading settings file: /home/pi/.node-red/settings.js
Error: The module ‘/home/pi/.node-red/node_modules/i2c-bus/build/Release/i2c.node’
———————–
Freaking me out! Any help would be appreciated.
remove/comment out references to i2c in settings.js
live happy
OMG! OMG! OMG! facepalm to me!
For almost 2 days I have had trouble with getting the node-red dashboard to show up.
The browser would only show the top row with the node-red logo in my chrome browser.
In my rasbie it was running the nodes. Finally I found out what the issue was.
F;ing ADBLOCKER blocked the login popup window. When I paused it for the page, the login window popped up and I could enter the admin and its password.
I will eat humble pie for dessert.
Btw, glances+bottle seems to work now.
I manually installed them via pip, as does The Script.
Piero
Hi Pete, Antonio,
using Raspbian on a Rpi3B with a just downloaded copy of the script, the habridge installation fails because the java installation fails… because the gpg key C2518248EEA14886 is missing.
I’ve not yet tried habridge, but java and habridge installation seems to work by manually installing the key.
Thanks all,
Piero
never had issues with that… anyway, Peter, add this line to latest script after line 1152:
sudo apt-key adv –keyserver hkp://pgpkeys.mit.edu:80 –recv-keys C2518248EEA14886 2>&1 | tee -a $LOGFILE
and now that i see that, the full block about java (1145-1157) should go BEFORE the habridge one, otherwise this one will not find java when installing…
All done (while re-installing Windows, Office and 100 other packages).
new node-red 0.20 is out, check changes here:
https://nodered.org/blog/2019/03/12/version-0-20-released
Hello Antonio
Thanks for the information re usb access to zero pi.
Iv looked at it and filed it as useful information for future problems to access
units via ssh
regards
Hello Pete and Antonio
So last update on pi zero with The Script OS raspian lite.
From LOG
/home/pi/.node-gyp/10.15.1/include/node/v8config.h:324:3: note: in definition of macro ‘V8_DEPRECATED’
declarator __attribute__((deprecated(message)))
^~~~~~~~~~
CXX(target) Release/obj.target/bindings/src/serialport_unix.o
CXX(target) Release/obj.target/bindings/src/poller.o
CXX(target) Release/obj.target/bindings/src/serialport_linux.o
SOLINK_MODULE(target) Release/obj.target/bindings.node
COPY Release/bindings.node
make: Leaving directory ‘/home/pi/.node-red/node_modules/@serialport/bindings/build’
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN node-red-project@0.0.1 No repository field.
npm WARN node-red-project@0.0.1 No license field.
+ node-red-node-serialport@0.7.0
+ node-red-contrib-ibm-watson-iot@0.2.8
+ node-red-contrib-play-audio@2.3.2
+ node-red-node-smooth@0.1.0
+ node-red-node-random@0.1.2
+ node-red-node-ping@0.0.16
added 153 packages from 98 contributors and audited 492 packages in 191.389s
found 0 vulnerabilities
My Grandaughters (2 0ff) did vibration tests and generally behaved as hooligans in testing this unit I have never seen the ammount terror made on a unit in my living
Eventually it became very sluggish in response to repeated rgb requests (started to loose contact then come back) presumably mosqitto got snowed under with msgs that I had to reboot
So final verdict the script with raspian lite works on pi zero
The pizero:-
is suitable for small tasks 1 room and no children.
Didnt try io ops only used mosquitto as a server.
The competition at the price 10€ is far better : – ESP32
and at 4€:- ESP8226
end
Regards
Hello Pete
pi zero with script
further news sqlite works no problem. Have set up 20 favourites of rgb in the
datbase with facility to change or add more records automatically from the colour at the moment for my grandaughter.
As I didnt select grafana and influx can not report on these
Hello Pete and Antonio
I tried as you suggested to use raspian lite which is my normal os. instead of dietpi
As the zero has no wired lan only wifi and I did not have a micro hdmi cable I had to go by the wifi route
I normaly contact my os by ssh.
So my procedure was this:
1) Burn 8Gb sd with raspian lite useing etcher.
2) MAKE an empty file and name it ssh with no extension (to enable ssh in raspian by boot). copy this file to boot partition in the sd card
3) Make another file called wpa_supplicant.conf
and enter my ssid and key for my wifi
e.g
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
network={
ssid=”FRITZ!Box 7490″
psk=”xxxxxxxxxxxxxxxx4611″
key_mgmt=WPA-PSK
}
Copy this file to the boot partition on the sd card
4) Put card in zero and power on
when the zero connects to your wifi you can connect by winscp ssh as normal
run get script run the script
ABSOLUTELY NO PROBLEMS only time it took 2hrs 3mins
ended at 0200hrs
I have loaded flows (7) webmin updated 4 progs including mosquitto
have darksky as per your way running.
My grandaughter is over the moon with controlling it and the rgb ights in her room
and it hasnt once lost contact.
It is naturally slower than piB but not that you really notice. No doubt if you really load it their would be problems. I was surprised that webmin ran as it is quite a load for this.
anyway the news is The script with raspian lite is usable on pizero W
Regards
Good!
There’s an other option: https://desertbot.io/blog/ssh-into-pi-zero-over-usb
Hello Pete and Antonio
actually took 49mins 23secs on zeropi but failed to do most important
part. No problem I only wanted it for grandaughters room to setup seperate
small server for her room controls, thought Id save a 3B, but now Iv got the
old (very old Pi ) working I will use that. The diet pi looks good especially the pre boot
setup configs in dietpi.txt and wifi.txt. I didnt have cable to be able to connect, so did all through wifi.
regards
try raspbian lite on the pi0, as dietpi changed a lot recently and could be that, i don’t use anymore nor i’ve tested it recently… i prefer stock distros like debian or ubuntu… let us know 🙂
hello Pete
forgot os diet pi on the zero
and raspian Lite on old rasberry
regards
i’ve no more any RPi Zero to test, nor i will: they’re just too limited for this kind of setup, with just half gb of ram and underpowered cpu… sorry…
I tend to agree with Antonio. If it works on the Zero, fine, but dont have any to test and wont have any doing forward. Sorry,
and just running script on that device will take about 6 hours, for what i can recall… madness…
hello Pete
ran same on rasperry pi zero W
runs perfectly except did not load npm and node-red
regards
Hello Pete
Just used latest script on my sons old raspberry pi dated 2011.12 which he (at the age of 38) was throwing out. Script worked flawlessly including grafina and influx
I did not use webmin. Used your new node-red + npm
regards
Glad thats working.
Hi Pete,
first thanks for the great work, i am silent user of your work for a while now and now finally want to start my iot system at home. and here is my question(s)!
first i install your script without “Grafana and Influxdb” and after seeing the Andreas Spiess video i decided to install the “Grafana and Influxdb”. while only checking the “Grafana and Influxdb” option ,it output
”
E:Unable to locate package grafana
Failed to start influxdb.service no such file or directory
.
.
”
(more in attached screenshot)
it seems bit strange to me, that it can’t find the package,where it clearly getting something from the repo.
now as a last resort i am willing to start over but i saw another strange command (i might be wrong here 🙂 )
in the script and at “Installing Grafana and Influxdb”
for both debian 8 and 9 you are using same repo
$VERSION_ID = “8” && echo “deb https://repos.influxdata.com/debian jessie stable”
$VERSION_ID = “9” && echo “deb https://repos.influxdata.com/debian jessie stable”
where influxdata have a specific repo for the debian stretch
“deb https://repos.influxdata.com/debian stretch stable”
so just to understand is there a specific need to use “jessie” for debian 9 as well?
*i am using debian9.7 on a VM
at the time that part was done, there was no stretch and the same setup worked for both… i need to take a look at that part, then… let’s see tomorrow, if i’ve a bit of time…
try this: https://pastebin.com/YmZLemxG
modified lines 1192 (for influx stretch version) and added 1198 (for grafana stretch version)
if you already run before, suggest removing these 2 files BEFORE running new script, as they will contain the old repository, duplicating it if running the script again:
/etc/apt/sources.list.d/influxdb.list
/etc/apt/sources.list.d/grafana.list
HI,
today i redo the complete installation with your new script, but i encounter the same problem.
——————————————————–
Installing Grafana and Influxdb
E: Unable to locate package grafana
E: Unable to locate package grafana-data
E: Unable to locate package grafana
Failed to enable unit: File influxdb.service: No such file or directory
Failed to start influxdb.service: Unit influxdb.service not found.
Failed to enable unit: File grafana-server.service: No such file or directory
Failed to start grafana-server.service: Unit grafana-server.service not found.
Failed to enable unit: File chronograf.service: No such file or directory
Failed to start chronograf.service: Unit chronograf.service not found.
——————————————————–
here is the complete log
https://pastebin.com/fvPYcaVw
to add that i can’t see if
“/etc/apt/sources.list.d/grafana.list”
is created .
tomorrow (today) i’ll do an install in a debian stretch vm and test it…
testing right now
bug confirmed, working on it
issues should be solved by this version, just sent to Peter, too: https://pastebin.com/MLmHQesS
grafana now has an official repository, the one we used till now seems messy… now it and influx are both installed correctly, just tested on Stretch in vm…
commented out “glances and bottle” because of python errors…
added “net-tools” to standard requirements, as stock debian is now missing standard “ifconfig, netstat, etc”
Thanks a lot, i will test it tonight.
just test it now, working perfectly, without any problem.
thanks again 🙂
new video by Andreas Spiess who uses the script to install a grafana/influxdb setup 🙂
I did now a fresh install of standard Raspbian, updated it and started Peter’s script.
Here is the log file: https://www.dropbox.com/s/gfu59je7c70qqa2/script.sh-2019-02-06_14h57m.log?dl=0
and here the node-red logs (I found two in different directories, (maybe they are the same):
https://www.dropbox.com/s/y4t9o7arpmx3m0c/nodered-install.log?dl=0
https://www.dropbox.com/s/g77ucxchhj1omlw/nodered-install1.log?dl=0
After that, I had to manually install Moment and after that, Node-Red, influxDB, and Grafana worked. Unfortunately, not all nodes were installed. So I had to install the node contrib-InfluxDB from the palette. Everything was done on a RPi3.
Maybe this helps.
could you (or anybody) please try this version?
https://pastebin.com/terk37AA
the ONLY change is that now nodes are installed 1 by 1 by this “for cycle”, instead of as before when ALL of them were installed in a single instruction… this way if 1 of them fails, well, 1) it’s reported explicitely in logs, and 2) cycle will just skip it and go to the next one…
PLEASE, when testing TURN OFF quiet mode in initial script menu, so i can get FULL logs, thanks… It’s untested… i just run it commenting out the actual npm install line to see if the for cycle was ok, and it seems so…
for addonnodes in moment node-red-contrib-config node-red-contrib-grove node-red-contrib-diode node-red-contrib-bigtimer \
node-red-contrib-esplogin node-red-contrib-timeout node-red-node-openweathermap node-red-node-google node-red-contrib-advanced-ping node-red-node-sqlite node-red-node-emoncms node-red-node-geofence node-red-contrib-moment node-red-contrib-particle \
node-red-contrib-web-worldmap node-red-contrib-sqldbs node-red-contrib-ramp-thermostat node-red-contrib-graphs \
node-red-contrib-isonline node-red-node-ping node-red-node-random node-red-node-smooth node-red-contrib-npm node-red-node-arduino \
node-red-contrib-file-function node-red-contrib-boolean-logic node-red-contrib-blynk-ws node-red-contrib-telegrambot \
node-red-dashboard node-red-node-darksky node-red-contrib-owntracks node-red-contrib-alexa-local node-red-contrib-heater-controller ; do
printstatus “Installing node \”${addonnodes}\””
npm $NQUIET install –save ${addonnodes} 2>&1 | tee -a $LOGFILE
done
Hi Antonio,
I tried the new file. Here the results:
https://www.dropbox.com/s/uhbuivvy7r662h5/script.sh-2019-02-07_08h27m.log?dl=0
https://www.dropbox.com/s/tocfrzpxzhederq/nodered-install_new.log?dl=0
It worked after reboot and I see many of the nodes. I am not sure if the influxDB nodes should be installed by the script. They are not there, but I was able to add them using palette install.
Looks much better now!
great! I think the problem was with one of the nodes failing install, so as the install line was a single one, this node caused ALL install failing… as it’s now, it will eventually fail the single node, who cares… i did not go into all log details to see which one is the bad guy, and the nodes installed is a list of common ones, mostly used by Peter himself or requested by others because useful… if Peter wants it’s easy to add influx node, too…
Peter, you can publish this version of the script on bitbucket, then… as said, the only change is that the nodes now are installed 1 by 1 via a “for cycle” instead of a single instruction with all the nodes at once…
well done (pat pat on my shoulder 🙂 )!
Yes well done.. and I’ve added some other stuff then published.
I am busy constructing a script that installs only the extra nodes by taking pieces from THE SCRIPT.
I notice that node-red-node-sqlite and node-red-contrib-sqldbs are still in the for loop. They should be installed separately?
While installing sqlite nodes there is no option –save. Doesn’t it mean that it is not registered in the package.json?
don’t know, honestly… because the nodes you install via packages.json are the ones with same permissions… of if you install one with unsafe-perm or installed globally, better to leave them out…
You don’t need the save option any more and now if sqlite rails it not take the rest down. Incidentally I am using RPI 2 and 3 and ALL features work.. and have survived upgrades.
So here are the results of the jury.
0 votes to node-red-node-sqldbs. This one tries to install sqlite v3.1.3 and blocks or crashed(Pi was unreachale from the web). I had to pull the plug. But here is the log https://pastebin.com/FyvvH3WK.
This is the script I used for this https://pastebin.com/Bu33dYuN
I followed the information in the package.json and it seems this node is virtually not maintained anymore, 2 years of inactivity.
There are also a number of nodes which are deprecated and the owners need to update the compatibility of newer versions.
SQLite itself is not a problem as it installs a binary v4.0.6 which is compatible with v10 of NR.
I suggest to leave out the sqldbs node from the standard extra nodes. Be aware that sqlite will be installed twice with the current SCRIPT. If it is necessary to install in the special way stays unclear.
I am just wondering if it is possible to change the utf8 validation node to a higher version than referenced in a lot of nodes.
It is not such a bad idea to split the installing of node modules from the main script!
i think you’ve found the assassin! Peter, strip out that node, if you look in logs it tries to reinstall sqlite, too, i think the problem was there, now it came out as nodes are installed one by one… i think nobody uses it, too, and it’s 2 years old, as Leo said…
about utf8 node, you’re free to update elseways, as in script there’s no version information so it should always be latest version…
SQLDBS now removed, other minor additions. Also fixed outstanding issue with bigtimer.
sqlite node is still installed twice!
does it work? If yes, who cares 😀
look in logs and see where it’s installed the second time… but in any case, it CANNOT be installed twice, you always get one, the second time in case is just reinstalled, harmless procedure… but if you tackle down what (re)installs that, we can fix it
It is in the list without –unsafe-perm and in the special command with –unsafe-perm and without –save
bcryptjs gave problems when trying to update from Package Manager, changed perms to pi.pi to solve. I suppose that is still a problem. I will try to generate the fault by resetting permissions to root.root.
Otherwise set permissions to pi.pi after installing all extra nodes, as you do in your update script.
that’s used just to manage credential encryption… leave it there as it is and live happy, or update it via npm, look at command in script
Sorry to say, but I had serious trouble both with PM and npm. I tried now with root.root and installed blockly without trouble. Probably another problem and a wrong fault message when I ran into trouble. In your update script bcryptjs is changed to pi.pi with all the others in node_modules and there are no problems with that.
BTW I will never install the extra nodes anymore other than the ones I need, there are to many deprecated nodes in this NR v10. The menu could have an extra choice for this.
all node-red setup is run as pi.pi, so setting it as root.root is just an error, we never said to do that… in case, just change owner before doing updates or installs:
sudo chown -R pi.pi ~/.node-red
that’s a tilde, not an hyphen… ALT 126 on keypad to do it
Just to clarify.!
By some reason the bcryptjs folder was owned by root.root when I got the problems. And there were more directories with it. No idea what went wrong, but that was with a system that would not update at all! That was a few weeks ago. Now things behave normally again.
The setting of bcryptjs to root.root was to try to force the error again, but no error happened. So I forget it, think it was a glitch somewhere in the setup.
I’ve removed the first one in each case.
:))
I would go in and separate out the NR code but I’m too busy sorting out BigTimer issues (moon) which I’m pretty convinced are down to the third party sunCalc – it’s almost enough to make me want to write my own suncalc.
I think you can get them from weather stations as well and forget about suncalc?
Maybe this?
https://developer.here.com/documentation/weather/topics/example-astronomy-forecast.html
thanks but needs to be an offline resource.
Suncalc actually works fine normally and no-one is stepping in re: the moon so I guess very few people are trying the moonup and moondown.
It is a regular service like the one for the sun. Why should this be different?
I think the moon part was added in later.. and generally you don’t have the issue of the sun always being up or never being up. getHours and getMinutes don’t like invalid times and so the flags alwaysUp and AlwaysDown I use as a check.
It’s all there in the Javascript.
Pete
I tried now at least 10 times to install the new script from scratch using several versions of the Raspbian image as well as the script options with no success.
The last time I copied the script to the ~directory and started it with the bash command and the following options: https://www.dropbox.com/s/7hz1v4ay04rq2wz/04-02-_2019_10-56-08.jpg?dl=0
It created the following log file: https://www.dropbox.com/s/s2c4rkyn3ibgkhq/script.sh-2019-02-04_10h55m.log?dl=0
After reboot node-red does not start and if I press:
pi@grafana:~ $ node-red-log
I get:
Started Node-RED graphical event wiring tool.
Error loading settings file: /home/pi/.node-red/settings.js
Error: Cannot find module ‘moment’
Started Node-RED graphical event wiring tool.
Error loading settings file: /home/pi/.node-red/settings.js
Error: Cannot find module ‘moment’
I am not very familiar with Linux, so maybe I made a simple mistake.
Hi Andreas, try this:
cd .node-red
npm install moment
or comment out the moment line in settings.js if you don’t use it
Peter, seems this damn “moment” node is causing various problems… is it needed? can we just strip it out from script, and who wants it will install himself?
or you can even leave it there, but remove or comment out the relative line in https://tech.scargill.net/iot/settings.txt which is the template used by script to create a proper settings.js file
Hi Antonio, thank you for your help. I installed moment according to your instructions and now it works and I was able to get influxdb connected to node-red and Grafana to influxdb.
I am working on a video using your script. This time for InfluxDB and Grafana…
it’s Peter’s script… i’m just a humble contributor 🙂
btw, strange “moment” that was not installed during setup, as it’s there in script… and you’re not the 1st one complaining about this node… in the past we had issues with sqlite (WTH it takes so long to install and its developers can’t release a binary package???), and with serialport and even with i2c nodes… never with moment, till recently…
IMHO moment is not the problem! During the running of the script all additional nodes are removed!!! , including moment js.
I am going to install a fresh system again to see where we are at the moment….
at the MOMENT i’m working and not able to fix…
everybody feel free in any MOMENT do provide proper patch to Peter…
y’all have the script and can modify and report back with a solution…
thanks
I believe it is the NR script which does the deletion – see NR guys about that?
don’t know, as nr script is run before installing nodes, and after that we just install a bunch of nodes, so if we’re INSTALLING, what should REMOVE those nodes… don’t know, honestly i need others testing script and reporting back… who have a spare rpi, or just a spare sdcard, try it, please… without loosing time testing grafana or apache or others, i don’t care about those, just install basic requirements+nodered, that’s the problem right now…
Additional info.
Andreas has the same problems I have. It throws away 514 nodes in the end of the node installing process. I suspect this is because of the nodes that deprecated and npm cannot cope with all the faults happening.
See my log:
https://drive.google.com/file/d/1ues9Y2J8p7MCdZ-2ECCFcZ22M1n5zkYy/view?usp=sharing
You most probably can restore all the nodes by installing again after the Script is ready, but I did something wrong somehow and I got lost in permission problems.
Here’s a thought. Miss NR out of the script install – and use the recommended NR install/upgrade script – I’m not sure how we got landed with NR install issues.
Peter, that’s exactly what your script does right now, look at line 797: https://bitbucket.org/snippets/scargill/qexexb/the-script-2019#script.sh-797
it will JUST download and run official script… JUST that…
then it will JUST install your usual bunch of nodes… JUST that…
tomorrow i’ll separate these nodes install in smaller groups with addons log writes, so we can address better WTH is going on…
p.s.: i moved definitely to HASS.IO with nodered addon… 0 headaches, it JUST works… just updated to last version which adds a simple flag to turn on/off the dark theme, and updates usual nodes (bigtimer is now at v2.1.4)…
and when hass.io doesn’t work are we back to a heavy backup session? You are braver than me, HASS.IO installation for me was not trivial and didn;t work properly so I wiped it.
For now I’m happy with RPI normwl – I’d be happier if the Node-Red guys could make this NODE update scenario more reliable and hassle-free then people wouldn’t pin this on the script.
Well all I know is that over a year ago, I downloaded the script, and ran it on a RP3. After that I copied the SD card, and duplicated it for a few other Pi’s
I’ve never seen the need to update anything, and it all just works. The only thing I do differently is get rid of the psuedo sudo security and change the pi user to user 0 and group 0,
In other words Pi is now root. If anyone thinks that prepending every command with sudo somehow makes a system more secure, well if a user named Pi exists, then the chances are it has sudo access with no password.
BTW, this also eleviates the need to chamod on every file and system resource.
I know I’ll get backlash for suggesting such an insecure installation, but given the Pi was built for hardware hackers and learners, why the hell make it harder than it needs to be
backup in hassio are GOOD… you create a snapshot, and download it, or you can have them automatically uploaded on google drive or dropbox… you can use the SAME snapshot to move your setup from a raspberry to a linux vm or what else can run hassio… the only drawback it’s that you’ll have “home assistant”, too, and in case you don’t use it it’s just there doing nothing more than wasting (few) resources…
you had issues with home assitant itself as far as i remember, but that needs a little study as it’s a HUGE world… i’m just starting scratching its surface, right now…
i can’t see errors in there, aside that damn sqlite which takes 50% of the full logs with its own crap…
please share the other log you have in your home folder, the nodered-install.log one, as that part being a separate script now, has it’s own logs…
Search for “514” and you will see the error. And since we have v10 there are those nodes which are deprecated and can’t be build without errors Sorry but I erased that log when lost in permissions, only saved the scriptlog.
Leo – please tackle th eNode-Red guys – this is down to their script/NR install.
If I could give a clear picture what it is happening!
If I look on the Internet there are 3 package managers npm, pnpm and yar. Lot of people mention the f..up of the lock file of npm under certain unknown circumstances.
All 3 use package.json, but the differences are also annoying. I decided, after a few tries, not to mix them in one install. So I am sticking to npm.
that line means nothing to me… there are few f..ing lines:
797 installs nodered via ITS OWN setup script:
https://bitbucket.org/snippets/scargill/qexexb/the-script-2019#script.sh-797
line 798 copy THAT script nodered (the one you deleted) and changes its permissions in your home folder…
line 800 installs a bunch of nodes, nothing fancy…
line 807 install that damn sqlite, and your error is related to that part of the install… well, that’s the way NR guys told Peter to install that damn sqlite… so, remove it and live happy, or go complaining with sqlite guys for doing a shitty job in packaging their on software and not being able to provide a binary npm package…
i’ve enough of all this mess not related to script but to other people scripts or install procedures…
It was not a line reference. The disaster happens on line 2858, suddenly 514 items are removed.
I am now doing an install with the NR method(nodenew) with installing the extra nodes including no sqlite. The extra Pi nodes are installed via this NR script.
We have to find a way to get a flawless install!
Not line 2858 of the script it isn’t. The script is 1159 lines in length.
It is in the log file line 2858. Don’t bother about it. Comes out of the blue.
Made an install without all those fancy extra nodes. 20 minutes and done! Copied in my flows and installing missing nodes by Palette manager no problem. SQLite needed an NR stop/start cycle. Copied my databases.
Absolutely no problem!
We are obviously getting a wrong URL when done via npm install with all the rebuilding. Does Palette seem to know where the bin is?
So what Peter already said. Install things without the nodes. After that every user is responsible to load necessary nodes for his/her nodes to add.
I found an issue using Antonio’s update script.
Although it recognized that there is a new version of contrib-bigtimer it did not install. I did a stop-start of node-red!
Why is the command to update made a comment?
it happens from time to time
that line is commented as i use the other method someone suggested on nodered forums: using “npm-check-updates” (ncu) to create the packages.json with latest versions, then running “npm install” which installs the missing ones and updates the installed ones… you can uncomment that line if you want, harmless AFAIK
OK I suspected already that there were more roads that lead to Rome.
I have been looking at the dependency tree and I am now very curious if there is not a better system to keep the same module in sync for all parents. Something like a normal linker will put only one object file into a program. What I see is a big number of modules in different versions because it is written so in the package.jsons.
I did some Googling and found this https://www.npmjs.com/package/next-update. Am I on the right road to a better system just like any other linked program? This is a weird situation!
you can one packages.json and change version numbers with just an asterisk, than it will download latest version
a good practice is saving every *.json file in .node-red folder… after reinstalling nodered, you can just put there your old packages.json and run npm install, which is now the suggested way to install and it’s the way many docker containers do…
there’s an other tool, alternative to npm, but we never used: yarn
I will have a look at yarn
I installed it looked some videos and concluded it is better than npm, but it is not a revolution. It is more sophisticated in the ways you can update packages. It holds a .cache with all your installed packages locally, so it is faster. I just ran it over the system with Node-RED installed by the Script and it respected at least all those packages not installed by other means and in other occasions, it took newer versions. So in a list you see sometimes more than one version of the same package. It has been organized flat and sorted so it is easy to find out which packages are loaded.
But if it is worth switching in the Script is doubtful.
I have just updated bigtimer yesterday – it might have been going through the system – current version is 2.1.3 (fixes a moonlight bug introduced in 2.1.1).
With Antonio’s script it was signalled there was an update, but because of the # sign it did not update automatically. In the palette manager it was also signalled. As Antonio said, it can be done automatically. You have written a blog about npm outdated and I used that for a while until I realised there were no updates anymore. Somehow my system went partially broke.
OK just a little report on my next try to install a new system with the renewed SCRIPT
I took the Raspbian without the recommended programs from the site.
I tried the new node install procedure and removed the node sqlite from the normal install(no double instals anymore) and move sqldbs also to the separate install line with safe perm.
The outcome was not perfect, but usable. SQL-nodes did not install, however Palette-Manager did install them!
There are some fault messages from deprecated nodes I will most probably never use them. Look in the logs of the install to see which ones are deprecated in this NR version
https://drive.google.com/file/d/1Txvvbi85twf3HzXHI3rwEHQh_YRVzrl-/view?usp=sharing
https://drive.google.com/file/d/1KH_D5vCDZV6aSDBzbvHDhGujfr0zg2c0/view?usp=sharing
The SCRIPT needs a small correction, in my opinion, to prevent double install of SQLite when using the NR script. May be we can convince the NR people not to include that question, by using a parameter like -u(unattended).
These exercises at least gave me some confidence I could build a complete new system from scratch.
BTW I tried the update script on an almost completely update system without any issues.( this one!: https://drive.google.com/file/d/1KH_D5vCDZV6aSDBzbvHDhGujfr0zg2c0/view?usp=sharing )
This was more than enough for me now, pfff, going back to where I was working on a week ago.
Hi guys, maybe this will help some people having some of the problems reported here. It may sound simple but SET CORRECT TIME on the raspberry pi just after starting up with the clean image. BEFORE you run the script. Some compilation actions go wrong and in a loop because of the time of the files and the current time of the Pi
Thanks for all the help. My RPI was built using the script about 6 months ago. Maybe I’ll start from scratch again.
I did actually. Please see final comment by “ukmoose”
https://discourse.nodered.org/t/rpi-serial-hangups/7398/4
Thanks anyway I’ll ask the question on that forum.
Thanks. Mostly, we get very few problems with the script, but those we do are often Node-Red and usually due to the SQL plug-in or the serial port on Raspberry Pi. I’m not sure who we got landed with Node-Red’s problems. However, just to say that everything I use was installed on Pi 3 using the script and I have the above and many more plug-ins working including a couple I added in not so long ago. If you follow any instructions the Node-Red guys come up with, NR should be fine.
just answered there: https://discourse.nodered.org/t/rpi-serial-hangups/7398/8?u=fragolinux
Unfortunately I struggle with linux. Am I supposed to be looking for a file called node-red-log? I’ve attempted to use the “find” command but have had no success.
If I may suggest asking Node-Red questions in the Node-Red forums – the IBM guys who created Node-Red are in there. Neither Antonio nor I created or maintain Node-Red.
don’t know what you’re doing… the (sub)script used by Peter’s script now for installing nodered is the official nodered team one: https://github.com/node-red/raspbian-deb-package/blob/master/resources/update-nodejs-and-nodered
and that take cares of downloading proper “node-red-log” and putting in right path (line 287) with correct permissions (line 292)… so, you just need to run “node-red-log” in console…
Hi
Does anyone know why “node-red-log” produces command not found.
I’ve loaded node red using the script.
check if it’s executable and where it is… remember that in linux the current dir is NOT part of the PATH variable, on the contrary of windows… so if you want to run a script in current folder, you have to prepend it with ./
./script.sh
to make it executable:
sudo chmod +x script.sh
eventually move it to /usr/local/bin, so it’s in path
I did an install of nodes by extracting from the SCRIPT and one I added myself.
I left out sqldbs as I noticed this node was not missing. Already present in the base install of node-red modules?
After copying in my databases everything(?) seems to be working.
So leave out node-red-contrib-sqldbs in the SCRIPT and the script might have succesfull install.
I will test that tomorrow, or if somebody else could do let it know here.
A bit different! I was probably just lucky by omitting sqldbs. I do not use this one, but sqlite which needs an extra node for the query. But together almost equal to sqldbs.
SQLite seems to be the problem.
After a very long time hanging there with many deprecated messages all my extra nodes thrown away (See line3161 of log). NR not starting because of moment not installed.
https://drive.google.com/file/d/1m39QiO3ch6Hg4n_xqc1APz47SOE7nPYO/view?usp=sharing (NR log)
https://drive.google.com/file/d/1v-t_qg4P6GyW6tw2oexbZLt51e9BchJi/view?usp=sharing (SCRIPT log)
I will try to reinstall them all except sqlite3 node(which is separate in Script)
Later more!
edit settings.js and just comment out moment… maybe you don’t even use it… sqlite node is actually a PITA which even nodered guys are looking for alternatives…
So I did a fresh install and it failed!
I miss lots of nodes. Looks almost the same problem as when installing or updating node. In node_modules folder there are more nodes left, but they don’t show up in the interface when NR has started
Node-RED cannot be started because of the missing node moment. When I take that line out of settings.js then NR starts, but then I see a lot of missing nodes. However, I can install them from the Palette manager.
This is serious because I cannot generate a new system from scratch by downloading Raspbian and the Script and use it as a backup and experimental system. I suppose nobody can get this done as I used a RPI3BP with the fresh downloaded Raspbian and the latest Script.
If you need more info please ask me what to produce so I can help solving these problems. I am willing to invest quite a bit of time in this.
MIJN logs:
https://drive.google.com/file/d/18Qi8gXjwtpQnoYLHhq_2fH_PR_gvtLMN/view?usp=sharing
https://drive.google.com/file/d/1PJ9lpZNFuOdxWwBE8KXViXZdB6vlo2zR/view?usp=sharing
There are quite a number of irregularities in the node-red log!
what script are you using? share it somewhere, as i modified it during christmas holydays and i can’t find any of my new additions in your logs… one logs refers to node v10 but installed in a way that should be different, the other log talks about node v8…
during install, did script halt and ask something… read here and next comments: https://tech.scargill.net/the-script/#comment-47438
Yes, it asked me whether I want to have v10 and I said yes! I used an image without the recommended programs so I see that NR is not found as I left the menu to the new Node and kept the 2 separate choices empty.
I downloaded the Script with the link provided in Blog wget…
I am now ready for a new attempt to run the script with the full version of Raspbian and I will throw away the Wolfram and LibreOffice including Webmin.
Will take an hour to see the result
I tested your identical setup in Christmas holydays and all worked fine, reported by others, too… Is there a new raspbian image out? Don’t know what to say…
just checked, raspbian is still at version i used, 2018-11-13: https://www.raspberrypi.org/downloads/raspbian/
official nodered install script (used by Pete’s script from 2019 version and going on), is still the same, 4-12-2018: https://github.com/node-red/raspbian-deb-package/blob/master/resources/update-nodejs-and-nodered
latest script using these specific version worked flawlessly less than a month ago, during tests…
if even this fails, try running the above nodered script manually, but in script it’s used just as it is, in fact it halts Peter’s script just to do its stuff, before going on with nodes…
and try NOT selecting hw support, maybe it’s that… but even that was tested…
I have been trying to get the script to work in a VMWare VM. There appears to be a permission problem with the npm install (After ‘Installing Nodes’). I got a load of permission denied , mkdir ‘/home/pi/.npm/_cacache/index-v5/xxx.
When I copied the npm install command from the script.sh (after cleaning) and ran it as sudo it installed (all-be-it with a few warnings and an error for serialport.
I can now access Node-Red control panel from the Type on Screen page
I am special,…. I know! It is always me to have this kind of problems…..
I going to build a new system with the latest version of all and default settings as presented when the script begins, no change of options.
B.t.w. when I try to install a new node I got the same trouble. I corrected that by copying back all my node_modules files from a backup disk. Had no problems after that with node-red, but I did not dare to update or add anything.
The only thing I see happening in the log that there have been thrown away 1260 files!!!
— removed 1260 packages and audited 166 packages in 47.676s, out of the blue.–
All trouble started when I was trying to update BigTimer node. First I ran into bcryptjs permission problem and after chown to pi:pi it did update and throwed away all my nodes except the standard ones.
I have some question about Mosquitto. Suddenly, after changing my SD card for a backup, I experienced some problems. Node-RED would not start and now Mosquitto is not starting automatically anymore. I am on Stretch and almost completely updated(only 3 packages still there).
What to do to get Mosquitto start automatically again and how to check if this, other than a reboot, if the settings are OK?
suggest installing webmin so you can visually see what’s enabled to start automatically on boot, check logs, etc… be prepared to a LONG install time, but after that, point your browser to port 10000 and go from there…
I did find it on System/Bootup… it says it is starting up(i did put it in /etc/rc.local) and running status unknown.
Meanwhile I have made some progress. Using linked clone VMs is a blessing. Unfortunately I need to use this…
Step 1. comment out the installation CD-Rom as source:
sudo nano /etc/apt/sources.list
Comment out (putting # in front) deb cdrom:[Ubuntu-Server 16.04 LTS _Xenial Xerus_ – Release amd64 (20160420.3)]/ xenial main restricted
This fixed installing NodeJS, NPM and with that Node-RED.
Step 2. Installing the moment module manually after the script ends: sudo npm install moment
This resolves the issue that Node-RED does not start.
Next issue to tackle is:
Noticing that still a lot of nodes are missing (by reference Pete’s BIG TIMER) and no Dashboard…
Although generated the en_US and en_GB locales the following error keeps nagging during script execution tests.
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = “en_US:”,
LC_ALL = (unset),
LC_CTYPE = “UTF-8”,
LANG = “en_US.UTF-8”
are supported and installed on your system.
perl: warning: Falling back to a fallback locale (“en_US.UTF-8”).
Hi Peter,
I am trying out your newest release of the script but it gives me some headaches. For whatever reason it will not install npm and with that no Node-RED is installed.
Trying to install it upfront, the scripts removes it and stuck in same error loop.
I am using Ubuntu 16.04 LTS.
Any suggestions?
I’m willing to bet this is another NodeJS issue
Try using old method, deselect new and check node + nodered
Hi all,
I think I have a similar problem as Tonyb. I’m using the 2019 “THE SCRIPT” on a dietpi and raspberry pi 3. My installation does not install mosquitto or apache and maybe other things too. Here is a snippet from my log file.
############################################################
Installing Mosquitto with Websockets
–2019-01-22 20:15:41– http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key
Resolving repo.mosquitto.org (repo.mosquitto.org)… 85.119.83.194, 2001:ba8:1f1:f271::2
Connecting to repo.mosquitto.org (repo.mosquitto.org)|85.119.83.194|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 3120 (3.0K) [application/octet-stream]
Saving to: ‘STDOUT’
0K … 100% 66.0M=0s
2019-01-22 20:15:41 (66.0 MB/s) – written to stdout [3120/3120]
E: Unable to correct problems, you have held broken packages.
Failed to start mosquitto.service: Unit mosquitto.service not found.
Failed to enable unit: File mosquitto.service: No such file or directory
############################################################
Installing Apache/PHP and Sqlite
E: Package ‘libapache2-mod-php5’ has no installation candidate
E: Package ‘php5-sqlite’ has no installation candidate
!!!! Apache+PHP+SQLITE+PHPLITEADMIN NOT INSTALLED! [0;97m
############################################################
My log file is too long to insert here but I have tried to attach it as a file renamed as a .jpg to see if that would work.
Any suggestions? Thanks!!
I have found a solution to my problem. I believe that there were some dependencies for mosquitto that were not being installed with THE SCRIPT. (Maybe DietPi was too light?) I was able to install mosquitto using the DietPi utility dietpi-software to create a working installation of mosquitto then re-run THE SCRIPT. Hope this helps other people.
don’t know why in those logs it’s trying to install php5… can’t stay on the changes dietpi does, it always worked, they sometimes change stuff and break things… just use raspbian, problem solved
I can’t comment on diet-pi as I’ve not used it for some time. I use only the full Raspbian (usually the version without the extras).
Hi again. re-ran the new script on the old 8gb sd card a new 16 gb sd card, same result everything loads but can’t open the web page ‘Site can’t be reached, refused to connect’. Everything else seems to work. I did notice that in the node red install list, I had ticks for most of the items but ‘-‘ for ‘Move global nodes to local’ and ‘NPM rebuild existing nodes’
As far as I can tell, logs are the same for both installs. Here’s the last one:
***************************************
Started : Mon 7 Jan 18:54:36 GMT 2019
Running for user pi at /home/pi
Failed to stop nodered.service: Unit nodered.service not loaded.
Found global nodes: :
Reading package lists…
Building dependency tree…
Reading state information…
Package ‘nodered’ is not installed, so not removed
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Reading package lists…
Building dependency tree…
Reading state information…
Package ‘nodejs-legacy’ is not installed, so not removed
Package ‘npm’ is not installed, so not removed
Package ‘nodejs’ is not installed, so not removed
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
dpkg: warning: ignoring request to remove nodejs which isn’t installed
dpkg: warning: ignoring request to remove node which isn’t installed
Reading package lists…
Building dependency tree…
Reading state information…
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Reading package lists…
Building dependency tree…
Reading state information…
curl is already the newest version (7.52.1-5+deb9u8).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
## Installing the NodeSource Node.js 10.x repo…
## Populating apt-get cache…
+ apt-get update
Hit:1 http://raspbian.raspberrypi.org/raspbian stretch InRelease
Hit:2 http://archive.raspberrypi.org/debian stretch InRelease
Reading package lists…
## Confirming “stretch” is supported…
+ curl -sLf -o /dev/null ‘https://deb.nodesource.com/node_10.x/dists/stretch/Release’
## Adding the NodeSource signing key to your keyring…
+ curl -s https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add –
Warning: apt-key output should not be parsed (stdout is not a terminal)
OK
## Creating apt sources list file for the NodeSource Node.js 10.x repo…
+ echo ‘deb https://deb.nodesource.com/node_10.x stretch main’ > /etc/apt/sources.list.d/nodesource.list
+ echo ‘deb-src https://deb.nodesource.com/node_10.x stretch main’ >> /etc/apt/sources.list.d/nodesource.list
## Running `apt-get update` for you…
+ apt-get update
Hit:1 http://raspbian.raspberrypi.org/raspbian stretch InRelease
Hit:2 http://archive.raspberrypi.org/debian stretch InRelease
Get:3 https://deb.nodesource.com/node_10.x stretch InRelease [4,612 B]
Get:4 https://deb.nodesource.com/node_10.x stretch/main armhf Packages [765 B]
Fetched 5,377 B in 2s (2,148 B/s)
Reading package lists…
## Run `sudo apt-get install -y nodejs` to install Node.js 10.x and npm
## You may also need development tools to build native addons:
sudo apt-get install gcc g++ make
## To install the Yarn package manager, run:
curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add –
echo “deb https://dl.yarnpkg.com/debian/ stable main” | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt-get update && sudo apt-get install yarn
Reading package lists…
Building dependency tree…
Reading state information…
The following NEW packages will be installed:
nodejs
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 13.5 MB of archives.
After this operation, 68.2 MB of additional disk space will be used.
Get:1 https://deb.nodesource.com/node_10.x stretch/main armhf nodejs armhf 10.15.0-1nodesource1 [13.5 MB]
Fetched 13.5 MB in 9s (1,418 kB/s)
Selecting previously unselected package nodejs.
(Reading database …
(Reading database … 5%
(Reading database … 10%
(Reading database … 15%
(Reading database … 20%
(Reading database … 25%
(Reading database … 30%
(Reading database … 35%
(Reading database … 40%
(Reading database … 45%
(Reading database … 50%
(Reading database … 55%
(Reading database … 60%
(Reading database … 65%
(Reading database … 70%
(Reading database … 75%
(Reading database … 80%
(Reading database … 85%
(Reading database … 90%
(Reading database … 95%
(Reading database … 100%
(Reading database … 55602 files and directories currently installed.)
Preparing to unpack …/nodejs_10.15.0-1nodesource1_armhf.deb …
Unpacking nodejs (10.15.0-1nodesource1) …
Setting up nodejs (10.15.0-1nodesource1) …
Processing triggers for man-db (2.7.6.1-2) …
/usr/bin/npm -> /usr/lib/node_modules/npm/bin/npm-cli.js
/usr/bin/npx -> /usr/lib/node_modules/npm/bin/npx-cli.js
+ npm@6.5.0
added 2 packages from 1 contributor and updated 14 packages in 58.92s
npm WARN deprecated mailparser@0.6.2: Mailparser versions older than v2.3.0 are deprecated
npm WARN deprecated nodemailer@1.11.0: All versions below 4.0.1 of Nodemailer are deprecated. See https://nodemailer.com/status/
npm WARN deprecated mimelib@0.3.1: This project is unmaintained
npm WARN deprecated mailcomposer@2.1.0: This project is unmaintained
npm WARN deprecated buildmail@2.0.0: This project is unmaintained
/usr/bin/node-red -> /usr/lib/node_modules/node-red/red.js
/usr/bin/node-red-pi -> /usr/lib/node_modules/node-red/bin/node-red-pi
> bcrypt@2.0.1 install /usr/lib/node_modules/node-red/node_modules/bcrypt
> node-pre-gyp install –fallback-to-build
node-pre-gyp ERR! Tried to download(404): https://github.com/kelektiv/node.bcrypt.js/releases/download/v2.0.1/bcrypt_lib-v2.0.1-node-v64-linux-arm-glibc.tar.gz
node-pre-gyp ERR! Pre-built binaries not found for bcrypt@2.0.1 and node@10.15.0 (node-v64 ABI, glibc) (falling back to source compile with node-gyp)
make: Entering directory ‘/usr/lib/node_modules/node-red/node_modules/bcrypt/build’
CXX(target) Release/obj.target/bcrypt_lib/src/blowfish.o
CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt.o
CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt_node.o
../src/bcrypt_node.cc: In function ‘char {anonymous}::ToCharVersion(v8::Local)’:
../src/bcrypt_node.cc:66:30: warning: ‘v8::String::Utf8Value::Utf8Value(v8::Local)’ is deprecated: Use Isolate version [-Wdeprecated-declarations]
String::Utf8Value value(str);
^
In file included from /root/.node-gyp/10.15.0/include/node/v8.h:26:0,
from /root/.node-gyp/10.15.0/include/node/node.h:63,
from ../../nan/nan.h:51,
from ../src/bcrypt_node.cc:1:
/root/.node-gyp/10.15.0/include/node/v8.h:2892:28: note: declared here
explicit Utf8Value(Local obj));
^
/root/.node-gyp/10.15.0/include/node/v8config.h:324:3: note: in definition of macro ‘V8_DEPRECATED’
declarator __attribute__((deprecated(message)))
^~~~~~~~~~
SOLINK_MODULE(target) Release/obj.target/bcrypt_lib.node
COPY Release/bcrypt_lib.node
COPY /usr/lib/node_modules/node-red/node_modules/bcrypt/lib/binding/bcrypt_lib.node
TOUCH Release/obj.target/action_after_build.stamp
make: Leaving directory ‘/usr/lib/node_modules/node-red/node_modules/bcrypt/build’
+ node-red@0.19.5
added 396 packages from 341 contributors in 131.503s
0.19.5
Installing extra nodes: node-red-node-random node-red-contrib-ibm-watson-iot node-red-node-ping node-red-contrib-play-audio node-red-node-smooth node-red-node-serialport :
> serialport@6.2.2 install /home/pi/.node-red/node_modules/serialport
> prebuild-install || node-gyp rebuild
prebuild-install WARN install No prebuilt binaries found (target=10.15.0 runtime=node arch=arm platform=linux)
make: Entering directory ‘/home/pi/.node-red/node_modules/serialport/build’
CXX(target) Release/obj.target/serialport/src/serialport.o
../src/serialport.cpp: In function ‘Nan::NAN_METHOD_RETURN_TYPE Open(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/serialport.cpp:41:49: warning: ‘v8::String::Utf8Value::Utf8Value(v8::Local)’ is deprecated: Use Isolate version [-Wdeprecated-declarations]
v8::String::Utf8Value path(info[0]->ToString());
^
In file included from /home/pi/.node-gyp/10.15.0/include/node/v8.h:26:0,
from /home/pi/.node-gyp/10.15.0/include/node/node.h:63,
from ../../nan/nan.h:53,
from ../src/./serialport.h:6,
from ../src/serialport.cpp:1:
/home/pi/.node-gyp/10.15.0/include/node/v8.h:2892:28: note: declared here
explicit Utf8Value(Local obj));
^
/home/pi/.node-gyp/10.15.0/include/node/v8config.h:324:3: note: in definition of macro ‘V8_DEPRECATED’
declarator __attribute__((deprecated(message)))
^~~~~~~~~~
../src/serialport.cpp:48:53: warning: ‘v8::Local v8::Value::ToObject() const’ is deprecated: Use maybe version [-Wdeprecated-declarations]
v8::Local options = info[1]->ToObject();
^
In file included from /home/pi/.node-gyp/10.15.0/include/node/node.h:63:0,
from ../../nan/nan.h:53,
from ../src/./serialport.h:6,
from ../src/serialport.cpp:1:
/home/pi/.node-gyp/10.15.0/include/node/v8.h:10046:15: note: declared here
Local Value::ToObject() const {
^~~~~
../src/serialport.cpp: In function ‘void EIO_AfterOpen(uv_work_t*)’:
../src/serialport.cpp:95:30: warning: ‘v8::Local Nan::Callback::Call(int, v8::Local*) const’ is deprecated [-Wdeprecated-declarations]
data->callback.Call(2, argv);
^
In file included from ../src/./serialport.h:6:0,
from ../src/serialport.cpp:1:
../../nan/nan.h:1674:3: note: declared here
Call(int argc, v8::Local argv[]) const {
^~~~
../src/serialport.cpp: In function ‘Nan::NAN_METHOD_RETURN_TYPE Update(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/serialport.cpp:113:53: warning: ‘v8::Local v8::Value::ToObject() const’ is deprecated: Use maybe version [-Wdeprecated-declarations]
v8::Local options = info[1]->ToObject();
^
In file included from /home/pi/.node-gyp/10.15.0/include/node/node.h:63:0,
from ../../nan/nan.h:53,
from ../src/./serialport.h:6,
from ../src/serialport.cpp:1:
/home/pi/.node-gyp/10.15.0/include/node/v8.h:10046:15: note: declared here
Local Value::ToObject() const {
^~~~~
../src/serialport.cpp: In function ‘void EIO_AfterUpdate(uv_work_t*)’:
../src/serialport.cpp:150:30: warning: ‘v8::Local Nan::Callback::Call(int, v8::Local*) const’ is deprecated [-Wdeprecated-declarations]
data->callback.Call(1, argv);
^
In file included from ../src/./serialport.h:6:0,
from ../src/serialport.cpp:1:
../../nan/nan.h:1674:3: note: declared here
Call(int argc, v8::Local argv[]) const {
^~~~
../src/serialport.cpp: In function ‘void EIO_AfterClose(uv_work_t*)’:
../src/serialport.cpp:188:30: warning: ‘v8::Local Nan::Callback::Call(int, v8::Local*) const’ is deprecated [-Wdeprecated-declarations]
data->callback.Call(1, argv);
^
In file included from ../src/./serialport.h:6:0,
from ../src/serialport.cpp:1:
../../nan/nan.h:1674:3: note: declared here
Call(int argc, v8::Local argv[]) const {
^~~~
../src/serialport.cpp: In function ‘void EIO_AfterFlush(uv_work_t*)’:
../src/serialport.cpp:231:30: warning: ‘v8::Local Nan::Callback::Call(int, v8::Local*) const’ is deprecated [-Wdeprecated-declarations]
data->callback.Call(1, argv);
^
In file included from ../src/./serialport.h:6:0,
from ../src/serialport.cpp:1:
../../nan/nan.h:1674:3: note: declared here
Call(int argc, v8::Local argv[]) const {
^~~~
../src/serialport.cpp: In function ‘Nan::NAN_METHOD_RETURN_TYPE Set(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/serialport.cpp:250:53: warning: ‘v8::Local v8::Value::ToObject() const’ is deprecated: Use maybe version [-Wdeprecated-declarations]
v8::Local options = info[1]->ToObject();
^
In file included from /home/pi/.node-gyp/10.15.0/include/node/node.h:63:0,
from ../../nan/nan.h:53,
from ../src/./serialport.h:6,
from ../src/serialport.cpp:1:
/home/pi/.node-gyp/10.15.0/include/node/v8.h:10046:15: note: declared here
Local Value::ToObject() const {
^~~~~
../src/serialport.cpp: In function ‘void EIO_AfterSet(uv_work_t*)’:
../src/serialport.cpp:285:30: warning: ‘v8::Local Nan::Callback::Call(int, v8::Local*) const’ is deprecated [-Wdeprecated-declarations]
data->callback.Call(1, argv);
^
In file included from ../src/./serialport.h:6:0,
from ../src/serialport.cpp:1:
../../nan/nan.h:1674:3: note: declared here
Call(int argc, v8::Local argv[]) const {
^~~~
../src/serialport.cpp: In function ‘void EIO_AfterGet(uv_work_t*)’:
../src/serialport.cpp:336:30: warning: ‘v8::Local Nan::Callback::Call(int, v8::Local*) const’ is deprecated [-Wdeprecated-declarations]
data->callback.Call(2, argv);
^
In file included from ../src/./serialport.h:6:0,
from ../src/serialport.cpp:1:
../../nan/nan.h:1674:3: note: declared here
Call(int argc, v8::Local argv[]) const {
^~~~
../src/serialport.cpp: In function ‘void EIO_AfterGetBaudRate(uv_work_t*)’:
../src/serialport.cpp:383:30: warning: ‘v8::Local Nan::Callback::Call(int, v8::Local*) const’ is deprecated [-Wdeprecated-declarations]
data->callback.Call(2, argv);
^
In file included from ../src/./serialport.h:6:0,
from ../src/serialport.cpp:1:
../../nan/nan.h:1674:3: note: declared here
Call(int argc, v8::Local argv[]) const {
^~~~
../src/serialport.cpp: In function ‘void EIO_AfterDrain(uv_work_t*)’:
../src/serialport.cpp:424:30: warning: ‘v8::Local Nan::Callback::Call(int, v8::Local*) const’ is deprecated [-Wdeprecated-declarations]
data->callback.Call(1, argv);
^
In file included from ../src/./serialport.h:6:0,
from ../src/serialport.cpp:1:
../../nan/nan.h:1674:3: note: declared here
Call(int argc, v8::Local argv[]) const {
^~~~
CXX(target) Release/obj.target/serialport/src/serialport_unix.o
CXX(target) Release/obj.target/serialport/src/poller.o
../src/poller.cpp: In static member function ‘static void Poller::onData(uv_poll_t*, int, int)’:
../src/poller.cpp:69:29: warning: ‘v8::Local Nan::Callback::Call(int, v8::Local*) const’ is deprecated [-Wdeprecated-declarations]
obj->callback.Call(2, argv);
^
In file included from ../src/poller.cpp:1:0:
../../nan/nan.h:1674:3: note: declared here
Call(int argc, v8::Local argv[]) const {
^~~~
CXX(target) Release/obj.target/serialport/src/serialport_linux.o
SOLINK_MODULE(target) Release/obj.target/serialport.node
COPY Release/serialport.node
make: Leaving directory ‘/home/pi/.node-red/node_modules/serialport/build’
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN node-red-project@0.0.1 No repository field.
npm WARN node-red-project@0.0.1 No license field.
+ node-red-contrib-ibm-watson-iot@0.2.8
+ node-red-node-ping@0.0.16
+ node-red-contrib-play-audio@2.3.2
+ node-red-node-smooth@0.1.0
+ node-red-node-random@0.1.2
+ node-red-node-serialport@0.6.8
added 273 packages from 125 contributors and audited 857 packages in 104.422s
found 0 vulnerabilities
The user `pi’ is already a member of `gpio’.
Finished : Mon 7 Jan 18:54:36 GMT 2019
Hi!
The new script worked great for me!
Thank you very much!
Let’s make home automation now!
Best regards!
Hi, tried the new script today to help with my node red problem. First time, looked to go ok although seemed to go around in circles on sqlite but when I logged on all the items on the log in page worked but couldn’t access node red. Apologies, newbie mistake but forgot to look at the error log file :(.
Second time the pi hung for 40+ minutes halfway down the new list of node red, first time through cleared this with all ticks.
Used the same script but selected old rather than new node red and that has worked ok.
I’ll retry the new method for node red in the next couple of days with a different sd card and report back.
well, if new version does not work, go complain with node-red guys, because it’s STRAIGHT their script, just downloaded and run by Peter’s one 🙂
that’s the suggested way to install it, i just made that a menu option…
after that, it installs the standard nodes using npm install, as ANYBODY would do manually… then installs that DAMN sqlite using the way Dave suggested to Peter to do, with –unsafe-perms…
script does nothing more that avoid you to run same stuff line by line manually, so don’t know what to do anymore…
save and share logs, anyway, or no help is possible…
Peter put new script online, it contains new setup (preset to ON, as in image) and old one too (the 2 other checks in next image)… OF COURSE *DO NOT* select both setups, use only the preselected one OR deselect it and enable the other 2… new setup uses the official raspberry pi suggested way to install nodered, and you’ll get node v10 then: https://nodered.org/docs/hardware/raspberrypi
AND, *AS ALWAYS*, this is ONLY for fresh systems, *DO NOT* run on preinstalled setups…
WoW! You are so efficient! I didn’t expect a fastest solution. Thank you very much!
I will trie it Sunday!
I’m just wondering how do you store your sensors datas without sqlite?
Have a nice weekend!
Wow! You are so efficient! That’s impressive!
Thank you so much! I will install it this evening.
Hi! Congratulations and thank you for all this work!
I’d like to use this installation script on my raspberry pi 3B+, but I’ve seen the warning about Node Red.
Does someone used it successfully recently?
Is it useful to wait for a fix or do I have to find an other solution?
Best regards.
Jean.
i’ll try running it today on a spare sdcard, let’s see if there’s an easy fix…
I’m decorating today so can’t help right now. However, putting the details of all nodes into a text file is a first step as you need to ensure that all nodes are in the package.json file, if not, they may be silently removed. Node 6 onwards incidentally does not need the –save option. Earlier versions did. Also I’m told that the SQLITE3 node needs npm -i –unsafe-perm node-red-node-sqlite3 and can take 20-30 minutes to compile. DaveCV is apparently looking at better-sqlite3 as a possible “upgrade.
I’ve not had time to progress this – the script messed up Node-Red for me but I got around it by upgrading an earlier clone of my setup. Indeed I’ve made umpteen clones, all fully up to date and using npm version 6.4.1 and node version 8.14.0
ok, got it working sqlite, too, using what Dave suggested you… just sent you updated script, all seems working fine again, we’re on latest node v10+nodered 0.19.5+npm 6.5.0
we should remove the redbot telegram node, as it’s now deprecated, and use the one in Csongor Varga video, i’ve set it up and it’s very good:
and we should add the credential encryption key to settings.js to allow recovery of flows services passwords, as suggested by nodered in logs…
sent new script with the –save option to add new nodes to packages.json
updated script sent to Peter, tested on a Raspberry Pi 3B+ using Raspbian Stretch Lite as base, no issues so far, EXCEPT sqlite…
i left previous install menu options (“nodejs” and “nodered”) in script but put them OFF, while now it’s ON the new “nodenew” which installs both NodeJS and Nodered using official recommended script ( https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/update-nodejs-and-nodered )… DO NOT choose both setups: you MUST use OR the predefined nodenew option, OR the other 2… if you want hardware support, of course add the other menu option which enables addon nodes (i2c and some tweaks)…
the new addon script itself will already install serialport and other RPI gpio related, if you answer yes when asked… YES, the script WILL HALT once reached the new official node script installer, which asks 2 questions (i didn’t want to alter it, it’s stock from nodered guys):
Are you really sure you want to do this ? [y/N] ? y
Would you like to install the Pi-specific nodes ? [y/N] ? y
i’m starting to HATE sqlite: the fact the sqlite guys are not able to release a proper precompiled npm binary for it is unbelievable, and takes MUCH part of the script install time… nodes install failed because of it, so i had to make its install a line on its own… and even so, i get these errors:
npm WARN node-red-project@0.0.1 No repository field.
npm WARN node-red-project@0.0.1 No license field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: utf-8-validate@1.2.2 (node_modules/utf-8-validate):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: utf-8-validate@1.2.2 install: `node-gyp rebuild`
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! sqlite3@4.0.4 install: `node-pre-gyp install –fallback-to-build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the sqlite3@4.0.4 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/pi/.npm/_logs/2019-01-03T10_52_05_528Z-debug.log
the nodered install script has its own log file (/var/log/nodered-install.log), which is copied in PI user home directory once install finishes…
oh, for who was expecting to have Node v8, no, it’s now v10 in latest official install script… look at lines 180 and 200 of:
https://github.com/node-red/raspbian-deb-package/blob/master/resources/update-nodejs-and-nodered
line 180 points to this url/folder, which contains node v10: https://nodejs.org/download/release/latest-dubnium
while line 200 explicitely downloads the v10 nodesetup script… who knows what’s in nodered guys: in last months we used script install v6 and we ended up with having v8 instead (which caused most of the sqlite/i2c/serial and update problems…)… now that the suggested version is v8, they install v10…
well, it’s up to you now, i moved my setups to Home Assistant and use its docker containerized addon for NodeRed… no more this mess… using the script you have all working now, EXCEPT sqlite, and it’s NOT our fault, now we’re using the suggested setup path (and even before we just used STOCK nodesetup.sh)… i have nodered running using this script, for me it’s ok, as said EXCEPT sqlite…
pi@raspberrypi:~ $ sudo systemctl status nodered.service
● nodered.service – Node-RED graphical event wiring tool
Loaded: loaded (/lib/systemd/system/nodered.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2019-01-03 10:59:33 GMT; 1min 21s ago
Docs: http://nodered.org/docs/hardware/raspberrypi.html
Main PID: 328 (node-red)
CGroup: /system.slice/nodered.service
└─328 node-red
Jan 03 11:00:02 raspberrypi Node-RED[328]: If the system-generated key is lost for any reason, your credentials
Jan 03 11:00:02 raspberrypi Node-RED[328]: file will not be recoverable, you will have to delete it and re-enter
Jan 03 11:00:02 raspberrypi Node-RED[328]: your credentials.
Jan 03 11:00:02 raspberrypi Node-RED[328]: You should set your own key using the ‘credentialSecret’ option in
Jan 03 11:00:02 raspberrypi Node-RED[328]: your settings file. Node-RED will then re-encrypt your credentials
Jan 03 11:00:02 raspberrypi Node-RED[328]: file using your chosen key the next time you deploy a change.
Jan 03 11:00:02 raspberrypi Node-RED[328]: ———————————————————————
Jan 03 11:00:02 raspberrypi Node-RED[328]: 3 Jan 11:00:02 – [info] Starting flows
Jan 03 11:00:02 raspberrypi Node-RED[328]: 3 Jan 11:00:02 – [info] Started flows
Jan 03 11:00:02 raspberrypi Node-RED[328]: 3 Jan 11:00:02 – [info] Server now running at http://127.0.0.1:1880/
At the moment the only way to update modules is start from scratch and run the script, copying my flows and database do some renaming in setings.js. This is not such a big thing as it runs on my spare RPI3B+ on the side.
There is an issue with updating which I cannot identify as you guys are not experiencing it. During running the Script it has problems with some serial and sqlite3.o nodes during node install where it is working for half an hour and finally comes through after a big delay and producing lots of output from the script. The only thing I have to try is a run with the same options selected with which you have made an updatable system. Now I leave out the hardware and GPIO and log2ram as well as deleting libreoffice and wolfram… Which options did you choose?
So I did a complete run from nothing to using this node-red update script ( https://nodered.org/docs/hardware/raspberrypi )
The outcome is not good. The update script ruined my node-red and will not start anymore . It did after applying the SCRIPT and copying my flow.
Seems to be I need to give sudo node-red-start. Then it starts node-red
This is the text I got in the Terminal:
Starting as a systemd service.
Started Node-RED graphical event wiring tool.
Error loading settings file: /home/pi/.node-red/settings.js
Error: Cannot find module ‘moment’
That’s it for now. It is not reliable on my Pi3B+ Most probably more after Christmas. And I did dare to use Antonio’s update script, because this was even more destructive on this Pi a few days ago! First exchange with bu and make a new copy bu.
we’ll take a look at script in these vacation days, but my update script will not do anything you shouldn’t do on your own, line by line, when updating a system… i use it for years now, on many system…
but today i’m playing with home assistant, i like HASS.IO as it has all needed addons with just a few click away… i installed nodered, mqtt, sqlite web, tasmoadmin, and have all working in https with automatic letsencrypt renewal, nginx reverse proxy and dyndns without writing A SINGLE line of code in a terminal… enabled plugins, configured the few parameters via web in json format, restarted services, done…
of course it’s not for who uses the script for a pure nodered setup, as home assistant was never in Peter’s plans, so using hass.io to have all the other stuff working is a bit like shooting at flies with a cannon… but how cool is the work that HA guys are doing… have both worlds talking together is the way to go for me now, as HA has a HUGE support, while creating automations in yaml is a bit “meh”… but as the 2 can interact very easily, who cares? 🙂
The only thing left for me is “not updating anything on the RPI” and keep bu’s by hand(thx to rpi-clone)
Right now I too am working with (fully up to date) clones. I’ve put a note in the blog.
I set up my system with the script. Personally I’m using node 8. I lost nodes on an update… Thanks to Node Red.. and reinstalled simply using the node red nojejs upgrade script on the NR site and then used rpi-clone on that. I’ve used my clone as a starting point and have done so ever since. Everything works.. and I’ve since added pivpn and pihole. Before every significant change I use rpi-clone as a safety measure.
I always keep 2 copies of a working system!
I do my experiments on a spare RPI!
It was a long time with no problems, but now I get more and more problems. We need to find the source of the troubles. However, my Homeserver is still functioning, although I cannot update it anymore. But as long as it works why updating! A new function on Big-timer?
I have a major problem updating modules. there is 1 node_modules bcryptjs folder with root owner and root user permissions. All others(there are many) seem to be pi,pi.
I am using a fresh system using the script and tried to update by using the Palette Manager. It failed and pointed to the error log in .npm/_logs
Can anyone give a medicine against this error messagein the .npm log?:
silly diff-trees filtering actions: includeDev false includeProd true includeOpt true
28 warn checkPermissions Missing write access to /home/pi/.node-red/node_modules/bcryptjs
29 timing stage:rollbackFailedOptional Completed in 3ms
30 timing stage:runTopLevelLifecycles Completed in 108334ms
31 silly saveTree node-red-project@0.0.1
31 silly saveTree └─┬ node-red-contrib-bigtimer@2.0.7
31 silly saveTree └── suncalc@1.8.0
32 verbose stack Error: EACCES: permission denied, access ‘/home/pi/.node-red/node_modules/bcryptjs’
33 verbose cwd /home/pi/.node-red
34 verbose Linux 4.14.79-v7+
35 verbose argv “/usr/bin/node” “/usr/local/bin/npm” “install” “–save” “–save-prefix=~” “–production” “node-red-contrib-bigtimer@2.0.7″
36 verbose node v8.11.1
37 verbose npm v6.4.1
38 error path /home/pi/.node-red/node_modules/bcryptjs
39 error code EACCES
40 error errno -13
41 error syscall access
42 error Error: EACCES: permission denied, access ‘/home/pi/.node-red/node_modules/bcryptjs’
42 error { Error: EACCES: permission denied, access ‘/home/pi/.node-red/node_modules/bcryptjs’
42 error stack: ‘Error: EACCES: permission denied, access \’/home/pi/.node-red/node_modules/bcryptjs\”,
42 error errno: -13,
42 error code: ‘EACCES’,
42 error syscall: ‘access’,
42 error path: ‘/home/pi/.node-red/node_modules/bcryptjs’ }
43 error The operation was rejected by your operating system.
43 error It is likely you do not have the permissions to access this file as the current user
43 error
43 error If you believe this might be a permissions issue, please double-check the
43 error permissions of the file and its containing directories, or try running
43 error the command again as root/Administrator (though this is not recommended).
44 verbose exit [ -13, true ]
A lot of other things are not working too! npm outdated gives no modules to be updated. I installed npm-check_updates says that nothing needs to be updated. What is going on?
as already said many times in previous comments:
sudo chown -R pi.pi ~/.node-red/node_modules
Why is this not in the SCRIPT? (short explaining pls!)
I am just a simple linux user and can you imagine those commands are absolutely witchcraft for me.
I was already on that trail, but I did not had the courage to give the command.
As you can see in the log bcryptjs is the first one which is wrong and I searched the blog for that phrase and draw a blank.
I understand this should do the trick, but now I have a very strange phenomenon. When I reboot node-red is not starting. When I try to do it with node-red-start it gives an error loading settings.js. I have compared this settings.js file with a working version, it is there and I raised access to anyone but still not loading?
because we never understood why this happens sometimes, and it’s something that is not useful while script is running: as script is meant to be run on freshly installed devices, the first time you run npm that folder is created and has already the right permissions… while updating some nodes, don’t know which one, the problem could appear… the bcryptjs package is one of the problematic one, especially on updating, if running on nodejs v8, that’s why we suggested to stay on node v6…
i added that line to my update script a while ago, you can use it:
https://gist.github.com/fragolinux/412aa50683eb32bb879a120c73110938
Somehow my nodejs updated to v8.11. This must be a setting, because I did not take action as I read not to update it.
Probably it is related to all the problems I have, but how to stay on v6???
unfortunately don’t know… if you look at script (lines 789 and 811 of https://bitbucket.org/snippets/scargill/ekon6o/the-script ), we download V6… but you end up with v8 after a while… why, is in nodejs guys minds…
During my experiments I used the commands you use in the script and that did not work at all! That is all before you advised me the owner change. I was than not aware I am running on nodejs v8.11.
as Peter said in his latest comment, you’re not the only one who lost his nodes, and we still didn’t realize what cause that… it’s NOT script related, for some reason npm update screws things up…
Tomorrow I will try your update script before I do anything else! I will report the result. What is deborphan -sz? This will work on a fresh system?
my script is just for automating periodical updates… instead of doing a bunch of updates commands, i condensed them there… the last lines are for homeassistant, remove them if not needed… deborphan searches for DEB packages that are not needed (have no requirements as dependencies in other packages) in your system anymore, and can be removed… think if you do “sudo apt-get install apache2”, it will install that AND tons of others… if you remove apache2, and do apt-get autoremove, you get rid of majority of them, but not ALL… deborphan is the King Herod of them 😀
That’s new to me, I must check. Took me ages to remove all Apache2 recently.
more info: https://debian-administration.org/article/134/Removing_unnecessary_packages_with_deborphan
Well, THAT looks like Christmas fun, I’m going to give it a shot…
previous link is old, but deborphan still works, in case there’s APT that does the same…
https://askubuntu.com/questions/286947/obsolete-packages-vs-orphaned-packages
https://linoxide.com/linux-how-to/remove-orphaned-packages-ubuntu/
I looked at the contents of node_modules. Only big-timer and suncalc folders as I tried the update for that node after chown.
All other folders vanished!
I hope you are using rpi-clone.
I made a complete new install and accidentaly missed the webmin install star. Now I miss it.
Can I use the piece of script in Script.sh and use it to install it afterwards?
Can’t think of any reason why not. Beware, Webmin takes ages to install.
also, i sent this along to the node guys and they think its a problem in the script also:
Just a quick question, the node-red-node-email node gets installed during the node-red installation as a default. There is a newer version available as the palette manager says there is an update, but it is not updatable from the palette manager as it is installed globally. It has to be uninstalled and reinstalled to work. Little bit of a pain.
as i already said here, now the email node REQUIRES nodejs8, which has caused MANY issues to A LOT of us, npm/bcrypt/hw related… so we’re sticking to v6, for now… i hope v8 issues are addressed soon (even docker nodered container guys suggest to use v6 for now…)
Instead of changing bcryptjs from root to pi, i left its owner as root but changed permisions then tried to install from command line:
pi@NodeRed2:~/.node-red/node_modules $ sudo chmod 777 bcryptjs
pi@NodeRed2:~/.node-red/node_modules $ cd $HOME/.node-red
pi@NodeRed2:~/.node-red $ npm install node-red-contrib-combine
+ node-red-contrib-combine@1.6.2
added 1 package from 1 contributor, removed 1326 packages and audited 1 package in 57.309s
found 0 vulnerabilities
REMOVED 1326 Packages!! BROKEN Again. I just don’t get why it removes all the other packages.
Would it be possable to install new nodes into a different folder so they dont delete all the other stuff, like in the httpstatic path?
Much as I love to try and help, these are questions for the Node Red guys.
Ok Pete,
Did as you suggested and ended up with the following solution to the above problems:
When node-red and npm are installed by the script, the errors occur. I’m not exactly sure why but it does. If I wipe my pi and reinstall stretch, then system updates, then npm, then node-red, then run the script omitting the npm and node-red install options, everything works perfectly. Solved Chris’s problem also. Great script and thank you for your help.
John
Good. Time to investigate further then
When I get back off holiday and into the UK and in front of a PI, perhaps Antonio and I will do a little experimenting. Thank you for that.
My Walk through of node red install with the script.(Headless).
1. Wipe and format sd card. Ease partition master seems to work better than SD Formatter.
2. Write raspbian image to card. Win32Disk Imager still works for me.
3. Before putting card in raspberry open it up in windows explorer, and in the root directory make a new blank text document and save it as ‘ssh’ (without quotes). This will enable SSH on boot. No need for Keyboard & Monitor !!!
4. Put card in Pi, connect wired network and power. Wait for it to boot.
5. Check router to see what IP address is.
6. SSH into it (I use Putty) Logon = pi, password = raspberry.
7. Check file system (you used to have to expand it to use all the card, but recently seems to be already expanded. df –h
8. sudo raspi-config and enable ssh, I2c and serial and disable serial console – others may have different requirements.
9. sudo nano /etc/ssh/sshd_config and Change #PermitRootLogin prohibit-password to PermitRootLogin yes (uncomment!) save and exit.
10. Restart sshd service using: /etc/init.d/ssh restart or sudo reboot
11. Set a root password if there isn’t one already: sudo passwd root
12. Now you can Log in as root.
13. Get the script wget –no-check-certificatehttps://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh
14. And then bash script.sh
15. Script will start, and after a few seconds will ask you to log out as root and log in as pi, then bash script.sh Again.
16. It will ask you to change user password, admin password then Pi password and root password.
17. Then you get the options menu, select what you want then GO!………wait . (pi3 Model B with all options about 35 -40 mins).
This method has always worked for me, but now i can’t load new nodes via pallet, got somthing to do with permissions of bcryptjs being root.
Pretty much how I do it but once I got a working setup a while ago I started cloning from rpi-clone backups so I can’t comment on new bcrypt issues.
Now reloading again, going to leave bcryptjs alone and try loading new nodes from command line in putty. also going mad!!!!!!!!!!! I have had the i2c-bus thing as well.
I have wondered that. It is far too easy to end up losing everything. Thank heavens for for rpi-clone on the Pi.
going mad now – after all that i get
Error: Cannot find module ‘i2c-bus’
when starting node red
Thanks peter, yes. I have installed a new copy of stretch on the pi, updated the system and ran the script – once as root as mentioned in the script which takes me in to the user pi than run the script again as pi. Also tried just running it again on a fresh system from the pi user and get same results. All goes well until I try and install the node-red-contrib-simple-gate node (which always fails) with the above messages. I am now reinstalling everything fresh again and will try the node install and report what happens.
Actually, I have spent too much time on this and I cheated. I put the problem node installations in the script and ran it again and they installed ok.
Node-red-node-email is a problem as it gets installed globally and cannot be updated from the palette, so needs to un-installed and reinstalled again. Have done this successfully before but with the new install I now get the following when I try and reinstall email node:
npm i node-red-node-email
npm WARN deprecated mimelib@0.3.1: This project is unmaintained
npm WARN checkPermissions Missing write access to /home/pi/.node-red/node_modules/bcryptjs
npm ERR! path /home/pi/.node-red/node_modules/bcryptjs
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall access
npm ERR! Error: EACCES: permission denied, access ‘/home/pi/.node-red/node_modules/bcryptjs’
npm ERR! { Error: EACCES: permission denied, access ‘/home/pi/.node-red/node_modules/bcryptjs’
npm ERR! stack: ‘Error: EACCES: permission denied, access \’/home/pi/.node-red/node_modules/bcryptjs\”,
npm ERR! errno: -13,
npm ERR! code: ‘EACCES’,
npm ERR! syscall: ‘access’,
npm ERR! path: ‘/home/pi/.node-red/node_modules/bcryptjs’ }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator (though this is not recommended)
For some reasons after playing with all of this for the past month or so I always seem to get in to situations where there are permission problems installing or updating nodes. Tried running from sudo on occasion and that only created worse problems. Somewhat frustrated by it all although I do really like your script, and I dont know what the cause of these problems are.
Have you tried with rasp jessie ? i have that on my main controller with node red v0.17. 7 and it has been running for over a year,maybe two, with no problems. Lately i have been experimenting a lot and i am scared of breaking my main node red pi, as another pi is a lot cheaper than a divorce, i am trying to get running the script for test purposes.
All reloaded, node red running but bcryptjs is still root. tried mr sharks:- sudo chown -R pi.pi ~/.node-red/node_modules, installed nodes ok, reboot, Broken!
re load try again tonight.
Fresh install.. going nowhere near sudo or root. will post on what happens in morning.
Chris
I’m having similar problem. Pi3 model B. after running script all seem ok, until you try to install extra nodes. after looking at the log file it is a permission problem with file /home/pi/.node-red/node_modules/bcrypt. when you look at it in winSCP you can see that its owner is ‘root’, (all other stuff in there is ‘pi’. So using putty as user ‘pi’ as i cant get into root (access denied) i did a sudo chown on that file to pi. i then went back to node red and tried installing again. it worked!! i installed another brilliant!! feeling very pleased with myself i went off to bed. Came back to it this evening and all seemed ok, Until i rebooted. node red no longer starts! I thought no problem i will just change the permissions back to root, reboot and try something else. But All the files and folders have gone !! all that is in node_modules is the two nodes i added.
P.S. when i run the script it says must be run as root the only way i can get to user root is sudo su. could this be relavent?
Anyone Help ???
many thanks Chris.
I have installed the script a few times on different RPis and even with a new installation with just stretch installed and the script, i get errors when i try and install a node. Get file permission issues. Here;s an example trying to install node-red-contrib-simple-gate: (on a new installation on stretch and the script only).
“Installation of module node-red-contrib-simple-gate failed:”
npm WARN checkPermissions Missing write access to /home/pi/.node-red/node_modules/bcryptjs
npm ERR! path /home/pi/.node-red/node_modules/bcryptjs
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall access
npm ERR! Error: EACCES: permission denied, access ‘/home/pi/.node-red/node_modules/bcryptjs’
npm ERR! { Error: EACCES: permission denied, access ‘/home/pi/.node-red/node_modules/bcryptjs’
npm ERR! stack: ‘Error: EACCES: permission denied, access \’/home/pi/.node-red/node_modules/bcryptjs\”,
npm ERR! errno: -13,
npm ERR! code: ‘EACCES’,
npm ERR! syscall: ‘access’,
npm ERR! path: ‘/home/pi/.node-red/node_modules/bcryptjs’ }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the …
The script would not cause permissions issues. Did you install as user Pi. On Raspberry pi install script as user Pi.
Hi Nicolass, I had the same problem running ‘the script’ on a dual boot PC using ubuntu 16.04. I found some information on the net about the npm issue, unfortunately I didn’t make a note of the source. Basically I installed npm manually and then selectively ran just the part of the script to load the node-RED nodes. Well that’s as much as I recall. Once done all the other stuff ‘the script’ installs runs beautifully.
Dr Fraggle and Mr Shark is the real expert on the Linux stuff and he may be able to provide far assistance than I can.
The work here is awesome. Thank you so very much. It looks like the script will not install under an amd64/x86 distro of linux. I have tried ubuntu 16.04 (I have two machines running it now on ubuntu 16.04), mint linux, raspberry pi desktop, and a couple other debian variants. The issue I have is the npm does not get installed — claims missing dependencies. The script works fine on rpi with noobs/rasbian. Any advice?
Thank you again
Hi, 2 times i insalled the script bu node-red flow control page not showing right. It has kinda CSS or design error. Any advise?
The script would not affect the NR CSS.
Hi All. I’ve been happily running my ubuntu 16.04 server that I loaded the script on for >18 months. Mosquitto/MQTT has been performing flawlessly. For a separate reason I happened to check the version of mosquitto broker and noticed mine is on version 1.4.8 but when checking the mosquitto.org changelog am seeing that its quite a few releases behind. I scanned this thread for any details about version and noticed a few comments but am not clear if I can upgrade to the newest version without creating some issues. I noticed some comments about websockets and also noticed that in the script the Debian repo is where mosquitto is installed from. Unfortunately i’m not as proficient in understanding all the nuances but do you think its risky to upgrade?
Of course part of me is thinking, if it aint broke don’t fix it, but I would generally like to get to the latest version if it makes sense.
Many thanks.
I’ve not deliberately upgraded Mosquitto in a long time – it may be it gets upgraded in apt-get update/upgrades but I’m not sure. Just make sure you backup before experimenting.
Actually I have v 1.4.10 build date fri 27th 2017 apparently. It works perfectly.
just to inform you that some nodes started to REQUIRE node v8 to operate, so problems are on the horizon, as we never found that v8 is very reliable or easy installable on our SBCs…
here the node that i had to remove from my flows, this morning… the EMAIL one… which for sure is NOT one of the less useful, so it’s a real problem…
[node-red-node-email/email] Error : Requires nodejs version >= 8.
if using Peter’s script, you’ll find these info in a prompt writing “nrlog”, just after a node update and following node-red restart
Hi again.
for your information:
I installed a new VM in VBox using an Ubuntu Server 16.04.05 Xenial and ran script
I got quite similar npm error.
Current test environment is:
VBox: 5.2.12
Host OS: Win 8.1
Guest OS: Ubuntu server 16.04.05
uname-a: Linux prueba3 4.4.0-131-generic #157-Ubuntu SMP Thu Jul 12 15:51:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Script version: None. Contains “..20/08/2018 Fixed Java ..”
Source: wget –no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh
BR
i’m trying your same setup (only difference: in vmware instead of virtualbox, but this is ininfluent), and i’m having problems downloading packages from ubuntu repositories today (those errors 400 in following transcript)… so maybe is your same problem… retry in a few days, or try debian stretch instead… it’s a repository problem, not related to script…
78 aggiornati, 4 installati, 0 da rimuovere e 0 non aggiornati.
È necessario scaricare 16,0 MB/155 MB di archivi.
Dopo quest’operazione, verranno occupati 304 MB di spazio su disco.
Continuare? [S/n]
Scaricamento di:1 http://it.archive.ubuntu.com/ubuntu xenial-updates/main amd64 bsdutils amd64 1:2.27.1-6ubuntu3.6 [51,2 kB]
Err:2 http://it.archive.ubuntu.com/ubuntu xenial-updates/main amd64 mount amd64 2.27.1-6ubuntu3.6
400 Bad Request [IP: 90.147.160.69 80]
Scaricamento di:3 http://it.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libblkid1 amd64 2.27.1-6ubuntu3.6 [107 kB]
Scaricamento di:4 http://it.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpam-systemd amd64 229-4ubuntu21.5 [115 kB]
Scaricamento di:5 http://it.archive.ubuntu.com/ubuntu xenial-updates/main amd64 udev amd64 229-4ubuntu21.5 [993 kB]
Scaricamento di:6 http://it.archive.ubuntu.com/ubuntu xenial-updates/main amd64 initramfs-tools-bin amd64 0.122ubuntu8.13 [9.742 B]
Scaricamento di:7 http://it.archive.ubuntu.com/ubuntu xenial-updates/main amd64 systemd-sysv amd64 229-4ubuntu21.5 [11,7 kB]
Scaricamento di:8 http://it.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libmount1 amd64 2.27.1-6ubuntu3.6 [114 kB]
Scaricamento di:9 http://it.archive.ubuntu.com/ubuntu xenial-updates/main amd64 uuid-runtime amd64 2.27.1-6ubuntu3.6 [25,9 kB]
Scaricamento di:10 http://it.archive.ubuntu.com/ubuntu xenial-updates/main amd64 open-iscsi amd64 2.0.873+git0.3b4b4500-14ubuntu3.6 [334 kB]
Scaricamento di:11 http://it.archive.ubuntu.com/ubuntu xenial-updates/main amd64 ubuntu-core-launcher amd64 2.34.2 [1.560 B]
Scaricamento di:12 http://it.archive.ubuntu.com/ubuntu xenial-updates/main amd64 snapd amd64 2.34.2 [14,2 MB]
Recuperati 15,9 MB in 16s (953 kB/s)
E: Impossibile recuperare http://it.archive.ubuntu.com/ubuntu/pool/main/u/util-linux/mount_2.27.1-6ubuntu3.6_amd64.deb 400 Bad Request [IP: 90.147.160.69 80]
E: Impossibile scaricare alcuni pacchetti. Potrebbe essere utile eseguire “apt-get update” o provare l’opzione “–fix-missing”.
Tante grazie for checking that.
I m going to retry during weekend. Hope to get it working.
Also, I am going to try to setup just one CPU instead of multiple as I am normally using.
BR.
Hi Pete & friends,
I am having issues with the-script running it in a fresh DietPi copy in a new virtualBox VM (DietPi_VirtualBox-x86_64-Stretch).
I got some messages that npm is not available such as:
Installing Node-Red
sudo: npm: command not found
Quite strange because in same log I can read:
/usr/lib
└─┬ npm@6.4.1
I found that there is an issue with /usr/bin/npm symbolic link.
I could fixed it, but have to reinstall Nore-Red to get a “cleaner” installation.
But I am not quite sure if installation is really finished.
Have you got same issue with npm?
such in case, full log is in: https://pastebin.com/bEd3EAvq
BR,
Sergio.
use armbian, problem solved… i’m tired to fix other people weird choices… i’m not against you, to be clear, but unfortunately, dietpi people tend to do too much alterations from version to version, i’ve seen that from time to time they suggest to start fresh and not upgrade… that’s a big no-no for me… so, i use dietpi rarely recently, and go for straight armbian xenial or stretch… if you want to use dietpi, then better stick at their own package management…
Hi,
Many thanks for your answer, I did not know that DietPi was so untidy.
in my case as DietPi VM was mentioned as a platform that has been tested sometime ago, I was sharing the issue, debugging and work-around done.
DietPi VM gives me a light and quick start base environment to run this Script, which is great to keep complete installations in a totally controlled manner , allowing me to make some NR tunning/tests without touching production setup and even not being at home, but I am taking your suggestion for the Pi install 🙂
Best regards,
Sergio.
in case of a vm, you can even go for stock ubuntu server… remember, only 16.04
Hi folks.
I have been told to get my system as Peter’s so I can run his flows and see what they are like on MY machine.
To do this I believe I have to run the script mentioned way up near the top of this.
(for clarity)
wget –no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh
Ok.
There are things in there about if it is running on a PI. Though I have a lot of Pi’s, for now I want to get it working on a bigger machine.
(x86 NUC)
So I have the script.
Reading it…. yeah, ok. A bit above me.
I am running/installing the stuff on a VM running Ubuntu. Not Raspbian, and there is no PI user.
Reading in the script I see:
IMPORTANT:-
## 1. Run ONLY as ROOT (or PI user on Raspberry Pi). If ROOT, PI user will be checked/created for you as the main
## script must be run as pi.
## User PI must be in the SUDO group. Other groups are added by the script.
Then a few lines down I see:
6. Do not access this script as SUDO.
I’m lost.
And if I am not installing it on a PI, I am at a loss how to do it.
I am trying to get Peter’s “base line” of what he has installed, so I can see what the stuff looks like, but am just not “getting it” with the instructions as they are.
Thanks in advance.
when i use an ubuntu vm for this stuff, i always create a “pi” user just during setup, so it’s already there when i run the script… as script was originally meant for raspberry, default user “pi”, it works ONLY with that user, unfortunately…
otherwise, do as such if you already installed and have a different user:
sudo -s -H
# give YOUR ACTUAL USER password at step above… after that, you ARE root
# start from here running the usual setup
cd && wget –no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh
bash script.sh
# after it created the user pi, logout and login with that user, or do:
su – pi
bash script.sh
or just reinstall ubuntu creating a pi user during setup, as i said… DON’T use ubuntu 18.04, only 16.04 is supported…
Thanks.
16.04…………..
Drats.
I started with 17.01 when I got the NUC.
I know it isn’t too difficult to get the older version.
Ok, that is a whole new can of worms with which to deal.
I don’t get why 18.04 isn’t “supported”? You mean for the build on which it is built?
it isn’t supported by Peter’s script, as none of the sbc has something different from debian stretch or ubuntu xenial, so why adapting the script to something that very few will use?
You can use it on newer ubuntu, but you’re on your own with all the problems you’ll find, don’t ask for help… i don’t plan to port it on 18.04 in the near future, or ever… because i see docker as the future of this…
and when my docker setup will be usable, there will be no differences as containers ignore the underlying OS, so you’ll be able to run docker on 18.04, too…
God that was quick, all working now
thanks a lot.
I have spent all day setting up and reading the comments etc
for latest script- This was the only problem I encountered by installation
regards from sunny Germany
for unknown reasons, some nodes change owner from time to time, or you used sudo when it was not supposed to use it, so the nodes were then owned by root instead of you… as it happens to me, too, i’ve added that line to my full system update script a lot of time ago, so i’ve not to think to it every time… it updates system and nodes, automatically:
https://gist.github.com/fragolinux/412aa50683eb32bb879a120c73110938
(remove last 4 lines if you don’t use homeassistant)
Hello Pete
Thanks for all the info and work you do.
Hope sunny Spain is that.
After doing a new sd card on raspian stretch and using script from Sept
I copied file flows.json from my previous setup to .node-red in new
I have the same problem as Loic74 September 19, 2018 at 7:32 pm
post to which I believe you gave him a solution
I cant seem to find your reply
Can you enlighten me please?
Not certain if its a problem credential password or permissions.
sudo chown -R pi.pi ~/.node-red/node_modules
Hi Peter, I have a little issue that I wonder if you might have an answer to?
I have been using ESPLOGIN and your software on a few devices for a few years now, and they how worked pretty much flawlessly. I recently started converting a few older switches (Sonoff S20) onto your software, where they used to run TASMOTA. The ROMS where erased first, and the switches work as expected now. However, I have noticed that two switches (from eight in total) seem to be running an Access Point and show a port 80 service on 192.168.4.1. I haven’t been able to access the server. When initially burnt with your software I was unable to access the web config on these two, so setup was done via serial connection.
Have you any idea why this may be? It seems odd that they have an AP but are also active on my own network too…
Thank you for your (and all other contributors!) efforts with the entire ESP-GO suite.
Thank you Peter,
And everyone involved in the script your work is outstanding.
It turns out – and Antonio is right here – that while I tried to use the latest NODE 8.x, Node-Red does not yet always seem to work correctly with this… indeed the IBM upgrade script uses Node 6.odd – so for now I’ve modified the script back to Node 6.x – if you’ve installed the lot on a PI and Node-Red isn’t having it, you can firstly use
sudo rm -r /home/pi/.node-red
then run the script again as PI user but this time with ONLY node + node-red ticked – and the update admin and user passwords options ticked.
Just done it myself on a new setup. The script by now refers to NPM 6.x, not 8.x
All worked but no extra nodes and no serial port so in /home/pi/.node-red I opened (in a text editor) package.json and replaced it’s (almost non-existent) content with:
{
“name”: “node-red-project”,
“description”: “A Node-RED Project”,
“version”: “0.0.1”,
“dependencies”: {
“i2c-bus”: “^4.0.1”,
“node-red-contrib-bigtimer”: “~2.0.2”,
“node-red-contrib-blynk-ws”: “~0.7.1”,
“node-red-contrib-chatbot”: “~0.15.0”,
“node-red-contrib-diode”: “^1.0.1”,
“node-red-contrib-esplogin”: “~1.3.7”,
“node-red-contrib-geofence”: “^0.2.4”,
“node-red-contrib-i2c”: “~0.5.2”,
“node-red-contrib-ibm-watson-iot”: “^0.2.8”,
“node-red-contrib-influxdb”: “~0.2.1”,
“node-red-contrib-owntracks”: “0.0.2”,
“node-red-contrib-play-audio”: “^2.3.2”,
“node-red-contrib-sqldbs”: “^1.1.8”,
“node-red-contrib-timeout”: “~1.1.0”,
“node-red-contrib-web-worldmap”: “~1.3.8”,
“node-red-dashboard”: “~2.9.8”,
“node-red-node-arduino”: “0.0.18”,
“node-red-node-email”: “~1.0.4”,
“node-red-node-ping”: “0.0.15”,
“node-red-node-random”: “^0.1.0”,
“node-red-node-serialport”: “^0.6.8”,
“node-red-node-smooth”: “^0.1.0”,
“node-red-node-sqlite”: “~0.3.5”,
“sqlitedb”: “^1.0.0”
}
}
Remember I’m doing this as user pi. If you don’t need serial and sqlite – leave them out as they take ages to update – no binaries out there apparently so a lot of “rebuilding” goes on.
I then made sure Node-Red was stopped (node-red-stop) – and that I was in the /home/pi/.node-re directory before (as pi user) typing:
npm rebuild
and
npm upgrade
This takes ages to run non-interactively. sqlite and the serial port are a pain here.
After some time – that did the trick (don’t comment about warnings, out of our hands). REBOOT.
All working. npn 6.41
I have it running successfully here on a PI3 and we’ll do a much neater job of this – but I move from Spain to the UK this coming weekend then have work to do on the house (including wiring in broadnand and computers – and a floor etc – so consider this a stop-gap.
Don’t ask what “npm audit” does, I didn’t run that.
a BIG advice: that command will remove .node-red and should be done ONLY on freshly installed boards, and only if you had problems with node v8 and want to go to v6 without rerunning full script from scratch… otherwise you’ll loose all your nodes and configs…
ah, for the missing hw support, you must check the relative menu option to have it, of course…
and for package.json, instead of all those nodes listed in script, we can have a package.json which installs them as you just did… tip: you can strip out versions and replace them with * to have a more generic version, like this i’m using on my docker setup: https://github.com/fragolinux/DockerIOT/blob/master/nodered/package.json
the npm audit will check all nodes if they have security issues and correct them, it’s a new addition in npm 6 i think:
https://docs.npmjs.com/getting-started/running-a-security-audit
https://blog.npmjs.org/post/173719309445/npm-audit-identify-and-fix-insecure
nodered guys have proper, official, nodered images, and they too stick to v6 (compare size of different v6 and latest tags…)… https://hub.docker.com/r/nodered/node-red-docker/tags/
i’m customizing these official images for my setup adding your nodes, and here, too, it took ages to install serial and sqlite… BUT… we could pack everything once done and release customized images on dockerhub, at the end…
this way reinstalling will just be downloading a new image and deploy a container, but this implies that every time some new node is released, we should re-release the customized image (the new layers of it, really)…
i think that the time needed to install could be just acceptable, as it’s the same if you run classic setup or dockerized one… which, in the end, is just the same process, just running on different places (host or container), always on some linux…
better stick to stock image and add a custom layer with our usual nodes, less work for me, and a little more time for who uses it, yup 😀
here’s why it takes ages to compile… 4 cores, only 1 used by nodered to compile…
Peter
I downloaded a fresh version of the ‘script’ today and it is still installing Node v8…
Tom
I am now travelling until 3rd of October with limited access to communications.
Sorry Peter, just realised that you are mid-move to the UK…
This can wait!
this is the line in latest script that downloads the official nodejs setup script, and as you can see, it points to v6 (v8 of course has an 8 in the url):
curl -sL https://deb.nodesource.com/setup_6.x > nodesetup.sh
we’re not reinventing the wheel, it’s the official way to install it, taken from nodejs site… but we noticed in last times that this, too, brings to a v8 setup once finished installing, but this time v8 works, on the contrary of the v8 script that seems to work very seldom…
Thanks for the info… not sure if I should risk v8 or not!
as said, i don’t know what are doing nodejs people… if you use node v8, often you get errors… if you use node v6, you end up getting v8 after setup is finished, but this time it works…
i think swapping comments on line 789-790 and 811-812 should solve, as this way it’s restored node V6 that we used for months… what i don’t understand is why v8, which both used flawlessly various times, without issues, last weekend, now fails…
Following on from my comment earlier this morning, this is the bit about setting up Raspberry Pi that I hate – plugging in a keyboard and monitor.
This is a good time to set the timezone and password and of course connection. I use hardwired ethernet, not WIFI.
In the RPI menu – go to configuration – I enable ssh, I2c and serial and disable serial console – others may have different requirements. This is a good time to point out that the Pi has built-in pullup resistors for I2c, other boards often do not.
First reboot coming up.
At THIS point the PI wants me to change password to something more secure.
Done…
Now I pop up a terminal on the Pi – and key in this mess…
sudo nano /etc/ssh/sshd_config
I need this info:
https://raspberrypi.stackexchange.com/questions/48056/login-as-root-not-possible
This will let me log in remotely as rooot after another operation. In the mentioned file, uncomment #PermitRootLogin and change the rest of the line to yes
Save with control-o (Enter) then control-x
Back at the terminal I change the ROOT password using sudo passwd root
I then make a note of the IP address (I could at this point have made this fixed), sudo reboot – that’s it, no more messing with Pi keyboard and screen.
I turn off the Pi, disconnect keyboard, screen and mouse. From now on I can do the lot from the comfort of my PC.
Using (in my case) Mobaxterm on the PC (I used to use WinSCP which is also great) I’m now talking to the Pi from the comfort of my desk as root user for a brief time.
On the Pi (as user root) we need to key in:
wget –no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh
and then bash script.sh
You will note that quite quickly, some preliminary software will load on the Pi. Within a minute or so you’ll be asked to log out as root and run the script as pi. I find it easy to simply switch to another session in Mobaxterm – logging in as user Pi.
Repeat:
bash script.sh
This time will take longer but most of it is just waiting (or more sensibly, having a coffee break). First however, in the menu that pops up on the screen, select any changes we want from defaults. using cursor and space bar. I enable hardware support, enable habridge and grafana, remove wolfram and office as I have no plans to use the Pi graphical interface from now on. When done use TAB, Ok
We then get to (optionally change the hostname and) update USER and ADMIN names and passwords at the end of which the script runs with no further interaction and could take a while.
Indeed it will – which is why I’m stopping this comment here.
I’m going to get to the bottom of this… I have a Rasperry Pi 3 Model B here and this morning am doing a fresh install including the latest full Raspian (26/7/2018) which I’m currently downloading from https://www.raspberrypi.org/downloads/raspbian/
I now use the free ETCHER program on Windows to grab Raspbian Stretch (full version zip download) in this case. This does NOT need to be run as Windows administrator and does a better job of checking SDs than Win23DiskImager. I’ll take this opportunity to recommend using only genuine Sandisk Utra or Samsung Evo/Evo+ SDs. I generally use the 16GB variety and that leaves plenty of room for expansion/wear. As I have a few 8GB SDs lying around, this morningI’ll be using one of thosea a Sandisk. Etcher incidentally does the copy then checks the SD, without user interaction. All this on my not-stunningly-fast Windows 10 PC takes several minutes to download then several minutes to install on the SD. This is not something I relish doing frequently. Windows takes great delight in suggesting formatting the SD just after you’ve waied ages to program it up – ignore that – if Etcher says “success” – you’re done.
The script for clarification is today best grabbed directly onto the PI as root user using:
wget –no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh
i.e. after grabbing in this case Raspbian operating system (a massive 1.6GB) and getting Raspian running, I then (using a screen and keyboard on the PI graphical desktop) enable remote root access and pass control to a PC (using Mobaxterm on the PC) at which point user ROOT on the PI can wget the script and run it. That takes a short length of time after which I switch to user pi and run the script fully. More on this later.
There may be an issue with the script. This is with a fresh install of 2018-04-18-raspbian-stretch an a Raspi 3.
I get this error when issuing this command node-red-start
Node-RED is not yet fully installed
/usr/bin/node-red-start: line 91: update-nodejs-and-nodered: command not found
I’ve tried multiple times with the same result.
Thank you DrFragle and MrShark
I will try this.
Is it ok however to re-run The Script on the same installation or should I start from a clean setup?
A clean installation is usually safest. I’d hold off on using the script for a day or so – we took out a recent mod and are about to put it back in.
Righto will wait for a bit, thank you both for the quick replies.
use script as it is right now, the recent addition was a lighter http server (caddy), which was optional and alternative to apache… unfortunately, while cleaning my github gists i wrongly deleted its configuration file, so i’ve to redo it… but, NO ETA, as it’s not an essential service and apache worked and works just fine… if i’ve spare time in weekend, i’ll try to have it back, but priority is my docker setup, right now, and i can work on it only in weekends as i’ve busy days at work and in the evening all i want is everything but a linux terminal…
Dear Pete,
Thank you very much for this very useful script. I ran it on a VM Debian 8 and it worked like a charm. I have a small issue however, hope you can help:
Somehow in NodeRed it seems I can only login with the admin account (not the User account although the User account logs in fine through SSH to the VM) and the admin account will not let me install new nodes. See log below.
Maybe I missed something?
Thanks in advance
Loic
__________________________________________________
npm WARN checkPermissions Missing write access to /home/pi/.node-red/node_modules/bcryptjs
npm ERR! path /home/pi/.node-red/node_modules/bcryptjs
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall access
npm ERR! Error: EACCES: permission denied, access ‘/home/pi/.node-red/node_modules/bcryptjs’
npm ERR! at Error (native)
npm ERR! { Error: EACCES: permission denied, access ‘/home/pi/.node-red/node_modules/bcryptjs’
npm ERR! at Error (native)
npm ERR! stack: ‘Error: EACCES: permission denied, access \’/home/pi/.node-red/node_modules/bcryptjs\’\n at Error (native)’,
npm ERR! errno: -13,
npm ERR! code: ‘EACCES’,
npm ERR! syscall: ‘access’,
npm ERR! path: ‘/home/pi/.node-red/node_modules/bcryptjs’ }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR!…
It worked, thank you :))
just found a nice, modern web gui for sqlite: sqlite-web (https://github.com/coleifer/sqlite-web)
very easy to install:
sudo pip install –upgrade pip
sudo pip install sqlite-web
sqlite_web -H ip.ip.ip.ip -p 81 -x -P dbs/iot.db
change the ip.ip.ip.ip to the ip of your board, otherwise the web will be available only on 127.0.0.1, so only from the board itself, locally…
-x is to avoid it opening a browser automatically
-P is to ask for a password to protect the web interface (you can delete the -P option if you don’t need a password)
-p is to change the http server port (default if -p is missing is 8080)
last parameter is the db to open, in this case Peter’s default one
that’s it 🙂
oh, forgot… if you put 0.0.0.0 as ip, it will work even if you change your local ip, of course…
the script has now a new option, default unchecked…
you can use Caddy (https://caddyserver.com), a very lightweight http server, INSTEAD of Apache for the services that the script is meant for: so, you can have phpliteadmin, phpsysinfo and the basic services access page Peter created on this http server, using less resources…
BARE IN MIND the capital “INSTEAD” i just wrote… you have to be SURE to UNCHECK apache and CHECK caddy if you want this, as they do the same stuff and use the same tcp ports and local folders, you can’t have them both… well, technically you can if you know what you’re doing and install caddy yourself, but that’s an other story…
unfortunately, i screwed up my gists, wrongly deleting some of them, so the caddyfile is now lost… if any of you has installed caddy from the script in the days between september 16 and 18, please share this file again:
/etc/caddy/Caddyfile
Dear Pete,
Thank you for this excellent script which worked like a charm for me (Debian 8 running on Synology VMM).
I am only having a small issue with NodeRed, it seems the created Admin account does not have the rights to install further nodes through the Palette Manager.
So I tried to install the nodes from the root account via SSH but it completely messed things up and had to start over again (I love VM…).
Log below. Could be obvious but I am (still) fairly illiterate on Linux, thanks for any help!
Loic
npm WARN checkPermissions Missing write access to /home/pi/.node-red/node_modules/bcryptjs
npm ERR! path /home/pi/.node-red/node_modules/bcryptjs
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall access
npm ERR! Error: EACCES: permission denied, access ‘/home/pi/.node-red/node_modules/bcryptjs’
npm ERR! at Error (native)
npm ERR! { Error: EACCES: permission denied, access ‘/home/pi/.node-red/node_modules/bcryptjs’
npm ERR! at Error (native)
npm ERR! stack: ‘Error: EACCES: permission denied, access \’/home/pi/.node-red/node_modules/bcryptjs\’\n at Error (native)’,
npm ERR! errno: -13,
npm ERR! code: ‘EACCES’,
npm ERR! syscall: ‘access’,
npm ERR! path: ‘/home/pi/.node-red/node_modules/bcryptjs’ }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR!…
Just a heads up guys – not sure where it is coming from – I have a Ubuntu VM built from 16.04 from THE SCRIPT.
I have been managing my own updates etc through webmin no problems.
Today i went to install contrib-moment through the palette manager and it deleted a heap of the nodes that had been installed via the script – the only reason i picked it up – was the Darksky node i was using refused to deploy after i restarted Node-red.
Apparently this could come down to the nodes that you provide as part of the script are not registered in package.json and there are changes between NPM5 and NPM6
I recently upgraded to Node-red 19 and also NodeJS 10 so this may have triggered it.
I have a snapshot of the VM from prior to the upgrade – will try and fire it up and see if installing the updated moment node causes the same issue and report back
mmm, about this… we can install nodes using a package.json and then just running “npm install” in the same folder, instead of npm installing every of them 1 by 1 as it’s now…
this way you’ll have a good starting point… i’ve already one, as this is the way i install nodes in nodered in the docker setup i’m working on, you can check it here:
https://github.com/fragolinux/DockerIOT/blob/master/nodered/package.json
Yep already manage it all myself (snapshot before doing an upgrade) and all good. I was just interested if that use case had been handled or not – then everyone would stay pretty much in lockstep when asking you questions etc – obviously docker will make all of this a LOT easier
Craig
Antonio/Peter,
What is you recommended path once this is in an running. I have created a Ubuntu 16.04 VM about 18 months ago from the script and it has been running with no problems. Was thinking about upgrading to the latest Node.js – is it best to just handle all of those things by ourselves once the system is in place – or is there some magic in the script to handle in place upgrades ??
No magic. Ensure you have backups.
script is NOT meant to be used for upgrades… it’s only for 1st run, initially… other than that, most of the installed software is added via repositories, so you can just update your system via apt to have it upgraded… but nodejs will not be upgraded this way, and on purpose: every node should be recompiled, if you upgrade node…
you can use my script to upgrade everything, not node, as said:
https://gist.github.com/fragolinux/412aa50683eb32bb879a120c73110938
as said, do backups…
just sent to Peter an updated script version, changelog:
removed menu options for MC, CU, SCREEN, MPG123, as i’ve added them per default to the requirements apt-get line (who uses them will have it, who don’t will not be harmed by their presence…)
fixed java install on latest raspbian…
added menu option HWSUPPORT (default UNCHECKED), that will enable i2c, gpio, serial, only for who needs it, reducing install time for those who just use script in a virtual machine or on an SBC just as centralized controller with no hw connected…
and of course the mods of yesterday to fix settings.js problems with latest nodered
new node-red out: 0.19 (video in next article)
https://nodered.org/blog/2018/08/14/version-0-19-released
last version that will work on nodejs <8…
I ran the updated script 8-1-18 and everything installed.
my password for node-red and dashboard are not working.
any ideas on a workaround so I can log into node-red ect.
Thanks for your hard work!
Brad.
share logs somewhere…
No idea as I;ve no idea why your Node Red login failed…. unless…. You don’t log in with PI or root – you log in with the admin name and pass you put in when running the script. Node-Red-Dasyhboard login is the user name and password you put in when running the script
maybe problems with bcrypt again… without logs, no solution…
I set a link to the logs, did you not get it?
https://drive.google.com/file/d/1vU2ixoEufQTaCWxjIBMNn22M_Sz9-_r_/view?usp=sharing
i see this at the end of node install, before nodered:
E: Unable to correct problems, you have held broken packages.
unfortunately no other info in logs… so, try using the v6 version script, not the test version as node8 still causes problems, as far as i can see…
and… java repository seems broken, too:
E: Package ‘oracle-java8-installer’ has no installation candidate
both are not script related, but repository related… nothing we can do, beside stick with previous versions…
I suspect the problem is in the script has been broken by the OS.
I suspect that previously, npm was installed as a prerequisite of node.js and now that is not happening so the script will need to be modified to install npm.
To test this, I just tried 2 installs on Raspberry Pi 3’s at the same time to get around issues being caused by my dodgy internet connection – the only difference between them was that one was on wifi and the other cat5.
Both started with a fresh SD card with the 6/18 version of raspbian. On both systems I ran apt-get update and apt-get upgrade then on one I installed npm.
From there the script ran identically up until it got to installing nod-red when the system I hadn’t installed npm on had an error.
Peter, let’s add a patch… in script (june edition), between lines 815 (FI) and 816 (IF) add this one:
sudo apt-get $AQUIET -y install npm 2>&1 | tee -a $LOGFILE
Added to last script and to exp version
My “the script” – the subject of this blog entry – installs NodeJS along with Node Red.
I tried several times with DrFragle’s script because I wanted to try out home assistant and it appears the nodes for that need node V8. Every time I wanted to add some nodes it would delete some others, so I went back to the official script. It failed a few times and I suspected it was due to flakey internet, but on inspection of the logs, it wasn’t installing npm so would fail with unknown command when npm was called.
I did an apt-get update and apt-get install npm before the script and it went through fine. I couldn’t see a line to install npm in the script so I’m wondering if it used to be installed as a dependant, but now needs to be explicitly installed.
I had the same case today, in a fresh Raspbian install, (Stretch 9).
Doing the “apt-get update and apt-get install npm” let my script run.
I got NodeJS 8.0
Sorry for the long post.
@Steve – I was running the script on a DietPI distro and I saw the same problem where the log file in the home/pi dir was owned by root and had not been updated when the pi user ran the script.
This happens when the root user runs the script (creating the pi user and copying the script log file to home/pi) and within the same minute the pi user tries to run the script again.
There are 3 line in the script which I changed to avoid this.
Line #288 – Added seconds to the temp filename so the pi user gets a unique name.
Lines #365 and #375 – It looks like the intention was to have the copied log file owned by pi (because of the “chown pi.pi”) but because the “-p” option was not used, the result of the “cp $LOGFILE /home/pi” command is that the /home/pi version is owned by root. I decided to reverse the commands do the “chown pi.pi” on the /home/pi version after the cp command.
Here is my version in case anyone needs the details.
Line#288
LOGFILE=$HOME/$0-`date +%Y-%m-%d_%Hh%Mm%Ss`.log
Line#365, 375
cp $LOGFILE /home/pi && chown pi.pi /home/pi/$LOGFILE && chmod 644 /home/pi/$LOGFILE
Sorry but there was a bug with my change to the script on line 365, 375.
It is better to just use the “-p” option to the cp command and leave the rest of the line the same.
Line#365, 375
chown pi.pi $LOGFILE && chmod 644 $LOGFILE && cp -p $LOGFILE /home/pi
I tried this yesterday as I managed to fry my install trying to update Node.js.
First try, the script didn’t work (I now think it was connection issues while installing).
Second try, I used Dr Fraggle’s script from his post on July 8 as I wanted Node V8 and that worked BUT none of my passwords that I set up during the script worked. I can get into Node-Red admin panel if I uncomment the login details in the settings.js config file (I tried to add my passwords there but that didn’t work) but I cannot access the dashboard.
Is there a way I can update the passwords and make them work? Or should I start from scratch and try the standard script again?
as said, there were (or still are) problems with some remote repositories from where the script downloads the needed packages… if repos are offline or broken, errors occur… but it’s not a problem of the script itself… take a look at logs in your home folder and search for “404”… if you find them, then some packages were not downloaded/installed… nothing we can do about it, unfortunately…
I stepped through your script and found that bcryptjs didn’t install so the password hashes weren’t where they should be. I was able install it and do the steps required to create the hashes and copy them into the settings.js file.
I’ll be sure to back up my SD card this time so I can recover more easily next time I screw something up.
Thanks for your work.
Note: For some reason the log was owned by root and therefore hadn’t been written to for the part of the install done by user pi.
that package, together with others like i2c, sqlite, serial, is one of those that for some reason are failing recently to be downloaded… retrying will often solve, but you should know what to retry, as you cannot rerun script as it is…
Hello peter, hello guys,
I have problems using the script script.sh on a new Raspberry Pi 3B , OS stretch.
script.sh: line 847: npm: command not found
sudo: npm: command not found
Then the script hangs at…
Unpacking python3-rpi.gpio (0.6.3~stretch-1) …
Setting up python3-rpi.gpio (0.6.3~stretch-1) …
Please help to get it to work
Thanks a lot.
I don’t see this being the script – I just set up 2 RPI3 B+, an RPI3 and an RPI2. There was an issue with Node the other day but that seems to be history – again it was not the script.
Hi guys,
Thanks for taking the time to make/update the script and help people to get it working.
I’ve been trying for days trying to get it to work for me. I have a Raspberry pi 3B and I’ve tried the script with Dietpi and Rasbian Stretch lite and I’ve been having the same problem with both.
script.sh: line 847: npm: command not found
sudo: npm: command not found
Then the script hangs at…
Unpacking python3-rpi.gpio (0.6.3~stretch-1) …
Setting up python3-rpi.gpio (0.6.3~stretch-1) …
I would appreciate any help you could provide.
Here is a copy of my log file. https://www.dropbox.com/s/h4pzllwvn7u7r99/script.sh-2018-07-13_05h06m.log?dl=0
See elsewhere discussion about deleting this file /etc/apt/sources.list.d/nodesource.list
Could you be a little more specific? I’m not finding any references to deleting nodesource.list.
Hi.
For my RPi2 b have the same problem… Couldn’t find the solution for this moment
https://nodered.org/docs/hardware/raspberrypi
and
dceejay Leader
July 11
If that fails then delete
/etc/apt/sources.list.d/nodesource.list
And then re-run
(The issue is that you are Nide 6, and pi have now released node8 and it starts to install that and while it should work… Obviously it isn’t. Deleting the file above and rerunning will make my script think its not installed and will update to 8 “our” way.)
Peter,
Thanks for all of your hard work, even on the weekend I see.
Your recommendation fixed my node-red issue but I ran in to a problem with settings.js. I did have to delete the nodesource.list file and rerun their install/upgrade command and then your script again. It worked but when attempting to load node-red I would get an error:
$ node-red
Error loading settings file: /home/pi/.node-red/settings.js
/home/pi/.node-red/settings.js:251
editorTheme: {
^
SyntaxError: Unexpected token :
at createScript (vm.js:80:10)
at Object.runInThisContext (vm.js:139:10)
at Module._compile (module.js:616:28)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object. (/usr/lib/node_modules/node-red/red.js:115:20)
I removed the extra }, just above editTheme (there are 2 sets back to back. That then gave:
Error: Cannot find the module ‘i2c-bus’
So I commented the variable at the beginning and the call in httpStatic. That got me an error on ‘moment’:
Error: Cannot find module ‘moment’
So commented that too and node-red would then start. I am sure I have broken more than I fixed but wanted to report my findings.
Have a good weekend
Glenn
Alaska
Ok, for me works those steps:
1) First I installed latest NodeRed (“bash <(curl -sL https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/update-nodejs-and-nodered)");
2) After rebooting I've successfully installed the Script.
The following did not work…. (I did have to cd to the cd node_modules after cd .node-red)…
cd .node-red
npm remove serialport node-red-node-sqlite i2c-bus node-red-node-serialport
sudo npm -s install –unsafe-perm –build-from-source serialport node-red-node-sqlite i2c-bus node-red-node-serialport
All sorts of mystery messages spewed out. Looks like a lot of syntax errors? I let the it all run until complete, rebooted. NR did start but a lot of the flow nodes were screwed up.
I restored the backup I had and all is back to what it was before the “fix”. That is the good news. Backups indeed do work! Yeepie.
I will live with the few warning when NR starts up. Maybe I should have coped all the install messages…but I didnt.
Hope some of this helps you get things working.
Cheers
Bob
I too had problems yesterday, will try again tonight.
Many of those messages are warnings not script related… but yes my confidence level in the change is not high yet hence leaving the original pre-node-8 version in place.
On my way out for the morning but for the record, this morning I took the unmodified script, rpi3 b+ and made a single change to reference latest node. all worked except passwords not in the nr settings file – added those from an older installation and it looks like all is ok up to now.
seems like some nodes are not ready to be used with node v8, so…
about password, take a look at logs, recently i had issues with bcrypt (which is used to generate the passwords) even on v6, so that could be the problem…
both, anyway (v8 and bcrypt) are not script related but due to the respective maintainers…
sure, better stick with v6 for the moment, as it’s an LTS version anyway… it works and there’s no hurry to go to a newer version… in the end, we use the nodes (which use node undenneath…), not node itself 🙂
What if I execute the following on my existing system?
sudo npm $NQUIET install –unsafe-perm –build-from-source serialport node-red-node-sqlite i2c-bus node-red-node-serialport 2>&1 | tee -a $LOGFILE
You think the warning will be fixed and wont cause any NEW problems? Not that the warnings are a big problem…its just a clean startup might make me sleep better 🙂
I will do a backup before…just in case.
Bob
you can remove those nodes, before, if you want… so you’re going to install them fresh… and after that, run the command but as follows (those vars work only inside the script…)
cd .node-red
npm remove serialport node-red-node-sqlite i2c-bus node-red-node-serialport
sudo npm -s install –unsafe-perm –build-from-source serialport node-red-node-sqlite i2c-bus node-red-node-serialport
Well after much success with an RPI2, my RPI3 B+ installation was looking grim with no Node-red.. So….
sudo npm install –unsafe-perm i2c-bus
that worked but no serial..
sudo npm install –unsafe-perm –build-from-source serialport node-red-node-serialport
That worked..
i’m testing in vm, where of course there’s no hw, but i think we can safely add this mod to script… when we see nodes with warnings, compile from source does the trick…
let me test the problems with nodejs v8 and i’ll release the updated script ASAP
please send me your logs to see what happened in your v8 install yesterday
not a good day to test anything… tried in vm using ubuntu 16.04, italian mirror is corrupted and i had tons of errors 404… reverted snapshot and retried, switching to german repository, same errors… i’ll test again in weekend… or have to test using debian…
Normally I keep on top of Node versions but looks like there were updates while I was off-planet, Node 8.11.3 it is.
Peter, i see interesting stuff here: https://github.com/node-serialport/node-serialport#raspberry-pi-linux
“If you’re going to use sudo or root to install Node-Serialport, npm will require you to use the unsafe parameters flag.
sudo npm install serialport –unsafe-perm –build-from-source
Failure to use the flag results in an error like…”
i think it’s best to use this version for these problematic nodes (i2c, serial, sqlite are ALL, from time to time, error prone…), and remove them from the long npm install line we have right now… this way is SURE they’re compiled for the specific board and right in time…
so, splitting this line:
npm $NQUIET install moment node-red-contrib-config node-red-contrib-grove node-red-contrib-diode node-red-contrib-bigtimer \
node-red-contrib-esplogin node-red-contrib-timeout node-red-node-openweathermap node-red-node-google node-red-node-sqlite \
node-red-node-emoncms node-red-node-geofence node-red-contrib-moment node-red-contrib-particle \
node-red-contrib-web-worldmap node-red-contrib-sqldbs node-red-contrib-ramp-thermostat node-red-contrib-graphs i2c-bus \
node-red-contrib-isonline node-red-node-ping node-red-node-random node-red-node-smooth node-red-contrib-npm node-red-node-arduino \
node-red-contrib-file-function node-red-contrib-boolean-logic node-red-contrib-blynk-ws node-red-contrib-chatbot \
node-red-dashboard node-red-node-darksky node-red-node-serialport node-red-contrib-owntracks node-red-contrib-opi-gpio node-red-contrib-alexa-local 2>&1 | tee -a $LOGFILE
into these 2 lines:
npm $NQUIET install moment node-red-contrib-config node-red-contrib-grove node-red-contrib-diode node-red-contrib-bigtimer \
node-red-contrib-esplogin node-red-contrib-timeout node-red-node-openweathermap node-red-node-google \
node-red-node-emoncms node-red-node-geofence node-red-contrib-moment node-red-contrib-particle \
node-red-contrib-web-worldmap node-red-contrib-sqldbs node-red-contrib-ramp-thermostat node-red-contrib-graphs \
node-red-contrib-isonline node-red-node-ping node-red-node-random node-red-node-smooth node-red-contrib-npm node-red-node-arduino \
node-red-contrib-file-function node-red-contrib-boolean-logic node-red-contrib-blynk-ws node-red-contrib-chatbot \
node-red-dashboard node-red-node-darksky node-red-contrib-owntracks node-red-contrib-opi-gpio node-red-contrib-alexa-local 2>&1 | tee -a $LOGFILE
sudo npm $NQUIET install –unsafe-perm –build-from-source serialport node-red-node-sqlite i2c-bus node-red-node-serialport 2>&1 | tee -a $LOGFILE
Yes it was the update to node.js LTS 8 that broke things. node -v shows v8.11.3 now. I also see some warning errors in the syslog file related to serial, etc, but they don’t break NR. At this stage I don’t want to do another clean install. Its been a struggle to get everything running. One of the issues was the OrangePi. It just wasnt stable. I dumped it and got a RaspberryPi 3 B+ and things have been MUCH better.
I did a fresh reboot now and copied the NR startup lines from the syslog file. Here is a link to those NR errors…
https://www.dropbox.com/s/6eshobtlgd5nwbg/NRerrors.txt?dl=0
Hopefully this info will be useful to fix things.
The log show there are some remaining warning errors due to epoll.node, sqlite3.node, serialport.node. It would be nice to clean up those problems…so any help on how to fix those would be appreciated.
Cheers
Bob
Well, I just took Antonio’s mod and installed the modified script on a fresh Raspberry Pi 2, with latest Stretch. After the usual RPI nonsense due to ssh not being set up by default along with root access, meaning I had to stick a monitor on the PI initially, Node-Red appears fine. Of course I had to copy my password from a previous installation. I could have sworn we put the code in there for Node-Red and ui password creation… anyway it works, no sign of any issues yet.
ssh, just the usual: create an empty “ssh” file in the folder which you have access in windows itself, just after flashing the sdcard…
indeed the node-red password creation is in script, from ages now… have to test in virtual machine, then… work for tomorrow, friends on the way now… 🙂
but in the end, if you run my modded script, you end up with the recent nodejs version, from official site, nothing more or else… it’s a part of what you do with that other script you run to update to version 8, which is specific to raspberry and made a mess on my rock64 when run…
as i said, i’ve serial port and i2c problems very often, on various boards… and now that you named it, i had problems with sqlite, from time to time… unfortunately nothing we can do in script: the way all of these 3 packages are installed is just NPM, so it does what it should and install process is beyond our intervention… we should check on their site or github issue pages to see what they report…
epoll: don’t even know what it is or what does install it…
about your “Yes it was the update to node.js LTS 8 that broke things”… what update? previous script installed version 6 and we never implemented an update script, beside the one found by Peter for raspberry…
Note to anyone looking in – this mod has yet to be tested on various machines.
FYI…
Some new Debian updates (that included nodejs) broke node-red. After many hours of trying to determine what caused the break…I discovered it was the addition of the I2C stuff Peter added to the NR settings.js file. I removed the additions (I dont need I2C) and NR launched OK. There is probably a more elegant fix, but wanted to post this here in case someone runs into this same issue. Here is some details about the error message…
Error loading settings file: /home/pi/.node-red/settings.js
The module ‘/home/pi/.node-red/node_modules/i2c-bus/build/Release/i2c.node’
was compiled against a different Node.js version using
NODE_MODULE_VERSION 48. This version of Node.js requires
NODE_MODULE_VERSION 57.
if you have time and you want to, can you try this version of the script, restarting from scratch? The only difference is that i swapped nodejs from v6 to v8, which is the new stable and anyway we should move to it soon…
https://pastebin.com/Gsy5EWQx
i think it should work flawlessly, but if not we need to:
1) move to v8 anyway, looking for eventual problems (i’ve none on my boards…)
2) move HW related stuff on a new menu item (default UNCHECKED), so those who want just a bare IOT controller system or virtual machine and NO physical sensors/actuators should be just fine…
if you look at other comments, mine too, you’ll see recent problems with i2c, and with serial in the past…
I have put it on bitbucket called scriptexp_july2018.sh and will try on one of my boards asap. Indeed it seems the IBM guys now recommend Node.js LTS 8.x
Thanks
Pete
eventually add a new note at beginning for the changelog, moving nodejs from v6 to v8, or what you think it’s best 🙂
Hi Pete / Antonio,
Great website / script.
I did a fresh install last night of Raspbian Stretch Lite + the script on a RPI3B
I’m also getting the corrupt /etc/rc.local file like Luca mentioned previously.
from pi home directory as user pi did:
wget –no-check-certificate https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh
bash ./script.sh
Only defaults where used.
On line 7 in /etc/rc.local, get an unterminated string:
exit 0 ” on success or any other
Easy to fix, but not sure if others getting same error.
Link to log of install.
https://www.dropbox.com/s/ohxtszr6ktiq5gr/script.sh-2018-07-01_01h00m.log?dl=0
Cheers
Nigel
I’ve never used the lite version and never suggested it is compatible.
no problems there, it’s just the basic linux without GUI (the Pixel one in case of the official raspberry), but as many of us use the various SBC headlessly, it’s fine 🙂
About to start travelling, AWOL until mid week.
no problems in log file, except that java repositories is changed again so no java installed… i’ll have to take a look, hoping java8 is still there, as java10 has some problems… but we use it only for habridge AFAIK, so no big issues…
about rc.local, please share the bugged one so i can take a look to what causes the corruption…
I tried this last night on an old B+ with DietPi. It seemed to run OK but had to leave overnight to complete. Tried to access the Web page this morning with success but everything else seems to have failed. No time this morning to check more but will update later. Anyone else had success running the script on a B+?
B+ is that the original pi? Ie not one of the version 2 variations? I don’t test that far back as they are really not powerful enough. Maybe someone else has.Even if everything works you will be disappointed with performance.
Pete
never tested on such an old device, but it should be like an RPi Zero or the like… you have a log file in your home folder, take a look at there to see what happened… a night is TOO MUCH, to me you had network issues and it couln’t download some stuff…
Success! How bizarre. I came back from work. Started the Pi up and now everything is working. Just created a test flow for MQTT and it’s spot on. Thanks very much for all your work on this! This Pi does have limited resources but will be fine for my tinkering.
just installed this on a pi 3b+ it took 35 mins to install and works great
love all the new programs which it installed. another deep learning curve now
!!!!! SUCCESS !!!!!
The script ran fine and all the utilities/programs installed OK and run fine from the web page interface.
I wanted to get back here to thank Antonio for hanging in there to get the script working for me. Your expertise and patience are world class. Your help is very much appreciated.
I wanted to also summarize what happened and point out a few things that could have shortened the time required to diagnose problems.
My first run of the script resulted in a bad node-red install. I knew this because the web page interface would not launch NR. I was ask to upload the script log. Nothing in the log pointed to the problem. Then I was ask to run nrlog. The log showed a truncated error list and wasn’t helpful. Turns out a complete NR log would have pointed to the problem. It was many days later that full NR install errors showed the problem was related to I2C. I discovered the problem after attempting a manual install of NR. The manual install of NR streamed many errors to the terminal indicating I2C was the issue. I did another clean op sys install, ran the script, edited the settings.js file to remove I2C, then rebooted. All was well. So the problem could have been resolved much quicker IF those NR errors would have been logged in the NR log. I’m not sure if that is a NR or script issue. I also think it would be a good idea to allow the user to disable I2C in the script, so an edit of the settings.js file is not required.
Hardware and op system:
Orangepi Zero with 512mb
Armbian_5.38_Orangepizero_Debian_stretch_next_4.14.14.img
Thanks again! Now I am off to understand and use all stuff I have installed! haha
Cheers
Bob
my bad, i thought the error was recent just after boot, so nrlog (which is an alias for a longer command and that colors its output) was enough, if the error was in latest lines… instead, it would have been better asking for full node-red log file, indeed…
I deleted all lines in the settings.js file that related to i2c. Manual install NR again and got no errors! Connected to opi0 via web and now the NR links open ok from Peter’s webpage.
I’m going to do a clean install again later and edit the settings file to remove those i2c lines and run the script again. I suspect all will be well. I will update again.
Im thinking I will have to run the script once to get the settings.js file..then edit it and run the script again?
no, you can just remove those 2 lines before rebooting after running the script, or you can comment these 2 lines (which are the ones that add those 2 lines to settings.js) before running Peter’s script:
886: sudo echo ” i2c:require(‘i2c-bus’), ” >> tmpfile
906: sed -i -e ‘s#\/\/\s*var fs = require(“fs”);#var fs = require(“fs”);\nvar i2c = require(“i2c-bus”);#’ /home/pi/.node-red/settings.js
Peter, what about adding a menu item to add or not the hardware related stuff? I2C, for example, is useless if you are installing in virtual machine… and have caused problems to me, too, with some boards… even standard raspbian does not enable serial, i2c, spi, in default install…
Yes, but the only thing we have to do with i2c surely is a node and that should not cause issue till used? The mosca mqtt broker looks to be broken but does not fail and kill nr until I attach an mqtt input to it?
Pete
i had problems, in various occasions, with the serial node and the i2c one for sure… sometimes it happens that npm complains about missing binaries and have to compile them, and it takes AGES on these ones…
problem is that i couldn’t address the problem, as rerunning script a few days after is totally fine… so, is something related to their maintainers and their updates to their packages, don’t know what else to say…
if someone wants an SBC (or a VM) to run just as a controller, demanding the actual sensor/actuator stuff to mqtt controlled nodes, these HW related stuff is useless…
and in any case, we will NOT remove them, for sure: i think the best option is to add an item to menu, unchecked by default, to allow everybody who needs them to add, on his discretion…
Or her discretion, indeed.
LOL, yes 😀
I think I finally discovered the NR install problem. I tried a manual NR install again and this time I noted the errors in the nrlog. All the errors relate to I2C. Seems the opi0 has i2c ability but its not enabled. I dug through some info about overlays etc to enable ic2, but didnt succeed. I also read your blog about i2c on the opi0 but you were using dietpi. I am using armbian debian. Does dietpi somehow pre-enable i2c?
Here are a few of the lines of the NR error log….
Error loading settings file: /home/pi/.node-red/settings.js
{ Error: Could not locate the bindings file. Tried:
→ /home/pi/.node-red/node_modules/i2c-bus/build/i2c.node
→ /home/pi/.node-red/node_modules/i2c-bus/build/Debug/i2c.node
→ /home/pi/.node-red/node_modules/i2c-bus/build/Release/i2c.node
→ /home/pi/.node-red/node_modules/i2c-bus/out/Debug/i2c.node
→ /home/pi/.node-red/node_modules/i2c-bus/Debug/i2c.node
→ /home/pi/.node-red/node_modules/i2c-bus/out/Release/i2c.node
→ /home/pi/.node-red/node_modules/i2c-bus/Release/i2c.node
→ /home/pi/.node-red/node_modules/i2c-bus/build/default/i2c.node
→ /home/pi/.node-red/node_modules/i2c-bus/compiled/6.14.2/linux/arm/i2c.node
So now what is your suggested next move? …
* Go to dietpi (thats no longer supported on the opi0),
* Help with enabling i2c on the opi0,
* Buy a different sbc (looks like the rasberrypi w is a good replacement)
* or ?
I’m leaning toward dumping the opi0 and going to the raspberrypi.
if you don’t need it, just disable/remove (as i always do, too…) the 2 i2c lines in settings.js and live happy 😀
Hi Peter,
I did use Andrea’s Spiess image before based on a outdated Dietpi image now which cannot be update anymore.
Because of this i wanted to install it again on a new version of DietPi (v6.8)
I ran into some issues:
* PIP did not seem to be installed (sudo: pip: command not found)
can be resolved by installing before (sudo apt-get install python-pip)
* dropbear SSH server is being used by default (did nout cause issues as far as i am aware of, but configuration might nt be altered by the script)
Did anyone ran into these issues as well?
1st question: strange, python-pip is in line 585 of script from ages, now…
https://bitbucket.org/snippets/scargill/ekon6o/the-script
for the latter, yes, it’s standard dietpi… you can switch to ssh via dietpi-config after 1st boot, or just by editing the config.txt and/or dietpi.txt in the fat32 partition of the sd card, even before putting it into your device, just after flashing the sd…
take a look at these 2 files, you can setup a lot of things before even doing the 1st boot… example, set wifi credentials, fixed ip, change dropbear for openssh, etc…
ah, you can use dropbear as it’s a valid ssh daemon, but it does NOT allow file transfers… so if you don’t need to use winscp or similar, don’t bother…
I’m getting rid of all boards with less than 512 meg RAM. Pointlessly self-limiting.
I will give dietpi a try as Im still getting bad NR installs. The dietpi website doesnt have any images for the orange pi zero…but there seems to be some floating around. I’ll try to find one and give it a shot. Crossing fingers!
fyi….after the latest script run…NR still bad. I ran the second script mrshark suggested…and it coughed up an error and never ran.
mmm… how much memory do you have on Opi0? Hope not the 256 model… otherwise could be that… you need a board with at least 512mb to work with the script…
The opi0 I have has 512mb…so thats not the problem.
I have never been on raspbian….I have always been on Armbian 5.38 stable Debian on an Orangepi Zero. I downloaded the latest script and see no changes to the one I have been using…so I would expect the same results…ie bad NR install.
OK…I will run the same script again and expect the bad NR install….then I will run that update script and hope for the best.
Thanks
I believe the info at the top of the script explains that we tested the script on Orange Pi Zero using Dietpi, NOT Armbian.
I am currently running Armbian 5.38 stable Debian GNU/Linux (stretch) now. I just did a clean install.
I want to be sure I understand your suggestions….
1. Run Peter’s script (should I select NR install or not?)
2. Run the NR update script
i suggested to run second script after peter’s one when you where on raspbian… now that you already changed distro, go for default setup in peter’s script and let’s see what happens… don’t run second one, report back only on peter’s script, we offer help on that one only…
I was just now looking at doing a manual install of all the stuff, looks pretty daunting. After your comment…I’m thinking maybe best to use the script to install everything EXCEPT NR. Then manually install NR. Good idea?
Maybe you can help with the command lines to do a manual install of NR? Also a list of all the script boxes to check to get all the stuff required. If I get a good install…at least that would isolate the area thats causing the problem.
I tried doing a manual install of NR on top of existing install…but it was a fail. However there was more diagnostic messages but I didnt save those messages :-(. At this point I am at a clean install of armbian and trying to decide what to do next. Your post was very timely…however I was hoping you would find the problem and I could run a new script and celebrate…haha.
If I do a manual install of NR and get those error message again…I will save them. I’m thinking its something in the settings.js file thats goofy.
yes, try armbian or dietpi (which is based on armbian and more lightweight, i like it a lot… even thought probably tkaiser will say it’s not good 🙂 )
or AFTER Peter’s script, try running the script suggested by Peter himself a few days ago: https://github.com/node-red/raspbian-deb-package/blob/master/resources/update-nodejs-and-nodered
it was just updated to latest nodered… it’s specific to raspbian, even though it “should” work on other distros, too
You may be onto something as I installed a few days ago. Here is the settings.js file….
https://www.dropbox.com/s/jn4trqh5zwqsj7e/settings.js?dl=0
Are we having fun yet?
Cheers
Bob
can’t see anything wrong… anyway, i’m planning to do a fresh install on a nanopi neo this weekend and i’ll what happens…
as said, i just did a fresh install using latest script on top of DietPi on a NanoPi Neo… standard menu selection + checked “grafana and influx”… well, “flawless victory”… all services up and running… so maybe something in raspbian? Don’t know and don’t have a spare sd/rpi to try…
there are some problems with those packages Peter added at the end of the script a while ago (bottle and glances), but absolutely not his fault, it’s just that those installs sometimes work and some don’t, i’ve already seen them going in error… just retrying a few days after and they’re fixed… nothing essential, BTW…
Do to my limited linux knowledge…I didnt know that was a hidden folder. I used winscp and I do see the .node-red folder. The settings.js file is in there too! Yes I am 100% sure I did a clean install. I reformat the sd before every install. I tried to start NR manually(node-red-start) and I get all the info on how to use NR etc, then see NR started as service, starting graphical event wiring tool then this…
Started Node-RED graphical event wiring tool.
Error loading settings file: /home/pi/.node-red/settings.js
{ Error: Could not locate the bindings file. Tried:
Same info from nrlog.
The settings.js file in the /home/pi/.node-red folder and its full of NR stuff…ie not empty. Seems like NR cant see that folder or rights to the file/folder are bad? The settings.js file has group sudo R properties too.
any new ideas?
can you upload the settings.js file somewhere? Remove the encoded passwords from latest lines if you want
but just seen that AGAIN a new nodered version is out, a few days ago… maybe that introduced some new lines in settings.js file… unbelievable…
https://github.com/node-red/node-red/releases
nrlog….
Started Node-RED graphical event wiring tool.
Error loading settings file: /home/pi/.node-red/settings.js
{ Error: Could not locate the bindings file. Tried:
The above is exactly what nrlog reported. I do not see a .node-red directory in the home/pi directory. The only files there are…
dbs, habridge, nodesetup.sh and the script log file.
sorry, don’t know what happened… on line 848 of the script (https://bitbucket.org/snippets/scargill/ekon6o/the-script) there’s a command:
mkdir .node-red
so AT LEAST the folder should be there… sure you didn’t run the script on top of an already run system? or deleted something?
the folders starting with a dot are HIDDEN… you can enable winscp to show them, or run from shell:
ls -a /home/pi
to see them…
Im running Armbian 5.38 stable debian stretch. Here is the log file…
https://www.dropbox.com/s/yqea1jctmtlrvbj/script.sh-2018-05-28_16h49m.log?dl=0
If there is a better op system please let me know. I have no preference.
Thanks for the help!
i see both installed in logs… can you run “nrlog” to see what’s there?
for habridge, sorry, never used, don’t know what to check there… i think it’s useless now, as newer nodes for node-red are out and can manage same thing easier…
I did several installs and all failed installing node-red and HA bridge. Everything else worked fine. Im installing on a orangepi zero. I read the comments about update of NR. I assume the script is fixed now? Any ideas why the NR install and HR bridge failed? Each install was to a clean flash drive.
What should I do to ensure a good install?
Thanks
You’ve not stated which operating system, not all combinations have been tested. Orange Pi Zero was some time ago. If you are on same version of operatinmg system it could be changes to Node Red?
script is fixed, indeed… as always, post logs… they’re in your home folder…
an intro to regular expressions in javascript: https://www.youtube.com/watch?v=VrT3TRDDE4M
Thanks Guys,
Using the supplied update script from the Node-Red guys worked perfectly to perform the upgrade.
I then rolled back the snapshot and tried the script supplied by Antonio – again this worked just fine – so either option will work for you it appears on a VM built from the Original Script as supplied by Peter.
Craig
excellent
THanks guys, will snapshot the VM and try both and then report back as to results from both
Craig
beware that you can’t run the script again on top of an already run install…
my comment was about Peter’s script, sorry, i was on the rush and thought about that… of course you can run my update script on top on an already installed system, that’s its purpose 😀
Peter, What is the recommended way to upgrade Node-Red once installed with your script ?
I have built a Virtual Machine about a year ago from the script and it has been running fine – i have kept all of its modules upto date with Webmin.
Now i want to upgrade Node-Red – is the official script from Nick Oleary OK to run on your deployed system ?
https://nodered.org/docs/hardware/raspberrypi
regards
Craig
In fact I have used his script to do the upgrade and move all remaining nodes off root to user pi… but only done thios in Raspbian Stretch on a Pi2 and 3
this is my script that i use to keep my systems updated… you can remove last 4 lines about home assistant if you don’t have it…
https://gist.github.com/fragolinux/412aa50683eb32bb879a120c73110938
This script does not update the nodes but throws them away in the end.
This is the same result I get when I try to update a node via the Palette manager. When I change owner and group as done in the script it will run with the same disaster.
Look in my log if there is any lead why this happens.
https://drive.google.com/file/d/1_8MakDX-z7V_Brunlzc1OeKyh1XUFrXk/view?usp=sharing
For now, I go back to a copy!
don’t know what to say… never had issues, they’re just a bunch of commands that you would manually run in shell one by one… and no errors in your log…
Peter,
Wonder if you investigated turning the script into a docker image ? Would be much easier for you to manage longer term and fairly easy from an install point of view (with adequate documentation)
Docker holds no interest for me.
i’m studying it for work and for sure i’ll try to apply to the script, too 🙂
no ETA, so don’t ask… it will be ready when it will be ready, if it will be ready…
guys, docker is f***ing awesome… working on it, there’s A LOT of stuff out there… but we can reduce memory footprint of the various services A LOT, without loosing functionalities and having a setup easier to deploy, update and backup… stay tuned!
Peter, do you mind creating a topic to discuss of this, so i can have help from others and have a quicker setup soon? Something like: “WIP: Docker”, or the like, you decide… Thanks in advance
I’m not by my machine but I will send the log file – not that I could see anything in it when I checked earlier. If it helps the extra brace and comma appears after the logging section and just before the editor theme section ( which being new may be the culprit). One other clue may be in that the script appears to have skipped the two lines which set up nrlog and the systemctl enabling of node red as I had to start it manually after fixing the problem and rebooting and nrlog is not there. Actually while typing this I managed to access the settings file and I note that the httpStatic section in my settings file which is the last section is missing its closing brace – I hadn’t noticed because this is not throwing an error.
In summary the last two sections of my settings file are editorTheme and httpStatic and the closing brace that should come after httpStatic appears before editorTheme and this may be linked to the fact that the script then skipped the nrlog and systemctl lines (although the intermediate lines relating to flowfile and require fs were carried out. As to why I cannot see.
Just to be clear this was a completely fresh first time install on a new install of the OS which had been upgraded to latest versions.
i think i’ve found the problem… now i’ve read your comment and i realized just a few days ago there was a NEW nodered version, 0.18.6…
take a look at last line of the changelog:
https://github.com/node-red/node-red/releases
“Add editorTheme.projects.enabled to default settings.js”
they enabled the project feature by default now… so, settings.js file is now different from before… and that’s the error, not the script itself, or not just it…
if i fix script now, it will work ONLY if you install it NOW and so having nodered 0.18.6… but, btw, what we advert is to run on fresh systems, so if you install it now, you’ll end up having that anyway…
let me update nodered and fix it…
to be continued
sorted out… Peter is going to update the script… the mod is to add this line after line 919 of ACTUAL script, to restore the correct number of curly brackets:
sed -i -e ‘s#\/\/ Customising#},\n\n \/\/ Customising#’ /home/pi/.node-red/settings.js
as said, problem introduced by nodered 0.18.6 which is out for 3 days, now…
Done – many thanks,
Thank you guys very much! Amazing work so quickly.
And to be clear, I got it working last night. Y’all need anything else from me?
Thanks Neil, it looks like they made a change to Node-Red just days ago. Sorted, apparently.
Howdy! Great work!
On a recent install of your script I ran into an issue where NodeRed wouldn’t start. i traced it down to a commented close brace in the settings.js file at line 244 after the install. I don’t know if I am on the only one running into the this issue but it was something I had to look at it since I use your script just about everywhere. Thank you for the amazing work and contributions.
Neal
I did an install a few days ago and the script worked just fine, I do hope you sort this out. Do let me know what happens. I can’t immediately see anything wrong with the script.
Hi – I had exactly the same issue on an install done on Saturday – latest version of raspbian stretch lite on a pi3b and latest copy of the script – ended up with superfluous closing brace and node red would not start
Do you have any ideas? I can see nothing wrong with the script but I could be missing something… I have just installed the script on a NanoPi K1+ in the last few days.
and i tested it on rock64, as well with no issues… and the node-red part is untouched from months, now…
please, both, provide the log files you have in your home directory, thanks
Sure thing!
Here you go.
https://www.dropbox.com/s/442xfd3cwazirn9/script.sh-2018-05-21_02h20m.log?dl=0
I can’t see anything wrong, I hope we can help these guys.
i stopped nodered on my board, saved all the content of the .node-red folder, emptied it, ran lines 872-919 of the script (the ones that create the settings.js and update it adding encrypted password and some other stuff): no problems at all… sure you didn’t run the script twice, even if there are some protection in script to prevent that?
In case you can, can you start over, and send me the 3 settings.js (1 as such, 1 with .bak-pre-crypt and 1 with today’s date in its filename) files you’ll find into your .node-red folder, BEFORE removing any damn bracket, to see WHERE it’s put so to find WHAT put it there…
you can strip your encrypted password if you want from the actual settings.js file (it’s in 2 lines towards the end of the file)
Hi, cannot download the script from http://bit.do/the-script, do you have any other address?
see bitbucket scargill – snippets
I downloaded (via wget) the script from:
https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh,
but it set up – not suprisingly! – a file called script.sh in my /root directory. I then used ‘bash ./script.sh’ to run it (successfully). Do you need to change the instructions to reflect this?
Done, updated blog entry, also change the permalink to “the-script” as against “a christmas script”. Earlier blo on the script renamed.
Hey Pete how are you, Im happy that you are back!
I was able to use your amazing script on jessy, and whezzy distros on a RP3, now Im doing a clean install on the latest stretch. I was going crazy because putty was ending the connection, and I found out that the raspy restarts suddently at the stage of installing nodes
I re run the script, without the extra nodes, all the rest of the things seems to work fine
I re run the install node red alone and same problem at the stage of installing nodes
Not a big problem, Node red works, I just needed to set ip up as auto start
but im not able to install the ui nodes
pi@raspberrypi:~ $ npm i node-red-dashboard
npm ERR! Linux 4.9.59-v7+
npm ERR! argv “/usr/bin/node” “/usr/bin/npm” “i” “node-red-dashboard”
npm ERR! node v6.13.1
npm ERR! npm v3.10.10
npm ERR! file /home/pi/.npm/encodeurl/1.0.2/package/package.json
npm ERR! code EJSONPARSE
npm ERR! Failed to parse json
npm ERR! No data, empty input at 1:1
npm ERR!
npm ERR! ^
npm ERR! File: /home/pi/.npm/encodeurl/1.0.2/package/package.json
npm ERR! Failed to parse package.json data.
npm ERR! package.json must be actual JSON, not just JavaScript.
npm ERR!
npm ERR! This is not a bug in npm.
npm ERR! Tell the package author to fix their package.json file. JSON.parse
npm ERR! Please include the following file with any support request:
npm ERR! /home/pi/npm-debug.log
Let me know If i’m doing something wrong here
Hello Peter,
I have follow the instructions to install the different programs in the script and they all seem to be ok. However, I don’t have any packages on Node-Red. I had node-red installed previously so I have uninstalled it and installed it again from your script. Still no new packages in node-red. Am I missing something?
Thanks
The script should automatically uninstall old NR on Rasbian. Script not tested on NOOBS. When it installs NR it should install a boatload of NR
nodes.
what am i missing when i get the web page Pete’s utilities up at 192.169.0.202 all the links work but not the node red two second and third in the list.
by the way i am not a coder and find all of this really hard going
Sorry guys,
it was a messy installation due to the poor internet connection.
I’ve done it again form the beginning and everything works now.
Marvellous!
Thank you for everything!
A few days ago I posted about how the latest DietPi somehow broke the-script but it did not make it past the moderators to show up on here.
I have since been messing with the system and discovered the problem is the clean DietPi install contains a /home/pi folder but no pi user. I found that deleting the /home/pi folder and then running the-script fixed the problems
Hope this is helpful to other new users (assuming it ever gets displayed.
Thank you, i was puzzled by this and threw the towel expecting a user/permissions nightmare without investigating. Serves me right that it was that easy to fix 🙂
First:
THANKS a million for “The Script” it was the only method that worked for me to get Mosquitto running. HOWEVER, nothing I seem to try gets Node-Red to connect to the Mosquitto broker.
I would not be bugging you except I have now spend 3 days trying to get Node-Red to connect to Mosquitto and am hopelessly lost. Given my 0.0001 level of expertise in Linux this is no surprise.
RPi 3: NOOBS_v2_4_5, loaded Stretch, updated, loaded “The Script” and a cup of coffee later, seemed everything is running. NO Mosquitto load errors [that’s a first].
I have 1 Sonoff flashed with ESPEasy, and used the information from videos by Csongor Varga to configure the Sonoff as “Sonoff_1”.
I used his supplied library @ https://flows.nodered.org/flow/5c8a19f0e862fd59457d3b8c2c906acc and only changed all references from Sonoff1 to Sonoff_1 and of course the IP address.
If I follow his instructions to the letter I always get a “disconnected” message in any MQTT node in Node-Red.
However, if I put something for an IP address that I know is wrong in “Edit MQTT-broker-node”, I get a “connecting” message but never connected.
My Sonoff [IP 192.168.1.101] “Controller IP” is set to 192.168.1.9 [RPi IP addr] Port 1883
In node-red I have tried “localhost:1883”, 127.0.0.1:1883, node-red@192.168.1.9:1883 and none of those work.
Putty:SSH into the RPi as I am clueless in Linux as to determining if anything is runnning I tried this.
pi@homeauto:~ $ mosquitto
1515598144: mosquitto version 1.4.10 (build date Mon, 29 May 2017 13:43:29 +0100) starting
1515598144: Using default config.
1515598144: Opening ipv4 listen socket on port 1883.
1515598144: Error: Address already in use
So that told me, I THINK, that Mosquitto is running.
During the script process anytime I was prompted for a login/password I used “homeauto” so hopefully that is what Mosquitto is expecting.
i’ve seen similar issues with mosquitto when starting from NOOBS… retry with a standard raspbian stretch, NO noobs, and let us know…
Thanks DrFragle, I have tried a manual load of Raspian/Stretch–>Node-Red & Mosquito, a NOOB –> Raspian/Stretch plus “The Script”, a Raspian/Jessie–> plus “The Script”, a Raspian/Stretch–> plus “The Script” and I still cannot get any indication that Node-Red is even aware that Mosquitto is running.
Even using a known running node configuration from Csongor Varga and only changing “Sonoff1” to “Sonoff_1” and changing the IP address to reflect the IP address of my RPi3 renders the same result “Disconnected”
After now 4 days, I am admitting defeat and will try to find another solution where I am not working for the software instead of the software working for me.
I can still greatly appreciate the huge effort and brilliance in creating “The Script” but it is obvious my skills as an 8 core assembly programmer are insufficient to this task.
Thanks again for your efforts.
don’t know what’s happening… is port 1883 available?
i usually put an inject node in an mqtt publish node, with some test topic, then a mqtt subscribe node to that topic, redirected to a debug node… always worked…
This would tend to tell me Mosquitto is running and listening on port 1883 but then again, I obviously know zero about Mosquitto.
pi@homeauto:~ $ mosquitto
1515598144: mosquitto version 1.4.10 (build date Mon, 29 May 2017 13:43:29 +0100) starting
1515598144: Using default config.
1515598144: Opening ipv4 listen socket on port 1883.
1515598144: Error: Address already in use
When it shows “disconnected” in Node-Red, does that mean Node-Read is not connected to Mosquitto or that Mosquitto is not connected to my device, in this case my single Sonoff which was flashed with ESPEasy?
These are my Sonoff settings.
Name: Sonoff_1
Admin Password:
SSID: xxxxxx
WPA Key:xxxxxxxxx
WPA AP Mode Key: configesp
Unit nr: 1
Protocol: OpenHab MQTT [have tried others]
Locate Controller:
Controller IP: 192.168.1.9
Controller Port: 1883
Controller User: homeauto
Controller Password: homeauto
Sensor Delay: 60
Sleep Mode: [not checked]
Optional Settings
ESP IP: 192.168.1.101
ESP GW: 192.168.1.1
ESP Subnet: 255.255.255.0
ESP DNS:
8.8.8.8
try using whatever mqtt client you know (for example: mqttlens or mqttbox, both are google chrome extensions), or via command line to check if it connects:
mosquitto_sub -h localhost -t “#” -u YOURUSER -P YOURPASSWORD
This is what was returned in Putty/SSH
mosquitto_sub -h localhost -t “#” -u admin -P homeauto
-56.00
-56.00
-55.00
-55.00
-55.00
-55.00
57.00
-58.00
-57.00
This looks like the WiFi signal strength from the Sonoff
Looks like the command you suggested is returning all the expected values to Mosquitto even though nothing appears on Node-Red. I added the device nomenclature in [xx]
-58.00 [Wifi]
0 [button]
0 [button]
-58.00 [Wifi]
0 [button]
0 [button]
21.00 [UpTime]
0 [button]
-63.00 [Wifi]
0 [button]
1 [button]
1 [button]
-63.00 [Wifi]
1 [button]
1 [button]
-59.00 [WiFi]
if you used the script, then the mqtt user will be admin, this user is created by the script, and the script prompts for the password right at the start.
Secondly you’re getting an adress already in use error when mqtt starts.Im not sure why this would be, seems strange that port 1883 is already in use. Unless there are 2 instances running.
you could try changing the port number to something else, and reconfigure node-red and the sonoff to use the different port.
sorry I just saw the updates, and I see mqtt is Ok.
I would still check the username youre using to connect, both in node-red and on the sonoff, as it should be admin, not homeauto.
Yeah that login was my typo, it is using admin.
Changing the port# in node-red gave me a lost connection error. Restoring it to 1883 removed the error but still disconnected in node-red.
Does, “disconnected” in node-red mean node-red is not connected to the server OR does it mean it cannot see the Sonoff?
It means disconnected from mqtt. Node -red only ever sees mqtt, as its a queue, It knows nothing of the sonoff, it just subscribes and publishes messages to the mqtt queue.
can you supply your mqtt connection and ssecutity settings, as they appear in node red
Ok, It is now working. But in all honesty I don’t know why. I have exported/saved the nodes so I can recreate this again.
Thanks for all who kept me going.
no, it’s just that mosquitto was already running, and running “mosquitto” cli command just tried to run it again and failed because there was already an active instance…
Hello I got exact same troubles. Newest version of everything.
Also new to Linux 7 days intence learning. I found soft off a temporary workaround. By killing the mosquitto and sudo restarting it.
I have admin in my mosquitto.passwords file.
Here is a screen log of how i restart mosquitto.
However same if restarting my rasp3b ther problem is there again. pi@pi:/etc $ sudo mosquitto -v
1550795747: mosquitto version 1.4.10 (build date Wed, 13 Feb 2019 00:45:38 +0000) starting
1550795747: Using default config.
1550795747: Opening ipv4 listen socket on port 1883.
1550795747: Error: Address already in use
pi@pi:/etc $ sudo mosquitto -d
pi@pi:/etc $ sudo ps -ef | grep mosquitto
mosquit+ 421 1 0 01:15 ? 00:00:00 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
pi 1549 1435 0 01:36 pts/0 00:00:00 grep –color=auto mosquitto
pi@pi:/etc $ kill 421
bash: kill: (421) – Operation not permitted
pi@pi:/etc $ sudo kill 421
pi@pi:/etc $ sudo mosquitto -v
1550795796: mosquitto version 1.4.10 (build date Wed, 13 Feb 2019 00:45:38 +0000) starting
1550795796: Using default config.
1550795796: Opening ipv4 listen socket on port 1883.
1550795796: Opening ipv6 listen socket on port 1883.
1550795800: New connection from 127.0.0.1 on port 1883.
1550795801: New client connected from 127.0.0.1 as mqtt_a0e9afaa.4abe5 (c1, k60, u’admin’).
1550795801: Sending CONNACK to mqtt_a0e9afaa.4abe5 (0, 0)
1550795801: Received SUBSCRIBE from mqtt_a0e9afaa.4abe5
1550795801: OfficeTemp (QoS 2)
1550795801: mqtt_a0e9afaa.4abe5 2 OfficeTemp
After a LOT of TIME spent trying to resolve this issue i have found a (the?) solution.
I used the chrome extention MQTTLens to figure it out.
My setup was a rasPi3 i set it up like in Andreas Speiss excellent video https://www.youtube.com/watch?v=JdV4x925au0
And i used a simple NodeMCU with an example of MQTT lib the difference between this example sketch and using MQTTLens was that in the MQTTLens you are forced to put in user and password. As soon as i implemented this in the arduino sketch it started to communicate.
The lib i used contained som dockumentation.
https://pubsubclient.knolleary.net/api.html
And so…when i ADDED “admin”,”admin” in this function problems was solved. Now the server and client can handle powerdowns and other restarts.
//MQTT
void reconnect() {
// Loop until we’re reconnected
while (!client.connected()) {
Serial.print(“Attempting MQTT connection…”);
// Create a random client ID
String clientId = “ESP8266Client-“;
clientId += String(random(0xffff), HEX);
// Attempt to connect
if (client.connect(clientId.c_str(),”admin”,”admin” )) {
Serial.println(“connected”);
// Once connected, publish an announcement…
client.publish(“outTopic”, “hello world”);
client.publish(“OfficeTemp”, teststring );
// … and resubscribe
client.subscribe(“inTopic”);
} else {
Serial.print(“failed, rc=”);
Serial.print(client.state());
Serial.println(” try again in 5 seconds”);
// Wait 5 seconds before retrying
delay(5000);
}
}
}
The mosquitto software in the raspi was not very helpfull to tell me that this problem simply was all about privileges.
put my harddisk to pc ran Paragon harddisk manager
I did a new build of raspian stretch on harddisk using Etcher
booted rpi3
ran the script again no errors
nodered dashboard now works
can only think was a corruption in a file or the harddisk
The script worked perfectly
sorry missed out that the reruns of the script naturally do not appear to overwrite what is uptodate in programmes
Hello
I have a problem after running the script on a rasppi 3.
The os is Raspian stretch and I have it set up booting from a hardisk 360GB
All appeared ok. I had a clean unused OS booting ok.
I ran the script removing wolfram, changing the user and pw for dashboard, and
the pw for nodered & MTTQ
I enabled Webmin
I saw no errors the script finished ok. Booted ok
Mqtt running ok
Nodered flow ok
put small flow in nodered, connected to mqqt ok
Opened dashboard prompted for user and pw entered both which I had noted
reaction was no autherisation.
I tried various combinations of pw etc
I have rerun the script 4 times noting exactly what I changed each time-
I have the same result, I can open nodered flows, mqqt, but not nodered dashboard
Anyone know of this issue or have a pointer to possible error I have made
I have in the simple test flow an inject timestamp to Text(ui) and debug. Debug gives out a valid timestamp
I am a newbie to nodered and Linux
Hey.. I found some more issues on the RPi Zero W (DietPi), e.g. with node-red I have one error when it tries to create the node-red directory
the-script: line 849: cd: .node-red: No such file or directory
and then later it tries to cd into that directory, which of course it can’t
the-script: line 872: cd: /home/pi/.node-red/: No such file or directory
I’m not complaining, more like a little wondering, whether it is as fool proof as I thought (read that), and yes, it is kind of fool proof, I just found it irritation that I run into these errors where in fact sudo is missing in front of the commands from time to time and whether this is a script issue, or an issue with my installation.
as always, publish somewhere full logs you have in your home directory
if you see script you’ll find, at line 847:
cd
mkdir .node-red
cd .node-red
so, it goes in your home folder, creates that new one and enters it
Why on earth that should fail… Are you sure your sd card is good, or is it mounted readonly? Did you enlarge the FS on first boot, and rebooted after?
Hey,
I tested this further, now on a VM, also for speed reasons, and there it worked fine. So seems there was something completely wrong with me user setup. Sorry for “wrong alarm”. I will check further
Hey, I just wanted to point out a potential issue I stumbled across.
I wanted to install the script on a Raspberry Pi Zero W (running DietPi) and ran into an issue with NodeJS. The NodeJS installation failed for me as the wget resulted in a Permission denied error. After utilizing the command again manually with sudo it worked. (I’m referring to line 798 of the script)
wget –no-check-certificate https://nodejs.org/dist/$LATESTNODE/node-$LATESTNODE-linux-armv6l.tar.xz -a $LOGFILE
Also I was curious, why you hardcoded the version 6.11.3 in the script when there are newer releases. on nodejs.org? I’m new with NodeJS so forgive me if this is a stupid question I’m asking here.
just couldn’t find a way to retrieve just “latest” version, as they use the “latest” word only in folder names and didn’t create the symlink to the “latest” file, as you can see: https://nodejs.org/dist/latest-v6.x/
From email – “Don’t use script with. Noobs. Not tested. Use full raspbian clean install”
So I did a new Raspbian install, no NOOBS, ran the script. And it all worked 🙂
Thank you.
oh, yeah, that too, i forgot about that…
I have tried The Script on a new Stretch install (not upgrade) and it kept choking. After some investigation I found that Mosquitto (MQTT server) will not install because of messed up dependencies. 🙁
please publish the logs you have in your /home/pi folder
I was mistaken. I thought it was Mosquitto but it seems to be sqlite. I had tried the-script a couple of few and it alway failed. So I started with a fresh image (NOOBS) and manually removed and installed packages. I got stuck on Mosquitto.
From internet searches I found that the install of Mosquito fails with Stretch, so I assumed my problem with the-script was Mosquitto, but looking through the log it did install with an older version.
My complete log is at https://gist.github.com/RudyFiero/e390a5b840177c27d527ab1910e52491
The last bit where it failed is this,
> sqlite3@3.1.13 install /home/pi/.node-red/node_modules/sqlite3
> node-pre-gyp install –fallback-to-build
node-pre-gyp ERR! Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v3.1.13/node-v48-linux-arm.tar.gz
node-pre-gyp ERR! Pre-built binaries not found for sqlite3@3.1.13 and node@6.12.2 (node-v48 ABI) (falling back to source compile with node-gyp)
make: Entering directory ‘/home/pi/.node-red/node_modules/sqlite3/build’
ACTION deps_sqlite3_gyp_action_before_build_target_unpack_sqlite_dep Release/obj/gen/sqlite-autoconf-3150000/sqlite3.c
TOUCH Release/obj.target/deps/action_before_build.stamp
CC(target) Release/obj.target/sqlite3/gen/sqlite-autoconf-3150000/sqlite3.o
I am not very familiar with Linux. (but with every problem learning more) The NOOBS image has node-red included under applications. After running the-script it is missing but it does work when run from a terminal window. It was the main reason I tried the-script. I have a older model B running Mosquitto on a DietPi install.
I hope you can get me past this bump. If not, I still appreciate any thoughts.
seems just a temporary error, try redoing this in a few days, or report directly to the sqlite3 node author, as there’s nothing wrong in script… it does “npm install” of the node, if the binary is not available it downloads sources and try to compile, if sources have problems script can’t fix them…
THANK YOU!!
GOD bless you!!!
It works perfectly – unless you try to start programs from /etc/rc.local!
Try running rc.local from the command line and you see the error.
Actually, since it errors, I don’t think it can be executing the chmod command anyway, so maybe this line could be deleted. (I think it’s only run for Raspbian)
the suggested mod is fine, Peter, you can modify that line, it’s safer this way…
p.s.: when i did that mod, there was a single exit 0 in the file, at the end…
Quote: ” Author: Antonio Fragola
Comment:
well, just don’t do it… rc.local is supposed to be the LAST started up script during boot up, not a script that you run every now and then… you want to run it as you do? Then know what you’re doing and how to deal with your issues…”
Thanks for that Antonio. It’s always a good idea to know what you are doing and how to deal with your issues.
I don’t recommend running /etc/rc.local from the command line EXCEPT in the circumstances where it is broken and you want to find the error.
sorry, i reread your comment and understood your issues, i modified my comment just 1 minute after publishing, but you read it anyway 🙂
my bad
Testing on a Raspberry Pi 3 this script breaks /etc/rc.local:
Before running it has
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will “exit 0” on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
# Print the IP address
_IP=$(hostname -I) || true
if [ “$_IP” ]; then
printf “My IP address is %s\n” “$_IP”
fi
exit 0
After it has
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will “chmod 777 /dev/ttyAMA0
exit 0” on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
# Print the IP address
_IP=$(hostname -I) || true
if [ “$_IP” ]; then
printf “My IP address is %s\n” “$_IP”
fi
chmod 777 /dev/ttyAMA0
exit 0
Your command on line 866
sudo sed -i -e ‘s#exit 0#chmod 777 /dev/ttyAMA0\nexit 0#g’ /etc/rc.local
matches two occurrences of exit 0, including one in the comments at the start.
Inserting a new line there results in a Bash error.
I suggest this instead to only match a line containing only “exit 0”
sudo sed -i -e ‘s#^exit 0$#chmod 777 /dev/ttyAMA0\nexit 0#g’ /etc/rc.local
By the way, why do you chmod 777 /dev/ttyAMAO?
Hi
I just ran the script 2 days ago on a Raspberry Pi 3 and everything worked first time – perfectly?
Re: AMA0 I probablyt had some issues with Node-Red accessing the serial port.
Hi Peter,
Firstly, Heartfelt gratitude for sharing this wonderful piece of script. Works smoothly for everything. However, I am facing trouble accessing the MQTT broker. I am trying the access the MQTT broker running on my Raspberry pi (raspbian stretch) from my Node MCU. The connection can’t be established. The Node MCU connects to the Wifi but returns an error (rc=5) connecting to the MQTT server. It says connection denied and I am not authorized to connect.
I need some help. May be this is not the right blog but a little pointer in the right direction would be great.
Thanks,
Aritra
Assuming you are using “the script” – MQTT needs a username and password. that will be “admin” and whatever you set up as the password in the script for “admin” user.
and in case of errors, try from pc, too, using one of the many mqtt clients: http://www.hivemq.com/blog/seven-best-mqtt-client-tools
Thank you so much. 🙂
As per update above – the script can now be downloaded as simply as, assuming root user initially in the root directory:
wget https://bitbucket.org/api/2.0/snippets/scargill/ekon6o/master/files/script.sh
give it execute permissions – go.
newbie user – how do i get to be root – su?
give it permissions? like 777?
Hi , I could not install the script in ubuntu-16.04.3-server VM.
This is the error:
E: Could not get lock /var/lib/dpkg/lock – open (11: Resource temporarily unavailable)
E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it? .Thank your help.
Marcelo
when this happens, system is autoupdating itself, so just wait… or did you run an other apt-get in other window and run the script while that was still running? Nothing to do with script, anyway
Peter and Mr Shark, Thank you for the ‘script’ and the set up of the VM, I used virtual box. I had one issue and that was when I went to install Ubuntu I could only see 32 bit files despite me having a 64 bit Windows 10 machine. This problem is caused by hyper-V being disabled in the BIOS. There’s quite a lot of information about that on ‘Google’ and once I found the correct location in the bios and enabled that functionality I could see the 64 bit files in the virtual box set. I think where Leo is getting confused, as was I, is that Peter’s excellent video describes the Virtual box install whereas the blog text describes the VMWARE version, which of course it states. In my case once I had the issue of the missing 64 bit files solved I followed the video and everything installed perfectly and after installation everything runs perfectly. I put a BigTimer flow on the virtual machine and used MQTT to send the data to MQTT-spy running on my Windows 10 desktop PC, in fact the same physical machine that also has the virtual machine. Windows and Linux in harmony!
Incidentally it 16 minutes to run the ‘script’ and load everything onto the virtual machine. A super job in getting the script to be so easy to use and so versatile. I am very new to Linux so it is a big help to install so much software easily.
Full walk through of a vm install in vmware, Peter please add to article
Now I can see how many wrong turns I took in my run on the VM installation. It is very nice and helpful. You have that so fast great work!
Took away the invalid option -g., next problem
>>>
root@Ubuntu-VM:/home/hoekbrwr# adduser –quiet –disabled-password –shell /bin/bash –home /home/pi –gecos “User” pi echo “pi:password” | chpasswd usermod pi sudo echo “pi ALL=(ALL) NOPASSWD: ALL” > /etc/sudoers.d/pi chmod 0440 /etc/sudoers.d/pi
adduser: Only one or two names allowed.
<<<
I think I already had to define the pi user during installation of the OS!
That’s it for today… Tomorrow more tries.
It is not as easy as it looks. All the details in the text are useless because with every other OS installed things look different to the blog text.
last comment, then i surrender: WHY ARE YOU RUNNING THOSE LINES BY HAND??? Instructions are CLEAR: login with whatever user you want, create a file, paste the script, save, give permissions, run it… if logged in user it’s not pi it will be created, if it’s pi it will be updated with needed permissions, then you have to login as pi and rerun the script and answer some VERY basic questions, PERIOD! DON’T OPEN THE SCRIPT, LEAVE IT THERE AS IF IT IS A BLACK BOX! RUN IT, NOTHING ELSE!
Sorry but this is still in the blog with a lot of other stuff not useful because this was for one specific system accurate. I will run The Script and then it should be OK.
no, the script is tested on many different system, and will work on a gui system (ubuntu for sure), too, if only it’s run as it’s advertised to be run…
You’ve lost me Leo – the script works well for a bunch of different systems – most of which are noted in the comments at the start. Don’t deviate from simply running the script – or we can’t help.
NOTHING BIG WRONG WITH THE SCRIPT! Rest assured it is almost flawless. unattended in the VM in 18 minutes on my 6 year old PC(a bit pimped SSD 8GB RAM)
I was following the instructions in the blog and it is not in sync with Ubuntu UI messages. When I start installing Ubuntu I am asked to make a user account and there I used my standard username. I think one should use pi as a user and set the pw at that point. Then the script will start without the extra round because there is a pi user. I tried this big command(found in your text) to create a pi user and I got all kind of strange errors. I am not a Linux guy and I tried to create the user with the User menu in the settings and that was not clear to me. Then Antonio got irritated by my complaints and I realised I had seen this earlier with another try that the script creates the pi user in the first round and then can be started and run to the end.
The text just above Bash script just did not fit for Ubuntu. I think it was written with another linux version when you first made the blog for VM installation. All things very confusing root, pi user admin(later on in the script)
Now I have run The Script and have a VM with Ubuntu.
How should I produce an image for my pi3 from that? Convert .vmdk to .iso? Can I boot it then?
You can’t… Flash latest raspbian, run script, done
I am lost. What is the purpose of running on a VM? I made a wrong conclusion it can be used to flashed to use as home controller on the tested hardware?
You did, indeed…
Nowhere is written that you can take a vm install and port on a SBC…
A vm is just an other option, for who doesn’t need a physical board…
I followed creation of pi. HOwever there is a problem!
>>>
root@Ubuntu-VM:/home/hoekbrwr# adduser –quiet –disabled-password –shell /bin/bash –home /home/pi –gecos “User” pi echo “pi:password” | chpasswd usermod pi -g sudo echo “pi ALL=(ALL) NOPASSWD: ALL” > /etc/sudoers.d/pi chmod 0440 /etc/sudoers.d/pi
chpasswd: invalid option — ‘g’
<<<
What is going on?
I have created pi user as administrator through menu, but cannot let it login automatically. Slow boat to China….
honestly, don’t know what the h… are you doing… why create the pi user from menu, if script creates it itself? and why not just creating it during install… and, repeat, customizing the graphic part of ubuntu is beyond this blog, there are TONS of tutorials online on how to do… why are you running those lines by hand and not by script? Restart from scratch, do a textual install, install the script, then do whatever you want with gui, not relevant for the script functionalities
I see you made an update for The Script.
As I think I am behind in updating my existing Pi3 and Raspbian Jessie I thought let’s try that VM method to create a new image with the latest version.
Then the question what OS to use! I got lost. You sort of advertised Raspbian Stretch, but when I read the comments of your masked friend that is not latest and best technology to use ?????????????? A little advice would be appreciated here.
No advertising – simply mentioned that it exists (and hence WILL be with us) and so we modified the script accordingly. Which one you use is up to you. I’m upgrading my PIs. I just did the install on a Pi3 – had to modify a separate Grafana setup to get it to work but most other stuff seems just fine. Backed up the SD and now I have a Pi2 working on the same setup.
masked friend, lol 😀
well, ubuntu is more up to date, so in vm you can use ubuntu xenial or just standard debian 9, and let us know what you get… i prefer xenial, everytime 🙂
So at last xenial is a little advice! So I take the desktop version as I am a linux illiterate. So I got this one: ubuntu-16.04.3-desktop-amd64.iso
Sorry to say. In VMWare Workstation 12 I am stuck at the globe asking for timezone. The flatted globe does not fit into my little window “Where are you”. I only see a part of a unactivated button and that’s where I am no further actions possible ESC ctrl-c Tab nothing! What is the trick to get out of that situation? see also png
There is a message which asks for Install Tools and that seems the only way out circumventing all following settings.
nothing to do with script, but try pressing enter where it says amsterdam, or press tab+enter… sorry, don’t know, use larger windows… or do the textual setup when it asks at beginning… you’ll still have graphic once finished…
https://tutorials.ubuntu.com/tutorial/tutorial-install-ubuntu-desktop#7
Cannot resize window. Something to do with display adapters not yet initialised. I must update Tools. I will start from scratch again. I took away 3D feature.
Interesting..
http://linux-sunxi.org/Xunlong_Orange_Pi_Zero
This states that PA17 is attached to a red light on the Pi Zero… not on mine it isn’t… at least, I can’t control it.
look at tkaiser commen, #14: there are various pcb revisions, newer seem to suffer less the heat, too
http://www.cnx-software.com/2017/07/30/how-to-setup-an-orange-pi-zero-diy-smart-speaker-with-google-assistant-sdk/#comment-544681
Hi there
“THE Script” seems to get stuck at Installing Node-Red
Should I wait for the odd 26days to pass, or is there something else I may try, pls?
This is the link to the logfile.
https://www.dropbox.com/s/8d5ibrj9v5i0hum/scargill.sh-2017-07-05_12h07m.log?dl=0
Kind regards
problems with github, not script… unfortunately their files are served in a round-robin fashion, or the like, and sometimes you end up in a server that’s not aligned with the others and have missing files… that’s what happens with that bcrypt file in your logs… i tried about 10 times to go to that url, 2 times i downloaded the file, 8 time i got a 404 page (not existing url)… so, just cross finger and retry…
I thank you Dr….
How do I then “reset” to start flawlessly from the top? eg, does not complain about directories, etc?
Is there a way to know I have d/l the GOOD script?
I will keep trying.
Again, thank you.
Rerun script selecting what’s missing… Script is good, but the parts it downloads may be broken…
Thank you!!
I understand, will confirm.
Kind regards
All seems fine!
Thank you!
Hi Peter,
Great video & script.
Just a suggestion,
In RASPBIAN why do not install PHP7?
Other for raspicam… http://elinux.org/RPi-Cam-Web-Interface#Internal_Motion
Because 7 gave some other boards issues – and because I had no particular use for 7. Previous versions work just fine for the likes of PHPLiteAdmin etc… nothing to stop anyone upgrading…
I modified your script and PHP 7 works fine.
Excellent, well if you send in the mod and let us know under what circumstances it works – we can add that as an option…
This is a website made for my son:
https://www.casaempuja.com/index.php/how-is-made-this-web/
As you can see is a demonstration that in Rasberry work together Apache2, MariaDb, PHP7 and WordPress …
Hi
Well it’s probably very nice – but twice I tried – the first time nothing – the second time looked promising but no data… the point of this was? We know all of those work together? Or were you just proud of the site?
Yes, they all work together.
But if I had to do it now, I would develop it in another way.
Probably with NGIX and skipping all the graphics.
You know, putting an ssh file in the boot before booting and doing it from the beginning via ssh.
Surely it is how you end up doing it.
By the way, I saw on your profile that you spent some time in Spain.
Do you understand Spanish?
I develop in English with difficulties.
I am indeed in Spain right now and for the rest of the summer – that is if we don’t all die from the heatwave. Sorry can’t help you with Spanish as my Spanish language is atrocious.
because raspbian is just… OLD… still based on Jessie, and as far as debian is done (and raspbian on it), they have a release time of about 3 years… Stretch is out, let’s hope in soon adoption, and support for the 64 bits specific of the rpi3… if you want php7 (which is faster than 5, but in our scenario is used only for phpliteadmin and little more, nothing vital to a iot setup), then go for an ubuntu 16.04 distro like armbian or dietpi, i think
You could say it is SO 2014 !
Ran the new and improved, experimental script last night and it ran without a hitch. Ran it over Real VNC, 2 hours and 19 minutes later, it was done.
Nicely put together.
Excellent – well I just ran it on a NEO 2 with DietPi and I think the only thing that failed was Mosquitto – will tackle that later today with Antonio.
unfortunately that distro is a mix of dietpi on top of armbian based on jessie, a bit of a mess… mosquitto is on version 1.3.4 so the problems in it not starting were related to websockets, in the end… other problem is that the mosquitto_passwd file works only interactively, so no scripting for that part… i could add a switch to check this specific version on this specific board, but is it worth it? If someone wants to go this route and use this dietpi, wait for the updated version Peter will publish and then you have to comment out the websockets part completely, so you end up with mosquitto passwordless but at least starting… too much of a hassle for now…
Problems with the script!
I downloaded the latest Raspbian and latest script and run into a big number of problems!
First I was wrong with the answer to give the PI3 the name of myboard. There is something wrong with the name setting. I did not want to change the name to myboard. I ended up with an empty name field in the preferences. OK I wanted the name to be raspberrypi and changed that after installing Raspbian. Also chosing the options in the menu is confusing there is no mentioning how to choose options other than the hint on top of the menu to choose(how???) I made the mistake by pushing a return after the namesetting to and the script started running, which was not the intention. Choose with arrow keys and Tab or space , unclear especially when you are not doing this regularly.
I get a lot of messages about email system not being alright, probably nothing because I do not use this but annoying!
I was a bit attending the flow of the script and I have seen an error when installing Webmin. As a result I cannot login to Webmin. Can I install it afterwards?
I backed up the files in .node-red from my flows and the database from my MySQLite data, but no flows appear in node-red. Then i tried stopping and starting node-red by node-red stop and node-red start but this gives errors because of a faulty package.json. I did that by overwriting it with the backup files . I only need to set back floss_*.json or also flows_*_cred.json?
I cannot login to my Root account although I have given exact answers when asking for my username and passwords in the script.
It does not seem to be so easy as the first time I ran the script on my Pi3.
Have there been introduced problems because of all adaptions for other than Pi3(/2?)
Are there any logs of the script to see if there were errors, I have no idea?
Repeating the procedure is the best option I think!
Situation update: 2nd time I did it alright and updated the ROOT an pi passwords which I unintentionally skipped the first time. Importing flows through clipboard by copying the JSON text from the flow* file. Only fill in username and password for the broker and we are going again.
odd and even are there of course.
space to select/deselect options, enter to confirm, easy peasy, you’re the first to complain about this… webmin complained about missing user and password, probably… if you screw up things, don’t try to patch a dead system, just restart flashing a clean distro… and if using latest testing version, you’ll find logs in the same folder of the script…
More details:
+ echo ‘Dpkg::Progress-Fancy “1”;’
+ sudo tee /etc/apt/apt.conf.d/99progressbar
+ echo ‘APT::Color “1”;’
+ sudo tee -a /etc/apt/apt.conf.d/99progressbar
+ username=user
+ userpass=password123
+ adminname=admin
+ adminpass=password123
+ newhostname=myboard
+ SECONDS=0
+ [[ RASPBIAN == *\R\A\S\P\B\I\A\N* ]]
++ whiptail –title ‘Main Raspberry Pi Selection’ –checklist ‘\nSelect items fo r your Pi as required then hit OK’ 30 73 23 quiet ‘Quiet(er) install – untick fo r lots of info ‘ ON prereq ‘Install general pre-requisites ‘ ON phone ‘Install o n Android Smartphone – see blog’ OFF mosquitto ‘Install Mosquitto’ ON apache ‘In stall Apache/PHP/SQLITE + PHPLITEADMIN ‘ ON nodejs ‘Install NodeJS’ ON nodered ‘ Install Node-Red’ ON webmin ‘Install Webmin’ ON screen ‘Install Screen’ ON java ‘Update Java’ ON wiringpi ‘Wiring Pi for the GPIO utility’ OFF mpg123 ‘Install M PG123’ ON modpass ‘Mod USER and ADMIN passwords (password123)’ ON phpsysinfo ‘In stall PHPSYSYINFO’ ON upgradenpm ‘Upgrade NPM to latest version ‘ OFF addindex ‘ Add an index page and some CSS’ ON passwords ‘Update ROOT and PI user passwords’ OFF installcu ‘Install CU for serial VT100 Terminal’ ON installmc ‘Install MC+M CEDIT file manager + editor ‘ ON installjed ‘Install JED file editor’ OFF habri dge ‘Install HA-bridge on port 82’ OFF log2ram ‘Install Log2RAM default 40Meg’ O FF wolfram ‘Remove Wolfram on a PI to save space’ OFF office ‘Remove LibreOffice on PI to save space’ OFF
+ MYMENU=
+ [[ ” != *\q\u\i\e\t* ]]
+ AQUIET=
+ NQUIET=
+ [[ ” == ” ]]
+ whiptail –title ‘Installation Aborted’ –msgbox ‘Cancelled as requested.’ 8 7 8
./script.sh: line 457: 2052 Killed whiptail –title “Installat ion Aborted” –msgbox “Cancelled as requested.” 8 78
+ exit
don’t have the smallest idea of what you did… lot of what you posted is not even in the script, sure not in latest… nowhere in script are comparison towards strings with all those asterisks… and raspbian is for original raspberry, not orange…
slashes, not asterisks, sorry
The first post is the normal output, second one is debug output of bash script… The slashes are added by debug in the output.
Raspbian is Raspbian server version downloaded from orange pi zero pages.
we never tried that version, xunlong images are sh!t…
use the version we suggest, dietpi or armbian
Will try armbian…
Now it works
well done 🙂
Raspbian server… orange pi zero (ssh login)… error:
[ ok ] Restarting ssh (via systemctl): ssh.service.
./script.sh: line 457: 1577 Killed whiptail –title “Installation Aborted” –msgbox “Cancelled as requested.” 8 78
pi@OrangePI:~$ /script.sh: line 457: 1577 Killed
Guys ( Mr. Scargill and all those that helped) Thank you all for this effort. And DrFragle and MrShark – thank you for the mention of DietPi – to this point my only choices were Raspbian (nothing wrong with it) or more efforts and NOOB..
I’ve just put DietPi on a Raspberry Pi 2 B with minimal additional software, then ran “The Script” as it was four days ago. And tada! working system.
The note about dos2unix above was a big help, too. Glad I learned to read a bit before jumping in and asking questions.
I’ve always gotten to a point (Pi loaded and ready to go) and then wondered where do I go from here? Well, I’ve got a better idea, now. And a couple Arduinos, a couple of NodeMCUs, and a box of sensors, so I think I might get somewhere, finally.
One, incompletely informed question – I see WEMO boards mentioned here…would those be any relation to the WEMO switches? Which means I could control the outdoor floods from my Pi?
Now, where did I put my log in credentials……
Wemo – Wemo D1 – check them out on Ebay or aliExpress – just a cheap, very small form factor ESP-12-based board, I bought a couple – there are also some header boards available like a relay board… so nothing special except they are a nice small size and inexpensive.
maybe it was just a problem with actual naming, as the esp8266 boards are called wemoS, not as plural of wemo but by themselves 🙂
the full name of the small board is Wemos D1 Mini, as there are also the d1, the d1 r2 (both arduino uno form factor) and the d1 pro and the d1 lite, same as the mini but with more memory the former, and a different chip the latter
Found/recreated those pesky login credentials. The WeMo Switch is actually a Belkin product – and apparently an American thing – according to Amazon, not available for shipment over seas. So I will have to do bit more digging on this side of the “pond”. I’ve had the switch in place for a couple of years for those evenings when I get home after dark. Use the phone to turn on the outside lights – get inside. Use phone to turn off lights.
There are a few how-tos that mention using an ESP8266 (or similar) and a relay to recreate the same functionality.
Again, thanks to all y’all that have contributed.
Oh WEMO – as in working with ALEXA – easy – there are a couple of software packages that do that – and if I am not mistaken the new software that ITEAD are putting in their SONOFFS does the same thing now.
This is probably the best replacement software (tested by me)
https://github.com/arendst/Sonoff-Tasmota
then there is this (untested by me)
https://community.home-assistant.io/t/sonoff-wemo-emulator/11001
And the ITEAD link where they talk about their product working with Amazon
https://www.itead.cc/sonoff-work-with-amazon-alexa
i think Peter uses HaBridge with esp8266 boards to have similar functionalities of the WEMO devices, he will confirm (or not) 🙂
Oh yes, one of the options in my script – HABRIDGE – and that can output MQTT and hence you can then talk to anything that handles MQTT.
Peter,
OK weird one here – did a Virtual Machine (Vmware Workstation 12 Pro) deployment of The Script – using the Ubuntu Server ISO as per instructions.
All worked fine – no issues whatsoever. Assigned a static IP address to the Virtual Machine and can login through SSH with no issues.
Node-Red comes up fine – however whenever i try and connect nodes to it i am gettting errors. Basically on the canvas all the MQTT nodes have a disconnected underneath them.
I exported some nodes from another Virtual machine i have been playing with (using Centos and my own manual install of most things) imported them into the canvas and the same thing happens.
if i change my node (it is an Arduino Mega with ethernet shield) to point from my Test-VM (where it is sending status messages and receiving Time Updates) to the new VM where the Script is i have all the nodes as disconnected.
I then tried MQQT-SPY and tried to connect to the topics – no good (but works fine for my other test VM)
I thought maybe a problems with IPTables/Firewall so i shutdown the UFW on Ubuntu but still the same issue – any ideas – something weird with non-standard port numbers, authentication etc ?
Housekeeping for The Script
Peter, Just downloaded the script – in the initial Menu once it is running – there is a typo in the option to Update NPM to latest – it says NPN
Otherwise looks good.
In your instructions in relation to VMware – you might also want to give some info on how to install VMware Tools into the VM
Hi – thanks for that – will fix it – be VERY wary about updating NPM to the latest version – I could not get Node-Red to start with it. I saw a comment in the Node-Red Google forums that it DOES – but isn’t the recommended option. That why I un-ticked that option by default.
Ok, script is fixed 🙂
Yep tried that with the NPM upgrade – took a snapshot first just to be safe !!
I love Virtual Machines !
Craig
This works on the CHIP! I like that I don’t have to worry about it too much with a 2500 mAh battery connected. The only challenge I had was that node-red won’t accept my login credentials from Safari. I kept thinking I had entered the password wrong and re-ran the script a few times till I figured it out. Chrome worked fine.
Any cautions with using the CHIP as it has no removable (i.e. easily replaceable) storage?
Excellent – well, the latest version uses log2ram so while it will use up some of your RAM it may improve long term life of the FLASH… As that has a battery you might be able to change what I think we set as an hourly cron job to a daily cron job and keep the writing down even further.
yes, i tested on the chip a while ago, i think about christmas, during the script rewrite we did that time… veeeeery slow, i think as the rpi1 and the rpi0/w… the good, it has an emmc… the bad, it’s only 4gb and slow, that, too… but sure for a device of 9$ with embedded storage, is quite good, have to test it’s gpio part, it has a good support and documentation on its site… what you did on the nas can sure be replicated on this little monster…
p.s.: actually the original log2ram script flushes its caches every hour, you changed to every day for yourself 🙂
I read somewhere you said your houses are connected, are your pi servers clustered? I’m not sure if that’s the right word though, but basically that they are redundant and that mqtt nodes can failover and publish to any of them?
Also, thank you for this resource. It’s amazing the amount of work you’ve put into this, and the result is outstanding. I’m working on a design for an open source water treatment controller, something that could handle a fish tank, a hydroponic system, or wastewater, and all of this is a big help with prototyping the control and monitoring side.
They are connected – each has it’s own MQTT broker and they share information. It is also a handy way to know if something has gone wrong because either can email me if the other is not responding. They are not clustered however. Right now my ESP code can handle two WIFI access points but only one MQTT server. That could get quite tricky – if one of them went off on a tangent and others didn’t.
AS Robin Williams said …. glad one can be of service 🙂
The script took approximately 12 hours to run on a Raspberry Pi model B1, but in the end it did finish!
I found two things that I would like to note:
1- the rc.local file gets mangled up somewhat, there are two lines on the script with a sed command to change “exit 0” to something else, and one of the two changes “exit 0” to “chmod 777 /dev/ttyAMA0 exit 0” in the comments in the top part of the script, and the line break gets mangled too and thus the rc.local file fails on startup.
I believe this line is the culprit:
sudo sed -i -e ‘s#exit 0#chmod 777 /dev/ttyAMA0\nexit 0#g’ /etc/rc.local
2- I appreciate that IPv6 is still not very much used, but I do not understand why you are shutting it down for good in the script, both in /etc/sysctl.conf and in /etc/avahi/avahi-daemon.conf
I use IPv6 and had to remove those lines to regain IPv6 access to my Pi. I noticed as I did start the script from an IPv6 SSH connection to my Pi, and as soon as I started the script, logging stopped as I killed my own SSH connection! 😀
A great script, which helps immensely on starting up with a more IoT-oriented software load on the Pi.
Many thanks, Peter!
Luca Bertagnolio
Hi there Luca
Yes the old Pi boards really are not up to it – I think I threw my last one in the bin. Of interest, comparing your 12 hours – Ununtu running in emulation on a PC takes about 15 minutes – and I think the Pi3 takes way less than an hour. Much depends on broadband speed of course but basically the original Pis are now showing their age.
That AMAO mod I’ll ask Antonio to take a look at – that’s a recent change – as for IPV6 -my take on this is that recently it has started to cause issues with people’s routers being unable to resolve the addresses so the simplest solution was just to disable it – in the future we’ll take another look at that. I am assuming those who use IPV6 specifically remain for now a minority – sorry it cave you problems.
Hi – (and thanks for the work you’ve put into this and for publishing…)
I’ve just run the script on a fresh installation (stretch lite) on a raspberry pi B +
and the only problem was that rc.local is still being mangled. Simple fix to edit but thought I’d let you know.
BTW – script took 2hrs 17mins.
I am again having a PI3-system up and running. As i got the feeling I ‘ll have to make a better supply,instgnated by your remarks and your newest blog entry, I gathered some components together and made my own UPS. I already had a lipo charger, a 1500mAh 3.7V, a buck converter up to 5V with those little V-meters on the lipo and the output. That whole thing was triggered by the misery I contracted for some reason and a bad supply could have been the reason. I only miss the elegant switching off when the battery goes really low by a power outage that stays out too long. I only have to really test how long it will last without mains, but this stabilizes the voltage much better. I found out the difference between the voltage a the micro USB differs about 0.2 V compared to the pin header value. I can reduce that when I connect my little 2,5 inch HDD on the USB which has extra power from another charger externally.
I estimate this kind of supply is not a gadget for unstable mains, but also when using a standard charger a necessity.
When I was running the script I have seen quite a number of warnings. Those might be ignored, but I am wondering if there is a log of all those messages running on the screen or did I miss a check somewhere in the starting menu?
I did something wrong when asked for the HOST name. There another name is suggested “myboard” and it says something like if you do not want a change just press OK. However the HOST name is changed to myboard from raspberrypi. I once renamed my HOST also and that gave all kind of permission problems so I will not touch that.
For some reason it took about 1 3/4 hour to finish, but the system is functioning at the same slow pace it did before I forget about that.
I will start working on a better back-up strategy. At the moment I will start with a copy of the SD card with the SD card copier which is standard in the raspbian jessie firmware. That should bring my up and running again much much faster than doing it from scratch.
There are warnings Leo but that’s what they are – warnings – of issues in other people’s scripts – over which of course we have no control – usually they are to do with unused variables – why on earth those are not turned off is beyond me.
On power supply switch off – see my latest blog – I’ve just hooked up a cheap OLED to my NANO and this can now be added to a supply to make it shut down the Pi or whatever gracefully and recover when the power recovers. All working.
No there is no log – but by all means make one…
If you don’t change myboard it will be myboard – change it to raspberrypi if you want to call your board raspberrypi.
The script takes anywhere from 7 minutes on a fast PC with Virtual Linux – to several hours on the lamentably slow Raspberry Pi Zero WIFI.. on the NEO 2here I think about half an hour or so – I’m quite getting to like this little board, I’ll like it better when Friendlyarm get around to releasing proper GPIO code and the ability to use more than one of the four UARTS.
I have read your latest blogs about uninterruptable supplies and GPIO, but that is , at least for me, shooting with a canon on a mosquitto(lol!). A whole new field of knowledge. I understand that if not running down the PI3 gracefully it might wrecken the SD card software in a lot of cases. I am surprised why your unit in Spain has never died when you say there is a horrible mains(solar supply?).
I have various backup solutions!
Thanks for this great script Peter!
I just installed an Orange Pi PC+ with it and it worked like a charm!
Excellent – enjoy – well it’s not just me – I’ve had lots of help from Antonio – and originally when I was REALLY rubbish at scripting, my friend Aidan Ruff was involved and we’ve had input and feedback from all sorts of people. I think I probably hold the record for testing hours 🙂
Thanks Peter, This is good information for beginner like me.
I just run the script from fresh install dietpi to my old raspberry Pi.
It seems Nodejs should be change to the latest node-v6.10.3
Done – well spotted – updated.
Thanks for your swift answer, but I do not understand the process you suggest. I have a filled otherthaniot.db file in the same directory. I do not have a database in SQLite yet. I like to have a database with the same name as the file, restoring the old situation.
sqlite databases are just files, if in your node-red flows you point to iot.db, for example, then rename your old file, containing your data, as iot.db, and restart your device
My database can receive new data, but phpliteadmin is not showing the records in the database! When I create the database in phpliteadmin with the name of the file in ~/dbs then I get an unknown error. That can be expected, but how to synchronize phpliteadmin to the file in ~/dbs??? B.t.w new records are created in the database from my restored flow, I see that on my dashboard graphics! So SQLite is functioning.
check /var/www/html/phpliteadmin/phpliteadmin.config.php
I will study that!
All kind of wrong permissions needed to change. Now database is accessible again! Other problems coming up like MQTT broker does not start anymore after reboot????????????
How to start the broker and how to set that it starts automatically at reboot?
honestly… you’re having too much problems with an evidently corrupted installation… save what you need and start from scratch… easier… or use a different sdcard in case forgetting something back…
Sigh, you are right!
I agree with Antonio – this is sounding like a corrupt installation – maybe SD issues, maybe power – but I would not be continuing – you’ll spend more time trying to fix than you would starting from scratch – check power – check that the SD card is a good one.
Sandisk ultra 16 GB from local seller. Supply between 4.98-5.08. How to check SD card?
read the blog post about sd cards and power supplies, no need to repeat, very well explained there…
already found it!
Now I only have to copy back the database with my gathered data into SQLite, I have a file in the same directory as where the iot.db is. How to import that file into SQLite?
rename the empty iot.db file to something else, and the full one to iot.db, restart
My pi3 nodered just suddenly died. It worked for months and now I cannot start node-red anymore, all kind of error messages with no clue what is wrong. I did an upgrade of Raspbian and it only gave worse error messages with Syntax error!
Is it possible to reinstall node-red and then run part of the Script to reinstall all extra nodes or… is a complete reinstall necessary?
I am now trying to reinstall from backup image from about 5 days back, but that is a separate excercise.
i think you can still recover your flows from the .node-red folder
about reinstalling, you can just rename .node-red folder and redo just all the npm part, without actually reinstalling node-red itself
I have done a new install ran the Script. I backed up the flows files in .nodered directory. I copied them back to their original location and nodered does not “see” them, stopped and started nodered. Any idea how I can activate my flows? There is a function import, but I only see a map example and cannot freely choose the backed up file!
OK I got them back by copying the contents of the .json file from a text editor to the clipboard and importing them from there.
Don’t forget the flows must have the same name as the domain name you’ve given the new setup.
Hi Pete,
Love your work.
The script worked like a charm.
It took over 10 hours to install on a Pi Zero…
I started on a similar project a couple of years ago with nodejs,
never got very far with it because of time issues.
Recently i wanted to start it up again and then found node-red.
Love it.. has a huge potential. Now i’ll start over with node-red.
Also like what you’re doing with the esp’s too.
Did a small test project with them, and to keep it small i use
the 128×64 i2c oled display’s for status and debuging. I use them on almost anything, they’ re so cheap these days.
I have question about your system:
Why do you use imperiHome on your phone?
You could do al that with the dasboard in node-red.
keep it going,
may thanks,
Kiki
Hi there Kiki – thanks for the kind words. Re: Imperihome – I think it depends on what order you read the blog – I am indeed now favouring the Dashboard over Imperihome. However….. there is one little niggle with Dashboard that as far as I know has never been resolved… if you only use Dashboard then the inputs and outputs stay in sync with your phone but imagine you have a timer running in the background – changing the state of things – it is sometimes necessary to refresh the display to make sure it is up to date. In other words I push changes regularly when the phone which is running Dashboard is operating…. and therein lies the issue – the password system – and I’ve mentioned this in the forums as have others… there is something not quite right about the security/password setup in that it regularly asks for the username and password when you go to the dashboard (only on Android phones, never on my Windows 10 PC) – and that’s a pain but not TOO bad.. BUT if you have any kind of refresh going as I do – you get half way through entering the password and the screen refreshes – indeed to use the dashboard reliably I’ve had to stop refreshing the screen as such…. the thing is – it should not be asking for username and password once you’ve stored it – but it does from time to time. That is currently unresolved… But generally speaking yes I’m finding it better to work with than Imperihome – which STILL is missing some generic controls – they seem intent more on supporting every device on the planet than providing generic controls that we can use in Node-Red – hence you’ll see I spent some time working with templates in Dashboard and various gauges etc..
Pi Zero – yes, very nice but WAY too slow and the script only serves to remind us how slow it can be. I must say – even with the outstanding issues (ease of use of GPIO) I’m favouring the FriendlyArm boards such as NEO etc… slightly more expensive but WAY more powerful and they are a helpful lot.
Hi Pete,
As I found out that “G P” is right:
“The buggy line was “log2ram”:
“Install Log2RAM default 40Meg” OFF \ ”
Behind the \ there was a TAB and then LF.
I also removed the tab to get it to work.
Thx.
Nadav
Hi Pete!
Until now, I have used only parts from your script. But now I tried to run it complete, and there was no menu displayed.
The buggy line was
“log2ram” “Install Log2RAM default 40Meg” OFF \
Behind the \ there was a TAB and then LF. I had to remove the tab to get it to work.
It is highly likely your diagnosis is correct as this was a brand new addition to the script. I have loaded up the script -ensured that it is a space before the backslash and saved – so hopefully others will not come across the problem.
Thanks for bringing this to my attention!!!
Ref ‘The Script’ & mosquitto:
I think mosquitto needs to have /var/log/mosquitto created and ‘chown’ed to mosquitto:mosquitto in order to log…
In my quest for minimum writes, I set it’s persistance path there, and then wondered ‘why no logs? why no db?’. only when I created the folder and ‘sudo chown mosquitto:mosquitto /var/log/mosquitto’ did a restart cause logs + db to be present.
Can someone check on another system to see if you have mosquitto logs, and if not, we may want to add this to ‘The Script’?
dietpi, in vm: log2ram active, mosquitto folder there and with correct owner…
ls -l /var/log/mosquitto/
-rwxrwxr-x 1 mosquitto mosquitto 2852 Apr 26 19:37 mosquitto.log
Mosquitto works perfectly with the script as is. I use it on all my installations.
must have been something about the state the Armbian OS was in when I installed – maybe *my* (Armbian pre-installed) log2ram did/does not correctly copy to log.hdd on shutdown, and after The Script install, I rebooted before the hourly copy….. mosquitto always worked, it just did not log…. Feels like the same non-copy bug was responsible for Apache not running too. Something to highlight in the main post maybe.
‘In at least one installation where log2ram is pre-installed (e.g. Armbian), it has been observed that folders created in /var/log/ by the installations may not get copied to the /var/log.hdd/ folder at reboot, resulting in services failing or misbehaving. This can be avoided by a manual copy after installation or by waiting an hour and checking /var/log.hdd/ matches /var/log/’.
Or we could add a ‘sudo log2ram write’ to the end of the script, which will (quietly?) fail if log2ram is not present, but solve the issue if it is?
It may just have been some one-off…..
Open an issue on the log2ram github if you have problems (and the last version of course).
Check you starting log, you will have error from systemd for the log2ram service. And there are the file /var/log.hdd/log2ram.log file to watch too.
Is there any reason why the script doesn’t install fail2ban to deter hackers?
And a related question, why do you choose to enable the root login on a Raspberry Pi?
I do occasionally see strangers attempting to login as pi but nowhere near the number who try as root.
because in general we use this setup locally… i don’t enable root on my systems, but others prefer using it, maybe just “habits”… i use vpn to connect to my services from outside… anyway, this is just an easy setup of a bunch of services, suited for most but not everybody, and whomever can modify their setup and make them suited to their needs… feel free to close everything, disable ssh root login, add ssh keypair login instead of user and pass, add fail2ban, etc
The script does not do a lot of things. I’ve never heard of fail2ban and I don’t add anything I’m not familiar with and feel comfortable with. Root? Because here are some things that are a pain in the backside to deal with, without Root. Generally speaking I run everything as user Pi.
Fail2ban:
If you open SSH to the internet on the default port, you very soon attract login attempts, lots of them.
grep “Failed password” /var/log/auth.log to see how popular this is with the hackers.
All fail2ban does is ban an IP address for 10 minutes after 6 attempts. Of course it’s configurable, that’s the default.
(On the Pi) sudo apt-get install and it starts working, no setup needed.
I’d also suggest that logins and passwords for node red, webmin etc should be defined at the top of the script rather than buried in the code.
Best regards,
well, users and passwords ARE defined at beginning of the script… and, again, fail2ban is useful only if you publish your setup on the internet, something most of us DON’T do…
Passwords deined at the beginning…
Lines 357 – 363 in the script:
username=”user”
userpass=”password123″
adminname=”admin”
adminpass=”password123″
newhostname=”myboard”
Only useful if you…
Yes that’s true. It’s just a suggestion for a simple enhancement.
question isn’t where password are (they’re at qute the beginning of the MAIN SECTION OF THE SCRIPT, the first part are general functions never touched and that a while ago were in a separate file, now included just for easier management), question is if you ever run the script or are questioning just for the sake of it… if you ever run it, you should know that there’s nothing you need to edit yourself, and that running it there’s an option in menu that uses THOSE defaults and asks for new ones, JUST AT THE BEGINNING OF THE INSTALL process… after those first seconds, you can go to take a coffee or what else, and return after about 15-30 minutes, depending from your board speed… and now, enough…
I was asking because i have on occasion recommended your script on the Raspberry Pi forums and I wondered why you did/did not do a few things that slightly worry me – including enabling root login and not installing fail2ban. But please don’t think I’m criticising your work, just asking questions.
Yes I do run it sometimes. I have never previously done so without modifications to suit my preferences.
However, as an experiment I just tried running the script unedited on a headless Raspberry Pi 3 with a fresh copy of Raspbian 2017-04-10 to which i added /boot/ssh and /boot/wpa_supplicant.conf.
It fell over at this point:
++ whiptail –title ‘Main Raspberry Pi Selection’ –checklist ‘\nSelect items for your Pi as required then hit OK’ 30 73 23 quiet ‘Quiet(er) install – untick for lots of info ‘ ON prereq ‘Install general pre-requisites ‘ ON phone ‘Install on Android Smartphone – see blog’ OFF mosquitto ‘Install Mosquitto’ ON apache ‘Install Apache/PHP/SQLITE + PHPLITEADMIN ‘ ON nodejs ‘Install NodeJS’ ON nodered ‘Install Node-Red’ ON webmin ‘Install Webmin’ ON screen ‘Install Screen’ ON java ‘Update Java’ ON wiringpi ‘Wiring Pi for the GPIO utility’ OFF mpg123 ‘Install MPG123’ ON modpass ‘Mod USER and ADMIN passwords (password123)’ ON phpsysinfo ‘Install PHPSYSYINFO’ ON upgradenpm ‘Upgrade NPM to latest version ‘ ON addindex ‘Add an index page and some CSS’ ON passwords ‘Update ROOT and PI user passwords’ OFF installcu ‘Install CU for serial VT100 Terminal’ ON installmc ‘Install MC+MCEDIT file manager + editor ‘ ON installjed ‘Install JED file editor’ OFF habridge ‘Install HA-bridge on port 82’ OFF log2ram ‘Install Log2RAM default 40Meg’ OFF ‘ ‘
Box options:
–msgbox
–yesno
–infobox
–inputbox [init]
–passwordbox [init]
–textbox
–menu [tag item] …
–checklist [tag item status]…
–radiolist [tag item status]…
–gauge
Options: (depend on box-option)
–clear clear screen on exit
–defaultno default no button
–default-item set default string
–fb, –fullbuttons use full buttons
–nocancel no cancel button
–yes-button set text of yes button
–no-button set text of no button
–ok-button set text of ok button
–cancel-button set text of cancel button
–noitem don’t display items
–notags don’t display tags
–separate-output output one line at a time
–output-fd output to fd, not stdout
–title display title
–backtitle display backtitle
–scrolltext force vertical scrollbars
–topleft put window in top-left corner
-h, –help print this message
-v, –version print version information
++ wolfram ‘Remove Wolfram on a PI to save space’ OFF office ‘Remove LibreOffice on PI to save space’ OFF
Wolfram Language 11.0.1 Engine for Linux ARM (32-bit)
Copyright 1988-2017 Wolfram Research, Inc.
In[1]:=
(Yes, I did edit the script and insert set -x to obtain the above)
Not complaining, just letting you know.
Thank you – I am going away for a few days so can’t progress this now but I just had someone use the script successfully barring one item which we fixed (nodeJS) today. I have root because I want it – it is not necessary for this script… and I’ve no interest right now in fail2ban – others can add these things if they want along with anything else. The script will work with a basic Raspbian – I cannot really comment on it’s operation outside of that basic scenario or if it is modified.
Sorted
As the script is open source – you are more than welcome to take what is there and make an improved version.
Nodejs in the /dist/latest-v6.x/ folder is now 10.2 instead of 10.1, which makes the installation fail, then nodered fails too.
Please note there is also nodejs 7.
I would bow to the opinion of the Node-Red guys as I only need nodejs for Node-Red – they generally avoid the very latest versions.
Fully agree on not using 7 or 10.2 for that matter. It is just that script now hard fails the nodejs installation. So my suggestion is to either change line 712 (and 711) to use 10.2 instead of 10.1, or use /v6.10.1/ instead of /latest-v6.x/
Should have been clearer about that.
just checked, and he’s right, nodejs 6.10.1 was deleted from node servers and now there’s only the newer 6.10.2… every mention to
node-v6.10.1
should be changed to
node-v6.10.2
in script, now…
he’s NOT right about nodejs7, as it’s not STABLE and we use only stable versions…
Fixed versions in the script… and yes I noted the blog is running slowly – complaining to service provider now..
Hi Pete/all,
just working with my Orangepizero, restoring it after an SD card corruption. Whilst looking for regular writes, I found that WebMin has folders on the SD card which it both logs to and uses for temporary files for serving up info.
‘The script’ installs log2ram? This effectively makes ‘/var/log’ a ram disk, but persists it at startup and shutdown, and once per hour?
As a test, in ‘/etc/webmin/miniserv.conf’ I’m changing
logfile=/var/webmin/miniserv.log
errorlog=/var/webmin/miniserv.error
pidfile=/var/webmin/miniserv.pid
env_WEBMIN_VAR=/var/webmin
to
logfile=/var/log/webmin/miniserv.log
errorlog=/var/log/webmin/miniserv.error
pidfile=/var/log/webmin/miniserv.pid
env_WEBMIN_VAR=/var/log/webmin
and
sudo cp -r /var/webmin /var/log/
and
sudo nano /etc/webmin/var-path
(change the path in there too).
sudo service webmin stop
then
sudo service webmin start
and now webmin is again running, but with no regular SD card writes (except once per hour when log2ram will store everything in var/log).
webmin probably does a LOT of individual small writes, which could do quite some damage to the SD card over time (e.g. the sessions db is in there….).
This may be a useful addition to the way the script installs webmin for those of us with fragile media?
Simon
p.s. My home-auto node-red flows also persist data to disk (graphs, etc.), so I’ve moved them to a folder within the /var/logs folder; far better a once-per-hour write than a once per minute – and no need to design SD protection into the flow… but you do risk up to an hours’ worth of data if you have an unexpected failure. I decided better lose an hour of data than 4 hours of my time restoring the whole setup with a risk of wife-rage because the hot water did not turn on!
mosquitto is still writing to SD occasionally.
I’m using ‘sudo iotop -a’ to observe disk writes – just leave it on for a while, and offending processes float to the top.
ohhh.. reading above, I don’t know of log2ram was a script install, or if it was part of the armbian OS image I started with. If it never got into the script, I highly recommend it :).
after a reboot, webmin did not start; seems log2ram did NOT copy /var/log/webmin/ to /var/log.hdd/ on a sudo reboot.
Since I believe the folder is only temporary files effectively, I’ve done a manual copy, then the reboot works. Also from brief reading, log2ram should copy it….
Log2RAM is currently NOT in the script – never has been. Currently considering adding this..
https://github.com/azlux/log2ram
Just a case of deciding defaults 40 meg allocated and whether to use cp or rsync. I’m thinking the latter and maybe 60meg?
i remember there were references to apache, which should be explicitely added as to start AFTER log2ram start, otherwise it would fail… maybe it’s the same for webmin… but on that github there are no more references to apache…
Good job your memory is better than mine – yes – Apache would not start – can you remember what we had to add to make it work?
i think the author reworked the script and so it’s not needed anymore to modify it… the mod should have been to add apache.service to the BEFORE line, if i remember well
https://github.com/azlux/log2ram/commit/01632f9cb1746d066af3826816fc4518494e08d7
Note: armbian has
https://github.com/armbian/build/tree/master/scripts/log2ram
(‘updated’ – mainly seems to be about config and script location)
log2ram script appears to be disabled at first look, but in armbian version, actually loads config from /etc/default/log2ram which overrides the enabled=false in the script.
Mine appears to be set for 50M, of which 28% is in use (including webmin folder + 7M of my own data).
Rsync was OFF; I just turned it on and it seems to be doing ‘reasonable’ things.
also found:
https://github.com/Fourdee/DietPi/issues/781
which may be relevant to some rpi users.
Maybe if we’re to use (as I am) /var/log as ram based perm store for NR, we should add the NR service to the ‘before’ line too….
(my apache2 was failing to start – but because /var/log/apache2 did not exist).
latest log2ram versions take care of this, read the beginning of the changelog: https://github.com/azlux/log2ram/commit/01632f9cb1746d066af3826816fc4518494e08d7
so maybe armbian is just still using an old version…
dietpi has its own log2ram version, directly selectable via dietpi-software companion script
ask the author on github to take a look, opening an issue there:
https://github.com/azlux/log2ram/commit/01632f9cb1746d066af3826816fc4518494e08d7
I am having problems with running the script on the latest release of raspbian jessie lite,
The problem starts when installing Node-Red,
–2017-04-21 05:00:23– https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/nodered.service
Resolving raw.githubusercontent.com (raw.githubusercontent.com)… 62.252.172.241
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|62.252.172.241|:443… connected.
GnuTLS: A TLS fatal alert has been received.
GnuTLS: received alert [0]: Close notify
Unable to establish SSL connection.
I have done a few google searches but cant seem to find any information on this.
Any help would be appreciated.
Hi Craig
The script has not been tested on Raspbian Lite, just on the full Raspbian – and amongst the things you can do is remove office and that maths program which collectively take up a lot of room.
no problem on jessie lite, i used that, i don’t like wasting space for a gui (plus the tons of gui programs) i will never run… their base is the same, only difference is the presence of the Pixel gui in the full version…
the problem of craig is the same of that guy we solved in chat a while ago: some of the github round-robin served servers have wrong ssl config… solution: add this line to /etc/hosts and retry:
151.101.128.133 raw.githubusercontent.com
Gave this a try and it worked fine.
Thank you for the quick fix on this issue.
solved, to put in script… it first tests if the actual github server returns a good certificate, in case does nothing, while in case of a wrong one, it adds a good host/ip association to /etc/hosts… Peter, add these 2 lines before the one at line 754, but after the ELSE on 753
echo | openssl s_client -showcerts -servername raw.githubusercontent.com -connect raw.githubusercontent.com:443 2>/dev/null | openssl x509 -inform pem -noout -text > /dev/null 2>&1
[[ $? -ne 0 ]] && echo 151.101.128.133 raw.githubusercontent.com | sudo tee -a /etc/hosts
please try latest script version, Peter added the check i wrote down here in comments, test it and let us know…
I’ve just tried with the recently updated script and still get the same error.
The first response to this issue i tried and that worked O.K.
I now have my Pi up and running but if you need me to test any more changes to the script regarding this issue just let me know.
i tried those lines, now in script, and here worked well… did you try before or after that mod i suggested? Which does nothing but the same thing, as 1st line in script now tests if ssl certificate is correct, and 2nd one adds the line you added manually in case 1st line returns an error… so, it should have worked…
can you remove that line from your /etc/hosts file, and run these 2 lines and report if they say OK or KO? or better, run these 2 lines first WITH and THEN without that line in hosts, and report both results? Thanks
echo | openssl s_client -showcerts -servername raw.githubusercontent.com -connect raw.githubusercontent.com:443 2>/dev/null | openssl x509 -inform pem -noout -text > /dev/null 2>&1
[[ $? -eq 0 ]] && echo OK || echo KO
I ran the script on NanoPi M1, left it over night and hostname dialog or connection timed out. I reran the script and came back in 1 hour and changed the hostname and rebooted. Logged in via ssh as pi user. Browsed to device http://hostname.local and saw the “Pete’s Utilities” page. WebMin worked, Node Red didn’t, OPI Mon worked, SQLite Admin and Sys Info worked too. Only Node Red didn’t work because of the 2nd run of the script and double entries in settings.json.
Note Red installation had multiple entries for admin and user Auth’s so once those were cleaned up in pi/.node-red/settings.js and rebooted it worked.
In The Script, line #6 of “IMPORTANT” states to not access(run?) the script as SUDO. But my understanding is that on systems without a pi user it has to be run first with sudo and then run as pi user.
The hostname dialog has now been moved to the front of the script.
If the Node-Red installation has mutliple entied for admin – the script or part of it has been run twice by some means or another.
If the system has no Pi user, running it the first time as ROOT will create the Pi user and will now move the script – with permissions modified, to the Pi directory – you then log in as Pi (password password unless you change it which is a good idea) and you run again and that completes the operation.
we can add a check if settings.js exists and has stuff in it already, before adding lines to it… adding to our TODO list…
in the version just released of the script, if you try to rerun the node-red setup it will skip the step and go on, preventing double lines… if you want to rerun, at lease rename settings.js, but best rename or move elsewhere the .node-red folder
I ran The Script last night on a NanoPi M1+ and it took longer than I could wait so I left it overnight. In the morning it had stopped from a connection timeout at the “change hostname” dialog box.
Maybe a better option would be to move all user input requirements to the top of the script so it can run to completion unattended after a few user input items at the beginning?
It IS all at the top except one unimportant item at end
I reran it and sure enough, it’s waiting for the hostname and finished. I could have looked at the script but wanted to see what it would do re-running it. 🙂
Wish List: move hostname prompt to the top and add a menu item to reboot when complete.
Wish fulfilment officer here. Wish fulfilled – also done various other fixes and made output less noisy (nodes). All updated now.
oh, the nodejs install from xz package, is less noisy too, but we tested only in vm, so you didn’t realize that… needs testing on a arm64 board (you know which 🙂 ), and/or a rpi0 or rpi0w… i added “pv” to have it show a nice progress bar
mmm, we can do that… ask at beginning, and leave the current ip and hostname at the end, where they are… 🙂
needs testing, as changing hostname while script running already brings some little delays at the end and some warnings, which i suppressed as we do a reboot soon after, and after reboot all is fine…
WOW! This is great!
Saved me a lot of work!
Thank you so much!
And – don’t try this on orange pi official rasbian distro – that is total crap and I wasted 3 days trying to install everything on it. On Ambrian it did everything for less than an hour.
You are right. I did it then, but ran into my first problem (infinite loop) and forgot about your comment. Sorry about that.
In the meantime, I found out, why the loop was infinite: If node-red still runs when I start your script it does not create a “virgin” settings.js. So, I added the line:
sudo service nodered stop;
at the begin of the script and now, it runs perfectly. Thank you very much for your patience!
oh, yes, that too…
thanks to you, preparing popcorns for your next video… hoping no lora, a good mailbag would be great 😀
The second problem was, that, everytime the script runs, it adds a section, which is already there, if node-red was installed by your script…
Now, it works (I followed the advice of Peter).
httpStatic: ‘/home/pi/.node-red/public’,
functionGlobalContext: {
os:require(‘os’),
moment:require(‘moment’),
fs:require(‘fs’)
},
adminAuth: {
type: “credentials”,
users: [{
username: “admin”,
password: “$2a$08$o9n62Pq6s5YVfD2C.inYKucfIOp26eYart4s0XeUB3Cbf7h.GIsrC”,
permissions: “*”
}]
},
httpNodeAuth: {user:”user”, pass:”$2a$08$z2m9DthU3ZrfFtDx7ceD..HlFOuJvdX2Plmdkpvg//YNfCYbJ2BpW”}
}
infact in my very first comment I told you to remove or rename the settings.js file, which would have been recreated by the script:
https://tech.scargill.net/a-christmas-script/#comment-32792
The script was looping endlessly, because of a lacking settings.js in my directory. Now, I am one step further. I get an error message at node-red start. But I have to run…
I was not able to install dos2unix with your command (it did not accept the -y). So, I went the way via notepad++. Now, I get no error message, but the script does not return anymore… (it also does not react to ^C. I have to logout…
This seems to be really hard… (for me, at least)
Andreas – as the script clearly works – would it not make life easier here to simply reload the operating system image and run again. It doesn’t take any interaction once it starts as you know.
You might be right… Did not think, it will be so complicated. But I always learn. So, I will start over. Thanks for your support!
you’re right, in the copy paste i missed the “install” part… damn… these are the correct lines, sorry
sudo apt-get -y install dos2unix
cd
wget http://pastebin.com/raw/9ct1Fmz4
dos2unix 9ct1Fmz4
bash 9ct1Fmz4
I changed to the pi home director and started the two commands (Wget and bash). Unfortunately, I always get the error message “line 5: $’\r’: command not found
“.
Then, I created a file using sudo nano and copy-paste raw. Same thing…
Looks like wrong file format to me at first glance – see what I said about Notepad++
as Peter said, wrong end of lines…
sudo apt-get -y dos2unix
cd
wget http://pastebin.com/raw/9ct1Fmz4
dos2unix 9ct1Fmz4
bash 9ct1Fmz4
Hi Pete,
I used your script in my newest video and wanted to thank you for this great job. From the comments, my viewers like it, too!
One question: Do you have any plans to include InfoxDB and Grafana in your script? It looks like these two are quite famous (and I was not able to get Grafana running)…
Hi Andreas
Thank you for the kind words… Grafana I’m more than happy with – InfluxDB – not so… I really would like to see someone do a SQLITE interface… I’ve had a couple of days of disconnects (power cuts in Spain) and some time ago a glitch -and there are values not necessarily out of range but wrong – InFlux simply won’t let you do something as simple as pick a record or records and delete them. So if you don’t mind I’m going to hang back on that a little in case a better, equally simple solution comes along – and of course – whatever it is has to work on everything from a Pi Zero, through various boards and phones to VMS – as does the rest of the script.
Hi Pete
I have some trouble with the credentials in node-red dashboard. Admin admin does not work for me…
Can you guide me to looking up the user and change the password without re-installing everything?
remove/rename settings.js file in /home/pi/.node-red then run this (modifying users and passwords as you wish, here are the defaults in Peter’s script), as user “pi”, of course…
http://pastebin.com/9ct1Fmz4
Thanks for your fast answer. Unfortunately, I am a complete Linux beginner. How do I “run” this script? I tried by just pasting it into the terminal and creating a .sh file. Both, unfortunately were not successful…
cd
wget http://pastebin.com/raw/9ct1Fmz4
bash 9ct1Fmz4
Don’t forget when running a script – it has to be in Linux format – that is line feeds only – you can ensure that in NotePad++ under EDIT – EOL CONVERSION – LINUX – and the script once created (it IS just a text file) needs EXECUTE permissions.
If you get the format wrong – the resulting error message is not helpful.
Perhaps this is what you’re looking for? If so no need to install anything – it’s in VS Code vanilla:
Oh hell – is it THAT easy…. will you look at that – all those other things on the bottom do things as well…. I’m liking this more by the minute.
Thanks for the feedback on this guys.Oh – and an update, too.
Ok, just got a couple of new RPi3’s and downloaded the latest Rasbian and wanted to install the system “headless”. After using the trick to have SSH enabled at boot up (which by the way don’t work for Ubuntu even if that is the directions) 🙁 Then after following you direction plus a few more to get a working password for root and pi added to the sudo user group I loaded you script to /home/pi then chmod to 744. When I tried to run it with just ./script.sh I got a (bash: ./script.sh: /bin/bash^M: bad interpreter: No such file or directory) error. When I did it (bash ./script.sh) it just bailed out back to command prompt. No menu, nothing. Can someone tell me what I am missing??? Thanks
Sounds to me like the file is saved in PC format…. needs to be saved in Linux format – that is – with only Linefeeds (I think that’s right) and not both CRLF. I’ve never understood the message when that is wrong.
the first line (called SHEBANG or HASHBANG, and starting with #! ) in a unix script tells the shell which interpreter to use to execute the following lines… the error is just saying that there’s no interpreter called:
/bin/bash^M
as it’s actually called
/bin/bash
the ^M is exactly what Peter said, the carriage return char, which should be stripped out…
you can do it in linux itself:
sudo apt-get install -y dos2unix
dos2unix script.sh
(the same package adds the unix2dos conversion tool, too, just in case…)
info: https://en.wikipedia.org/wiki/Shebang_(Unix)
Yes, that’s what I had done.. I read your note but choose the wrong EOL. I saved it as Mac. It is running now, wo hoo!! Now to see what else I can foul up.
Thanks.
This new Microsoft Visual Code editor – whereas in Notepad++ it is as easy as selecting EDIT then the format – I’m not finding a similar menu item in the Microsoft editor – which is a bit of a pain – it would be nice if the editors would flash up somewhere what format you’re using…
this?
https://github.com/Microsoft/vscode/issues/2957#issuecomment-200362773
not exactly what you ask, but…
found… honestly, this is done way better and easier in n++… anyway:
open vs code
ctrl-p
ext install beautify
on the left, install the one by hookyqr (has this icon: https://marketplace.visualstudio.com/items?itemName=HookyQR.beautify )
reload vs code
then open a file, and press F1 (or ctrl-p, then > )
write: eol (and press enter)
choose the desired eol…
the extension itself does MUCH more than this, try to write beutify on a js, or css, or html file…
Yes, to the Node-Red ftp node (node-red-contrib-ftp). I coundn’t find the directory where jquery was located, but I will look again. Will do the same using a wildcard to see which version of the vnc server might already be installed with Raspbian. Might just need to do a configure for automatic startup at boot.
Thanks.
Just a suggestion,
How about adding the following to the install list?
jquery, vnc (remote viewing) and the ftp node.
Thanks for the good work!!
jquery is available in Node-Red, VNC…. erm seems to be already on most boards but worth checking for….. ftp node? You mean in Node-Red?
What I meant by JQUERY was… if you’re doing anything with Node-Red Dashboard – like using a template – you have full access to jQuery in there without having to insert a link
Maybe this is not your field, but I will try this question to you.
I reinstalled my system on a pi3 a few days ago when an npm update miserably failed and I lost all my nodes. As I am a Linux illiterate this was the only thing I could do.
Now I found SD Card Copier and made a backup SD card with success, I swapped the written card with the original. As I did not like to do this job by hand every time I searched for another method and found this: https://github.com/aweijnitz/pi_backup.
As I have an old 2,5 inch laptop hard drive I wanted to use for automatic backupping. Then the problems started as I found out I have a read-only file system and this HD cannot be written to with the above mentioned script. Really nothing goes!
I suspect this behaviour could have been setup by the Christmas script, but that is strange. I know you are not a linux specialist yourself I cannot understand why this is the normal behaviour of the pi3.
I have read some articles about corrupt SD cards caused by power outages. I now think I know why you are so keen on having this backup board with a lipo on it. My experience until now is that during all my resetting the pi3 by flipping the power switch myself when I thought things were screwed up I never had a SD card failure. You also have this experience in your house in Spain, where the power is not that reliable too. I found this article https://hallard.me/raspberry-pi-read-only/ that tells it is a good thing to have a read-only file system, but I will not fiddle with my system as described there.
On the side of all this I noticed a thing about the pi user being a member of the sudo group. In the blog is is written you have to be sure pi is a member of it. What I see when opening the file manager then pi is listed as a member of group pi in the permissions box of the file manager. I have no idea if this is wrong and will have influence on the problems I experience!
Can any one guide me out of this situation and explain how to be able to use the backup script on an external drive?
So for other people having same problems:
I installed gnome-schedule, gparted and I could work without learning Linux deeply. I edited the backup script to my needs and now I have a working backup automatically going every night. I formatted the external disk to NTFS so I could detach it from my “dead” RPI just in case and I can write a new image from the latest backup with Win32 DiskImager on my PC to the “dead” SD-card. Occasionally I make a copy with SD Card Copier, as extra safety. All with very little attention needed.
Why not integrate this in the startup script to make live easier for Linux illiterates.
looking through available source & firmware, looks like it’s going to be wired only for me for the moment; no time to port wifi drivers this week…..
Ping times are awful (anywhere from 3ms to 2000ms; mostly 200-300ms).
Wifi comes back up on restarting my bthhv2b. But, my wireless router is NOT my dhcp server, so it may not be an equivalent test….. Saw your topic on the armbian forums…
ahhh.. just looked at logs. It did not reconnect to the SAME router (I have 2, and it knows about both…), so not a suitable test…. Interesting that it’s happy to migrate though.
just to report, the script worked on Orange pi zero with Armbian_5.24_Orangepizero_Ubuntu_xenial_default_3.4.113.7z
reported execution time 46 minutes.
Thankyou 🙂
Let me know how your WIFI works – particularly recovery from access point failure (just unplug) and general reliability… that is my one reservation about the Zero and I had no luck at access point failure recovery using Armbian (fine on hardwired ethernet).
Wow! I am sooo far behind on all this. However, learning is fun?
Hello Peter
I have to admit that I am stumped. I have been using Mosquitto for about 6 months but recently I had a crash, lost my installation on my Pi 3, and reinstalled all. Now I cannot get Mosquitto to work. I get this message when I enter this:
sudo mosquitto_sub -d -t Humidity
the error I get is:
Connection Refused: not authorised.
I used the script to install Mosquitto and Node-Red. Can you point me in the right direction please? I have searched the web but have not made any progress for the past several days. As I said, prior to the crash I had collected 100’s of megabytes of data and controlled lighting in my home. Now, I cannot get Mosquitto to work at the simplest level.
Regards,
Bill Miller
seems an auth error… verify that a user+pass is present in /etc/mosquitto/passwords
https://community.openenergymonitor.org/t/mqtt-server-issue-connection-refused-not-authorized/1828/15
I’ll second that – seems like a username and password issue.
Hello
Thanks for your work.
Two comments :
For real Raspberry, I would recommend to add to your script the log2ram tool. This is helping to save SD card and avoid corruption (logs are stored in RAM and dump to SD only once an hour per default). -> https://github.com/azlux/log2ram
For your Debian stuff, you should have a look to the new X86 Raspberry/Pixel official image (if you’ve not had the info yet). This is made by raspberry org themself to run on PC or MAC and this should be pretty easy to have it on VM with persistence. This will be as close as possible to the regular raspbian … -> https://www.raspberrypi.org/blog/pixel-pc-mac/
Happy new year.
Sebastien
Hi there Sebastien
Thanks for that and happy new year. Raspberry Pi – do you want to work up the exact instructions to add Logram and put in here why it’s a great idea – if it is reliable we can always addt it to the Raspberry Pi menu!!! I’m all for offering to ditch logs actually – ie run in RAM but don’t save. I only ever seem to use logs to check why something went wrong – not really interested weeks later and it would lengthen life of SD ??
Debian – I have PIXEL running in VIRTUALBOX – works a treat – and the script works perfectly on it – it’s a good quick way to get Debian and of course includes SUDO which Debian often doesn’t . Don’t get me wrong however, it is NOT a Raspberry Pi emulator… sound controls are missing – there is no RASPI-CONFIG and no hardware – like ports or anything… Shame – had they thought it through – a little usb board with ports on, made to LOOK like you have ports – would have been nice. Still – it is free and it works. Shame they’re not ready to make it work on other boards because some of the alternatives to Pi, while having great hardware have really rubbish software.
pixel raspbian was not meant for virtual machine (so the extra steps needed to have it working in a vm – why not doing as ubuntu, an icon on the desktop of the live cd, INSTALL, done???), and having it on actual physical hardware can bring audio and other stuff working… for ports, you can just passthrough them to the hypervisor of choice, not always easy but doable… clearly no gpio or other hw, only boards you can already have working via usb, like an arduino via firmata, or other boards Pete already tested in his blog entries in the past…
frankly, the only advantage raspbian has is that’s official… i don’t care about the gui, i use ssh into my system every time, so pixel is useless for me… and, to be honest, it seems windows 95 with its icons… better than the former ones, but quite oldstyle anyway… but everybody says LOOK’MA, DAS ICONS! (there are TONS of better looking icon themes out there, from DECADES…)
oh, and pixel has no clear instructions to put it in a vm, you have to dig into comments in official blog page to know how to do…
raspbian has nodered and other stuff in it, but usually is old itself, too…
ubuntu has newer packages already in its standard repositories, while dietpi has the best starting point for SBC, with low memory, low space needed, logs in ram, and many other optimizations…
once setup, are all the same, you don’t bother about what’s underneath, your interest is in the service you use above the OS… which are all the same…
dietpi is sure my distro of choice, right now…
if nodered is your sole purpose, http://thethingbox.io is a good choice, too…
Sebastien, this looks interesting.
Along these lines I’ve been using CIFS to access disk space on my network shares (CIFS was easier to explain to friends than NFS or AFS). For when I must have local storage I dropped on a cheap 120G SSD drive and cross link it to the drive. I need to sit down and figure out how to mount the SSD as the writable part and the SD as just readable.
in case someone want to try, detailed instructions on how to have a LOCAL BLYNK SERVER up and running… spent the morning testing…
https://gist.github.com/fragolinux/a5e7d6153afab4c52d91d41ff9860b8c
Has anyone successfully run the script on a Pi 1 loaded with DietPi. I’m having no luck.
“npm command not found”
And yes I have a PI3 running well, but I have a couple of early PIs which I would like to give to my son.
PLEASE NOTE ALL – a BUG crept into NODE-RED-CONTRIB-BIGTIMER – I have updated this and fixed this bug which would have stopped bigtimer running.
To fix on existing setups, go to your .node-red directory (probably /home/pi/.node-red
and as PI user, from there merely…
npm install node-red-contrib-bigtimer
Update already but please wait until 9am today (Friday 6/1/2017) or later. The version you should end up with is 1.4.88 or later.
why “install” and not “update”?
I suppose you could do update – but that’s two commands you have to remember 🙂 Both do the job.
A most awesome & handy script – thanks Pete & Antonio! Used it last night on my RBPi3 and it’s up and running.
I just finished running your script in Virtualbox 5.1.12 and everything appears to have worked flawlessly. I have used Virtualbox many times over several years for various reasons so that part was easy. The script itself took less that 15 minutes but I did not time it to get an exact time (Win 7 i5 2500K host). My “real” server is running Ubuntu 14.04 desktop and I’m looking to upgrade it to Xenial. I have been using MySQL vs SQLlite but your script has everything else I use plus more.
Thanks for a great way to test things before I try upgrading my real server.
John
Hi john
Well, the purpose of using SQLITE is to keep writing down to a minimum for SD memory on little SBC boards such as the pi – for simple logging, MYSQL though it would be my preferred DB is well over the top. Still – no doubt you can install that separately on your VM – and if you look at the script itself you’ll see it isn’t that hard to add sections – just be aware if you add anything to the menu – you need to change 2 values at the top of the menu to make it larger (because it isn’t intelligent enough to autosize).
Just getting back to the virtualbox installation of the script on Xenial and I can’t seem to find Bigtimer in node-red. It is in .node-red/node_modules and I was able to update it to 1.4.88 using npm. However it still does not appear as an available node on the node-red web page. esplogin, grove and timeout appear under the scargill heading but no bigtimer. I don’t understand enough about node-red to diagnose such issues. Can anyone point me in the right direction?
All working – DietPi VM, Debian VM, Ubuntu VM
apache+php7 install on standard debian+pixel sorted out… they report 32bit arch as i686… patch sent to Pete, which is trying pixel now, i tried debian, everything fine now…
npm, still don’t know, as it’s not affected by php part of the script…
Not quite….
they don’t know how much work needs this to be tested…
Thanks Peter,
Trying to run the script on a Virtualbox running MX-Linux (Jessie) and cannot get pst the npm installation part. Attached image shows the errors. I’ve tried a couple of times with no luck.
Any ideas?
Clearly, NPM was not installed. So you need to look back in that installation and see the earlier section to find out why not.
Interestingly the only issue I’m currently having on VirtualBox is with Apache – in both the PIXEL and DEBIAN 8.6 images, Apache won’t load and so of course PHPLITE INFO won’t load (I’m sure we’ll fix that today and that it is to do with getting the right installation image) – but NPN on VirtualPC – Pixel, Debian and Ununtu are all fine
If anyone wants to jump in on this and help – by all means.. I’m thinking on the Apache thing we’re going to need a menu option so the system knows this is a VM on a PC… but that still leaves the question of which apt-get etc.
Any ideas on Scott’s issue? I’ve no experience AT ALL of MX-Linux either natively or on a VM.
problem is always the same (for apache): the output of:
cat /etc/os-release
and
uname -m
as these are what we use to select the right repository…
for npm, don’t know…
I’ve purged the virtualbox OS and started over. Get the same problem. npm seems not to be insstalling. The text below is what comes prior to the image II posted just above here.
====
==== Installing Installing Node-Red – Total Time: 00h:10m:02s
====
sudo: npm: command not found
–2017-01-05 12:04:48– https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/nodered.service
Resolving raw.githubusercontent.com (raw.githubusercontent.com)… 151.101.20.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.20.133|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 888 [text/plain]
Saving to: ‘/lib/systemd/system/nodered.service’
/lib/systemd/system/node 100%[===================================>] 888 –.-KB/s in 0s
2017-01-05 12:04:48 (14.6 MB/s) – ‘/lib/systemd/system/nodered.service’ saved [888/888]
–2017-01-05 12:04:48– https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/node-red-start
Resolving raw.githubusercontent.com (raw.githubusercontent.com)… 151.101.20.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.20.133|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 2111 (2.1K) [text/plain]
Saving to: ‘/usr/bin/node-red-start’
/usr/bin/node-red-start 100%[===================================>] 2.06K –.-KB/s in 0s
2017-01-05 12:04:49 (9.68 MB/s) – ‘/usr/bin/node-red-start’ saved [2111/2111]
–2017-01-05 12:04:49– https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/node-red-stop
Resolving raw.githubusercontent.com (raw.githubusercontent.com)… 151.101.20.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.20.133|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 226 [text/plain]
Saving to: ‘/usr/bin/node-red-stop’
/usr/bin/node-red-stop 100%[===================================>] 226 –.-KB/s in 0s
2017-01-05 12:04:49 (2.63 MB/s) – ‘/usr/bin/node-red-stop’ saved [226/226]
Failed to get D-Bus connection: Unknown error -1
please, WHICH distro are you using? We tried script on STANDARD dietpi, raspbian, ubuntu and debian, NOTHING else…
This Debian Jessie, distro is MX-Linux https://mxlinux.org/ . Tried both 32 and 64 bit versions.
The computer is a Panasonic Toughpad with the same distro as the Host, using Virtualbox.
I’m willing to test!
Let me download the Ubuntu. Never been a fan of Ubuntu, but I will give it a go.
please, while reporting bugs, EXPLICIT what script version you are using (and try to use ALWAYS the latest…), and which distro… if it’s not 1 of those I listed, JUST SAY IT, we have no crystal balls to guess, nor we can make script work on EVERY single version of distros out there…
Repaired this part of the problem, but editing the /etc/ls-release file. Changed the Metamorphosis to jessie, that way nodesetup recognizes the distro and proceeds.
Still can get past the installation of the nodes tho. Appears to a systemd problem.
Fun playing.
Any WinSCP variants for Linux?
Not a Linux man I’m afraid – perhaps others can help. I’d be surprised if there is anything as pretty.
filezilla, or every desktop environment in linux has bindings to access a SSHFS, examples in comments here: http://stackoverflow.com/questions/299412/is-there-any-winscp-equivalent-for-linux
filezilla
Just completed a rather long script install on a Raspberry Pi 2 having added a timestamp to the beginning of each section – I seem to recall some folk liking to keep track – also added (thanks Antonio) a progress bar. The screen is still WAY too noisy for me but we’re working on that.
The script despite taking ages, worked perfectly – bog standard Raspberry Pi2 with the November Raspbian.
The FriendlyArm S2 also works but I’m just kitting that out with a new eMMC-based Debian to try again, leaving it to run overnight. I’ll do a writeup on this board soon, probably tomorrow.
I just registered to share a comment about your script. A few days ago I booted the Raspbian “Pixel for PC” from a USB flash drive onto an old Toshiba Satellite laptop (M115-S3094). This afternoon I tried running your script on that machine. It worked! I rebooted and now have node-red running at 127.0.0.1:1880 with your large array of nodes to try out. Webmin is also working. I haven’t tried anything else yet. Just wanted to let you know that the script works even with “Pixel for PC”. I have a very limited knowledge of Linux so to me this seems quite a trick! Thank you and Happy New Year!
Hi
Thanks for the kind words. Etc did you manage to get remote scp access to Pixel for PC?
as far as i could see in the few minutes i used it, it has no ssh server… so
sudo apt-get install ssh # (to install ssh server+client)
sudo passwd # to give root a password
then it should work, as always
I struggled with scp on the Pixel OS – always timed out when trying to send a text file to my Windows 10 machine. I was able to setup and use a PuTTY connection. I had to turn on the SSH server (OpenSSH_6.7, already there, shown in Webmin) and allow authentication by password. Finally, I installed WinSCP and was able to drag and drop a text file from the Pixel machine into a directory on the Windows 10 computer.
look for ssh into Peter’s script, there is a block about it that enables password login for root, disables dns to speedup the password answer, and removes timeouts…
Thank you for your help. I located the ssh block in the script, then I viewed the sshd_config file and saw the appended lines about UseDNS and ClientAlive… I have learned that Pixel for PC version 1.1 has the ssh server off by default. I don’t understand Linux command syntax well enough to know if that would have any effect on the script. I do see the last line in the block restarts the ssh server, but the Webmin interface said the ssh server was off. Thanks again, no response necessary. The problem is most likely operator error. Right now I have a working system and plenty to learn!
use raspi-config: https://www.raspberrypi.org/blog/a-security-update-for-raspbian-pixel/
or, as per this link https://www.raspberrypi.org/documentation/remote-access/ssh/ :
“For headless setup, SSH can be enabled by placing a file named ‘ssh’, without any extension, onto the boot partition of the SD card.”
these are the lines i added/modified to sshd_config:
PermitRootLogin yes # to allow root login directly… i would not do this, but some wants…
UseDNS no # to reduce the time between inserting user and the following password prompt
TCPKeepAlive no # these 3 lines are for disable ssh timeouts
ClientAliveInterval 30
ClientAliveCountMax 100
Just tried it on a fresh install of Armbian (Ubuntu 16.04.1 LTS) on my Olimex A20-Olinuxino-Micro, and it seemed to work flawlessly (meaning I didn’t notice any major errors, and all the components seem to be functioning), in case anyone has the same platform and is interested in trying it.
Magic – thanks for the feedback
Hi Peter,
Great video & script. I’ve installed Ubuntu Xenial Desktop as a VM in Virtualbox and followed your instructions. However, the script has stalled at the point below. Tried twice but getting the same result.
================= Installing Node-Red ====================
make: Entering directory ‘/usr/lib/node_modules/node-red/node_modules/bcrypt/build’
CXX(target) Release/obj.target/bcrypt_lib/src/blowfish.o
CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt.o
CXX(target) Release/obj.target/bcrypt_lib/src/bcrypt_node.o
SOLINK_MODULE(target) Release/obj.target/bcrypt_lib.node
COPY Release/bcrypt_lib.node
make: Leaving directory ‘/usr/lib/node_modules/node-red/node_modules/bcrypt/build’
make: Entering directory ‘/usr/lib/node_modules/node-red/node_modules/ws/node_modules/utf-8-validate/build’
CXX(target) Release/obj.target/validation/src/validation.o
SOLINK_MODULE(target) Release/obj.target/validation.node
COPY Release/validation.node
make: Leaving directory ‘/usr/lib/node_modules/node-red/node_modules/ws/node_modules/utf-8-validate/build’
make: Entering directory ‘/usr/lib/node_modules/node-red/node_modules/ws/node_modules/bufferutil/build’
CXX(target) Release/obj.target/bufferutil/src/bufferutil.o
SOLINK_MODULE(target) Release/obj.target/bufferutil.node
COPY Release/bufferutil.node
make: Leaving directory ‘/usr/lib/node_modules/node-red/node_modules/ws/node_modules/bufferutil/build’
try running these 2 lines, i think that is the bcrypt module… maybe updated today and causes problems, as yesterday was all fine…
cd ~/.node-red/
sudo npm -s install bcryptjs
I’ve done another install on my Pi3 with a clean image of Jessie. I used the defaults on the script and I got through to the end without any errors. I did get a warning of an unused variable ‘status’ when the sqlite was installing though.
Comments:-
I though my Pi had crashed to start with until I saw the -qq switches on the update and upgrade. I’ll remove the -qq next time I run it.
I did like the elapsed time feature of the older scripts (now gone).
Some packages recommend ‘apt-get autoremove’. Maybe put this at the end of the script?
Otherwise – awesome!! Well done Peter & Antonio.
Hi Keith
Can you clarify – Pi3 – you say Jessie – was this a generic Jessie or are you talking about Raspbian – very important to clarify.
The qq options are to try to quieten down the “noise” coming off the installs – after all a BOATLOAD of messages telling us what has been successful are actually of no use – it would be nice indeed to figure out a reliable way of ONLY showing error messages – as they often get lost in the “noise”. The old elapsed feature – yes I agree with you but that was based on a notion of interaction – incidentally the old script is still there for those who want to use it.
Apt-Get autoremove – I see no reason why we should NOT do a cleanup at the end – Antonio – your thoughts?
yes, as said on skype, when you install “something”, it brings together an other lot of packages… when you remove “something”, those addons packages are left behind… to remove these packages not needed anymore (as the packages using them was already removed), you do the “autoremove”… to remove the downloaded (let’s say the setup ones, in windows style) and already installed packages, you do the “clean”, too
Hi Peter,
Sorry, yes it was Raspbian straight from the raspberrypi.org website.
I liked the elapsed time as it gave some idea of performance. You often mentioned how many minutes it took to install on your blog.
I’m a ardent console watcher, so like to see the ‘noise’ 🙂 It gives me a warm feeling that things are still moving along as opposed to being stuck.
in me, too 😀
we added a (no)quiet menu option yesterday, you can use it now… but time depends on your menu selection, so compare is valid only on default selections… and can change from microsd to microsd on the same platform…
Hope you’re happy with the new mods which include Timing!
Added menu option to have a quiet or normal output during install
Very interesting stuff! As I have an issue with the loss of mouse events in Chrome on Jessie I will try another installation on an 8Gb SD with the proposed Ubuntu image.
I am a bit confused about all usernames and passwords needed. Should I use PI or pi as a user name? An overview of all needed un&pw’s where to use what and when to create would be appreciated.
Another item I have not touched is access from outside of home through internet, what to do on router(s) to reach my home server from outside?
Keep on going the good work!
I’ll answer as back home
Will give it a try with VMWare and that takes some time. The description is very extensive so I might be able to come all the way through. So no big hurry anyhow, but thx for the swift attention to this.
Does not show the same options as described although the same Ubuntu image.
There comes a question BIOS or EFI and default is BIOS, but I have no idea whether this is right. I can try EFI for a better result, but that is a shott in the dark. No question about adding SSH or GRUB.
Display settings are not usable. No display is detected. It uses host settings and does not completely fit in window of VMware.
How to change to ROOT user? I am a Windows user. This will take some googling.
full album of the VM creation in Vmware: http://imgur.com/a/X8xLi
you could use Vmware Player, too, as it’s free and despite the name, allows to create VMs, too…
in vm, no real difference between bios and efi, choose what you want…
this setup uses UBUNTU 16.04.1 LTS SERVER, and so, no desktop environment… if you use a desktop iso, be sure to install vmware tools at the end, to enable graphic support (it’s in the vmware player/workstation menus)…
root is useless in ubuntu, it has no password, when you need to do something ubuntu asks for YOUR password, as first user created has SUDO rights, so can do administrative tasks without ever know root password or become root… but, many windows users used to be “administrator”, tend to enable it… bad security habit…
in a system with root enabled, an attacker only need its password to be harmful… in a system with a regular user, attacker has to know user name, user password, and THEN root password, if used has no sudo rights… but, your choice…
Thx.
I installed on W10-64 the desktop version. You are showing install on linux. Diff is menu with SSH and you did not cross Samba server, probably needed for W10 PC’s on the network.
Pete is advising some work in ROOT. Should I remove Root user?
I will start from scratch again in EFI mode with one of the premade VM’s you mention hereafter.
Terrible mistake I have filled in as guest the host W10. Will try again with guest = linux Ubuntu
or you can download premade VMs (appliances), from: http://www.osboxes.org/ubuntu/#ubuntu-16-04-vmware
there are for virtualbox, too
you need to install the vm tools, after first start
I will try again with this premade + vmtools!
Do I take Xenial from your link as the YAKKETY YAK is almost 3 times bigger?
stay with LTS (long term support) version, we test script ONLY on their latest ubuntu version (16.04 right now), not GENERAL latest one…
the username and password that are asked by Peter’s script are used for:
user: only for node-red dashboard
admin: mosquitto, phpliteadmin, node-red main interface
webmin uses it’s own access, which is root or pi, and their respective password
I have now upgraded the script to add that advice to the password section.
Do I have to take new clone?
ALWAYS use latest script, NEVER ask for help using old ones, as this could bring problems, to you and to us not understanding what’s causing what else… in case, just say I’M USING VERSION FROM XYZ (DATE)… but better using latest, as said…
Hi, I was stuck at the node-red login window. idk the correct password and usename?
the ones you put in during script setup
for the internet access… without discussing security issues which will take way too much time… you need to create “virtual servers” or “nat” or “port forwarding” or whatever they are called in your internet router… if you have access to its admin interface, or you need to ask your ISP to do it… and open, for example:
external port 1880 –> yourdeviceIP port 1880 (for node-red)
external port 1883 –> yourdeviceIP port 1883 (for mosquitto)
external port 80 –> yourdeviceIP port 80 (for apache)
leave webmin and ssh away, for your good, if you don’t know what you’re doing… and use strong passwords…
I will try that! For which protocols do I need port forwarding, only TCP or also others?
only tcp, for these services
I’ve been trying this script on a Raspberry Pi 3 with Jessie downloaded today.
A couple of points:
1 I see the whole menu if in command line mode, but on an X terminal a few characters are missing on the left hand side – so you can’t select the options.
2 I get a lot of errors eg Failed to fetch http://packages.dotdeb.org/dists/jessie/InRelease Unable to find expected entry ‘all/binary-armhf/Packages’ in Release file (Wrong sources.list entry or malformed file)
3 As with the old script, it attempts to use NPM to install things and fails because NPM is not installed. “sudo: npm: command not found” Presumably NPM is already installed with other operating systems? Could it be added to the prerequisites?
4 A logfile is useful to trawl through for error messages. You can get one with the command ./script.sh 2>&1 | tee script.log
Regards,
jb
both right… just sent patches to Peter, i did tests on dietpi and ubuntu, all right, missing raspbian, if someone want to test, in latest script change this (from line 367):
if [[ $OPSYS != *”UBUNTU”* ]]; then
#enable php7 on debian jessie – no need for xenial
echo “deb http://packages.dotdeb.org jessie all” |sudo tee /etc/apt/sources.list.d/dotdeb-php7.list
echo “deb-src http://packages.dotdeb.org jessie all” |sudo tee -a /etc/apt/sources.list.d/dotdeb-php7.list
wget https://www.dotdeb.org/dotdeb.gpg -O – | sudo apt-key add –
sudo apt-get update
fi
with this:
if [[ $OPSYS == *”DEBIAN”* ]]; then
#enable php7 on debian jessie DIETPI
echo “deb http://packages.dotdeb.org jessie all” |sudo tee /etc/apt/sources.list.d/dotdeb-php7.list
echo “#deb-src http://packages.dotdeb.org jessie all” |sudo tee -a /etc/apt/sources.list.d/dotdeb-php7.list
wget https://www.dotdeb.org/dotdeb.gpg -O – | sudo apt-key add –
sudo apt-get update
fi
if [[ $OPSYS == *”RASPBIAN”* ]]; then
#enable php7 on debian jessie RASPBIAN
echo “deb http://repozytorium.mati75.eu/raspbian jessie-backports main contrib non-free” |sudo tee /etc/apt/sources.list.d/mati75-php7.list
echo “#deb-src http://repozytorium.mati75.eu/raspbian jessie-backports main contrib non-free” |sudo tee -a /etc/apt/sources.list.d/mati75-php7.list
sudo apt-key adv –keyserver pgpkeys.mit.edu –recv CCD91D6111A06851
sudo apt-get update
fi
unfortunately, dotdeb repository has not arm packages, had to find them elsewhere…
Right everyone – … menu fixed – narrowed in for smaller screens and simplified.
I’ve implemented the changes above suggested by Mr Shark. I have to go out – I will not be back until tomorrow mid-day – here’s hoping these changes sort the Pi out.
i think in the raspbian section, if wolfram and openoffice are checked to be removed, this should happen BEFORE initial general update/upgrade in prerequisites… no need to update something and then remove it after 😉
Good idea – done – brought straight after usernames and passwords – might speed things up for some.
1 Peter reduced menu size
2 my patch in comments here, and already included in latest online script, solved php7 on raspbian
3 no npm problems, just installed latest raspbian (november 2016), launched script, everything is up and running…
i made install in 2 batch, just to check everything working…
1st run: modpass prerequisites mosquitto apache screen mpg123 phpsysinfo addindex installcu installmc wolfram office
2nd run: modpass (to reinsert password for node-red and webmin) nodejs nodered webmin java upgradenpm
not a single problem…
pi@raspberrypi:~/.node-red $ npm -v
4.0.5
pi@raspberrypi:~/.node-red $ node -v
v4.7.0
pi@raspberrypi:~/.node-red $ npm view node-red version
0.15.2
pi@raspberrypi:~/.node-red $ npm list –depth=0
/home/pi/.node-red
├── bcryptjs@2.4.0
├── node-red-contrib-bigtimer@1.4.86
├── node-red-contrib-blynk-websockets@0.0.3
├── node-red-contrib-boolean-logic@0.0.3
├── node-red-contrib-config@1.0.1
├── node-red-contrib-esplogin@1.1.9
├── node-red-contrib-file-function@1.1.2
├── node-red-contrib-gpio@0.9.1
├── node-red-contrib-graphs@0.3.5
├── node-red-contrib-grove@1.0.7
├── node-red-contrib-isonline@1.1.15
├── node-red-contrib-ivona@0.0.2
├── node-red-contrib-moment@2.0.0
├── node-red-contrib-npm@1.0.0
├── node-red-contrib-particle@0.0.4
├── node-red-contrib-timeout@1.0.4
├── node-red-contrib-web-worldmap@1.0.22
├── node-red-dashboard@2.2.1
├── node-red-node-arduino@0.0.12
├── node-red-node-darksky@0.1.14
├── node-red-node-emoncms@0.0.11
├── node-red-node-geofence@0.0.16
├── node-red-node-google@0.1.0
├── node-red-node-ledborg@0.0.17
├── node-red-node-openweathermap@0.1.14
├── node-red-node-ping@0.0.13
├── node-red-node-pushbullet@0.0.10
├── node-red-node-random@0.0.8
├── node-red-node-smooth@0.0.9
├── node-red-node-sqlite@0.1.2
└── raspi-io@7.1.0
Hi Pete,
Nice video. Always good to see things in action. Can I suggest maybe a video on messing with Node Red Dashboard, be nice to see its capabilities.
Yes that is something I planned to do soon.
Hi John – did you use the latest script as of today… I just installed on an Ubuntu real machine – and Debian and Ubuntu VMs without issue… but not on a Pi3. The issue with NPM was resolved ages ago – perhaps on a Pi the existing version is causing a problem?
As for X terminal if the terminal is too small when you start the script it is no good – that is a “feature” of the menu script… just make the terminal a little larger.
John looks like you are right – apparently the Pi does not support PHP7 – looking at this now. Need a Pi tester or 3.
Hi Pete,
I’ll help with the testing on the Pi 3. I’m flashing a fresh Jessie image to a sd card now. Does it matter if I run apt-get upgrade before I run the script or should I just start with the default image?
Pete, I’m happy to help test with Raspberry Pi if you want. I have a lot of experience with shell scripting too.
“The issue with NPM was resolved ages ago”
On a pi with the latest Jessie, NPM is not installed (or if it is, it’s not on $PATH)
I THINK that the Pi issue is now resolved and 2 changes made today to SSH and some helpful text for menus – please report only issues with software from this evening’s date ONWARDS – and thanks to EVERYONE for their help – shell scripting experience always comes in handy – as you can see!
New update – should now check to ensure it is user PI that has logged in! If root and there is no user PI it should create one and ask you to log out and try again. !! Thanks to Mr Shark for this one.
And while we’re talking about SBCs – I just found a fantastic array of boxes for them… CHEAP too.
https://www.aliexpress.com/af/din%252Brail%252Benclosure/4.html?isAffiliate=y&initiative_id=SB_20161228055832&site=glo&g=y&SearchText=din+rail+enclosure&needQuery=n
Pete,
There’s a typo that prevents NPM from updating:-
Line 268
“upgradednpm” “Upgrade NPM to latest version ” ON \
I think that this should read:-
“upgradenpm” “Upgrade NPM to latest version ” ON \
to match line 715
if [[ $MYMENU == *”upgradenpm”* ]]; then
Fixed – thanks.
Also, lines 721 and 722 should be swapped?
720 # Add CU to enable serial VT100 mode for terminals
721 printf “${ICyan}============================ Installing CU…etc
722 if [[ $MYMENU == *”installcu”* ]]; then
Yes, done 🙂
Pete,
The ‘CANCEL’ works – all the time – even if I tab to ‘OK’ & hit enter.
Yup, I could see that. That was this morning’s addition – fixed…
I have a question..
My rpi with ‘the script’ has a blinking power led.
Everything I read suggests a blinking led means a power supply issue and the device doesn’t boot.
Mine is working fine, although the one sonoff I have connected frequently reports lots of lwt messages inferring disconnects.
I tried a 24 hour ping test, none missed.
So is there anything in ‘the script’ which could be using the power led to report something?
Absolutely not, nothing touches the LED. I have a Pi4 and a pair of Pi3s running the same code – on flashes, the other two stay on constantly. Power supply issues?
what psu are you using? The led says you’re underpowered… if going in graphic mode you should have a low voltage indicator, look image…
here you’ll find a python script to report undervoltage via ssh: https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=82373&start=75
use a good psu, bad ones are 1st cause of sd data corruption…
more info: https://tech.scargill.net/raspberry-pi-3-4-battery/
and on many other blog articles
Thank you for the pointers. I was confident that it wasn’t the power supply as I tried a couple.
What I didn’t do was replace the ps cable. Which should have been obvious as many are below standard to say the least.
Pete,
OK – I’m testing on the pi 3.
Line 534
if [[ $OPSYS == *”RASPBERRY”* ]]; then
Is there a NEW OS? 🙂
No wonder we couldn’t land something on Mars… must’ve been late at night. Fixed. Thank you. Keep going…
Hi Pete,
Here’s another one:-
Line 255
MYMENU=$(whiptail –title “Main Raspberry Pi Selection Checklist” –radiolist \
This only allows you to check ONE option.
I think it should be :-
MYMENU=$(whiptail –title “Main Raspberry Pi Selection Checklist” –checklist \
Thanks for the ‘CANCEL’ handler.
Keith
Wow – how did THAT get in there (we’re testing non-pi boards right now)- thank you Keith – fixed.
Hi Pete / Antonio,
I’ve used Pete’s script several time in the older format and had few problems. I ran the new one this morning with the menu system and found this problem on my Raspberry pi 3 loaded with Raspbian:-
The script does not detect my Raspbian OS correctly. It thinks that it’s not a genuine Raspbian OS. Here’s my OS:-
pi@raspberrypi:~ $ cat /etc/os-release
PRETTY_NAME=”Raspbian GNU/Linux 8 (jessie)”
NAME=”Raspbian GNU/Linux”
VERSION_ID=”8″
VERSION=”8 (jessie)”
ID=raspbian
ID_LIKE=debian
I changed this line in your script:-
if [[ $OPSYS == *”RASPBIAN”* ]];then
to
if [[ $OPSYS == *”raspbian”* ]];then
and it detects my OS correctly.
Many thanks to both of you for your great work.
Keith
Correct, Keith – I’ve now put a case converter in to make all results upper-case – so that should not happen again – please report back any other issues as this is all new. I’ve also just put a handler in for the CANCEL option which now works.
I’ve been doing lots with running Ubuntu(Kubuntu,Lubuntu,Xubuntu,etc) from liveCD ISO files on my HD and with a partition or USB thumb drive labeled casper-rw there’s an overlay(persistent) filesystem capability too.
So my thought is, instead of installing Ubuntu in a VM, why not download the Ubuntu ISO file and setup a VM with the CDROM pointing to the file. Then create a small 8GB hard disk image and boot the vm into the live mode. partition and change the label of the 8GB partition to casper-rw and reboot. Now you have a quickly booting Ubuntu system to experiment loading up your home automation stuff(docker, node red, etc).
For the noobs it gets them through the installation process fast and lets them play in Ubuntu land for a bit. If they like it, they can then generate another VM but this time go for the installation of Ubuntu so it’s more able to handle updates.
or just use pendrivelinux to create the same thing, way easier (just a box to check):
the purpose of the vm was to speed up the script development, not to have a persistent or production environment… what we did in a weekend could have taken weeks on an actual SBC…