Raspberry Pi Monster Script Part 1

It all started innocently enough…

Updated 10/10/2015: All I wanted to do was control some ESP8266 and Arduino boards – I spent hours looking at various packages and ultimately decided that none of them really did what I wanted – I resolved to learning about the Raspberry Pi (though frankly any small Linux device would have done – but for power to cost ratio the Pi2 is pretty hard to beat unless the Orange Pi comes up trumps  – still waiting for delivery).

Having gone through the cycle of installing Raspbian on my little Pi2 and then manually installing the likes of Apache, PHP, Node-Red, MySQL and more, over and over again either because I screwed up the installation or tried an experiment too far…. after a while I was really starting to tire of the repetition. I decided to look into SCRIPTS.

So I’ll try not to make too many assumptions here, I’ll assume you are able to get Raspbian on your Pi2. I’m a Windows user so I’ll explain this from the perspective of someone sitting at a Windows machine though most of this does not concern such dependency.

So typically looking around, having installed the basic setup and now wanting to move on, people use the default PI login and then use the SUDO (Super User Do) command to give root privileges – so – I made the decision to log in as ROOT and save a lot of typing – still – you’ll note I used it so often I actually know what it means – that’s sad). I’m sure I’m just about to get arguments back about security. Save it guys.

SO – following this article you should be able to end up with a Raspberry Pi setup like mine which has all the tools I need to make a central control base for home control.. but the point of the article really is to give you a base so you can make your own script to do what you want.

My installation consists of the following:

• The latest Raspbian on the latest base.
• TightVNC Server (for remote PC graphical access which I use less and less)
• Apache webserver with PHP
• SQLITE server with some basic databases
• Sqlite with PHPLiteAdmin
• Webmin
• Node
• Node-Red
• A ton of nodes for Node-Red including my own, nodes for MySQL, SQLite, Email, Serial and much more
• Mosquitto MQTT
• I’ve installed the Internet time updating software for Webmin so that the Pi2 will go off to the web for the correct time
• Email (SMTP)
• Sound including speech and MP3

So clearly some questions come out of this – why two types of SQL? I started with MySQL – a powerful database which works well on the Pi2, but I starting to get concerns about it’s effect on the SD memory – MySQL does a lot of writing.  With that in mind I went off looking to alternatives and found SQLite – which is very different, not entirely compatible but lends itself to being a lot lighter on SD writes!

Getting Raspbian onto the Pi in the first place

First things first,  I’m assuming you can put Raspbian Wheezy on the Pi2. There is a slightly later version but I had minor issues, it was new so I figured… let’s stick with the one that works for now. There are a ton of ways to do this – usually involving formatting an SD with SDFORMATTER then copying some files over to the SD, putting it into the Pi2, turning on and following instructions. EASYBOOT is my choice and is Excellent as it not only does the downloading of Raspbian for you but also allows you to do backups to USB memory. Format the SD, grab the EASYBOOT zip, copy the files onto the SD and run the PI with network connection – I prefer wired – select Raspbian Wheezy and some time later when it is ready to go, make it the default and reboot – and you’re up and running.

PERSONALLY at this point I reboot again – and in BerryBoot do a backup to USB memory – makes things a lot easier if you screw up later on.

You can run Raspbian from the SD, or from a memory stick or even from a Diskstation or similar on your network…   If you are running off the internal SD card, buy a decent, fast, brand name SD.  I’m using 16Gbyte SDs and I have a 16GB USB in there too for backups (the backups don’t take the full 16GB, mine tend to be 2-3gig at a time).

I have an installation which starts on a card and actually runs everything on a 2.5” hard drive – an old 120GB job.  You could use a larger SD but then if you wanted to back this up with something like Win32DiskManager, it just takes that much longer. Windows tools for backing up this stuff are not very sophisticated. Berryboot 2 is the best solution I’ve come across so far.

The next step

So you’re sitting with a Pi2 and some kind of monitor, it has asked you to login as Pi user and you’re there with a prompt or Raspi-Config running. If not – you can just run “sudo raspi-config”. You’ll probably want to change the password for PI user and under advanced you may or not want to enable SPI, I2C and SERIAL. Unless you have a reason not to, I suggest enabling them. You may or may not wish to use the last option to update the raspi-config package itself. If not you can always do that later.

Back at the prompt:

sudo passwd root

This will ask you for a password – and in my case that is the last time I ever have to use “sudo”.

AT THIS POINT you do not ever need to look at the Pi2 through it’s own monitor again except for backup and restore – you can do it all from a PC – which most likely means a bigger screen, better keyboard and mouse !

So, when the Pi was powering up I hope you made a note of it’s IP ADDRESS as you’ll need that.  Indeed if you know how to fix IP addresses in your router, I suggest doing that so the IP address will remain fixed. You can do that all in the Pi but it’s a lot easier if your router supports associating an IP address with a MAC address – to ensure you get the same address whenever you power up. If you never intend to access the Pi externally all of this is probably irrelevant – but if you want external access then you’ll need to fix the address so that you can set up port forwarding on your router.

Armed with an IP address, on the PC, grab WinSCP and also install Putty somewhere on the PC. Why? Because WinSCP is the best way by far to access the Pi (easiest) and it has  rather awful terminal but will automatically use Putty if it is available – trust me – copy-paste, ease of use, I’ve tried LOADS of programs and after months, this combination works best.

Graphical desktop or Terminal?

When I started off, I wanted graphical access on the Pi so everything I did was oriented that way. Experience has shown that this is not a good idea… why? Well, the editors on the Pi are NEVER going to be as good as the tools you likely have on your PC.  The browsers for the Pi are AWFUL – far better to access on a PC with a proper browser – and so on and so on. So now, with the benefit of experience I’m wondering why I was so intent on filling up the Pi with visual tools I was unlikely ever to use. Don’t get me wrong, remote access to the Pi will be very visual, no more so than with Node-Red – but that is browser-based, no need for graphical ability on the Pi itself.

WinSCP is easy enough to setup – you mainly tell it about the IP address of your Pi (or PIs if you have a few) and the root name (“root”) and password you setup earlier – that’s it. From there on you are in a file explorer-type window with drag and drop and if you need a terminal it’s a button press away.

Typical login screen above – on the left, my list of devices (WinSCP does a lot more than just talking to Pis – I use it to connect to my service provider, my DiskStation and much more.  In this case, the protocol is SCP (this is already built into the Pi) I provide IP, user name and password and that’s it.

Here we are connected up to one of my Pi units – as it happens the one that uses a hard drive – though that is utterly irrelevant here as Linux doesn’t use “disk drives” as such – everything is just a folder. the BOOT folder happens to be on the SD and everything else happens to be on the hard drive – but that’s quite irrelevant here. Left hand side is my PC, right hand side is the Pi. You can move files back and forth or indeed take files and folder on your desktop and drag and drop into the right window (but beware, PERMISSIONS on Linux are not the same as a PC – I’m making life there a little easier by being ROOT user at the expense of a little security).

So there are two or three main methods of installing stuff – one involves APT-GET (Advanced Packaging Tool) which will go off and get an installation for you. Another involves WGET which basically grabs, say a ZIP file or a whole directory and dumps it on the Pi, so you can subsequently unzip it and put it’s innards somewhere. It is worth while looking up these two commands as you’ll see them a lot… mind you – the point of a script is so that you don’t actually have to know much about this – but really if you want to modify my work to suit your own needs, the more you know the better and no I’m NOT a Linux fanatic, I’m a PC guy…  but a little knowledge goes a long way.

Once you have NODEJS up and running there’s a package called NPM which is also used for installation. But don’t let any of this worry you.

So a SCRIPT is a plain text file, usually with suffix .SH which you run – and it goes off and does things for you almost as if you’d typed instructions by hand – but with a lot more capability such as variables and functions. I had to learn this the hard way – all I’ll say is thank goodness for Google.

Here’s a simple script we’ll call “test.sh” which contains:

echo “hello”

and when you run the script it will do… nothing. My first run-in with permissions. A script has to “execute” and if you don’t give it permission to do that, it will fail by default!  But hey, WinSCP to the rescue./

So I’m assuming at this point you have Rasbian running, you’ve done the root password thing and you’re sitting with WinSCP waiting to have some fun.

I use NotePad++ on my PC for good reason and so I’ll assume you have that.

You should have put PUTTY somewhere on your PC (the one I found didn’t have a proper installation program so I just put it in Program Files(86) and under  PREFERENCES – integration, applications tell it where PUTTY is. Under PREFERENCES in WinSCP you might want to set the default EDITOR to be Nodepad++

Here we are creating a file on the Pi. It can be edited on the PC just by double-clicking it and when you hit save it will save back to the Pi.

So – TWO things we need to get out of the way… The file (test.sh) needs PERMISSION to execute and needs to be saved in a LINUX format – that is with new lines (\n) but no returns (\r)

When editing a file in NotePad++, the latter is easy. EDIT – EOL CONVERSION – UNIX – simple. Here’s our file.

In the file put the echo statement above.  Save (control-S). You should now see the file on your Pi.

So now we access the Pi – under COMMANDS – OPEN IN PUTTY – you gain access to your PC for actually DOING things – it is going to ask you for the root password you created earlier – I’ve not found a way around this.. You don’t need to get into any Putty setup nonsense – it just opens up , you enter a password and you are in.

At this point I should explain PASTE in Putty… anything in your clipboard – just right click the mouse and it will be pasted into the command line in putty. Easy.

And so here we are in the root directory… you will try your new test.sh file – and it will fail.

I think this was the point at which I considered giving up and installing Windows on the Pi – but only briefly.

Back in WinSCP –  right click the file – PROPERTIES – thank heavens – tick-box properties –

See that X I’ve ticked for user OWNER – that will allow scripts to operate.

and now….

Doddle. So now we have the ability to edit a text file – and make things happen in a script!

In the next part, we’ll go into this in depth and end up with a working script which will install all the goodies I’ve mentioned above – reducing the complex job of installing loads of packages to basically a nice coloured screen with the need to type in some passwords while drinking coffee.

Honestly it gets easier and a lot more fun from here on. And now – onto part 2. https://tech.scargill.net/raspberry-pi-monster-script-part-2/

But please note that “the script” has moved on a LONG way from this – see the current article on the subject..