Easy Backups and Webmin

This article started off as two separate subjects – and has now gone SO much further – you really need to read this one…

So I’m messing here with a NanoPi NEO PLUS2  (though this applies to other board too) – and I’ve had all sorts of issues getting SSD1306 drivers working on them – well, not so much getting them to work – that’s easy – more stopping this infernal “segmentation error”. I needed a backup solution. Well, I found more than one !

So it occurred to me that if I had 2 identical boards, I could back one up to the other a hell of a lot more quickly than copying SDs – while  testing ideas which might go wrong.

Antonio and I came up with some sync code using RSYNC – that could be run on a good machine and, when pointed to the machine I’d just messed up, would delete anything new I’d put on that machine and restore it to a perfect copy of the master  – except of course for the hostname which I’d rather leave alone!  We wanted this to be an alias rather than a script – and along the way discovered that you can’t send parameters from the command line – to an alias.

But you CAN send parameters to a function – to old Linux hacks this is common sense – but to me this was brand new! SO – check this out.

syncto()
{
if [[ $1 = “” ]]; then
echo “Sorry – no IP specified. Use syncto ip_address”
else
sudo rsync -aAXv / –delete –exclude={“/run”,”/etc/hosts”,”/etc/hostname”,”/dev/*”,”/proc/*”,”/sys/*”,”/tmp/*”,”/run/*”,”/mnt/*”,”/media/*”,”/lost+found”} root@$1:/
fi
}

A lot easier than it looks, rsync is a standard synchronisation tool for Linux and is here copying remotely over the network from the current machine to the one specified by the IP address (can’t seem to use hostnames – tell me if I’m wrong) but excluding several directories which you don’t need to copy/sync and the options include verbose and stopping it over-writing the boot partition – so just the main file system.

So you can just run this function from the command line in Ubuntu (and most likely Debian as well) and nothing will happen – it merely stores the function. You could store it in /etc/bash.bashrc for permanent use –  as root (or with sudo) just add it to the end of etc/bash.bashrc and start up a terminal – and you’re done.

To use – let’s say you are sitting on the good machine and you want to send updates to the one that is all messed up.  Let’s call that machine 192.168.1.29

syncto 192.168.1.29

That’s it – I’m assuming you have SSH on both machines – I have it on all of mine (openSSH) so I can remote into them  – so unless you have certificates set up – it will ask for the host password for the other machine – and then promptly go off and ensure the other machine has any new files deleted and anything missing added so it becomes a copy of the machine you are on. Reboot the destination machine once finished and you’re done.

Node-Red users may at this point say “but – the flow files contain the hostname- they will be wrong if you backup to a differently named unit” – well, no. According Dave C-J of Node-Red fame, in the Node-Red settings file you can simply put “flowfile: ‘flows.json” and do away with the hostname dependency – assuming you are also backing up the settings.js file of course-  which in this instance you are.

But that brings us to Webmin – something else I usually install in “the script”.  Well, it turns out that Webmin is constantly making file changes all the time – something I’m not happy about. now I know how often it is doing it.

If you have Webmin installed, to stop Webmin coming up at power on you need this..

sudo update-rc.d -f webmin disable (or enable).

To actually stop and start webmin…

sudo service webmin stop  (or start)

For now I’m leaving it off by default and turning it on when I need it. Don’t like constant writing to SD.

So this is GREAT for copying to another machine on the network – but what about cloning?

Cloning to SD – just like the Raspberry Pi SD backup – at last

The Raspberry Pi has a great desktop-based cloning tool for SD which makes a copy of a working system – now – don’t lecture me about cloning live systems – we know it has potential to make a bad copy due to changed files – but if you ensure you keep activity to an absolute minimum – it does seem to work.

But then – what about the likes of the NanoPi which is not a Raspberry Pi and not 32 bit but 64 bit etc.. Well, it turns out that at least on the NanoPi Plus2 and hopefully on many other boards – the command line drive rpi-clone works a treat.

This cloning tool is NOT like copying an SD where you have to have exactly the same size SD. I copied an original system based on a 16GB SD (but not using anywhere near half of that) to an 8GB card and then removed the original SD, plugged in the new, smaller one which then just  worked – the copying process took about 12 minutes in my case. Stunningly useful if you’ve not had this before.

https://github.com/billw2/rpi-clone – this worked – but lacked amazingly lacked a –delete option which meant the clone if used over and over would actually end up getting bigger as any files deleted on the original would be left on the copy!!  Also there was no unattended version which limited it’s use for incremental syncing … and someone had suggested adding a hostname change – but this had not been implemented and the pull request version had a bit missing anyway and…

To cut a long story short, I pulled a copy of this excellent script, added the bits I needed and the hostname option and now it sits here…  https://github.com/scargill/rpi-clone/blob/master/rpi-clone

This works a treat – well, on a NanoPi Plus2 anyway. Stick an SD into a usb slot – and off you go…   rpi-clone sda (assuming that’s the name that appears in your /dev directory when you plug in the SD)  – add options –d to delete stuff on the clone that has been deleted in the original –u for unattended cloning once you’ve done the first one (incremental backups only take a minute or two) and –o for Hostname change in case you want to do that (you’ll then be forced to supply a new hostname).

But before you go rushing off – we did contact Bill (the original link) and not only did he confirm that the delete code is indeed built into his original – but he will be putting our changes into his code – I’ve supplied him with info on the NEOs (who’s boot looks nothing like Pi but the general 2-partition principle is the same) and he’s going to be doing a new version – so keep an eye on that software. This has already been a god-send for me after experimenting and messing up the operating system, being able to just pop another SD in and also back that up pretty quickly is wonderful for experimenters.

All of which is just marvellous – all it needs is a little colour – but I’ve tackled that elsewhere.

Update September 2018 – RPI-CLONE 2 incorporates some of the points raised in this article and works a treat for Raspberry Pi. For other boards, especially those with eMMC, I’ve had to go looking elsewhere – Armbian – often the best operating system solution for many SBCs which may get little software support from their manufacturer (Orange Pi for example) offers a copy-from-SD-to eMMC option and as it turns out, given a very basic installation of Armbian, on for example the Orange Pi +2E, an RSYNC solution can be made easy to implement. See EASY BACKUP 2. Not QUITE as easy as rpi-clone but then this does handle eMMC installations.