Easy Backups and Webmin

This article started off as two separate subjects – and has now gone SO much further – you really need to read this one…

So I’m messing here with a NanoPi NEO PLUS2  (though this applies to other board too) – and I’ve had all sorts of issues getting SSD1306 drivers working on them – well, not so much getting them to work – that’s easy – more stopping this infernal “segmentation error”. I needed a backup solution. Well, I found more than one !

So it occurred to me that if I had 2 identical boards, I could back one up to the other a hell of a lot more quickly than copying SDs – while  testing ideas which might go wrong.

Antonio and I came up with some sync code using RSYNC – that could be run on a good machine and, when pointed to the machine I’d just messed up, would delete anything new I’d put on that machine and restore it to a perfect copy of the master  – except of course for the hostname which I’d rather leave alone!  We wanted this to be an alias rather than a script – and along the way discovered that you can’t send parameters from the command line – to an alias.

But you CAN send parameters to a function – to old Linux hacks this is common sense – but to me this was brand new! SO – check this out.

if [[ $1 = “” ]]; then
echo “Sorry – no IP specified. Use syncto ip_address”
sudo rsync -aAXv / –delete –exclude={“/run”,”/etc/hosts”,”/etc/hostname”,”/dev/*”,”/proc/*”,”/sys/*”,”/tmp/*”,”/run/*”,”/mnt/*”,”/media/*”,”/lost+found”} root@$1:/

A lot easier than it looks, rsync is a standard synchronisation tool for Linux and is here copying remotely over the network from the current machine to the one specified by the IP address (can’t seem to use hostnames – tell me if I’m wrong) but excluding several directories which you don’t need to copy/sync and the options include verbose and stopping it over-writing the boot partition – so just the main file system.

So you can just run this function from the command line in Ubuntu (and most likely Debian as well) and nothing will happen – it merely stores the function. You could store it in /etc/bash.bashrc for permanent use –  as root (or with sudo) just add it to the end of etc/bash.bashrc and start up a terminal – and you’re done.

To use – let’s say you are sitting on the good machine and you want to send updates to the one that is all messed up.  Let’s call that machine


That’s it – I’m assuming you have SSH on both machines – I have it on all of mine (openSSH) so I can remote into them  – so unless you have certificates set up – it will ask for the host password for the other machine – and then promptly go off and ensure the other machine has any new files deleted and anything missing added so it becomes a copy of the machine you are on. Reboot the destination machine once finished and you’re done.

Node-Red users may at this point say “but – the flow files contain the hostname- they will be wrong if you backup to a differently named unit” – well, no. According Dave C-J of Node-Red fame, in the Node-Red settings file you can simply put “flowfile: ‘flows.json” and do away with the hostname dependency – assuming you are also backing up the settings.js file of course-  which in this instance you are.

But that brings us to Webmin – something else I usually install in “the script”.  Well, it turns out that Webmin is constantly making file changes all the time – something I’m not happy about. now I know how often it is doing it.

If you have Webmin installed, to stop Webmin coming up at power on you need this..

sudo update-rc.d -f webmin disable (or enable).

To actually stop and start webmin…

sudo service webmin stop  (or start)

For now I’m leaving it off by default and turning it on when I need it. Don’t like constant writing to SD.

So this is GREAT for copying to another machine on the network – but what about cloning?

Cloning to SD – just like the Raspberry Pi SD backup – at last

The Raspberry Pi has a great desktop-based cloning tool for SD which makes a copy of a working system – now – don’t lecture me about cloning live systems – we know it has potential to make a bad copy due to changed files – but if you ensure you keep activity to an absolute minimum – it does seem to work.

But then – what about the likes of the NanoPi which is not a Raspberry Pi and not 32 bit but 64 bit etc.. Well, it turns out that at least on the NanoPi Plus2 and hopefully on many other boards – the command line drive rpi-clone works a treat.

This cloning tool is NOT like copying an SD where you have to have exactly the same size SD. I copied an original system based on a 16GB SD (but not using anywhere near half of that) to an 8GB card and then removed the original SD, plugged in the new, smaller one which then just  worked – the copying process took about 12 minutes in my case. Stunningly useful if you’ve not had this before.

https://github.com/billw2/rpi-clone – this worked – but lacked amazingly lacked a –delete option which meant the clone if used over and over would actually end up getting bigger as any files deleted on the original would be left on the copy!!  Also there was no unattended version which limited it’s use for incremental syncing … and someone had suggested adding a hostname change – but this had not been implemented and the pull request version had a bit missing anyway and…

To cut a long story short, I pulled a copy of this excellent script, added the bits I needed and the hostname option and now it sits here…  https://github.com/scargill/rpi-clone/blob/master/rpi-clone

This works a treat – well, on a NanoPi Plus2 anyway. Stick an SD into a usb slot – and off you go…   rpi-clone sda (assuming that’s the name that appears in your /dev directory when you plug in the SD)  – add options –d to delete stuff on the clone that has been deleted in the original –u for unattended cloning once you’ve done the first one (incremental backups only take a minute or two) and –o for Hostname change in case you want to do that (you’ll then be forced to supply a new hostname).

But before you go rushing off – we did contact Bill (the original link) and not only did he confirm that the delete code is indeed built into his original – but he will be putting our changes into his code – I’ve supplied him with info on the NEOs (who’s boot looks nothing like Pi but the general 2-partition principle is the same) and he’s going to be doing a new version – so keep an eye on that software. This has already been a god-send for me after experimenting and messing up the operating system, being able to just pop another SD in and also back that up pretty quickly is wonderful for experimenters.

All of which is just marvellous – all it needs is a little colour – but I’ve tackled that elsewhere.

Update September 2018 – RPI-CLONE 2 incorporates some of the points raised in this article and works a treat for Raspberry Pi. For other boards, especially those with eMMC, I’ve had to go looking elsewhere – Armbian – often the best operating system solution for many SBCs which may get little software support from their manufacturer (Orange Pi for example) offers a copy-from-SD-to eMMC option and as it turns out, given a very basic installation of Armbian, on for example the Orange Pi +2E, an RSYNC solution can be made easy to implement. See EASY BACKUP 2. Not QUITE as easy as rpi-clone but then this does handle eMMC installations.


22 thoughts on “Easy Backups and Webmin

  1. Hi can anyone help with “permission denied” when using this rsync script. I have rebuilt the destination PI with the recent script and tried root access but nothing works.

  2. Sorry don’t do linux.
    I added the syncto function to the end of etc/bash.bashrc using sudo nano.
    When I ran syncto xxx.xxx.xxx.xxx I got “command not found”
    Then I tried running just the line “sudo rsync -aAXv / ………… xxx.xxx.xxx.xxx:/”
    I got some dialogue then “Permission denied”
    Any advice?

    1. I don’t do Linux either – so I write things down when I figure them out on the chance someone who DOES do Linux has answers. No idea. I’ve never seen the point of giving PI user SUDO control then still restricting what that user can do.

  3. I am considering upgrade to Stretch, but there is a warning they changed to new db format(2nd link). I use sqlite from node-red and they say everything will be converted , but is this new db system MariaDB compatble with sqlite? Can i still use sqlite, confusing!

    1. MariaDB is a fork of MySQL and is replacing MySQL in Stretch. This should not affect anything you do with sqllite or indeed any other database except MySQL. It does mean that any MySQL databases will be upgraded a process which cannot be reversed

      1. thx for this info. The remark in the upgrade could be clearer in respect of sqlite. I can go on now.

  4. Hi Pete,

    I have more than 100 old raspberrys with old raspbian – works fine, but I like to upgrade it to the latest armbian version.
    apt-get upgrade doesn’t do it’s job because it upgrades only the installed packages, it doesn’t change between debian (raspbian) versions.

    Can I use rsync to transfer a new installed raspbian to the old one ?
    Rewriting the sd-cards is imposibile, because I don’t have physical acces to the raspberrys – I have access via ssh only.

    Maybe is a stupid solution, but please give me any other suggestion.

    1. 1st: you can upgrade jessie to stretch
      2nd: better you look out some tutorial on “screen”, if you don’t want to loose access to the remote system (some way it has to update ssh, and then you’re out unless of using screen or a parallel ssh session…)
      3rd: https://www.52pi.com/blog/7-upgrade-raspbian-jessie-to-raspbian-stretch

      not for the faint of heart…

    1. The really good thing about writing a blog is – every time I come up with an idea – someone tells me it’s already been done – which means I then don’t have to do any work… I took a QUICK look at Unison – it did seem OTT in terms of documentation but when I got to the bit about using it, looked simple enough. I may have a play later. Thanks for that – the more easy solutions the better.

  5. Right – that solves that one – perfect copy of a running (but not doing much) NanoPi Plus2 using rpi-clone – I’ll add it to the script – could not be easier – just took a 16GB original SD and cloned an 8GB working copy. Marvelous.

  6. Excellent tip!
    Just a niggle, wouldn’t it be better to call the “backupto” command “restoreto”?

    1. You are free to call it anything you wish 🙂 I’m backing up a working system to another device – hence backupto – a name which took all of 2 seconds to dream up and which should not be considered iconic.

          1. CloneTo ™ – I like it – but really it isn’t a clone as we’re only doing one of the partitions. I would LOVE something like the Raspberry Pi took which creates a new SD and copies the lot – ready to plug in.. but then that is a graphical environment tool only as far as I know. This’ll do for now and the incremental backup is really fast.

            1. It can be done, easily… 😉
              But when I’m back to work and have a spare Pi to work with…
              DD the boot partition to a new SD card, create an ext4 one and rsync the root Fs…

              1. Hold on not so fast. So… we have an SD sitting in a USB device… DD the boot partition, create ext4, rsync. all that sounds like a decent cloning system – fire some code at me and I’ll test on my two boards here – could be a good one.

Comments are closed.