Backups the easy way
One of my biggest issues with the Pi has always been – backups. Using a PC to backup a 16GB SD is a pain and of course necessitates turning the Pi off to remove the SD. Well, now there’s a better way.
A slightly less painful way than backing up the whole SD with a PC is to use BerryBoot which can do a good job of running Raspbian and at power-up giving you the option to store a compressed version of the SD to USB memory.
It was with sadness then that I updated Raspbian today to the latest version.
sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install piclone geany usb-modeswitch
sudo apt-get install python-pigpio python3-pigpio
Nice theory but in practice, on reboot, I ended up with an utterly dead board – something to do with the update and BerryBoot. I’ve reported this to the BerryBoot author.
But here’s the thing – the latest version of Raspbian Jessie now for the first time includes a live backup facility. Once you have this you’ll wonder why it was not always this way – especially if, like me you’re not a Linux fanatic.
It does not stop there however. Not only will this new utility copy from live to SD (I’ve yet to figure out how to run the SD copier from the command line but it runs from menu – accessories – SD card copier without issue) but also – and I’m quite excited about this – you don’t need the same size SD – which means expansion or reduction is now a snap – thank heavens for that.
So in order to test this new backup facility – I grabbed the latest Raspbian (not the NOOB version) and blew it onto a16GB SD with Win32DiskManager as usual. I started up the Pi3 and put another identical SD into a convertor and stuck the USB convertor into a spare USB slot. I ran the backup, powered down, swapped SDs over and rebooted – no problem at all. Fantastic.
For my second test – I ran my script on the Raspberry Pi 3 so it was fully up to date with all of my tools – Apache, PHP, Mosquito, Node-Red, SQLite etc. – on a 16GB card. I then made a backup the same way as above to an 8GB card… the transfer took place while writing this blog – all in around 15 minutes.
I powered off the Pi3, put in the 8GB card and powered up. IT WORKED!!!!
With that success behind me I decided to get brave. I took the new card and put it into a Raspberry Pi 2 – surely this could not possibly work? IT WORKED!!!!
For some this may be nothing – for others a life-saver – so now it is easy – without even powering down to make copies of a working system – even on different size cards and it would seem interchangeable between Pi2 and Pi3. Other Pi-like manufacturers take note!! Now I can be WAY more adventurous as it is so easy to make backups.
PigPio
While I was on I thought I’d try the new pigpio GPIO interface.
Firstly I started up the pigpio daemon – at this point being new to me I have no idea how to make this run at power up – nor do I know if there is a reason you should NOT run this at power-up.
sudo pigpiod
Bear in mind that everything was originally set up on a Pi3 – but by this time I was running the SD on a Pi2, not the Pi3. I like doing things interactively when I’m learning so I simply typed python <enter>
I was greeted with >>
I fastened a LED/resistor combo (I always keep a few ready with female connectors on them for testing) between connector pin 6 (ground) and 12 (GPIO18).
>> import pigpio
>> mypi=piggpio.pi()
>> mypi.write(18,1)
And it worked – the light came on and off when I used the same command with 0 as the last parameter. I’m dying to try the other more complex commands in a Python program. indeed today everything just seems to be working – the sun is shining, my gadgets are working – what more could one want. I have one duff Electrodragon (more on that later) but apart from that…
Recently I switched to another way for my stuffs running on Raspberry Pi: put everything into a docker container, then I can do live backups with pretty small size. (about 1GB for my home automation related stuffs). If I screw up something, I just revert back to a previous copy of container. Also can have script to automate the backup.
A downside is that it doesn’t backup the host OS (I use OSMC), for that I just backup Kodi settings when needed.
I use Xbian for media centre (kodi based) and they use BTRFS as thier file system. This can be set up to make backups from a live system to a network server/NAS.
Mine are set so that each pi backs up to the server in the early hours every day, but only keeps the last 7 images (to save space).
It works well.
For Linux backup I swear by http://www.hashbackup.com/
Not quite what you’re wanting here but for incremental backups even to S3 or B2 great!
Not sure if you noticed it in the announcement of this new version Pete but down at the end of the announcement was a short note about an additional feature. This copies a wpa_supplicant configvfile from boot to where it should be. This means you no longer have to have a keyboard and monitor attached at any stage of setting things up.
I’m sure a lot of folks on here have learnt the trick of keeping a copy of their setup and working from there but now you don’t even have to do that, just keep a copy of your wpa_supplicant file and remember to copy it to boot after you’ve flashed the SD card.
Simon
I’d missed that Simon – so even MORE reasons to upgrade – this easy backup will allow people to be even more experimental sure in the knowledge that they can pop the previous SD in should things go wrong… marvellous stuff. I need to work on my script now to automate even more of the setup – like the security files for Node-Red which I still do manually.
If you like to do things interactively you really need to look into ipython (command line or Jupyter Notebook).
I took a look – I could not even figure out what platform it’s supposed to run on. They make a LOT of assumptions about people’s knowledge.
It runs on any platform that runs Python. Installing it is quite simple: https://ipython.org/install.html
Then you just run ipython instead of python whenever you want to use Python interactively, or ipython notebook if you want a nice web-based interface. In any case you get tab completion, online help, etc., etc.
It’s a bit strange that the Jupyter (formerly known as IPython Notebook) instructions recommend the use of Anaconda, even on Linux: https://jupyter.readthedocs.io/en/latest/install.html but as you can see you can also install it using pip.
If you instead want a really outdated version you can also install it via apt-get.
Another way to do backups would be to use dd and copy the whole SD card while it’s inside the Pi while simultaneously running a NFS share.
DD ?
dd is the command line tool for Linux that does the same thing as Win32DiskImager in Windows.
Let’s say you have a network share to your Windows computer mounted in /mnt/winshare inside Raspberry Pi. The following command would backup the SD card directly to your computer.
sudo dd if=/dev/mmcblk0 of=/mnt/winshare/backup.img bs=16M
If you want a progress bar.
sudo apt-get install pv
sudo dd if=/dev/mmcblk0 | pv | dd of=/mnt/winshare/backup.img bs=16M
I just did this upgrade, but skipped line 3 because those tools are all GUI and I’m running Jesse Lite. Like you, I’d love to have a command line version of piclone.
I took a quick look at the piclone source. There is a “backup” script, it does the usual linux stuff to make a backup. Does this script do what piclone does? https://github.com/raspberrypi/piclone/blob/master/src/backup
I asked for a command line version of piclone on the Raspbian forum. Hopefully they get on that quickly.
Spent a little time comparing the “backup” script to the source for piclone and they share a lot of commands – looks to me like they do the same thing. If I can find my USB SD card adapter I’ll give it a try. Happy to have others try as well.
This might be based on https://github.com/billw2/rpi-clone
Looking at it, the GUI might only provide an easy way to input the source and destination locations.
That program – so it says, must be run as root – I’m not running this as root…
Hmm okay, I’m genuinely curious what’s behind then. I tried to find some source code or description of it, without success though –> i.e. nothing more as already linked above.