No doubt there are many of you out there with Raspberry Pi Single Board Computers who have for some time, like me been using “rpi-clone” to successfully clone/backup their PI to known good SDs like Sandisk or Samsung SDs… maybe initially with Raspbian Stretch or even earlier versions… and who have now moved to the “Buster” upgrade as recommended by RPI. Well – there is (was) a potential problem using RPI-Clone if you took the upgrade (as against fresh start) approach as many of us needed to do in order to preserve months or years of work. Read on over the break…
I've now cloned my RPI systems more often than I could possibly count.
In creating the clone, the procedure varies depending on whether you are cloning to a new SD/USB or one previously used on the same (partition) setup. When making a new clone where the destination was not previously initialised with the same setup, it is necessary to reboot then use the -f for FORCE option in RPI-CLONE. That option forces resizing of the ROOT (file) partition to handle differing SD sizes (larger and smaller) and it does it well and is only needed for SDs freshly introduced. Up to now this has worked as the size of the BOOT partition (small) has been consistent but now on BUSTER it has grown. If you attempt to clone from an original which was upgraded from Stretch without expanding the BOOT partition, then the clone may well fail. At the time of writing this blog entry, RPI-clone does not know about changing the size of the BOOT partition. By September 2019 this had been resolved so see the later blog entry who's permalink is here.
So, my 16GB original SD had an upgrade to Buster on it but without a change to the BOOT partition size. Good enough to run BUSTER but NOT a large enough BOOT partition to handle the cloning procedure as well. I initially thought that earlier (upgraded Buster) clones using -f were forcing the new SD to resize the (FAT) BOOT sector to the larger BUSTER standard of 120MB, as against the earlier 44MB of Stretch, in fact the clones STILL had only 44 meg BOOT sectors.
As RPI-Clone did not (at the time of this blog entry) handle the boot partition resizing, I ended up putting my original (well, a clone) clone into a new RPI4 USB socket while running the RPI4 from a new Buster installation on SD. I had to manually add (using the RPI terminal program – easy aside from the pain of attaching a screen and keyboard to the RPI4) GPARTED to the SD installation then running that with my SD in it’s USB adaptor – and CAREFULLY using GPARTED to firstly take off 200 MEG from the end of partition 2, then moving P2 right by the same amount. I then expanded P1 from 44MB to around 130MB without shifting it’s left position. I then saved that new modified Buster clone, powered off, put my SD into the PI4 SD slot and powered up. From there I could make a NEW first clone using the RPI-clone –f option. Having now prepared a clone with the right BOOT partition size I could now make a test change to my original clone and use the second clone to re-clone WITHOUT a reboot or –f option.
Here in more detail from the start with a desktiop RPI on a virgin Buster…. in the terminal…
- sudo apt-get install -y gparted
- sudo gparted
I then put into the USB socket a working upgrade clone – which had the partition SDA1 at 44236 blocks size (1k SD blocks hence 44 MB BOOT partition)
and told GPARTED to process SDA rather than it’s working operating system on mmcblk0
I then used the PARTITION – UNMOUNT command to remove the little keys from the partitions, i.e. to ensure the SDA partitions were not mounted, After that I shrunk SDA partition 2 (SDA2) by a roughly a couple of hundred meg and shifted it’s start position right accordingly. I moved the right side of SDA1 right to increase SDA1 to fill up the gap without shifting it’s left side.
When I set this off going, SDA2 resized, took about half an hour, but when it came to SDA1, it failed, maybe I was supposed to leave a gap or maybe I inadvertently shifted the left of SDA1. I tried resizing SDA1 again (very quick) and this time ensured I didn’t move the left side AT ALL and reduced the size of SDA1 to 134 Meg to leave a gap. When I set it going, everything just worked. i.e. I turned off the Pi4, swapped SDs and the newly size-changed SD with all my data still intact just worked.
I rebooted and made a clone of the clone using –f option. So now I am able able to use THAT clone to make another….. I powered off, swapped SDs, cloned again with no –f option…. this would be quick, but I’d rebooted. 5 minutes later I had my clone – but could I do it again, this time without the –f option again AND without without a reboot or –f option? YES.
I now had a complete upgrade from Stretch, using Buster and able to clone, without losing any of my data or setup.
there is a lot of arrogant and often incorrect or out-dated nonsense out there about backing up/cloning Raspberry Pi. My subscribers deserve better. The informatiuon you see here is how I handled the boot partition resizing until BillW (rpi-clone) added the option to handle this to rpi-clone itself.
Having now read the RPI-CLONE README properly (not easy) it turns out there is an –s option you can put on the far right of the rpi-clone command to alter /etc/hosts and /etc/hostname if you want to change the hostname of your clone/copy. Please note that an alternamive way to do this - nmtui - does NOT update the hosts file.
In coming to the conclusions above, I tried the commands suggested below: on my Pi3 with a known good SD which only the previous day had made a “good” backup on that same Pi3:
and the result of the test was:
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/root 971040 268781 702259 28% /
devtmpfs 123728 379 123349 1% /dev
tmpfs 124880 10 124870 1% /dev/shm
tmpfs 124880 556 124324 1% /run
tmpfs 124880 3 124877 1% /run/lock
tmpfs 124880 14 124866 1% /sys/fs/cgroup
/dev/mmcblk0p1 0 0 0 - /boot
tmpfs 124880 13 124867 1% /run/user/997
tmpfs 124880 13 124867 1% /run/user/1000
which seemed fine – then this test:
sudo tune2fs -l /dev/sda
Second test failed (for reference) as it looks like you can only test partitions, not the complete SDA.
SO I completely shut down my PI (sudo shutdown now) then turned off then turned back on…
I am of course doing all of this via my PC, using Mobaxterm in an ssh session with the headless Pi as usual…
In case anyone thinks this means a lot of typing... here’s my latest list of aliases… in /etc/bash.bashrc go to /etc and edit with “sudo nano bash.bashrc” - this lot are on the end of the file.
alias mqttlog='tail -f /var/log/mosquitto/mosquitto.log | ccze -A'
alias nrlog='sudo journalctl -f -n 50 -u nodered -o cat | ccze -A'
alias space='df -h|grep -v udev|grep -v tmpfs|grep -v run'
alias cls='python /home/pi/cls.py'
alias stop='sudo shutdown now'
alias boot='sudo reboot'
alias clone='sudo rpi-clone -U sda'
alias cleanclone='sudo rpi-clone -f -U sda'
I’m about to change clone and cleanclone to add the –s flag on the right with new manually added hostnames just as soon as I figure out how to add arguments to aliases.
When I make clones now I see, using my PARTITIONS alias…
major minor #blocks name
1 0 4096 ram0
1 1 4096 ram1
1 2 4096 ram2
1 3 4096 ram3
1 4 4096 ram4
1 5 4096 ram5
1 6 4096 ram6
1 7 4096 ram7
1 8 4096 ram8
1 9 4096 ram9
1 10 4096 ram10
1 11 4096 ram11
1 12 4096 ram12
1 13 4096 ram13
1 14 4096 ram14
1 15 4096 ram15
179 0 15558144 mmcblk0
179 1 131072 mmcblk0p1
179 2 15261184 mmcblk0p2
8 0 31260672 sda
8 1 131072 sda1
8 2 30963712 sda2
See mmcblk0p1 and sda1 – at 131072 – well, under STRETCH those blocks WERE 44,000 (or 44MB) or thereabouts, no good for Buster and I kept cloning the OLD boot size to the NEW copy – NOT big enough. Not any more.