RPI4 Network Lesson

This blog entry was last updated mid-September 2019 and covers a wide range of issues I’ve had and resolved since the introduction of the Raspberry Pi 4 single-board computer (SBC) and the “Buster” update to the (Linux Debian-based) Raspbian operating system, in particular in setting up the networking and hostname which has changed somewhat since STRETCH on the RPi-3. Enjoy, technical readers using the RPi may find this very useful…

On the subject of rpi-clone

I recently had an update From BillW of rpi-clone “fame”. I use this tool several times a week on my Raspberry PIs before doing updates – works a treat to make seamless clones of working systems – and despite concerns of some I’ve never knowingly lost data. However, the move to RPi-4/Buster caused some concerns as rpi-clone does not (did not) resize the boot partition.. and the new operating system version uses a larger boot partition than predecessors.

I have tested the latest revision of rpi-clone (https://github.com/billw2/rpi-clone) numbered 2.0.22, changes made only recently to add a “–p” option (without the quotes) – this will clone an RPi-3 SD and increase the boot partition to 256M, the standard for RPi-4.  I’m now using that size for RPi-2, RPi-3 and RPi-4

I use the unattended flag (-U) and in order to do the boot resize you need to do a complete copy (slow) the first time to force the 256M boot partition.

sudo rpi-clone -f -p 256M -U sda

If you wish to set up a new hostname while you are there, then add the –s option followed by a hostname. For example:

sudo rpi-clone -f -p 256M -s myclone -U sda

Now then – you MAY think it is just as easy to use the “sudo nmtui” command line utility to change the host name – WRONG – as I just found out – that utility changes only the /etc/hostname file, NOT the /etc/hosts file (don’t ask why). Using nmtui to do the job, next time you reboot after a hostname change – you’ll find a complaint on the command line.  Rpi-clone to the rescue as it updates BOTH /etc/hostname and /etc/hosts file.

When doing a “normal clone” it all gets simpler (and faster)

sudo rpi-clone -U sda

All this of course is assuming you are making a clone on SDA, typically an SD or USB stick mounted in a USB socket on the RPi..  I already cloned an RPi-3 using the new rpi-clone update – to a USB stick – and noted that the boot sector was indeed enlarged. I’ll come back on this when I’ve done the same to an SD as you cannot currently boot an RPi-4 from USB. I guess I should have done an SD backup first really.

On the utility “nmtui” and cloning

I’ve now realised that the hostname of a cloned pi should preferably be set in rpi-clone – not nmtui – as yesterday I had an issue with an RPi4 clone – I wanted to change the static IP address I’d originally set with nmtui, as the new clone was to be used on an RPi-3.

I noted on fitting the SD into the destination RPi-3, that “nmtui” had put in a temporary IP address as the original address was not working – turns out that for some reason just under the fixed IP address field, somehow the MAC address of the original board had been inserted… which rendered the entry unusable on a new board, hence the temporary address. After some head-scratching. using nmtui I removed the MAC address and saved, without removing the temporary entry. By the time I rebooted, all was well.

End of Mid-September update

In the past, I’ve set up countless RPi-2 and 3 SBCs over time, usually headless and using the file /etc/dhcpcd.conf to set up a static IP address, It worked every time. Continue reading for some hopefully useful info…

When I got my new RPi-4, I upgraded all my older systems to BUSTER, not one of the easiest things to do and indeed not recommended on the RPI-forums. (Use a clean installation, they say. GREAT unless you happen to have years’ worth of complex, only partly documented code on your RPi-2 or 3).

Well, I did upgrade, the change is all documented on the blog – and of course when it came to setting a static IP address I never thought twice about it. I used my Buster-upgraded installation several times on the RPi-4 without issue.

That was fine until recently when, frustrated with a few packages of little relevance being “held back” by “sudo apt-get upgrade” or to use the commonly accepted abbreviation “sudo apt upgrade”.

It turns out that if you use “sudo apt dist-upgrade” you get all the packages available. Following that by “sudo apt autoclean” gets rid of any old versions lying around and you’re all set.

I checked out two of my RPi-3 boards with this thoroughly up to date setup and all was well until the first time I tried this on the RPi-4 – no IP address. I don’t use the graphical desktop and hence could not even see if the board was working.

Reluctantly I fitted a screen, keyboard and mouse to the RPi-4 – and as user PI at a command prompt I could then play with the board (like I have nothing better to do). But nothing I did would bring up the Ethernet connection. I even checked /etc/network/interfaces as recommended by one chap on the web… No.

Then my friend Antonio reminded me that “the script” has a couple of commented out lines that allow for permanent setting of graphical mode and vice-versa. I put the RPi-4 into graphical mode and reset. That helped but still no Ethernet. In the setup on the graphical desktop you can set a fixed IP address. I tried that – that seemed to work – but setting up by the desktop is not ideal for me.

The dist-upgrade seemed to have enabled WICD – despite (according to Steve Lenehan in here) it being no longer supported. That being the case I was happy to stick with dhcpcd.conf – I promptly disabled then totally removed WICD. Antonio (Mr Shark concurs that WICD is now old hat and is not supported.

Solved:

A file called “/etc/wicd/wired-settings.conf” – that existed in my older images but with  IP etc set to “none” – Antonio thought that it might override “/etc/dhcpcd.conf” – well, it took over from dhcpcd.conf  – I got rid of it when I removed the /etc/wicd folder and all was well on my new RPi-4 except that…. in /etc/dhcpcd.conf, I gave the RPi the fixed address of 192.168.1.30 – that worked a treat, but sadly so did 192.168.1.20 – an address I could find no reference to in any file in /etc/ – yet there it was… I could SSH into either address.

According to the graphical desktop I’ve disabled WIFI so that wasn’t it. Turns out (thank you Antonio for spotting this) that while my manual update of /etc/dhcpcd.conf was of course working and setting up the static address as it always has, with the combination of RPi-4 and Buster somehow, nmtui (Network Manager Text User Interface) had silently added in the second address (192.168.1.20) on DHCP. For now I’ve disabled Network Manager but it looks like the real answer, as nmtui has been around for a long time, would seem to be to comment out any manual mods to /etc/dhcpcd.conf and set a static address (192.168.1.30) in nmtui.

That didn’t work well at least using an eth0 entry. What DID work well was to put a manual address into /etc/dhcpcd.conf as has always worked and killing Network Manager (once only) like this:

sudo systemctl disable network-manager

sudo apt remove -purge network-manager

sudo rm -rf /etc/network-manager

I did this remotely using Mobaxterm on my PC, after which a reboot and all was wonderful.

Ok, as an alternative this seems to work and can be done from SSH – i.e. remotely.

With Network Manager un-installed – and running on dhcp.conf, I installed Network Manager (sudo apt install network-manager) – I used nmtui to set up eth0 with a static 192.168.1.30 address…. and then:

sudo systemctl disable dhcpcd.service

sudo reboot

That STILL left me with 30 and 20 (keep up) so THIS time I deleted eth0, left “Wired Connection 1” in but made it static 192.168.1.30, then booted. All works a treat, no more 192.168.1.20

Unbelievable. So now I have Buster on RPi-4, upgraded from Stretch without starting from scratch and it seems, all working well.

From the same program, while setting up your IP address, you can set up the hostname OR you can do that during a clone.

Incidentally, those two lines selecting command line or desktop, commented out… below..

# you may want to use these on an RPi or elsewhere to force either
# a graphical or command line environment
#sudo systemctl set-default multi-user.target
#sudo systemctl set-default graphical.target

Ways of changing network settings

According to Mr Shark:

There are at least 5 different methods of setting up networking in Linux, each has its own config file, each can manage static AND dynamic ip, each is independent and can cause the collisions we’ve seen above… Let’s try to clarify, I’ll just list them with basic details, and give proper online GOOD documentation… a RTFM hasn’t ever harmed anyone…

Choose ONE of these, and be sure the others are disabled, not with just lines commented out in their config file… we had issues because of commenting out lines in /etc/dhcpcd.conf file while using Network Manager… but commenting out lines in that file will just revert its daemon to its standard behaviour, which is requesting IPs via dhcp… so the double ones, fixed by nmtui, and dynamic by dhcpcd…

In double quotes the service/daemon name, which you should enable/disable and/or start/stop via the usual “sudo systemctl enable|disable|start|stop|status SERVICENAME” command using:

1) the good old “networking” service.
It is configured via the /etc/network/interfaces config file, info: https://wiki.debian.org/NetworkConfiguration

2) the “wicd” service, which was installed by previous (to Buster) Raspbian editions if you used the GUI version, and deprecated by Buster (you end up having it only if you upgrade from previous editions and did not follow full instructions on Raspberry site to remove it and other stuff after upgrading…) This is configured via the /etc/wicd/*.conf config folder/files, info: https://wiki.archlinux.org/index.php/Wicd

3) the usual “dhcpcd” Raspbian suggested method which is configured via the /etc/dhcpcd.conf file, info: https://wiki.archlinux.org/index.php/Dhcpcd

4) the almost standard method in every modern distribution, “network-manager” – it is configured via various tools, in console you can use “sudo nmtui” (Network Manager Text User Interface), very easy to do, info: https://wiki.archlinux.org/index.php/NetworkManager

5) the newer method, the “netplan” way uses the format which you’ll find in MANY other services today (Docker and Home Assistant in primis), which is the YAML file format, that implies you format and indent the file in a PRECISELY spaced style, otherwise you’ll get errors…
managed via the /etc/netplan/* files, info: https://help.ubuntu.com/lts/serverguide/network-configuration.html

My own preference after experiencing the alternatives is to use (4) nmtui and I always use a static IP address for the Raspberry Pi, mainly because I cannot get ESP8266s to talk to my MQTT broker based on host name.

We hope this is clear. Now, and if you want more details, as suggested above, RTFM.