So I’ve been running a little 2 node rpi kubernetes cluster for over a year now, bootstrapped with Ansible and Helm (source). I picked Ubuntu Server at the time because I think the official 64-bit Raspbian OS was still young or maybe not even out at the time (can’t quite remember) but I’ve found myself fighting with Ubuntu an awful lot culminating in a major version upgrade to “jammy” last night that has wrecked one of my nodes. It even tried to delete the running kernel during the upgrade but caught itself and asked me to confirm, wtf. I’ve never experienced a Linux upgrade this bad. Yeah, “jammy” is right. Luckily I use a separate NAS for persistence. So I’m breaking up with Ubuntu, which I think is the cool thing to do these days anyway, and using this as an opportunity to rebuild and clean up my IaC.
I am most familiar with Red Hat distros (Fedora/CentOS daily drivers for years now, RHEL servers at work) though I’m not familiar with the ARM ecosystem there. Ive also been wanting to try NixOS for a while but looking at some of the rpi config last night had me a little concerned because it felt unfamiliar. Then of course there’s the Official Raspbian OS, 64 bit support should be solid by now.
What OS are you using for your Raspberry Pi servers? Any I should definitely avoid?
I’m weird and use ArchLinux ARM on my Raspberry Pi computers. I think it’s much easier to admin, especially if you don’t need video accelleration, but I use Arch on the daily, so that’s probably why I feel that way.
I also find that Fedora was pretty nice as well, but felt too bloated for what I needed.
Finally, Alpine was amazing. I used to use it as my daily driver for a while as well, and it is nice, lean, and easy to use. The main downside is that it uses the musl libc meaning sometimes packages won’t work, or things won’t compile. That was very uncommon though and the exception , not the rule.
The main problem I’ve had on ALL of those distributions were the clock. The Raspberry Pi doesn’t have a built-in clock, so you need to use NTP to pull the time down, or else it’ll be extremely out of sync. This means setting up your timezone, etc. RPi OS does this for you, but most DIY distros (Alpine, Arch) will not, so you’ll need to set that up.
I’ve used alpine for minimal container images, but never as a workstation or server (or arch for that matter). Config management isn’t an issue, I already ansibilized my config and a significant amount is removing crap I don’t want for Ubuntu so maybe going minimal and installing exactly what I need is would be cleaner. Hmmm. Tempting.
The main thing about it is that although Ubuntu is pretty much ready to go, Alpine not so much. It does take some fiddling around with, but I find because it is so minimalistic, ansible playbooks just keep working again and again update after update. I guess the question is when do you want to put your time in. All at the beginning, and then just keep minimal maintenance (DIY distros like Alpine) or just do a bit more maintenance with a super easy install and bringup (More wholistic distros like Ubuntu)