I wrote a blog post detailing my homelab setup throughout 2023. It includes the hardware I use, as well as the applications I selfhosted. I also detailed how I automate my home Kubernetes cluster and how I back up my data.

  • johntash@eviltoast.org
    link
    fedilink
    English
    arrow-up
    8
    ·
    11 months ago

    Storage is hard to do right :(

    If you can get away with it, use a separate NAS that exposes NFS to your other machines. Iscsi with a csi might be an option too.

    For databases, it’s usually better to not put their data on shared storage and instead use the databases built in replication (and take backups!).

    But if you want to go down the rabbit hole, check out ceph, glusterfs, moosefs, seaweedfs, juicefs, and garagehq.

    Most shared file systems aren’t fully posix compliant so things like file locking may not work. This affects databases and sqlite a lot. Glusterfs and moosefs seen to behave the best imo with sqlite db files. Seaweedfs should as well, but I’m still working on testing it.

    • Hexarei
      link
      fedilink
      English
      arrow-up
      4
      ·
      11 months ago

      Yep, as someone who just recently setup a hyperconverged mini proxmox cluster running ceph for a kubernetes cluster atop it, storage is hard to do right. Wasn’t until after I migrated my minor services to the new cluster that I realized that ceph’s rbd csi can’t be used by multiple pods at once, so having replicas of something like Nextcloud means I’ll have to use object storage instead of block storage. I mean. I can do that, I just don’t want to lol. It also heavily complicates installing apps into Nextcloud.