Escaping the FAANG Part 3. - An Update
It's been almost a year since I started ordering parts for a homelab server to replace some of my dependence on the rent-seeking cloud vultures of the FAANG. In the intervening period, the little server has already become critical personal infrastructure, but I dropped the ball on documenting my journey as the half-finished blog posts languished and the software that I tried failed me.
While I continue to maintain the deluded optimism that I will find the perfect software to support my needs, and then will be able to finish writing about it, I thought it might be worth a post to snapshot my progress to date; the imperfect patchwork of solutions that I have so far, and the distance that still awaits.
The Good
Notes and Todo lists
My brain is too small to keep track of all the little pieces of information that float through it. For over two decades, I've relied on some sort of digital note taking. The current iteration is just plain text notes that I need to be able to edit anywhere. On the iPhone I currently use 1Writer, and on my laptop I use Obsidian. Pretty much anywhere else I use vim and a handful of shell scripts to edit them.
The only difficult part is sync. After evaluating many options, I realised that there just isn't a reliable free (as in freedom) way to sync anything to the Apple ecosystem, so I use dFropbox to sync the notes to my iOS devices, currently the Dropbox free-tier supports enough devices to make this possible. I know this is likely a temporary solution, but it works for now.
I had hoped to use Syncthing or Nextcloud for this, but sadly their mobile clients just aren't there yet. Dropbox can afford a dedicated team to work on this, and have the clout to force Apple to support their efforts, so they are able to achieve a level of reliability that is lacking in the FOSS options.
I use Nextcloud to sync the notes to my laptop. Because using 2 sync engines can lead to conflicts, I have a hacky little shell script in a cron tab that makes sure that these conflicts are resolved into the notes. It also backs the notes up into a git repository and pushes to github. Once a day, my todo list is emailed to me from another cron script.
It sounds messy, but the overall effect is that I can edit notes anywhere, and within a few minutes, know that they are synced everywhere and backed up. This means I feel confident offloading some of my brain into this system. I would struggle to be productive without all of this now, so this is pretty much critical infrastructure for me.
Tailscale
One of the things I was worried about when setting up all of this was how I would let my mobile devices connect to the homelab without the security difficulties of running things publicly.
Tailscale has pretty much resolved all of these concerns - I can think of the 10-15 machines that I want to communicate with one another as on a single network and tailscale handles all of the intricacies of making that happen.
I would normally feel a bit icky relying this heavily on proprietary solutions that I don't (yet) pay for, but the tailscale business model makes sense to me, doesn't want to exploit me, and the underlying technology is similar enough to wireguard that I feel confident that I could replace them if they got too evil.
The short story is that I'm a massive fan of their service, and it's playing a critical role in my setup.
It didn't rely on the server, but I replaced gmail with fastmail, and other than a few small warts, everything seems to be working perfectly.
Rclone
Using rclone to mirror files between locations has become a critical part of my workflow - I rely on it entirely for backups, and it's the tool I reach for most often when copying data (I rarely use rsync any more).
ZFS
I guess a year is a short amount of time to evaluate a RAID, but my zpool continues to chug along and handle around 2 terabytes of data without any issue.
The Mediocre
Samba, and File sharing
What use is a NAS if you can't access the data within it? I've found accessing my data pretty slow from my laptop using NFS and SAMBA, presumably due to lacklustre support from Apple in macos. Sadly, despite much head-wall-banging, I haven't discovered any potential road out of this mess. Editing photos and video remains painfully slow for me.
Nextcloud and Syncthing
I already mentioned the difficulties syncing plain text to my mobile devices in the Todo list section, but Syncthing and Nextcloud proved disappointing in that regard. I am currently using syncthing to keep my lightroom catalogue mirrored to the server (it cannot reside on a remote volume, so it needs to be mirrored rather than on NAS). This seems to work, but I have reservations about it.
Nextcloud gives me mixed feelings. On the one hand I love their mission, and they seem to want to address a lot of the problems that I want to solve. On the other, they are woefully under-resourced to do all the things they want to, which leaves them spread too thin, and failing to do many of their goals well enough.
I use their file sync for my todo lists as mentioned above, but I don't use their calendar server (cannot sync with google calendar), or many of their other capabilities. I have used the photo backups part of their ios app to manually backup my iphone photos, but it is so slow and unreliable that I do this infrequently, and don't feel comfortable relying on it.
The Ugly
Automatic restarts
My little server runs in my office, and when a guest unplugged it so they could charge their phone, I experienced my only downtime this year. When I plugged it back in, I was disappointed how few of my services came back up automatically. I have a lot of work to do with systemd configs to make this a seamless process.
I'm also not sure if there's a way to make this fully automatic. I use full disk encryption which relies on me entering a password at boot time. I'm not sure how to remove this bottleneck on my time.
As it stands, I simply rely on Ubuntu's phenomenal reliablility to maintain my personal SLA.
Unfinished Business
I have many unfinished projects. I've played about with serving media to Kodi from Jellyfin. My encrypted volumes are not where I want them. I have only dipped a toe into the home-assistant world. I'd love to get rid of my reliance on lightroom for photos. I still use google calendar.
Perfect is the enemy of the good, but mediocrity is even easier. I've not yet removed my reliance on cloud services, but I've taken a step in the right direction, and some things are getting there.