Containers? On MY machine?

“It’s more likely than you think."

The Uninteresting Part

The back-story. I won’t be offended if you skip this part. I get it; I always scroll past that part on recipes websites. At least there are no ads here :D

I’ve been messing with computers ever since I can remember, and hosting my own services since around 2009. Things changed a lot and often along the way, of course, and one of the major changes was that I started looking into LXD and eventually migrated all my services into separate containers.

A side-effect of playing around with containers is: I started dealing more and more with Alpine Linux since it’s a very popular distribution to use in that context. It turned out that I quite enjoy Alpine, so I recently decided to give it a spin as my main, day-to-day operating system.

One of the things I try to do with my systems is to keep them as “minimal” as possible (yes, “minimal” means different things to different people. I like setting up everything from scratch, explicitly install every single package I need and know exactly how everything works, as much as possible. No kitchen sink, no blackbox magic). My first distribution was Slackware, I hopped around for a bit until I landed on Archlinux and stuck with it for several years (I still use it on all my machines except the XPS, which is running Alpine now).

One of the things that happens when you run the same system over a certain length of time is: you install things, configure them, try them out, uninstall some, keep others, install new things, etc., etc. Which inevitably leads to a bit of clutter (~/.config files, /etc/ files, and so on), if you’re not diligent with removing all of those (which I am not).

So, in an effort to keep my brand-new, fresh-and-clean Alpine Linux install clutter-free (but mostly as an experiment and just for fun) I decided to keep the packages installed on the host at a minimum and install extra things in containers. As you can probably guess, I chose LXD to run these extra things. There are other, possibly better-suited, options out there. I do realize that, but I like LXD and I’m having fun with the experiment, which is what matters to me.

Side-note: as a bonus, those “extra things” run in a sandboxed environment. (woo!)

The More Interesting Part

The “recipe” (finally!).

The Host

For the most part, on top of alpine-base, the kernel and drivers, I have:

• bluez
• foot
• git
• gpg
• lxd
• nftables
• openssh
• pipewire / wireplumbler
• sway / swaylock / swayidle
• tmux
• vim
• wayland
• wireguard
• wl-clipboard
• wpa_supplicant

At the time of writing, the total count of /etc/apk/world is 60 packages. Granted, this number doesn’t really mean much. Different distros package things differently. Sometimes things get split-up or merged, but I’m writing this here so I can look back and compare, because why not?

The dotfiles

Like many people do, I keep my dotfiles in a git repository, which I have cloned locally in ~/devel/dotfiles/ and I symlink from ~/.config/ and others to there:

aerc -> /home/chimo/devel/dotfiles/.config/aerc/
irssi -> /home/chimo/devel/dotfiles/.config/irssi/
lxc/
mbsync -> /home/chimo/devel/dotfiles/.config/mbsync/
mpv -> /home/chimo/devel/dotfiles/.config/mpv/
ncmpcpp -> /home/chimo/devel/dotfiles/.config/ncmpcpp/
newsboat -> /home/chimo/devel/dotfiles/.config/newsboat/
pulse/
qutebrowser -> /home/chimo/devel/dotfiles/.config/qutebrowser/
sway/config -> /home/chimo/devel/dotfiles/.config/sway/config
sway/swaybar/ -> /home/chimo/devel/swaybar/

The containers

I currently have the following containers running:

aerc

Mail client.

devel

Ephemeral container for development stuffs (right now there's nginx, php, etc. on there)

hugo

Static site generator. To preview posts I'm writing for this blog.

irssi

IRC client.

mbsync

Syncs my mailboxes locally.

mopidy

Music server. Play music from various sources, MPD daemon.

mpv

Watch YouTube, Twitch.

ncmpcpp

Music player client. MPD client for mopidy container (above).

newsboat

RSS reader syncing with my FreshRSS instance.

nextcloud

`nextcloudcmd` running periodically to sync files between my Nextcloud instance and my local machine.

opensmtpd

SMTP "relay" to send mail from aerc.

This allows me to "send" emails while offline and they'll get delivered whenever I get online again. This setup was inspired by Drew DeVault's blog post.

qutebrowser

Browsing the intertubes.

rbw

Password manager integrating with my vaultwarden instance.

signal

Instant messaging.

tut

Mastodon client.

GUI and Audio in Containers

I’ve setup a “gui” lxd profile and a “audio” lxd profile based on the forum posts below. Whenever a container needs either one, or both, I attach the appropriate profile(s) to it and things “just work”(TM).

• [Howto] Use the Host’s Wayland and XWayland Servers inside containers
• Audio (via Pulseaudio) inside Container - Note: I don’t use pulseaudio, but the concept applies to the Pipewire socket.

File Sharing

No, not the peer-to-peer kind. :)

Whenever containers need to share files, for example: I want “aerc” and “mbsync” to share the same mbox directory. Or whenever the host and containers need to share files, for example: I want a container to read config files from ~/devel/dotfiles, I use LXD’s shared folders (idmapped mounts) mechanism.

I’ve automated the creation of my containers and have pre-sets to apply either the “gui” and/or “audio” profiles and setup the environment as-needed.

I can also provide one-off LXD profiles for containers that have specific needs.

The script is pretty rudimentary. I have a bunch of TODOs planned and a bit of a revamp after some “lessons learned”, but it does the trick for my use-case.

Again, I know things that do this already exist. I’m also aware of cloud-init and all that. But I prefer something that would work on a bare-metal host, on cloud-init environments, as well as non-cloud-init environments, etc.

Most importantly, I’m just tinkering and having fun. /shrug

Container Backups and Updates

I have a script that shows me pending updates in my swaybar, and another script I can run to update one, several, or all containers. It takes a snapshot before applying any upgrade.

I’ve written about how I handle updates and backups on my VPS before. Those processes run on a schedule since the VPS is running 24/7.

Interacting with the Containers

I have a set of scripts in ~/.local/bin/ (this folder is part of my $PATH environment variable) that translate calls to container binaries into a lxc exec command.

This is a terrible explanation. Maybe some examples are better.

I have ~/.local/bin/aerc.sh in which the content is essentially just lxc exec aerc -- aerc.

Some of the other scripts, such as newsboat.sh set some options: lxc exec newsboat -- sh -c 'newsboat -c ~/.config/newsboat/cache.db -C ~/.config/newsboat/config'

Others (ex: irssi.sh) have different behaviours depending on the arguments they are called with.

There might be better approaches, but this works for me and I find it quite flexible.