Moving My Puppet Master to OpenBSD

I recently had some network trouble that isolated a system I use to host FreeBSD jails to be without network. The bummer of it is, that I used this particular system for hosting most of my primary working system. Things like CI and Puppet and such all lived in neat little jails, and while all the little jails and all the little services are still on disk I have no access. I suspect the hand-me-down Cisco switch is the culprit, but after years of service, meh.

Let me say also, that what follows was done with the knowledge that at any time I could just plug a USB flash drive in and ZFS sync over my jails and jail config files. This is something I may yet do for a couple jails. However, for Puppet, I thought I’d just see how it went on one of my OpenBSD servers. Quite well actually. I even got to test my backups in the process.

Here are some of the guts.

Deploying r10k

Since I do all my development on my laptop, all of the code necessary to deploy a new environment was already there. I just decided to use r01k to deploy the rest of the modules that I relied upon, but maybe don’t develop on. In any case, this was quick.

r10k deploy --config ~/Org/n3kl/r10k.yaml environment bootstrapnew -pv

My r10k.yaml was about as simple as it could get.

:cachedir: /Users/zach/Org/zleslie/pp/var/r10k
:sources:
  main:
    basedir: /Users/zach/Org/zleslie/pp/environments
    remote: git@git.zleslie.info:infrastrucuture/pp.git

:purgedirs:
  - /Users/zach/Org/zleslie/pp/environments

Then just copy over the code with a quick:

rsync -aP environments/ root@obsd.l.zleslie.info:/etc/puppet/environments/

and since I use directory environments, I needed set the default_manifest.

puppet config set server obsd.l.zleslie.info
puppet config set ca_server obsd.l.zleslie.info
puppet config --section agent set environment bootstrapnew
puppet config set default_manifest site.pp
puppet config set environmentpath /etc/puppet/environments

Then I used the bacula to restore a few key files:

# Ain't no FreeBSD
sed  -e 's/\/usr\/local//g' /tmp/bacula-restores/jails/pm0/usr/local/etc/puppet/hiera.yaml > /etc/puppet/hiera.yaml
ln -s /etc/puppet/hiera.yaml /etc/hiera.yaml

# Get some CA back
mv /tmp/bacula-restores/jails/pm0/var/puppet/ssl/ca/ /var/puppet/ssl/

# Eyaml keys and extra filemounts
mv /tmp/bacula-restores/jails/pm0/usr/local/etc/puppet/* /etc/puppet/

Then just to confirm I’ve got what I need, I’ll just list the certs.

puppet cert -la

Looks good enough to me. Now I just need to keep the dns_alt_names from before and update my dns.

puppet config set dns_alt_names obsd.l.zleslie.info,puppet.zleslie.info,puppetca.l.zleslie.info,puppet.l.zleslie.info
puppet cert print obsd.l.zleslie.info
/etc/rc.d/puppetmasterd start

Now it looks as though I am back in business. I’ve already got my git repos pushing to the correct place.

Update DNS for redirecting the CNAME
Install PuppetDB
pkg_add puppetdb

Adjust /etc/puppetdb/conf.d/puppetdb.ini

[jetty]
# Optional settings:
host = localhost
port = 8080
# Required settings:
ssl-host = obsd.l.zleslie.info
ssl-port = 8081
ssl-key = /etc/puppetdb/ssl/private.pem
ssl-cert = /etc/puppetdb/ssl/public.pem
ssl-ca-cert = /etc/puppetdb/ssl/ca.pem

puppetdb ssl-setup

The ssl-setup command failed since it seems it requires jetty.ini, but here only existed puppetdb.ini. Oh well, I think this is just packaging and I was able to get the pieces in place. The ssl files were copied correctly.

puppet config set storeconfigs true
puppet config set storeconfigs_backend puppetdb

/etc/puppet/puppetdb.conf

[main]
server = obsd.l.zleslie.info
port = 8081
soft_write_failure = false

Deploying r10k#

Deploying r10k