For the past several years, I’ve been running the majority of my lab services out of FreeBSD jails. I don’t have but a handful of services to run and some underpowered hardware to put them on, but jails have grown to become a staple in how I deploy my private infrastructure. One such service that I run is LDAP. I got a wild hare yesterday and though I’d glue the two of them together with a bit of Puppet.

Background

To run Jails, there isn’t but a little bit of information that is required to get a node on the network and available to run services. Really just some information about the host like the default interface to use and perhaps the default path for jails. A long time ago I wrote a very brief puppet-jail module to manage these settings and handle the creation of the jail and bringing it online. I stored the information about the jails in Hiera, and when I needed to create a new jail, I would modify the Hiera data and commit.

After using jails in this way for a while, the tedium of committing to create a jail, then another to destroy it, then another to remove the jail entirely got a bit too much to bear.

This led me to start thinking of being able to modify an external data source so I could do so without modifying any of my Puppet code for deployment. The typed nature of LDAP led me this direction, although I’m only using the ‘Directory String’ data type in the LDAP schema below.

Prerequisites

Before taking a look at the details here, its good to know that there are three major components of this setup.

Puppet

Puppet is being used as the enforcer of truth here, but the idea here can be applied elsewhere. I’m using two modules to get the job done and a bit of Future parser (or current parser, depending on your world view).

ldapquery()

The puppet-ldapquery module was one that I’d started because I’d been wanting to glue Puppet to LDAP for a while, and an ldapquery() function seemed like just the ticket to store data in LDAP for use in Puppet manifests.

puppet-jail

The puppet-jail module is pretty rudimentary, and I almost abandoned it in favor of skoef’s jails module, but part of the work I did this weekend was to migrate back to my own jail module.

LDAP

The data store of choice here is LDAP, but likely any database could be used. I’ve just been using LDAP for a while and thought it would apply neatly.

FreeBSD and Jails

Jails are what I’m working with, but the concept could likely be applied to other container solutions or virtualization platforms. I just don’t care about any of those.

Some details

Here I’ll just cover the boring technical bits to get them at least documented.

The LDAP Schema

I’ve spent far too much time hand crafting an LDAP setup, so I’ll pass right over that bit. Its enough to say that you need LDAP and the ability to modify the schema.

Here is what I’m using to replace the cn=jail schema object in my tree.

dn: cn={16}jail,cn=schema,cn=config
changetype: modify
replace: olcAttributeTypes
olcAttributeTypes: ( 1.3.6.1.4.1.34380.600 NAME ( 'ipAddress' )
    DESC 'The v4 address for the jail'
    EQUALITY caseIgnoreMatch
    SUBSTR caseIgnoreSubstringsMatch
    SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )
olcAttributeTypes: ( 1.3.6.1.4.1.34380.601 NAME ( 'ipAddress6' )
    DESC 'The v6 address for the jail'
    EQUALITY caseIgnoreMatch
    SUBSTR caseIgnoreSubstringsMatch
    SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )
olcAttributeTypes: ( 1.3.6.1.4.1.34380.602 NAME ( 'hostName' )
    DESC 'The hostname to set for the jail'
    EQUALITY caseIgnoreMatch
    SUBSTR caseIgnoreSubstringsMatch
    SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )
olcAttributeTypes: ( 1.3.6.1.4.1.34380.603 NAME ( 'defaultInterface' )
    DESC 'The netwrk interface for the Jail host'
    EQUALITY caseIgnoreMatch
    SUBSTR caseIgnoreSubstringsMatch
    SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )
olcAttributeTypes: ( 1.3.6.1.4.1.34380.604 NAME ( 'zpoolName' )
    DESC 'The netwrk interface for the Jail host'
    EQUALITY caseIgnoreMatch
    SUBSTR caseIgnoreSubstringsMatch
    SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )
olcAttributeTypes: ( 1.3.6.1.4.1.34380.605 NAME ( 'ensure' )
    DESC 'The state the jail should be in: present, absent, running, stopped'
    EQUALITY caseIgnoreMatch
    SUBSTR caseIgnoreSubstringsMatch
    SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )
-
replace: olcObjectClasses
olcObjectClasses:  ( 1.3.6.1.4.1.34380.666 NAME 'jailInfo'
    DESC 'Information about a specific jail'
    AUXILIARY
    SUP top
    MAY ( ipAddress $ ipAddress6 $ cn $ ensure )
    MUST ( hostName  )
    )
olcObjectClasses:  ( 1.3.6.1.4.1.34380.667 NAME 'jailHost'
    DESC 'Information about a jail host'
    AUXILIARY
    SUP top
    MAY ( cn $ ou $ defaultInterface $ zpoolName) )

The above OIDs are almost certainly incorrect, but and you will need to modify this to fit your environment. Also note, this is just the first pass.

Schemas are pretty boring, but I can see it being useful to actually create an attributeType for each of the possible jail properties. In time, perhaps.

Usage in manifest

I’ve got a profile::jail::host class that I include (as one might assume) on the hosts that will be running the jails. A bit of setup manifest I have looks something like the following.

$base = "cn=${::hostname},ou=jail,dc=zleslie,dc=info"
$host_attributes = [
  'defaultInterface',
  'zpoolName',
]
$host_data = ldapquery('(objectClass=jailHost)', $host_attributes, $base)[0]
$basedir = '/jails'

class { 'jail::setup':
  usezfs    => true,
  zpool     => $host_data['zpoolname'],
  interface => $host_data['defaultinterface'],
}

Once the host is prepped with a bit of defaults, you can create a bunch of jails like so.

Jail::Zfsjail {
  ensure             => present,
  source             => '/var/tmp/staging/profile/10base.txz',
  bootstrap_template => 'profile/jail/bootstrap.sh.erb',
}

$jail_attributes = [
  'cn',
  'ipAddress',
  'ipAddress6',
  'hostName',
  'ensure',
]

$jails = ldapquery('(objectClass=jailInfo)', $jail_attributes, $base)
$jails.each |$j| {

  if $j['ensure'] {
    $ensure = $j['ensure']
  } else {
    $ensure = 'present'
  }

  jail::zfsjail { $j['cn']:
    ensure            => $ensure,
    config            => {
      'host.hostname' => $j['hostname'],
      'ip4.addr'      => $j['ipaddress'],
      'ip6.addr'      => $j['ipaddress6'],
    }
  }
}

That should be enough to lookup jail information from LDAP and create the associated Puppet resource.

There are a few assumptions about the structure of you LDAP database in the above, but it should be clear enough.

The OU ou=jail,dc=zleslie,dc=info is the root of everything jail. Under that, the jailHost objectClasses are created, one for each host specified by here with $hostname. Then under each jail host, are the actual jail objects.

So cn=myjail,cn=myhost,ou=jail,dc=zleslie,dc=info would be of objectClass jailInfo and could contain all the needed bits.

Review

Now what? What we have now is a system that allows us to leverage Puppet to manage the state of Jails, without actually using the Puppet as a source of truth for which jails to be running. To me this is pretty great, since now with a bit of scripting I can create a record for a jail, run Puppet on the associated host and profit. Since the master does a query to LDAP each time the ldapquery() function is encountered, the data is always up to date. This should make for pretty speedy deployment of new jails, and the destruction of old jails.

Whats next?

I’m planning round out this exercise a bit so I can manipulate the jail data in LDAP and manage the jail life cycle from creation, certificate signing, initial Puppet run until finally revoking and destroying the jail. If I moved to LDAP as a classifier from a script I could classify the node/jail on first run as well, which could have all kinds of useful items.

There is definitely room for improvement in the above but in the six hours or so I’ve had this working, it sure is nice to have the configuration that Puppet is managing decoupled from the hiera data we used to rely on.