Testing your Puppet manifests using Vagrant

We’ve recently started using Puppet to provision our servers really seriously at work. All of our new servers are systematically provisioned using Puppet, and we now have a rule that nothing on servers can be changed by hand, every single bit needs to be managed by Puppet.

Managing servers using Puppet is extremely useful and powerful, but you also need to know what you’re doing because a small untested change in your Puppet manifests could introduce instability (or worse!) in your production environment.

Quality is one of my top concern. In infrastructure management, quality is often measured from stability. Thus, I quickly felt the need to thoroughly test any change to our Puppet manifests in a completely isolated environment.

My search for the holy grail led me to the nirvana of Puppet testing: a combination of Vagrant and librarian-puppet.

Vagrant

With Vagrant, you describe a virtual machine in a simple configuration file (a “Vagrantfile”), and then using the Vagrant command line tools, you can create, destroy or re-provision one or more virtual machines in a quick and repeatable manner. In the background, Vagrant uses the free Oracle Virtualbox for running the virtual machines (more recently, new “providers” have been added such as VMWare or even an experimental Amazon EC2 provider). You can also easily connect to your virtual server using SSH to validate what Puppet has done to your server. This is ideal for testing Puppet as you can scrap your VM whenever you need and restart from scratch in less time than it takes to make coffee.

Librarian Puppet

Librarian-puppet manages your Puppet module dependencies. In a file called Puppetfile, you describe which modules your infrastructure depends on, then librarian-puppet does the rest. It installs dependencies, upgrades them or removes them when needed. This is an invaluable tool to control when and how your module dependencies are updated.

A quick kickstart project to get you running

If you want to try the workflow of managing a Puppet-provisioned virtual server in Vagrant, I’ve created a sample project that will help you jumpstart your setup. This project is available on my Github account.

See the README file for details on what to install and how to run it. A few Puppet best practices are also described in that README file.

Let me know if you have any question! Enjoy!

Advertisements

3 thoughts on “Testing your Puppet manifests using Vagrant

  1. Great! I’ve been using the Vagrant/Puppet combo on my local dev. environment when experimenting on a few of my pet projects… but it never made it to production. I was afraid of the instability you mentioned, especially when I felt the Puppet scripts were (unfortunately) tightly coupled to the Vagrant setup. I guess you brought it to the next level! Did you setup automated tests in a CI tool?

  2. Thanks Alex! I honestly have yet to feel a need to write unit tests for our Puppet code. I usually prefer to do live tests for small changes by validating what’s happening on my Vagrant box. What’s interesting is that by mixing Git branches into this, I can start my box on the “master” branch (the one that’s in production), then checkout a working branch, re-provision it and see what happens. This way I can (quite) safely test changes that would be applied on a box if I was to promote this code to production. Of course this is possible only if you adopt the “hands-off” mentality and never make a manual change on a server.

    I think my lack of feeling a need for unit tests might come from the fact we don’t have a lot of homegrown modules. In the context of modules, unit testing seems to make sense, especially when you’re targeting different platforms and a multitude of possible setups. But as part as your main manifests (those that combine community modules to actually come up with a whole), I have trouble justifying a full suite of unit tests, especially as I think tests would tend to be extremely brittle.

    It’s still something that’s on my list of things to explore though… maybe I’m just missing something 😛

  3. Pingback: A few Puppet best practices | Remember to Breathe

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s