Saturday, February 17, 2018
Home » Automation » Automating the automation: Using Puppet to manage Continuous Integration with Jenkins

Automating the automation: Using Puppet to manage Continuous Integration with Jenkins

Continuous Integration (CI) is a practice which runs unit tests in an automated fashion by either running on a schedule or polling your VCS for changes to a particular project. This best practice helps to catch code-related problems at an early stage. This also helps with reverting your code base to a bug-free state quite easily, since you will know which commit in your VCS is causing the failure.

Our Challenge

Here at Datapipe, we have an extensive library of puppet modules that have unit tests written using rspec-puppet. We found that a lot of time was being spent on creating builds for these modules on our Jenkins server. We automate a lot of our infrastructure already and decided that the same should be done for CI.

Enter Puppet-Jenkins

While researching the best way to automate our Jenkins builds using Puppet, we found the Puppet-Jenkins module originally written by rtyler and now being maintained by the Jenkins staff. This module handles the installation and configuration of Jenkins, as well as installing Jenkins plugins and adding Jenkins build jobs. There were many factors to consider when deciding to implement this module’s functionality in our own environment. We needed:

– A way to install required ruby versions and gems for each build (using rvm)
– The ability to manage the ssh key that Jenkins used to clone repositories from our git server
– A way to manage sudo rules for the Jenkins user for specific unit tests
– A means of easily specifying a custom template for a Jenkins build or plugin

How We Did It

Since we use the Puppet best practice of roles and profiles, we created a profile class to handle prerequisites (ruby, gems, Jenkins ssh key, etc) and a few defined types to handle Jenkins plugins and custom Jenkins build jobs. Using hiera in combination with the create_resources function, we were able to specify lists of ruby versions, as well as ruby gems for each ruby version, Jenkins plugins to install and Jenkins build jobs to create. The Jenkins jobs are created by supplying an XML file (as an ERB template). There is a default XML template used for most common builds. We also added the option to specify a custom erb template, via a named parameter, for jobs that may slightly differ from the default.

The Takeaway

Now, rather than fiddling with the Jenkins web GUI to create a new build, we can just add a couple of lines to a hiera YAML file. During the next Puppet configuration run, the job will exist in Jenkins.

About Pete Tierno

Pete Tierno
As an automation engineer at Datapipe, Pete helps enterprises bridge the gap between development and operational teams. Pete writes about automation tools.

Check Also

We’re Moving!

Back in September, we announced the exciting news that Datapipe has joined forces with Rackspace to create the world’s leader in multi-cloud managed services. Since then, the Datapipe and Rackspace teams have been hard at work to ensure a smooth integration, one that preserves each team’s best qualities and maintains the exceptional customer experience that both Datapipe and Rackspace customers have come to expect.