If you're among the rising number of Git users out there, you're in luck: You can automate pieces of your development workflow with Git hooks. Hooks are a native Git mechanism for firing off custom scripts before or after certain operations such as commit, merge, applypatch, and others. Think of them as henchmen for your Git repo. Pre-operation hooks act as bouncers, guarding your repo with a velvet rope. And post-operation hooks are your Man Friday, faithfully carrying out follow-up tasks on your behalf.
Installing hooks for a Git repository is fairly straightforward, and well-documented. In this article, we focus on using Git hooks to augment continuous integration practices, starting with an example that makes combining Git and continuous integration (CI) less painful. The code is written in Ruby. Fortunately, Ruby is a language that highly prizes readability, so even if you don't know Ruby, you can easily follow along.
Automate CI Configuration for Git Branches
One of the blessings of Git is how easy it is to branch off and develop in isolation. This means the master stays releasable, you get the freedom to experiment, and your teammates aren't derailed if code from the experimentation proves to be half-baked. One challenge of Git, however, is how many branches a team ends up with — scores of active branches, most of which live for only a few days. Who is going to take the time to set up continuous integration for all those piddly little branches? Your henchmen, that's who.
To automatically apply CI to new development branches, you'll use the "post-receive" hook type. These are server-side hooks, triggered after pushes to the repository are completed. In such cases, you can use the post-receive hook to fire off a script that programmatically clones a master's CI configs and applies them to new branches using the CI server's exposed API. It might look something like this, when using the open-source and hugely popular Jenkins CI server:
#!/usr/bin/env ruby
# Ref update hook for creating new Jenkins job
# configurations for newly pushed branches.
#
# requires Ruby 1.9.3+
require 'yaml'
require 'net/https'
require 'uri'
require 'rexml/document'
include REXML
# load ci-config.yml from hook directory
def load_config
hookDir = File.expand_path File.dirname(__FILE__)
configPath = hookDir + "/ci-config.yml"
puts configPath
raise "No ci-config.yml found." unless File.exists? configPath
YAML.load_file(configPath)
end
# Grab the configured Jenkins server
config = load_config
raise "ci-config.yml file is incomplete: missing jenkins_server" unless
config["jenkins_server"]
server = config["jenkins_server"]
raise "ci-config.yml file is incomplete: username, password, url and
default_job are required for jenkins_server" unless
server['url'] and server['username'] and
server['password'] and server['default_job']
# iterate through updated refs looking for new branches
ARGF.readlines.each { |line|
args = line.split
oldVal = args[0]
newVal = args[1]
ref = args[2]
if /^0{40}$/.match(oldVal) and ref.start_with?("refs/heads/")
# new branch!
# retrieve the jenkins job config
# TODO only need to do this once!
uri = URI.parse(
"#{server['url']}/job/#{server['default_job']}/config.xml")
req = Net::HTTP::Get.new(uri.to_s)
req.basic_auth server['username'], server['password']
http = Net::HTTP.new(uri.host, uri.port)
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
http.use_ssl = uri.scheme.eql?("https")
# execute the request
response = http.start {|http| http.request(req)}
raise "Bad response from jenkins, is your ci-config.yml correct?"
unless response.is_a? Net::HTTPOK
# parse the config.xml from the response
doc = Document.new response.body
doc.root.get_elements(
"//branches/hudson.plugins.git.BranchSpec/name").each {
# overwrite branch to be our new ref
|elem| elem.text = ref
}
# create a new request to upload the modified config.xml
newJob = ""
doc.write newJob
newJobName = ref["refs/heads/".length..-1].gsub("/", "-")
uri = URI.parse("#{server['url']}/createItem?name=#{newJobName}")
req = Net::HTTP::Post.new(uri.to_s,
initheader = {'Content-Type' => 'application/xml'})
req.basic_auth server['username'], server['password']
req.body = newJob
# upload the new job
response = http.start {|http| http.request(req)}
raise "Failed to post new job to jenkins" unless
response.is_a? Net::HTTPOK
end
}
With this hook in place, you need only push a dev branch to the repo, and it will automatically be put under test. (It's possible to run CI builds against branches using a build parameter to represent the target branch, but that muddles the build history. The cloning approach provides a clean, clear history.) Applying every last facet of the CI scheme to branches isn't necessary — for example, running each and every branch through the load test gamut might be overkill. But even if you skip the load and UI tests, and run just unit and API- or integration-level tests, these are huge wins.
The risk of introducing defects into master is greatly reduced by testing on the branch before merging. Developers can also work more efficiently and confidently because of the frequent feedback on changes (instead of the old merge-then-pray technique). And for teams who include testing as part of their definition of "done," managers and scrum master types catch a break. With the Git hook automatically putting branch code under test, the team's practices and values are being enforced without the need for nag-mails or raised eyebrows during stand-up.
Vet Merges to Master
Two hallmarks of coding craftsmanship are an affinity for automated tests, and adherence to stylistic rules (such as avoiding empty try/catch blocks or duplicated code). Despite best intentions, everyone neglects best practices from time to time. That's where Git hooks come in. Pre-receive hooks living in the central repository qualify incoming pushes, making sure they're good enough to get past the velvet rope. Let's look at three hooks designed to protect master from slip-ups made on development branches.
Require Passing Branch Builds
The whole point of working on a development branch is to isolate yourself and create a space to experiment (read: "break stuff"). So it's natural to see failing tests on the branch while development is in progress. When it's time to merge to master, however, things had better be tidied up. This can be enforced programmatically with a hook that checks to see whether the incoming push is a merge to master, and if so, verify that all tests are passing on the branch before processing the merge.
If you happen to be using Bamboo, you can cleanly fetch test results for a given commit. If you use Jenkins or its predecessor, Hudson, you can fetch a set of recent build results then parse through them to see which builds ran against the commit in question. (This hook, and those that follow are implemented for the Bamboo CI server, but they can be implemented in more or less the same way on all CI servers.)
#!/usr/bin/env ruby
# Ref update hook for verifying the build status of
# a topic branch being merged into
# a protected branch (e.g. master) from a Bamboo server.
#
# requires Ruby 1.9.3+
require_relative 'ci-util'
require 'json'
# parse args supplied by git: <ref_name> <old_sha> <new_sha>
ref = simple_branch_name ARGV[0]
prevCommit = ARGV[1]
newCommit = ARGV[2]
# test if the updated ref is one we want to enforce green
# builds for exit_if_not_protected_ref(ref)
# get the tip of the most recently merged branch
tip_of_merged_branch =
find_newest_non_merge_commit(prevCommit, newCommit)
# parse our Bamboo server config
bamboo = read_config("bamboo", ["url", "username", "password"])
# query Bamboo for build results
response = httpGet(
bamboo,
"/rest/api/latest/result/byChangeset/#{tip_of_merged_branch}.json")
body = JSON.parse(response.body)
# tally the results
failed = successful = in_progress = 0
body['results']['result'].collect { |result|
case result['state']
when "Failed"
failed += 1
when "Successful"
successful += 1
when "Unknown"
if result['lifeCycleState'] == "InProgress"
in_progress += 1
end
end
}
# display a short message describing the build status for
#the merged branch and abort if necessary
if failed > 0
# at least one red build - block the branch update
abort "#{shortSha(tip_of_merged_branch)} has #{failed}
red #{pluralize(failed, 'build', 'builds')}."
elsif in_progress > 0
# at least one incomplete build - block the branch update
abort "#{shortSha(tip_of_merged_branch)} has #{in_progress}
#{pluralize(in_progress, 'build', 'builds')} that have not
completed yet."
else
# all green builds - allow the branch update
puts "#{shortSha(tip_of_merged_branch)} has #{successful}
green #{pluralize(successful, 'build', 'builds')}."
end
Enforce Code Coverage Requirements
Along with successful test runs, you want to make sure that new code added on development branches is tested as thoroughly as code already on master. This ensures that the overall test coverage level of the project doesn't drop when a development branch is merged back in. This, too, can be checked with Git hooks.
A simple Git hook can verify that coverage on the branch meets the minimum threshold. To enforce this, a hook can be created to compare the coverage rate on master with that of the branch, and reject the merge if the branch's coverage is inferior.
