Just about every software engineer has had the experience of onboarding with a new team and spending a day, a week or more getting their development environments situated so they can actually run the application they signed up to work on and start committing code and being productive. Some teams try to improve upon this by maintaining a team wiki that documents setup instructions and may even include setup scripts. Unless these are actively maintained and curated they may cause more harm than help, leading the new developer down dead ends and on wild goose chases.

The best solution is less narrative and more executable documentation. One tool that many developers use to facilitate this and one we use on the infrastructure automation team at CenturyLink Cloud is Vagrant. Vagrant is a tool that makes it easy to share virtual environments across different virtualization platforms and makes destroying and recreating those environments an easily repeatable process. It also provides a mechanism that allows you to code in your native environment while your app runs in a VM.

In my first few months on the CenturyLink Cloud team, I worked on a Windows machine using Hyper-V for virtualization. Now I’m on an Ubuntu desktop using VirtualBox and we have other team members using VMWare Fusion on a Mac. Despite our different platforms, by using Vagrant we can all work off of the same configuration.

In this post I will share how our team configures Vagrant to handle developer VM provisioning. You can download Vagrant from https://www.vagrantup.com/downloads.html”>https://www.vagrantup.com/downloads.html and install the latest version if you would like to follow along.

Light weight and easy to source control

The key artifact of a Vagrant setup is the Vagrantfile. As long as Vagrant is installed, this simple text file is all one needs to share and consume an environment. Here is an excerpt from our Vagrantfile:

VAGRANTFILE_API_VERSION = "2"

chef_recipe = ["chef_workstation::default"]

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  config.omnibus.chef_version = :latest

  config.vm.box = “hashicorp/precise64”

  config.vm.hostname = “chef-workstation”

  config.vm.synced_folder “../../”, “/T3NDevOps”]

  config.vm.provider “hyperv” do |hv|

    hv.ip_address_timeout = 240

    chef_recipe << "chef_workstation::openvpn"

  end

  config.vm.provider :virtualbox do |vb, override|

    vb.customize [“modifyvm”, :id, “—natdnshostresolver1”, “on”]

    vb.customize [“modifyvm”, :id, “—natdnsproxy1”, “on”]

    vb.memory = 2048

  end

  config.vm.provision “chef_solo” do |chef|

  chef.cookbooks_path = [[:host, File.absolute_path(”../../chef-repo/cookbooks”)]]

  chef_recipe.each {|recipe| chef.add_recipe recipe}

    chef.json = {

      "chef_workstation" => {

        "user" => "vagrant",

        "group" => "vagrant",

        "chef_user" => "#{USER}”

    }

  }

  end

end

As you can see, this configures some basic VM properties which can vary by hypervisor provider. Check out the Vagrant documentation site for complete details on all Vagrant configuration settings. The most important setting here is the config.vm.box. In Vagrant, the “box” points to the base image. The box must have a name and optionally a URL. When no URL is present as seen here, Vagrant will look for the box at https://vagrantcloud.com/. So the box here would be pulled from https://vagrantcloud.com/hashicorp/boxes/precise64.

This webpage on Vagrant Cloud shows that it has boxes available for VirtualBox, Hyper-V and VMware Fusion. Vagrant has packaging conventions for all virtualization providers it supports. This convention includes the creation of an archive that contains the image file and any other key metadata files necessary for that provider to recreate the VM. Users of Vagrant can store these archives, which can be quite large on Vagrant Cloud if they purchase a pro account or they can use a free account and simply point to another URL where the box file is kept. For example, for personal projects, I keep a Windows box at https://vagrantcloud.com/mwrock/boxes/Windows2012R2.This points one to download the box from CenturyLink Cloud Object Storage at: https://wrock.ca.tier3.io/win2012r2-virtualbox.box for VirtualBox or https://wrock.ca.tier3.io/win2012r2-hyperv.box for Hyper-V. The great thing about this is it means I don't have to stick a huge image file in source control, instead the small Vagrantfile acts as a pointer to where the image can be found.

Repeatable provisioning scripts

Your Vagrant image can be a simple bare OS install, a complete environment or something in between. It’s usually best to keep the image as light as is reasonable and let a provisioner handle the remainder of the setup. Vagrant’s plugin model supports all sorts of provisioner plugins including a shell provisioner that simply contains one or more shell scripts as well as provisioners for almost any popular configuration management solution. You will notice that our file uses Chef and points to our chef_workstation cookbook that contains our provisioning recipes.

This cookbook will transform the bare Ubuntu 12 image to a fully functioning chef workstation customized for all of the things that our environment requires. For instance it will:

  • Install and configure [Docker](https://www.docker.com/)
  • Pull public as well as internally forked Ruby Gems both from [RubyGems.org](https://RubyGems.org/) and our own Git repos and install them
  • Ensure the correct versions of Chef and many of the infrastructure testing tools we use are installed
  • Ensure our internal command line tools for managing our lab environment is installed and in the users path
  • So if someone comes to us and says they want to create some cookbooks for their infrastructure, all we have to tell them is to run:

    git clone https://our/private/git/repo
    
    cd repo/Vagrant/WorkStation
    
    vagrant up
    

    With that, they will have a fully functioning environment no different from ours in just a couple of minutes.

    The build server uses the same provisioning script

    Here is another great technique. Use the exact same scripts that setup your developer environments to also provision your build server agents. This way when your tests pass and your code works locally, you have a high degree of confidence that your builds will pass in your CI pipeline. Further, just as easily as you can tear down your Vagrant VM and reproduce a clean environment, you can also rebuild your build agents to the same known state.

    But I hate working in a VM

    So do I. I have historically done the majority of my development natively where all of my favorite tools are and I don’t have to hassle with marshaling files back and forth from host to VM. Well Vagrant includes a simple but powerful feature called synced folders to accommodate a workflow allowing one to code natively but run virtually. Depending on the OS of the host and guest, Vagrant uses either rsync, NFS or SMB to sync contents from one or more folders on the host to the VM and back. In our Vagrantfile we sync from two levels above the location of the vagrant file, the root of our Git repo. So now I can edit natively in my favorite text editor or IDE (sublime text) and have all edits immediately available to be run on my VM.

    Our team spent a fair amount of time and effort to get the Vagrantfile setup and provisioning scripts just right. Also, whenever we change our environment tooling and requirements, we make sure these are reflected in these scripts. Ultimately, our team agrees that this up-front investment has saved us much more time in the long run. Plus, it dramatically reduces the friction of onboarding other engineers into our environment and gets them working on real projects more quickly.