Devops – The Last Mile
Jay Flowers
jay.flowers@gmail.com
http://jayflowers.com
Workspace Management
The act of defining, creating, and
maintaining the software tool stack
that comprises a developer’s or
tester’s computer configuration.
Why?
Problem?
Why do we need Workspace
Management?
Do we have a problem?
As workspace configurations drift
the more pervasive and sever “works
on my box” syndrome is encountered.
They drift from each other as well
as away from production.
The Norm
• Manually
• Tribal knowledge
• Some documentation, generally out of
date
• Lots of options, room for variation
• Unreliable results
• Difficult to reproduce same results
The Goal
•
•
•
•
•
•
•
Fully automated
Infrastructure as code
Institutionalized knowledge
Code as documentation
No options
Tested
Reliable, repeatable results
Needs
• We need a way to unobtrusively
manage all the developer/tester
workspaces on a project
• We need all the software and
configuration that makes up the
workspace to be centrally managed
• We need all the workspaces to look
and function identically
• We prefer to use the same tool to
manage workspaces as we use to
manage servers
Chef
• Automation for installing and configuring
software
• Open Source
• Supports Windows, OSx, and Linux
• Why Chef and not Puppet?
– Internal DSL, more powerful than Puppet’s external DSL
– 100% free, Puppet is only partially free
• Can bootstrap to create setup application
• System level CM
• Testable with Vagrant or EC2
– Higher quality from agile testing
DEMO
HTTP://JAYFLOWERS.COM/MISC%20DOWNLOADS/WORKSPACE-SETUP.EXE
Chef Modes
• Server/Client
– Secure communication, authentication, and
authorization with public/private keys
– Servers control execution of clients
• Solo
– Client without a server
– Execution is initiated at the command line on the
client
When to use Solo and when
to use Server/Client
• Solo for workspace management
– Updates controlled by workspace owner, on
demand
– Cookbooks stored in workspace
• Server/Client for everything else
– Update controlled by server, pushed to clients
– Cookbooks stored on server
• We can use Chef to deploy the
applications we build!
Chef Anatomy
• Server/Client or Solo
• Cookbooks (written to be idempotent)
– Resources and Providers – used to perform typical actions
(make user, create directory, install package, etc…)
– Attributes – parameters specific to the cookbook
• Nodes
– The client list of cookbooks to apply
• Roles
– Lists of cookbooks to be applied
– Helpful in organizing nodes into groups (e.g. frontend apache
server)
• Databags
– Parameters specific to the target nodes and or organization
Bootstrap Solo Configuration
require 'minitest-chef-handler'
file_cache_path "c:/tools/chef/cache"
cookbook_path [File.dirname(__FILE__) + "/Chef/cookbooks"]
file_backup_path "c:/tools/chef/backup"
role_path "c:/tools/chef/roles"
json_attribs File.dirname(__FILE__) + "/node.json"
handler = MiniTest::Chef::Handler.new(:path =>
File.join(Chef::Config[:cookbook_path], "*", "test", "*test*.rb"))
report_handlers << handler
log_level :debug
log_location STDOUT
Node.json
{
]}
"run_list": [
"recipe[git]",
"recipe[tortoisegit]",
"recipe[get_git_source]",
"recipe[springsource_sts]" ,
"recipe[java]",
"recipe[sysinternals]",
"recipe[virgo]",
"recipe[gradle]",
"recipe[notepadplusplus]",
"recipe[maven]",
"recipe[groovy]"
Java Cookbook
windows_package "java" do
source node['java']['url']
checksum node['java']['checksum']
action :install installer_type :custom
options "/s /v \"/qn INSTALLDIR=#{node['java']['java_home']}\""
not_if { File.directory? node['java']['java_home'] }
end
Git Clone Cookbook
dir = node['get_git_source']['dir']
directory dir do
inherits true
recursive true
action :create
end
Git Clone Cookbook
ruby_block "clone_git_repo" do
block do
url = node['get_git_source']['url']
Chef::Log.info("Cloning the repo #{url} to directory #{dir},
this could take a while...")
git = Chef::ShellOut.new("git clone #{url} #{dir}")
git.run_command
puts git.stdout
if git.stderr != ''
puts "error messages: " + git.stderr
end
# Raise an exception if it didn't exit with 0
git.error!
end
action :create
Git Clone Cookbook
not_if do
Chef::Log.info("Checking if #{dir} is a working git repo")
git = Chef::ShellOut.new("git status", :cwd => dir)
git.run_command
puts git.stdout
if git.stderr != ''
puts git.stderr
end
if git.exitstatus == 0
true
else
false
end
end
end
Chef Foots the Bill
• Written to easily manage large numbers
of servers, translates to workspaces too
• Installs and configures software and
operating systems creating reliable and
repeatable results
• Creation of new servers and workspaces
becomes so easy allowing us to treat
them as disposable resources
• Can be used to repair/heal broken
workspaces and servers
Benefits
• What works on my box works
everywhere else…
• Dramatically reduced time to create
new workspace
• Workspace updates are non-events
• Heal broken workspace
• Can be managed by unskilled team
members
• Pairs well with other scripting
(e.g. automated deployments)
Cucumber
•
•
•
•
Test Tool
Open Source
Tests are written in plain text
Tests are implemented in Ruby, the
same language as Chef is
implemented in…
• Tests use virtualization libraries
to create, control, and destroy
test instances/environments
Simple Example
Cucumber Example
Minitest Example
Test Driven
• These tools enable a test driven
approach
• Tests can be written before Chef
cookbooks are written
• Tests can be executed in a local
workspace as well as on a CI
server
Test Resources
•
•
•
•
ChefSpec - https://github.com/acrmp/chefspec
MiniTest - https://github.com/calavera/minitest-chef-handler
Test Kitchen - https://github.com/opscode/test-kitchen
Examples
– https://github.com/opscode-cookbooks/apache2/
– https://github.com/opscode-cookbooks/mysql/
A LOOK
UNDER THE HOOD
HTTPS://GITHUB.COM/JFLOWERS/VIRGO.SAMPLE-GREENPAGES