Automating
Deployments with
Octopus Deploy
“A L L P R O B L E M S A R E B O R I N G P R O B L E M S U N T I L T H E Y ' R E Y O U R O W N ”
- G A L I N A “ R E D ” R E Z N I KO V ( O R A N G E I S T H E N E W B L A C K )
Who am I?
Ian Paullin
http://ianpaullin.com (Twitter: @ianpaullin)
C#, ASP.NET developer for 5 years with Avanade and 9 years of .NET development experience
Not ALM or DevOps guy (seriously reconsidering)
Computer nerd who likes too much tech
Proud Seahawks fan
Soon to-be Dad
What do I do?
November 2013 – Jim Szubryt from Accenture contacted me with opportunity to tackle
automated TFS deployments for all of Accenture CIO
Chance to fix this issue once and for all (delusions of grandeur)
Opportunity to test out two tools head-to-head and implement enterprise-wide automated
deployment solution throughout Accenture world-wide
◦ Release Management for Visual Studio (InCycle/Microsoft)
◦ Octopus Deploy
Why present Octopus Deploy
Deployments are a neglected aspect of software development; they’re a last-mile problem
I’ve worked at several companies over the course of 9 years; deployments were always a hassle
My personal goal: make all organizations use Octopus Deploy to push automated deployments
Today’s presentation: to actually help you with your deployments using Octopus Deploy
Experiences with deployments
Excel spreadsheet check lists
Email as (poor) documentation
When something goes wrong, everyone is frantically trying to figure out the problem
Rolling back to previous version takes longer especially if database changes need to be reverted
Multiple deployment types (web,db,ssrs,services,etc.) drastically prolong deployment
Public deployments done on Saturday night (late)
◦ Conference call with a few developers, some leads and the IT staff who actually does the deployment
◦ A handful of people knows what’s going on
◦ Everyone smoke tests the site
The “real” reason I tout Octopus
“I abhor deployments”
Survey
How many people do deployments?
How long does it take on average?
How frequently do you deploy and to what environments?
What kind of apps/technology do you deploy?
What size company?
What technologies?
TFS Automation at Accenture
November of 2013 – Present – testing 2 products:
◦ InCycle/Microsoft Release Management 2013 for Visual Studio
◦ Octopus Deploy
After 3 months of testing, Octopus was the winner
◦ Completely web-based (no desktop client)
◦ Better licensing (very affordable)
◦ Leveraged NuGet (very flexible)
Pilot phase from February 2014 – May 2014
June 2014 into production
◦ Goal of 120 different projects on-board and deploying to production by March 2015
◦ Currently have approximately 160 projects on-board
◦ Average 1000 deployments/week; 4000 deployments/mo.; projecting 8000-12000 deployments/mo. 2015
How does Octopus work?
Octopus uses NuGet packages as a storage mechanism
Using the OctoPack, TFS build template can generate NuGet packages and push them
automatically to a specified NuGet server
NuGet server stores compiled projects as NuGet packages available for deployment at any time
Agents (tentacles) are installed on servers you want to deploy to (2 modes: listening or polling)
You can define your environments and assign servers (tentacles)
A deployment process will fetch your NuGet package and run the defined steps in parallel
(across servers)
Deployment process is orchestration of steps (powershell) by environment, server roles
How does Octopus work? (con’t)
Why is Octopus better?
◦ NuGet is the better approach
◦ No need to recompile projects (if you need to rollback)
◦ Centralized storage; TFS build drop folders insecure and waste storage
space
◦ Prevent tampering of config files with locked-down NuGet server
(security)
◦ NuGet gallery for all (internally or externally)
Why is Octopus better? (con’t)
Powershell is the best choice for Windows deployments
Octopus records all PowerShell output by default
Store all web.config settings for all environments
Can automate web.config xml transformations
Can deploy to Azure and Amazon
Can use to deploy SharePoint, SSRS/SSIS, Windows Services, etc.
DSC (desired state configuration) – next big thing in PowerShell
Anything can be done with PowerShell!
Why is Octopus better (con’t)
SQL Server not required! (from 2.0 – 2.6)
◦ Uses Embedded RavenDB
◦ Octopus 3.0 will move back to SQL Server (enterprise-ready)
IIS not required!
◦ Uses NancyFX (Self-hosting)
Octopus REST API
◦ They consistently use their own API throughout all components
◦ You can call the API to automate/script your own solutions
Octo.exe, DbUp, Octopus Step Template Library are all open source
Force process over manual changes (best practices)
Other features of Octopus
Agents (tentacles) can use any port to connect to Octopus Server
◦ Many company networks may have very restrictive port access
Able to do Continuous Deployment on Check-in or scheduled releases
◦ Octo.exe can be called to trigger release on Octopus Server
Integrates with Active Directory or own user database
Active community suggestion forum
Great support
Rapid development schedule
Meh, our manual deployments are fine
People love to stick to the idea of simple, even if its burdensome, and prone to mistakes
Solid automation trumps manual any day; manual always seems good enough until everything
goes wrong of course, and then..
Let’s get started!
Three major components to getting started in Visual Studio
1. OctoPack – a NuGet package to install for your project
a. Hooks into TFS build template to create NuGet package and push to server
2. NuSpec file – a file that tells what files to store in the NuGet package
a. Use command line “nuget.exe spec <project name>” to create nuspec file in project folder
3. Change TFS build template (MSBuild arguments)
a. Include the following three parameters:
/p:RunOctoPack=true
/p:OctoPackPublishApiKey=<YourCustomApiKey>
/p:OctoPackPublishPackageToHttp=<YourNuGetServerUrl>
Configure Deployment process for deploying your NuGet package
Make sure your tentacle is installed and configured for your environment you’re targeting
Note: You don’t need the OctoPack, but it
makes life a whole lot easier
OctoPack requirements
◦ .NET 3.5 runtime or higher
◦ Web Application Projects (not Website compatible!)
◦ Injects Octo.targets and commands in *.csproj file
Alternative to the OctoPack?
◦
◦
◦
◦
Build packages from command line in Visual studio using NuGet
Create PowerShell script to mimic same behavior
Run MSBuild local desktop and pass paramters to OctoPack
Download New-Nugetpackage PowerShell Script: http://bit.ly/1sTxsbo
How to configure Octopus Deploy?
(con’t)
When your project NuGet package is in NuGet server feed, Octopus can see the package and
deploy to specific environment/servers
Define deployment process per project
Configure config file variable replacements (if you need to)
Can configure manual interventions for approval/denial (not required)
Email notifications (not required)
Downloadable Step Templates from library.octopusdeploy.com (no powershell knowledge
necessary!)
Basic rule: If you can get want you want deployed in a NuGet package, Octopus can deploy it
What about “X” type of deployments?
Database deployments
◦ Recommended practice is to have a Windows Console app that is a script runner; Octopus has
free dll (DbUp) available on GitHub/NuGet that allows Windows Console to execute SQL
scripts in folder in console app
◦ Check out http://dbup.github.io
SSRS deployments
◦ GitHub project “Deploy-SSRSProject.ps1” has PowerShell script to deploy reports to Reporting
Services asmx web service target
◦ Check out https://gist.github.com/jstangroome/3043878
Anything can be deployed as long as you can manipulate contents into a NuGet package
Demo
Cover Deployment Process, Steps, Variables and Versioning
Deploy using Octopus API
Simple Web Deployment
Build and deploy to development environment
Promote to staging and production
Crazy Implementations
Do you ever just get down on your knees and thank God that you know me
and have access to my dementia?
- George Costanza
Customization Examples
Provisioning of Teams (permissions), Environments, Projects and NuGet feeds via PowerShell in
Octopus Deploy
◦ Manually provisioning teams was too time consuming; created PowerShell to work with Octopus REST API
◦ You can use the NuGet feed in the Octopus Server but we use the free NuGet server from Nuget.org.
◦ Provisioning each feed manually takes time so we automated the process via PowerShell and added the
NuGet server with a tentacle – our PowerShell generates a random API key as well
◦ When provisioning is finished, we receive an email with all the project information to send out to the
requesting team
Created windows service to insert new user accounts (or remove) in Octopus teams
◦
◦
◦
◦
We have specific teams that have permissions to deploy in production only
Automating user management with custom windows service using C#, Octopus libraries and REST API
Service scans Octopus Project Groups for IDs to find match in external database
Inserts missing users into appropriate teams; removes users who aren’t in external database
Warts and road bumps
NuGet NuGet.exe and NuGet servers
◦ NuGet Developers seem to break compatibility or have numerous unaddressed issues
◦ Each new version has it’s issues; be careful in using latest and greatest
◦ Default NuGet.Server sucks with lots of packages – testing NuGet.Lucene/Klondike
(https://github.com/themotleyfool/NuGet.Lucene)
NuGet Feed & Package Management
◦ Challenging to manage numerous NuGet packages and feeds
◦ Few commercial options; hard to manually manage or automate across enterprise
IIS configuration (for NuGet server only)
◦ Default NuGet upload size is 30MB; must configure IIS for larger packages
◦ http://help.octopusdeploy.com/discussions/problems/184-30mb-default-maximum-nuget-package-size
Users and Team Management can be tricky
◦ With custom roles, anything is possible but difficult to implement
(layering of teams with appropriate permissions to specific environments)
Warts and road bumps (con’t)
Most errors are not Octopus Deploy but technologies around Octopus
◦ OctoPack issues largely with NuGet.exe – unhelpful error messages
◦ Networking issues can make connecting tentacles to server
Intended user base (currently) is for small-medium sized organizations.
◦ No true multi-tenant capabilities; we’ve managed with permissions and individual NuGet feeds
◦ Octopus 3.0 is a major leap into enterprise arena
Scalability
◦ It can scale*
◦ CPU/RAM/Storage/Network bandwidth – all crucial
◦ NuGet package management also very critical
Warts and road bumps (con’t)
Automation requires diligent education
◦ As with most products, users need to be educated – deployments need a lot of education due to
numerous technologies involved (IIS, SQL Server, SSRS, SSAS, SSIS, Azure, etc. etc.)
◦ Automation can be poorly done - we’ve had conference calls, recorded videos, documentation and
users can still struggle with automating deployments in a reliable, efficient manner
◦ People tend to stick to old habits; they use Octopus as if they were doing manual deployments
High-level best practices
Getting automation right requires lots of practice!
◦
◦
◦
◦
It’s a process: practice, practice, practice some more, then do it! (Use Dev as a base for practice )
Staging, testing environments should not require more alterations; if so – fix from dev and start over
All non-production environments are practice for production!
Aim for consistency in process across environments! Less differences = higher probability for success
What benefits have we seen?
High time savings
◦ Reduced deployment times (as much as several hours)
◦ Simple web projects can deploy in 1 to 3 minutes (avg)
◦ Example team: manual deployment time from 4.25 hrs to 1.5 hrs (still can be better!)
Savings in time translates to more frequent releases
◦ Teams able to deploy themselves – less support staff needed (IT support)
◦ The easier and faster it is to deploy, the more likely you’re going to do it
◦ Feedback loop frequency increases quality
Saving time + less staff overhead = significant financial savings
◦ Teams able to deploy on their own w/o support teams
Yeah, well, what about new stuff?
Constant community feedback at:
◦ https://octopusdeploy.uservoice.com/
Roadmap available (based from community feedback)
◦ http://octopusdeploy.com/roadmap
Aggressive schedule (2-4 week sprints; numerous patches/updates)
For reference, in 2014, they’ve had 7 releases! (From 2.0 to 2.6)
This year they’ve making major changes in 3.0:
◦ Switching back SQL Server for customers and for HA/DR (high-availability/disaster recovery)
Well, they must be pricey, right?
Perpetual license! Once you buy it, it’s yours plus one year of updates
Renewals are for updates and support
Renewal pricing is 50% of original cost
Octopus Licensing:
◦ Free
$0
◦ Professional $700
◦ Team
$2000
◦ Enterprise
$5000
5 projects
10 agents
5 users
20 projects
20 agents
20 users
60 projects
60 agents
60 users
unlimited projects, agents and users!
For more information, check out http://octopusdeploy.com
Reasons NOT to use Octopus Deploy
Lessons learned
Almost _anything_ can be done (through PowerShell)
PowerShell expertise is not as mandatory thanks to PowerShell scripts available online (github)
and the Octopus Step Template library (at http://library.octopusdeploy.com)
User management is tricky, but can be _very_ granular
No real integration with CI is a benefit
Once you get it going for your environments, you want to continue adding even more to your
process
In large environments, automation is key; need to figure out policies and philosophy
REST API and Octopus client/platform library are awesome for extending/customizing Octopus
to your needs
Conclusions
Octopus Deploy is the tool to reliable, maintainable and fast deployments
Leveraging NuGet to the fullest extent and thus implicitly enforcing practices (release versioning,
nuget package versioning, NO MANUAL INTERVENTION etc.)
Auditing and automatic documenting of all output from deployment process beats email,
spreadsheets and chat history any day
Octopus Deploy’s components separates it from other solutions
No direct integration (IDE or CI server) allows creative solutions and flexible configuration
Other solutions are either too cumbersome, heavy-handed or outdated
Octopus’s product transparency and velocity easily surpasses competitors
Pricing is extremely affordable
Go forth and deploy!