DevOps & Continuous Delivery Accelerating the delivery of value to the business David Myers February, 2013 Mobility, big data, analytics, social collaboration and cloud are creating a new wave of business opportunities and IT challenges Platforms accelerates its reporting process from 20 days to several hours and cuts reporting costs by 50 percent New Era Analytics, Mobile and Social Web, e-business and SOA Transactio n Systems Interconnected Solutions Action at real time speeds Multiple Applications Some shared data Dedicated Systems Post Processing in Warehouses Single Database Batch Processing Managed one Solution Time to Business Action Processing occurs NOW !!! Business impact visualized Action taken immediately Strategic and Tactical differentiator Time 19602 1990- 2010- New era systems integrate existing operational systems with rapid delivery of new client-facing apps Social, Local, Mobile Smart Infrastructure Analytics CRM HR Legacy DB ERP Manage workloads and maintain security 3 Rapid innovation in the cloud User experience and mobile management The need for continuous delivery Significant pressure on business to: Innovate to create new business value by employing cloud, mobile and social channels and leveraging big data - Systems of Engagement Integrate with transactional systems - Systems of Record Balance speed with risk, compliance and quality Continuously deliver software-driven innovation and business value 4 Different application components evolving at different speeds Use the Pace-Layered Application Strategy to Guide Your DevOps Strategy – Gartner, Oct 2012 Impact of evolving customer and market expectations User Experience Today Emerging Primary Workload Type Systems of Record (Transactional) Systems of Engagement (+ Record) Big Data, Analytics, Mobile/Social Channels Delivery Model Planned Incremental (DevOps) Development and Operations 100s and Costly Team Sizes 10s with built-in DevOps automation Release Frequency Months to Years Days to Weeks, based on business opportunity Integration Frequency Weeks Continuous Infrastructure Deployment Days Minutes Time to Value Planned Opportunistic Operational Model Systems Management Built into application, Recovery Oriented Computing, Continuous Availability Service Sourcing Develop Consume and Assemble (Public and Private) 6 Today IT leaders must balance the optimization of existing systems and innovation through new engagement models Optimization • Focus on consolidation, virtualization, cost cutting • Align IT with needs of the CFO 7 Innovation • Focus on mobile, intelligent infrastructure, rapid development • Align IT with needs of the CMO So what is DevOps? Dev + Ops (Thank you…tip your waitress) A way of re-integrating IT to deliver business value faster and better Siloed delivery challenges Customers Dev & Test Teams Line of Business 1st Gap Desire for fast and continuous innovation Requirements Addressed by... Agile Dev Operations Team 2nd Gap Code & Tests Business Services Addressed by... Dev Ops Agile is not enough… Agile Dev Functional Testing Acceptance Testing Production Operator Setup (weeks) Install Agile builds are piling up Test and Ops teams have increased pressures to keep up with increased loads but continue to use waterfall approaches and traditional tools. DevOps is… A set of principles and values that facilitate collaboration across disciplines to… People 1. Enable rapid evolution of deployed business services 2. Reduce risk, decrease cost, and improve quality Process Tools DevOps Principles and Values • Collaborate across disciplines • Develop and test against a production-like system • Iterative and frequent deployments using repeatable and reliable processes • Continuously monitor and validate operational quality characteristics • Amplify feedback loops People Process Tools How do we make this happen? Automate everything Track and Plan Version everything Test everything Track and Plan everything Automate Application Artifacts Application Code Dashboard Runtime Environment Definition Version Instrumentation and Configuration Instrument and Audit everything Test Dashboard everything Instrument and Audit 13 Today’s landscape Selective and siloed solutions limit visibility across people, process and tools and create gaps in the delivery process Line of Business Software Development Test Operations GAP GAP GAP Poor alignment Manual handoffs Big Bang Release • Portfolio management • Agile developer tools • Test infrastructure automation • Cloud, virtualization • Business process management • Build automation • Test Management • Continuous integration • Test Automation • Infrastructure provisioning automation • Big Data/Analytics • App and middleware deployment automation 14 Challenges meeting market & business pressures with quality software Up to 4-6 41% 51% 45% experience delays in integration, configuration and testing of applications applications rolled back due to quality issues escaping into production experience delays due to troubleshooting and fine-tuning issues in production Software Development Line of Business GAP Test GAP Operations GAP 3-4 Weeks Weeks to deliver a simple change average time to isolate a defect 15 Time is now for DevOps Trends accelerating the need for Continuous Delivery Business Agility Development and Test Cloud Agile Development DevOps Cloud Computing Production Cloud Operational Discipline So what is Cloud Computing? A user experience and a business model Cloud computing is an emerging style of IT delivery in which applications, data, and IT resources are rapidly provisioned and provided as standardized offerings to users over the web in a flexible pricing model An infrastructure management and services delivery methodology Cloud computing is a way of managing large numbers of highly virtualized resources such that, from a management perspective, they resemble a single large resource. This can then be used to deliver services with elastic scaling Service Consumers Monitor & Manage Services & Resources Datacenter Infrastructure Access Services IT Cloud Cloud Administrator Service Catalog, Component Library Component Vendors/ Software Publishers Publish & Update Components, Service Templates 17 Traditional Dev - Ops Hand-off Installation Instructions RedHat Linux 1. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. 2. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Apache Web Server 1. Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo. 2. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt. Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet, consectetur, 3. adipisci velit, sed quia non numquam eius modi tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem. Python 1. Ut enim ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam, nisi ut aliquid ex ea commodi consequatur? 2. Quis autem vel eum iure reprehenderit qui in ea voluptate velit esse quam nihil molestiae consequatur, 3. vel illum qui dolorem eum fugiat quo voluptas nulla pariatur? Provisioning using Cloud #!/usr/bin/env ruby class DevopsDeployer def initialize(build_url, build_id) @log = Logger.new(LOG_FILE) @log.level = LOG_LEVEL @iaas_gateway = IaasGateway.new(HsltProvider.new(), LOG_FILE, LOG_LEVEL) @server_instance = nil .jsp .html rtc_build_system_provider = RtcBuildSystemProvider.new( RTC_REPOSITORY_URL, RTC_USER_ID, RTC_PASSWORD_FILE) @build = rtc_build_system_provider.resolve_build( build_url, ENV['buildResultUUID'], build_id) @build_system_gateway = BuildSystemGateway.new( rtc_build_system_provider, LOG_FILE, LOG_LEVEL) end .java def add_build_stamp template_file = WEB_APP_ROOT + "/app/templates/pages/page.html" @log.info "Adding build ID stamp #{@build.id} to \ #{template_file}" chef # Read in the file's contents as a string, replace recipes # the build_id, then overwrite the original contents .sh # of the file text = File.read(template_file) new_text = text.gsub(/\{\{ build_id \}\}/, "<a href=\"#{@build.uri}\">#{@build.id}</a>") File.open(template_file, "w") { |file| file.puts new_text } end Source Artifacts # ... Source Control Management Infrastructure as Code Delivery Pipeline Fundamentals .jsp .html .java .sh Build, Package, & Unit Test Application Binaries & Platform Configuration Deploy chef recipes Source Artifacts Source Control Management Deployable Artifacts Library Environment Running System Pattern of incremental adoption Maturity Build Automation Continuous Integration MultiPhase Automated Testing Continuous Deploy and Release DevOps Vision for Continuous Delivery Systems of Record Systems of Engagement Application Release Management & Automation Build Test Integration Production Business Owners Customers IaaS PaaS Continuous Delivery Pipeline: An iterative set of quality checks and verifications that each piece of application code must pass during lifecycle phases before being released to production. Results using DevOps Search Algorithm Release Process at Google Release Process at Wealthfront Release Automation at Etsy Release Automation at flipkart Examples from Etsy 2009 2012 6-14 hours 15 mins “The deployment army” 1 person Highly orchestrated and infrequent Rapid release cycle Special event – highly disruptive Commonplace – happens so often we cannot keep up Blocked for 6-14 hours, plus minimum of 6 hours for redeploy Blocked for 15 minutes, next deploy will only take 15 mins. Release branch, database schemas, data transforms, packaging, rolling restarts, cache purging, scheduled downtime Mainline, minimal linking and building, rsync, site up Slow Complex Special Fast Simple Common http://www.slideshare.net/mikebrittain/continuous-delivery-the-dirty-details 30+ deploys a day… • How do you build features so quickly? o Software Deploy ≠ Product Launch o Deploys frequently gated by config flags (canary releases) o Validate in production, hidden from public. • What’s in a deploy? o Small incremental changes to the application o New classes, methods, controllers o Graphics, stylesheets, templates o Copy/content changes o Turning flags on/off, or ramping up • Quickly responding to issues o Security, bugs, traffic, load shedding, o adding/removing infrastructure. o Tweaking config flags or releasing patches. • No Release Manager Where to get more information? • Enterprise DevOps blog o http://ibm.co/JrPVGR • 6 Ways for Enterprises to Adopt DevOps blog o http://ibm.co/xq71xY • Pulse 2012 DevOps Interview o http://ibm.co/OBqS2z www.ibm.com/software/rational © Copyright IBM Corporation 2012. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others. 27