White Paper 5 Steps to Eliminating Database Change Risk 1 | 5 Steps to Eliminating Database Change Risk Table of Contents Two-thirds of change requests require a schema update……...……………………………..……...2 Overview - Composite Risk Management………………………………...………....………...……....2 Identify the hazards……...….…….…………………………………....…………………………….......3 Assess hazards to determine risk……………….…..….…………………....………………..…...…...4 Develop controls and make risk decisions……...………………...…………………………...…….…5 5 steps to eliminating database change risk….……………………………………...……….….….…6 Implement controls…….……………….…….……..……………………….………………………....…7 Supervise and evaluate…………………………………………………………………………………...7 Datical DB…….……………….…….……..……………………….………………………....…...…...…8 About Datical…….……………….…….……..……………………….………………………...……..…10 2 | 5 Steps to Eliminating Database Change Risk 5 Steps to Eliminating Database Change Risk “Updating the schema carries a 78% probability that your application will “break” somewhere along the release path to production.” In new research conducted by Simon Management Group (SMG) among Fortune 1000 companies, we’ve learned that two-thirds of change requests require a schema update. Updating the schema carries a 78% probability that your application will “break” somewhere along the release path to production. If you are currently managing your schema changes manually, you are setting yourself up for unnecessary risk in your application lifecycle. Enterprise Risk Management has gained a lot of attention recently, especially since the financial crisis of 2008. Groups like the Committee of Sponsoring Organizations (COSO), a joint initiative between accounting organizations, and the International Organization for Standardization, ISO, have published frameworks for managing risk in the enterprise. These frameworks bear many similarities to the U.S. Army’s process for managing risk – Composite Risk Management. Intuitively this makes sense – why expend the effort to analyze the nature of risk as it applies to the enterprise, when one can adapt a comprehensive methodology that combat leaders have successfully employed for centuries? This white paper presents an overview of Composite Risk Management, and discusses the process of updating your database schema within the application release process as a practical example. Overview - Composite Risk Management (CRM) There are several guiding principles for applying the framework – how do they apply to your application lifecycle? Integrate CRM into all phases of missions and operations Every phase of your application lifecycle has unique risks associated with it Make risk decisions at the appropriate level Manage risk at the appropriate level, and know when an issue must be elevated 3 | 5 Steps to Eliminating Database Change Risk “Risk occurs when the state of the database is unknown. Will the changes that “worked” in your dev environment also work in QA, staging or production?” Accept no unnecessary risk Apply the process cyclically and continuously Accept no level of risk unless the potential gain or benefit outweighs the potential loss The process of managing risk is never complete Do not be risk averse At the heart of risk lies uncertainty, which also carries an equal chance for opportunity The CRM framework consists of five steps: 1. Identify hazards 2. Assess hazards to determine risk 3. Develop controls and make risk decisions 4. Implement controls 5. Supervise and evaluate Identify the hazards Some common hazards that organizations typically face when updating their database include: Hazard: Misalignment of version numbers between application and database code Risk occurs when only certain changes or feature requests will be deployed in a release – someone must manually determine what gets deployed. Hazard: Database configuration “mismatches” across environments Risk occurs when the state of the database is unknown – will the changes that “worked” in your development environment also work in QA, staging or production? 4 | 5 Steps to Eliminating Database Change Risk “. . have you ever run across an SQL script containing 2,000 lines of code? If a problem occurs, someone will have to manually inspect those lines to locate and troubleshoot the issue.” Hazard: Too much information in a single database release Risk occurs when single changes contain too much information – have you ever run across an SQL script containing 2,000 lines of code? If a problem occurs, someone will have to manually inspect those lines to locate and troubleshoot the issue. Hazard: Troubleshooting the database manually Risk occurs when an issue is identified and you can’t easily eliminate the database as a cause – productivity is decreased and all too often the project is delayed. Hazard: Uncertainty about the impact of database changes Risk occurs when there is drift between environments – how do you know that what you intended to occur in production will actually execute correctly? Assess hazards to determine risk “Hazards are assessed and risk is assigned in terms of probability and severity of adverse impact of an event/ occurrence.” So, there are two key inputs – the probability of a hazard occurring, and the hazard’s impact on the task at hand. The intersection of these variables determines the level of risk. The military employs a tool called a Risk Assessment Matrix for this purpose (see Figure 1). Figure 1 5 | 5 Steps to Eliminating Database Change Risk Probability should be scientific, based on industry benchmarks or metrics from your organization. Severity is largely subjective and should be defined according to the impact to your organization. Now, let’s assign risk to each hazard according to the matrix (see Figure 2). Figure 2 When assigning risk to the task (i.e. updating the database schema), the rule of thumb is to use the highest level identified from individual hazards – updating the schema is assigned a risk level of “High.” Develop controls and make risk decisions Controls are developed which will either reduce the risk of a hazardous event occurring, or eliminate the risk entirely. To be effective, each control must meet the following criteria: 1. Suitability – The control must remove the hazard or mitigate residual risk to an acceptable level. 2. Feasibility – The organization must have the capability to implement the control. 3. Acceptability – The benefit gained must justify the cost in resources and time. 6 | 5 Steps to Eliminating Database Change Risk 5 Steps to Eliminating Database Change Risk Hazards are reassessed to determine the residual risk level (i.e. level of risk after the control has been implemented). Risk decisions are based on residual risk only. If residual risk is higher than acceptable by management, additional controls are developed and the hazards reassessed. 1. Hazard: Misalignment of version numbers between application and DB code a. Control: Treat database changes like Tier 1 artifacts i. Check database changes into your source control repository ii. Synchronize database changes with application code 2. Hazard: Database configuration “mismatches” across environments a. Control: Maintain awareness about the state of the database i. Maintain a log that tracks the current state of the database ii. Maintain a log that tracks what changes have been made to specific databases 3. Hazard: Too much information in a single database release a. Control: Deploy database changes as atomic units – cohesive units with your application code i. Break database changes down into the smallest units possible 4. Hazard: Troubleshooting the database manually a. Control: Provide proof that the database is not at fault i. Require reporting within your organization to build an audit trail which can be quickly referenced 7 | 5 Steps to Eliminating Database Change Risk 5. Hazard: Uncertainty about the impact of database changes on production a. Control: Maintain awareness concerning drift in your environments i. Employ automation that provides a log of changes made to specific environments ii. Strive to ensure environment integrity and consistency Now, we reassess each hazard and determine the residual risk (see Figure 3). Once again applying our rule of thumb, updating the database now carries a residual risk of “Moderate.” If management decides that this is acceptable, then we are complete with our risk assessment. Figure 3 Implement controls This is why your manager makes the big bucks – he’s the one who has to enforce the controls. We’re betting that he tells you that you’re now in charge of your organization’s risk management process. Supervise and evaluate Now, all you have to do is design and implement four daily logs and a reporting system. Oh, and you’ll also have to go to everyone every day for at least three months to make sure that they’re using the logs and reporting system correctly. After a period of time, you’ll also have to check all of the logs against actual data to evaluate whether 8 | 5 Steps to Eliminating Database Change Risk “[Datical DB] empowers your developers to accurately and efficiently update the schema simultaneously with their application updates, using an object-oriented approach so they don’t have to worry about SQL syntax.” the system is working. If it worked, you should immediately buy a powerball ticket. If it didn’t work quite the way you planned, it’s time to troubleshoot the process, identify the discrepancies, and begin anew… Repeating a key point: “...controls are developed which will either reduce the risk of a hazardous event occurring, or eliminate the risk entirely.” Updating schemas is inherently risky. Developing a robust logging and reporting system is time-consuming and frustrating in addition to being risky. Why not avoid the heartache and eliminate the risks entirely? That’s where Datical comes in. Datical DB Datical DB is a database change management solution, based on Liquibase, specifically developed to help you eliminate risk and uncertainty when updating your database as part of the application release process. Unlike the current DBA tools available, Datical DB employs a holistic approach while seamlessly integrating with your current process and tools. It standardizes the process of updating your database schema so that you can keep application and database code in synch throughout the application release process: Maintains an automated running log of all executed database changes and compares database configurations to provide instant visibility into the current state of your database. Organizes and tracks your SQL scripts while providing the entire team with visibility into what’s happening in the database. Empowers your developers to accurately and efficiently update the schema simultaneously with their application updates, using an object-oriented approach so they don’t have to worry about SQL syntax. Augments your DBA’s ability to facilitate the process, troubleshoot problems, and recover from disaster. Enables you to eliminate configuration drift between databases and perform fresh installations in a matter of seconds. Most importantly, it provides you with proactive environmental intelligence concerning the impact of changes on the database, before you deploy. 9 | 5 Steps to Eliminating Database Change Risk “Datical DB makes it very easy to reduce errors and application downtime, speed time to market and lower your cost.” Instead of providing yet another framework that requires you to spend a lot of money, time and effort to change your existing systems but still doesn’t provide change integration, Datical DB simply works with the systems you have. Whether that’s Source Control, Build Systems, Testing Systems and existing Automation Frameworks – Datical DB makes it very easy to reduce errors and application downtime, speed time to market and lower your cost by: Eliminating database change risk and bottlenecks in your develop-test-deploy process Accelerating releases and automating deployment processes Knowing how changes will impact production and any other environment Improving compliance with auto-documenting change management processes Increasing quality of service with a holistic, repeatable, databaseagnostic approach We invite you to learn more at www.datical.com. About Datical Datical solves critical business pains by leveraging proven open source projects, such as Liquibase. We enhance and extend open source projects while removing complexity. We make them easy to use. We make them open. We make them flexible, secure and fully supported. But first, we complete extensive market validation research so we know we’re solving the right problems and adding the right features, in an easy to use, non-disruptive way. We’re not interested in checking off boxes on a feature comparison checklist. We’re focused on solving our customer’s most complex database change problems, protecting critical data and reducing risk. That’s how we deliver serious value. And our customers see quick results. We believe it’s because we’re fanatically market driven and have taken a different, model-based approach in architecting our solutions. This makes it possible to fuel automation with environmental intelligence, visibility and flexibility – extending the very best open source projects for a future proof enterprise. 10 | 5 Steps to Eliminating Database Change Risk 600 Congress Avenue Suite 1650 Austin, Texas 78701 www.datical.com 949.DATICAL info@datical.com 1 United States Army (2006). FM 5-19, Composite Risk Management. Washington D.C. Headquarters, Department of the Army. Simon Management Group (2012). Datical Market Validation Study. Austin, TX. © Copyright 2013. All rights reserved. Datical, the Datical Logo and Datical DB are trademarks of Datical, Inc. All other company and product names are trademarks or registered trademarks of their respective holders. 2