WH I T E P A PE R
9 Metrics DevOps Teams
Should Be Tracking
9 Metrics DevOps Teams
Should Be Tracking
Change is never easy, but when beginning any transformation, it’s important to identify
specific objectives and to determine how those objectives will impact the bottom line.
Because DevOps is a more of an adoption in culture it can be hard to put tangible metrics in
place to determine its success.
That is exactly what Payal Chakravarty from IBM dealt with during her team’s
transformation from Agile to DevOps. The team from IBM ended up creating a “DevOps
scorecard” to track key indicators of the team’s progress.
What does success look like? Ultimately this questions led to documenting what they
chose to be the success criteria for their transformation – “Ship code frequently without
causing a customer outage.”
Over time and through their experiences, the team found a “more granular way to track
success,” and broke down their self-defined mantra into “quantifiable success metrics that
could be represented in a scorecard.”
9 Metrics DevOps Teams Should Be Tracking
9 Metrics DevOps Teams Should Be Tracking
2
Here are the 9 metrics the IBM team used to track their progress:
1. Deployment Frequency
How often is the team deploying new code? “This metric should trend up or remain stable
from week to week.”
2. Change Volume
How many user stories and new lines of code are being deployed? IBM suggests another
important parameter to track around this metric is the complexity of change.
3. Lead Time (from development to deployment)
This is the cycle time from when new code starts development to when it successfully
gets deployed into production. Cycle time is an important indicator of efficiency in the
process — when tracked using value stream mapping, it can help the team to visualize
areas in the process which need improvement, such as handoff times between work
centers. “Lead time should reduce as the team gets a better hold of the lifecycle.”
4. Percentage of Failed Deployments
What is the percentage of deployments which have caused an outage or negative user
reaction? DevOps’ emphasis on building quality in from the beginning should reduce this
metric over time. This metric should be reviewed together with change volume. According
to IBM, “If the change volume is low or remained the same but the percent of failed
deployments increased, then there may be a dysfunction somewhere.”
5. Mean Time to Recovery (MTTR)
When failure does occur, how long does it take the team to recover from the issue? The IBM
team states this is a “true indicator of how good [the team is] getting with handling change.”
Spikes in MTTR are fine for complex issues which the team has never encountered before,
but the overall trend for this metric should decrease over time.
6. Customer Ticket Volume
“This is a basic indicator of customer satisfaction,” and an insightful metric to track. From
the team’s own defined success criteria, the goal of their DevOps transformation was
to ship more frequently without causing customer outages, and the number of tickets
generated by users is a good indicator of how well they’re doing in achieving that goal.
7. Percent Change in User Volume
“Number of new users signing up, interacting with [the team’s] service and generating
traffic.” Tracking this metric can help ensure that your infrastructure is able to meet
demand.
9 Metrics DevOps Teams Should Be Tracking
3
8. Availability
Were any SLAs violated, and what is the overall uptime for the product or service? If you can
maintain healthy uptime even given fluctuations in user volume, you’re tracking pretty well.
9. Performance (Response Time)
“This metric should remain stable irrespective of % change in user volume or any new
deployment,” and indicates that the product or service is operating within predetermined
thresholds.
These metrics are a great start for any organization. Whether your organization is
struggling to measure current DevOps success or just starting on its DevOps journey.
The fact is that companies who can successfully adopt DevOps initiatives can deliver
applications faster — going from what was once measured in weeks or months, to days
or even hours. This means delivering value to your customers in ways that your
competitors can’t.
Want to learn more about DevOps Best Practices? Check out our white
paper The Business Case for DevOps.
WHITE PAPER
If you want to learn more about how Datical is bringing DevOps to the
database, contact us or see at demo at www.datical.com.
The Business Case for DevOps
www.datical.com | info@datical.com | Tel: 949.DATICAL
Copyright ©2019 Datical, Inc. Datical, all products prefaced by the word Datical and the Datical logos are either registered trademarks or trademarks of Datical, Inc. in the United
States and/or other countries. All other products mentioned are either registered trademarks or trademarks of their respective corporations. 201902
9 Metrics DevOps Teams Should Be Tracking
4