Design your own
KPI based Scorecard
Wednesday September 23
2.00 PM – 6.00 PM
Mustang 6
Workshop overview
•
•
This session is designed for professionals who operate in the field of IT
performance management and allows them to create a KPI based
scorecard using KPIs that actually make sense to the business
The following are the objectives for this session:
–
–
–
–
Understand the principles of successful IT Performance Management
Are able to identify what KPIs are relevant to their business
Learn to identify quality KPIs
Using a Software as a Service environment can configure a scorecard and
dashboard
– Are able to interpret information in a dashboard and use this for effective
decision making
•
This workshop focuses on the process for the creation of an IT KPI based
scorecard and will not go into the details of data collection, data quality or
data transformation
2
Workshop Agenda
Session
Title
Description
Type
Time
1
The current state
of IT Performance
management in
your organization
Introduction of the participants and
Discussion on maturity of IT Performance
Management in your organization
Assignment
2.00 – 2.10 PM
2
Current state of
the industry
An overview of the current state of the IT
Performance Management industry
Presentation and
Group Discussion
2.10 – 2.30PM
3
IT Performance
Management
An overview of the IT Performance
Management process, terminology and
concepts
Presentation
2.30 – 3.00 PM
4a
How to
Defining IT Performance management
objectives and identifying key processes
relevant to your organization
Presentation
3.00 – 3.45
4b
Define / Measure
Define requirements for IT Performance
Management: objectives, audience, scope
Individual Exercise
3.45 – 4.30 PM
5
Scorecard Design
Create an IT KPI based Scorecard based
on the current state of your organization
and your requirements
Presentation and
Group Exercise
4.30 – 5.30 PM
6
Review of
Scorecards /
Dashboards
Analysis of a number of Scorecard and
Dashboard examples
Presentation and
Group Discussion
5.30 – 5.45 PM
7
Questions and
Answers
Q and A session and definition of next
steps
Group Discussion
5.45 – 6.00 PM
3
Session 1
CURRENT STATE OF YOUR
ORGANIZATIONS‟ IT
PERFORMANCE
The current state of IT Performance Management
in your organization
Q1) What is your opinion on the following statement:
“ For too many organizations IT is a black box. Projects and systems are so
complex that few CIOs can predict a direct impact on the business, making
it hard to win budget and resources even in prosperous times”
Agree
Agree to some extent
Disagree
Not sure
Source: Information Week 2008 - Hunting the Elusive IT Dashboard
The current state of IT Performance Management
in your organization
Q2) To what extent does IT management in your organization / your
customers have a holistic picture on and decision making information
about the performance of IT?
They don‟t have a clear picture
They could, but information is fragmented and dispersed over
tools and reports
They do, but a lot of effort goes into consolidating various
reports into one view
They have a holistic picture of their overall IT performance
thanks to one or few regular concise reports
There is no clear trend to be distinguished
The current state of IT Performance Management
in your organization
Q3) How would you rate the maturity level of your organization (or that of
your customers) in terms of its IT performance management.
Level 0: IT Performance Management is non-existent.
Level 1: There is ad hoc monitoring in isolated areas.
Level 2: Some measures are set with a clear link to business
goals but are not communicated. Measurement processes
emerge, but are not consistently applied.
Level 3: Efficiency and effectiveness are measured and
communicated and linked to business goals and the IT
strategic plan. Continuous improvement is emerging.
Level 4: There is an integrated performance measurement
system linking IT performance to business goals by global
application of a documented framework. Continuous
improvement is a way of life.
The current state of IT Performance Management
in your organization
Q4) What is the main driver for IT Performance Management?
Reduction of IT cost
Establish the progress toward achieving goals
Optimal resource allocation
Be compliant with internal or external regulations
Require an insight into performance against
service levels
Identify internal improvement opportunities
Other
The current state of IT Performance Management
in your organization
Q5) What do you consider to be the key success factors for effective IT
performance management?
Having support from the business
Proper guidance on the implementation
Having a formalized service management framework in
place (ITIL, COBIT)
Having a useful software application to capture and
convey performance measures
Having an industry standard for metrics (common
language)
Having the proper resources in place (budget, people,
infrastructure)
Other. Please specify:
The current state of IT Performance Management
in your organization
Q6) Which tools are you currently using to report IT performance?
Business Intelligence (e.g. IBM, Hyperion, SAP)
Manual reporting (e.g. Excel)
Internally developed software solution
IT performance solution from external vendor (Metricus, M42 )
IT service management solutions (HP OpenView, BMC
Remedy…)
Service Level Management Monitoring Tools (Digital fuel,
Oblicore...)
Other. Please specify:
We don‟t use any tools to report IT performance
Session 2
CURRENT STATE OF IT
PERFORMANCE
MANAGEMENT
IT Performance Management market maturity
65% of IT Managers agree with an InformationWeek statement that says IT managers
have no idea what is happening inside their IT organization (according to an ITpreneurs
survey)
“For too many organizations IT is a black box.
Projects and systems are so complex that few
CIOs can predict a direct impact on the business,
making it hard to win budget and resources even
in prosperous times.
And when the CIO can't get a clear picture of the
real-time data that underlies critical applications,
infrastructure, and projects, IT too often ends up
reacting to issues after users and customers are
having problems.” (Information Week March
2008)
Source: Information Week 2008 - Hunting the Elusive IT Dashboard
Source: “Trends in IT Performance Management,” 2008/9, ITpreneurs’
survey and interviews among 99 IT executives and consultants
12
IT Performance Management market maturity
The majority of organizations are not reporting on IT Performance due to the time
and energy involved in doing this
They could, but information is fragmented and
dispersed over tools and reports
Collecting data
from various sources
Converting data
into logical numbers
Building complex
Excel sheets
Highly Ineffective
They do, but a lot of effort goes into
consolidating various reports into one view
They have a holistic picture of their overall IT
performance thanks to one or few regular
concise reports
They don't have a clear picture
0%
Percentage 2009
Source: “Trends in IT Performance Management,” 2008/9, ITpreneurs
Survey and interviews amongst IT executives and consultants
5% 10% 15% 20% 25% 30% 35% 40% 45%
Percentage 2008
13
IT Performance Management market maturity
IT Performance Measurement is emerging and the basics are getting in place but
there is no proper communication as yet
45%
Level 0
Level 1
Level 2
40%
35%
30%
Level 3
25%
20%
Level 4
15%
10%
IT Performance Management is non-existent.
There is ad hoc monitoring in isolated areas.
Some measures are set with a clear link to
business goals but are not communicated.
Measurement processes emerge, but are not
consistently applied.
Efficiency and effectiveness are measured and
communicated and linked to business goals and
the IT strategic plan. Continuous improvement is
emerging.
There is an integrated performance measurement
system linking IT performance to business goals by
global application of a documented framework.
Continuous improvement is a way of life.
5%
0%
Level 0
Level 1
Level 2
Level 3
Level 4
Note: this is a self assessment
Source: “Trends in IT Performance Management,” 2008/9, ITpreneurs
Survey and interviews amongst IT executives and consultants
14
IT Performance Management market maturity
The reduction of cost and the creation of clarity on the performance of IT in relation
to business goals are the main drivers for IT Performance management initiatives
Reduction of IT cost
Establish the progress toward achieving goals
Optimal resource allocation
Require an insight into performance against
service levels
Identify internal improvement opportunities
Be compliant with internal or external regulations
Other
0%
20%
40%
60%
80%
Source: “Trends in IT Performance Management,” 2008/9, ITpreneurs
Survey and interviews IT executives and consultants
15
IT Performance Management market maturity
Organizations look at IT best practices for support in identifying metrics and realize
they need to have both resources in place and buy-in from the business
Having a formalized service management framework in place
(ITIL, COBIT)
Having the proper resources in place (budget, people,
infrastructure)
Having support from the business
Having an industry standard for metrics (common language)
Having a useful software application to capture and convey
performance measures
Proper guidance on the implementation
Other
0%
percentage 2009
10%
20%
30%
40%
50%
60%
70%
80%
90%
percentage 2008
Source: “Trends in IT Performance Management,” 2008/9, ITpreneurs
Survey and interviews amongst IT executives and consultants
16
IT Performance Management market maturity
Manual reporting using Excel is favored in most organizations today followed
by the reporting capabilities provided by service management tools
Manual reporting (e.g. Excel)
IT service management solutions (HP OpenView, BMC
Remedy)
Internally developed software solution
Business Intelligence (e.g. IBM, Hyperion, SAP)
Service Level Management Monitoring Tools (Digital fuel,
Oblicore)
Other
We don't use any tools to report IT performance
IT performance solution from external vendor (Metricus, M42 )
0%
Percentage 2009
10%
20%
30%
40%
50%
60%
70%
80%
percentage 2008
Source: “Trends in IT Performance Management,” 2009, ITpreneurs
Survey and interviews IT executives and consultants
17
Session 3
IT PERFORMANCE
MANAGEMENT PROCESS
18
The IT Performance Management Process
IT Performance Management is about measuring, improving, and
demonstrating the value of IT
“IT Performance Management is the effective combination of methods,
metrics, data, and tools that enables organizations to define KPIs that are
relevant to them, understand their current performance against
predetermined goals, and enables organizations to build on this information,
initiate improvement activities, and achieve optimal IT performance in line
with business requirements”
* Metricus definition for IT Performance Management
19
The IT Performance Management Process
IT Performance Management is closely related to IT performance
measurement.
They are sometimes mistaken for each other. Strictly speaking, Performance
Management is the larger domain and includes performance measurement as a
component.
IT Performance Management
Define
Measure
Manage
Improve
IT Performance Measurement

Performance measurement is the process of assessing progress toward achieving
predetermined goals*. Performance Management builds on that process, adding the relevant
communication and action on the progress achieved against these predetermined goals.
* Wikipedia
20
The IT Performance Management Process
In this workshop we will touch on the first three phases of the IT Performance
Management lifecycle: define, measure and manage
Define
Initiate improvement
activities based on historic
data and scenario planning
Improve
Define what is important
to you and what you
should measure
Measure
Measure your IT performance
through best practice KPIs
and performance data
Manage
Manage the ongoing process and
present decision making
information to relevant
stakeholders in your organization
21
What drives IT Performance Management
“
What is not defined cannot be controlled.
What is not controlled cannot be measured
what is not measured cannot be managed or improved
”

Just like other business departments, IT has to continuously improve and ensure alignment with
the business

Ultimately the only way for IT management to demonstrate value and control is by defining,
measuring and managing IT performance

A great idea, but the idea often gets stuck at not being able to successfully measure IT
performance and not being able to bring everything together into a view that allows the IT
management to take informed IT decisions.
22
What drives IT Performance Management
Business Strategy drives IT
strategy and results in
defining, managing and
optimizing IT processes and
activities
This needs to be balanced
against the availability of
quality data and processes
that can provide information
from the bottom up
Business
Strategy
IT Balanced
Scorecards
IT Strategy
CIO Dashboard /
IT process health
drives
drives
IT Processes
ITIL/COBIT
Process
Scorecards
drives
IT Activities
IT Performance
Analysis
IT Performance
Metrics and
Measures
23
Structured KPIs drive performance
management for various users
IT KPIs (that are aligned with the business strategy) play a role at each
decision-making level in the organization and enable the organization to
take informed IT decisions in line with business requirements
Business
Strategy
Business Leadership
IT Balanced
Scorecard
drives
IT
Strategy
IT management
IT process health
dashboards
drives
Processes and
Activities
IT departments
Timely,
accurate,
and
quality
IT metrics
and
measures
(KPIs)
Functional and
process scorecards
24
From Measures to Decisions
Operational
Data
Measures
KPIs
Scorecards
Dashboards
Decisions!
IT Balanced
Scorecard
25
Operational Data Source
An Operational Data Source (ODS) represents data collected by a system or
process associated with the delivery and support of IT services.

A comprehensive understanding of ODS's within the context of the information ecosystem of an
organization is essential to the structured definition and development of IT Service Metrics.
ODS examples
Service Desk
e.g. FrontRange HEAT,
HP Service Manager,
ServiceCenter, CA
Unicenter, Service Plus,
Servicedesk, Numara
Footprint / Trackit, In
house applications, Open
Source applications
Manual
Decentralized,
disparate data related
to specific
users/processes
Custom
Internal IT applications
developed specifically
for IT support/delivery
functions
Enterprise Systems
Management
e.g. IBM Tivoli, HP
OpenView suite, CA
Unicenter, Nagios, Open
Source Applications
Enterprise Resource
Planning
e.g. SAP, Oracle Financials
26
Data Management
Business objectives change, which has an impact on the requirements
for data. Thus keep in mind that this process is dynamic and requires
continual management
Data
Change
IT strategy
Insight
IT
processes
and
activities
Data
integration
process
?
Integration
Data
Warehouse
KPI based decision
making information
27
Measures
Measures are distinct sets of data derived from mathematical calculations.
 Measures are quantifiable, for example, size, volume, or percentage,
and involve aggregation of data elements, for example, sum, average,
min, max, or count.
 In and of themselves, measures may or may not be meaningful.
However, measures represent building blocks for the metrics
required to make business decisions.
 Examples:
Calculation
Measure
Sum (incidents)
Volume of incidents
Sum (Incidents Resolved) where „Met Target‟ = „Y‟
Volume of incidents resolved within target resolution time
Average (Database Availability)
% Database Availability
Sum (IT Resources) where „Trained‟ = „Y‟
Number of IT resources trained
28
Metrics
Metrics consist of one or more measures combined with a mathematical
calculation and a standard presentation (format) for the output.
Dimensions
Time
Daily
facilitates
comparison
Measures
Volume of Incidents Resolved
Within Target Resolution Time
division
Categorization
Priority = „Critical‟
functional
categorization
Metric
% Critical Incidents
Resolved
Within Target
Volume of Incidents
 Metrics are associated with two dimensions: a time dimension and a functional categorization
dimension.
 Metrics are used in the quantitative and periodic assessment of a process that is to be
measured.
 Metrics should be associated with targets that are set based on specific business objectives.
 Metrics are associated with procedures to determine the measures required and procedures for
the interpretation of metric results.
29
Acronyms that are used for „metrics‟
Numerous terms and acronyms exists for labeling „metrics‟

Impact related - Key Performance
Indicator (KPI), Critical Success Factor
(CSF), Outcome Measure, Performance
Indicator

Operational related – IT Service Level
Agreement – SLA, IT Operational Level
Agreement (OLA), and Service Level
Objective (SLO)

Term „Metric‟ often used interchangeably
with „Target‟, „Benchmark‟ and „Goal‟

Clear definitions are required in order to
facilitate the necessary communications,
and to set appropriate expectations, with
users.
CSF
Outcome
Measure
KPI
IT Performance
Metric
OLO
SLA
SLO
30
Characteristics of good metrics
31
Metric Presentation
Concept
Description
Dashboard
•
A dashboard is a graphical display of the status of a
selected set of key metrics
Scorecard
•
Scorecards are the consolidated tabular and graphical
display of sets of metrics related to particular business
functions.
Visualization
•
Visualization reporting solutions show relationships
between selected metrics and assist in performing
impact analysis
Interactive Data Analysis
•
Online interactive analysis of data providing „slice and
dice‟ and „drill-down‟, „drill-through‟ capabilities. Known
as OLAP (Online Analytical Processing).
Scheduled Reports
•
Individual reports related to specific data requirements.
Operational Reports
•
Reports available directly from operational systems
used to support processes
Monitoring
•
Near real-time display of detailed data related to the
delivery of a service.
32
Metric Presentation
In theory various users in the organization have access to
information that is timely and relevant only to them
Concept
Dashboard
Scorecard
Visualization
Executive
Managers
Functional
Managers
Interactive Data Analysis
Scheduled Reports
Process Managers
Operational Reports
Monitoring
Operations
33
Metric Presentation
In practice many IT managers want to have access to very
detailed reports. Also, scorecards or interactive data reports are
only sporadically used
Concept
Dashboard
Scorecard
Visualization
Executive
Managers
Functional
Managers
Interactive Data Analysis
Scheduled Reports
Process Managers
Operational Reports
Monitoring
Operations
34
Dashboards
“A dashboard is a visual
display of the most
important information
needed to achieve one or
more objectives;
consolidated and arranged
on a single screen so the
information can be
monitored at a glance”
Intelligent Enterprise, 2004
Green IT dashboard
 Highly visual, data aggregated from
various sources
 Provide a one glance view on the
current state of the
organization/process or activities.
 Typically used by CIOs, IT
Management and business users.
35
Scorecards
„Scorecards‟ focus on the consolidated presentation of metrics and
presents an accurate view of the here and now compared with predefined
goals
Service Desk Scorecard
 Graphical representation of trends
and alignment with defined targets is
provided.
 They are supported by static and
interactive reports, as well as
diagrammatic representations of
metric performance and linkages
between metrics.
 Categorization of metrics within a
scorecard is typically related to
functional or process views.
Metricus Scorecard from the Service Desk Module
36
Session 4
DEFINE REQUIREMENTS FOR
IT PERFORMANCE
MANAGEMENT
37
Method 1) GQM – goal, question, metrics
GQM is a technique to derive measures for project control starting from
high-level goals, passing through the decomposition in several questions
to answer.
Author: Victor Basili in the early „80s (NASA Goddard Space Flight Center)
Measurement object: software projects
Application: GQM is one of the most well-known and used measurement
approaches for establishing a measurement program
Source: L.Buglione & A.Abran © 2000
38
Method 1) GQM – goal, question, metrics
Using the Goal Driven Method
Business Objectives
Increase Pace of
Service Delivery
…
Improve Service
Availability & Stability
Deliver Projects
on Time / on budget
Reduce outage minutes
Reduce number of incidents
…
Process Effectiveness
Incident
Reduce MTTR
Reduce MTBF
Problem
Decrease days to root
cause determination
Decrease days to
implement permanent fixes
Change
Decrease problems
due to change
Decrease issues during
change implementation
Process Governance
Complete
Are mandatory
fields filled out
Correct & Valid
Are data entered according to
process policies & standards
Does the data reflect what
actually happened
Source: David Chiu of BMO Financial
Detailed
Do text fields contain
appropriate level of
details
39
Method 2a) Balanced Scorecard
BSc is a multidimensional framework for “translating (organizational) strategy
into action” at all levels of an enterprise, by linking objectives, initiatives and
measures to an organisation‟s strategy
Authors: Kaplan & Norton (HBS) in the early „90s, originating from a French method
called Tableau du Bord (turn of 20th century)
Main measurement object: whole organisation / a SBU
Application: BSc is one of the most relevant management practices of last 75 years,
according Harvard Business Review Journal
Source: L.Buglione & A.Abran © 2000
40
Business strategy
IT strategy
Method 2b) Balanced Scorecard and COBIT
COBIT provides a bridge between business risks, control needs and technical by
providing a comprehensive framework for delivering value while managing risk
and control over data and information throughout the lifecycle.
Authors: ISACA in the „90s, originating from being an „auditors check list‟
Main measurement object: The entire IT organization: IT resources need to be
managed by a set of naturally grouped processes. COBIT provides a framework that
achieves this objective.
Source: COBIT 4.1, ISACA
42
Method 3) COBIT
Aligning business goals with IT goals using COBIT, example for DS5
Source: COBIT 4.1, ISACA
43
Assignment
We are going to use the following model from COBIT to define a process based KPI
scorecard
Business Goals
Step 1 define business goals
Step 2 define IT goals
IT Goals
Step 3 process
goals
IT process goals
IT
management
IT process
health
dashboards
drives
IT
departments
Functional and
process
scorecards
IT KPIs
Step 5
Scorecard
Step
4
KPIs
44
Use the following template for step 1, 2, 3
Business goal
IT goals
Processes
45
Case description
At the Coffee Company, coffee is not merely a bean or a beverage.
It is the furthest thing from a product or a commodity. Coffee
involves relationships and responsibilities. It is a process involving
high standards and tough decisions. More than anything, coffee
has the connective force to enrich peoples‟ lives.
The Coffee Company is a gourmet coffee company, serving highquality coffee blends and a specialty selection of coffee by the cup.
Espresso and related food and beverage products complement the
offerings. Along with hot and cold liquid servings, the Coffee
Company offers gourmet beans in retail bags through a retail
channel as well as an online channel. All offerings are provided in an
atmosphere that whispers of the warmth and convenience of your
living room.
From 1991 to date, the Coffee Company experienced and realized
significant growth in size, revenue, and number of stores. From only
one store in downtown Toronto, the company has grown to 12,000
retail stores and 1 online store.
the current economic situation has made the Coffee Company
shift its focus from expansion to consolidating business with
its existing stores and focus on ensuring the uninterrupted
availability of its coffee products and services.
The coffee production process
Farming,
drying,
extracting
Roasting
Coffee grows best in a
warm, humid climate
with a relatively stable
temperature of about
27C all year round.
The world's coffee
plantations are
therefore found in the
so-called coffee-belt
that straddles the
equator between the
tropics of Cancer and
Capricorn.
Distribution
After roasting, coffee is
packaged and sent to
one of our five regional
distribution centers in
Vancouver, Rotterdam,
Singapore, London and
Houston. The distribution
centers supply our retail
stores manage the
distribution from the
regional internet sales.
Un-roasted 'green' beans are
shipped to Rotterdam in the
Netherlands and Vancouver,
Canada from locations as
close as Hawaii and as far
away as Indonesia. Varietals
are then roasted or mixed
with other beans to create
blends, such as our popular
VOC or Da Vinci blends.
Retail
stores
customers
Online
store
Our outlets are company
owned and franchise
locations worldwide. The
distribution centers
supply our stores twice a
week or more depending
on demand. Each
retailer is strongly
connected to the
community as well to
ensure delivery of fresh
products such as breads,
bagels, etc.
Fragmented information from IT
Retail store managers are getting
tired of constantly having to wait
more than 5 minutes on the
telephone with the service desk.
Before they get someone on the
line..They have a shop to run!
So as a result they
don‟t call but bypass
the service desk and go
directly to the
specialists
I am the incident manager, and I
am proud of closing incidents
within targets by my first level
support team
A change weekend is always
exciting for the IT organization..
Will there by any store with
problems with their online
systems or does everything go
well? In case of issues, the
challenge is to find out what went
wrong and how to revert. Testing
plans are not maintained
centrally and there is no review
of implemented changes
Our CMDB is not very
easy to use and as a
result its‟ not kept up to
date always
Change management is a
rush job, and our change
manager agrees to all
requests from the
business as they feel
everything is important.
It seems that we keep
reopening incidents as our
solutions don‟t seem to
work, what‟s going on. We
need a knowledge base..
Session 5
CREATE AN IT KPI BASED
SCORECARD
49
Tips and Tricks for selecting KPIs
When you are presented with a KPI wish list that looks too good to
be true, it probably is.
1. Show me how
well we are
managing issues
in IT
CIO/Executive
Management



2a. Sure! Will
this scorecard
do?
Consulting
“Middlemen”
2b. We don‟t know
if it‟s possible but
that‟s not our
problem, we need
the sale.
3. We don‟t
have the
processes or
data in place…
IT Service
Transition Team
There is a gap between theory and practice, and there are not that many metric practitioners out
there
What looks and sounds nice is often not practical and possible.
Don‟t be put in a situation with „middlemen.‟ Work direct with the users of KPIs to ensure that
what is requested is actually feasible.
50
Tips and Tricks for selecting KPIs
Use only use KPIs when you know the data is available and only use
KPIs if the results can be tied to higher level operational, tactical or
strategic objectives



The horse and the cart: Only use metrics when you
know the data is available.
Confirm with all stakeholders the actual need and
planned use of each of the reported metrics
The requirement for metrics should not drive the
requirement for data; process and strategy should.
51
Tips and Tricks for selecting KPIs
Ensure that your „effort-to-insight ratio‟ remains positive
Insight
provided by
the KPI
Cost of
data collection



The cost for collecting data for a metric should weigh up against the insight that you are
retrieving from the metric.
The cost categories are data collection, Business Intelligence, and report development.
Costs are predominantly qualitative whilst benefits are both qualitative and quantitative.
52
Tips and Tricks for selecting KPIs
Garbage in = Garbage out: Poor data quality in IT services negatively affects
the ability for informed decision making throughout the entire organization.
 IT metrics should be developed and
presented with the same rigor as financial
accounting metrics.
 Consider: Integrity of data, consistency
with prior periods reporting, materiality
(Value of data must exceed cost of
reporting)
High
Quality of
IT Metrics
Low
High
Low
Data Quality
53
Tips and Tricks for selecting KPIs
Recognize that data integration will be the greatest challenge in developing
KPIs, scorecards, and supporting reports.

Focus on consolidating the data required to
support metrics and scorecards.

Don‟t assume „one approach fits all‟ for data
integration.

Build a 'data resource network,‟ including
developers, support and operations (DBAs),
and application SMEs.
Email
Word
Excel
PowerPoint
Face to Face
Telephone
Meetings
Memos/Letters
Unstructured
Help Desk
Monitoring
Systems
Structured
54
Tips and Tricks for selecting KPIs
Less is more, and it is recommended to start with only a handful of KPIs
rather than going with dozens of them

KPIs are put in place to be used to manage and
too many KPIs make managing to them difficult

For every KPI processes need to be in place to
collect data for every KPI and do something
with the results of the KPI.

Start small, keep it simple, and build upon
achieved successes.

Recognize that metrics will change over time
based on the changing values required from the
metrics and the growing maturity of the
organization
55
Tips and Tricks for selecting KPIs
Involve the right people when you are designing your IT Performance
Management initiative
Audience
Rationale
Executive Senior
Management
•
•
Identify Key People
•
•
•
Identify Metric Users
•
•
•
•
•
Implementing metrics and scorecards solutions is a strategic initiative that
requires C-level sponsorship (CIO, CEO).
This is generally recognized as the #1 Critical Success Factor to incorporate
any Performance Management Initiative or implementing Business Intelligence
software
The people that really matter to a process must always be part of it. This is
often a group so small that you can call the members out by name and count
them on one hand.
Key people need to be identified for requirements gathering, ownership of
metrics and reports, assistance with developing metric/scorecard solutions,
assistance in architecting and configuring the infrastructure required, and
support for data collection and metrics presentation.
Performance Management software solutions can provide guidance and
templates for scorecards and reports for specific users in your organization
Ask users what they need!
Collect key information for users.
Map metrics, scorecards, and reports to users.
Develop an authorization and application security model.
Keep it simple! Start on the premise of allowing users to see all information
and selectively revoke rights where appropriate.
56
Metric Requirements - SMART Now we have identified the Processes that have our key interest and that
are important for us when we want to meet the objectives of the business.
The next step is the identification of specific KPIs for the selected
processes.
First some guidelines around metrics:
 Metrics that are used need to be SMART:
 Specific
 Measurable
 Action oriented
 Realistic
 Time bound
57
Metric Requirements - SMART The definition of an IT Performance Metric needs to be structured in alignment
with defined attributes and valid measures
 This helps avoid the „100,000 feet‟ syndrome of documenting
„wish lists‟ of what sound like nice metrics to have, but in reality
are not meaningful, not possible, too costly, and so on.
Good Example
Poor Example
[% Calls Abandoned]. A fundamental Call
Center/Service Desk metric, this divides the
number of calls offered by the number of calls
abandoned. Both measures are readily available
and attributes such as time dimension,
categorization fields, targets, user base, and so on
are straightforward to obtain. (To be discussed
more in the section – „IT Performance Metrics –
Attributes.‟
Time used to resolve unavailable services.
What services are included? What is the definition of
unavailable? How is unavailable time equated with
effort to resolve? Is it linked through incidents? If so,
are the processes in place to record time?
58
Metric Requirements - SMART IT Performance Metrics need to be quantitative, allowing users to actually
measure progress and performance against the IT Performance Metric .
 The provision of an IT KPI should prompt the users to ask
questions.
 Supporting data needs to be directly available and/or
mechanisms available to obtain support data.
 Typically, quantitative metrics are supportable while qualitative
metrics are hard to support.
Good Example
Poor Example
[SRs Open] is a valuable IT service metric
but needs to be supported by details of
open service requests.
% Projects with Predefined Benefit
is often based on after the fact
„guestimates‟ as to whether predefined
benefits were available. For the metric to
be of value, information on what predefined
project benefits need to be stored in
conjunction with baseline project details.
59
Metric Requirements - SMART An IT Performance Metric should contain information that can be directly acted
upon.
 This could involve questions being asked as to why a metric is a specific
value, providing information on achieving defined levels of service, or
automatically instigating remediation action within a particular IT service
process.
Good Example
Poor Example
[% Service Requests Resolved On
Initial Contact] is often a metric contained
within a Service Level Agreement (SLA).
Failing to meet a defined contractual
target, say 85% will require action to be
taken in order to avoid possible penalties.
Incidents Created
While this is often interesting to look at,
particularly across a period of time, and it is
certainly easy to measure, it‟s hard to make
an educated decision based solely on the
results in isolation. This is actually a
measure that needs to be categorized with
IT performance reference data, such as
service or classification, in order to be of
value.
60
Metric Requirements - SMART An IT Performance Metric should be realistic from a data perspective, that is, the
data associated with underlying measures in the calculation of the metric needs
to be available.
 An IT Performance Metric also has to be justifiable from the initial and ongoing
costs. The effort in collecting the metric logically is lower than the value derived
from decisions related to the metric.
Good Example
Poor Example
[% Incidents Escalated]
The volume of incidents and an indicator if
an incident was escalated should be
readily available from the operational data
source associated with Incident
Management.
% of Problems Recorded and Tracked
It is difficult to detect that a problem is not
recorded or tracked unless it actually has
been recorded and tracked!
61
Metric Requirements - SMART An IT Performance Metric should be collected at regular time intervals, that is,
have an associated time dimension.
 This may be hourly, weekly, monthly, quarterly, and so on.
 Metrics without a time dimension should be referred to as milestones.
Good Example
Milestones
[% Incidents Caused by Changes] is a
metric that should be analyzed over time
to ensure a downward trend. Data should
be available every day.
% of Data Elements Contained within
Enterprise Data Model. The creation of
an Enterprise Data Model is a largely
investigative activity aimed at
understanding what data elements exist.
Consequently, the denominator for this
metric, the number of data elements,
cannot be determined until an Enterprise
Data Model actually exists. As a result, this
should be an IT Performance Milestone –
„Enterprise Data Model Complete.‟
.
62
Assignment
Based on identified processes, we are going to look at and select a number of IT KPIs
and create a scorecard for this:
Business Goals
Step 1 define business goals
Step 2 define IT goals
IT Goals
Step 3 process
goals
IT process goals
IT
management
IT process
health
dashboards
drives
IT
departments
Functional and
process
scorecards
IT KPIs
Step 5
Designing a
Scorecard
Step
4
KPIs
63
Assignment
Step 1: identify the most relevant KPIs for
Change, Service desk / incident management
Step 2: write down which KPIs
are most relevant for your IT KPI
based scorecard and why
Process name:
Metric
Importance (1-10)
Rationale (why)
Note: keep the
„tips and tricks‟ in
mind!
64
Use the following template for step 4
Process name:
Metric
Importance
(1-10)
Rationale (why)
65
KPI Examples: Cost Control
% IT Costs - Travel
Reduce travel cost by providing viable alternatives, such as Web conferences or VOIP and e-learning
education instead of classroom training.
Energy Cost per IT Facility
Energy cost per IT facility gives insight into the energy cost per facility. Monitoring this information
periodically helps organizations in optimizing the energy consumption.
% Projects Related to
Projects should not be created if they are not directly related to specific IT or business objectives. The
Business
combination of [% Projects Related to IT] and [% Projects Related to Business] should be 100%.
% Inactive Application
Inactive accounts represent overhead on security systems, both from a performance and an
Accounts
administration perspective. Importantly, they make the task of auditing security more difficult due to
unnecessary overhead. If effective security management processes are in place, the number of
inactive accounts should be minimal.
% IT Resources - Contractors IT departments will almost always have a requirement for contractors. The level of contractors needs
to be balanced with the potential impact on overall IT resource moral as well as costs associated with
projects.
The % of IT costs that can be
This is a difficult metric to measure; however, the actual process of attempting to measure this metric will yield
directly connected to business
results through mapping IT costs to business value drivers.
value drivers, for example,
sales/services increase due to
increased connectivity
The average cost per user license It is common for an IT organization to purchase far more licenses than required. For example, it is estimated
for a particular application and
that 80% of Business Intelligence software licenses are not used. Reducing the average cost per user license is
types of applications
a combination of ensuring that the correct number of users are determined and that costs are negotiated
efficiently with IT vendors.
A measure indicating how
This metric can be used to measure the effectiveness of the allocation of costs from IT to the business. While
satisfied the business is with the subjective in nature as the business will rarely be satisfied with IT costs, it can be used to gauge improvements
allocation of costs from IT
in the distribution of costs.
The % of IT staff that has
It is important within any organization that resources receive the necessary training to ensure that they are
participated in IT-related training aware of current knowledge and to assist with career development. This metric provides details on what % of
over the last 12 months.
IT personnel have received training over the previous 12 months. The level of training is also significantly
proportional to effectiveness and the costs associated with IT support.
KPI Examples: Ensure continuous service
% Availability SLAs Met
The % of services delivered for which the requirements in the Service Level Agreement for the availability of
services is met
% CIs Monitored
The % of configuration items that are actively monitored by an infrastructure monitoring tool. Only configuration
items that are candidates for monitoring i.e. infrastructure components such as servers, networks and databases
are included in this metric.
The number of scheduled backups that failed in a selected time period
% Database Backup Failures
% Database Restoration
Failures
The % of database restoration attempts that failed during a selected time period.
% Databases Backup/Restore The % of databases that are scheduled for backup for which backup testing and verification of restoration has
been performed
Verification
% Databases Scheduled for
Backup
The % of databases that required backup that have successfully been schedule for backup.
The % of total server backups that failed during a selected time period. Failure is defined by the non completion
of a backup. This includes both scheduled backup's that did not start and backups that failed during the backup
process.
% Server Restoration Failures The % of server backup restoration attempts that failed during a selected time period
% Server Backup Failures
% Servers Backup/Restore
Verification
The % of total servers that have had a full backup and restoration tested and validated.
% Servers Scheduled for
Backup
The % of services in an IT infrastructure that are scheduled for regular backups. The nature of backups will vary
across servers based on the software and applications hosted on a server. This metric looks at all servers and
provides an enterprise IT perspective.
Average Training Days for IT
Resources
The average number of training days for IT resources over the previous 12 months
IT Continuity Plan Reviews
Number of reviews per period of the IT continuity plan
KPI Examples: Project Management
Projects Completed on Time
The number of projects completed that were completed within the target end date.
Projects Completed Within Budget The number of projects completed where the total actual costs were less than the total budgeted costs.
Projects Completed on Time and
Within Budget
Projects Open with Milestones
Missed
The number of projects that were completed by the target completion date and had total actual costs less than
budgeted costs.
The number of projects currently open where one or more milestones related to the project have not been met.
Projects Scheduled
The number of projects at a given point in time that are scheduled to be started in the future. Scheduled projects
are those that have been created, subsequently approved, but not yet commenced.
The number of projects that are currently open, that is, implementation or development has started but has not
been completed.
Projects Open
Projects Completed
The number of projects marked as completed during a specified timeframe. The definition of completed is that
the intended functionality to be provided by the project has been accepted by the users of the functionality.
Projects Championed by Business The number of IT projects created were initiated from the business and that have funding and full support from
the business.
Projects Completed with
A measure of the number of projects that have appropriate documentation associated with implementation tasks,
Documentation and Testing Plans such as testing and post-implementation tasks, such as support and training.
Projects Created
The number of new projects created.
% Projects Related to Business
Projects should not be created if they are not directly related to specific IT or business objectives. The
combination of [% Projects Related to IT] and [% Projects Related to Business] should be 100%.
Projects - Predefined Benefit
The number of projects created that have specified defined tangible benefits to the business or IT if successfully
implemented.
KPI Examples: Manage Quality
% IT Processes Meeting QA
Objectives
The % of IT processes that meet the QQ objectives that were defined
% IT Processes Reviewed by QA The % of IT processes that have been reviewed by conducting a formal QA
% IT Resources - Quality Training The % of IT resources who have received IT Quality related training within the previous 12 months.
% Projects Meeting QA
Objectives
The % of IT projects that meet the QQ objectives that were defined
% Projects Receiving QA Review The % of IT projects that have been reviewed by conducting a formal QA
% Stakeholders Understanding IT The % of stakeholders that understand the IT policy
Policy
KPI Examples: Enable Operation and use
% CIs with Operations
Documentation
The percentage of Configuration Items that have associated operations documentation
% Incidents Resulting from
Inadequate Documentation
The percentage of incidents that can be appointed to inadequacies in the documentation made available with
new or changed applications
% SRs Resulting from Inadequate The % of SRs created due to the requestor not having sufficient training in functionality related to an IT service
e.g. configuration of new email accounts in Outlook or accessing functionality within an ERP system or on the
Training
corporate portal.
The % of service requests created as a result of lack of documentation related to an IT service. For example, a
% SRs Resulting from Lack of
user calls the service desk because they cannot find information on the corporate portal on how to connect to the
Documentation
instant messaging system.
The % of stakeholders that attended training for a new application after release
% Training Attendance New
Applications
Average Change Documentation
Update Time
Average time it takes for documentation to be updated after a change
Customer Satisfaction Training/Documentation
Incidents Resulting from
Inadequate Documentation
The average customer satisfaction score for IT Training/Documentation surveys sent in a selected period. This
is representative of the overall satisfaction and confident that the business has in IT's ability to delivery training
and IT related documentation required by the business to use IT services. A good range of values for
measurement is 0 to 10
The volume of incidents that can be appointed to inadequacies in the documentation made available with new or
changed applications
SRs Resulting from Inadequate
Training
The volume of Service Requests that can be appointed to inadequacies in the training that was presented to
stakeholders after release of the new application or after a change made to the applicatoin
KPI Examples: Change Management
Changes Created
Changes Implemented
Average Change implementation time
The number of requests for change (RFC's) created. In IT Service Support, a change is the addition,
modification, or removal of approved, supported or baseline hardware or software components. This can
include network, application, environment, and system components, or other IT components, including
documentation. All changes should relate to a configuration item.
The % of critical priority changes implemented within an agreed target time.
The average length of time required to implement requests for change.
% Changes Implemented within Target - Critical The % of critical priority changes implemented within an agreed target time.
% Changes Implemented within Target - High
The % of high priority changes implemented within an agreed target time.
% Changes Implemented within Target - Medium The % of medium priority changes implemented within an agreed target time.
Average Cost per Change
% changes failed
Average cost per change
The % of changes that failed during the implementation phase of Change Management.
% changes rejected
The % of changes implemented that classified as emergency changes. Emergency changes are those
which require circumvention of routine change management processes due to the urgency of business
requirements and changes to the IT infrastructure. A comprehensive Change Management process will
include a process for handling Emergency Changes
The % of changes created where the risk of incidents occurring within the IT infrastructure is high. A
comprehensive Change Management process will include a process for handling High Risk Changes
The % of change requests that are analyzed by IT and subsequently rejected.
% changes due to CMBD issues
The % of changes that were made as a result of incorrect information provided by the CMDB
% changes post implementation feedback
The % changes that received feedback after completion of the implementatoin
% changes post implementation review
The % of changes that were reviewed post implementatoin
% changes process compliance
The % of changes that correctly followed the defined change management process
% changes audited
The % of changes that were audited post implementation
% changes causing incidents
The % of changes implemented that resulted in one or more incidents occurring.
% changes causing problems
The % of changes implemented that resulted in one or more problems incurring
% emergency changes
% high risk changes
% changes implemented without a back out plan The % of changes that were implemented without a defined back out plan
KPI Examples: Change Management
% changes implemented without CI
testing
% changes implemented without testing
The % of changes that were implemented without proper CI testing
% changes specified inaccurately
The % of changes that were not specified correctly
% changes with incorrect data
The % of changes that were closed but contained 1 or more incorrect data components e.g. wrong
categorization fields entered, incorrect history, timestamps entered incorrectly, missing solution or closure
description, etc.
% Changes without sign off
The % of changes that were incorporated without formal sign off
The % of changes implemented that did not go through the formal testing phase incorporated within the
Change Management process.
% changes requiring scheduled outages The% of changes that require a service outage to be scheduled in order for implementation of the change to
occur.
KPI Examples: Service Desk
% IT Level 1 Support with
Backup
% SRs Resolved on Initial
Contact
IT Resources - Level 1 Support
% Calls Abandoned
Calls Answered
Call Answer Time
Customer Satisfaction - Service
Operations
An important IT metric that is often used as a benchmark between IT organizations. The lack of backup
personnel can directly effect the delivery and support of IT services, which in turn can effect the overall
operations of an organization. This is a particularly important metric for the Service Desk because of the
customer visibility involved.
An important service desk metric as the ideal scenario is that service requests related to phone calls are
resolved during the first call, minimizing the impact on the customer and the amount of process required by IT
support. This metric is also a key driver of customer satisfaction because it is an 'interaction' metric - one
whereby the service desk is exposed to the customer.
The number of IT resources classified as Level 1 Support. Level 1 Support typically refers to agents at the
Service Desk who are the first point of contact for service requests, incidents, and change requests from the
business. They will try to resolve 'how-to' questions, routine simple changes, such as password resets, and
ensure that more complex issues are assigned (dispatched) to the correct Level 2 and 3 resources.
Provides insight into how efficient the Service Desk is at answering IT support telephone calls from customers. A
high abandonment rate may indicate inadequate staffing or inefficient processes in terms of the collection of
information from the customer. Also, an important metric in terms of customer satisfaction because while small
minorities of these abandoned calls are accidents or mistakes, the majority represent either dissatisfied
customers (those who hung up) or customers who are about to become dissatisfied (those whose calls were
terminated).
Measures the number of calls that were answered by the Service Desk. This is related to <Calls - Offered> and
<Calls - Abandoned> in that the difference between <Calls - Offered> and <Calls - Abandoned> are mostly the
calls that were answered (the exception are the calls that were terminated during the initial pre-recordings). So,
as the % of calls abandoned decreases, the % of calls answered should increase. Each call answered becomes
a candidate for a service request, incident, or change request to be created.
A measure of how long it took to be answered from the time the customer was placed in a queue to the actual
answer time. The time placed in a queue is normally immediately after the automated menu messages.
The results of survey responses for Service Delivery surveys sent.
Avg Daily Incidents Handled per Relates to the calls that Service Desks can handle and is an important metric relevant to the planning of Service
Service Desk Agent
Desk resources.
Average Cost per Call
Average cost per call to the Service Desk.
% IT Staff Turnover - Level 1
Support
High Service Desk staff turnover will have a negative effect on key service request resolution metrics, such as [%
SRs Resolved on Initial] and [% SRs Resolved within Target].
KPI Examples: Incident Management
% Incidents Misrouted
'Misrouted incidents add to the overall resolution time and, as such, impact the customer and add cost to IT operations. A high number and
% of misrouted service requests could indicate fundamental flaws and problems in the incident management process.
% Incidents Reopened
Primarily an operational metric that provides information on the effectiveness of the Service Desk and IT Operations resolution process.
The ideal process is that an incident is resolved and then automatically closed within a set timeframe. If an issue recurs, a new incident is
created. Incidents should not be reopened after initial resolution if the issue logged has been appropriately actioned and resolved.
% Changes Causing Incidents
If the change management process is functioning effectively and the CMDB is accurate, this metric should be very low because all changes
should occur during scheduled outage times. Note: When analyzing this metric, care needs to be taken as if the incident root-cause
analysis process is failing then this metric will be artificially low.
Costs associated with level 2 are typically higher than for level 1 due to the additional expertise required. As IT operations becomes more
efficient, this metric should fall, resulting in faster resolution times and reduced costs.
% Incidents Dispatched to Level 2
% Incidents Resolved by Known Errors
% Incidents Dispatched to Level 3
Known error information assists in the incident resolution by providing an IT operations resource with answers to questions previously
asked. This assists in achieving target resolution rates and reducing overall costs. This is stated as an 'above target' metric, meaning that
while the overall volume of incidents and incidents resolved by known errors should reduce, but the % of incidents solved by known errors
should increase.
Costs associated with level 3 are typically higher than for level 1 and 2 because of the additional expertise required. As IT operations
become more efficient, this metric should fall, resulting in faster resolution times and reduced costs.
% Incidents Resolved by Workaround
While workaround solutions play an important part in maintaining the overall stability of IT services delivery, this metric should decrease as
an IT environment matures.
% Incidents Dispatched
Provides an indication of IT operations efficiency in processing incidents. Ideally, over time, the volume dispatched decreases as a % of the
total incidents created.
% Incidents Void
Void incidents should be monitored to ensure that upward deviations from longer term averages do not occur. Such deviations may
represent data quality or data entry problems. They could also highlight issues with automated incident creation mechanisms.
Average Incident Dispatch to Own
Duration
This metric indicates how responsive IT support staff are at accepting ownership of incidents. Increasing trends can indicate issues with the
process of service request ownership. Hourly or daily spikes can indicate staff rostering issues. Over time, the [Incidents - Dispatch to Own
Duration] should decrease, but increase relative to the [Incidents - Create to Dispatch] duration. This will result in the [Incidents - Create to
Own] duration decreasing.
% Incidents Owned within Target Prompt ownership of critical priority incidents indicates efficient service support processes and increases the probability that incidents will
Critical
be resolved within the target time. Ownership of critical priority incidents within target has increased importance due to the customer
visibility into processes within IT, that is, because of the potential business of critical priority service requests, the customer will have high
expectations regarding ownership and subsequent resolution.
Incidents created
The volume of incidents that are created
st
% incidents resolved by 1 level
The % of incidents that were resolved by the first level support team
% incidents resolved within target critical The % of critical incidents resolved within the defined target time. Also referred to as 'Resolution Met' or „TTR Met' (Time to Resolve Met).
A typical target time for resolution of critical priority incidents is 2 hours.
KPI Examples: Incident Management
% incidents resolved within target – high
The % of high priority incidents resolved within the defined target time. Also referred to as 'Resolution Met' or
„TTR Met' (Time to Resolve Met). A typical target time for resolution of critical priority incidents is 2 hours.
% incidents resolved within target – low
% incidents caused by changes
The % of low priority incidents that were owned within the defined target time. Also referred to as 'Ownership
Met‟, „TTO Met' (Time to Own Met), or 'Accept Met'. A typical target time for ownership of low incidents is 4
hours
The % of incidents caused by the implementation of a change. Refer [Changes - Created]
% incidents caused by CMDB issues
The % of incidents that were caused by data errors within the configuration management database.
% incidents linked to testing errors
The % of incidents that can be linked to errors made during testing
% incidents auto generated
The % of incidents that were automatically generated by IT monitoring tools.
% incidents resulting from inadequate
documentation
The % of incidents that are the outcome of inadequate documentation
% incidents resolved by known errors
The % of incidents that were resolved by access to information available regarding known errors. This
information is typically stored in a 'Known Error' or Knowledge Management database. The source of the
information should be Problem Management processes where known errors are identified.
KPI Examples: Problem Management
% Problems with Incorrect Data
Shows the % of problems that were identified as having one or more invalid data components. Poor data quality
associated with operational IT systems adds significant cost to a business.
Problems Open - Missed Target Used to monitor the volume of critical problems that are open and have missed target resolution. Increasing
Resolution - Critical
volumes highlight risk to the business and potential resource or IT infrastructure issues.
% Incidents Linked to Problems
Average Problem Create to
Close Duration
Problems Created
% Proactive Problems
Problems Open
% Problems Completed within
Target - Critical
% Problems Root Cause
Identified
% Problems RFC Created
Average Problems per Release
In theory, all incidents are caused by problems. But, in the practical context of implementing IT service
frameworks, not all incidents result in a problem record and resolution through the formal problem management
process. This metric provides an indication of the % of incidents that are linked to the resolution of a problem.
The goal of effective incident and problem management processes should be to increase the % over time while
decreasing the overall volume.
Can be used over time to track the efficiency of IT operations in resolving and closing problems. Increasing
trends can indicate resource issues and instability of the IT infrastructure. However, care needs to be taken when
analyzing this measure because some problems can be open for extended periods of time due to difficulties in
capturing the data required for problem diagnosis
Used to monitor the overall number of problems generated within an IT infrastructure. A high level of problems
indicates inherit instability and potential problems with all IT service processes as problems are minimized via
effective incident, configuration, and change management.
Used to monitor how many problems are initiated by IT operations in a pro-active manner, that is, they start a
new project because they believe that something could go wrong or they want to ensure that a given future
problem does not happen.
The presence of proactive problems indicates a mature, efficient problem
management process.
Used to help identify resource issues within IT operations and assist with analyzing the overall stability of the IT
infrastructure.
Provides an indication of the effectiveness of Problem Management processes. Can also be used as an indicator
of how many serious problem exists - those that require more time to resolve because of extensive analysis and
diagnosis requirements.
One of the fundamental goals of problem management is to identify the root cause of the related incidents and/or
service requests. Efforts can be made to rectify the root causes, and information can then subsequently be fed
into knowledge bases and known errors.
Provides information on how many problems require requests for change in order to be closed. The
implementation of one of more changes associated with a problem can be a sign that tangible outcomes are
occurring because of problem management processes or that the IT infrastructure requires significant change to
correct issues. Given this, this metric needs to be analyzed with respect to the overall maturity of IT services
delivery.
Can be used to track the overall effectiveness with which IT implements releases. Problems related to releases
can indicate serious flaws in the Release Management process.
Session 6
ANALYSIS OF A NUMBER OF
SCORECARD AND
DASHBOARD EXAMPLES
77
A number of example dashboards/scorecards for
review
View a number of examples from Metricus
78
Metricus – IT Balanced Scorecard
IT KPIs – 25 practical, pragmatic, usable KPIs
Trending
Trending information to analyze KPIs in detail
Settings
Manage definition, set target, tolerance, data criteria
Collaborate
Provide comments with graphs,
manage issues and documents
79
Metricus – Service Desk
IT KPIs – 25 practical, pragmatic, usable KPIs for
managing your service desk operations across the
world
Dashboards
Call handling information, information on Service Requests and
Incidents
Drill down into the details
Benchmark different service desk
sites across the world and drill
down into the details of available
data
80
Metricus – Service Request Management
IT KPIs – Every Module includes at least 25 KPIs with
information on the practical use, adoption and
benefits of KPIs.
Add your own
KPIs
Simply add your
own KPIs or
modify KPIs to
the Module and
publish them to a
dashboard
Analyze to the greatest extent possible
Dynamic charts allow you to analyze data and create charts that are most
relevant to you
81
Metricus – Change Management
Six Sigma charts
For selected KPIs you have the ability
to create Six Sigma control charts or
histograms. This functionality helps
you reduce defects in your processes
Comprehensive Dashboards
A typical Metricus Dashboard Template that provides a
comprehensive picture on your current IT performance and
historical information.
These two elements together allow you to take adequate
decisions for the future
82
Metricus – IT Balanced Scorecard
For Visio experts
Metricus allows you to create and
upload Visio charts and link KPIs to
Visio components. For every Module
specific templates are available but
you can also create your own
Manage KPI categories
For all areas in the IT balanced
scorecard KPIs are available. We
categorized them according to
customer feedback, but you can
create your own sub categories
83
IT Performance Management and best practices:
ITIL®
IT Performance Management is
strongly advocated in ITIL.
Measuring, Managing, and Improving ITIL
processes has its roots in the Deming Circle
that is advocated in ITIL. The Plan, Do,
Check, and Act (PDCA) Cycle requires to
define a baseline and continuously measure
progress against that baseline.
ITIL addresses the use and benefits of KPIs
and IT Performance Management in every
process, as well as having an entire lifecycle
phase fully dedicated to IT Performance
Management, Continual Service
Improvement (CSI).
ITIL Service Lifecycle © Crown Copyright 2007 Reproduced under license from OGC
84
IT Performance Management and best practices:
ITIL®
The 7-Step Improvement Process is the most important element of Continual
Service Improvement.
This process follows the same logical steps as defined in the IT Performance Management lifecycle and
can help you define what is important to you, measure progress against defined goals, and effectively
utilize your resources to optimize your IT organization.
The 7-Step Improvement Process © Crown Copyright 2007 Reproduced under license from OGC
85
IT Performance Management and best practices:
ITIL®
Strengths
 ITIL is strong in terms of processes surrounding service
support. This has translated into broadly accepted sets of
KPIs to measure Incident Management, Change
Management, Problem Management, and so on.
 ITIL emphasizes effective Configuration Management. An
accurate CMS is fundamental to the delivery of IT
Performance Management.
Weaknesses
A process is not implemented
until measured
An improvement is not noted until measured
against a baseline
Justification for an ITIL project requires a
measured baseline and an improvement target
 The ITIL publications are limited in the specification of
KPIs to be used.
 There is not a lot of consistency in the definition of KPIs
and information on KPI utilization between the various
ITIL domains.
 The Continual Service Improvement is intended to be
used throughout all phases of the service lifecycle,
however in reality is only considered late in most
implementation projects.
Without metrics, an ITIL project will soon lose
steam and eventually fail
ITIL Service Lifecycle © Crown Copyright 2007 Reproduced under license from OGC
86
IT Performance Management and best practices:
ISO/IEC 20000
ISO/IEC 20000 provides an external
quality mark for IT Service
Management organizations
Service Delivery Processes
Capacity Management
Service Continuity and
Availability Management
Service Level Management
Service Reporting

ISO/IEC 20000 promotes the adoption
of an integrated process approach to
effectively deliver managed services to
meet business and customer
requirements.

Adopting ISO/IEC 20000 formalizes
the measurement component of IT
processes for organizations because
they have to demonstrate and attest
control over IT processes.

This includes requirements such as “A
process should be in place to identify,
measure, report, and manage
improvement activities,” and “Reports
shall be produced to meet customer
needs, including trend information,
satisfaction analysis, and so on.”
Information Security
Management
Budgeting and Accounting
for IT Services
Control Processes
Configuration Management
Change Management
Release Processes
Relationship Processes
Resolution Processes
Incident Management
Business Relationship
Management
Problem Management
Supplier Management
Release Management
87
IT Performance Management and best practices:
ISO/IEC 20000
Strengths

ISO/IEC 20000 formalizes the need for
well-defined and thought-through
process adoption.

There are many documentation
requirements in ISO/IEC 20000 and, as a
result, the chances of finding the data
required for effective IT Performance
Management are likely to be high.

A specific process called „service
reporting‟ dictates reporting
requirements.
Weaknesses

There are no KPIs in ISO/IEC 20000.

Even though ISO/IEC 20000 formalizes
many of the operational IT processes, it
does not provide guidance on good KPI
selection and adoption.
88
Thank you
Contact details:
insert photo
Arjan Woertman
Arjan.woertman@itpreneurs.com
+31 (0)10 71 10 260
89