How to get the Answers
to your most
important Questions
An Introduction to Story Metrics
An Introduction to Story Metrics
1
Martin Klubeck
Strategy & Planning Consultant
Office of Information Technologies
University of Notre Dame
574-631-5447
mklubeck@nd.edu
An Introduction to Story Metrics
2
An Introduction to Story Metrics
3
Agenda
• Metrics Overview
• Demonstrate the Process
• Hands-on
An Introduction to Story Metrics
4
What Is A Metric?
A Common Language
• Data
• Measure
• Information
An Introduction to Story Metrics
5
Branches
Leaves
Measures
Limbs
Information
Data
Trunk
Metric
Roots
The Question
An Introduction to Story Metrics
6
A Complete Story
An Introduction to Story Metrics
7
Anthologies May Be Better
An Introduction to Story Metrics
8
Metrics…
• Are question driven
• Tell a complete story
• Include:
•
•
•
•
Data
Measures
Information
Other Metrics
An Introduction to Story Metrics
9
Help Desk Example
Data
Measures
Number of trouble calls
Number of opened cases
Number of closed cases
Number of employees
Number of survey responses
Number of calls per hour
Number or cases closed by worker
Information
Number of calls for each hour compared to number of workers on shift.
Average length of time to close a case, grouped by type
Average customer satisfaction rating
An Introduction to Story Metrics
10
Help Desk Example
Average number of
open cases from
1999-2002
225
manning
12
200
11
175
10
150
9
125
8
100
Min manning
50
7
6
25
METRIC
Max manning
5
0
0
Explanation:
The manning over the academic year was not in line with the
number of trouble calls received – based on data collected over the last
three years.
Result:
We’ve re-aligned our manning for the coming year to match the level of need
each month.
An Introduction to Story Metrics
open cases
manning
11
Why Use Metrics?
To Help You …
•
•
•
•
Gain support from above
Provide visibility
Improve
Make better “data informed” decisions
An Introduction to Story Metrics
12
How Not To Use Metrics
To “support my case*”
To “motivate” the staff
To “manage” the staff or others
To evaluate individual performance
An Introduction to Story Metrics
13
How To Use Metrics
• Explain how they will and WON’T be used
• Investigate
• Share – Close the loop
An Introduction to Story Metrics
14
Who Will Use The Metric?
• Customer Community
• Management
• Owners and Workers
• Leadership
An Introduction to Story Metrics
15
Top Five Warning Signs that this training
has failed
1. The
boss says –
“I’ll know it when I see it.”
2. “We’ve been collecting this data for five years and
no one is using it.”
3. “Do we have any data on...?”
4. “They don’t trust the data.”
5. “Sounds interesting, let’s collect it.”
An Introduction to Story Metrics
16
What Is The Root Question?
The 5 Whys
An Introduction to Story Metrics
17
Drawing A Picture
• Focus on “how it looks”
• A picture is worth
a thousand words
• Send a clear message
An Introduction to Story Metrics
18
An Introduction to Story Metrics
19
An Introduction to Story Metrics
20
An Introduction to Story Metrics
21
An Introduction to Story Metrics
22
An Introduction to Story Metrics
23
Metrics Tools
• Implementation Guide – adds rigor and structure
• Answer Key – “cheat sheet” for developing metrics
• Process Documentation – How to
An Introduction to Story Metrics
24
The Complete Metric “Pallet”
•
•
•
•
•
•
•
•
•
•
•
•
•
Metric Name
Purpose
Metric Area/Category
Customer
Graphical Representation
Explanation
Metrics Analysis
Measures used to Develop Metric
Collection Schema
Schedule
Assumptions and Constraints
Related Metrics and Data Dependencies
Lessons Learned
An Introduction to Story Metrics
25
The Primary Metric Colors
• Purpose
• Graphical Representation
• Explanation
• Metrics Analysis
• Measures used to develop Metric
An Introduction to Story Metrics
26
Delivery
Effectiveness
Information Needs
Customer view
Value to
Organization
Customer Satisfaction
Customer Service
Cost
Efficiency
Business view
Time
Quality
Employee Satisfaction
Training
Human
Resources
Work Environment
Worker view
Reward & Recognition
Management
of Resources
Project/program Status
Resource Allocation
Visibility
Management view
Strat. Planning & Goal Attainment
Communications
Priority Setting
An Introduction to Story Metrics
27
Hands-On Session
An Introduction to Story Metrics
28
Designing your first metric
1. What is your root question?
2. Visualize the Answer
3. Identify the Measures
4. Find the data
An Introduction to Story Metrics
29
Summary
Embrace the Opportunity
Start with the Question
Draw the Picture
Tell a Complete Story
Repeat - Tweak - Delete
An Introduction to Story Metrics
30
Q&A
An Introduction to Story Metrics
31
Martin Klubeck
Strategy & Planning Consultant,
Office of Information Technologies, University of
Notre Dame
574-631-5447
Klubeck.1@nd.edu
An Introduction to Story Metrics
32
Effectiveness
Ratio of Promoters to detractors
Reponse to Customer Satisfaction Survey
Grad/Professional (718)
0.3
Fifth year (40)
0.1
Fourth year (660)
0.1
Third year (592)
0.1
Second year (787)
0.1
First year (772)
0.1
Students (3574)
0.1
1.1
Faculty & Staff (1280)
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
The higher the ratio the better - a ratio of 1 (promoter = detractor) is the threshold
Signif icant securit y incident s
Core/Critical service availability
(file / storage, enterprise applications, email, and netw ork services)
Target is >= 99.90%
99.92%
2003
99.90%
99.90%
99.90%
99.89%
99.88%
2004
99.86%
99.84%
99.82%
2005
99.81%
99.80%
0
99.78%
99.76%
2
4
6
8
10
Threshold = 2, Target is less t han 2
2005
2004
2003
2002
An Introduction to Story Metrics
33
Efficiency
Client-Staffing Ratio
Time To Resolve
6
45,000
70.0
4.9
5
4.9
38,504
4.5
37,260
35,000
30,782
34,268
30,000
3
3.3
3.1
2.9
61.8
59.5
217
60.0
52.9
4
250
66.6
40,000
200
192
183
50.0
166
25,000
2.8
20,000
2.2
2
15,000
150
40.0
30.0
100
10,000
1
5,000
in days
0
2004
2003
17.8
15.9
15.4
14.1
50
0
2005
20.0
10.0
2002
5.5
6.2
6.3
7.0
2005
2004
2003
2002
0.0
Overall average time to resolve
Target = <= 5 days
Average time to resolve - Help Desk
Target = <= 2.75 days
0
Number of overall cases
Students to OIT employees
OIT Spending
$9,000
Faculty to OIT employees
Staff to OIT employees
Centralized
OIT Expenditures
(in Millions)
25%
23%
$8,000
21%
20%
$7,803
$6,928
$7,000
$700.0
$6,813
$6,473
$800.0
15%
$6,000
$5,000
10%
$4,000
5%
4.80%
$668.7
$626.3
4.70%
$600.0
4.70%
$589.2
4.60%
$500.0
4.50%
$400.0
$3,000
$2,884
$2,351
$2,000
2%
1%
$2,498
0%
$2,476
0%
4.40%
$300.0
$200.0
$100.0
4.30%
$20.8
$7.7
4.30%
$19.6 $10.0
$21.1
2004
2003
$5.8
$0.0
-6%
$1,000
-5%
4.60%
4.20%
4.10%
2005
-7%
$0
-10%
2005
Per student
2004
Per faculty and staff
2003
Per Student Change
2002
Actual ND Expenditures (Millions)
Actual OIT Expenditures (Millions)
Renovare (ERP) expenditures (Millions)
OIT & Renovare (ERP) Expenditures / University Expenditures
Per faculty and staff change
An Introduction to Story Metrics
34