Metrics and the Cost of
Quality
Chapter 18
Why Metrics!
 A strategy without metrics is just a wish. And
metrics that are not aligned with strategic
objectives are a waste of time.
– Emery Powell
 Be careful what you measure -- you might just
get it.
 If you don’t keep score, you’re only practicing.
– Tom Malone
 You get what you inspect, not what you expect.
The Real Bottom Line: Mission
and Measures
 One of the most powerful management disciplines, the
one that more than any other keeps people focused and
pulling in the same direction, is to make an organization’s
purposes tangible. Managers do this by translating the
organization’s mission – what it, particularly, exists to do
– into a set of goals and performance measures that make
success concrete for everyone. This is the real bottom
line for every organization – whether it’s a business or a
school or a hospital. Its executives must answer the
question, “Given our mission, how is our performance
going to be defined?”
– Magretta & Stone, Management. 2002, p. 129
Why Metrics Now!
 The “never satisfied” customer.
 Managing the “total” supply chain.
 Shrinking product life cycles.
 More (not necessarily better) data.
 Profit margin squeezes.
 Presence of an increasing number of
alternatives.
So What
 Surviving in this new environment means
working with:
–
–
–
–
Less lead time
Less inventory
Less cost
More reliance on the supply chain
 Surviving means having “better” metrics.
What are Metrics?
 A verifiable measure stated in either
quantitative (e.g., 95% inventory accuracy) or
qualitative (e.g., as evaluated by our
customer,we are providing above average
service) terms. A metric is intended to close
the gap between value, strategy, and specific
activities.
 Metrics
–
Measure, direct, teach.
Metrics – The Firm’s Instrument
Cluster
Metrics Categories
 Metrics
– Individual measures.
 Metrics Set
– Number of metrics used to evaluate the
performance of an individual or unit.
 Metrics System
– System for integrating, coordinating and
alignment metrics, objectives, and plans.
TYPES OF METRICS
 Individual vs. Systems
 Operational vs. Financial
– Utilization vs. Internal Rate of Return (IRR)
• Traditional focus - financial
– Critical issue
• Operational improvements do not always result in
financial results
 Output vs. Process
 Internal to the firm vs. supply chain metrics
Types of Metrics
 Product/Output metrics
– Measure results, not the performance of the
processes that gave rise to this performance.
 Process metrics
– Pertains to the performance of the process
– Typical measures -- cost, time, quality, agility
(the ability to adapt), capacity, variability
Output vs. Process Metrics
 You have to drive to Detroit. You would
like to leave by no later than noon. You
need to arrive at no later than 3:00 pm.
 Output metric - 3 PM arrival time met
 Process metrics
– departure time - 1:00 PM
– average speed - 65 mph
Importance of Process Metrics
Experience is a wonderful thing. It enables you
to recognize a mistake when you make it again and again - and again. Process metrics enable
you to recognize a potential mistake before you
make it.
Key Point
 Output metrics - critical to top
management
 Process metrics - critical to those working
in execution
Metrics Process




I - Intent (what you want to achieve and why?)
D - Deployment (how do you want to achieve this goal?)
A - Assignment (who will be held accountable?)
S - Standard (how will you measure achievement?)
• Issues of form, source, feasibility, appropriateness.
 E - Evaluation and Reward
I.D.A.S.E. Process in Detail
 Intent
– Desired outcome
– What is it that we are trying to measure?
– Why is it that we are measuring this?
– How does this process/outcome support our objectives
at the functional level? At the corporate level?
– What would happen if we were to fail on this
dimension?
– Who are the “champions” for this outcome?
I.D.A.S.E. Process in Detail
 Deployment
– How are we going to achieve this outcome?
– What type of data do we need to effectively and
–
–
–
–
accurately implement this metric?
Do we have this data? If not, what proxy data can we
use? Impact of the proxy data?
Will this be implemented as a predictive or outcome
metric?
Impact of the metric on the metric set?
Is this metric similar to any others currently in use?
I.D.A.S.E. Process in Detail
 Assignment
– Recognize that there are two different people
involved in a metric
• Those who monitor the outcomes
• Those who manage the processes that create the
outcomes
– There are different behaviors
• Scanners -- large number of metrics-- looking for
gaps
• Searchers - deal with fewer metrics -- interested in
problem solving
I.D.A.S.E. Process in Detail
 Standard
– The basis of comparison
– Defines what is acceptable (and unacceptable)
behavior
– Different types of metrics
– Can be generated from the benchmarking
process
– Can and should be dynamic
Standards
 Absolute
– EPA
– Few in number
 Relative
– Relative to our performance
• Past performance
• Best possible future performance
• Pros and cons
Standards - 2
 Relative
– Relative to best within group
•
•
•
•
Need to develop appropriate group
Best within the group teach others
Worse within the group learn from others
Group membership must be viewed as dynamic
– Relative to a “benchmark”
• Best in firm
• Best in industry
• Best in class
FAC
Critical Traits of
Standards
F - Feasible
A - Appropriate
C - Comparable
An Example of Metrics in
Action
 OEE
– Operational Equipment Effectiveness
– Availability * Performance * Quality
– Note what we are measuring
• Quality
• Uptime
• Cost
– Note what we are not measuring
• On-time delivery
Metrics Set
 The set of measures that we use to monitor or
evaluate the performance of a specific person or
function or activity.
 Metrics set brings together two elements
– Set of metrics
• Load
– Capacity to interpret or act on the measures
• Capacity
 Metrics set recognizes that the process is
capacitated.
Metrics Set
 Ideal number of measures
– 5 +/- 3
 If we overload the manager, then we get:
– Frustration
– Confusion
– Satisficing
– Negotiation
– Cherry picking
Metrics System
 Metrics System
– Total system responsible for coordinating and
aligning metrics
• Alignment
– Top to bottom
• Coordination
– Across functions
– Why are Alignment and Coordination so
important?
• The lessons of the “telephone” game.
Alignment and Coordination
 Alignment occurs first
– Coordination follows
 Alignment involves a translation problem
– Restating higher order objectives into supporting
activities for the lower levels
– As we go from top to bottom, objectives narrow and
level of details increase
 Challenges
– Translation gets “messed” up
– Multiple bosses
Attaining Alignment
 Approaches
– Bill of Material Approach
– Balanced Scorecard Approach
– Theory of Constraints Approach
– Strategic Profit Impact Model
Approaches for Managing the
Metrics System
 Bill of Material approach
– Explosion of metrics from the top down
– Converting each higher level outcome into a series of
lower level outcomes that must be achieved to ensure
that the higher level outcome is met.
– Problems
 Balanced Scorecard
– Approach
– Problems
Balanced Scorecard Approach
Approaches for Managing the
Metrics System
 Theory of Constraints Approach
– Must look at the entire system and identify the
constraint
– Now measure the performance of the
constraint
– The constraint drives the performance of the:
• Firm
• Supply chain
– Constraint must always be working
• Measure this
Approaches for Managing the
Metrics System
 Theory of Constraints - continued
– Elements
•
•
•
•
Profit (P)
Operating Expenses (E)
Throughput Time (T)
Inventory (I)
Strategic Profit Model
Recent Findings
 Metrics are a source of fear
 Different groups perceive metrics
differently
– Resolution strategies:
• Negotiation (group optimization)
• Satisfaction of minimum targets (order
qualifiers)
• Decompose the problem into sub-problems
(silo management)
• Treat everything as important (cope as you can)
• Find new metrics we can agree upon
• Ignore most and focus on what I can do well
• Seek divine guidance
Recent Findings
 Metrics must be managed as a
system
 Metrics for motivation and those
for process control are different
– Global vs. local
 Metrics explosion
 Ceteribus Paribus
Metrics Framework
Metric Type
Output
Focused
Aggregate
Process
Line of
Control
Point Control
Dimensionality
Multidimensional
Motivation
Embedded
Line of
Monitoring
Traits of Effective Metrics
– Focused on the critical few, not trivial
–
–
–
–
–
–
many
Linked to value
Measure directly
Appropriate display
Appropriate frequency of updating
Acceptable time lags between
generation of metric and report
Appropriate standard used
The “Cost of Quality” Metrics
 Types of Quality Costs
– Prevention
– Appraisal
– Internal Failure
– External Failure
Classifying Quality Costs
1-10-100 Rule
 Theory Says:
– Prevention – 1
– Inspect and Prevent – 10
– Given to Customer – 100
1-10-100 Rule
 Practice Shows:
– IBM Rochester: A400 Server
• 1-13-92 (hours work required)
– Corning Glass: Glass pressings
• 1-80-400 ($ cost)
– Texaco: Refinery
• 1-10-100 ($ cost)
 Cost of Customer Defections
– Companies can boost profits 25 –
85% by decreasing defections 5%
Cost at Points in the Process
Use of a Cost Basis
 Why?
– Allow comparison across different periods
 Typical Bases
– Internal failure as a percent of total production costs
– External failure as an average percent of net sales
– Procurement appraisal costs as a percent of total
purchased material costs
– Operations appraisal costs as a percent of total
production costs
– Total quality costs as a percent of production costs
Quality Cost Basis Example
January
February
Cost Category
Product A
Product
B
Product A Product
B
Prevention
$2,000
$4,000
$2,000
$4,000
Appraisal
$10,000
$20,000
$13,000
$21,000
Internal
Failure
$19,000
$106,000
$16,000
$107,000
External
Failure
$54,000
$146,000
$52,000
$156,000
Total
$85,000
$276,000
$83,000
$288,000
Standard
Direct Labor
Costs
35,000
90,000
28,000
86,000
Quality Cost Basis Solution
Use of the System
 Categorizing costs
 Estimating true external failure costs
 Application of cost bases
 Tracking improvement
 ROQ