Metrics:
Metrics: The
The Good,
Good, The
The Bad,
Bad,
and
and The
The Ugly
Ugly
Can (John) Saygin, Ph.D.
Assistant Vice President for Sponsored Project Administration
Office of the Vice President for Research
Professor, College of Engineering
Center for Advanced Manufacturing and Lean Systems
Phone: 210.458.5194
can.saygin@utsa.edu
Outline
•
•
•
•
•
Lean: What does it mean?
IOM 2013 Report
From Data to Metrics
Success Stories of Lean
We want to hear from you
LEAN
LEAN: SEE WASTE and ELIMINATE IT
Waste (Non-Value Added):
Anything that adds
cost, time, effort without
adding value
DEFINE VALUE!!!
Inventory
Transport
UnderUtilization of
Human Talent
OverProcessing
Wastes
Motion
Waiting
Defects
OverProduction
HOW DO YOU MEASURE WASTE?
• Metric: a standard of measurement
• Performance Metric: Standards of
measurement by which efficiency,
performance, progress, or quality of a plan,
process, or product can be assessed.
• Deviation 
|Actual| - |Target|
LEAN METRICS
Lean Metrics – the appropriate measurements
and goals for the Lean Improvement activities
Commonly used Time-based Lean Metrics
•
•
•
•
Individual Cycle Time
Total Cycle Time
Queue Time
Total
And many more…
• non-time based metrics, such as: cost,
customer satisfaction, on-time delivery, and
quality.
TRAP… “averages”
PROCESS… FLOW
• "If you can't describe what you are
doing as a process, you don't know
what you are doing."
- W. Edwards Deming
DEFINE-MEASURE-ANALYZE-IMPROVE-CONTROL
DMAIC must be a “continuous process”
• Define the system, the voice of the customer and their
requirements, and the project goals, specifically.
• Measure key aspects of the current process and collect relevant
data, including controllable and uncontrollable factors.
• Analyze the data to investigate and verify cause-and-effect
relationships. Determine what the relationships are, and attempt to
ensure that all factors have been considered. Seek out root cause of
the defect under investigation.
• Improve the current process based upon data analysis using
techniques: Future process
• Control the future state process to ensure that any deviations from
target are corrected before they result in defects. Implement
control systems and continuously monitor the process.
METRICS
Success with Lean Metrics
• To successfully use lean metrics:
– Standardize measurements
• Make sure the results are accurate and consistent
– Metrics should be easy to collect
• Gather data where it is most useful
– Make the Lean Metric Visual
• Make information accessible
• Goal: Predictable Output
– Stable Output is more
important than spikes of
performance
10
outstanding
Continuous Improvement: A Mindset
Process Design
& Improvement
Do the Right Thing
WHAT
Programmatic
Performance Metrics
Process Flow
Validation
Process Flow
Automation
Do it Better
Do it Right
HOW
Operational
Performance Metrics
EFFECTIVENESS
EFFICIENCY
(Performance directly linked
to Desired Outcomes)
(Rate of Desired Outcomes)
11
Eliminate Whenever Possible
Laws , Regulations, Rules
at various levels
Policies
Functions
Processes
5 Why’s…
Do not target the symptoms
Get to the Root-Cause
12
INSTITUTE OF
MEDICINE
2013 Report
IOM Report (2013)
Mission, Vision, Goals
Complexity, Inconsistency, Cost
“Learning Health
Care System”
Continuous
Improvement
CTSA 2.0
Additional
Complexity:
Network
(12 sites in 2006 to
61 sites in 2013)
Lack of… Metrics
IOM 2013 Report – Page 8
Learning Healthcare System
Lean Enterprise
Are they different?
Where to Start?
Lean in IOM 2013 – Page 67
Evaluation -- Page 81
Evaluation Individual CTSAs (pp.82-83)
15 Metrics (Page 84)
June 2013 Meeting
Programmatic Performance Metrics
Operational Performance Metrics
FROM DATA TO
METRICS
“In God we trust…
All others, bring data.”
W. Edwards Deming
A) It is raining.
B) The temperature dropped 15 degrees and then it started raining.
C) If the humidity is very high and the temperature drops substantially
the atmospheres is often unlikely to be able to hold the moisture so it
rains.
Data represents a fact or statement of event without relation to other
things.
Information embodies the understanding of a relationship of some sort,
possibly cause and effect.
Knowledge represents a pattern that connects and generally provides a
high level of predictability as to what is described or what will happen next.
Reference: "Data, Information, Knowledge, and Wisdom"
by Gene Bellinger, Durval Castro, Anthony Mills
The Big Picture
Reference: "Data, Information, Knowledge, and Wisdom"
by Gene Bellinger, Durval Castro, Anthony Mills
Where to focus
when determining metrics…
Actionable Information… Not just data
Tools: Data Collection, Analysis,…
Job by itself…
Myopic Nature: Single Metric!!!
Interpretation leads to action…
Stakeholders
Simple
Source:
Source -- http://www.juiceanalytics.com/writing/choosing-right-metric/
SUCCESS
STORIES OF LEAN
An Example from UTSA Human
Resources…
http://camls.utsa.edu/
HR at UTSA
Background
• Human Resource (HR) managing
“On-Boarding Process” of new hires
– Orientation, tax and insurance paperwork,
computer and email accounts, requests for keys, telephone
line, parking, etc.
• Problems:
– New hires often take 2 weeks to be truly “on-board”
– HR team spends hours on missing data & error corrections
everyday
– Lots of “waiting” among offices
HR at UTSA
The Improvement Project
Objectives:
Methodology:
• Shorten new hire time
• Improve work readiness
• Increase compliance
• Lean training for entire office
• Value stream mapping
and implementation planning
HR at UTSA
The Findings
Root Causes
Solutions
• Disconnected operations
• Isolated resources
• Ineffective (error prone)
paperwork process
• Redesigned workflow (new
“value stream map”)
• Partners with OIT, ID Card
Office, Parking, etc.
• Standardized web forms
HR at UTSA
The Results
• New process provides integrated resources at the
Orientation for New Employees
• New hires are ready on Day 2 versus 2 Weeks
• 100% accuracy of I-9’s
• $231,319 net savings
in 1st year
IT IS TIME TO
HEAR FROM
YOU…
Think of processes or functions that
you perform…
Write on a piece of paper:
• The Good: You are comfortable with it. You do
it well.
• The Bad: You are not comfortable with it.
However, you work so hard and you get it
done.
• The Ugly: You are not comfortable with it.
Your performance varies when doing it.
Do it yourself
Do it as a team…
Discuss as a team and compile your notes on a
large sticky paper
Good
Bad
Ugly
Any questions,
comments
Can (John) Saygin, Ph.D.
Assistant Vice President for Sponsored Project Administration
Office of the Vice President for Research
Professor, College of Engineering
Center for Advanced Manufacturing and Lean Systems
Phone: 210.458.5194
can.saygin@utsa.edu
As an Engineer…