Managing with Data
Washington State Association of Counties
(WSAC)
June 6th , 2013
Brian Willett, Performance Analyst
My Premise
Learning HOW to manage with data is just as
important as learning WHAT to measure.
Performance measures are a management TOOL, just
like a knife or a screwdriver.
The TOOL can be used for both GOOD and EVIL.
INTENT and SKILL separate a home cook from a master
chef from a serial killer.
2
The Yellow Bead Company Employees
Supervisor (1)
• A “natural leader”
• Believes in holding
people
“accountable”
• Able to “motivate”
employees
• Able to enforce
strict adherence
to procedure
Inspectors (1-2)
• Can distinguish
the color yellow
• Can count up to
“20”
Recorder (1)
• Legible
Penmanship
• Can calculate the
average of 5 whole
numbers
Timer (1)
• Can operate a
digital stopwatch
with basic
training
Willing Workers
• Willing to “do their
best”
• Able to follow
strict production
procedures
3
3
The Yellow Bead Company Procedures
Objective: “Produce” as many yellow beads as possible.
Step 1 – Hold the bead box upside down, press the white button on the side, and
empty the “trap”.
Step 2 – Flip the bead box over and shake it exactly 5 times.
Step 3 – Place the bead box on the table and press the white button on the side
once.
Step 4 – Flip the bead box upside down and show the production results to the
inspectors .
Step 5 – Inspectors independently count the number of yellow beads and tell the
agreed upon number to the Recorder. The Recorder writes that number on
the scorecard.
Step 6 – Pass the bead box to the next Willing Worker and repeat the process four
more times (Total of 5 times).
Step 7 – After the 5th time through, pause work until told to resume and have the
Recorder calculate the average production of the team for the quarter.
Step 8 – The Recorder writes down how much time elapsed from start to finish.
4
4
The Yellow Bead Company debrief
Anybody can achieve almost any goal by…
…redefining the terms.
…distorting or faking.
…running up costs.
Dr. W. Edwards Deming
“Numerical goals lead to distortion and faking, especially
when the system is not capable of meeting the goal.
Anybody will meet the quota (goal) allotted to them,
especially if they are not responsible for the losses
generated.”
“Perhaps I can find new ways to motivate them?”
5
5
It’s the process and system, not the people
Freedom to
improve/change:
• Tools
• Methods
• Rules
• Equipment
• Facilities
• Staffing/scheduling
Personnel
6%
Process
94%
• Performance
evaluations
• Customer service
training
• Pay for
performance
• Goals & targets
• Motivational
speakers &
posters
6
What is a system?
Outcomes:
Organizational Benefits
User Benefits
Financial Stakeholder Benefits
Suppliers
Inputs
Process
Outputs
Users &
Brokers
(Customers)
7
Foofaraw and Twaddle
“All of the empowered, motivated,
teamed-up, self-directed, incentivized,
accountable, reengineered, and
reinvented people you can muster
cannot compensate for a dysfunctional
system. When the system is functioning
well, these other things are all just
foofaraw. When the system is not
functioning well, these things are still
only empty, meaningless twaddle.”
Peter Scholtes
8
What Does Change Look Like on a Run Chart?
Process-Level Shift: Eight or more
successive points on the same side of the
median (baseline)
Trend: Seven or more successive points
in the same direction (increasing,
decreasing, or staying exactly the same)
Warning: Seeing a trend can depend on your
sampling frequency
Non-Random Variation Patterns: Look
for repeating cycles and clusters
Don’t look too hard for these – If they are present they
tend to jump off the page
9
9
Analysis questions and management options
Is there evidence of abnormal variation?
(trend/process level shift)
…If yes, put your finger on the abnormal variation and
ask, “What happened here to cause it?”
…If no, is the data capable of meeting or exceeding the
target 100% of the time?
…if no, you have two options:
1. Change the target so you can meet or exceed it 100%
of the time, or…
2. Fundamentally change the underlying
processes/systems.
10
Why not just worry about meeting targets?
In this example, actual
performance has never
come in below the target,
but the trend is stable and
undesirable.
If this were on a typical
Red/Yellow/Green
scorecard, it would be
Green every time
If you just base your
evaluation on whether
they hit the target or not
in the past, you are
missing what is likely to
happen in the future.
Better
y
Target
Time
11
No time? No money?
12
Additional readings and references
Understanding Data, T.S. Marshall, 1990.
The New Economics, Dr. W. Edwards Deming, 1994.
The Leader’s Handbook, Peter Scholtes, 1998.
We Don’t Make Widgets, Ken Miller, 2006.
Extreme Government Makeover, Ken Miller, 2011.
A Performance Management Framework for State and Local
Government, National Performance Management Advisory
Commission, 2010. http://www.pmcommission.org
13
More Information? Help?
Local Government Performance Center
Website: http://www.sao.wa.gov/PerformanceCenter
Twitter: www.twitter.com/LocalGovPerform
Brian Willett
Performance Analyst
(360) 725-9731
Brian.Willett@sao.wa.gov
Larisa Benson
Director of Performance Audit
(360) 725-9720
Larisa.Benson@sao.wa.gov
14