Measurement for
improvement
Mike Davidge
22
Measurement for
improvement
“Measurement is for
improvement not judgement.”
D. Berwick
3
Model for Improvement
4
7 Steps to measurement
1 Decide aim
2 Choose measures
3 Define measures
6 Review
measures
7 Repeat
steps
4-6
4 Collect
data
5 Analyse
& present
5
Measures checklist – a handy reminder
 Section 1
–
–
–
–
Rationale
Definitions
Data required
Goals
 Section 2
– Collect
– Analyse
– Review
6
Clarifying aim is crucial
1 Decide aim
2 Choose measures
3 Define measures
6 Review
measures
7 Repeat
steps
4-6
4 Collect
data
Take the LIFT test.
Would you be able to describe your aim in a
couple of sentences?
5 Analyse
& present
Take 5 minutes to agree your aim
7
Choosing the right measures
1 Decide aim
2 Choose measures
3 Define measures
6 Review
measures
7 Repeat
steps
4-6
4 Collect
data
5 Analyse
& present
8
Types of measure
 Outcome measures
– Reflect the impact on the patient
– For example: unplanned return to ITU or crash calls.
 Process measures
– Reflect the way you work
– For example: % compliance with Sepsis 6 bundle.
 Balancing measures
– reflect unintended consequences
– For example: if you have implemented changes to reduce your post
operative length of stay, you also want to know what is happening
to your post operative readmission rate. If this has increased then
you might want to question whether, on balance, you are right to
continue with the changes or not.
9
2
Driver diagram
AIM
To improve
recognition and
timely
management of
patients
identified with
sepsis in ED
and CDU by
achieving 90%
compliance with
evidence based
therapy
(SEPSIS 6) by
March 2013
SECONDARY DRIVERS
PRIMARY
DRIVERS
11
2
3
3
Early identification
of patients with
possible sepsis in
ED, CDU and
Wards
Ensure sepsis
best management
practices in ED,
CDU and wards
Seamless
transitions
• Timely triage
• Timely notification to, and assessment
by nurse and doctor
• Early and repeated lactate
measurements
• Monitoring and communication of
progress
• Early aggressive administration of IV
fluids
• IV antibiotics administered within 1
hour
• Blood cultures taken before IV
antibiotics are given
• Education of staff in sepsis as time
critical illness
• Effective communication between
ED , CDU and SCAS
• Effective communication and
transition with in-patient wards
• Patient shadowing and information
for patients
10
Choose your measures
 Use the driver diagram
 You have 5 minutes to
customise to your system
 Now take another 5
minutes to decide which
drivers and change ideas
to measure
 Homework: Continue to
discuss!
AIM
To improve
recognition and
timely
management of
patients
identified with
sepsis in ED
and CDU by
achieving 90%
compliance with
evidence based
therapy
(SEPSIS 6) by
March 2013
SECONDARY DRIVERS
PRIMARY
DRIVERS
11
2
3
3
Early identification
of patients with
possible sepsis in
ED, CDU and
Wards
Ensure sepsis
best management
practices in ED,
CDU and wards
Seamless
transitions
• Timely triage
• Timely notification to, and assessment
by nurse and doctor
• Early and repeated lactate
measurements
• Monitoring and communication of
progress
• Early aggressive administration of IV
fluids
• IV antibiotics administered within 1
hour
• Blood cultures taken before IV
antibiotics are given
• Education of staff in sepsis as time
critical illness
• Effective communication between
ED , CDU and SCAS
• Effective communication and
transition with in-patient wards
• Patient shadowing and information
for patients
11
Definitions – an Achilles heel
1 Decide aim
2 Choose measures
3 Define measures
6 Review
measures
7 Repeat
steps
4-6
4 Collect
data
5 Analyse
& present
12
Collect
1 Decide aim
Decisions, decisions
 What - All patients or a sample?
2 Choose measures
 Who – is collecting?
3 Define measures
 Where – is the data located?
 How – hospital system or audit?
 When – Real time or retrospective?
6 Review
measures
7 Repeat
steps
4-6
4 Collect
data
What is your baseline?
5 Analyse
& present
13
14
Where to measure?
Start ?
Decision
Point ?
Handover ?
End ?
14
It’s not an add-on
Organise everything around value-added
(front line) work processes
W Edwards Deming
All value-adding work is inherently local;
All improvement is inherently local; therefore,
As you implement a data collection system,
You mustn't destroy clinical productivity
Instead, you must
Integrate data collection into workflow at the front line
15
Admission and Recognition
Bundles
 Detail actions that should take place on a regular,
routine basis such as:
–
–
–
–
Observations
Calculating and recording NEWS score
Querying sepsis if the score is high
Communicating NEWS and risk to whole team
 The aim is to embed these actions and behaviours into
normal everyday practice
 So a high compliance with these bundles is
demonstration that practice is based upon the best
available evidence
16
Planning & testing your data collection
 What, Who, When, Where
and How?
 You have 5 minutes to
discuss your data
collection plan
 And decide your first
small test of change
(PDSA)
 Homework: Run the
PDSA
17
Analyse
1 Decide aim
“The type of
presentation you use
has a crucial effect on
how you react to data”
2 Choose measures
3 Define measures
6 Review
measures
7 Repeat
steps
4-6
4 Collect
data
5 Analyse
& present
18
How we traditionally assess performance:
2 point comparisons
Last
quarter
62
This
quarter Change
66
+6%
Why has the number of crash calls gone up? Our
service is getting worse. We need to do something!
What decision are you going to make?
19
What’s a person’s normal body
temperature?
20
In the real world,
everything varies....
How many patients
did we admit today?
How long
does it
take you
to get to
work?
Is my
temperature
always the
same?
How long does it
take to take a
patients BP?
21
“Data contains both signal
and noise. To be able to
extract information, one
must separate the signal
from the noise within the
data.”
Walter Shewhart
22
There are two types of variation
 While every process displays variation:
 some processes display controlled
variation (common cause)
– Stable pattern of variation = noise
– constant causes/ “chance”
 while others display uncontrolled variation
– pattern changes over time = signal
– special cause variation/“assignable” cause eg infection or
hypothermia
We should display data in a way that shows which is present
23
Revisiting Crash calls
Last
quarter
62
2006
2006
Jan
Jan
3131
2012
2007
2007
Jan
Jan
3232
This
quarter Change
66
+6%
2011
Feb
Feb
3636
Mar
Mar
2727
Apr
Apr
1818
May
May
2222
Jun
Jun
4040
Jul
Jul
2323
Aug
Aug
3131
Sep
Sep
4242
Oct
Oct
1919
Nov
Nov
3131
Dec
Dec
1616
Feb
Feb
3232
Mar
Mar
5454
Apr
Apr
2222
May
May
5151
Jun
Jun
1717
Jul
Jul
2424
Aug
Aug
1111
Sep
Sep
2727
Oct
Oct
2424
Nov
Nov
2626
Dec
Dec
1616
24
As a run chart..
Complaints during 2006 to 2007
60
50
40
30
20
2006
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
0
Jan
10
2007
Number
25
• Plot data in time order
• Calculate and display median as a line
Run charts
• Analyse chart by studying how values
fall around median
Complaints during 2006 to 2007
60
50
40
30
20
2006
2007
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
0
Jan
10
26
Percentage of sepsis patients that receive sepsis six with one hour
NHH
from Oct 2011 to Oct 2012 - All Wards
120
100
% compliance
80
60
40
20
10/09/20
12
10/08/20
12
10/07/20
12
10/06/20
12
Weeks
10/05/20
12
10/04/20
12
10/03/20
12
10/02/20
12
10/01/20
12
10/12/20
11
10/11/20
11
10/10/20
11
0
Data from an in hours Outreach service
During this time sepsis was a major contributor to death in
15% of cases
27
Annotate your charts!
28
Review measures
It is a waste of time collecting and
analysing your data if you don't
take action on the results
1 Decide aim
2 Choose measures
Question 1
3 Define measures
Where will the measures be
reviewed?
6 Review
measures
7 Repeat
steps
4-6
5 Analyse
& present
4 Collect
data
Question 2
When (how frequently) will
we review them?
29
Where do you put stuff you
want everyone to know?
Why do we hide track and
trigger scores at the foot
of the bed and then audit
them infrequently?
30
Insert name of presentation on
31
Putting Important Information In a
Prominent Place
 Communicates to the whole team, all
the time
 Quickly exposes where staff have
difficulty with performing
observations/calculating score
 Promotes education and training for
possible eventualities
32
7 Steps to measurement
 You may not get it
right first time! You may need
several attempts to get it right
for you
1 Decide aim
2 Choose measures
3 Define measures
6 Review
measures
7 Repeat
steps
4-6
4 Collect
data
5 Analyse
& present
33
Next steps – the project plan
 Measures
1 Decide aim
– Do you have an agreed set of measures? If
not, how and when will you get them agreed?
2 Choose measures
 Definitions
– Who will complete measures checklists for all
remaining measures and by when?
3 Define measures
 Review meetings
– Have you agreed when you will review your
measures? Set a date for the first meeting
6 Review
measures
7 Repeat
steps
4-6
 Test your process
4 Collect
data
– When are you going to follow the 7 step
process for your first measures?
5 Analyse
& present
34
Thank you and here’s
to effective
measurement!