Cumulative Sum (CUSUM) charts for
medical student peripheral venous
cannulation;
development of a difficulty-adjusted CUSUM
Dr Harry Murgatroyd
SpR Anaesthesia
Leeds Teaching Hospitals Trust
Sumaiyah Kola
Medical Student
Leeds University Medical School
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Qui ckTime™ and a
TIFF ( Uncompressed) decompressor
are needed to see this pi cture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Runcie CJ. Assessing the performance of a consultant anaesthetist
by control chart methodology. Anaesthesia. 2009; 64(3): 293-296
Developed initially to look at
industrial processes
CUSUM Chart
Learning Curves
Monitors performance
Audit of quality
clinical practice
Used to determine
competency
Graphical presentation over time
Theory
Set:
definition of success / failure
acceptable failure rate
unacceptable failure rate
error
Collect:
binary data
Algorithm:
Score falls with success
Score increases with failure
Graph
Boundary Lines
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Sequential
cannulation attempts
CUSUM
2
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
-2
Success is seen as
a fall in the graph
-4
-6
Failure seen as a rise
in the graph
-8
-10
Each point represents a
single cannulation attempt
-12
Score derived using
the CUSUM formula
Problems
•Patient variability
•Standard CUSUM
•Constant failure and success rates
•Risk adjusted CUSUM
•Complicated
•Not intuitive
•Failure rates
•Set by user
•Can affect results considerably
Comparing acceptable failure rate
6
4
2
0
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
-2
-4
-6
-8
Acceptable failure rate 0.1
-10
Acceptable failure rate 0.2
Acceptable failure rate 0.4
-12
Medical student project
• Aims
– Proficiency at
intravenous
cannulation
– Plot individual
CUSUM charts
– Develop a
‘difficulty-adjusted’
CUSUM technique
• Time Scale:
– 5 weeks
How to insert an intravenous cannula 101
Methodology
• Setting
– Teaching hospital
– Elective surgical lists
• Procedures
– Verbal consent
– Peripheral venous cannulation
• Standard technique
• Size of cannula appropriate to surgical
procedure
– Data collection
•
•
•
•
Success or failure
Appearance of vein
Size of cannula
Patient awake or anaesthetised
Conventional CUSUM
• Definition of ‘success and failure’
• Acceptable and unacceptable failure rates
– Consultant consensus
– Literature
– 0.2 and 0.4 respectively
• Calculation
– Published formulae
– Error rates = 0.1
– MS Excel
de Oliveira. Anesth Analg 2002;95:411-6.
Williams et al. BMJ 1992;304:1359-61.
Conventional CUSUM
Failure rates
Upper and lower
boundries
0= failure,1= success
Data is plotted sequentially
Running total,
CUSUM
Example if “IF”
formula in Excel
Difficulty Adjusted CUSUM
• Difficulty score
– Appearance of vein
– Cannula size
– Awake or anaesthetised
• Different failure rates
– Two stages
• Vein adjusted
• All three variables
– Intervention line
• Average of all prior lines
Difficulty Adjusted CUSUM
Vein adjusted
Vein
appearance
Acceptable
failure rate
Unacceptable
failure rate
Visible,
palpable
0.15
0.3
Just visible
0.3
0.6
Three variable methodology
Table shows the scoring of each of the
recorded variables. These are then added up
to give the total score for the cannulation
attempt
Shows the standard CUSUM
formula, whilst incorporating
different failure rates and scores
dependent on the difficulty of the
variables recorded.
Spreadsheet showing the final
added up scores of the different
variables. Hence including the vein,
consciousness and cannula size.
Using “IF” formulas the correct
value of S is selected from the table
above and the CUSUM then plotted
in the same way as before.
Difficulty Adjusted CUSUM
• Successful difficult cannulation
– Large fall in score
• Failed difficult cannulation
– Small rise in score
• Successful easy cannulation
– Small fall in score
• Failed easy cannulation
– Large rise in score
Conventional CUSUM vs Vein DA-CUSUM
An example CUSUM and difficulty-adjusted CUSUM chart for student A
3
2
C onventional C U SU M
D A -C U SU M
D A -C U SU M intervention line
C onventional C U SU M intervention line
CUSUM score
1
0
0
-1
-2
-3
-4
-5
10
20
30
40
P atient number
50
60
Conventional CUSUM vs Three Variable DA-CUSUM
3
Standard
Including Difficulty
2
1
0
1
-1
-2
-3
-4
-5
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31
33
35
37
39
41
43
45
47
49
51
53
55
Three variable DA-CUSUM for two students
2.5
2
1.5
CUSUM
1
0.5
Sumi
Dave
0
1
3
5
7
9
11
13 15
17
19 21
23
25 27
29
31 33
-0.5
-1
-1.5
-2
Series
Monday Morning
35 37
39
41 43
45
47 49
51
53 55
POSITIVES of CUSUM
LIMITATIONS of CUSUM
•Objective
•Only technical skills
•Simple Calculations
•Must have binary outcome
•Shows improvement in
learners
•Relies on logbooks and
honesty of user
•Early detection of poor
performance
•Time consuming
•Allows comparisons
between students
Bolson S, Colon M. Int J Health Care Qual Assur
2000;12:433-438.
Kestin IG. BJA 1995;75:805-809.
•Open to manipulation
•Does not show
improvements that do not
change binary outcome
Difficulty adjusted CUSUM
• POSITIVES
• LIMITATIONS
• Potentially corrects for
patient variability
• Easier and more intuitive
than other methods of
adjustment
• Failure rates set by the
user
• The more variables
‘corrected’ for, the more
layers of estimation and
inaccuracy
• Loss of statistical element
of conventional CUSUM
Summary
• Easy technique
– Handheld devices
– Electronic logbooks
• Objective
• Can be adjusted for patient
variability
• Allows
– Charting of ‘learning curve’
– Comparison between practitioners
– Identification of poor performance