Statistical Process Control – An Overview
Public Health Intelligence Training Course – March 2011
1
better information --> better decisions --> better health
Introduction
•
•
•
•
•
2
Public health practice commonly makes comparisons between areas,
groups or institutions.
Methods based on ranking, e.g. league tables, percentiles, have a number
of flaws.
Ranking makes the assumption that differences between organisations are
the results of better or poorer performance. It takes no account of inherent
system differences.
Just because institutions produce different values for an indicator, and we
naturally tend to rank these values, doesn’t mean we are observing
variation in performance.
All systems within which institutions operate, no matter how stable, will
produce variable outcomes due to natural variation.
better information --> better decisions --> better health
Introduction
•
The questions we need to answer are:
–
–
–
–
•
•
3
Is the observed variation more or less than we would normally expect?
Are there genuine outliers?
Are there exceptionally good performers?
What reasons might there be for excess variation?
Alternative methods based on understanding variation may be more
appropriate.
Statistical process control is one such method and helps to answer these
questions through the use of control charts.
better information --> better decisions --> better health
Introduction to Control Charts
Why use control charts?
Control charts are used to monitor, control, and improve
system or process performance over time by studying variation and its
source.
What do control charts do?
• Focus attention on detecting and monitoring process variation over time
• Distinguishes special from common causes of variation, as a guide to local
or management action.
• Serves as a tool for ongoing control of a process
• Helps improve a process to perform consistently and predictably
4
better information --> better decisions --> better health
Types of Variation
1.
2.
3.
5
Common-cause or process variation is variation that is completely random;
special-cause or extra-process variation is non-random i.e. is the result of an
event or action.
Special cause variation can be exhibited within or outwith control limits i.e trends,
step functions, drift etc.
In any system variation is to be expected. Using statistical techniques we define
the limits of variation (control limits and zones). Interpretation of the data relative
to these limits or zones identifies points that are worthy of investigation.
better information --> better decisions --> better health
Definitions
• A process is said to be ‘in control’ if it
exhibits only “common cause” variation.
– This process is completely stable and predictable.
• A process is said to be ‘out of control’ if it
exhibits “special cause” variation.
– This process is unstable.
6
better information --> better decisions --> better health
Basic control chart layout
Upper control
limit
Warning zones
Under Run Hours as a % of Allocated Hours
14.00%
Centre line
(usually mean
or median)
12.00%
Zone A
10.00%
Zone B
8.00%
Zone C
6.00%
Zone C
4.00%
Zone B
Zone A
2.00%
0.00%
Apr08
May- Jun08
08
Jul08
Aug- Sep08
08
Oct08
Nov- Dec- Jan08
08
09
Feb- Mar- Apr09
09
09
May- Jun09
09
Jul09
Aug- Sep09
09
Oct- Nov- Dec- Jan- Feb- Mar09
09
09
10
10
10
Date
Lower control limit
7
better information --> better decisions --> better health
Types of control charts
•
•
8
Control charts are plots of the data with lines indicating the target value
(mean, median) and control limits superimposed.
The common types are based on statistical distributions:
– Poisson distribution for counts, rates and ratios; e.g number of violent
crimes, number of serious accidents
– Binomial distribution for proportions; e.g where the response is a
category such as success, failure, response, non-response
– Normal distribution for continuous data e.g measures such as height,
weight, blood pressure
better information --> better decisions --> better health
Types of control charts
1.
2.
9
Conventional control charts (run charts)
– The indicator of interest is plotted on the y-axis, against time or the
unit of analysis on the x-axis.
– Control charts can be plotted with small numbers of data points
although their power is increased with more data.
Funnel plots
– A type of chart where the indicator of interest is plotted against the
denominator or sample size.
– This gives it the characteristic funnel shape
better information --> better decisions --> better health
Using control charts and SPC methods
•
•
10
Control charts can help us to present and interpret our information more
intelligently.
They can be used
– To detect unusual or outlying patterns, e.g. poor performance,
outbreaks or unusual patterns of disease
– In health profiling and assessing levels of performance
– To decide whether or not targets are being met
– In assessing health inequalities
better information --> better decisions --> better health
Examples – Run Charts & Control Charts
Run Charts:
•
Display of data points plotted in chronological order
•
Ideally 25 data points are required
•
Centre line (mean or median) is included to identify types of variation
Control Charts:
•
A Run chart plus control limits and warning limits (optional)
•
Control limits are set at 3 standard deviations above and below the mean
Warning limits are set at 2 standard deviations above and below the mean
•
These limits provide an additional tool for detecting special cause variation
11
better information --> better decisions --> better health
Run chart – Time to work
09:21
09:07
08:52
12
better information --> better decisions --> better health
Fri
Thurs
Wed
Tues
Mon
Fri
Thurs
Wed
Tues
Mon
Fri
Thurs
Wed
Tues
Mon
Fri
Thurs
Wed
Tues
Mon
Fri
Thurs
Wed
08:24
Tues
08:38
Mon
Time arrived at work
09:36
13
better information --> better decisions --> better health
Fri
Thurs
Wed
Tues
Mon
Fri
Thurs
Wed
Tues
Mon
Fri
Thurs
Wed
Tues
Mon
Fri
Thurs
Wed
Tues
Mon
Fri
Thurs
Wed
Tues
10:48
10:33
10:19
10:04
09:50
09:36
09:21
09:07
08:52
08:38
08:24
Mon
Time arrived at work
Run Chart – Out of control
Special Cause Rule Number 1: Shifts
For detecting shifts in the middle value, look for eight or more consecutive points
either above of below the center line. Values on the center line are ignored, they
do not break a run, and are not counted as points in the run.
Micrograms/ML
SERUM GENTAMICIN LEVELS - TROUGH
2.2
1.7
1.2
0.7
0.2
1
2
3
4
5
6
7
8
9
10 11 12
13 14 15 16 17 18 19 20 21
Blood Samples
14
better information --> better decisions --> better health
22 23 24 25
Special Cause Rule Number 2: Trends
For Detecting trends, look for six lines between seven consecutive points all going
up or all going down. If the value of two or more consecutive points is the same,
ignore the lines connecting those values when counting. Like values do not make or
break a trend.
Number of Adverse Drug
Reactions
ADVERSE DRUG REACTIONS
5
4
3
2
1
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Week Number
15
better information --> better decisions --> better health
Special Cause Rule Number 3: Zig-Zag Patterns
Any non-random pattern may be an indication of a special cause variation. A
general rule is to investigate where 14 consecutive points go up and down
alternately.
DIASTOLIC BLOOD PRESSURE
MEASUREMENT
120
115
110
105
100
95
90
85
80
75
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
INDIVIDUAL PATIENT READINGS
16
better information --> better decisions --> better health
Special Cause Rule Number 4: Cyclical Patterns
A non-random cyclical pattern may be an indication of a special cause variation.
For example, a seasonal pattern occurring across months or quarters of the year.
7
6
Observations
5
4
3
2
1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
0
Time
17
better information --> better decisions --> better health
Special Cause Rule Number 5: Points Outside Limits
A point or points outside control limits is/ are evidence of special cause. Control
limits are calculated based on data from the process.
ABNORMAL PAP TEST FOLLOW-UP PROCESS
TIME IN DAYS
70
Mean = 35
60
50
UCL
40
30
20
10
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
COLPOSCOPY PATIENTS
18
better information --> better decisions --> better health
21
22
23
24
25
Determining if the process is out of
control – Control Rules
• One or more points fall outside of the control limits
• 8 or more consecutive points on same side of centre line
• 7 successive points all going up or down
• 14 consecutive points going up and down alternately
• 2 out of 3 consecutive points in zone A or beyond
• 4 out of 5 consecutive points in zone B or beyond
• 15 consecutive points in zone C (above and below)
19
better information --> better decisions --> better health
Answers to Handout
20
better information --> better decisions --> better health
8+ points on same side of centre line
21
better information --> better decisions --> better health
16 points going up and down
22
better information --> better decisions --> better health
Common cause
23
better information --> better decisions --> better health
Common cause
24
better information --> better decisions --> better health
7 points decreasing
25
better information --> better decisions --> better health
Ap
rM 08
ay
-0
Ju 8
n0
Ju 8
lAu 08
gSe 08
p0
O 8
ct
-0
N 8
ov
D 08
ec
-0
Ja 8
n0
Fe 9
b0
M 9
ar
-0
Ap 9
rM 09
ay
-0
Ju 9
n0
Ju 9
l-0
Au 9
gSe 09
p0
O 9
ct
-0
N 9
ov
D 09
ec
-0
Ja 9
n1
Fe 0
b1
M 0
ar
-1
0
Under Run Hours as a % of Allocated Hours
4 out of 5 points in zone B or beyond
12.00%
26
10.00%
8.00%
6.00%
4.00%
2.00%
0.00%
Date
better information --> better decisions --> better health
Acting on Variation
Special or common cause variation?
Special
Common
Is the process capable?
Search for and
eliminate
differences in causes
between data points
27
Yes
Do
nothing
No
Search for and eliminate
causes common to all
data points
better information --> better decisions --> better health
Management of Variation
Inappropriate Action
Appropriate Action
Special Cause Variation
28
•Identify and study the special
cause.
•React to special cause
- If it is a negative impact,
prevent it or minimise impact.
-If it is a positive impact, build
into process.
•Change the system to react to
special causes
Common Cause Variation
•Recognise that the capability will not
change unless the process is changed.
•Work to reduce variation due to
common causes
•Do not react to individual occurrences
or differences between high and low
numbers.
•Treat every occurrence as a special
cause
better information --> better decisions --> better health
Summary
• Understanding the causes of variation has reformed
industry
• Application to healthcare has provided important insight
to inform improvement
• Effectively highlights areas meriting further investigation
through simple data presentation
29
better information --> better decisions --> better health
Chart Instability
Instability is defined as:
No. of control rule violations
Total no. of points entered
• Charts can be ranked according to their instability
• Good way of prioritising the charts to investigate
• Can be used as an ‘Early Warning System’ to identify
problem charts before they become a real issue
30
better information --> better decisions --> better health
Funnel plots
• Conventional control charts are used for
count data, proportions and continuous
variables
• Funnel plots are used for discrete/count data
(e.g. deaths and hospital admissions)
– Can be used for proportions, directly standardised
rates, indirectly standardised rates and ratios, and
rate ratios.
31
better information --> better decisions --> better health
Example 1: rate of mortality at 120 days
following admission to a surgical specialty
• In this example each data point is a hospital (all hospitals in NHS
Board X are shaded blue).
• The number of people admitted to a surgical specialty is represented
on the horizontal axis, which essentially means that smaller hospitals
appear towards the left hand side of the graph and larger hospitals
towards the right.
• The proportion of people who died within 120 days of admission to
hospital is represented on the vertical axis – the higher up the data
point, the higher the rate of mortality would appear to be.
• The funnel formed by the control limits (and from which the graph gets
its name) is wider towards the left hand side. This is simply so the level
of activity (in this case, the number of admissions) is taken into
account when identifying ‘outliers’ (i.e. the larger the denominator, the
most stable the data points are).
32
better information --> better decisions --> better health
Elective admissions to any surgical
specialty: overall mortality at 120 days
3.00
Mortality rate(%) at 120 days
2.50
2.00
1.50
1.00
.50
.00
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
Number of Patients
33
better information --> better decisions --> better health
16,000
18,000
Transurethral Prostactectomy for
benign disease: overall mortality at
120 days
12.00
Mortality (%) at 120 days
10.00
8.00
6.00
4.00
2.00
.00
0
50
100
150
200
250
Number of Patients
34
better information --> better decisions --> better health
300
350
Issues with control charts
• In the “any surgical specialty” example, there are many
areas which lie outside the control limits
• Such a large number of points outside the control limits
is known as overdispersion
• It arises when there are large numbers of events, and
case-mix or other risk factors (e.g. deprivation) are not
accounted for
• In this example, the overdispersion is probably due to
the variation in procedures covered and different uptake
of these procedures across the Scottish hospitals.
35
better information --> better decisions --> better health
How to handle overdispersion?
• In performance management, we try to identify differences that can
be attributed to differences in organisational performance.
• In this case it’s usual to adjust the control limits or the data to
eliminate potential sources of variation, such as case-mix and
demography.
• This has the effect of creating a ‘level playing field’.
• In public health practice, we are likely to be interested in such
sources of variation for their own sake (lung cancer example).
• Rather than eliminate them, we want to draw attention to them and
understand the reasons behind them.
• We tend not to alter control limits, and display the variation as it
actually is.
36
better information --> better decisions --> better health
Example 2:
lung cancer mortality rates
by local area
37
better information --> better decisions --> better health
Further information
http://www.apho.org.uk/
resource/item.aspx?RID
=39445
38
http://www.indicators.scot.
nhs.uk/SPC/Main.html
better information --> better decisions --> better health