Applying Change Point Detection in
Vaccine Manufacturing
Hesham Fahmy, Ph.D.
Merck & Co., Inc.
West Point, PA 19486
hesham_fahmy@merck.com
Midwest Biopharmaceutical Statistics Workshop (MBSW) - MAY 23 - 25, 2011
Outline

Definitions



Case studies



Detection methods
CUSUM and EWMA estimators
CUSUM and EWMA
SSE
Conclusions
2
Methods of Detection

Visual (Simple but Subjective)




Raw data; run chart
CUSUM chart
EWMA chart
Analytical (Complicated but Objective)


Change-Point estimators; i.e. CUSUM, EWMA
Mathematical Modeling; i.e. MLE, SSE
3
Types of Variation
 Common Causes – natural (random) variations that are
part of a stable process
Machine vibration
Temperature, humidity, electrical current fluctuations
Slight variation in raw materials
 Special Causes – unnatural (non-random) variations
that are not part of a stable process
Batch of defective raw material
Faulty set-up
Human error
Incorrect recipe
4
Cumulative Sum Control Chart

CUSUM: cumulative sum of deviations from average

C t  max 0, y t   0  K   C t1

Ct  accumulate d deviations above/belo w 0






A bit more difficult to set up
More difficult to understand
Very effective when subgroup size n=1
Very good for detecting small shifts
Change-point detection capability
Less sensitive to autocorrelation
5
Exponentially Weighted Moving Average

EWMA: weighted average of all observations
t -1
Et    1 -   yt  j  1    E0
j
t
j 0
A bit more difficult to set up
Very good for detecting small shifts
Change point detection capability
Less sensitive to autocorrelation

“EWMA gives more weight to more recent observations
and less weight to old observations.”
CUSUM
0

1
Shewhart
6
Process Shifts
Step
Linear
Nonlinear
Others
7
Process Model
t
in-control state
out-of-control state

Change point,
"Unknown"
8
Change-Point Estimation Procedures

CUSUM Change-Point Estimation Procedure (Page 1954)
ˆCUSUM  t : Ct  0, 0  Ci  H i  t  1,....T  1, CT  H 
Ct  accumulate d deviations above/belo w 0

EWMA Change-Point Estimation Procedure (Nishina 1992)
ˆEWMA  t : Et  0 , 0  Ei  UCLi i  t  1,...., T  1, ET  UCLT 
9
Process D ata
12
y
Example:
11
10
9
8
1
Actual Change-Point= 20
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
t

C t  max 0, y t   0  K   C t1

U pper CU SU M Chart
5
H 5
3
C+
K  0.5
4
2
1
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
t
ˆCUSUM  t : Ct  0, 0  Ci  H i  t  1,....T  1, CT  H 
Most Recent Reintialization at t =22
10
Process D ata
12
Example:
y
11
10
9
8
1
Actual Change-Point= 20
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
t
Et   yt  1   Et 1
  0.2
E0  0  10
EW MA C hart
11.0
UCL=11
E
LCL=9
10.5
10.0
9.5
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
t
ˆEWMA  t : Et  0 , 0  Ei  UCLi i  t  1,...., T  1, ET  UCLT 
Most Recent time Et   at t =19
11
450
425
400
375
350
325
300
275
250
225
200
175
150
125
100
75
50
25
Run Chart of Prodx
Real Example
R un C har t of Prodx
33.0
32.0
31.0
30.0
29.0
Sample
12
CUSUM vs. EWMA
C USU M of Prodx
140
130
120
100
90
80
70
EW MA of Pro dx, Lamda=0.001
60
50
31.14
31.13
20
31.12
31.09
Avg=31.0870
31.08
31.07
Sample
425
375
325
275
225
175
31.06
125
375
350
325
300
31.10
75
Sample
275
250
225
200
175
150
125
100
75
50
-10
31.11
450
0
425
10
400
EWMA
of Prodx
30
25
40
25
Cumulative Sum of Prodx
110
13
EW MA of Prodx, Lamda= 0.2
32.5
  0.2
31.5
Avg=31.087
31.0
30.5
30.0
450
425
400
375
350
325
300
EW MA of Prodx, Lamda= 0.5
275
250
225
200
175
150
125
100
75
50
29.5
25
Sample
32.5
32.0
31.5
Avg=31.087
31.0
30.5
30.0
450
425
400
375
350
325
300
275
250
225
200
175
150
125
100
75
50
29.5
25
  0.5
EWMA of Prodx
EWMA of Prodx
32.0
Sample
14
Simulated Case Studies

To test different methods' (control
charts/analytical) capability for identifying
change-points



Case # 1: small shifts/drifts
Case # 2: mirror image of case # 1
Case # 3: large shifts/drifts
15
Case #1
Chhar
art
off Y
Y
RRuunn C
t o
44
33
22
Run Chart
Chart ofof YY
Run
11
00
-1
-1
-2
-2
-3
-3
-4
-4
-5
-5
-6
-6
00
25
25
50
50
75
75
100
100
125
125
Sample
Sample
150
150
175
175
200
200
225
225
250
250
16
Case # 1
I ndividual Measur ement of Y
4
1
3
1 & 6: In-control process; N(0,1)
: -ve linear trend; 0.1 sigma
3
: Step shift; N(-3,1)
4
: Step shift; N(-1.5,1)
5
: +ve linear trend; 0.1 sigma
3
4
5
6
2
1
0
-1
Y
2
2
-2
-3
-4
-5
-6
-7
0
25
50
75
100
125
150
175
200
225
250
Obs
17
CUSUM (V-Mask)
C USU M of Y (Al pha=0.0027)
Samples 40 to 60
100
3
Run Chart of Y
2
75
1
50
0
Cumulative Sum of Y
-1
-2
60
58
56
54
52
50
48
46
44
42
40
-3
Sample
25
0
-25
Run Chart of Y
-50
4
3
-75
2
1
Run Chart of Y
-100
0
0
25
50
75
100
125
150
175
200
225
250
Sample
-1
-2
Diagnostic Sequence Plot
-3
-4
-5
-6
0
25
50
75
100
125
Sample
150
175
200
225
250
Detection Criterion: Slope Change
18
CUSUM; 
 0.01
C USU M of Y (Al pha=0.01)
100
Cumulative Sum of Y
75
  0.0027
C USU M of Y (Al pha=0.0027)
100
75
50
25
0
-25
-50
Cumulative Sum of Y
50
-75
25
-100
0
0
-25
25
50
75
100
125
150
175
200
225
250
Sample
-50
-75
-100
0
25
50
75
100
125
150
175
200
225
250
Exact Profile
Sample
19
CUSUM – Target Adjusted
CUSUM of Y - Target= 0
C USU M of Y - Targ et= 0
-100
Cumulative Sum of Y
-150
Cumulative Sum of Y
0
-200
-250
-300
-100
50
75
100
125
150
175
200
Sample
-200
C USU M of Y - Target= 0
50
Cumulative Sum of Y
-300
0
0
25
50
75
100
125
150
175
200
225
250
Sample
-50
-100
-150
0
25
50
75
Sample
100
125
150
20
EWMA;
  0.5
EW MA of Y (Lamda=0. 5)
Samples 40 to 60
2
3
1
1
0
0
-2
Sample
60
58
56
54
52
50
48
46
44
42
-3
EWMA of Y
-1
40
Run Chart of Y
2
-1
Avg=-1.11
-2
-3
-4
-5
0
25
50
75
100
125
150
175
200
225
250
Sample
Detection Criterion: Slope Change
21
EW MA of Y (Lamda=0.001)
-1.02
-1.04
-1.06
EWMA of Y
-1.08
-1.10
Avg=-1.1080
-1.12
-1.14
-1.16
EWMA;
  0.001
-1.18
-1.20
0
25
50
75
100
125
150
175
200
225
250
Sample
Same Profile
C USU M of Y (Al pha=0.01)
100
Cumulative Sum of Y
75
50
25
0
CUSUM
-25
-50
-75
-100
0
25
50
75
100
125
Sample
150
175
200
225
250
22
Case # 2 “Mirror Image”
I ndividual Measur ement of Y
7
1
6
1 & 6: In-control process; N(0,1)
2
3
4
5
6
5
4
3
4
5
: +ve linear trend; 0.1 sigma
: Step shift; N(3,1)
: Step shift; N(1.5,1)
: -ve linear trend; 0.1 sigma
3
2
Y
2
1
0
-1
-2
-3
-4
0
25
50
75
100
125
150
175
200
225
250
Sample
23
CUSUM Chart
C USU M of Y
Case # 2
Cumulative Sum of Y
100
0
C USU M of Y (Al pha=0.01)
0
100
25
50
75
100
125
150
175
200
225
250
Sample
Cumulative Sum of Y
75
Mirror Image
50
25
0
-25
-50
Case # 1
-75
-100
0
25
50
75
100
125
Sample
150
175
200
225
250
24
CUSUM – Target Adjusted
CUSUM of Y - Target= 0
350
300
Cumulative Sum of Y
250
200
150
100
50
0
0
25
50
75
100
125
150
175
200
225
250
Sample
25
EWMA Chart;
  0.01
EW MA of Y-Lamda=0. 01
1.8
1.7
1.6
1.5
EWMA of Y
1.4
1.3
1.2
1.1
1.0
Avg=0.992
0.9
0.8
0.7
0.6
0.5
0
25
50
75
100
125
150
175
200
225
250
Sample
26
EWMA Chart;
  0.2
EW MA of Y - Lamd a=0.2
4
3
EWMA of Y
2
1
Avg=0.99
0
EW MA of Y - Target= 0, Lamda= 0.2
4
-1
3
0
25
50
75
100
125
150
175
200
225
250
EWMA of Y
Sample
2
1
0
µ0=0.00
-1
0
25
50
75
100
125
Sample
150
175
200
225
250
27
EWMA Chart;   0.5
EW MA of Y-Lamda=0.5
5
4
EWMA of Y
3
2
1
Avg=0.99
0
-1
-2
0
25
50
75
100
125
150
175
200
225
250
Sample
28
Case # 3
I ndividual Measurement of Y
1 & 6: In-control process; N(0,1)
17
2
: +ve linear trend; 0.5 sigma
13
3
: Step shift; N(12.5,1)
4
: Step shift; N(9.5,1)
6
5
4
3
2
1
15
11
Y
9
7
5
3
5
: -ve linear trend; 0.38 sigma
1
-1
-3
-5
0
25
50
75
100
125
150
175
200
225
250
Sample
29
CUSUM
C USU M of Y
400
Cumulative Sum of Y
300
200
100
0
-100
-200
-300
0
25
50
75
100
125
150
175
200
225
250
Sample
30
CUSUM – Target Adjusted
CUSUM of Y - Target=0
600
500
Cumulative Sum of Y
400
300
200
100
0
-100
-200
-300
CUSUM of Y - Target=0
1000
1400
1300
1200
1100
Cumulative Sum of Y
Cumulative Sum of Y
C USU M of Y - Target=0
-400
-500
-25
0
25
50
75
100
1000
900
800
700
600
125
Sample
500
400
300
0
50
75
100
125
150
175
200
Sample
0
25
50
75
100
125
150
175
200
225
250
Sample
31
EWMA;
  0.5
EW MA of Y, Lamda=0.5
16
14
12
EWMA of Y
10
8
6
Avg=5.45
4
2
0
-2
0
25
50
75
100
125
150
175
200
225
250
Sample
32
EWMA;
  0.2
EW MA of Y, Lamda= 0. 2
16
14
EWMA of Y
12
10
8
6
Avg=5.45
4
2
0
-2
0
25
50
75
100
125
150
175
200
225
250
Sample
33
EWMA;   0.1
EWMA of Y
EW MA of Y, Lamda=0.1
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
-1
-2
Avg=5.45
0
25
50
75
100
125
150
175
200
225
250
Sample
34
Detection by SSE



Pick a window of about 30 points including
the “investigated point”
Fit a two-phase regression using all possible
change-points & calculate the SSE
Plot possible change-points vs. their SSEs
35
60 to 90 W i ndow
35 to 65 Wi ndow
3
6
32
44
31
2
5
30
29
4
Y
26
SSE
0
38
3
27
Y
40
28
36
2
25
90% CI
-1
34
24
1
32
23
-2
0
22
21
-3
30
28
-1
20
55
30
35
40
45
50
55
60
65
Examples
from Case # 2
70
Obs
Left Scale:
Right Scale:
Y
65
75
80
85
90
95
Obs
Left Scale:
Y
Right Scale:
SSE
Overlay Pl ot
5
4
15
4
3
14
3
2
13
1
12
1
0
11
0
-1
10
-1
105
110
115
120
125
130
135
140
145
Y
16
SSE
5
105
70
SSE
110 to 140 Wi ndow
Y
60
2
110
115
120
125
130
135
140
145
Obs
Obs
Left Scale:
Right Scale:
Y
SSE
36
SSE
1
42
35 to 65 W i ndow
60 to 90 W i ndow
10
70
17.5
7.5
60
15
5
50
12.5
2.5
40
0
30
7.5
-2.5
20
5
-5
10
2.5
100
90
70
10
60
SSE
Y
SSE
Y
80
50
40
35
40
45
50
55
60
65
70
20
55
60
65
70
Obs
Left Scale:
Y
Right Scale:
Left Scale:
Examples
from Case # 3
SSE
80
85
90
95
Y
Right Scale:
SSE
160 to 190 Wi ndow
110 to 140 Wi ndow
15
14
11
60
10
55
9
50
25
13
45
11
Y
20
SSE
8
12
Y
75
Obs
40
7
35
6
10
15
9
8
7
10
105
110
115
120
125
130
135
140
145
30
5
25
4
20
3
15
155
160
165
Right Scale:
Y
SSE
175
180
185
190
195
Obs
Obs
Left Scale:
170
Left Scale:
Right Scale:
Y
SSE
37
SSE
30
30
Conclusions



Change-point problem is general and can be applied
in many applications such as 4 parameter logistic
regression and degradation curves.
Another application in manufacturing processes
includes detection of the change-point for process
variance.
It is preferred to combine both analytical and visual
techniques; in addition to process expertise; to get
accurate results.
38
References
• Fahmy, H.M. and Elsayed, E.A., Drift Time Detection and Adjustment
Procedures for Processes Subject to Linear Trend. Int. J. Prod.
Research, 2006, 3257–3278.
• Montgomery, D. C., Int. to Stat. Quality Control, 1997, (John Wiley:
NY).
• Nishina, K., A comparison of control charts from the viewpoint of
change-point estimation. Qual. Reliabil. Eng. Int., 1992, 8, 537–541.
• Pignatiello, J.J. Jr. and Samuel, T.R., Estimation of the change point
of a normal process mean in SPC applications. J. Qual. Tech., 2001,
33, 82–95.
• Samuel, T.R., Pignatiello Jr., J.J. and Calvin, J.A., Identifying the time
of a step change with X control charts. Qual. Eng., 1998, 10, 521–
39
527.
Acknowledgements



Lori Pfahler
Julia O’Neill
Jim Lucas
40