Setting Specifications
Statistical considerations
Enda Moran – Senior Director, Development, Pfizer
Melvyn Perry – Manager, Statistics, Pfizer
B
i St
ti ti
Basic
Statistics
Population
p
distribution
(usually unknown).
Normal distribution
described by μ and σ.
15
1.5
True batch assay
1.0
Distribution of
possible values
0.5
0.0
98
99
100
We infer the population from
samples by calculating x and s.
1.5
1.0
1.5
1.5
Sample 1
Average
98.6
True batch assay
y
True batch assay
y
1.0
Sample 2
Average 98.7
1.0
Distribution of
possible values
0.5
99
100
Distribution of
possible values
0.5
0.0
0.0
98
True batch assay
y
Distribution of
possible values
0.5
0.0
Sample 3
Average 99.0
98
99
100
98
99
100
2
I t
Intervals
l
30
28
26
24
Population
average
22
20
18
16
0
10
20
30
40
50
60
70
80
90
100
•100 samples of size 5 taken from a population with an average of 23.0 and a standard deviation of 2.0.
•The highlighted intervals do not include the population average (there are 6 of them).
•For
For a 95% confidence level expect 5 in 100 intervals to NOT include the population average
average.
•Usually we calculate just one interval and then act as if the population mean falls within this interval.
3
I t
Intervals
l
Point Estimation
The best estimate; eg MEAN
Interval Estimation
A range which contains the true population parameter or a future
observation to a certain degree of confidence.
Confidence Interval
- The interval to estimate the true population parameter (e.g. the population
mean).
mean)
Prediction Interval
- The interval containing the next single response.
Tolerance Interval
- The interval which contains at least a given proportion of the population.
4
F
Formulae
l ffor Intervals
I t
l
Intervals are defined as:
x ± ks
A
Assuming
i a normall di
distribution
t ib ti
• Confidence (1-α) interval
0 .5
⎛ 1⎞
CI = x ± ⎜ ⎟ t α s
⎝ n ⎠ 1− 2 ,n −1
• Prediction (1-α)
(1 α) interval for m future observations
0. 5
1⎞
⎛
PI = x ± ⎜ 1 + ⎟ t α s
n ⎠ 1− 2m ,n −1
⎝
• Tolerance interval for confidence (1-α) that proportion
( ) is
(p)
i covered
d
(n − 1)⎜⎛ 1 + 1 ⎞⎟ z(21− p )
TI = x ±
⎝
χ
n⎠
2
α ,n −1
2
s
5
P
Process
Capability
C
bilit
3s
Process capability is a measure of the
risk of failing specification. The spread of
the data are compared with the width of
th specifications.
the
ifi ti
3s
The distance from the mean to the
nearest specification relative to half
the process width (3s).
The index measures actual
performance. Which may or may
not be on target i.e., centred.
LSL
⎧USL − x x − LSL ⎫
Ppk = min⎨
,
⎬
s
3s ⎭
3
⎩
x
x − LSL
USL − x
USL
6
P
Process
Capability
C
bilit – Ppk
P k and
dC
Cpk
k
• Ppk should be used as this is the actual risk of failing specification
specification.
• Cpk is the potential capability for the process when free of shifts and drifts.
Random data of mean 10 and SD 1, thus natural span 7 to 13.
Added shifts to simulate trends around common average
average.
With specs at 7 and 13 process capability should be unity.
Process Capability of Shifted
I Chart of Shifted
15
LSL
1
14
UCL=13.589
13
Ind
dividual Value
When data is with trend Ppk
less than Cpk due to method of
calculation of std dev.
12
11
USL
Within
Ov erall
P rocess D ata
LS L
7
Target
*
USL
13
S ample M ean
10.0917
S ample N
30
S tD ev (Within) 1.16561
S tD ev (O v erall) 1.95585
P otential (Within) C apability
Cp
0.86
C P L 0.88
C P U 0.83
C pk 0.83
O v erall C apability
_
X=10.092
10
Pp
PPL
PPU
P pk
C pm
9
0.51
0.53
0.50
0.50
*
8
7
6
LCL=6.595
6
1
4
7
10
13
16
19
Observation
22
25
O bserv ed P erformance
P P M < LS L
0.00
P P M > U S L 100000.00
P P M Total
100000.00
28
10
12
14
E xp. O v erall P erformance
P P M < LS L
56966.34
PPM > USL
68512.70
P P M Total
125479.03
UCL=13.038
13
12
11
LSL
USL
Within
Ov erall
P rocess Data
LS L
7
Target
*
USL
13
S ample M ean
10.0917
S ample N
30
S tD ev (Within)
0.982187
S tD ev (O v erall) 1.14225
P otential (Within) C apability
Cp
1.02
C P L 1.05
C P U 0.99
C pk 0.99
_
X=10.092
10
O v erall C apability
Pp
PPL
PPU
P pk
C pm
9
8
LCL=7.145
7
1
4
7
10
13
16
19
Observation
Cpk
C
k uses average moving
i
range SD (same as for control
chart limits).
Cpk
p is close to 1 at 0.83.
Process Capability of Raw
I Chart of Raw
IIndividual Value
8
E xp. Within P erformance
P P M < LS L
3995.30
PPM > USL
6296.59
P P M Total
10291.88
Ppk uses sample SD. Ppk less
than 1 at 0.5.
22
25
28
7
O bserv ed P erformance
P P M < LS L 0.00
P P M > U S L 0.00
P P M Total
0.00
8
E xp. Within P erformance
P P M < LS L
822.51
P P M > U S L 1533.13
P P M Total
2355.65
9
10
11
12
E xp. O v erall P erformance
P P M < LS L 3397.73
P P M > U S L 5446.84
P P M Total
8844.57
13
0.88
0.90
0.85
0.85
*
When data is without trend
Ppk is same as Cpk. Only
small differences are seen.
Cpk effectively 1 at 0.99.
Ppk close to 1 at 0.85.
7
M
Measurement
t Uncertainty
U
t i t
LSL
Good
parts
t
almost
always
passed
p
Bad parts
almost
always
rejected
j t d
±3σmeasurement
USL
σmeasurement
σtotal
Bad parts
almost
always
rejected
±3σmeasurement
The grey areas highlighted represent those parts of the curve with the
potential for wrong decisions, or mis-classification.
8
Misclassification with less
variable
i bl process
σtotal
LSL
±3σ
3σmeasurement
USL
σmeasurement
±3σmeasurement
9
M
Measurement
t Uncertainty
U
t i t
Precision to Tolerance Ratio:
How much of the tolerance is taken up by
measurement error.
This estimate may be appropriate for evaluating
how well the measurement system can perform
with respect to specifications.
Gage R&R (or GRR%)
What percent of the total variation is taken up by
measurement error (as SD and thus not additive).
6 × σ MS
P /T =
Tolerance
Tolerance = USL − LSL
USL = Upper
pp Spec
p Limit
LSL = Lower Spec Limit
σ MS = Std. Dev. of Measurement Sys.
%R & R =
σ MS
× 100
σ Total
Use Measurement Systems Analysis to assess if the assay method is fit for purpose.
It is unwise to have a method where the specification interval is consumed by the
measurement variation alone.
10
Specification example –
T l
Tolerance
IInterval
t
l
Data from three sites used to set specifications
specifications.
16.5
16.0
Tolerance interval found from pooled data of 253
batches.
Tolerance interval chosen as 95% probability that
mean ± 3 standard deviations are contained.
Sample size:
n=253
Mean ± 3s
Mean = 14.77
Tolerance
interval
((95%
%/
99.7%)
s=0.58
R
Range
13 03 - 16.51
13.03
16 51
12 89 - 16.65
12.89
16 65
If sample size was smaller, difference between
these calculations increases.
As the sample size approaches infinity the TI
approaches mean ± 3s.
15.5
15.0
14.5
14.0
13.5
13.0
1
2
3
Code
N
5
10
15
30
∞
k multiplier
6.60
4.44
3 89
3.89
3.35
2.58
Table of values for 95%
probability of interval
containing 99% of
population values
values.
Note at ∞ value is 2.58
which is the z value for
99% coverage of a normal
population
11
Specification Example St bilit
Stability
Batch history
(process and
Total SD (p
measurement)
Stability batch
with SD
(measurement)
Shelf life set from 95% CI on slope from three clinical batches.
Need to find release criteria for high probability of production batches
meeting shelf life based on individual results being less than
specification.
Review
R
i
off batch
b t h history
hi t
will
ill llead
d tto a process capability
bilit statement
t t
t
against release limit.
12
Specification Example St bilit
Stability
Release limit is calculated from the rate of
change and includes the uncertainty in the
slope estimate (rate of change of parameter)
and the measurement variation).
8
7
6
⎛
s2 ⎞
RL = Spec - ⎜ Tm + t T 2sm2 + a ⎟
⎜
n ⎟⎠
⎝
Specification = 8
Shelf life ((T)) = 24
Parameter slope (m) = 0.04726
Variation in slope (sm) = 0.00722
Variation around line (sa) = 0.86812
t on 126 df approx
approx. = 2
N= 1 for single determination
Release limit = 5.1
5
4
3
2
1
0
0
6
12
18
24
30
MONTHS
A similar approach can be taken in more complex situations; e.g,, after
reconstitution of lyophilised product
product. Product might slowly change during cold
storage and then rapidly change on reconstitution.
Allen et al., (1991) Pharmaceutical Research, 8, 9, p1210-1213
36
13
C i with
Coping
ith lilimited
it d d
data
t
Problems with n<30.
•
•
•
Difficult to calculate a reliable estimate of the SD.
Difficult to assess distribution.
Difficult to assess process stability.
With very limited data the specifications will be wide due to the large multiplier
multiplier.
•
This reflects the uncertainty in the estimates of the mean and SD.
Is there a small scale model that matches full production scale?
How variable is the measurement system?
Alternatives are:
Example with 4 values
•
Min / max values based on
scientific rationalisation
•
•
•
Mean ± 3s
Mean ± 3s
13.2
27.0
Mean ± 4s or minimum Ppk
p = 1.33.
Mean ± 4s
10.9
29.3
Tolerance interval approach
TI (95/99.7)
-1.5
41.7
N(20 4) – 19.9,
N(20,4)
19 9 16
16.9,
9 22
22.1,
1 21
21.3
3
14
Updating
p
g Specifications
p
When, why, and a scenario for consideration
•
Specifications for CQAs might require revision during the product lifecycle. For example, new clinical data and
i t lli
intelligence
on continued
ti
d use off a product
d t might
i ht d
drive
i a change
h
tto a CQA specification
ifi ti
•
Scenario:
What if a process producing a new product approaching regulatory approval is ‘incapable’ of meeting the clinicallytested CQA in the long-term? For example.........
Proposed Spec.
Process
capability
estimate
CQA
Proposed Spec.
If spec.
required
t be
to
b sett
at the
Process
capability
estimate
CQA
Clinical
experience /
New spec.
Clinical
experience
clinicallyclinically
tested
level.........
Clinical
/ Validation Lots
Clinical
/ Validation Lots
An ‘incapable’ process
•
How could we manage this situation? Complex. Many factors need to be considered if setting specifications at
clinically tested level (will
(
be discussed through this workshop))
•
IF it must be done for risk reduction purposes, and if there is sufficient uncertainty that the Proposed Spec. for the
CQA could cause harm, a possible approach may be to move towards the clinically-tested level in stages.
- Accrue approx.
pp
n=25-30 batches/lots to consider estimates of future p
process capability
p
y acceptable.
p
Adjust spec. to a revised process capability estimate.
- Accrue further lots of the stabilised process to approx. n=85. Make FINAL spec. adjustment to a revised
process capability estimate.
15
Summary
• Process capability & measurement uncertainty
• Setting spec.s - large number of datapoints
• Setting spec.s – small number of datapoints
16