Perspective on Use of Statistical Tools
in Pharmaceutical Manufacturing
Karthik Iyer (CQE, CSSBB)
Senior Policy Advisor
CDER/OC/OMPQ
March 9th, 2012
AOAC Conference
* This presentation reflects the views of the author and should not
be construed to represent FDA’s views or policies.
1
Agenda
•
•
•
•
Enforcement Action Examples
CGMP References
ASTM Standards
Conclusions
2
Recent warning letters and other compliance issues
•
•
•
•
Examples involving
Incorrect application of sampling plans
Equipment changes and process
capability
Container closure - determining baseline
defect rates
Recall example – application of ASTM
E2709
3
1. Warning Letter – sampling plans
• Firm using sampling plans incorrectly
– Pooled X vials, used only 1 reportable value, but used
n=X in sampling plan.
– ….based your lot or batch acceptance/rejection
criteria on a single reportable value averaged from a
pooled sample.
…
For ….., you are collecting 3 pooled samples (each pool
= 10 vials). This equates to a lot disposition action on
3 reportable values with corresponding AQL of X%
and LQ of X% respectively. This is not equivalent to
an X or X plan as claimed in your SOP.
4
1. Warning Letter – sampling plans
– Response to 483 indicated firm did not know
how to use and interpret sampling plans
correctly.
– Firm did not understand concepts of
Acceptable Quality Level (AQL) and Limiting
Quality (LQ) and Operating Characteristic
Curve (OC) of a specific sampling plan.
5
2. Warning Letter – equipment comparability and process
capability
• Four (4) tablet products, various strengths
– Initial process qualification used a single-sided tablet
press. During routine production, however, these
products were also being manufactured using a
double-sided tablet press.
– Compression using the double sided press was not
qualified.
– Firm’s response to the FDA 483 attempted to show
statistical equivalence between the single and double
sides presses.
6
2. Warning Letter – equipment comparability and process
capability
• The firm’s written response referenced the Cpk values for processes
using a double-sided tablet press and the single-sided tablet press.
• FDA evaluation of the FDA 483 response
– The Cpk value alone was not an appropriate metric to demonstrate
statistical equivalence. Cpk analysis requires a normal underlying
distribution and a demonstrated state of statistical process control. The
firm did not address these issues in their response.
– Statistical equivalence between the two presses could have been
shown by using either parametric or non-parametric (based on
distribution analysis) approaches and comparing means and variances.
Neither of these approaches was employed. Firm did not use the
proper analysis to support their conclusion that no significant differences
existed between the two compression processes.
7
2. Warning Letter – equipment comparability and process
capability
• Issues –
– Data did not support proper statistical
conclusions.
– Firm did not understand underlying
assumptions required to conduct Process
Capability calculations.
– Firm did not conduct proper statistical
analysis to demonstrate equivalence between
two operations.
8
3. Warning Letter – container closure quality and
baseline defect rates
• Product – LVP in a dual chamber bag
– Numerous complaints of leaks, bursts, and premature activation
during 2 ½ year period.
– Root cause - variability in the film thickness that influenced the
sealing of the bags. Bags have two seals and their strength (or
weakness) relative to each other led to different failure modes.
– Critical defects compromising sterility and stability.
• Poor history for the supplier of this container closure
system.
• Incoming acceptance activities, as well as in-process
and finished product release activities, were found
inadequate.
9
3. Warning Letter – container closure quality and
baseline defect rates
• Issues –
– In responding to the 483, the firm equated customer
complaints to true manufacturing defect rate. They did
not understand that market incident data may not
track with the quality of the product prior to release.
– The proposed sampling plans to identify these known
potential defects were not based on appropriate
statistics.
– Firm did not understand sampling plan used for lot
release.
– Firm could not justify using a riskier sampling plan
(higher probability of accepting bad product).
10
Application of ASTM E2709 - Standard Practice for Demonstrating Capability
to Comply with an Acceptance Procedure.
Tablets,
Q value: 70%
Background: Firm was having recall issues due to dissolution failures on
stability. Dissolution data was analyzed using ASTM 2709. Sample data
below shown for 2 lots (Each row is a different lot). If evaluated correctly,
these lots would have been flagged as high risk for failure.
Unit
6
Unit
7
Unit
8
Unit
9
Unit
10
Unit
11
Unit
12
USP PASS
or
FAIL
ASTM E2709
Probability @
95%
confidence
Unit
1
Unit
2
Unit
3
Unit
4
Unit
5
96%
72%
82%
74%
102%
70%
97%
63%
71%
78%
74%
60%
78%
14%
17%
Pass
0.14%
77%
73%
90%
95%
92%
59%
73%
94%
60%
72%
62%
85%
78%
13%
17%
Pass
0.14%
Mean
SD
RSD
11
CGMP References
•
•
•
•
211.110(a) & (b)
211.165(d)
211.180(e)
Preamble for 21 CFR 210, 211
12
Key elements in these requirements
•
•
•
•
•
•
Control procedures
Monitor the output
Performance
Variability in the characteristics of in-process
material and the drug product
….derived from previous acceptable process
average and process variability estimates
(where possible)
….determined by...suitable statistical
procedures (where appropriate)
13
Statistics
• Can sample tablets at any stage of process and analyze for:
–
–
–
–
Weight.
Content Uniformity.
Dissolution.
Other critical quality attributes and or parameters of interest.
• Can make decisions at any stage of process with respect to:
– Ability for a lot to pass USP UDU and or Dissolution tests in the future.
(ASTM E2709)
– Confidence in sampling. (ASTM E2334 & ASTM E122)
– Capability and Performance analysis. (ASTM E2281)
– Statistical Process Control Charts. (Monitor Variation, ASTM E2587)
• Following tools illustrate making inferences about untested units on
a particular attribute, variable and or parameter with respect to
sample size and an associated confidence.
14
Voluntary Consensus Standards:
US Government Agencies
• OMB Circular A119
– Federal Participation in the Development and Use of
Voluntary Consensus Standards and in Conformity
Assessment Activities (Rev. Feb 10, 1998)
– directs agencies to use voluntary consensus
standards in lieu of government-unique standards
except where inconsistent with law or otherwise
impractical
– intended to reduce to a minimum the reliance by
agencies on government-unique standards.
http://www.whitehouse.gov/omb/circulars/a119/a119.html
15
Sample Size Effect on RSD Limit @ 95% Confidence / 95% Probability
ASTM E2709
7.00
Standard Practice for Demonstrating
Capability to Comply with an Acceptance
Procedure
n=10
5.00
RSD
n=20
4.00
n=30
n=100
3.00
n=500
2.00
n=1000
1.00
0.00
85.1
87.5
90
92.5
95
97.5
100
102.5
105
107.5
110
112.5
114.9
Sample Mean
Sample Size Effect on RSD Limit @ 99% Confidence / 95% Probability
7.00
6.00
5.00
RSD
One tool to analyze
Uniformity of Dosage
Units
6.00
n=10
n=20
4.00
n=30
n=100
3.00
n=500
2.00
n=1000
1.00
0.00
85.1
87.5
90
92.5
95
97.5
100
102.5
Sample Mean
105
107.5
110
112.5
114.9
16
ASTM E2709 Explanation
Standard Practice for Demonstrating Capability
to Comply with an Acceptance Procedure
• Slide shows the relationship between sample size and
tolerance for variability. As sample size increases, so does
the tolerance for variability.
• The analysis was performed using ASTM E2709-10. The
RSD limits on the y-axis represent the maximum variability a
lot can possess to ensure with 95 or 99% confidence that
there is at least a 95 or 99% probability a lot will comply with
the USP Uniformity of Dosage Units test based upon a given
sample size, confidence level, and sample mean.
– For example: If you sampled 30 units and had a sample
mean of 95%, then the maximum RSD value for those 30
units would be ~3.0% to be 95% confident that there is at
least a 95% probability a future sample from the lot would
pass the USP UDU test.
17
ASTM E2334
Setting an Upper Confidence Bound For a Fraction or
Number of Non-Conforming items, or a Rate of Occurrence
for Non-conformities, Using Attribute Data, When There is a
Zero Response in the Sample
Confidence vs Sample Size
100
90
80
Confidence
70
60
Pd=1%
50
Pd=0.5%
40
Pd=0.065%
30
20
10
0
6
10
12
24
30
100
150
300
500
1000
Sample Size
18
ASTM E2334 Explanation
Setting an Upper Confidence Bound For a Fraction or
Number of Non-Conforming items, or a Rate of Occurrence
for Non-conformities, Using Attribute Data, When There is a
Zero Response in the Sample
• Slide shows the relationship between Confidence and
Sample Size. As sample size increases, so does
confidence demonstrated.
• The analysis was performed using ASTM E2334-09.
Keeping the maximum percent defective constant (1,
0.5, and 0.065%) a line was generated to show how
sample size effects the confidence demonstrated in
having no more than the maximum percent defective. A
zero response was assumed (that is zero defects in the
sample) and a binomial distribution was used.
– For example: If you desire a percent defective of no more than
0.5% and sample 30 units, then you are only ~15% confident
that your lot has no more than 0.5% defects.
19
ASTM E2334
Setting an Upper Confidence Bound For a Fraction or
Number of Non-Conforming items, or a Rate of Occurrence
for Non-conformities, Using Attribute Data, When There is a
Zero Response in the Sample
Sam ple Size vs Percentage of Non Conform ities
40
Percentage Non Conforming
35
30
25
20
99%
Confidence
15
95%
Confidence
10
5
0
10
20
30
50
Sam ple Size
500
1000
20
ASTM E2334 Explanation
Setting an Upper Confidence Bound For a Fraction or
Number of Non-Conforming items, or a Rate of Occurrence
for Non-conformities, Using Attribute Data, When There is a
Zero Response in the Sample
• Slide shows the relationship between the upper
confidence bound on percent defects and sample size.
As sample size increases the upper confidence bound
on percent defects decreases.
• The analysis was performed using ASTM E2334-09.
Keeping the confidence level constant (95 and 99%) a
line was generated to show how sample size effects the
upper confidence bound percent defects. A zero
response was assumed (that is zero defects in the
sample) and a binomial distribution was used.
– For example: If you want to be 99% confident that there is no
more than 1% defective units in your lot, then you must sample
~460 units with a zero response.
21
ASTM E122
Standard Practice for
Calculating Sample Size to Estimate, With Specified
Precision, the Average for a Characteristic of a Lot or
Process
Estimate Precision vs Sample Size
10.00
9.00
Precision Range
8.00
7.00
σ=1.0
6.00
σ=3.0
5.00
σ=5.0
4.00
σ=6.0
3.00
σ=10.0
2.00
1.00
0.00
10
20
30
50
Sample Size
500
1000
22
ASTM E122 explanation
Standard Practice for
Calculating Sample Size to Estimate, With Specified
Precision, the Average for a Characteristic of a Lot or
Process
• Slide shows the relationship between sample
size and precision. As sample size increases,
so does your estimate precision.
• The analysis was done using ASTM E122-09.
Lines were generated using different sample
sizes to show the effect it has on your estimate
precision.
– For example: If you sampled 30 units and your sigma
value was 6, then your sample average is ~ +/-3.5%
of your true population average.
23
ASTM E2281
Standard Practice for
Process and Measurement Capability Indices
Confidence in Manufacturing Capability
3.50
Manufacturing Capability
3.00
2.50
2.00
95% Confidence
99% Confidence
1.50
1.00
0.50
0.00
6
10
12
20
24
30
50
Sample Size
100
250
500
1000
24
ASTM E2281 Explanation
Standard Practice for
Process and Measurement Capability Indices
• Slide shows the relationship between a reported
Process Capability Index (Cpk (3.14)) and
sample size. As sample size increases, so does
the reported Cpk.
• When reporting a Cpk, a lower 95 or 99%
confidence bound should always be the value
reported. As this value accounts for the sample
size in which the Cpk was estimated.
– For example: If you sampled 30 units and estimated
a Cpk of 3.14, then the value reported should be ~2.5
(that is I am 99% confident that the Cpk for my
process is at least 2.5). The analysis was done using
ASTM E2281-08.
25
ASTM E2281
Standard Practice for
Process and Measurement Capability Indices
Confidence in Manufacturing Performance
3.50
Manufacturing Performance
3.00
2.50
2.00
95% Confidence
99% Confidence
1.50
1.00
0.50
0.00
6
10
12
20
24
30
50
Sample Size
100
250
500 1000
26
ASTM E2281 Explanation
Standard Practice for
Process and Measurement Capability Indices
• Slide shows the relationship between a reported
Process Performance Index (Ppk (2.79)) and
sample size. As sample size increases, so does
the reported Ppk.
• When reporting a Ppk, a lower 95 or 99%
confidence bound should always be the value
reported. As this value accounts for the sample
size in which the Ppk was estimated.
– For example: If you sampled 30 units and estimated
a Ppk of 2.79, then the value reported should be ~2.2
(that is I am 99% confident that the Ppk for my process
is at least 2.2). The analysis was done using ASTM
E2281-08.
27
ASTM E2587
Standard Practice for Use of Control Charts in
Statistical Process Control
•
SPC (Statistical Process Control) Charts are a
collection of very effective statistical-graphical
tools which can be used to:
–
–
–
•
Understand and diagnose your data.
Track performance to identify problems, or shifts in
performance (good or bad).
Control or adjust the process to maintain desired
performance.
Can be applied for data based on Incoming, Inprocess, or Lot release samples.
28
ASTM E2587
Standard Practice for Use of Control Charts in Statistical Process Control
Variable X-bar-R chart
Voice of the Process
Is the process in a state of control?
Sample Mean
5
2 5
103
UCL=103.048
5 5
2
_
_
X=101.868
102
101
6
5
6
5
LCL=100.689
1
100
1
1
11
21
31
41
51
Sample
61
71
81
91
Sample Range
4.5
UCL=4.326
3.0
_
R=2.046
1.5
0.0
LCL=0
1
11
21
31
41
51
Sample
61
71
81
91
29
ASTM E2587
Standard Practice for Use of Control Charts in Statistical Process Control
• Chart is used to detect special causes of
variation during manufacturing.
• Control is determined against standard 8
rules established by Dr. Walter Shewhart.
• Preceding chart is called X-bar-Range with
Subgroup size of 5 tablets (each point is
an average of 5 individual results).
• Control limits reveal true variability of the
process.
30
ASTM E2587
Standard Practice for Use of Control Charts in Statistical Process Control
Attribute nP chart
Pass/Fail test for incoming tablet bottles (nP chart)
The tablet bottle is either pass or fail (Binary response)
Sample 100 bottles per incoming lot
12
UCL=11.38
Sa mple Count
10
8
6
__
NP=4.9
4
2
0
LCL=0
1
4
7
10
13
16
19
22
25
28
Sample
Here we assume the failure rate is 5 bottles/100 bottles
Need to know historical defect rate from supplier
31
Conclusions
• What have we covered?
– Enforcement action examples and CGMP references.
– Use of statistics to quantify relationship between
confidence associated with attribute, variable and or
parameter of interest with respect to the sample size
collected.
• To make inferences on untested units.
• Can be applied to In-coming, In-process, or Finished
samples.
• Can be used for real time manufacturing and or
annual/periodic product reviews.
32
Conclusions
• Other Statistical Tools
– Sampling plans
• Do they describe Consumers Risk?
• How are true defect rates calculated to use a particular
sampling plan?
– Confidence, Prediction, and Tolerance Intervals
• Is the correct statistical tool being applied for the
right analysis?
• Do the tools help answer questions about
product quality and process performance?
33
Conclusions
• The specific statistical tools and analysis
depends on what variables, attributes,
parameters are being used to monitor process
performance and product quality.
• The preceding example is just one set of
statistical methods available to monitor process
performance and product quality.
• Statistics is a tool to elicit information to confirm
that a specific manufacturing process is
producing quality product.
34
Acknowledgements
• Alex Viehmann – CDER/OPS
• Grace McNally – CDER/OC
35