Evaluating Product
Characteristics for Acceptance
Mark D. Johnson, PMP
ASQ CMQ/OE & CQE
AbbVie Inc.
Agenda
•
•
•
•
Product Characteristics
Defect Rating Criteria
Sampling Techniques
Questions
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
2
Product Characteristics
• Identify characteristics important to
confirming product performance
• These might be items such as:
Component dimensions for device components
Content uniformity for tablets
Potency and impurities for a drug product lot
Force to activate an autoinjector
Correct tablet coloring for different dosage
strengths of same product
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
3
Product Characteristics
Product characteristics can be based on:
• Compendial performance expectations
• Current quality attribute testing for a similar
product
• New tests required to assess unique features of a
new product
• Product features designed to address specific user
needs
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
4
Defect Rating Criteria
When a pharmaceutical, medical device or combination product is defective,
three defect types are possible:
• Design - the pharmaceutical was manufactured correctly, but anticipated
or unanticipated side effects caused by the drug use resulted in patient
harm or injury; the medical device was poorly designed or not properly
tested, creating a situation where the products might be defective and
dangerous.
• Manufacturing - the product was designed to meet requirements, but
manufactured improperly or contaminated somehow during the
manufacturing process, potentially exposing the patient to harm.
• Marketing – the product’s instructions, warnings, or recommendations for
the use of the drug are not sufficiently clear or appropriate to help
consumers use the product correctly.
Reference: HG.org Legal Resources Website
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
5
Defect Rating Criteria
Examples of the three defect types are:
• Design – An infusion pump screen design is cumbersome or confusing to
users, which causes a delay in therapy. For example, the “Start Infusion”
key may be located next to the “Power” key, and a user may turn off the
infusion pump and lose programmed pump settings instead of initiating
infusion. In some cases, programmed settings are lost when a user turns
the pump off, and the infusion settings have to be re-entered after the
pump restarts.
• Manufacturing – Excess water has entered a drug substance batch
manufacturing run, due to a faulty valve in the purified water system. This
resulted in elevated impurity and reduced potency levels within the batch.
• Marketing – A patient does not adequately comprehend the medication’s
use instructions that require him to eat a meal and drink water prior to
taking his daily dose.
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
6
Defect Rating Criteria
• Product characteristics need to be examined to determine
the impact of not meeting their acceptance criteria
• These characteristics might be based on similar attributes
assessed for a comparable product
• Likewise, they may be identified and confirmed during the
Process Validation Lifecycle activities recommended to
document development, verification, validation and future
manufacture of the product
• This examination may be based on evaluating the impact of
the defect upon product functionality or clinical effect to a
user
• Also, risk-based evaluation can be used to identify the
impact of a product defect
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
7
Defect Rating Criteria
• Product sampling based on identification of defects was
initially used in the Aerospace industry
• This approach was adopted within the pharmaceutical
industry in the 1970s and 1980s
• There is no industry-wide consensus on a specific percent
defective level used to characterize Critical, Major, Minor or
Cosmetic defects
• Dr. Wayne Taylor, a recognized sampling expert, cited the
following percent defective levels were commonly used to
assess medical devices for product defects:
 Critical (Health and Safety): 0.065%, 0.1%, 0.25%
 Major (Functional): 0.25%, 0.4%, 0.65%, 1.0%
 Cosmetic: 2.5%, 4.0%
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
8
Defect Rating Criteria
Product Attribute Typical Assigned AQLs
Critical
0.04%, 0.065%, 0.1%, 0.15%, 0.25%
Major Functional
Minor Functional
Cosmetic Visual
0.25%, 0.4%, 0.65%, 1.0%
0.65%, 1.0%, 1.5%
1.5%, 2.5%, 4%, 6.5%
• Acceptance Quality Limits (AQLs) in Pharma / Devices
• These attribute categories help identify the foci of our sampling activities.
• Are we only interested in assessing superficial product characteristics, or do
we need to assess critical product operations or functions as well?
• We use these attribute categories to assure our processes operate to meet or
surpass our expectations for observing defective product.
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
9
Defect Rating Criteria
Defect rating criteria may be defined as:
• Critical: A nonconformity likely to present a hazard to
health
• Major Functional: A nonconformity which may cause the
product to be unfit for use, significantly degrades its
function or performance, or is likely to generate a
complaint
• Minor Functional: A nonconformity which may cause the
product to function poorly or a user inconvenience, but
may still be fit for use, or possibly generate a complaint
• Cosmetic Visual: A nonconformity detrimental to the high
quality image of a product that will not affect usability or
functionality, but affects only appearance of the product
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
10
Defect Rating Criteria
These criteria can be based on:
• Similar defects experienced with a similar product
• Historically observed defects for the product’s dose
modality (e.g., oral solution, tablets, capsules)
• Risk-based assessment of hazardous situations/failure
modes experienced with the product
• Means of evaluating product performance vs. current
test methods or other necessary standards
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
11
Defect Rating Criteria
Assessment of these criteria is completed by:
• Laboratory testing of product vs. required quality
attributes
• Functional testing of product to meet operational
requirements
• Visual examination for any noticeable defects
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
12
Sampling Techniques
• To assess product characteristics, an adequate
sample of sufficient confidence must be used
• Based on the defect level of the characteristic of
interest, different confidence levels might be
considered for validation purposes:
Critical (e.g., % defective ≤ 0.4%), validate to ≥ 90%
confidence
Major (e.g., 0.40% < % defective ≤ 2.5%), validate to ≥
50% confidence
Minor (e.g., % defective > 2.5%) validate to ≥ 20%
confidence
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
13
Sampling Techniques
Similarly for process validation activities, an
adequate level of confidence is also required:
• New process/product or major process change,
validate to ≥ 90% confidence
• Routine revalidation, validate to ≥ 50% confidence
• Minor process change or accelerated testing,
validate to 20-50% confidence
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
14
Sampling Techniques
There are different methods available to sample product
for inspection:
• Simple Random Sampling – Selecting samples so that each
unit has an equal chance of being selected
• Stratified Random Sampling – Selecting samples
deliberately from each time period or location in a batch
• Nested Sampling – Selecting units from locations within a
batch and obtaining multiple samples from each location
• Systematic sampling – Selecting units periodically over time
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
15
Sampling Techniques
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
16
Sampling Techniques
Validation vs. Product Acceptance Sampling Plans – What
are the differences?
• Validation Sampling




Cannot assume process is good, because it has not been validated yet
Need to prove it's good
"Guilty until proven innocent“
Need to use tighter sampling plan than one used for product acceptance (e.g., will reject
when data shows high confidence (e.g., 90% or 95%) that product performance vs.
acceptance criteria is worse than acceptable % defective)
• Product Acceptance Sampling
 Assumes process is good, since process successfully validated
 "Innocent until proven guilty“
 Will only reject when data shows low confidence (e.g., 5-10%) product performance vs.
acceptance criteria is worse than the AQL
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
17
Sampling Techniques
If you pass the “n=575, a=0”
PPQ sampling plan, you have
90% confidence (e.g., (1-0.1
Pr(Accept)*100%) you will
accept a lot ≤ 0.4%
Defective.
If you pass the “n=32, a=0”
Product Acceptance
sampling plan, you have 12%
confidence (e.g., (1-0.88
Pr(Accept)*100%) you will
accept a lot ≤ 0.4%
Defective.
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
18
Sampling Techniques
• Once the process has been validated, the confidence of
the sampling plan can be relaxed to routinely assess
batches for large quality defects.
• ANSI Z1.4 provides attribute sampling plans to be used
for routine lot inspection and acceptance
• ANSI Z1.9 provides variables sampling plans to be used
for routine lot inspection and acceptance
• Both plans provide switching rules for transitioning
from normal to tightened, tightened to normal, normal
to reduced and reduced to normal sampling practices,
dependent on the continuing documented quality
history of the product
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
19
Sampling Techniques
• Once you’ve determined the product characteristics to
assess vs. their required acceptance criteria, you can
build a sampling approach
• Two types of sampling plan approaches typically used:
 Attributes: units assessed within each sample can only
pass or fail the acceptance criteria
 Variables: mean and standard deviation are calculated for
a sample of units, then a range is calculated using a factor
suitable to the defect level and sample size for comparison
with the acceptance criteria
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
20
Sampling Techniques
• Attribute characteristic examples:
Fill volume: >20 mL
Tablet defects: cracked, chipped
Extraneous matter: visible particulate, rubber, hair
• Variables characteristic examples:
Component dimensions: Length = 2 ± 0.25 mm
Ejection force: ≤ 10 Newtons
Crush test for tablet: ≥ 100 Newtons
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
21
Sampling Techniques
• Attribute Sampling:
Easy to conclude result, either pass/fail
Many samples might be required to provide adequate
confidence in pass/fail decision
• Variables Sampling:
Variables data contains more information than
percent nonconforming
More descriptive
More statistical information and power
Fewer samples required for same statistical power
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
22
Sampling Techniques
Operating Characteristic (OC) Curve
0.4% Defective Attribute Sampling Plans
Probability of Acceptance
0.95
n
90% conf 575
50% conf 175
20% conf 55
Tightened 50
Normal
32
n
c
0.50
c
0
0
0
0
0
sample size
acceptance number
0.20
0.10
0.00
0.0 0.4
2.0
4.0
6.0
8.0
Lot Percent Defective
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
23
Sampling Techniques
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
24
Questions…
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
25
Thank You!
20 May 2015
Midwest Biopharmaceutical Statistics
Workshop
26