A Qualitative Decision Making Tool to Aid in
Defining the Number of Lots for a Process
Validation Campaign
Leslie Sidor — Amgen Inc.
2012 Midwest Biopharmaceutical Statistics Workshop
May 22, 2012
Outline
 Where is Amgen today
 Why a qualitative tool
 Tool development
 Conclusions
Operational Excellence
2
Lifecycle Approach to Process
Commercialization Began 10 years Ago
Magnitude of post approval changes can
take process back into Commercialization
Amgen
Commercialization
Process
Amgen Production
Activity
FIH Process
Development
Comm. Process
Development
Process
Char.
P3
Campaign
Phase 1 & 2 Campaigns
Val. &
File Prep
Validation
Campaign
PAI & Launch
Prep
Commercial Production
Amgen’s old paradigm:
Validation is a single event in
time using 3 lots
Operational Excellence
3
Post-approval
Development
Lifecycle Approach to Process Validation
Began 10 years Ago
Magnitude of post approval changes can
take process back into Commercialization
Amgen
Commercialization
Process
FIH Process
Development
Amgen Production
Activity
Comm. Process
Development
Phase 1 & 2 Campaigns
Process Validation
Scope
As defined in the FDA Process
Validation Guidance (Jan 2011)
Stage 1
Process Design
Process
Char.
P3
Campaign
Conf. &
File prep
Conf.
Camp.
Stage 2
PPQ
PAI & Launch
Prep
Postapproval
Development
Commercial Production
Stage 3
Continued Process Verification (CPV)
Stage 2 will flex in size to assure “process is capable of
reproducible commercial manufacturing”
Operational Excellence
4
Traditional Statistical Tools to assess
process capability Require N > 30 lots
 Control charts
–
–
–
–
Lot release
In process controls
Analytical method performance
Stability data
 Process capability indexes
Inadequate sample
size creates under or
over reaction to
signals
– Lot release
– In process controls
Need to leverage qualitative information in Stage 1 to avoid
statistical issues
Operational Excellence
5
Goal: Leverage information from Stage 1 using
a decision making tool to define size of PPQ
Stage 1
Process Design
Amgen
Activities
Stage 1
•Design of Experiments
•Multiple Risk Assessments
•Critical Quality Attribute Matrix
•Comparability
•Etc.
Stage 2
PPQ
Stage 2
•PV protocols
•Validation report
•Product comparability
•Process comparability
•NC trending
•Etc.
Stage 3
CPV
Stage 3
•Control charting for numerous
quantitative parameters
•Stability trending
•Process capability assessment
•Analytical method performance
•NC trending
•Quality System performance
•Etc.
Integrate Stage 1 information + indirect performance of
the facility using a Decision Making Tool
Operational Excellence
6
Attributes of the Decision Making Tool using an
index
 Tool maps qualitative factors into a numerical index
to describe factors that impact ability to control the
process
 Tool uses qualitative characteristics of performance
– Direct measures
– Indirect measures
 Tool needs to be…
–
–
–
–
Objective
Simple to use
Adjustable to allow for continuous improvement
Appropriately specific
Operational Excellence
7
Relative Ranking for Decision Making
MATRIX
Indicator Characteristics
Owner
Targeted Owner (5 Points)
______
Flag
Targeted Flag (7 Points)
______
Class
Top 25% (0 Points)
______
Middle 50% (1 Point)
______
Bottom 25% (3 Points)
______
Unknown Class (5 Points)
______
Vessel Type
Oil or Chemical Tanker (1 Point) ______
Freighter 10+ Years (2 Points)
______
Passenger Ship (1 Point)
______
Low Commodity Carrier (2 Points)______
Performance Characteristics
Detentions (Past 12 Months)
0 (-5 Points)
1+ ( __ Detentions × 10 Points)
Priority 1 Deficiencies (Past 18 Months)
0 (-5 Points)
1-2 (0 Points)
3+ ( __ Deficiencies × 5 Points)
Casualties (Past 3 Years)
0 (0 Points)
1+ ( ___ Casualties × 3 Points)
PSC Exam – No Priority 1 Deficiency
Less than 3 Months (-5 Points)
Between 3 and 6 Months (-2 Points)
Greater than 6 Months (0 Points)
Total Indicator Score
Total Performance Score
______
Total Matrix Score: ___________
Ballast Water Exam Required: Yes _____ No ______
U.S. Coast Guard Boarding Required: ______

Score
17+
7-16
4-6
3
_______
_______
_______
_______
_______
_______
_______
_______
_______
_______
_______
Vessel Priority
I
II
III
IV
Source: US Coast Guard, Field Demonstration Workshop on Performance-Based Inspection of Vessels
Entering the St. Lawrence Seaway (Prioritizing Vessels for US Coast Guard Inspection), April 2000
Operational Excellence
8
For Internal Use Only.
Amgen Confidential.
Step 1: Define what the index will represent

What does this index represent?
– Is it consequence based: the final number represents the
impact from consequences of interest.
Items of
interest for
scoring
Scoring
Factors
List of possible sources
of variation that can
impact process
understanding
Scoring of the most
impactful sources of
variability for a
molecule
Factors affecting
the process
understanding
– Consequence of the index: description of the process by
understanding sources of variability
Scoring of Performance Characteristics that describe
process understanding. Index will drive the size of the PPQ
campaign
Operational Excellence
9
2. Brainstorm factors that could affect the
index

Team members: 16 staff over 8 functional areas

Over 100 factors were identified

Factors were classified into 8 Performance
Characteristics
-Process
-Product
-Raw Materials
-Facility
-Risk
-Regulatory
-Analytical Methods
-Documentation

Factors were modified to remove ambiguous language
(e.g.: robust, approximate, successful, etc.)

DISADVANTAGE: Miss a key functional area, factor list
will be incomplete
Operational Excellence
10
Step 3: Identify specific situations for
which specific actions are required
Question
Answer
Action
Is this a new facility?
Yes
Proceed to PPQ, but Perform the
maximum amount of work based on
the action threshold
Is process locked?
No
Do not proceed to PPQ
Is a PPQ protocol in
place?
No
Do not proceed to PPQ
These questions are asked before the tool is used
and overrides any decision from the tool
Operational Excellence
11
Steps 4 & 5: Select a basic scoring or indexing &
develop scoring scales for each factor
Performance
Characteristic
Factor
No: Less process
understanding
Product
Product comparability has been
demonstrated at commercial scale
No =1
DP only: Experience with multiple DS
ages
Yes=0
Process
Process tracking active in Clinical
manufacturing
No=1
Raw Materials
All raw materials have been used for
the same application at commercial
scale
No=1
Average SCORE
Use a binary response and compute % in
the “No’s”
Operational Excellence
12
75%
Considerations for scoring
 All factors are not created equal
 Apply appropriate weighting: Should a “No”
for some factors be worth 2 points?
 Possibilities for average score
– Compute straight % for the “NO’s” (assumes each factor = 1
point if NO)
– Weight some factors greater than 1 point, then compute %
Operational Excellence
13
Step 6: Set action threshold for the index
•Option 1: Define # of lots
•Option 2: Define # of lots + a transition plan
Score (% “No’s)
# of Lots
for PPQ
+ Transition plan into CPV
> 80%
>6
+ monitor PPQ parameters for next 15 lots;
then move to CPV
>60% to < 80 %
5 to 6
+ monitor PPQ parameters for next 10 lots;
then move into CPV
>40 % to < 60%
3 to 4
+ monitor PPQ parameters for next 5 lots;
then move into CPV
< 40%
1 to 2
Move into CPV
No Decision has been made at Amgen to define
the action threshold
Operational Excellence
14
Final steps are not complete
 Step 7: Organize the scoring scale, index calculation
and action thresholds into an SOP
– A white paper will be written describing development and
rationale
– An SOP or training will be needed on how to use the tool
– Documentation of the tool user’s answers and the
justification of the answers will be incorporated into the tool
 Step 8: Verify the documentation makes sense and
refine as needed
– Does the score make sense?
– Did we miss some factors?
– Does the weighting need further refinement?
Operational Excellence
15
Conclusions
 Statistical tools are not appropriate for low
sample sizes
 Implement a qualitative decision making tool
that leverages Stage 1 information
– Follow systematic approach to develop tool
– Test tool to look for gaps before implementation
– Evaluate documentation for ease of use
 Overall size of PPQ campaign also needs to
consider comparability and stability
requirements
Operational Excellence
16
References
 Guidance for Industry Process Validation: General
Principles and Practices, January 2011
 Levy, P.: Determining and Justifying the Number of
Process Validation Batches: Making Initial Batch Release
Decision. Presented at ISPE: Lessons Learned from 483s
Process Validation Track, February 2012
 Field Demonstration Workshop on Performance-Based
Inspection of Vessels Entering the St. Lawrence Seaway
(Prioritizing Vessels for US Coast Guard Inspection); April
2000
Operational Excellence
17

ABSTRACT:

The new FDA Process Validation guidance recommends a lifecycle approach for process validation
by allowing the manufacturer to incorporate process and product understanding into the timing and
size of the validation campaign (referred to in the FDA document as Process Performance
Qualification or PPQ). This paper describes a qualitative decision making tool to help determine an
appropriate size of the PPQ. The tool is based on criteria from seven key focus areas that aid in the
understanding of process capability and product consistency. These seven areas are product
understanding, process understanding, facility information, raw materials, status of analytical
methods, regulatory considerations and level of documentation. The tool lists performance
characteristics for each focus area. The user then lists a binary response of “yes” or “no” for each
characteristic. The justification for the number of lots for a process validation campaign is then
determined based on the percentage of “no” responses.
Operational Excellence
18