Regulatory Challenges in
Bioassay Practices
Tim Schofield
Managing Director & Head of Non-Clinical Services
Arlenda, Inc.
tim.schofield.arlenda.com
Presented at the 35th Annual Midwest
Biopharmaceutical Statistics
May 22, 2012, Ball State University, Muncie, Indiana
Outline
 What is a bioassay?
 Bioassay guidelines
 Regional differences in bioassay practices
 US viewpoint on bioassay practices
 QbD for analytical methods
What is a bioassay?
ICH definition and requirements
 Bioassay (Biological Assay) – ICH Q6B (paraphrased)
 Definition: The measure of the biological activity using a suitably
quantitative biological assay (also called potency assay or bioassay),
based on the attribute of the product which is linked to the relevant
biological properties.
 A valid biological assay to measure activity should be provided by the
manufacturer. Examples of procedures used to measure biological
activity include:
• Animal-based biological assays, which measure an organism's
biological response to the product
• Cell culture-based biological assays, which measure biochemical or
physiological response at the cellular level
• Biochemical assays, which measure biological activities such as
enzymatic reaction rates or biological responses induced by
immunological interactions
What is a bioassay?
FDA requirements
 Potency definition
 Specific ability or capacity of the product, as indicated by
appropriate laboratory tests or by adequately controlled clinical
data obtained through the administration of the product in the
manner intended, to effect a given result. [21 CFR §600.3 (s)]
 Tests for potency
 Tests for potency shall consist of either in vitro or in vivo tests, or
both, which have been specifically designed for each product so
as to indicate its potency in a manner adequate to satisfy the
interpretation of potency given by the definition in § 600.3 (s) of
this chapter. [21 CFR § 610.10]
What is a bioassay?
Distinguishing properties
 Measurement of activity rather than mass
 Specific to the mechanism of action
 Usually reported as relative potency to a reference standard
 Highly variable
 10% to 50% RSD (versus 1% to 2% RSD for chromatographic assays)
 Biology rather than chemistry
 Resource intensive
 Time as well as materials
 Usually performed on samples in a complex matrix
 Therapeutic proteins purified from cell culture
 Vaccines produced in living systems
Bioassay guidelines
 Originally USP <111> and EP 5.3
 <111> was split into two chapters, USP <1032> Design and
Development of Biological Assays and USP <1034>
Analysis of Biological Assays
 <1033> Biological Assay Validation added to the suite
“Roadmap” chapter
(to include glossary)
6
Bioassay guidelines (cont.)
 All but chapter <111> are above 1000 and therefore
“informational”
 Not intended as enforceable (as chapters below 1000)
 However, the chapters provide a set of guiding principles which
might be considered by regulators in their reviews
 Chapter <111> is left to support monographs which
reference it
 USP is working towards addressing product-specific references
to prepare it for further revision
7
Regional differences in bioassay practices
 Assessing “linearity” and similarity
 “Linearity” is the goodness-of-fit to the processing model
 Similarity is the equivalence of the bioassay model parameters
•
Parallelism in parallel line analysis
•
Equivalence of asymptotes and Hill coefficient in 4 parameter logistic
regression
•
Equivalence of intercepts in slope ratio analysis
 Parallel line/curve versus slope ratio analysis
 For some vaccines the US requires parallel line analysis while the EU requires
slope ratio analysis
 Design and performance characteristics are different for the two approaches
 Clinical specifications versus process consistency
 Growing expectation in the US that potency specifications should be supported by
clinical studies
Regional differences in bioassay practices
Assessing “linearity” and similarity
 Significance testing versus equivalence testing
Laboratory B
0.8
p = 0.08 (p > 0.05, i.e., not
significantly different)
Standard Data
Test Data
0.4
Standard Line
Conclude parallel!
Rewarded for poor assay
performance
Log10 Response
Test Line
0
0.5
1
1.5
2
2.5
-0.4
-0.8
-1.2
Log10 Concentration
Laboratory A
0.8
Standard Data
Test Data
Conclude nonparallel!
Penalized for good assay
performance
Standard Line
0.4
Log10 Response
p = 0.02 (p < 0.05, i.e.,
significantly different)
Test Line
0
0.5
1
1.5
-0.4
-0.8
-1.2
Log10 Concentration
2
2.5
Regional differences in bioassay practices
Assessing “linearity” and similarity (cont.)
 Paradox: significance tests
reward poor work and penalize
good work
 The greater the precision in the
data, the more likely you will fail
the significance test
a:
no evidence of a difference
in slopes; however, possibly outside
acceptance limit (not similar)
no evidence of a difference in slopes;
but inside acceptance limit (similar)
a difference in slopes; however,
within acceptance limit (similar)
b:
c:
 Solution – use an equivalence
test
 Determine an acceptable range in
a metric related to “linearity” or
similarity (LAL,UAL)
 Demonstrate (TOST) that there’s
acceptable similarity
• CI includes 0 = no evidence of a
difference
• CI within (LAL,UAL) = similar
UAL
0
N=4
N=6
N=8
a
b
c
LAL
Note: Rewarded for good work
10
Regional differences in bioassay practices
Assessing “linearity” and similarity (cont.)
 Hurdles to an equivalence approach
 European regulation adheres to EP 5.3
 The revision of EP 5.3 does not allow for other approaches
 Some points of view
• “Statistical significance is scientifically important”
• The significance approach can be moderated by an examination of
the results
•
Use of historical variability
•
Moderation of significance level
•
May lead to subjectivity
• “Calibration” to the equivalence approach
•
Engineer significance approach to duplicate equivalence approach
•
More straight forward to apply equivalence approach
 Establishment of an equivalence margin
11
Regional differences in bioassay practices
Assessing “linearity” and similarity (cont.)
 Approaches for assigning an equivalence margin – operational
approaches
 Approach 1 – based on “process capability” of the bioassay
• Addresses only the producer’s risk
• There is no penalty for a poorly designed bioassay
 Approach 2 – based on “process capability” using a tolerance
interval on the confidence interval
• Penalizes poor assay runs
Approach 2
Difference of Slopes
Difference of Slopes
Approach 1
-
-
12
Regional differences in bioassay practices
Assessing “linearity” and similarity (cont.)
 Approaches for assigning an equivalence margin –
quality approaches
 Approach 3 – based on discrimination between “good” and “bad”
behavior
• Protects both producer’s and consumer’s risk
• However, how do you define/generate “bad” behavior
• Some sensitivity to clinical correspondence to “bad” behavior
 Approach 4 – ad hoc limits
• Based on product or assay knowledge
• Should consider potential impact to product quality
13
Regional differences in bioassay practices
Bioassay design and data processing (cont.)
►
There are currently regional differences in the accepted processing of
some vaccines (flu)
–
►
EU requires slope ratio while North America requires parallel line analysis
In slope ratio analysis
variability of RP is impacted by
increased variability of
influential regression points
 Slope is influenced by
extreme points
►
Simulation results
 True relative potency equal to
2.00
 Impact of increase in assay
%RSD
 n=100 simulated assays per
condition
Slope Ratio Analysis
14
12
10
8
Standard
6
Test
4
2
0
0
Assay
%RSD
1%
10%
20%
1
2
3
4
5
Estimated RP (%RSD)
Slope Ratio
Parallel Line
1.99
(4%)
1.99
(2%)
2.06
(45%)
2.02
(19%)
1.95
(159%)
2.21
(48%)
• Little impact on relative potency determination with
increase in assay %RSD
• Dramatic increase in RP %RSD for slope ratio assay
• Assay design should be adapted to parallel line
approach (geometric doses)
US viewpoint on bioassay practices
Validation of assay format
 Some US regulators believe that bioassay validation should be a
verification of the procedure for obtaining a “reportable value”
 Groupings in time have different variability characteristics than the sum
of the variance components
 Emphasis on product release ignores other uses
 Some merit to this if the validation is not designed to address the issue
of short term versus long term variability
• Replicate the bioassay under the same set of ruggedness conditions
4.2
Regression
4
3.8
3.6
3.4
3.2
3
2.8
0
3
6
9
12
15
Time (Month)
18
21
24
Managing variability
Beyond random and systematic variability (cont.)
 Retest rules have the potential to lead to
truncation bias in the reportable value
•
e.g., retest when a measurement is
outside the “quantifiable range” of the
bioassay
Dilutional Linearity
200
Response
 Risk of truncation error and
range
150
100
50
0
1
 Potential solutions
• Assign a value to the low/high result
(e.g., ½ the LLOQ in clinical assays)
• Demonstrate a range which
supports low/high potency samples
(without retest)
• Retest the series using an adjusted
dilution scheme
10
100
1000
Concentration
Lot 1
Lot 2
Test
Lot 3
Avg
Retest
Lot 4
Lot 5
16
US viewpoint on bioassay practices
Bioassay characterization
 USP allows for “Use of validation
results for bioassay
characterization”
 Use of variance components to
adapt bioassay format
• Numbers of runs (assays) and
replicates to efficiently manage
bioassay variability
• Identify potentially significant
sources of variability
•
Update technique or training
•
Replicate over significant
factors
 ˆ 2

ˆ 2


Run  Replicate

k
n

k
Format Variability  100   e
 1 %




Format variability for different combinations of number of
runs (k) and number of minimal sets within run (n)
Number of Runs (k)
Reps (n)
1
2
3
6
1
7.2%
5.1%
4.1%
2.9%
2
6.4%
4.5%
3.6%
2.6%
3
6.0%
4.2%
3.4%
2.4%
6
5.7%
4.0%
3.3%
2.3%
Variance
Component
Estimate
Var(Media Lot)
0.0000
Var(Analyst)
0.0014
Var(Analyst*Media LOt)
0.0000
Var(Run(Analyst*Media Lot))
0.0019
Var(Error)
0.0022
QbD for analytical methods
 Industry has begun to recognize that analytical methods generate
a product – measurements
 Like pharmaceutical products, measurements should have
adequate quality to meet their intended use – decision making
 The fundamental goals of product development are:
 Safety and efficacy (hitting the clinical target)
 Variance reduction
 The fundamental goals of analytical development are:
 Accuracy (hitting the analytical target)
 Variance reduction
18
QbD for analytical methods (cont.)
 Many of the concepts associated with QbD for pharmaceutical
products translate to concepts related to analytical methods
Process Concept
Analytical Counterpart
Target Product Profile (TPP)
• Target clinical performance, manufacturing,
and commercial requirements
Critical Quality Attributes(CQAs)
• Potency
• Aggregation
• Purity
Specifications (acceptance criteria)
• 80% to 125% potency
• Purity > 95%
Analytical Target profile (ATP)
• Target analytical performance, testing
laboratory, and customer requirements
Performance attributes (validation parameters)
• Precision
• Sensitivity
• Accuracy
Acceptance criteria
• %GCV < 10%
• LLOQ > 1 ng/mL
Critical process parameters
• pH, time, temperature
Process control strategy
• Comparability protocols
• Tech transfer
Continuous verification
• Continuous review and updating of process
knowledge
Critical assay parameters
• pH, time, temperature
Assay control strategy
• Comparability protocols
• Method transfer
Continuous verification
• Continuous review and updating of analytical
knowledge
19
QbD for analytical methods (cont.)
 The bioassay should be fit for its intended use throughout the
bioassay lifecycle
 Should perform adequately to support decisions
 Decisions are made day-to-day using bioassays
 During development
• Which formulation provides the best stability?
• Does a particular process step impact potency?
• What is the self-life of the product?
 During manufacture
• Should a manufactured lot be released to the market?
• Should a process change be implemented?
• Has a manufactured lot maintained potency over it shelf-life?
• Can a new potency standard be used in the bioassay?
20
QbD for analytical methods (cont.)
 All decision are made with risk, and risk is costly
 Risks during development
• Risk of deciding “the process” is suboptimal when it is fit– producers risk
– results in excessive development costs or program failure
• Risk of deciding the process is fit when “the process” is flawed –
consumers risk – results in excess downstream costs to fix the problems
• Risks during manufacture
• Risk of failing satisfactory product –regulatory burden and lost revenues
• Risk of passing unsatisfactory product – potential risk to the “customer”
 Decision risk can be managed in several ways
 Use sound scientific reasoning and/or experience to guide decisions
 When decisions are made on the basis of empirical evidence, decision risk
is associated with the strength of the evidence
21
QbD for analytical methods (cont.)
 Statistical opportunities – supporting evidence based
development and control
 Statistical thinking
• Understanding variability
• Managing variability
• Communicating uncertainty
 Bioassay models and analyses
 Bioassay optimization
 Bioassay maintenance
• Statistical process control
• Standard qualification/calibration
• Method transfer
• Method comparison – in vivo to in vitro
22
Summary
 Bioassays are utilized throughout the biological product lifecycle
to make key development and quality decisions
 Statistical approaches facilitate decision making and help mitigate
the risks of bioassay variability
 Industry and regulatory statisticians should work together to
support bioassay development, and should help promote best
practices in implementation