A Bayesian Approach to
Parallelism Testing in Bioassay
Steven Novick, GlaxoSmithKline
Harry Yang, MedImmune LLC
Manuscript co-author
John Peterson, Director of statistics, GlaxoSmithKline
Warm up exercise
When are two lines parallel?
Assay Response
• Parallel: Being
everywhere equidistant
and not intersecting
Standard
Test
log10 Concentration
• Slope
• Horizontal shift places
one line on top of the
other.
When are two curves parallel?
Assay Response
• Parallel: Being
everywhere equidistant
and not intersecting?
Standard
Test
log10 Concentration
• Can you tell by checking
model parameters?
• Horizontal shift places
one curve on top of the
other.
Where is parallelism important?
• Gottschalk and Dunn (2005)
– Determine if biological response(s) to two substances are similar
– Determine if two different biological environments will give similar
dose–response curves to the same substance.
• Compound screening
• Assay development / optimization
• Bioassay standard curve
80
20
0
• Desire for dose-response
curves to be parallel.
40
60
• E.g., seeking new HIV
compound with AZT-like
efficacy, but different
viral-mutation profile.
100
Screening for compound similar to
“gold-standard”
1.0
1.5
2.0
2.5
log10 - Inhibitor (
3.0
M)
3.5
4.0
Change to assay procedure
• E.g., change from fresh to frozen cells.
• Want to provide same assay signal window.
– Desire for control curves to be parallel.
Fresh
Frozen
Day 0
Day 5
Day 10
Assess validity of bioassay
used for relative potency
• Dilution must be parallel
to original.
Assay Response
– Callahan and Sajjadi (2003)
Original
Dilution
Lot 1
Lot 2
log10 Concentration
Replacing biological materials used
in standard curve
• E.g., ELISA (enzyme-linked immunosorbent)
assay to measure protein expression.
– Recombinant proteins used to make standard
curve.
• Testing clinical sample
• New lot of recombinant proteins for
standards.
– Check curves are parallel
– Calibrate new curve to match the old curve.
• Potency often determined relative to a
reference standard such as ratio of EC50
– Only meaningful if test sample behaves as a
dilution or concentration of reference standard
• Testing parallelism is required by revised USP
Chapter <111> and European Pharmacopeia
Linear: Two lots of Protein “A”
3.5
Estimated Concentrations
Lot 2 is 1.4-fold higher than Lot 1
Lot 2
3.0
Lot 1
2.5
log 10 Signal
Log10 Signal
4.0
Linear Standard Curve: Protein A lots 1 and 2
0.5
1.0
1.5
=0.14
2.0
log10 Concentration
2.5
If the lines are parallel…
Assay Response
• Shift “Lot 2” line to the left
by a calibration constant .
•  is log relative potency of
Lot 2.
Lot 1
Lot 2
Draft USP <1034> 2010
log10 Concentration
Testing for Parallelism in bioassay
Typical experimental design
4.0
Serial dilutions of each lot
3.5
Several replicates
Fit on single plate (no plate effect)
Lot 2
3.0
Lot 1
2.5
log 10 SignalSignal
Log
10
Linear Standard Curve: Protein A lots 1 and 2
0.5
1.0
1.5
2.0
log10 Concentration
2.5
Tests for parallel curves
Linear model
f θ1 , x   a1  b1 x
•
f θ2 , x  a2  b2 x
Hauck et al, 2005; Gottschalk and Dunn, 2005
H0: b1 = b2
H1: b1 ≠ b2
ANOVA:
T-test
F goodness of fit test
2 goodness of fit test
May lack power with small sample size
Might be too powerful for large sample size
A better idea
• Callahan and Sajjadi 2003; Hauck et al. 2005
– Slopes are equivalent
• H0: | b1 − b2 |≥ 
• H1: | b1 − b2 |< 
Nonlinear: Two lots of protein “B”
3.5
Estimated Concentrations
Lot 2 is 1.6-fold higher than Lot 1
3.0
Lot 1
Lot 2
2.5
log 10 Signal
4.0
4.5
FPL Standard Curve: Protein B lots 1 and 2
2.5
3.0
3.5
4.0
=0.21
4.5
log10 Concentration
5.0
Tests for parallel curves
4-parameter logistic (FPL) model
f θ1 , x   A1 
B1  A1
1  expD1 log(x)  D1C1
f θ 2 , x   A2 
B2  A2
1  expD2 log(x)  D2C2 
• Jonkman and Sidik 2009
– F-test goodness of fit statistic
• H0: A1 = A2 and B1 = B2 and D1 = D2
• H1: At least one parameter not equal
May lack power with small sample size
Might be too powerful for large sample size
• Calahan and Sajjadi 2003; Hauck et al. 2005;
Jonkman and Sidik (2009)
– Equivalence test for each parameter = intersection-union
test
• H0: |1 / 2|  1 or |1 / 2|  2
• H1: 1 <|1 / 2| < 2
i = Ai, Bi, Di , i=1,2
1. Equiv. of params does not provide assurance of parallelism (except for linear)
2. May lack power with small sample size
3. Forces a hyper-rectangular acceptance region
Our proposal
“Parallel Equivalence”
Definition of Parallel
• Two curves f θ1, x and f θ2 , x are parallel if
there exists a real number ρ such that
f θ1 , x  f θ2 , x   
for all x.
4-parameter Logistic Curves
Assay Response
Assay Response
Straight Lines
log 10 Concentration
log 10 Concentration
Definition of Parallel Equivalence
• Two curves f θ1, x and f θ2 , x are parallel
equivalent if there exists a real number ρ such
that
f θ1, x  f θ2 , x      for all x  [xL, xU].
• It follows that two curves are parallel
equivalent if there exists a real number ρ such
that
f θ , x  f θ , x     
max
xL  x xU
1
2
• It also follows that
f θ , x  f θ , x     
max
min

xL  x xU
1
2
Are these two lines parallel enough when xL < x < xU ?
Assay Response
Assay Response
< ?
xL
xU
log10 Concentration
xL
xU
log10 Concentration
Linear-model solution
f θ1 , x   a1  b1 x
• Linear model:
f θ2 , x  a2  b2 x
f θ , x   f θ , x     
max
min

x L  x  xU
 min

1
2
f θ , x   f θ , x     
max


x x L , xU
1
2

 xL  xU
  b1  b2 
 2

Just check the endpoints


  a1  a2  b2


Parallel equivalence = slope equivalence
a
max
min

x L  x  xU
1
 b1 x  a2  b2 x  b2    
 min a1  b1 xU  a2  b2 xU  b2    
wlog


 xL  xU
  b1  b2 
 2



  a1  a2  b2



 xL  xU
a1  b1 xU  a2  b2 xU  b2 b1  b2 
 2

 xU  xL 
 b1  b2 

 2 
Same as testing: | b1 − b2 |< 


  a1  a2  b2  


Parallel Equivalence
• FPL model:
f θ1 , x   A1 
B1  A1
1  expD1 log(x)  D1C1
f θ 2 , x   A2 
B2  A2
1  expD2 log(x)  D2C2 
f θ , x  f θ , x     
max
min

xL  x xU
1
2
• No closed-form solution.
– Simple two-dimensional minimax procedure.
Assay Response
Assay Response
Are these two curves parallel enough when xL < x < xU ?
< ?
xL
xU
log10 Concentration
xL
xU
log10 Concentration
Testing for parallel equivalence
H0:
f θ , x  f θ , x     
max
min

H1:
f θ , x  f θ , x     
max
min

xL  x xU
xL  x xU
1
1
2
2
• Proposed metric (Bayesian posterior
probability):

p  Prmin
 
max
xL  x  xU

f θ1 , x   f θ 2 , x      | data

Computing the Bayesian posterior probability
• For each curve, assume
Data distribution:

Yij θ i , x j ~ N f θ i , x j ,  2

i =1, 2 = reference or sample
j = 1, 2, …, N = observations
Prior distribution:

θi ~ N θi(0) , V(0)

 2 ~ IGsh, sc 
Posterior distribution proportional to:


 
L θ1, θ2 , 2 | data  θ1  θ2   2
• Draw a random sample of the i of size K from
the posterior distribution (e.g., using
WinBugs).
• The posterior probability

p  Prmin
 
max
xL  x  xU

f θ1 , x   f θ 2 , x      | data

is estimated by the proportion (out of K) that
the posterior distribution of:
f θ , x  f θ , x     
max
min

xL  x xU
1
2
3.5
lo g 10 Signal
3.5
Lot 1
Lot 2
2.5
3.0
Lot 2
3.0
Lot 1
2.5
lo g 10 Signal
4.0
Linear Standard Curve: After adjustment
4.0
Linear Standard Curve: Before Adjustment
0.5
1.0
1.5
2.0
log10 Concentration
2.5
0.5
1.0
1.5
2.0
log10 Concentration
= 0.14 (100.14=1.4-fold shift)
= 0.07  90% probability to call parallel equivalent
2.5
4.0
3.5
lo g 10 Signal
3.5
Lot 1
3.0
3.0
Lot 1
Lot 2
2.5
Lot 2
2.5
lo g 10 Signal
4.0
4.5
FPL Standard Curve: After Adjustment
4.5
FPL Standard Curve: Before Adjustment
2.5
3.0
3.5
4.0
log10 Concentration
4.5
5.0
2.5
3.0
3.5
4.0
log10 Concentration
= 0.33 (100.33=2.15-fold shift)
= 0.52  90% probability to call parallel equivalent
4.5
5.0
Simulation: FPL Model
Based on protein “B” data
3.5
3.0
Lot 1
Lot 2
2.5
log 10 Signal
4.0
4.5
FPL Standard Curve: Protein B lots 1 and 2
2.5
3.0
3.5
4.0
log10 Concentration
4.5
5.0
Simulation: FPL model
• Similar to Protein “B” protein-chip data.
• Concentrations (9-point curve + 0):
• 0, 102(=100), 102.5625, 103.125, …, 106.5(=3,200,000)
• Three replicates
• xL = 3.5=log10(3162) & xU = 5=log10(100,000)
• δ = 0.2.
•  = 0.02, 0.04, 0.11, and 0.21 (%CV = 5, 10, 25, 50)
• For each Monte Carlo run, I computed:

p  Prmin
 
max
xL  x  xU

f θ1 , x   f θ 2 , x      | data

Scenario
Original
New
Max
A1
B1
C1
D1
A2
B2
C2
D2
Diff
2
4.5
4
-1
2
4.5
4
-1
0
2
2
4.5
4.7
-1
0
3
2.445
4.945
4.7
-1
0.15
4
2
4.5
4.7
-1.312
0.15
5
2.61
5.11
4.7
-1
=0.20
6
2.25
4.75
4.7
-1.5
0.30
1
5,000 Monte Carlo Replicates
Example data (CV=10%)
Diff:
0
0
0.15
0.15
Original
0
Scenario 1
2
4
6
0.30
New
8
0
Scenario 2
0.20
Scenario 3
2
4
6
8
0
Scenario 4
Scenario 5
Assay Response
4
3
2
2
4
6
8
0
2
4
6
8
log10 Concentration
0
2
4
6
4
6
Scenario 6
5
0
2
8
8
50
5
Pr > 0.9
Scenario 5
Pr > 0.9
xL xU
p
0.0
5
10
25
50
5
% CV
10
50
Pr > 0.9
0.5
50
25
Scenario 6
1.0
25
10
25
50
0
25
Scenario 4
Pr > 0.9
0.
99
7
0.
49
98
1
1
10
0.
99
6
0.
49
8
1
Pr > 0.9
Scenario 3
10
2e
-0
4
8e
-0
4
Scenario 2
Pr > 0.9
5
5
0.30
0
50
0.20
0.
02
3
0.
00
1
2e
-0
4
25
1
Scenario 1
10
0.15
0.
98
98
0.
27
62
0.
00
44
5
0.15
0.
1
0
0.
99
94
0.
87
94
0.
36
22
0.
14
0
1
Diff:
Frequentist statistical power
Scenario
Max Diff
%CV=5
%CV=10
%CV=25
%CV=50
1
0
1.00
1.00
1.00
0.50
2
0
1.00
1.00
1.00
0.50
3
0.15 (shape 1)
1.00
0.99
0.28
< 0.01
4
0.15 (shape 2)
1.00
0.88
0.36
0.14
5
δ = 0.20
0.10
0.02
< 0.01
< 0.01
6
0.30
0.00
0.00
< 0.01
< 0.01
Summary
• Straight-forward and simple test method to
assess parallelism.
• Yields the log-relative potency factor.
• Easily extended.
Extensions
• Instead of f(, x), could use
– f(, x) /  x = instantaneous slope
– f-1(, y) = estimated concentrations
What’s next?
• Head-to-head comparison with existing methods
• Choosing test level and , possibly based on ROC
curve? – Harry Yang paper
• Guidance for prior distribution of 1 and 2.
References
1. Callahan, J. D. and Sajjadi, N. C. (2003), “Testing the Null
Hypothesis for a Specified Difference - The Right Way to Test
for Parallelism”, Bioprocessing Journal Mar/Apr 1-6.
2. Gottschalk P.J. and Dunn J.R. (2005), “Measuring Parallelism,
Linearity, and Relative Potency in Bioassay and Immunoassay
Data”, Journal of Biopharmaceutical Statistics, 15: 3, 437 463.
3. Hauck W.W., Capen R.C., Callahan J.D., Muth J.E.D., Hsu H.,
Lansky D., Sajjadi N.C., Seaver S.S., Singer R.R. and Weisman
D. (2005), “Assessing parallelism prior to determining relative
potency”, Journal of pharmaceutical science and technology,
59, 127-137.
4. Jonkman J and Sidik K (2009), “Equivalence Testing for
Parallelism in the Four- Parameter Logistic Model”, Journal of
Biopharmaceutical Statistics, 19: 5, 818 - 837.
Thank you!
Questions?