Using the Gamma Distribution to Determine
Anti-drug antibody Screening Assay Cut Points
with Non-normal Data
for
Midwest Biopharmaceutical Statistics Workshop
May 24-26, 2010
Brian Schlain
Nonclinical Statistics
ANTI-DRUG ANTIBODY (ADA) TESTING
high volume?
yes
no
screening
assay
yes
confirmation
assay
positive?
yes
positive?
titration assay
output titer
NEUTRALIZING ANTIBODY ASSAY (NAB) TESTING
Anti-drug Antibodies (ADA) detected?
yes
yes
high volume?
no
titration assay/
output titer
screening assay
yes
positive?
Types of Immunogenicity Assays
Anti-Drug Antibody (ADA) Testing with Binding Assays
 Radioimmunoprecipitation Assay (RIPA)
 Enzyme-Linked Immunosorbent Assay (ELISA)
*Standard sandwich ELISA
*Bridging ELISA
*Electrochemiluminescence (ECL; IGEN)
 Optical Sensor-based
* Surface Plasmon Resonance (SPR; Biacore)
*Guided Mode Resonance Filter (BIND; BD Biosci)
Neutralizing Antibody (NAB) Testing with Blocking Assays
Related to drug mechanism of action
Converted PK assays
Cell based: Some of the reagents genetically engineered
and stored as cell lines
Example ADA Assay Positive Control
Response Curve
Example NAB Assay Positive Control
Response Curve
8×12 Example Screening/Confirmation Assay Plate Layout
1
2
3
4
5
6
7
8
9
10
11
12
A
1S
1C
5S
5C
NC
LPC
9S
9C
13S
13C
17S
17C
B
1S
1C
5S
5C
NC
LPC
9S
9C
13S
13C
17S
17C
C
2S
2C
6S
6C
NC
LPC
10S
10C
14S
14C
18S
18C
D
2S
2C
6S
6C
NC
HPC
10S
10C
14S
14C
18S
18C
E
3S
3C
7S
7C
NC
HPC
11S
11C
15S
15C
19S
19C
F
3S
3C
7C
7C
NC
HPC
11S
11C
15S
15C
19S
19C
G
4S
4C
8S
8C
NC
HPC
-C
12S
12C
16S
16C
20S
20C
H
4S
4C
8S
8C
NC
HPC
-C
12S
12C
16S
16C
20S
20C
Screening Assay Cut Points
Fixed: mean and sd do not change across plates.
– CP=meanX ± a×sdX
– X=Sample assay signal response
Floating: mean changes, but sd remains constant
across plates.
– Multiplicative: CF=meanY ± a×sdY
Y=X/NC
CP=CF×NC
– Additive: CF=meanZ ± a×sdZ
Z=X – NC
CP=CF + NC
Dynamic: mean and sd both change with every plate.
Screening Multiplicative Cut Point CF Determination
• Screen data for outliers
• Normalize:Y=
Sample/NC
non-normal
Model distribution of
normalized values
normal or lognormal
• Gamma
• Weibull
Estimate CF
• Nonparametric
percentile
Validation Phase
CP=CF×NC
ADA Screening Assay Cut Point CF Estimation
under Normal Theory
CUT POINT RULE
sample>cut point, judge screening positive.
cut point=CF avg(NC).
NC: pooled negative sera control
 NORMAL THEORY CORRECTION FACTOR (CF) ESTIMATION
(Guttman I, Statistical Tolerance Regions: Classical and Bayesian, 1970)
n negative sera samples.
R (or normalized value)=Sample/Avg(NC).
mean (MR) and sd (SR) of R.
CF=MR + tdf,α SR×(1 + 1/n)0.5 no transformation .
CF=exp{MR + tdf,α SR×(1 + 1/n) 0.5 } log transformation.
df=n – 1.
α=targeted false positive rate=0.05.
NAB Screening Assay Cut Point CF Estimation
under Normal Theory
CUT POINT RULE
sample < cut point, judge screening positive.
cut point=CF avg(NC).
NC: pooled negative sera control
 NORMAL THEORY CORRECTION FACTOR (CF) ESTIMATION
(Guttman I, Statistical Tolerance Regions: Classical and Bayesian, 1970)
n negative sera samples.
R (or normalized value)=Sample/Avg(NC).
mean (MR) and sd (SR) of R.
CF=MR - tdf,α SR×(1 + 1/n)0.5 no transformation .
CF=exp{MR - tdf,α SR×(1 + 1/n) 0.5 } log transformation.
df=n – 1.
α=targeted false positive rate=0.05.
2-Parameter Gamma Density
Functions
Gamma approaches normal as k increases.
2-parameter Gamma
3-parameter Gamma
f ( x | k , , s ) 
( x  s)
k 1
exp[( s  x ) /  ]
 ( k )
k
x  s (threshold parameter)
Some common gamma fitting methods:
maximum likelihood estimation (MLE)
-SAS UNIVARIATE, R OPTIM.
-becomes unstable when k approaches 1
method of moments (ME)
modified moment estimation (MME) (Cohen and Whitten)
Maximum Likelihood Estimation (MLE) of 2parameter Gamma
Unstable when estimate of k close to 1.
ADA distributions tend to be unimodal with k>1.
NAB distributions tend to be unimodal with k<1.
-Fit gamma to 1/R=Avg(NC)/Sample.
Estimation of Gamma Parameters
by Method of Moments (ME)
Gamma central
moment
Empirical central
moment
Mean
S + θk
=m1 (mean)
Variance
θ2k
=m2
(variance)
Third standard
moment
3 
2
k
=m3 /(m2 )3/2
(std.
skewness)
ME for 3-parameter Gamma
Gamma
parameter
Moment estimator
k (shape)
=4m23 /m3
θ (scale)
=m3 / (2m2 )
2
s (threshold) =m1 - (2m22 )/m3
Modified Moment Estimators (MME) for 3-parameter
Gamma (using SAS PROBGAM function)
Cohen AC, Whitten BJ, Modified moment estimation for the 3-parameter
Gamma distribution, J. of Qual. Tech., 18, 1, 1986, 53-62
G ( X 1 | sˆ , ˆ , kˆ ) 
1
n 1

G (W 1 | s   2 / ˆ 3 ;   ˆ 3 / 2 ; k  4 / ˆ 3 )  PROBGAM
2
U 1  (W 1  sˆ ) / ˆ  (W 1  2 / ˆ 3 ) /( ˆ 3 / 2 )
W 
X  mean
sd
W 1  W 2  ...  W n
ˆ  sd  ˆ 3 / 2
sˆ  mean  2  sd / ˆ 3
2
kˆ  4 / ˆ 3
(U 1 , 4 / ˆ 3 ) 
2
1
n 1
Calculating the Cut Point Correction Factor as a
Gamma Percentile
Cut point correction factor (CF) calculated using SAS GAMINV
function:
CF  ˆ  GAMINV ( P , kˆ )  sˆ
P=.95 if false positive rate targeted at 5% (ADA assay)
-NAB assay: Model reciprocal of normalized values so
that skewness > 0.
θ=scale parameter
s=threshold parameter
k=shape parameter
CP=CF×NC
3-parameter Gamma Estimation
MLE preferred to MME or ME.
– SAS UNIVARIATE
– R OPTIM
MME generally better than ME.
MLE unstable for k near 1. (Johnson and Kotz recommend
k>2.5).
MME can be calculated for any k.
MME comparable to MLE with increasing n or α3 (Cohen and Whitten).
For  3 
2
k
< 0.10, consider normal distribution (Cohen and Whitten).
Standard Gamma Simulations Comparing MLE and
Nonparametric Percentile Cut Point Estimators
(targeted false positive rate=5%)
Shape n
Param.
2
30
60
120
240
4
30
60
120
240
8
30
60
120
240
16
30
60
120
240
MLE
se
5.085
4.827
4.794
4.786
8.010
7.856
7.834
7.829
13.458
13.314
13.289
12.278
23.564
23.390
23.363
23.355
1.029
.557
.379
.264
1.206
.695
.487
.335
1.469
. 888
.610
.427
1.862
1.128
.780
.545
FP
rate
.050
.051
.050
.049
.054
.051
.050
.049
.053
.050
.049
.048
.051
.048
.047
.046
se
.037
.023
.016
.011
.036
.023
.016
.011
.035
.023
.015
.011
.033
.022
.015
.010
Nonpar.
Est.
4.719
4.744
4.739
4.743
7.701
7.750
7.754
7.752
13.085
13.153
13.141
13.147
23.005
23.089
23.106
23.105
se
.924
.648
.471
.337
1.161
.799
.582
.415
1.482
1.043
.735
.532
1.920
1.308
.958
.684
FP
rate
.065
.057
.054
.052
.065
.057
.053
.052
.065
.056
.053
.052
.065
.056
.053
.051
se
.043
.028
.020
.014
.044
.028
.020
.014
.044
.029
.020
.014
.044
.028
.020
.014
Abbreviations: n=sample size; η=gamma shape parameter; MLE=maximum likelihood
estimator; se=standard error; FP rate=false positive rate; Nonpar. Est.=nonparametric
estimator.
Standard Gamma Simulations Comparing MLE with
Normal Based Cut Point Estimators
(targeted false positive rate=5%)
Shape n
Param.
2
4
8
16
30
60
120
240
30
60
120
240
30
60
120
240
30
60
120
240
MLE
se
FP
rate
se
5.085
4.827
4.794
4.786
8.010
7.856
7.834
7.829
13.458
13.314
13.289
12.278
23.564
23.390
23.363
23.355
1.029
.557
.379
.264
1.206
.695
.487
.335
1.469
.888
.610
.427
1.862
1.128
.780
.545
.050
.051
.050
.049
.054
.051
.050
.049
.053
.050
.049
.048
.051
.048
.047
.046
.037
.023
.016
.011
.036
.023
.016
.011
.035
.023
.015
.011
.033
.022
.015
.010
Normal
zmethod
4.272
4.301
4.306
4.322
7.234
7.259
7.273
7.283
12.604
12.628
12.641
12.650
22.525
22.547
22.569
22.584
se
FP
rate
se
.640
.466
.332
.237
.808
.572
.412
.286
1.044
.734
.517
.367
1.357
.948
.677
.477
.083
.077
.074
.072
.079
.073
.071
.069
.074
.069
.067
.066
.070
.066
.064
.062
.041
.028
.020
.014
.039
.026
.019
.013
.038
.025
.017
.012
.036
.024
.017
.012
Normal
predict.
interval
4.385
4.357
4.334
4.322
7.395
7.338
7.312
7.302
12.834
12.740
12.697
12.677
22.851
22.706
22.648
22.623
se
FP
rate
se
.662
.473
.335
.237
.834
.581
.415
.287
1.076
.745
.521
.369
1.397
.961
.681
.478
.076
.073
.072
.072
.072
.070
.069
.068
.067
.066
.065
.065
.062
.062
.062
.061
.039
.027
.019
.014
.037
.026
.018
.013
.035
.025
.017
.012
.033
.023
.016
.012
Abbreviations: n=sample size; η=gamma shape parameter; MLE=maximum likelihood
estimator; se=standard error; FP rate=false positive rate; normal z-method=
mean +1.645×sd; Normal Predict. Interval=upper limit of a 1-sided prediction interval based
on normal theory.
Standard Gamma Distribution Simulations Comparing
MLE with Log-normal Cut Point Estimators
(targeted false positive rate=5%)
Histogram of ADA ELISA Trial Pre-dose
Screening Assay Normalized Values (n=175)
outlying
values
Sample
#29
Sample
#34
MME Gamma Q-Q Plot (with outlying
sample 34 excluded)
outlying value
Sample # 29
MME Gamma Q-Q Plot
(with outlying samples 29 and 34 excluded)
Histogram of ADA Elisa Trial Pre-dose Normalized Values
(with outlying samples 29 and 34 excluded)
skewed to the
right
SAS UNIVARIATE EDF Goodness of Fit Test p-values
(outlying samples 29 and 34 excluded)
test
gamma
normal
log-normal
Shapiro-Wilk
-
<.0001
.031
KolmogorovSmirnov
.146
.010
.025
Cramer-von
Mises
.089
.0050
.073
AndersonDarling
.151
.0050
.060
Gamma Parameter and CF Estimates
(outlying samples 29 and 34 excluded)
MME
MLE
mean (of norm. values)
1.270
sd (of norm. values)
0.170
skewness (of norm. values)
0.902
W1 (smallest stand. value)
-2.102
n
173
α3
.474
.727
θ (scale)
.040
.061
s (threshold)
.553
.809
k (shape)
17.796
7.578
CF (upper 3.9% percentile of
gamma)
1.60
1.60
CP=CF×NC
target upper gamma percentile=
5% - 100%×(2/175)=3.9%
Gamma Distribution CF Determination
Target false positive (fp) rate=5%
Estimated percentage of outlying samples is 1.1% (=100%×2/175)
Target fp rate – percentage of outlying samples =5% - 1.1%=3.9%
CF=upper 3.9% percentile of fitted gamma=1.60
– CF=θ×GAMINV(p,k) + s=1.60.
p=1- 0.039=.961; k= 17.796; θ=.040; s=.553.
CP=CF×NC
Empirical fp rate=5.7% (=100%×10/175)
References
Schlain B et al., A novel gamma-fitting statistical method for anti-drug
antibody assays to establish assay cut points for data with non-normal
distribution, JIM, V. 352, Issues 1-2, 31Jan. 2010, pp. 161-168.
Guttman I, Statistical Tolerance Regions: Classical and Bayesian,
1970).
Cohen AC, Whitten BJ, Modified moment estimation for the 3parameter Gamma distribution, J. of Qual. Tech., 18, 1, 1986, 53-62.
Cohen AC, Whitten BJ, Modified moment and maximum likelihood
estimators for parameters of the three-parameter gamma distribution,
commun. Statist.-Simula. Computa., 11(2), 197-216 (1982).
Bowman KO and Shenton LR, Properties of Estimators for the Gamma
Distribution, Marcel Dekker, 1988.
Johnson NL and Kotz S, continuous Univariate Distributions, Vol. 1,
Houghton Mifflin company, 1970.
Krishnamoorthy K, Mathew T, and Mukherjee S, Normal-based
methods for a gamma distribution: prediction and tolerance intervals
and stress-strength reliability, Technometrics, Vol. 50, No. 1, pp. 6978.
BACKUP SLIDES
Standard Gamma Simulations Comparing MLE,
Nonparametric, and WH Cut Point Estimators
(targeted false positive rate=5%)
Shape n
Param.
MLE
se
FP
rate
se
Nonpar.
Est.
se
FP
rate
se
2
5.085
4.827
4.794
4.786
8.010
7.856
7.834
7.829
13.458
13.314
13.289
12.278
23.564
23.390
23.363
23.355
1.029
.557
.379
.264
1.206
.695
.487
.335
1.469
. 888
.610
.427
1.862
1.128
.780
.545
.050
.051
.050
.049
.054
.051
.050
.049
.053
.050
.049
.048
.051
.048
.047
.046
.037
.023
.016
.011
.036
.023
.016
.011
.035
.023
.015
.011
.033
.022
.015
.010
4.719
4.744
4.739
4.743
7.701
7.750
7.754
7.752
13.085
13.153
13.141
13.147
23.005
23.089
23.106
23.105
.924
.648
.471
.337
1.161
.799
.582
.415
1.482
1.043
.735
.532
1.920
1.308
.958
.684
.065
.057
.054
.052
.065
.057
.053
.052
.065
.056
.053
.052
.065
.056
.053
.051
.043
.028
.020
.014
.044
.028
.020
.014
.044
.029
.020
.014
.044
.028
.020
.014
4
8
16
30
60
120
240
30
60
120
240
30
60
120
240
30
60
120
240
WH
(cube
root)
4.910
4.803
4.746
4.730
7.975
7.851
7.789
7.765
13.447
13.291
13.214
13.180
23.487
23.274
23.190
23.152
se
FP
rate
se
.738
.506
.353
.249
.918
.632
.443
.306
1.174
.802
.560
.393
1.503
1.024
.723
.505
.051
.052
.052
.052
.051
.051
.051
.051
.050
.050
.050
.050
.050
.050
.050
.050
.031
.021
.015
.011
.030
.021
.015
.010
.030
.021
.015
.010
.030
.021
.015
.010
Abbreviations: n=sample size; η=gamma shape parameter; MLE=maximum likelihood estimator;
se=standard error; FP rate=false positive rate; Nonpar. Est.=nonparametric estimator;
WH=Wilson-Hilferty estimator.
Need for further research on WH
How well does it perform when the
real distribution is not quite a
gamma, but the gamma is the best
approximation that can be found?