Priors, Utilities, Elicitation
& Pharmaceutical R&D
Andy Grieve
Statistical Research Centre
Pfizer Global R&D
Outline
• Use of Bayesian Methods in Pharmaceutical R&D
• Three Prior Elicitation Examples
• Acute toxicity – LD50
• Sample Sizing & Confidence Intervals
• Counting Tablets in Dosing Dogs
• Elicitation for Internal Company Decision Making
– Portfolio Management
2
Are Bayesian Methods Acceptable in
Drug Development?
Not Forbidden by Regulation
3
Extracts from Drug Regulations
= International Conference on
Harmonisation
E4 : Dose Response 1993
Agencies should be open to the use of various
statistical and pharmacometric techniques such as
Bayesian and population methods, modelling, and
pharmacokinetic-pharmacodynamic approaches.
4
Extracts from Drug Regulations
= European Medicines
Evaluation Agency
CPMP Biostatistical Guidelines 1994
.. the use of Bayesian or other approaches may be
considered when the reasons for their use are clear and
when the resulting conclusions are sufficiently robust to
alternative assumptions
5
Extracts from Drug Regulations
E9 : Statistical Principles in Clinical Trials
1998
Essentially same as CPMP Guidelines
.. the use of Bayesian or other approaches may be
considered when the reasons for their use are clear and
when the resulting conclusions are sufficiently robust
(to alternative assumptions)
LUKEWARM !!!
6
Why ?
- Trust
Clinical_Trials List (1996)
• Background
• Hair-thinning
• Researcher Bias
“If I hadn’t believed it, I wouldn’t have seen it
with my own eyes”
7
Trust
• Feeling that use of “subjective priors may allow
unscrupulous companies and/or statisticians to
attempt to pull the wool over the regulators
eyes.” (Greg Campbell – FDA Centre for
Devices & Radiological Health)
• If it were that easy they are not very good and
we probably need new regulators
8
Trust
• Stephen Senn - “nowhere is the discipline of statistics
conducted with greater discipline than in the
pharmaceutical industry”
• Nowhere will Bayesian statistics be conducted
with more discipline than in the pharmaceutical
industry
• Document
9
Trust
•
Document
• Where did the prior come from ?
• Is it based on data? Is it subjective ?
• “to present a Bayesian analysis in which the company’s own prior
beliefs are used to augment the trial data will in general not be
acceptable to a regulatory agency” (O’Hagan & Stevens, 2001)
• “the frequentist approach is less assumption dependent and can
provide the statistical strength of evidence required for a
confirmatory trial that may be lacking in a more assumption
dependent Bayesian approach” (Chi, Hung & O’Neill –
Biopharmaceutical Report, Vol. 9, No.2, 2001)
•
SENSITIVITY ANALYSIS
10
Trust
• We work in a Frequentist World
Remember Acceptance of Bayesian methods is Lukewarm
•
•
We will be asked about false positive rates
We will be asked about the impact of multiple looks at the
data
• We need to be calibrated
11
Assessing a Prior in Acute toxicity
12
Motivating Example
Dose
(mg/kg)
# of
Animals
# of
Deaths
500
1000
2500
5000
5
5
5
5
1
2
3
2
Based on these
data we wish to
determine the
LD50 to classify
the drug according
to the following classification scheme (Swiss Poison Regs.)
Toxicity Class
1
2
3
4
5
Range of
LD50 (mg/kg)
<5
5-50
50-500
500-2000
2000-5000
13
Model
• Data triplets
• { di , ni , ri } : i=1,..,k
• Probabilities of response
• pi : i=1,…,k
• Logistic Model
• log [ pi / (1-pi)] = a + blog(di)
Probit Model :
pi =F(a + blog(di))
• Median Lethal dose (LD50)
• log(LD50) = -a/b = m
14
Bayesian Solutions
• Likelihood Function
ni -ri
k
L(a, b | X)   G(a  b log( di )) ri 1 - G(a  b log( di )) 
i=1
• Prior distribution - p(a,b) (b > 0 )
• Define : m=-a/b
• Inference
- log(LD50)

p(m | X) =  bp(-mb, b | X)db
0
mU
P(mL  m  mU | X) =  p(m | X)dm
mL
15
Likelihood Contours - Motivating
Example
Dose
# of
# of
(mg/kg) Animals Deaths
500
5
1
1000
5
2
2500
5
3
5000
5
2
16
Likelihood Function - Hypothetical
Example
Dose
# of
# of
(mg/kg) Animals Deaths
100
3
1
1000
3
2
Normal Analytic
Approximation
17
Motivating Example
Dose
(mg/kg)
# of
Animals
# of
Deaths
500
1000
2500
5000
5
5
5
5
1
2
3
2
Experienced toxicologists
will know that they need
to span the LD50 with the
doses they choose.
The choice of doses contains information concerning the
toxicologist’s beliefs about the likely value of the LD50.
18
Choice of Prior
1) Tsutakawa (1975) : logit
• Choose doses d1 & d2
s.t. P(d1<LD50<d2)=0.5
1
p2 >p1
• Implies p1 and p2 uniform over the
half square
• p(a,b) :
logit
probit
 n.c.p.
 BN (truncated)
p2
(Grieve , 1988)
• Implies knowledge of pi :  i ≠1,2
0
0
p1
1
19
Choice of Prior
2) Tsutakawa (1975) - logit
• Choose doses d1 & d2
• Specify modal responses probabilities
• Assume n.c.p. for p1 and p2
p
p (1 - p 2 )
 li = 1  (mi - 2)pˆi
l1 -1
1
• p(a,b) :
(1 - p 1 )
pˆ1and pˆ 2
logit
probit
m1 - l1 -1
l 2 -1
2
m2 - l 2 -1
 n.c.p.
 BN (truncated)
(Grieve, 1988)
• Implies knowledge of pi :  i ≠1,2
20
Choice of Prior
3) Grieve (1988) - probit
Toxicity Class
1
2
3
4
5
Probability
0.05
0.15
0.40
0.30
0.05
• Suppose p(a,b) is bivariate normal :
  a 0    a2
BN   , 
  b 0    a b


 a b  

2
 b  
• Can the parameters be determined ?
• Not uniquely !!!
a0
b0
a
, c2 =
, c3 =
, c4 = 
• The c.d.f. of –a/b depends only on : c1 =
a
•
•
•
•
•
b
b
Implying any 4 probabilities are sufficient to determine c1,c2,c3 and c4
Any one of the 5 parameters is also needed
Modal slope ? How about median ?
Feedback
21
Determining a Prior for Sample Sizing
22
Sample Sizing CIs : Simon Day (Lancet, 1988)
2n patients :
(1-a)% CI
Width
x1 , x2 , s
2
2
: x1 - x2 - t s
n
:
,
2
x1 - x2 - t s
n
2
w = 2t s
n
Acceptable Width =
w0
n=
8t s
2
2
w0
23
Alternative Approach : Grieve - Lancet, 1989
P( w  w0 ) > 1 - 
Required :


2
= P 2t s
 w0  > 1 - 
n


2 
 s2
w0 n
= P 2  2 2  > 1 - 
8t  

Solve by search
2
24
Simon Day’s Example
Two Anti-Hypertensives
Difference in Diastolic BP - 95% CI
 = 10 mm Hg, w0=10
n=32
Grieve - Lancet , 1989
1-
n
0.5 0.8
32 37
0.9
39
0.95
41
25
Never be absolutely certain of anything
Bertrand Russell
A Bayesian approach is an
unconditional approach accounting for
uncertainty in parameters
26
Beal - Biometrics , 1989
“ A prior estimate of  …… is
needed. This clearly introduces
some uncertainty regarding the
required sample size “
Conditionality
27
Relation Between  and n

n
7
8
9 10 11 12 13 14
21 27 33 39 46
54 62 71
Suppose we have some idea about the
likely value of  through a probability
distribution
28
Unconditional Approach
P( w  w0 ) =

P( w  w0 | ) p( )d
2
2
2
2
29
Where do we get p(2) from ?
• Previous studies
• Expert opinion - subjective ?
•
Estimate of
2 :
2
s0
based on 0 d.f.
• Inverse-Gamma prior
p(  ) =
2
0 / 2
( s / 2 )
2
0 0
exp[ - 0 s02 /( 2 2 )]
2 ( 0  2 ) / 2
 ( 0 / 2(  )
30
Conditional Formula
 2 w02 n 
P( w  w0 ) = P   2 2  > 1 - 
8t  

Unconditional Formula – Grieve(1991)
2 

w0 n
P( w  w0 ) = P F ,0  2 2  > 1 - 
8t s0 

31
Elicitation of Inverse-Gamma
• Expert provides  L2 and  U2
s.t.
0 / 2
2
2
2
(

s
/
2
)
exp[

s
/(
2

)] 2
2
0 0
0 0
p(  ) =  2
d = p0
(


2
)
/
2
2
0
L
 ( 0 / 2(  )
• Not enough information – assume upper and
lower limits are (1-p0)/2 percentiles
• Solve directly or modify algorithm in Martz and
Waller(1982 – Bayesian Reliability Analysis),
Grieve (1987,1991)
 U2
32
Illustrative Example
Probability ( 8 <  < 13 ) = 0.8
Implies
2
s
0=14.66 , 0 =95.55
33
Relation Between P(w<w0) and n
n
51
52
53
54
55
56
57
P(w<w0) 0.873 0.882 0.892 0.899 0.906 0.912 0.918
In this example accounting for uncertainty
increases the sample size by 40 %
34
Elaborating a Prior for Tablet Counting
35
Checking the Dosing of Dogs
•
dogs dosed on mg/kg basis
• adjusted weekly
•
Example
• Unit Dose
• Weight
• Required dose
:
:
:
36 mg/kg
19.2 kg
691 mg
36
Pre-Manufactured Tablet Strengths
4-5
0 -11
0-4
0-4
300 mg
25 mg
5 mg
0.5 mg
2
3
3
2
691 mg
37
Dog Dosing
• Tablets placed in a gelatine capsule
• Are the correct number of tablets in the capsule ?
38
Possible Approaches
• Do Nothing
• Hope - No : Inspection
• Acceptance Sampling
• too few samples - 308 capsules/wk
• checking creates errors
• Check Everything
• checking creates errors
• Weigh Capsules & Contents
39
Tablet /Capsule Weights
Tablet Strength (mg)
Mean (g)
St. Dev.
300
0.602
0.0036
25
0.298
0.0035
5
0.150
0.0013
0.5
0.075
0.0008
Capsules
0.701
0.0410
40
Statistical Model
•
•
•
•
•
T tablet sizes
2
tablet weights are : N(m i ,  i ) i = 1,.., T
2
capsule weights are
:
N(m C ,  C )
Ni tablets of each size chosen
Total weight w is distributed as

N m C 

N m
T
i
i =1

T
i
,  
2
C
i =1

N i 

2
i
41
Hypothetical Example
2 X 300 mg
3 X 25 mg
3 X 5 mg
2 x 0.5 mg
=
=
=
=
600 mg
75 mg
15 mg
1 mg
691 mg
• Given a total weight of 3.397g (simulated)
• What can we say about the likely numbers of tablets?
42
Dog Dosing - Solution (1)
• Co-primal Weights
• 3 : 7 : 13 : 23 instead of 1 : 2 : 4 : 8
43
Dog Dosing - Solution (2)
• Co-primal Weights
• 3 : 7 : 13 : 23 instead of 1 : 2 : 4 : 8
• Pre-Weighing of Capsules
44
Dog Dosing - Weights
Tablet Strength (mg)
CV (%)
300
0.6
25
1.2
5
0.9
0.5
1.1
Capsules
5.8
45
Solution (3)
• Co-primal Weights
• 3 : 7 : 13 : 23 instead of 1 : 2 : 4 : 8
• Pre-Weighing of Capsules
• Prior distribution
belief in ability to count to 5
greater than
belief in ability to count to 19
46
Elaborating a Prior
Grieve et al (1994)
• Suppose a technician tries to count to M tablets
of a given strength
• A model of the process could be :
• The total of M tablets is “achieved” by M individual
operations each attempting to count to 1
• An error can be made in either direction : xj=0,1 or 2
• The total count is : x1+x2+ …. + xM=N
47
Elaborating a Prior
• Suppose the probability distribution of results
from a single count is given by :
x
0
1
2
P(xj=x)
q
r
p
with p.g.f. - P(t)=q+rt+pt2
•Assuming independent counts the p.g.f. of N is :
PN (t ) = P(t ) N = (q  rt  pt 2 ) N
=
M
M - c1
M!
k1 k 2 M - k1 - k 2 2 M - 2 k1 - k 2
q
r p
t


k1 = 0 k 2 = 0 k1!k 2 ( M - k1 - k 2 )!
• Giving :
M - [( n 1 ) / 2 ]
M!
k 2M - n - 2k n  c - M
P( N = n ) =
q
r
p

k = max( 0 ,M - n ) k ! ( 2 M - n - 2k )! ( n  k - M )!
48
Elaborating a Prior
ADVANTAGES
• A prior distribution need not be elicited for every M
• Elaboration ensures consistency
• If Mk=Mj+1 then P(Nk=Mk) < P(Nj=Mj)
DISADVANTAGES
• Need to elicit p,q (r=1-p-q)
• Assumptions
49
Feeding Back
Values of r ( p=q=(1-r)/2)
M
2
4
6
8
0.9
0.95
0.99
0.995 0.999
P(N=M)
0.815
0.904
0.980 0.990 0.998
P(N=M1)
0.090
0.048
0.010 0.005 0.001
P(N=M)
0.680
0.821
0.961 0.980 0.996
P(N=M1)
0.147
0.086
0.019 0.010 0.002
P(N=M)
0.581
0.750
0.942 0.971 0.994
P(N=M1)
0.183
0.117
0.029 0.015 0.003
P(N=M)
0.507
0.689
0.924 0.961 0.992
P(N=M1)
0.204
0.141
0.037 0.019 0.004
50
Dog Dosing - Conclusions
•
•
•
•
Such a scheme is practicable
Computations trivial
Pre-weighing essential
Prior distribution essential
• Perfect for robotification
51