Overview of Bayesian Methods for
Safety Assessment
Karen L. Price, PhD
Eli Lilly and Company
On behalf of the DIA Bayesian Scientific Working Group (BSWG)
Company Confidential
© 2014 Eli Lilly and Company
Outline
• Brief overview of DIA BSWG
• Overview of use of Bayesian methods for safety
assessment
• Bayesian network meta-analysis with focus in
safety data
• Bayesian methods for safety trials
• Conclusion
Who are we?
Group of representatives from Regulatory,
Academia, and Industry, engaging in scientific
discussion/collaboration
– facilitate appropriate use of the
Bayesian approach
– contribute to progress of
Bayesian methodology
throughout medical product
development
Mission
To facilitate the appropriate use of Bayesian
methods and contribute to progress by:
• Creating a scientific forum for the discussion and
development of innovative methods and tools.
• Providing education on best practices for
Bayesian methods.
• Engaging in dialogue with industry leaders, the
scientific community, and regulators.
• Fostering diversity in membership and
leadership.
Opportunity Statement
• Bayesian methods provide framework to leverage
prior information and data from diverse sources.
• Bringing together academic, industrial, and
regulatory representatives is essential to
overcome hurdles.
• Provides opportunity to influence
proactively by engaging in
scientific discussion.
• Improved patient outcomes.
Safety Subteam
• Opportunity/Goals
• Current analytical approaches may be oversimplified and
knowledge of/experience with proper methods inadequate
• Some statistical challenges include: power, multiplicity,
complexity of data, continual assessment, signal refinement
• Bayes provides great promise
• 3 initial areas of focus
• Meta-analysis/Evidence Synthesis: chair David Ohlssen
• Safety Trials: chair Karen Price
• Signal Detection: chair Larry Gould
• Initial deliverables: white papers, publications, sessions
Some Advantages of Bayesian Methods
• Ability to incorporate prior information
• Natural for evidence synthesis or meta-analysis
• Handling multiplicity through borrowing strength
and hierarchical modeling
• Appealing in dealing with rare events as the
model modulates the extremes
• Ability to handle complex problems via unified
modeling, taking all the uncertainty into account
• Allowing direct probability inferences on different
scales
• “Safety assessment is one area where
frequentist strategies have been less
applicable. Perhaps Bayesian approaches in
this area have more promise.”
-- Chi, Hung, and O’Neill; Pharmaceutical Report, 2002
“If I were to predict where Bayesian ideas will
have great impact in the years ahead I would
highlight drug safety – not only during the
development of a drug but also postmarketing.”
-- Grieve; Pharmaceutical Statistics, 2007
8
Overview of Some Areas of
Implementation
1.
2.
3.
4.
5.
6.
7.
8.
Safety signal detection
Safety signal evaluation
Meta-analysis for analyzing adverse event data
Continuously monitor an event of interest in an
ongoing trial
Joint modeling for evaluation of safety/efficacy
outcomes
Estimating the dose-response relationship of
adverse events
Mixed treatment comparisons or network metaanalysis for safety data
Safety Trials
Screen shot of Pharmaceutical Statistics Special
Issue
www.diahome.org
10
Recent Publications from DIA BSWG
Pharmaceutical Statistics Special Issue:
Bayesian Methods in Medical Product Development and Regulatory Review
•
The current state of Bayesian methods in medical product development: Survey
results and recommendations from the DIA Bayesian Scientific Working Group:
Fanni Natanegara, Beat Neuenschwander, John W. Seaman, Nelson Kinnersley, Cory R.
Heilmann, David Ohlssen, George Rochester
•
Bayesian Methods for Design and Analysis of Safety Trials: Karen Price, H Amy Xia,
Mani Lakshminarayanan, David Madigan, David Manner, John Scott, James Stamey, Laura
Thompson
•
Guidance on the implementation and reporting of a drug safety Bayesian network
meta-analysis: David Ohlssen, Karen Price, H Amy Xia, Hwanhee Hong, Jouni Kerman,
Haoda Fu, George Quartey, Cory Heilmann, Haijun Ma, Bradley Carlin
•
Use of Historical Control Data for Assessing Treatment Effects in Clinical Trials:
Kert Viele, Scott Berry, Beat Neuenschwander, Billy Amzal, Fang Chen, Nathan Enas, Brian
Hobbs, Joseph G Ibrahim, Nelson Kinnersley, Stacy Lindborg, Sandrine Micallef, Satrajit
Roychoudhury, Laura Thompson
Therapeutic Innovation and Regulatory Science, submitted
•
Methods and Issues to Consider for Detection of Safety Signals from
Spontaneous Reporting Databases. Report of the DIA Bayesian Safety Signal
Detection Working Group. Larry Gould, Ted Lystig, Yun Lu, Haoda Fu, Haijun Ma, and
David Madigan
BAYESIAN NETWORK METAANALYSIS WITH FOCUS IN
SAFETY DATA
(BASED ON OHLSSEN, ET AL)
Network meta-analysis
Study 1
A
PL
Study 2
B
PL
Future
study
Additional
Studies
C
A
C
PL vs A: B
PL vs C
Of Interest Cvs A
AC: Active Comparator
13
MTC : Random Effects Model
(taken from NICE DSU documents)
rik ~ binomial (nik , pik )
i1  μt
i1
First arm in study i
p
μ  log
1 p
ik  μt +t
i1
i1 ,tik
Relative treatment effect
between 1st arm and kth arm
treatment effect
of 1st arm
t
i1 ,tik
Consistency assumption
μti1 ~N(0,1000)
kth arm in study I
k=2,..,K
d 23  d13  d12
d 24  d14  d12
...
d ( s 1), s  d1s  d1, s 1
~ N (dti1 ,tik , 2 )
between trial
standard deviation
Network meta-analysis Trelle et al (2011)
Cardiovascular safety of non-steroidal anti-inflammatory drugs
 Primary Endpoint was myocardial




infarction
Data synthesis 31 trials in 116 429
patients with more than 115 000
patient years of follow-up were
included.
A Network random effects metaanalysis were used in the analysis
Critical aspect – the assumptions
regarding the consistency of
evidence across the network
How reasonable is it to rank and
compare treatments with this
technique?
Trelle, Reichenbach, Wandel, Hildebrand, Tschannen, Villiger, Egger, and Juni. Cardiovascular safety of non-steroidal anti-inflammatory drugs
network meta-analysis. BMJ 2011; 342: c7086. Doi: 10.1136/bmj.c7086
15
Poisson network meta-analysis model
Based on the work of Lu and Ades (LA) (2006 & 2009)
b is the control treatment
associated with trial i
• μi is the effect of the baseline treatment b in trial i and δibk is the trialspecific treatment effect of treatment k relative to treatment to b (the
baseline treatment associated with trial i)
• Note baseline treatments can vary from trial to trial
• Different choices for µ’s and  ’s. They can be: common (over studies),
fixed (unconstrained), or “random”
• Consistency assumptions required among the treatment effects
• Prior distributions required to complete the model specification
16
Comments on Trelle et al
• Drug doses could not be considered (data not
available)
• Average duration of exposure was different for
different trials
• Therefore, ranking of treatments relies on the
strong assumption that the risk ratio is constant
across time for all treatments
• The authors conducted extensive sensitivity
analysis and the results appeared to be robust
Key Aspects of Ohlssen, et al.
• Summarizes Bayesian network meta-analysis
• Extends the Lu and Ades (LA) model via a variety of
alternative model parameterizations
• Particularly in the context of rare events, estimation of
model parameters can be challenging for LA model
• Outcomes can be particularly sensitive to the choice of
model, emphasizing need for sensitivity analysis and
transparency regarding assumptions/limitations
• Highlights benefit Bayesian approach provides for
decision making (including with multiple outcomes)
• Provides reporting guidelines
Reporting Guidelines
• Ohlssen et al provides a checklist for use when
conducting a safety meta-analysis
• Checklist includes four main sections:
Introduction, Methods, Results, and Interpretation.
• Each main section includes various items relevant
to that section
• The user of the table should evaluate each item
and can utilize the last two columns to confirm
whether or not each item has been addressed
and to add any relevant comments
BAYESIAN METHODS FOR
DESIGN AND ANALYSIS OF
SAFETY TRIALS
(BASED ON PRICE, ET AL)
Overview of Paper
• Reviews challenges associated with safety trials
• Describes several opportunities for use of
Bayesian methods to enhance safety trials
• Discusses several case examples
Recommendations: Overview of
Bayesian Opportunities for Safety Trials
Opportunity
[1] Bayesian methods to
determine sample size
Key References
Adcock; Wang and Gelfand;
Brutti, De Santis, and Gubbiotti;
Gaydos et al.
[2] Frequent interim analyses
Connor and White et al.
[3] Bayesian Meta-analysis
Spiegelhalter et al.;
Stangl and Berry;
Sutton et al.
23
Recommendations: Overview of
Bayesian Opportunities for Safety Trials
Opportunity
[4] Sequential meta-analysis
[5] Borrowing historical
information
Key References
Cheng and Madigan;
Higgins, Whitehead, Simmonds;
Ibrahim et al.;
Zeggini and Ioannidis
Berry et al.;
Hobbs et al.
[6] Continuous monitoring of
events
Xia et al.;
Yao et al.
[7] Hierarchical modeling
Gelman and Hill;
Gelman et al.;
DuMouchel
24
Recommendations: Overview of
Bayesian Opportunities for Safety Trials
Opportunity
[8] Post approval
studies/Surveillance studies
Key References
FDA Guidance;
Murray, Carlin, and Lystig
[9] Logistical planning related
to enrollment rates and
landmark event rate
Gajewski, Simon, and Carlson;
Bagiella and Heitjan;
Ying and Heitjan;
Donovan, Elliott, and Heitjan
[10] Bayesian interpretations
and predictions
Spiegelhalter;
Berry et al.
25
Case Example: Sequential Monitoring of
AEs
• Sequential Bayesian methods enable regular updating
of knowledge as data accumulate
• Cheng and Madigan illustrated this approach with Vioxx
• Presented a Bayesian sequential meta-analysis of the
placebo-controlled trials
• The analysis began with a “family of priors”
• Proposed a simple graphical summary of the metaanalysis showing the posterior probability over time that
the true relative risk of CVT events exceeds two
particular thresholds
• The following figure shows the posterior probability that
the true relative risk exceeds 1.1 over time
Case Example: Sequential Monitoring of
AEs, cont.
0.4
probability
0.6
0.8
1.0
Posterior Probability True RR Exceeds 1.1
0.0
0.2
skeptical prior
skeptical posterior
cautious prior
cautious poster ior
reference prior
reference posterior
1999
2000
2001
2002
time
2003
2004
Moving Forward
• Safety Meta-analysis guidance from FDA (draft
published, opportunity to comment)
• Continued growth in use for signal assessment
• Opportunities for increased use for safety trials
• Expanded use for evaluation of benefit/risk profile
(at least for key benefits/risks)
Conclusion
• Safety assessment is complex with numerous
statistical challenges
• DIA BSWG is actively working to ensure the use
of Bayesian methods in the context of safety are
appropriately used by increasing awareness and
providing best practice guidelines
• Bayesian methods provide advantages in the
context of safety signal assessment
Thank you!
Questions?
Backup
MTC Case Example: Code
(random effects)
proc mcmc data=b missing=ac nmc=10000 diag=ess outpost=o1;
parms sd /slice;
parms m;
parms sd_m /slice;
prior sd sd_m ~ uniform(0, 100);
prior m ~ normal(0, prec=0.0001);
beginnodata;
tau = 1 / (sd * sd);
tau_phi_prec = 1 / (sd_m * sd_m);
endnodata;
random mu ~ norm(m, prec=tau_phi_prec) subject = study
monitor=(mu);
random d2 ~ normal(0, prec=0.0001) subject = trt2 monitor=(d2);
MTC Case Example: Code
(random effects)
random delta2 ~ norm(d2, prec=tau) subject = study monitor=(delta2);
random d3 ~ normal(0, prec=0.0001) subject = trt3 monitor=(d3)
zero="0";
if rep eq 3 then do;
taud = tau * 2 * 2 / 3;
w2 = delta2 - d2;
sw3 = w2 / 3;
md3 = d3 + sw3;
end;
else do;
md3 = 0;
taud = 1;
end;
random delta3 ~ norm(md3, prec=taud) subject = trt3 monitor=(delta3)
zero="0";
MTC Case Example: Code
(random effects)
ph = logistic(mu); /* control arm */
model r1 ~ binomial(n1, ph);
ph = logistic(mu + delta2);
model r2 ~ binomial(n2, ph);
if rep = 3 then do;
ph = logistic(mu + delta3);
llike = lpdfbin(r3, n3, ph);
end;
else do;
llike = 0;
end;
model general(llike);
run;