DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Adaptive Trial Designs in Pediatric
Studies: A Survey of Methods and
Practices
Alfred Balch, PhD
Associate Professor, Pediatrics
Division of Clinical Pharmacology & Clinical Trials Office
Department of Pediatrics,
University of Utah School of Medicine
Methods in Progress
18-Oct- 2012
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Outline
• What are adaptive trials and why do we need to do them?
• Confirmatory Trials
– Group Sequential
» Alpha-spending approach
» Seamless Phase II-III
– Sample Size Re-Estimation
• Dose Finding Trials
–
–
–
–
–
Up and Down Designs: 3+3 and others
CRM
Bayesian Models
Adaptive Randomization
Using Efficacy and Toxicity: Utility Response
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
What are adaptive Trials?
“By adaptive design we refer to a clinical study design that uses
accumulating data to decide how to modify aspects of the study as it
continues, without undermining the validity and integrity of the trial. The
goal of adaptive designs is to learn from the accumulating data and to
apply what is learned as quickly as possible. In such trials, changes are
made “by design,” and not on an ad hoc basis; therefore, adaptation is a
design feature aimed to enhance the trial, not a remedy for inadequate
planning.
“Adaptive Designs in clinical drug development- An Executive Summary of the
PhRMA working Group” (2006)
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
What kind of adaptation?
• Number of Patients Enrolled:
– Terminate for Futility or for Early Success
– Sample Size-Re-Estimation
• Variance-Based
• Proportion-of-Completers based
• Type of Patients Enrolled
– Enroll Future Patients with Likely Success
• Patient Characteristics: Genotype, Age, Gender
– Enroll Patients from Successful Centers
– Change Assignment/Sampling to minimize covariate biases (?)
• Balance treatment Assignment
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
What kind of adaptation?
• Better Medical Practice Based On Partial Information:
– “Play the winner”
• Preferentially Enroll Patients into “Better” Treatment Groups
– Need to preserve patient randomization to treatment, if possible
• Dose-Finding- Enroll in “Better Dose”
– Better Regimens: Dose-Interval and Route
– Optimal (maximal information) Dose
• Preferentially Recruit “Better” Patients
– Need to preserve randomization to treatment, if possible
• Advantageous Co-therapies for all incremental enrolled patients
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
What’s the big deal?
• Why were earlier trials inflexible-what are the issues?
– Trials for Drug Registration need to maintain nominal level of
significance-need to account for multiple looks at the data
– This is not achieved for:
• Multiple looks at the data without statistical adjustment for multiplicity
– Armitage et al.(1969- unadj pval=0.142 for 5 looks nominal 0.05)
• Change in Endpoint
• Complicated Multiple Hypotheses
• Data-Driven changes in primary analysis
– The largest “response” may not be reproducible.
– In survival-type trials, bad news comes early. Need to be aware,
take early results with appropriate skepticism.
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Why do we “Need” more flexibility?
Armin Koch: BfArM (Bundesinstituts für Arzneimittel und Medizinprodukte)
• Sample size not correct:
“Quote marks” and coloring are mine!
– For example: Study assumptions were not accurate
• Was blinding maintained during sample size recalculation? If not, this is multiple look.
• Endpoint not well-chosen:
– Wrong doses or treatment
– Suboptimal statistical methodology
•
•
(Was blinding maintained when Stats were changed?)
Change of objective:
–
–
–
–
–
–
Superiority was an over-ambitious aim
Change in criteria for inclusion or exclusion
CRF’s limited or too extensive
composite endpoint does not differentiate between treatments
treatment effect in a different variable
wrong responder-definition
11
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Adaptive Method 1:Group-Sequential methods
– Preplanned (protocol/SAP) interim looks at the data
• k = number of looks (including final)
– Usually presented as equal increment group sizes
• Z1, Z2, … Zk Calculated Test Statistics at ith look (i=1,k)
– Based on all data observed up to look i
•
•
•
•
•
z1,…zk critical values
Stop and declare efficacy if Z i > zi for any i (including i=k)
Perhaps stop for futility if Z i < -zi (or alternate threshold)
For k=1 look this is just standard normal critical value
Test needs to adjust z-criterion so overall alpha=0.05 (or 0.025
or…) for this procedure
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Sequential Test Statistics: Normal Case
– In normal distribution case, in ith look, adjust Critical value of Zi to allow
for multiple looks at the data, taking into account that results are
correlated (k interim groups of size m)
This is a random walk like process….
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Pocock’s Method:
• Critical Values for K-multiple looks (Pocock (1977))
– Z remains the same for all looks…
– Define critical values, Cp(K,α)
Pocock Critical Values for 2-Sided Tests
K
1
2
3
4
5
10
20
α=0.01
2.576
2.772
2.873
2.939
2.986
3.117
3.225
α=0.05
1.96
2.178
2.289
2.361
2.413
2.555
2.672
α=0.10
1.645
1.875
1.992
2.067
2.122
2.27
2.392
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
O’Brien and Fleming Boundary
O.F. Critical Values for 2-Sided Tests
K
1
2
3
4
5
10
20
α=0.01
2.576
2.580
2.595
2.609
2.621
2.660
2.695
α=0.05
1.96
1.977
2.004
2.024
2.040
2.087
2.126
α=0.10
1.645
1.678
1.710
1.733
1.751
1.801
1.842
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Families of Boundaries:
• Wang and Tsiatis (1987))
– Use parameter Δ
» Δ=.5 => Pocock
» Δ=.0 => O’Brien Fleming
– Delta parameter emphasizes/de-emphasizes final Z
• Lots of additional work on optimal spending function(s)– Many generalize Wang & Tsiatis further
» Kim & Demets(1987)
» Hwang, Shih, De Cani (1990)
» 2 spending functions based on α and β
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Alpha-Spending Approach
K. K. Gordon Lan (PhD
in Mathematical
Statistics from Columbia
University) joined J&J in
2005 after holding
positions at
(NHLBI/NIH),
George Washington
University, Pfizer and
Sanofi-Aventis. He has
published more than 60
research papers in
professional journals,
and has given
He is a Fellow of the
ASA.
2012 Deming
Conference Website
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Group-Sequential methods (cont)
Keaven Anderson has a Ph. D. from Stanford University and is the head
of late-stage oncology statistics at Merck Research Laboratories where he
has worked since 2003. He is the primary author and maintainer of the
open source R package gsDesign for designing group sequential trials.
Deming Conf on Applied Statistics
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Group-Sequential methods
– Why more looks don’t cost much….Bonferoni is much too conservative:
– Random Walks…think coin toss where can win or lose 1 dollar
• “Real Case” is continuous analog where Brownian motion replaces Markov Chain
– X1…Xn are iid on {-1,1} where
• P(Xi = 1) = ½
• P(Xi= -1) = ½
– Sn+1= Sn + Xn+1
– Observe S1, S2, …Sn
– How big can Si get before the coin clearly isn’t fair? Test at 0.05 level of
significance….
– 2n equally likely outcomes- count paths which exceed a threshold value.
This section is optional…
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Group-Sequential methods
– Terminate Trial of 5 Tosses for success when a score of 4 is
reached. Interim looks at 4 and 5 tosses. (K=2 looks)
– For Five Coin Tosses What outcomes give this result? (Assuming a
fair trial) – 32 equally likely possible outcomes:
• THHHH, HTHHH, ….HHHHT
• And for stopping at 4 tosses
• HHHHT, HHHHH
– But wait, we counted HHHHT twice (really just HHHH)
• So 6 total outcomes result in terminate for success
• P-value = 6/32 for this procedure under Ho
– Note: This doesn’t change with an additional look at n=1 toss!
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Group-Sequential methods
– Terminate Trial of 5 Tosses for success when a score of 4 is
reached. Interim looks at 4 and 5 tosses.
– For Five Coin Tosses What outcomes give this result? (Assuming a
fair trial)
• THHHH, HTHHH, ….HHHHT
• And for stopping at 4 tosses
• HHHHT, HHHHH
– But wait, we counted HHHHT twice (really just HHHH)
• So 6 total outcomes result in terminate for success
• P-value = 6/32 for this procedure under Ho
– Note: This doesn’t change with an additional look at n=1 toss!
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
-20
-10
0
Sum
10
20
30
Random Walk with 100 coin tosses, Fair Coin
0
20
40
60
number of trials
80
100
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
What is the
power? At .7
10
20
30
Stop for Success at 30
0
What is the level
Of significance of
This test?
Sum
40
50
60
Random Walk with 100 coin tosses, p=.7
0
20
40
60
number of trials
80
100
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Random Walk with 100 coin tosses, p=.5
5
0
-5
Sum
10
15
interim look at 60 patients
0
20
40
60
80
100
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Summary: Group-Sequential methods
– Under Ho, Adjusted Z scores form a continuous martingale
process.(like a random walk- increments are iid) :
E(S n+1 | S1…Sn)= Sn
Under Ha there is a drift parameter
– Pocock (1977)
– O’Brien & Fleming (1979)
– Generally O’Brien and Fleming is FDA-preferred; not as wide a boundary in
initial looks as Pocock (“spend alpha late”).
– Additionally, DSMB or statistician can choose to use alpha-spending
(typically spend most of the alpha on final analysis, less in early stage for
extreme case in early look).
– Regulatory trials total alpha=0.025
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Interim Looks and Blinding
– Logistically don’t want a lot of interim looks at the data, need to clean
and lock data for each look and deal with DSMB logisticsMathematically, interim looks with small alpha (big Z) are not a big
deal.
– Unblinding can be a risk, knowing the trial is not terminated can bias
the future conduct of the trial.
– Interim looks can save a lot of subjects when the data are a lot better
or a lot worse than expected. Also if they are not provided in the
protocol, they will happen anyway!
– There is excellent free software available to do Group Sequential
calculations.
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
gsDesign
– Suite of GS software developed by Keaven Anderson at Merck
– Example: (Runs under r-statistical programming language)
– n.fix is the number reqd. for fixed sample size (dep on delta,sigma)
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
gsDesign
Normal test statistics at bounds
3.25
2.99
2.69
2.37
2.03
Normal critical value
2
Bound
0
Lower
Upper
-2.03
-2
-2.37
-2.69
-2.99
-3.25
-4 N=164
200
N=328
N=492
400
N=656
600
Sample size
N=819
800
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Sample Size Re-Estimation
– Variance Based
• Suppose initial sample size calculation is based on poorly known
estimate of variance? Can we fix this without unblinding the trial?
• Variance and Mean estimates for normal distribution are independent
• Independent statistician can calculate sample variance for observed
data.
• Trial statistician can use this estimate directly or use a
weighted/Bayesian update of variance estimate to recalculate overall
sample size.
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Sample SizeRe-Estimation
– Dropout Rate Based:
• Independent statistician can calculate dropout rate for observed data.
• Protocol can specify number of completed for sample size
• What are the unblinding risks??
– Are dropouts biased to treatment?
– Will dropouts affect actual trial conduct/termination and hence impact α?
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Adaptive Exploratory Trials
• In early trials, dosing frequency, route and
total amount may need to be modified
based on patient outcomes
• Focus on learning (not confirming),
successful outcome is a good decision
about next trial (if!) and its design; should
have ideas about dose, population,
indication.
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Adaptive Exploratory Trials
– Oldest: Mood & Dixon (1948) up and down
• Idea: Converge on Tox rate of (say) 50%
– Most common in oncology: 3+3
– Improvements proposed (Ed Korn, NIH,
Gezmu & Flourney (2006);
• Let R(dj) be the number of toxic responses for
group j given dose dj , then R(dj) has a binomial
distribution with probability dependent on dose.
• Use logistic regression to model Dose-Response
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Traditional Adaptive FIH Oncology Exploratory
Trials: 3+3
• Dose follows Fibonacci Sequence
– 1,1,2,3,5,8
– Or modified so increments decrease as dose increases
• 3+3 Design (somewhat adaptive)
– Enter 3 at lowest dose
– 0/3 DLT escalate to next dose
– 1/3 DLT expand dose cohort to 6:
» 1/3 + 0/3 treat next 3 at next higher dose
» 1/3+1/3 This is MTD dose
» If >2 DLT of 3 then reduce dose and repeat
» Stop and DLT when <= 2/6 DLT
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Up and Down Designs- Generally
– Define Cu.CL so that next cohort of patients is
randomized to dose
– dj-1 if R(dj) ≥ Cu
– dj+1 if R(dj) ≤ CL
– dj if CL < R(dj) <Cu
Cu , CL are pre-defined to reflect risk
tolerance.
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Up and Down Design (continued)
• Features:
• Next dose-cohort is based on information
gathered from trial (modification: dj+1 can depend
on Rj ). Dose sequence defines a Markov Chain
• More aggressive dose-expansion when no
problems at low dose(s)
• Generalizes and Improves much used “3+3
design”.
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Bayesian Adaptive FIH Oncology Exploratory Trials:
Continual Reassessment Method(CRM)
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Bayesian Adaptive Ph I-II trial:Escalation With
Overdose Control(EWOC)
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Bayesian Adaptive Ph I-II trial:Escalation With
Overdose Control(EWOC)
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Bayesian Adaptive Ph I-II trial: EWOC
• Let Y be an indicator of DLT
– FY(y) be the CDF of Y and define x* and x**
such that
• P(Y = 1|x = x∗) = 0,
• P(Y = 1|x = x∗∗) = 1−ϵ
(1−ϵ is “known” to be < ϴ) so prior on ϴ is
supported on [x*,x**]
Initial dose, d1 less than x**, is chosen
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Bayesian Adaptive Ph I-II trial: EWOC
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Bayesian Adaptive Ph I-II trial: EWOC
• EWOC software is available free
André Rogatko Ph.D
Director, Biostatistics and Bioinformatics at
Cedar Sinai. Dr. Rogatko has more than
100 published peer-reviewed articles
From 2010
Deming Conference Website
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Not to be
confused with
EWOK!
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Traditional Non-Oncology FIH
• Dose Increase by ½ log10
–
–
–
–
–
1,3,10,30,100
Single Ascending Dose (SAD) then
Multiple Ascending Dose (MAD)
Emphasis on PK, PD, PG Endpoints
First Dose Based on Animal Tox
• 6+2 Design
– 2 subjects receive placebo at each dose
» Control for safety and PD/Response
» Stop with dose 1-2 DLT (in active trtd)
– Not adaptive-may change dose, monitoring, etc., but usually
not in a pre-planned way.
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Traditional Non-Oncology FIH
– MCP-Mod- Adjust Dose-Reponse Function
– Use Scheffe’ to allow multiple looks at Dose-response significance
– Jose Pinheiro
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Bayesian non-oncology FIH
• Adaptive Randomization
– Use CRM strategy, but with utility replacing safety
» Utility function can combine safety/efficacy
– First Dose Response Based on Animal Tox, update DR curve
with human data as cohort data becomes possible
– Preferentially randomize into “best dose”
– May need to span low doses to get human data.
– SAE is still a stopper!
– Design can be parallel or nested, dosing can be direct dose, or PKbased (safe concentrations, AUC, Cmax)
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Adaptive design methods
• Innovations in clinical trials methodology
have the potential to:
– Improve the quality of knowledge, protect
human subjects, and improve the efficiency of
clinical research
• Adaptive trials are often ethically required
and will happen whether planned or not.
– Better to plan them as far as possible
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Review: Adaptive Trials
• Adaptive designs can be classified into
two categories:
– Exploratory/Dose Finding
– Confirmatory
• An adaptive design allows modifications to
be made to trial and/or statistical
procedures of on-going clinical trials.
• Adaptation may/should be pre-specified by a rule:
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Concerns
• Concern that the actual patient population after the
adaptations could deviate from the originally target patient
population
• Could lead to an increased overall type I error rate (to
erroneously claim efficacy for an infective drug).
• Analysis needs to be adjusted appropriately for adaptive
design
• Major adaptations of trial and/or statistical procedures of ongoing trials may result in a totally different trial that is unable
to address the scientific/medical questions the trial intends to
answer.
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Benefits of all adaptive trials
– Patient welfare is paramount, need to be able
to ethically enroll patients in ongoing
doses/arms of trials
– In pharmaceutical/clinical research and development has
become popular due to its flexibility and efficiency for
identifying potential signals of clinical benefit of the test
treatment under investigation.
– The value of adaptive clinical trial methods in exploratory
phase (phase II) clinical research is generally well
accepted.
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Resources*
Websites:
http://cran.r-project.org/web/packages/gsDesign/gsDesign.pdf
Really great documentation of Keaven Anderson’s free gsDesign suite
for the free r statistical language.
https://biostatistics.mdanderson.org/SoftwareDownload/
Good variety of programs, including Adaptive Bayesian designs
http://cran.r-project.org/web/packages/CRM/CRM.pdf
R Program to do CRM –check other places around CRAN as well!
http://arxiv.org/pdf/1011.6479.pdf
Andre Rogotku “Escalation with Overdose Control”-software also
available from the author (Cedar-Sinai)
* Presentation will be made available after class
46
DEPARTMENT OF PEDIATRICS
Division of Clinical Pharmacology
Resources
Knowledgeable about adaptive design and helpful for this talk!
Dr. Michael Spigarelli
Professor: Pediatrics, Internal Medicine and Pharmacy
Dr. Catherine M. T. Sherwin, PhD, FCP Assistant Professor, Pediatrics
Books
Statistical Methods for Dose-Finding Experiments
Ed. Sylvie Chevret (2006) John Wiley and Sons
Good collection of Adaptive Design methods mostly for early
development.
Bayesian Adaptive Methods for Clinical Trials
Scott M. Berry, et al. (2011) CRC Press
All Bayesian- good approach, esp. for Oncology
Group Sequential Methods with Applications to Clinical Trials
C.Jennison, B. Turnbull (2000) CRC Press
Nice Summary of GS methods and underlying theory-well developed
47