Piloting and Sizing Sequential Multiple
Assignment
Randomized Trials in Dynamic
Treatment Regime Development
2012 Atlantic Causal Inference Conference
May 25, 2012—Johns Hopkins
Daniel Almirall & Susan A. Murphy
Outline
• Dynamic Treatment Regimes
• Sequential Multiple Assignment
Randomized Trial (SMART)
• External Pilots
– Tailoring Variables
– Transition to Next Stage
– Assessment Schedule
– Sizing a Pilot SMART
2
Dynamic treatment regimes are individually
tailored sequences of treatments, with treatment
type and dosage changing according to patient
outcomes. Operationalizes clinical practice.
k Stages for one individual
Patient data available at jth stage
Action at jth stage (usually a treatment)
3
Dynamic Treatment Regimes
• A dynamic treatment regime (DTR) is a
sequence of decision rules, one per treatment
stage.
• Each decision rule inputs one or more tailoring
variables and outputs a treatment action.
• The tailoring variables are (summaries of)
patient data available at each stage.
4
Example of a Dynamic Treatment
Regime (DTR)
•Adaptive Drug Court Program for drug
abusing offenders.
•Goal is to minimize recidivism and drug
use.
•Marlowe et al. (2008, 2009)
5
Adaptive Drug Court Program
non-responsive
low risk
As-needed court hearings
+ standard counseling
As-needed court hearings
+ ICM
non-compliant
high risk
non-responsive
Bi-weekly court hearings
+ standard counseling
Bi-weekly court hearings
+ ICM
non-compliant
Court-determined
disposition
6
Sequential, Multiple Assignment,
Randomized Trial (SMART)
At each stage subjects are randomized among
alternative options. For k=2, data on each
subject is of form:
Aj is a randomized treatment action with known
randomization probability.
7
•Usually the treatment options for A2 are
restricted by the values of one or more
summaries of (X1, A1, X2)
• These summaries are embedded
tailoring variables; they are embedded
in the experimental design.
• The embedded tailoring variable(s)
restrict the class of DTRs that can be
investigated using data from the
SMART.
8
Pelham ADHD Study
Continue, reassess monthly;
randomize if deteriorate
Yes
8 weeks
Begin low-intensity
BMOD
AssessAdequate response?
BMOD + Med
Random
assignment:
No
BMOD++
Random
assignment:
Continue, reassess monthly;
randomize if deteriorate
Yes
8 weeks
Begin low dose
Med
Med ++
AssessAdequate response?
Random
assignment:
No
BMOD + Med
9
ADHD: Embedded Tailoring
Variable
• Early response is determined by two teacherrated instruments, ITB and IRS.
• Binary embedded tailoring variable
• R=0 if ITB<.75 and one or more subscales of
IRS >3; otherwise R=1.
• R is the embedded tailoring variable.
10
External Pilot Studies
• Goal is to examine feasibility of full-scale trial.
–
–
–
–
–
Can investigator execute the trial design?
Will participants tolerate treatment?
Do co-investigators buy-in to study protocol?
To manualize treatment(s)
To devise trial protocol quality control measures
• Goal is not to obtain preliminary evidence
about efficacy of treatment/strategy.
– Rather, in the design of the full-scale SMART, the
min. detectable effect size comes from the science.
11
Embedded Tailoring Variable
• Don’t use an embedded tailoring variable
unless the science demands it.
• If you have an embedded tailoring variable
make it simple (e.g. binary measure of
(non-) response)
– Non-responders likely to fail if continue on current
treatment OR responders unlikely to gain much
benefit if they stay on current treatment.
– Usually need to use analyses of existing data to
justify the use of the tailoring variable
12
Jones’ Study for Drug-Addicted
Pregnant Women
Decrease scope/intensity
2 wks Response
Random
assignment:
tRBT
Random
assignment:
Continue on same
Continue on same
Nonresponse
Increase scope/intensity
Random
assignment:
2 wks Response
Decrease scope/intensity
Random
assignment:
Continue on same
rRBT
Random
assignment:
Nonresponse
Continue on same
Increase scope/intensity
13
Missing Tailoring Variable
• How to manage missingness in the
embedded tailoring variable for purposes
of randomizing/assigning subsequent
treatment?
– VERY different from handling missing data
in a statistical analysis.
– Tailoring variable is part of the definition of
the treatment and experimental design.
14
Missing Tailoring Variable
• Need to formulate a fixed, pre-specified
rule to determine subsequent treatment if
tailoring variable is missing.
– Unexcused visit==non-response
– Use a rule that depends on all observed data,
including the data collected when the subject
again shows up at a clinic visit.
– Try out the rule in pilot.
15
Assessment Schedule
• How often should the tailoring variable be
measured?
• Example: Alcoholism study with weekly
assessments of days of heavy drinking.
– Weekly assessments were insufficient and likely
a pilot study would have detected this.
16
Oslin’s ExTENd Study
Naltrexone
8 wks Response
Random
assignment:
Nonresponse if
HDD >1
Random
assignment:
TDM + Naltrexone
CBI
Nonresponse
CBI +Naltrexone
Random
assignment:
8 wks Response
Naltrexone
Random
assignment:
TDM + Naltrexone
Nonresponse if
HDD>4
Random
assignment:
Nonresponse
CBI
CBI +Naltrexone
17
Outcome Assessment
versus
Tailoring Variable Assessment
• Keep these separate.
– Tailoring variable assessment done at clinic visit by
clinical staff or clinical lab or participant. Outcome
assessment done at research visit by independent
evaluator or independent lab or participant.
• Autism & Adolescent Depression Examples
• Try out in Pilot Study
18
Transition Between Stages
• Clinical staff disagree with when 2nd stage
treatment is introduced.
• Non-responding subject refuses 2nd stage
treatment.
– This may be VERY important scientifically
– Cocaine/Alcoholism Example
• Test in Pilot
19
Sample Size for a SMART Pilot
• Primary feasibility aim is to ensure
investigative team has opportunity to
implement protocol from start to finish with
sufficient numbers
– If investigator has good evidence to guess the
response rate: Choose pilot sample size so that
with probability q, at least m participants fall into
the sub-groups (the “small cells”)
– If little to no evidence concerning response rate,
size the study to estimate the response rate with a
given confidence interval width.
20
Pelham ADHD Study
Continue, reassess monthly;
randomize if deteriorate
Yes
8 weeks
Begin low-intensity
BMOD
AssessAdequate response?
BMOD + Med
Random
assignment:
No
BMOD++
Random
assignment:
Continue, reassess monthly;
randomize if deteriorate
Yes
8 weeks
Begin low dose
Med
Med ++
AssessAdequate response?
Random
assignment:
No
BMOD + Med
21
Sample Size for a SMART Pilot
• There are 2 treatment actions in stage 1, kR
treatments for responders, kNR treatments for
non-responders. Investigator chooses q (say
80%) and m (say 3), and assumes overall nonresponse rate pNR (say 50%).
• Solve
P[:5N ¡ kR m ¸ B ¸ kN R m]2 ¸ q
for N, the total sample size, where
B » Bin(:5N; pN R )
22
Discussion
• SMART clinical trial designs are of
growing interest in the clinical sciences.
• Because these designs are very new, they
require a great deal of leadership on the
part of the statistical community.
• The payoff for the statistician is
– Inform clinical science in a novel manner
– Unusual and novel trial data for methodological
development
23
This seminar can be found at:
http://www.stat.lsa.umich.edu/~samurphy/
seminars/ACIC_2012.ppt
Reference:
Almirall D, Compton SN, Gunlicks-Stoessel M, Duan
N, Murphy SA. Designing a Pilot SMART for
Developing an Adaptive Treatment Strategy. To
appear in Statistics in Medicine
24