Using Stata Graphs to
Visually Monitor the
Progress of Multi-centre
Randomized Clinical
Trials
Alexandra Whate & Glenn Jones
General Background
• Clinical trials are increasingly multi-centre
and global
• Routine meetings about trial status become
logistically difficult and expensive to hold
• Investigators have different levels of training
and motivation in carrying out clinical trials
• Monthly performance monitoring reports have
traditionally been text and table based adaptations need to be made
Our Context
• In 6 trials Investigators from over 30 countries
collect data and submit them to the DMC
• Monthly Performance Monitoring Reports are
created by the DMC and submitted to Investigators
and Technical Officers (TO)
• Used to evaluate investigator performance
• Used to monitor study progress
– Accrual, follow-up, survival, adverse events
– Provide investigators with the “big picture” to
encourage continued accrual, adherence to protocol
and patient follow-up
– Detection of emerging issues or bias
Principles to Optimize Monthly Reports:
• Visual – to overcome language barriers
• Relatively Simple – to match the level of
training of Investigators
• Discriminating – To easily identify those that
are adhering to protocol and those who are
not
• Efficient – Quick to read and understand
• Strategic – Motivating Investigators; minimize
bias; ensure patient safety
Applications of Stata
1. Monitoring Accrual
2. Monitoring Survival
3. Monitoring Clinical Trajectories
4. Monitoring Follow-up
1. Accrual
Accrual should be relatively rapid
All centres should participate in
adding patients to the study on a
regular basis
Using Stata To Monitor Accrual
• Line plots show total accrual over the time of
the study (standard)
• Bar graphs show monthly accrual
• Bar graphs can also be made ‘by centre’ to
show accrual for each centre involved
• These can demonstrate
– Trends in accrual
– Interruptions due to issues with staffing and
equipment
Total Accrual
140
120
100
80
60
40
20
0
12
11
9
6
Aug 2009
Sept 2009
June 2009
April 2009
2
March 2009
Jan 2009
July 2008
June 2008
May 2008
April 2008
March 2008
Feb 2008
6
July 2009
3
2
1
8
6
May 2009
5
6
Feb 2009
4
6
Dec 2008
Jan 2008
5
Nov 2008
Dec 2007
5
Oct 2008
3
6
Sept 2008
3
6
7
Aug 2008
3
Nov 2007
5
Oct 2007
14
12
10
8
6
4
2
0
Graph bar var, over(month var, label(angle(ver))) blabel(total) ytitle ylabel(0(2)14) ylabel(,angle(hor)) saving(graphstata)
twoway line var var, clwidth(medthick) ylabel(,angle(hor)) xlabel(,nolabel) xtitle("") saving(graphlinestata)
graph combine graphstata.gph graphlinestata.gph, col(1) title(“Total Accrual”)
0
Oct 2007
Nov 2007
Dec 2007
Jan 2008
Feb 2008
March 2008
April 2008
May 2008
June 2008
July 2008
Aug 2008
Sept 2008
Oct 2008
Nov 2008
Dec 2008
Jan 2009
Feb 2009
March 2009
April 2009
May 2009
June 2009
July 2009
Aug 2009
Sept 2009
Country 1
4
3
2
2
4
2
1
2
3
1
1
33 3
3
2
1
3
2
1
1
3
2
11
2
1111
1
1
11
1
2
Oct 2007
Nov 2007
Dec 2007
Jan 2008
Feb 2008
March 2008
April 2008
May 2008
June 2008
July 2008
Aug 2008
Sept 2008
Oct 2008
Nov 2008
Dec 2008
Jan 2009
Feb 2009
March 2009
April 2009
May 2009
June 2009
July 2009
Aug 2009
Sept 2009
2
Oct 2007
Nov 2007
Dec 2007
Jan 2008
Feb 2008
March 2008
April 2008
May 2008
June 2008
July 2008
Aug 2008
Sept 2008
Oct 2008
Nov 2008
Dec 2008
Jan 2009
Feb 2009
March 2009
April 2009
May 2009
June 2009
July 2009
Aug 2009
Sept 2009
6
Oct 2007
Nov 2007
Dec 2007
Jan 2008
Feb 2008
March 2008
April 2008
May 2008
June 2008
July 2008
Aug 2008
Sept 2008
Oct 2008
Nov 2008
Dec 2008
Jan 2009
Feb 2009
March 2009
April 2009
May 2009
June 2009
July 2009
Aug 2009
Sept 2009
0
Oct 2007
Nov 2007
Dec 2007
Jan 2008
Feb 2008
March 2008
April 2008
May 2008
June 2008
July 2008
Aug 2008
Sept 2008
Oct 2008
Nov 2008
Dec 2008
Jan 2009
Feb 2009
March 2009
April 2009
May 2009
June 2009
July 2009
Aug 2009
Sept 2009
4
Oct 2007
Nov 2007
Dec 2007
Jan 2008
Feb 2008
March 2008
April 2008
May 2008
June 2008
July 2008
Aug 2008
Sept 2008
Oct 2008
Nov 2008
Dec 2008
Jan 2009
Feb 2009
March 2009
April 2009
May 2009
June 2009
July 2009
Aug 2009
Sept 2009
Accrual by Centre
Country 2
Country 3
10
8
6
4
4
5
3
2 2
3
222
11
2 2
11 1 11
3
3
11
2
11
Graphs by centre
graph bar var, by(centre) over(month var, label(labsize(vsmall) angle(ver))) ytitle("number of patients
accrued") ylabel(0(2)10) ylabel(,angle(hor)) blabel(total, gap(.25)) title(“Accrual by Centre”)
1
Country 4
Country 5
Country 6
10
8
6
3
4
2
1
Monitoring Randomization
• We must ensure that randomization
processes are working
– Require by the end of accrual that there are
equal number of patients on each treatment
arm of the study
– Requires that patients are being placed at an
equal rate on the study arms – real time
randomization
Arm 1
bihist var, by(var)
Date of Randomization
Arm 2
01oct2009
01sep2009
01aug2009
01jul2009
01jun2009
01may2009
01apr2009
01mar2009
01feb2009
01jan2009
01dec2008
01nov2008
01oct2008
01sep2008
01aug2008
01jul2008
01jun2008
01may2008
01apr2008
01mar2008
01feb2008
01jan2008
01dec2007
01nov2007
01oct2007
8
6
4
2
0
2
4
6
8
Two-Arm Trial
Jul-05
Sep-05
Nov-05
Jan-06
Mar-06
May-06
Jul-06
Sep-06
Nov-06
Jan-07
Mar-07
May-07
Jul-07
Sep-07
Nov-07
Jan-08
Mar-08
May-08
Jul-08
Sep-08
Nov-08
Jan-09
Mar-09
May-09
Jul-09
Sep-09
Nov-09
Multi-Arm Trial
1
.8
.6
.4
.2
0
DOrand
Arm 1
Arm 2
Arm 3
cdfplot var, by(txarm) legend(col(4)) xlabel(,angle(ver))
Arm 4
2. Monitoring Survival
Ethical requirement to monitor patterns of
mortality over the span of the study
Monitoring Survival
• Kaplan-Meier plots allow us to estimate event
rates over time
– We create a “survival” variable with date of event or
last follow-up minus the date of randomization
gen SURV=date of last follow-up – date of randomization
stset SURV, failure(death==1)
• Stata allows for graphical demonstration of
survival estimates for different baseline groups
(different disease, stage of disease etc) especially
important for sample size
sts graph, cim censored(single)
risktable(,title(Risk)) ylabel(,angle(hor))
ylabel(#10) xlabel(#20) xtitle("days from
randomization")xlabel(,angle(ver))
Kaplan-Meier survival estimate
450
500
550
600
400
350
300
250
200
150
100
50
0
1
.9
.8
.7
.6
.5
.4
.3
.2
.1
0
6
3
1
0
days from randomization
Risk
110
88
62
41
32
25
95% CI
18
13
9
Survivor function
sts graph, cim censored(single)
risktable(,title(Risk)) ylabel(,angle(hor))
ylabel(#10) xlabel(#20) xtitle("days from
randomization")xlabel(,angle(ver))
Kaplan-Meier survival estimates
1.00
0.90
0.80
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0
20
40
60
80
100
120
140
160
180
200
220
240
260
280
300
320
340
360
380
400
420
440
460
480
500
520
540
560
580
1
days from randomization
Risk
M0:
M1:
69 66 62 57 52 46 41 34 31 26 25 23 21 20 18 15 14 12 11 10 8
41 36 34 28 20 16 13 9 8 7 7 7 5 4 3 3 3 2 2 1 1
Disease 1
8
1
7
1
5
1
Disease 2
3
1
2
1
0
1
0
1
0
1
0
0
3. Clinical Trajectory
Demonstrates the extent to which
protocol-required activities are properly
ordered and on time
Demonstrates the homogeneity or patterns
in clinical trajectories and identifies clear
outliers (protocol violation, patient choice)
Clinical Trajectory
• In Stata, graphic representations of clinical
trajectories can be created by plotting dates of
interest:
1.
2.
3.
4.
5.
6.
Date of diagnosis
Dates of imaging/scans/biopsies
Date of surgery
Date of randomization
Dates of treatment
Dates of follow ups
• Plot deceased and living patients separately
Clinical Trajectories of Deceased Patients
Oct-09
Jul-09
Apr-09
Jan-09
Oct-08
Jul-08
Apr-08
Jan-08
Oct-07
Jun-09
May-09
Apr-09
Feb-09
Mar-09
Jan-09
Dec-08
Nov-08
Oct-08
Sep-08
Aug-08
Jul-08
Jun-08
May-08
Apr-08
Mar-08
Feb-08
Jan-08
Dec-07
Nov-07
Oct-07
Jul-07
date of randomization
Follow-up
2nd Treatment
CT scan
3rd Treatment
Biopsy
Randomization
1st Treatment
Death
twoway scatter var var var var var var var var if deathflag==1, xti("date of randomization")yti("") xlabel(,angle(ver))
ylabel(,angle(hor)) legend(col(4)) ti("Clinical Trajectories of Deceased Patients") || scatter var var if status==1,
msymbol(x) mcolor(gold) legend(lab(8 "Death")) xlabel(#20) ylabel(#10)
Country 1
Country 2
Country 3
Country 4
Country 5
Country 6
Oct-09
Jul-09
Apr-09
Jan-09
Oct-08
Jul-08
Apr-08
Jan-08
Oct-07
Jul-07
Oct-07
Nov-07
Dec-07
Jan-08
Feb-08
Mar-08
Apr-08
May-08
Jun-08
Jul-08
Aug-08
Sep-08
Oct-08
Nov-08
Dec-08
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Oct-07
Nov-07
Dec-07
Jan-08
Feb-08
Mar-08
Apr-08
May-08
Jun-08
Jul-08
Aug-08
Sep-08
Oct-08
Nov-08
Dec-08
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Oct-07
Nov-07
Dec-07
Jan-08
Feb-08
Mar-08
Apr-08
May-08
Jun-08
Jul-08
Aug-08
Sep-08
Oct-08
Nov-08
Dec-08
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Oct-09
Jul-09
Apr-09
Jan-09
Oct-08
Jul-08
Apr-08
Jan-08
Oct-07
Jul-07
date of randomization
follow-up
treatment 1
randomization
treatment 2
CT scan
treatment 3
biopsy
death
Graphs by centre
twoway scatter var var var var var var var if death==1 , xti("date of randomization")yti("")
xlabel(,angle(ver)) ylabel(,angle(hor)) legend(col(4)) || scatter var var, msymbol(x) mcolor(gold)
legend(lab(8 “death")) xlabel(#20) ylabel(#10) by(centre)
Scatter Plots for Living Patients
Clinical Trajectories
Jan-08
Feb-08
Mar-08
Apr-08
May-08
Jun-08
Jul-08
Aug-08
Sep-08
Oct-08
Nov-08
Dec-08
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09
Aug-09
Sep-09
Oct-09
Nov-09
Oct-09
Aug-09
Jun-09
Apr-09
Feb-09
Dec-08
Oct-08
Aug-08
Jun-08
Apr-08
Feb-08
Dec-07
Date of Randomization
Follow Up
CT scan
Treatment1
Biopsy
Treatment 2
Randomization
Treatment 3
twoway scatter var var var var var var var, msize(small) xti("Date of Randomization") xlabel(,angle(ver))
ylabel(,angle(hor)) legend(col(4) ti("Clinical Trajectories")yline(18048)xlabe(#25)ylabel(#10)
4. Tracking Follow-Up
Identify patients that are at risk of
being lost to follow-up
Follow-up is required by protocol in
specific intervals
Ensuring Regular Follow-Up
• Each month we are interested in monitoring
the date of last follow-up for each patient to
determine whether follow-up submission is up
to date
– We can use a strip plot separated by country to
show the date of last follow-up for each patient
– Dot plots can be used to isolate patients based on
follow-up date (mlabel)
– Goal is to inform Investigators about the specific
cases that are missing follow-up
• Allows synchronization of records
DOlfu
Country 1
Country 2
Country 3
Country 4
Country 5
Country 6
Country 7
Country 8
Country 9
stripplot var, over(var) xlabel(,angle(ver)) xline(18110) separate(centre) legend(col(3))
Oct-09
Sep-09
Aug-09
Jul-09
Jun-09
May-09
Apr-09
Feb-09
Mar-09
Jan-09
Dec-08
Nov-08
Oct-08
Sep-08
Aug-08
Jul-08
Jun-08
May-08
Apr-08
Mar-08
Feb-08
Jan-08
Dec-07
Nov-07
Oct-07
Strip Plot: Follow-Up
Dot Plots
SA030
CH034
Sep-09
IN040
CR015
SA027
CH006
SA023
CH032
CR011
CH010
CH012
CH030
CH019
CH017
CH031
CR013
CH020
SA025
CH028
CR003
CR007
CH027
CH025
CH033
SA031
SA026
SA033
SA028
SA032
SA034
SA029
CH008
TL005
CH026
TL006
Aug-09
CH029
IN038
IC003
SA015
IN037
IN036
IN039
Jul-09
IN035
IN034
IN031
IN032
IN033
IN030
CH001
IC002
CH021
Jun-09
IN027
INO16
IN029
IN018
IN012
May-09
IN015
IN024
CR010
CR012
CR009
IN025
dotplot var, ylabel(,angle(hor)) ylabel(#5) mlabel(patid) mlabsize(tiny) xscale(off) yline(18110)
Strategic Monitoring of Follow-up
• Scatter plots and strip plots give a visual clue
to the proportion of patients that are out of
date for follow-up
• A more sophisticated look at combined followup for the whole trial allows us to determine
the proportion exactly
– Plotting cumulative date of last follow-up with
CDF plots estimates permanent loss to follow-up
Ideal Cumulative Follow-up
Cumulative Follow-up
1
.9
.8
.7
.5
.4
.3
.2
.1
date of last follow-up
Sep-09
Aug-09
Jul-09
Jun-09
May-09
0
Apr-09
%
.6
CDF Plot - Cumulative Follow-up
Cumulative Follow-up
1
.9
.8
.7
%
.6
.5
.4
Line to denote
the time since
which follow-up
forms should
have been
received
.3
.2
.1
Nov-07
Dec-07
Jan-08
Feb-08
Mar-08
Apr-08
May-08
Jun-08
Jul-08
Aug-08
Sep-08
Oct-08
Nov-08
Dec-08
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09
Aug-09
Sep-09
Oct-09
Nov-09
0
date of last follow-up
cdfplot var, ylabel(#10) xlabe(#5) xlabel(,angle(ver)) xline(18130) xtitle(date of last follow-up) ytitle (%) legend(col(3))
Determining Contribution
• CDF plots can be made by centre to show how
each centre is contributing to the global CDF
• Then scatter plots can also be modified to list
specific patients that are behind on follow-up
– We generate cumulative date variables
cumul var, gen(newvar)
and scatter these dates to create a “Scatter CDF
plot”
– By plotting only those patients that are overdue
for follow-up we can “zoom in” on the CDF plot to
identify the specific patients that are contributing
to sections 1 or 2 of the overall CDF plot.
Cumulative Follow-Up by Centre
Cumulative Follow-up
Nov-07
Dec-07
Jan-08
Feb-08
Mar-08
Apr-08
May-08
Jun-08
Jul-08
Aug-08
Sep-08
Oct-08
Nov-08
Dec-08
Jan-09
Feb-09
Mar-09
Apr-09
May-09
Jun-09
Jul-09
Aug-09
Sep-09
Oct-09
Nov-09
%
1
.9
.8
.7
.6
.5
.4
.3
.2
.1
0
date of last follow-up
Country 1
Country 3
Country 5
Country 7
Country 2
Country 4
Countrry 6
Country 8
Country 9
cdfplot var, by(centre) ylabel(#10) xlabe(#5) xlabel(,angle(ver)) xline(18110) xtitle(date of last follow-up) ytitle (%) legend(col(3))
Scatter CDF
.7
TL006
IN038
.65
SA015
IN036
IN039
IN035
.6
IN034
IN033
IN032
IN031
CH001
.55
IN027
CH021
IN030
INO16
IN029
IN018
.5
IN012
CR010
.45
IN037
CR012
CR009
IN015
IN024
IN025
Aug-09
Jul-09
Jun-09
May-09
Apr-09
.4
date of last follow-up
Country 1
Country 2
Country 3
Country 4
Country 5 Country 6
twoway (scatter cdf var if centre==1, mlabel(patid)) (scatter cdf var if centre==2, mlabel(patid)) (scatter cdf var if centre==3, mlabel(patid))
(scatter cdf var if centre==4, mlabel(patid)) (scatter cdf var if centre==5, mlabel(patid)) (scatter cdf var if centre==6, xlabel(,angle(ver))
legend(col(6)) xlabel(#5) ylabel(,angle(hor)) mlabel(patid))
Follow-Up Over Time
• We can merge serial .dta files (just key
variables) to enable simultaneous plotting of
each months CDF for dates of last follow-up in
patients that are/were alive
• This shows the structure of follow-up across
the months of the study
CDF Plot-Merged Over Time
Oct-09
Sep-09
Aug-09
Jul-09
Jun-09
May-09
Apr-09
Mar-09
Feb-09
Jan-09
Dec-08
Nov-08
Oct-08
Sep-08
Aug-08
Jul-08
Jun-08
May-08
1
.9
.8
.7
.6
.5
.4
.3
.2
.1
0
DOlfu
Jan 2009
Feb 2009
March 2009
April 2009
May 2009
June 2009
July 2009
Aug 2009
Sept 2009
Summary
• Stata’s graphing capabilities can be used to
monitor accrual, survival and follow-up status
and other aspects of protocol adherence
• The time interval between reports can be
short
– reports can be produced efficiently using do files
• Minimum Investigator training is required to
interpret reports about the whole study and
their own performance
• The trial can be visually monitored by TO’s and
the DMC
Summary
• Rapid identification of problems by centre and
patient can lead to more timely responses
• Different patterns of problems can be
detected for which different strategies for
resolution can be pursued – this can give
direction to Investigators as to how to solve
problems
• Using Stata to visually monitor trails helps to
improve patient safety and trial quality
Questions?