EXAMINING THE RELIABILITY AND
USEFULLNESS OF INTERIM ANALYSIS
DATA: CASE OF ANTIPSYCHOTIC DRUG
TRIALS
Jonathan Rabinowitz (1), Nomi Werbeloff (1), Stephen
Levine (2)
(1) Bar Ilan University, Ramat Gan, Israel
(2) Haifa University
2
What is NEWMEDS?
NEWMEDS is
an international consortium of scientists that has launched one
of the largest ever research academic-industry collaboration
projects
to find new methods for the development of drugs for
schizophrenia and depression.
EFPIA companies: H Lundbeck A/S, Abbott, AstraZeneca AB, Eli Lilly and
Company Ltd, Janssen Pharmaceutica NV, Novartis Pharma AG, Orion
Corporation, Pfizer Limited, F. Hoffmann-La Roche AG, Institut de
Recherches Servier
Universities: King’s College London (UK), Karolinska Institutet (Sweden), The
University of Cambridge (UK), Central Institute of Mental Health (Germany),
CSIC (Spain), The University of Manchester (UK), Bar Ilan University
(Israel)
SME‘s
Psynova Neurotech Ltd (UK), deCODE genetics (Iceland), GABO:mi (Germany)
3
Who supports NEWMEDS?
NEWMEDS is funded by the Innovative Medicines Initiative
(IMI), a unique large scale public-private partnership
between the European Union (represented by the
European Commission) and the pharmaceutical industry
(represented by the European Federation of
Pharmaceutical Industries and Associations, EFPIA).
IMI aims to put Europe at the forefront of
biopharmaceutical innovation and to support more
efficient discovery and development of better medicines
for patients.
www.imi.europa.eu
4
The Consortium
5
Key data for NEWMEDS
Start date:
1 September 2009
Runtime:
5 years
Structure:
11 Workpackages
Clusters:
System based animal models (A)
Translational Technology (B)
Patient Stratification (C)
6
Schizophrenia Database
Data from: Astra Zeneca, Janssen, Lilly, Lundbeck, Pfizer
64 Industry sponsored studies
34 placebo controlled
30 active comparator
25,900 patients
16,105 study drug
7,119 active comparator
2,676 placebo
1 NIMH sponsored study CATIE 1,493 patients
1 European Union sponsored study EUFEST 498 patients
7
Depression Database
Data from: Astra Zeneca, Lundbeck, Pfizer and Lilly
39 placebo controlled Industry sponsored studies
12,217 patients
8,260 active drug
3,957 placebo
8
Methodological question
Interim analyses yield effect size information that assist in deciding about potential
sample size adjustments.
To what extent are the results of the first cohort of patients to be enrolled similar
to the results of the subsequently enrolled patients?
In the rush to complete studies quality may decline reducing effect sizes over time.
Method: Studies split at midpoint of enrolment.
Results: Difference in placebo vs. active treatment difference in effect size (Cohen’s
d) between cohorts:
Mean
Median
Antipsychotic trials
-0.01
-0.03
Antidepressant trials
0.05
0.04
Overall the first and second cohorts enrolled yielded similar results.
9
Effect size differences placebo vs. antipsychotic treatment in first
and second cohort of patients enrolled ordered by difference:
>Negative difference greater effect first cohort, positive greater effect second cohort
>9 trials differences were positive, in 4 differences were negative, where first cohort showed
more difference these differences were small (less than -.08).
>In 7 of the 20 studies the first cohort well outperformed the second cohort (from -12 to -.44).
Effect size differences placebo vs. antidepressant treatment in
first and second cohort of patients enrolled ordered by
difference:
1.5
1.0
0.5
0.0
1
3
5
7
9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39
-0.5
-1.0
-1.5
>In 22 trials the differences were positive, in 17 the differences were negative.
>In 12 studies these differences were small (less than .08 in favor of either cohorts).
Using first cohort results to re-estimate sample size
-Repower study, as if designing a new study.
Ignores the fact that first cohort results are fixed.
-Repower second cohort based on conditional probability
Makes use of the fact that first cohort results are fixed.
Sample size needed for second cohort estimated using formula
for adding z scores.
1. Convert Cohen’s d to z = (d*sqrt(n))/2
2. Z needed for second cohort=(1.96*SQRT(n1^2+n2^2)-n1*z)/n1
(Based on formula for to sum weighted z scores.)
3. Compute P of z p=(1-NORMSDIST(z))*2
4. Compute sample size using d at time 1 and the p value from
previous step.
(a) Using power software.
(b) Resampling-bootstrapping (sample sizes increased
until obtaining needed p value).
Study ES 1,p= , n=placebo/active
ES 2, p= , n=placebo/active
Difference of ES2-ES1
Repowering study
based on first
cohort, total needed
for study and
additional patients
needed for second
cohort, if any.
ES1=.25, p=.15, n=134
(32/102)
ES2=.18, p=.30, n=137 (38/99)
Diff=-.07
Boot-strapping
Sample size calculation for second part 90%, 2-tail
90%, 2-tail
Z needed 1.31, p=.19
576-137=439
450-137=313
Z needed 1.45, p=.15
309-63=246
280-63= 217
Z needed 1.06, p=.29
508-203=305
400-203=197
Z needed 1.46, p=.14
300-55=245
270-55= 215
950 (n=230/620)
Add: 950-137=813
A
ES1=.33, p=.22, n=58 (18/40)
ES2=.08, p=.74, n=63 (25/38)
Diff=-0.25
438 (n=146/292)
Add: (438-63)=385
B
ES1=.24, p=.089, n=205
(50/155)
ES2=.35, p=.015, n=203
(48/155)
Diff=.11
976 (n=244/732)
Add: (976-203)=773
C
D
Conditional probability
ES1=.34, p=.21, n=59 (21/38)
ES2=.53, p=.06, n=55
(n=17/38)
Diff=.19
13 April 2015
411 (n=137/274)
Add: (411-55)=356
14
Conclusion
Results suggest that earlier cohorts in studies of antipsychotic
and antidepressant medications can reliably serve to estimate
trial effects in interim analyses and to efficiently adjust sample
size calculations using conditional probability.
13 April 2015
15