Efficacy Data in regulatory settings, DSBS January, May
2013
H. Lundbeck A/S
Outline
Part 1: Objectives and Endpoints in test strategies
Part 2:
– Integrated Data Analysis: Purpose,
Requirements, Terminology
– Methodology for Pooled and Meta Analysis
– Applications to filing of Vortioxetine
H. Lundbeck A/S
2
Part 1: Endpoints in RCTs
Secondary Endpoints are Increasingly important
for differentiation of products
•
•
•
highly competitive markets
demands from authorities
Publishing on clinicaltrials.gov
H. Lundbeck A/S
3
Definitions of Endpoints in RCTs:
’Good old Days’: Primary, Secondary and Exploratory
Now:
H. Lundbeck A/S
Primary:
More or less as before
Secondary:
Key Secondaries
Other Secondaries
Exploratory:
perhaps bigger than before
4
Regulatory view
Primary Endpoint: Multiplicity control in case of e.g. several
doses
Key Secondary Endpoints should be under proper
multiplicity control together with the primary and can
potentially be included in labelling text and promotional
material. Will normally require significant primary
Other Secondary Endpoints can (normally) not be included
in labelling text but have to go on ’www.clinicaltrials.gov’
Exploratory endpoints can (normally) not be included in
labelling text but does not have to go on
’www.clinicaltrials.gov’
- Unclear whether secondary analyses have to go on .gov
H. Lundbeck A/S
5
Authority Requirements to
protocols and SAPs (EMA+FDA)
•
Clinical formulation of objectives
•
Clear correspondence between objectives and endpoints
Testing Strategy
•
Primary and Key secondaries should be selected based on
‘Objectives’
•
Only one endpoint per objective. No redundancy
•
Only one analysis method (population) per endpoint
H. Lundbeck A/S
6
Objectives and Endpoints
Objective
Endpoint
Analysis Methodology
Similar for other objectives.
Select one row within each objective
Often a mix is seen in protocols
H. Lundbeck A/S
7
Primary Analysis
Objective
Endpoint
Analysis Methodology
: Secondary analysis method of primary endpoint adressing primary objective
Can not be used as key secondary
H. Lundbeck A/S
8
Key Secondary Analysis I
Objective
H. Lundbeck A/S
Endpoint
Analysis Methodology
9
Key Secondary Analysis II
Objective
H. Lundbeck A/S
Endpoint
Analysis Methodology
10
Example: Depression Study
Primary Objective:
Evaluate the efficacy of LuAA21004 compared to Placebo on depressive
symptoms (in patients with MDD).
Key Secondary Objectives:
Evaluate the efficacy of LuAA21004 compared to Placebo on
1.
2.
3.
Global Status
Functioning
Anxiety
Assessments/endpoints
adressing objective
H. Lundbeck A/S
MADRS, HAM-D (Response,Remission)
CGI-S, CGI-I (Response, Remission)
SDS, work/family/social/total
HAM-A, HAM-D Subscale
11
Hierarchical Testing
MADRS
Depression
Global Status
CGI-I
Functionality
work/social/family
SDS
HAM-A
Anxiety
One endpoint per objective
Two doses: alfa=2.5% in each sequence
H. Lundbeck A/S
12
Primary Objective
Objective
Endpoint
Analysis Methodology
OC
LOCF
Depression
MADRS
MMRM
HAM-D
Non-par
Response
Remission
Response/Remission considered redundant, not a separate objective.
However, special interest in EU
H. Lundbeck A/S
13
Response and Remission
Response and Remission:
– attractive for profiling
– attractive for pricing
– difficult to formulate as separate objective
EMA: Particular Clinical Relevance + Redundant
FDA: Arbitrary and Inadequate Definition
+ Redundant
H. Lundbeck A/S
14
EMA: Responders
MADRS
MADRS 50% Response
CGI-I
MADRS Remission
SDS
”Branching”, overall α>5%
Confirming Clinical Relevance
HAM-A
-
H. Lundbeck A/S
proceed as long as p<0.05
15
Number of Key Secondary Endpoints
– no formal requirement or limitations
– limited through non-redundancy within and
between objectives
– ’Rule of thumb’: 4-5 tests within each dose
– chose hierarchi according importance and
’hit-likelihood’
– status of non-tested endpoints can be
unclear
H. Lundbeck A/S
16
Testing Strategy
– How to report p-values outside testing strategy or after
stop within sequence ?
– Phrasing ’Statistical Significance’ should be reserved for
results from testing strategy
Tip: Phrasing ’seperated from placebo’ has been introduced
in accepted Lundbeck publications and in filing documents.
Other possible phrasings:
H. Lundbeck A/S
Nominal significance
Nominal p<0.05
Nominal evidence
Potential significance
17
Part II
Integrated Data Analyses in
Regulatory Setting
H. Lundbeck A/S
18
Integrated Data Analyses
When a clinical development program enters
registration phase a need for integrated analyses arises:
• Regulatory requirements
• Questions during approval phase
• Profiling after approval
H. Lundbeck A/S
19
Terminology
Integrated Data Analysis
Pooled Analysis
Meta Analysis
H. Lundbeck A/S
20
Terminology
Definitions of Meta-Analysis:
FDA:
Meta-analysis refers to the analysis of analyses...the statistical analysis of
a large collection of analysis results from individual studies for the purpose
of integrating the findings. (Glass, 1976) Examples of related terms used in
literature include: analysis of combined data, combined analysis, analysis of
pooled data, and pooled analysis. No matter what term is used, the objective
is to use appropriately sound methods when formulating an integrated
analysis.
ICH E9 + EMA:
The formal evaluation of the quantitative evidence from two or more trials
bearing on the same question. This most commonly involves the statistical
combination of summary statistics from the various trials, but the term is
sometimes also used to refer to the combination of the raw data.
H. Lundbeck A/S
21
Meta-analysis Definition (google)
•
Statistical solutions Software: Meta-analysis is a statistical
technique in which the results of two or more studies are
mathematically combined in order to improve the reliability of the
results. Studies chosen for inclusion in a meta-analysis must be
sufficiently similar in a number of characteristics in order to accurately
combine their results. When the treatment effect (or effect size) is
consistent from one study to the next, meta-analysis can be used to
identify this common effect. When the effect varies from one study to
the next, meta-analysis may be used to identify the reason for the
variation
•
Wikipaedia: In statisitics, a meta-analysis refers to methods
focused on contrasting and combining results from different studies,
in the hope of identifying patterns among study results, sources of
disagreement among those results, or other interesting relationships
that may come to light in the context of multiple studies
H. Lundbeck A/S
22
Terminology
Integrated Data Analysis
Meta Analysis
Pooled Analysis
H. Lundbeck A/S
Meta Analysis
23
Terminology
Integrated Data Analysis
Pooled Analysis
Pooled Analysis
H. Lundbeck A/S
Meta Analysis
24
Terminology in this presentation
Integrated Data Analysis
Pooled Analysis
H. Lundbeck A/S
Meta Analysis
25
Terminology
Meta Analysis (AD):
A specific statistical methodology based on
summary statistics or aggregate data from each
trial (AD Meta Analysis)
Pooled Analysis (IPD):
Statistical analysis based on data pooling at
individual patient data level, that is, combination of
raw data. (IPD Meta Analysis)
H. Lundbeck A/S
26
Pre-requisities for Integrated analyses
Similarity of Studies with respect to
• Clinical endpoints
• Study designs
• Populations
H. Lundbeck A/S
27
FDA: ISE requirements
• Integrated summary demonstating substantial
eveidence of effectivenes
• Evidence to support recommended dosing in
labelling
• Analyses in subgroups: Sex, age, race
• Dosing in specific subgroups
- So, actually no specific demand for integrated
analysis, could just be side by side presentation
H. Lundbeck A/S
28
EMA: Points to consider on Meta-Analysis
• Not a requirement
• Cannot normally serve a primary
• Cannot save individual negative studies
• Needs prespecification
H. Lundbeck A/S
29
EMA: Pre-requisits for acceptance of results
from Meta-analysis as pivotal evidence
Pre-specification
• Statistical Methodology
• Arguments for Inclusion and exclusion of
studies
• Plan for evaluation of robustness of
results: subgroup, subsets of studies etc..
• Populations
H. Lundbeck A/S
30
EMA: Accepted Purposes of Meta-Analyses
for supportive evidence
• Precise estimate of treatment effects
• Confirm effect in subgroups
• Secondary outcomes requiring more power
• Evaluate dose-response
• Evaluate conflicting study results
H. Lundbeck A/S
- similar to FDA ISE
31
Pooled Analysis
H. Lundbeck A/S
32
Properties of Pooled Analysis
Intuitively attractive using individual patient data
Flexibility in having original data (subgroups, outliers etc.)
Complex statistical modelling possible/necessary
Assumptions on variability, baseline dependence, sites etc.
Heterogeneity not straightforward
Risk of getting meaningless comparisons
Design and convergence issues when using MMRM
Not really recommended by FDA?:
Correspondence in Relation to AA21004:
’pooling on patient level is in general not recommended’
H. Lundbeck A/S
33
AA21004 Data Package for MDD
8 Studies for Major Depressive Disorder
Differences between Studies:
1.
2.
3.
4.
5.
6.
7.
8.
Duration, 6-week 8-week
Doses: 1, 2.5, 5, 10, 15 ,20
Primary endpoint: MADRS, HAMD-24
Method for primary (ANCOVA LOCF, MMRM, nominal/window)
Test Strategy (step-down/alfa-split)
Differences in key secondaries: SDS, Response, CGI
Region
Results
H. Lundbeck A/S
35
AA21004/Vortioxetine Studies for MDD
Short-Term
Long-Term
HLu
11492,
PoC
HLu
11984,
DF
TAK
305
HLu
13267
TAK
315
TAK
316
TAK
303
TAK
304
Hlu
12541,
Elderly
HLu 11985,
Relapse
prevention
6 weeks
8 weeks
8 weeks
8 weeks
8 weeks
8 weeks
6 weeks
8 weeks
8 weeks
OL: 12 weeks
DB: 24-64 weeks
PBO
PBO
2.5 mg
5 mg
10 mg
PBO
1 mg
5 mg
10 mg
PBO
PBO
PBO
PBO
PBO
2.5 mg
5 mg
PBO
PBO
5 mg
5 mg
10 mg
60 mg
DUL (AR)
60 mg
DUL (AR)
5 mg
10 mg
5 mg
10mg
15mg
20mg
15mg
20mg
60 mg
DUL (AR)
60 mg
DUL (AR)
20mg
225 mg
VEN (AR)
60 mg
DUL (AR)
EU/Asia
/CA
EU/Asia
/CA
EU/ZA
/Asia
EU/ZA
US
US
US
US
EU
/CA/US
EU/CA/Asia
Positive
study
Failed
study, but
supportive
Positive
study
Positive
study
Positive
study
Positive
study
Failed/nega
-tive study
Negative
study
Positive
study
Positive study
(MADRS,
LOCF)
(MADRS,
LOCF)
(HAM-D24,
MMRM)
(MADRS,
MMRM)
(MADRS,
MMRM)
(MADRS,
MMRM)
(HAM-D24,
LOCF)
(HAM-D24,
LOCF)
(HAM-D24,
LOCF)
(Time to relapse)
H. Lundbeck A/S
36
Methodology for Pooled Analysis
Example: Standard ANCOVA model
MADRS, LOCF, Week 8
Alternatives (SAS):
1.
model MADRS_DL = MADRS_BL ARMCD; (Naive)
2.
model MADRS_DL = MADRS_BL ARMCD STUDY;
3.
model MADRS_DL = MADRS_BL ARMCD SITEID(STUDY);
4.
model MADRS_DL = MADRS_BL*STUDY ARMCD SITEID(STUDY);
Further Modelling:
H. Lundbeck A/S
Random STUDY*ARMCD; Random Treatment*Study Effect
Repeated group=study; Heterogeneous Variability
37
Example: Treatment versus Placebo
TTreatment Arm only in one Study
Original Study
Estimate
-6.42
S.E
1.36
P-value
<0.0001
-4.60
1.11
<0.0001
N: 112 versus 105
Pooled Analysis
N: 112 versus 1290
- substantial difference in estimate
H. Lundbeck A/S
38
Interpretation of Pooled
Methodology
Misleading Estimates
MMRM design and convergence problems
Modelling does not seem to account for all study differences
A lot of effort can be done to make the pooled analysis do
what the meta-analysis seems to do automatically
Seems not to be the best choice for AA21, but was used for
small subgroups
H. Lundbeck A/S
39
Meta Analysis
H. Lundbeck A/S
40
Properties of Meta Analysis
Analysis of analyses
Original data not needed
(survey setting not so relevant for AA21)
Only relevant comparisons are retained
Works on any treatment estimate (+/-SE)
logistic regression, Cox, ANCOVA, SES
Well-established method for heterogeneity
Less powerfull ?
Pairwise Comparisons mainly
H. Lundbeck A/S
41
Methodology for Meta Analysis
Trials: i=1,…,k
Fixed Effects Modelling:
ai = true treatment effect in trial i
âi= estimated treatment effect in trial i
vi = variance of âi
wi = 1/vi, weights
Estimated effect: â = Σwiâi/Σwi
Variance of estimate: 1/Σwi
Test H0: ai=0:
(Σwiâi)2/Σwi ~ ϰ2(1)
ref: Encyclopaedia of Biostats. page 2570-2578
H. Lundbeck A/S
42
Der Simonian (1976)
Methodology for Meta Analysis with
heterogeneity: Random Effects
Test for heterogeneity of effects
Q = Σwi(âi – â)2 ~ ϰ2(k-1)
I2=max(0,100*(Q-k)/Q)
I2 describes the percentage of total variation across studies that is due to heterogeneity rather than chance (ref: Higgins, 2007)
>50% considered problematic
Random effects in case of Heterogeneity:
ai ~ N(a*,σ2) , σ2 estimated using Q
wi*= 1/(vi+σ2)
â*= Σwi*âi/Σwi*
Variance of estimate: 1/Σwi*
Test: (Σwi*âi)2/Σwi* ~ ϰ2(1)
H. Lundbeck A/S
43
(ref: Der Simonian)
Meta Analyses in SAS
PROC MEANS;
H. Lundbeck A/S
44
Plan for Meta Analyses on AA21004
for regulatory purposes
• ‘Prespecification’ in separate SAP
• To be shown in 2.7.3
• Applied for sub-groups: gender, baseline severity
• Pooled analyses for small subgroups
- not all studies finalised at planning stage
H. Lundbeck A/S
45
Preliminary Meta Analysis without two nonfinalised studies. Differences to Placebo
Removed for confidentiality reasons
Dose Response ?
10 better than 5 ?
Fixed or Random ?
H. Lundbeck A/S
46
Meta Analysis with all studies
Removed for confidentiality reasons
Dose Response ?
15 mg ?
H. Lundbeck A/S
47
Meta Analysis Summary
Differences to Placebo
Removed for confidentiality reasons
H. Lundbeck A/S
48
Pooled versus Meta Analysis of AA21004
•
Severe heterogeneity complicates interpretations
•
Confounding with Region US/Non-US
•
For both analysis types it is mandatory that interpretations involving
comparions across treament arms take the individual study results into
account.
•
The random effects model has less power in the presence of heterogeneity
but estimated treament differences change only slightly. Does not solve all
heterogeneity problems.
•
Random effects not feasible in pooled MMRM, but gets close to Meta results
for LOCF
•
Neither method completely satisfactory
•
Mixed treatment comparison (MTC) meta-analysis allows several treatments
(doses) to be compared in a single analysis while utilising direct and indirect
evidence
H. Lundbeck A/S
49
Meta Analysis in the filing documents
•
Need to downplay due to severe heterogeneity
•
Demonstrate Region issue US/Non-US
•
Results across subgroups: age, bmi, gender, severity
•
Argumentation for dose
H. Lundbeck A/S
50
Planned talk at 8 January 2013
…… 4 Months Later ……..
H. Lundbeck A/S
51
Lu AA21004, Vortioxetine, Brintellix
August 2012:
September 2012:
Filing EU
Filing US
January 8:
January 17:
Planned Talk at DSBS
Day 120 Q’s received
April 15:
Day 120 Q’s answered
June 7:
Day 180 Q’s
H. Lundbeck A/S
52
Day 120 Questions
• Focus on US versus non-US
• No focus on pool- versus meta- approach
• Some value of meta-analyses in terms of dosing and subgroup
arguments
• Testing strategy issues only in relation to PROs
H. Lundbeck A/S
53
Guidelines
EMA: Points to consider on applications with metaanalysis, (2001)
FDA: Guidance for industry. Integrated Summary
of Effectivenesss ISE (2008)
ICH E9: Statistical Principles for Clinical Trials
H. Lundbeck A/S
54