EMA DRAFT GUIDELINE ON SUBGROUPS
DISCUSSION
April 2014
Kristian Windfeld, Biometrics, H. Lundbeck A/S
Scope
Guidance to assessors in interpretation of subgroup analyses and
associated regulatory decision making
Possible implications for indication and inclusion of data in clinical
studies section of SmPC
“Enjoy the result you have found by exploratory
data analysis, for you will not find it again”
Confuseus
NOT FOR PROMOTIONALUSE
2
Highlights of draft guideline
Frame for discussing and prioritising subgroups in protocol
Focus on exploratory subgroups rather than those part of
confirmatory test strategy
Heterogeneity of target population
Methodological challenges (false positives/negatives)
Credibility of subgroup findings
Plausibility
Pre-specification
Replication
Common scenarios
Overall positive study with adverse subgroup findings
Overall borderline/negative study, possible ID of subgroup with positive
benefit/risk
NOT FOR PROMOTIONALUSE
3
Definition of subgroups
Based on pre-randomization intrinsic/extrinsic factors
Factor types
Unordered categorical (e.g. region)
Ordered categorical (e.g. disease severity)
Based on continuous measures
Usually subgroups based on single factor. Combinations may be of interest
Risk scores
Based on biological measures – possible misclassification
Discussion:
How to pre-specify and justify cutpoints for continuous measures? Should you
use cutpoints in the first place for continuous variables?
Is it reasonable to study subgroups based on one factor at a time? When/how to
study factor combinations?
NOT FOR PROMOTIONALUSE
4
The multiplicity problem
Multiple testing problem
Risk of false ”positives” is recognized – but no excuse for not investigating
subgroups…
Risk of approving drug in subgroups not benefitting also important
”Cautionary principle”: replicated evidence cannot be required to confirm
credibility of an untoward effect of the experimental treatment
Effective in subgroup
Yes
No
Yes
Decision to approve
in subgroup
No
E1
E2
Discussion:
How can we achieve adequate protection against false ‘positive’ findings while
satisfying the need to study homogeneity of effect in population?
What do you think about prioritization of sensitivity of the investigation (by not
adjusting for multiplicity) and the credibility considerations provided to ensure
specificity of the approach? Is there a reasonable balance?
NOT FOR PROMOTIONALUSE
5
Statistical methods
Statistical interaction tests appear not to be encouraged (said not to be well
understood…)
(unadjusted) p-values + plots with estimates and CIs (e.g. Forest plots)
Visual inspection of Forest plots; some guidance text about when to be concerned given
(Subgroups with CIs 2x or 3x width of overall effect that are not overlapping with overall effect
CI…)
Shrunk estimates may be used to reduce the problem of extreme random subgroup
findings by chance
P-value adjustment not recommended because the subgroup analyses are triggers
for further investigation
Commonly used scale versus scale relevant for B-R decision (relative vs absolute)
Discussion:
What is your opinion about analyzing the same outcome on both relative and absolute scale?
(Statistical versus regulatory (B/R) considerations…)
How do you find the interpretability of unadjusted p-values/CIs in Forest-plots? How should
difference between plausible vs exploratory subgroups be addressed?
Do you see a role for ‘data mining’ methods such as recursive partitioning, lasso, etc.?
NOT FOR PROMOTIONALUSE
6
Prioritising the exploratory analyses
Addressing multiplicity by prioritization:
Key subgroup (used for stratification, plausibility)
Truly exploratory (demographic, disease characteristics,…)
Maximize a priori discussion to minimize a posteriori discussion…
Potential disincentive for sponsor to plan subgroup analyses,
arguing for non-plausibility
Discussion:
Why should a sponsor pre-specify and prioritize potentially prognostic and
predictive factors with a potential label restriction as a consequence? Is it better
to not pre-specify?
How do you see the relevance of multiplicity correction for the plausible factor
based subgroup investigations?
NOT FOR PROMOTIONALUSE
7
Credibility of subgroup findings
Key considerations regarding credibility of subgroup findings:
Plausibility (+pre-specification)
Replication
Possibility to look at ≥2 sources of evidence better than a pooled estimate(!)
Sponsor may use lack of pre-specification to argue for lack of credibility –
not accepted (cf sponsor incentive discussion…)
Discussion:
How do you see the apparent preference for doing subgroup analyses by study
rather than pooled, given the replication consideration?
NOT FOR PROMOTIONALUSE
8
DSBS consolidated comments
We plan to submit DSBS consolidated comments via EFPIS
Please email your (company/group consolidated) comments to
krwi@lundbeck.com by 30 April 2014
EFSPI deadline: April/May
EMA deadline: 31 July 2014
NOT FOR PROMOTIONALUSE
9