Predicting the Probability of
Developing a Successful
Cancer Therapy
Tony Sabin, CSG Feb 2010
Contents
 Brief Overview of Clinical Drug Development Process
– Aims of Clinical Development
– Stages of Development
– Cost
 Decision Making in the Pharmaceutical Industry
– Historical Perspective
 Bayesian Model for Predicting Success in Pancreatic Cancer
2
Aims of Clinical Development
 Stop development of poor treatments as soon as possible
–
–
–
–
avoids giving patients ineffective/unsafe therapies
avoids escalating costs of development
allocate funds to develop more promising treatments
make more patients available for promising treatments
 Decision making process should possess a
– High chance of failing a poor treatment
– Low chance of failing a good treatment
3
Clinical Development Process
Phase 1
First in
Human
FIH
Phase 3
Phase 2
Proof of
Concept
PBA
Learn
Dose
Find
Confirm
EOP2
CTF
Confirm
Filing
Launch
CTL
Launch
Post marketing
Phase 4
4
First in Human Trials
 Single centre, healthy volunteer studies
– Occasionally done in patients e.g. when potential toxicity
precludes use of normal volunteers
 Single dose safety
 Increasing doses in small cohorts based on safety
 Single dose PK - Determine absorption, distribution,
metabolism & excretion
Questions
Is it possible to dose without gross safety signals?
What is the single dose PK profile?
5
Proof of Concept Trials
 Volunteer/patient studies, few centres
 Repeat dose safety
 Repeat dose PK/PD
 Different formulations, schedules and doses
 Increasing doses in cohorts based
on safety and PD/efficacy
Questions
Is there a range of safe doses where it may be possible to
observe the intended benefit?
What is the best formulation and schedule of dosing?
Is there any sign of the intended efficacy?
6
Dose Optimization Trials
 Patient studies, more centres
 Clinical durations of treatment
 Short term biological/clinical endpoints that are good
predictors of likely phase III endpoints
 Few doses against a control (active and/or placebo)
Questions
Which dose is likely to give the best balance between efficacy
and safety?
Are we likely to be successful in a phase III?
What is the likely cost effectiveness of treatment?
7
Confirmatory Trials
 Patient studies, many centres and regions
 Clinically relevant endpoints
 One or two doses compared against a control
 Studies may take a long time to run for chronic treatments
 Prior agreement of design with regulatory authorities
Questions
Can we provide robust evidence of clinical efficacy?
Is there any evidence of ‘case identifiers’?
What is the safety profile of treatment in the target population?
Can we successfully file for marketing authorization?
8
Observational Trials
 How is the treatment actually being used in practice
 Is the treatment being used on label
 Are there patients who should be getting the treatment
9
Decision Making
Quote: If we knew what it was we were doing it
wouldn’t be called research, would it? (Albert
Einstein)
Success Rates from FIH to Registration by
Therapeutic Area
Kola and Landis. Nature Rev Drug Discov 2004;3:711.
11
Success Rates by Phase and Therapeutic Area
Kola and Landis. Nature Rev Drug Discov 2004;3:711.
12
Cost and Number of Compounds
at Each Stage
 Cost $
# Compounds
 304M
5000 – 10000
Screened in Discovery
 96M
250
Entering pre-clinical
 72M
10
Entering Phase 1
 48M
3-5
Entering Phase 2
 224M
2
Entering Phase 3
 56M
1
Regulatory submission
 800M
1
Approved and launched
Source: PhrMA and EFPIA Pharm Report 2007
13
There is a high attrition rate within drug
development
 Considerable amount of attrition in late phase development
which means incurring the majority of drug development costs.
– This is not good for R&D productivity
– This is not good for cost of medicines
 Finding ways to make better decisions earlier on in
development would be a big step in improving R&D productivity
– How can statistics help?
– Focus on the end of Phase 2 decision point
– Bayesian Model for Predicting Success in Pancreatic Cancer
14
Bayesian Approach to
Predicting Success in Phase 3
Pancreatic Cancer
Pancreatic Cancer
 Fourth leading cause of cancer-related deaths in the US
– 32,240 deaths and 42,470 new cases in the US during 2009
 Usually diagnosed at a late stage and is largely
unresponsive to current medical therapy
– Highest cure rates in people with localized disease which are
amenable to surgical resection (10-24% 5 years survival rate)
– 90% present with unresectable or metastatic disease with a 5year survival rate < 1%
 Current acceptable treatment is gemcitabine (chemotherapy)
– 1000mg/m2 once a week for 3 weeks then one weeks rest
(repeated until disease progression)
16
Pharmacology of Apoptosis
RTK
inhibitors
Bcl-2
inhibitors
Bcl2, BclXL,
Mcl1
Akt
Flip
FLIP
Akt/PI3K
inhibitors
IAP
Smac mimetics
17
PI3K
Common Endpoints in Oncology
Monitor the change in tumour size over time and classify as
progressed, stable, partial response or complete response at a time
point
•
Progression Free survival time (Time to progression or death). Compare trts
using a Hazard Ratio, difference in PFS rate at a time point
•
Objective Response Rate (% best response of CR or PR). Compare trts using
difference in proportion of responders
Overall Survival Time (Time to death). Compare trts using Hazard
ratio or the difference in OS rate at a time point
18
The Model for Decision Making at EOP2
Probability of
Success wrt
Efficacy
Project
Deliverables
Probability of
Success wrt
Safety
19
Anticipated
Competitive
Situation
Combine
Information
Anticipated
Regulatory
Situation
Anticipated
Reimbursement
Situation
GO?
NO GO?
Current Business
Situation
(Amgen)
End of Phase 2 Evidence Diagram
Literature
Control Group
Response Rate
Predictors
Time Trends
Differences
Predictors
Prior Belief
Phase 2-3
Endpoint
Relationships
Prior
20
Phase 2
Phase 3
Treatment
Treatment
R
Diff
Control
R
Diff
Control
Phase 2 Endpoint
Phase 3 Endpoint
New Data
Prediction
The competition to be first to market is fierce:
Reliable short term endpoints in P2 are a must
Preclinical
Phase I
Phase II
1
6
2
3
4
5
7
21
Phase III
Launched
Our Goal: Probability of Success in Pancreatic
Cancer
100
Probability of Success in Ph 3
90
80
70
Better than industry average
Uninformative
Sceptical
60
50
Optimistic
Light Sceptical
40
Light Optimistic
30
20
10
0
-15
-10
-5
0
5
10
15
Diff in 6m Survival Rate in Phase 2 (Test - Control)
22
20
The Process
 Step 1: Literature search and data abstraction
 Step 2: Determine the expected treatment difference for the
Phase 2 endpoint and population
– Factor in the observed results from the P2 study, prior
knowledge of the control group behaviour from the literature in
the P2 population
– Factor in prior belief on the whether the drug will work
 Step 3: Use the relationship between P2 and P3 endpoints
to calculate the treatment difference in the P3 endpoint
 Step 4: Determine the probability of success in Phase 3
– Incorporate the Phase 3 design
23
Pancreatic Cancer Example
 Conduct a randomized controlled Phase II study:
– Test+gemcitabine combination versus gemcitabine alone
– Primary Phase 2 endpoint is 6 month survival rate
– PFS originally not thought to be relevant in this disease (see
later – this process can change medical opinion)
 Proposed Frequentist Phase III study design:
– 2 arms (Test+gemcitabine, gemcitabine alone)
– Primary endpoint is OS.
– Analyze after 379 events
 Our focus is on relating the difference in 6m survival rate to
the OS hazard ratio
24
Step 1 – Pancreatic Cancer Literature Search
 Inclusion criteria
– Adult patients with locally advanced or metastatic pancreatic cancer
– All randomized controlled comparative studies that were published in
English in year 2000 or later, in which gemcitabine was used either
alone or in combination with other therapies.
 Exclusion criteria
– Studies where patients were given concurrent radiotherapy or local
regional modalities such as surgery, which might have influenced
survival
– Cross over studies where the assessment of survival times was
impaired
– Non randomized study
– Information on patient survival times was not available
25
Step 1 – Pancreatic Cancer Literature Search
 Search Method
– Studies were identified by targeting Medline, Embase, the
American Society of Clinical Oncology web site, published
meta-analyses and the internal knowledge of Amgen’s clinical
and regulatory groups.
– Studies identified were screened for inclusion by both Amgen
Biostatistics and Amgen Clinical.
26
Step 1 – Pancreatic Cancer Literature Abstraction
27
Step 1 - Literature Search results
 136 hits
 30 studies identified for detailed analysis
 22 with Gemcitabine only control
 Use this data to develop the prior inputs to our model and
check the choice of endpoint makes sense
28
Meta Analysis of Gemcitabine 6 Month Survival Rate
(P2 endpoint – metastatic + LA subjects)
Meta Analysis Forest Plot
6 Month Surv iv al (Proportion)
(Gemcitabine Control)
Study Reference
31
29
33
32
3
17b
30
18
2
1
12
14
4
7
20
16
19b
19b
15
21
9
6b
Heterogeneity
Q = 39.1 p= 0.0096
I-Sq = 46.3%
Estimate [95% CI]
0.480 [ 0.312, 0.648]
0.560 [ 0.412, 0.708]
0.600 [ 0.452, 0.748]
0.440 [ 0.293, 0.587]
0.470 [ 0.350, 0.590]
0.500 [ 0.383, 0.617]
0.570 [ 0.467, 0.673]
0.510 [ 0.409, 0.611]
0.410 [ 0.322, 0.498]
0.450 [ 0.373, 0.527]
0.600 [ 0.524, 0.676]
0.620 [ 0.544, 0.696]
0.510 [ 0.435, 0.585]
0.570 [ 0.495, 0.645]
0.520 [ 0.446, 0.594]
0.520 [ 0.456, 0.584]
0.530 [ 0.471, 0.589]
0.420 [ 0.362, 0.478]
0.520 [ 0.462, 0.578]
0.490 [ 0.432, 0.548]
0.490 [ 0.437, 0.543]
0.490 [ 0.439, 0.541]
Fixed
Random
Weight
0.9%
1.2%
1.2%
1.2%
1.8%
1.9%
2.4%
2.6%
3.3%
4.4%
4.5%
4.5%
4.6%
4.7%
4.7%
6.4%
7.5%
7.6%
7.7%
7.7%
9.4%
10.0%
0.508 [ 0.492, 0.524]
0.511 [ 0.487, 0.534]
0.2
Program: pan_group_forest.sas
Output: meta_group_m6.cgm (Date Generated: 07JAN2010: 13:48)
29
0.3
0.4
0.5
0.6
0.7
0.8
Gemcitabine 6m survival predictions for the exact
population (control priors)
Number of
Studies
Estimate
95% CI
All studies
22
0.511
(0.487, 0.534)
Metastatic Subgroups
6
0.464
(0.400, 0.514)
Metastatic Subgroups and
Studies with ≥ 90%
Metastatic Subjects
9
0.472
(0.431, 0.514)
Meta-Regression
Estimating for 100%
Metastatic Subjects
22
0.482
(0.445, 0.517)
Method
30
Do the endpoints predict survival?
SPEED Pancreatic Cancer
All Treatment Group Responses
Median Ov erall Surv iv al by Median Progression-Free Surv iv al
10
9
Median OS v
Median PFS
Median OS(Months)
8
7
6
5
4
3
2
0
3
4 Cancer
5
SPEED
Pancreatic
All TreatmentMedian
Group
PFSResponses
(Months)
Median Ov erall Surv iv al by 6 Month Surv iv al
1
2
6
7
8
The size of the circles are proportional to the number of patients.
10
Program: panc_group_plots.sas
Output: all_os_pfs.cgm (Date Generated: 17DEC2009: 11:21)
9
Median OS v
6 Month
Surv Rate
Median OS(Months)
8
7
6
5
4
3
2
0.0
0.1
0.2
0.3
0.4
0.5
6 Month Survival
The size of the circles are proportional to the number of patients.
Program: panc_group_plots.sas
Output: all_os_m6.cgm (Date Generated: 17DEC2009: 11:21)
31
0.6
0.7
0.8
0.9
1.0
2
Developing the Endpoint Relationships
 Phase 2 Endpoints
– 6 month survival rate
– PFS
 Phase 3 Endpoint
– Overall Survival
 Predict the relationship between the treatment difference in
the P2 endpoint with the treatment difference in the P3
endpoint
32
OS Hazard Ratio and 6M Survival Rate Difference
SPEED Pancreatic Cancer
Phase 2 - Phase 3 Endpoint Relationship
Ov erall Surv iv al by 6 Month Surv iv al
1.6
Overall Survival (HR)
1.4
1.2
1.0
0.8
0.6
0.4
-0.4
-0.3
-0.2
-0.1
0.0
Difference in 6 Month Survival
The size of the circles is inversely proportion to the OS SE(LnHR)
Program: panc_reln_plots.sas
Output: diff_oshr_6m.cgm (Date Generated: 14JAN2010: 14:17)
33
0.1
0.2
0.3
0.4
OS Hazard Ratio and
PFS
Hazard
Ratio
SPEED
Pancreatic
Cancer
Phase 2 - Phase 3 Endpoint Relationship
Ov erall Surv iv al by Progression-Free Surv iv al
1.6
Overall Survival (HR)
1.4
1.2
1.0
0.8
0.6
0.4
0.4
0.6
0.8
1.0
PFS (HR)
The size of the circles is inversely proportion to the OS SE(LnHR)
Program: panc_reln_plots.sas
Output: diff_oshr_pfshr.cgm (Date Generated: 14JAN2010: 14:17)
34
1.2
1.4
1.6
Do we need to adjust the HR for other predictors of OS
Hazard Ratio?
SPEED Pancreatic Cancer
Ov erall Surv iv al Hazard Ratio
by Percentage of Metastatic Subj ects
1.6
OS HR v
% Metastatic
Overall Survival (HR)
1.4
1.2
1.0
0.8
0.6
0.4
40
50
SPEED Pancreatic Cancer
70
Ov60erall Surv iv al Hazard
Ratio
by Percentage
of ECOG
0/1 Subj
ects
Percentage
of Metastatic
Subjects
80
90
100
80
90
100
1.6 of the circles is inversely proportion to the OS SE(LnHR)
The size
Program: panc_reln_plots.sas
Output: diff_oshr_meta.cgm (Date Generated: 14JAN2010: 14:17)
OS HR v
% ECOG 0/1
Overall Survival (HR)
1.4
1.2
1.0
0.8
0.6
0.4
40
50
60
70
Percentage of ECOG 0/1 Subjects
The size of the circles is inversely proportion to the OS SE(LnHR)
Program: panc_reln_plots.sas
Output: diff_oshr_ecog.cgm (Date Generated: 14JAN2010: 14:17)
35
Meta-Regression for OS Hazard Ratio
SPEED Pancreatic Cancer
Meta Regression for Ov erall Surv iv al HR
by Month 6 Difference in Surv iv al Rate
1.6
Overall Survival Hazard Ratio
1.4
1.2
1.0
0.8
0.6
0.4
-0.20
-0.15
-0.10
-0.05
0.00
0.05
0.10
0.15
Month 6 Difference
The radius of the circle is inversely proportion to SE(LHR)
Predicted Mean HR (95%CI) for a fixed Month 6 difference is shown
Program: panc_reln_model.sas
Output: mreg_os_m6diff.cgm (Date Generated: 20JAN2010: 12:10)
36
Parameter
Posterior Distribution
Mean
SD
95% Cr.Int.
b (slope)
t (random effects SD)
-1.777
0.0233
0.3419
0.01759
(-2.518, -1.164)
(0.0015, 0.0664)
0.20
Step 2: The expected difference in 6m survival
between treatments
Ph 2 RCT
Gem Lit
6m
Gem P2
6m
AMG P2
6m
This is our best
estimate of the
trt difference
Equivalent
to our Ph2
population
diff
6m
survival
Eg. If the gem P2 <
expected then the 6m rate
in both the gem and AMG
arms are reduced by the
same amount
6m
survival
37
The observed
difference is
maintained.
The variance
alters
0.2
0.0
-0.2
-0.4
Rsq=75.3%
Phase 2 result
modulated
variance for the
control behaviour
-0.8
-0.6
log Hazard Ratio (Overall Survival)
0.4
Step 3: Predicting the OS result at the EOP2 (the
posterior distribution for the OS HR)
-0.4
-0.2
0.0
38
Difference in 6 mnth Survival Rate
0.2
0.4
Step 2: The expected difference in 6m survival
between treatments
Gem Lit
6m
Gem P2
6m
AMG P2
6m
Prior Belief of
the ability of
the drug
6m
survival
Sceptical
Uninformative
Optimistic
Diff is pulled towards the prior belief
39
Prior Distributions for Phase 2 Treatment
Difference
Uninformative Prior
No prior knowledge assumed, but somewhere
between a 50% reduction and a 50% increase.
-60
-45
-30
-15
0
15
30
45
60
Phase 2 Treatment Difference
Sceptical Prior
Expect treatment difference to be zero, but a 20%
chance that difference could be > 15%.
-60
-45
-30
-15
0
15
30
45
60
Phase 2 Treatment Difference
Optimistic Prior
Expect treatment difference to be 15%, but a 20%
chance that difference could be negative.
-60
-45
-30
-15
0
15
Phase 2 Treatment Difference
40
30
45
60
Light Sceptical/Optimistic: Downweight above to 25% P2 sample size
Incorporating prior belief ultimately allows us to build up a bound for the PoS
If we observe <X% diff in our P2 study the PoS is only Y% for even the most
optimistic of you.
If we observe >X% diff in our P2 study the PoS is still Y% for the real sceptics
0.2
0.0
-0.2
-0.4
Rsq=75.3%
Expected Diff in 6m
survival rate
-0.6
log Hazard Ratio (Overall Survival)
0.4
Step3: Converting the P2 to the P3 endpoint
-0.8
S
-0.4
-0.2
U
O
0.0
41
Difference in 6 mnth Survival Rate
0.2
0.4
Phase 3 Endpoint Treatment Difference
6M - OS Relationship
Probability
Probability true
HR<1 (i.e we
have an effect)
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
1.2
1.3
Overall Survival Hazard Ratio
42
1.4
1.5
1.6
1.7
1.8
1.9
2.0
Step 3: Posterior Distribution for OS Hazard Ratio
Posterior Distribution for OS Hazard Ratio
by Month 6 Difference in Surv iv al Rate
and Type of Prior Distribution
2.0
1.8
OS Hazard Ratio (95CrI)
1.6
1.4
1.2
1.0
0.8
0.6
0.4
Prior Distribution
Non-Informative
Optimistic
Sceptical
0.2
0.0
-0.20
-0.15
-0.10
-0.05
0.00
Month 6 Difference
Program: panc_posterior_reln_plots.sas
Output: post_os_m6diff_all.cgm (Date Generated: 02FEB2010: 14:57)
43
0.05
0.10
0.15
0.20
Determining the PoS in Phase 3
 PoS will depend upon the Phase 3 study design
– The power, significance level and hypothesis drive the number
of events required
– The more events you include in your analyses, the smaller the
variance and the more likely you are to reach statistical
significance
 We draw a sample result from the Phase 3 endpoint
distribution and apply the variance associated with the
chosen Phase 3 study design. We then determine if the
result is statistically significant.
 We repeat the process multiple times and determine the
proportion of times that we see a statistically significant
result. This is the probability of success.
44
Step 4: Probability of Success
100
We can also change the Phase 3 design
to optimize the PoS – do a larger study
Probability of Success in Ph 3
90
80
70
Better than industry average
Uninformative
Sceptical
60
50
Optimistic
Light Sceptical
40
Light Optimistic
30
20
10
0
-15
-10
-5
0
5
10
15
Diff in 6m Survival Rate in Phase 2 (Test - Control)
45
20
Summary
 The PoS we select will depend upon the current business
situation and our strategy to risk
 The industry average for oncology is a 40% success rate in
P3
 If we observe < 7% difference the PoS < 40% for the
optimists
 If we observe >10% difference then the PoS > 40% for the
sceptics
46
The power of the research: Examples
0.2
-0.2
0.0
Rsq=0.3%
-0.4
log Hazard Ratio (Overall Survival)
0.4
Using PFS for Soft Tissue Sarcoma
-0.4
47
-0.2
0.0
log Hazard Ratio (PFS)
0.2
0.4
The power of the research: Examples
Log Hazard Ratio (Overall Survival)
ORR in NSCLC
0.4
0.2
0.0
Rsq=23.9%
-0.2
-0.1
0.0
0.1
Difference in Objective Response Rate (%)
48
0.2
Log Hazard Ratio (Overall Survival)
The power of the research: Examples
PFS in NSCLC is not so
good
0.4
0.2
0.0
Rsq=42.6%
-0.2
-0.4
-0.2
0.0
Log Hazard Ratio (Progression Free Survival)
49
0.2
0.4
0.0
0.5
PFS in CRC seems good
Rsq=65.5%
-0.5
log Hazard Ratio (Overall Survival)
1.0
A good one to finish on
-0.8
50
-0.6
-0.4
-0.2
0.0
0.2
Log Hazard Ratio (Progression Free Survival)
0.4
0.6