Optimal GNG decision rules for an
adaptive seamless Ph2/3
oncology trial
Cong Chen
Linda Sun
BARDS, Merck & Co., Inc
Typical situation in oncology


Success rate in Ph3 is low, cost for Ph3 is high and
competition is fierce
Data at Ph2 to Ph3 transition point
–
–
–
2
Plenty of data on early efficacy endpoints, of which the most
important is progression-free-survival (PFS)
Limited data on the clinical (late) endpoint, which is typically
overall survival (OS)
PFS is a reasonable surrogate biomarker of OS but can
hardly be considered a validated surrogate endpoint
Ph2 to Ph3 transition

Sequential
–

Statistically seamless
–

Ph2 data are combined with Ph3 after proper
multiplicity adjustment due to dose selection
Operationally seamless
–
–
3
Ph2 followed with Ph3
Ph3 starts immediately after Ph2
Ph3 data won’t be combined with Ph2
GNG for seamless transition

Today’s focus
–
–
–
4
How to effectively
incorporate surrogate
biomarker data into the
decision matrix?
How to derive objective
GNG bars from a benefitcost ratio perspective?
How to fully realize the
potential of a seamless
design with proper risk
mitigation?

Not today’s focus
–
–
–
–
Multiplicity adjustment
and dose selection
Validation of assumptions
made for setting GNG
bars
Cost-effective futility
analysis of Ph3 to further
mitigate the risk of a Go
decision
Technical details
Learning of the day


5
Relative effect size between the
clinical endpoint and an early
endpoint, and its application to
GNG decisions
Benefit-cost ratio analysis for
deriving objective GNG bars
Relative effect size γ (Thing 1)


Ideally, estimation of
distribution is based on
appropriate meta-analysis of
relevant historical data
Mean r and variance σ2
–
–
6
Smaller r implies it takes
greater early effect to
achieve same benefit in
later endpoint
Smaller σ2 implies greater
predictability of benefit in
later endpoint from early
endpoint data
Data utilization
Conventional

Back-calculate effect size of
clinical interest for an early
endpoint
–

–
–
7

40% hazard reduction in PFS ~ 
25% hazard reduction in OS
Base upon early endpoint data
for deriving GNG bar
–
Proposed
Uncertainty about relative
effect size is not accounted for
Role of late endpoint data is
less clear (“review issue”)
Lack of a contingency plan and
frequent revisit of decisions
Explicitly incorporate (r, σ2) into
estimation of treatment effect
on clinical endpoint
Directly base upon the
estimate for deriving GNG bar
–
Empirical relative effect size
from the trial is compared to
historical estimate before a
GNG decision is made
Application to estimation of OS effect

weight*OS effect + (1-weight)*γ*PFS effect
–
Weight may be chosen to be inversely proportional to
variance after incorporation of σ2 into estimation


Go if the joint estimate meets certain criterion and
observed OS effect is consistent with or greater that
predicted by the PFS effect
–
–
8
When there is no OS data weight=0 but distribution of γ is still
incorporated into the decision matrix
Statistical criterion: type I/II errors and variants
Business criterion: max return on investment
Benefit-cost ratio analysis (Thing 2)

Ph2 POC trial is resourced for (α, β)=(0.1,0.2) for
detecting an early endpoint effect of Δ
–
–

What is the optimal GNG bar?
–
–
–
–
9
Sample size is ~80 patients in oncology when Δ refers to a
50% hazard reduction on PFS
Implicit GNG bar is ~0.6Δ associated with α=0.1
Senior manager: “The bar is too low!”
Team member 1: “How large is Ph3?”
Team member 2: “The drug is promising.”
Commercial: “The market is HUGE.”
A benefit-cost ratio analysis

Probability of Go if probability of drug truly active in
the setting is POS
–

Expected total sample size (SS)
–

Ph2 SS + Prob(Go)*Ph3 SS
Benefit-cost ratio (return on investment) when cost is
measured by SS
–
10
Prob(Go) = (1-POS)*α+POS*(1-β)
Power of carrying active drug (1-β) to Ph2 divided by
expected total SS
Maximum benefit-cost ratio
POS
Optimal (α, β)
0.1
(6.7%, 26.7%)
Optimal empirical
GNG bar
0.71Δ
0.2
(7.2%, 25.3%)
0.69Δ
0.3
(8.0%, 23.7%)
0.66Δ
Ph3 SS is assumed to be 400 patients in above analyses
Bars should be set higher than 0.6 Δ to make it cost-effective
11
Application to a P2/3 oncology study
12
Study design


A 3-arm operationally seamless Ph2/3 trial in
platinum resistant ovarian cancer with an
option of converting to sequential Ph2/3
Primary hypothesis
–
–
13
Test drug is superior to pegylated liposomal doxirubicin
(PLD) in OS
OR test drug is non-inferior to PLD in OS with a margin of
hazard ratio of 1.1 and superior to PLD in a set of pre-defined
safety endpoints
Ph2 to Ph3 transition

A GNG bar to P3 is pre-set based on complete Ph2 data
–

~210 patients in 1:1:1 randomization to two dose levels of test
drug (high and low), and PLD with 4-months of minimum follow-up
Dose decision and preliminary GNG decision are made at an
interim analysis right after all patients are enrolled to trigger a
seamless transition
–
–
Go if conditional power of meeting the end of Ph2 bar is >80%
Hold otherwise until Ph2 completes


14
Go if bar for end of Ph2 is met and No Go otherwise
Complete Ph2 data and OS data from extended follow-up are
used as a prior for helping set cost-effective futility bars in Ph3
interim analyses
Flow chart
Ph2: ~135
PFS events
1m
4m
60 pts
f/u
60 pts
f/u
60 pts
f/u
380 pts
Ph3: ~508
OS events
380 pts
Data cut-off
for IA
15
GNG to Ph3
Joint estimate of OS effect

Historical estimate of γ
–
r=0.5 at log(hazard ratio) scale, and σ2= 0.22


0.15*OS effect + 0.85*γ*PFS effect
–
16
50% hazard ratio on PFS implies ~70% hazard ratio on OS
(95%CI: 54%, 93%)
In this case study, 0.15 provides a robust estimate of weight
that approximately minimizes the variance when event
numbers and other parameters are in a space of interest
Key input to benefit-cost ratio analysis


Test drug is equally likely to be superior, equivalent
and inferior to PLD based on industry benchmark
Commercial values and approvability based on a poll
from key stakeholders
–
–

Relative cost before transition to Ph3 relative to the
total cost of Ph2/3 program
–
–
17
Relative value (superiority vs non-inferiority) is 5.6
Relative approvability (superiority vs non-inferiority) is 2.3
~40% for seamless transition
~25% for sequential transition
GNG bars

GNG bar at end of Ph2 based on a benefit-cost
ratio analysis
–
–

GNG bar at interim of Ph2 for seamless transition
–
–
18
Expected benefit: Average power of Ph3 over prior
adjusted with outcome (superiority or non-inferiority) and
its associated approvability and commercial value
Expected cost: Cost before transition + Cost after
transition*(average probability of Go to Ph3 over prior
associated with the GNG bar)
Conditional probability of meeting end of Ph2 bar is >80%
Assumption about r is double checked before GO
Key drivers behind benefit-cost ratio

In our case study, the optimal end of Ph2 bar is
generally robust to the input parameters
–

Drivers with greatest impact on optimal GNG bar
–
–
19
The bar corresponds to ~8% hazard reduction in terms of
joint estimate of OS effect (implied OS improvement ~1.2
months)
Prior belief of drug activity: the stronger the belief  the
lower the optimal bar
Cost structure: the higher the up front cost  the lower
the optimal bar
Optimal GNG bars at end of Ph2
Relative Cost of
Ph2 to whole
program
Implied OS
improvement in
optimized bar
Superior – 11%
Equivalent – 22%
Inferior – 66%
25% - sequential
~ 1.7 months
40% - seamless
~ 1.3 months
Superior – 33%
Equivalent – 33%
Inferior – 33%
25% - sequential
~ 1.6 months
40% - seamless
~ 1.2 months
Superior – 50%
Equivalent – 33%
Inferior – 17%
25% - sequential
~ 1.5 months
40% - seamless
~ 1.1 months
Assumption of
drug activity
20
1.0
0.5
0.0
Observed Hazard Ratio of OS (MK/Control)
1.5
GNG boundary at end of Ph2
0.6
21
0.8
1.0
1.2
1.4
Observed Hazard Ratio of PFS (MK/Control)
1.0
0.5
0.0
Observed Hazard Ratio of OS (MK/Control)
1.5
Check assumption about γ
0.6
22
0.8
1.0
1.2
1.4
Observed Hazard Ratio of PFS (MK/Control)
1.0
0.5
0.0
Observed Hazard Ratio of OS (MK/Control)
1.5
GNG boundary at interim analysis
0.6
23
0.8
1.0
1.2
1.4
Observed Hazard Ratio of PFS (MK/Control)
1.0
0.5
0.0
Observed Hazard Ratio of OS (MK/Control)
1.5
Check assumption about γ at IA
Boundary for IA
Boundary for FA
Predicted OS Effect
95% CI of Predicted OS Effect at IA
95% CI of Predicted OS Effect at FA
0.6
24
0.8
1.0
1.2
Observed Hazard
Ratio of PFS (MK/Control)
1.4
Discussion




25
Relative effect size γ (and its distribution) between OS and PFS
or, more generally, between a late endpoint and an early
endpoint holds key to statistical properties of GNG bars.
Assumptions about key parameters such as relative effect size,
benefit, cost, and POS are implicit in all major clinical decisions,
which are often heuristic and subjective in practice.
With max return on investment in mind, optimal GNG bars are
derived after explicit incorporation of the assumptions into a
utility function (e.g., benefit-cost ratio).
Statisticians can help formulate the problem, and help
streamline the decision process.
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
26
Chen C, Sun L, Chih C. Evaluation of Early Efficacy Endpoints for Proof-of-concept Trials, Journal of
Biopharmaceutical Statistics 2011, accepted.
Song Y, Chen C. Optimal Strategies for Developing a Late-stage Clinical Program with a Possible Subset
Effect, Statistics in Biopharmaceutical Research 2011, accepted.
Robert A. Beckman, Jason Clark, and Cong Chen. Integrating Predictive Biomarkers and Classifiers into
Oncology Clinical Development Programs: An Adaptive, Evidence-Based Approach. Nature Review Drug
Discovery 2011, volume 10, 735-749.
Chen C, Sun, L. On quantification of PFS effect for accelerated approval of oncology drugs. Statistics in
Biopharmaceutical Research 2011, DOI: 10.1198/sbr.2011.09046.
Chen C, Beckman RA. Hypothesis testing in a confirmatory Phase III trial with a possible subset effect.
Statistics in Biopharmaceutical Research, 1, 431-440 (2009).
Chen C, Beckman, RA. Optimal cost-effective Go-No Go decisions in late stage oncology drug
development. Statistics in Biopharmaceutical Research, 1, 159-169 (2009).
Chen C, Beckman RA. Optimal cost-effective Phase II proof of concept and associated Go-No Go
decisions. J. Biopharmaceutical Statistics, 1, 431-440 (2009).
Song Y, Chen C. Optimal strategies for developing a late-stage clinical program with a possible subset
effect. ASA Proceedings of the Joint Statistical Meetings 2009, 1408-1422.
Sun L, Chen C. Evaluation of early endpoints for go-no go decisions in late-stage drug development. ASA
Proceedings of the Joint Statistical Meetings 2009, 2273-2283.
Chen C, Beckman RA. Optimal cost-effective designs of proof of concept trials and associated Go-No Go
decisions. Proceedings of the American Statistical Association, Biometrics Section, (2007).
Technical details

Weighted estimate of treatment effect at information t
–
wΔt+(1-w)γδt where (Δt, δt) are treatment effect on (clinical endpoint, early endpoint)
and γ is estimated to be N(r, σ2)




Estimated treatment effect on clinical endpoint at final analysis
–
–
tΔt+(1-t)Δ1-t where Δ1-t is the treatment effect after information time t, independent of Δt
Joint distribution of wΔt+(1-w)γδt and tΔt+(1-t)Δ1-t is obtained after the variancecovariance between the two are derived from the above



27
w=var(γδt)/(var(γδt)+var(Δt)) where var(γδt)= σ2(var(δt)+ δt2)+(rδt)2
Corr(Δt, δt) may be estimated from a re-sampling based method or WLW method when both
are time-to-event variables
Easily extended when there are more than one early endpoints of interest
Conditional power, predicted power, and various other statistics of interest are easily obtained
once the conditional distribution is obtained.
Exact distribution may also be obtained but is more complicated. Normal approximation is
close enough for planning purpose.
Joint distribution of weighted estimates at two time points can be obtained
similarly, which is used for calculation of conditional probability in case study