Joel Singer,
Programme Head,
Methodology and Statistics,
CIHR Canadian HIV Trials Network
What is a Non-inferiority
Trial?
A trial to show that a particular
therapeutic agent or policy is no
worse than some other standard.
What is a Non-inferiority
Trial?
Type 1: Both agents have been
shown superior to placebo/no
treatment
• Type 2: Only one agent has been
shown superior to placebo/no
treatment
What is a Non-inferiority
Trial?
• The standard (STD) should have
clinical efficacy: (i) that is of
substantial magnitude; (ii) is
precisely estimated; (iii) estimates
that are relevant to the setting in
which the trial is being conducted.
ICH Guidelines
• A suitable active comparator could
be a widely used therapy whose
efficacy in the relevant indication has
been clearly established
• and
ICH: Constancy Assumption
• quantified in well-designed and well
documented superiority trials and
which can be reliably expected to
have similar efficacy in the
contemplated active control trial.
Example of non-Constancy
• Suppose vancomycin, has been
shown to provide a large
improvement, compared to no
treatment, in cure rate in patients
with various infections.
Example of non-Constancy
Suppose a high prevalence of
vancomycin-resistant enterococci
(VRE) has developed. NI trial:
comparing an experimental antibiotic
(EXP) with vancomycin is conducted
where VRE is prevalent
Example of non-Constancy
A biased estimate of the efficacy of
EXP would be obtained if the
constancy assumption is falsely
presumed to hold in the NI study.
What is a Non-inferiority
Trial?
• Because proof of exact equality is
impossible, a prestated margin of
noninferiority for the treatment effect
in a primary patient outcome is
defined.
What is a Non-inferiority
Trial?
• Non-inferiority trials are intended to
show whether a new treatment has
at least as much efficacy as the
standard or is worse by an amount
less than Δ,often on the premise that
it has some other advantage
What is a Non-inferiority
Trial?
• Both participants and outcome
measures in a noninferiority or
equivalence trial should be similar to
those in trial(s) that established the
efficacy of the reference treatment.
What is a Non-inferiority
Trial?
• The required sample size is
calculated using the confidence
interval (CI) approach, considering
where the CI for the treatment effect
lies with respect to both the margin
of noninferiority and a null effect.
What is a Non-inferiority
Trial?
• Sample size depends on the level of
confidence chosen, the risk of type II
error (or desired power), and Δ.
What is a Non-inferiority
Trial?
• A prestated margin of noninferiority
can be specified as a difference in
means or proportions or the
logarithm of an odds ratio, risk ratio,
or hazard ratio.
What is a Non-inferiority
Trial?
• The meaning and clinical importance
of measures such as relative risk,
odds ratio and relative hazard will be
dependent on the base rate of
outcome
What is a Non-inferiority
Trial?
• A prestated margin of noninferiority
is often chosen as the smallest value
that would be a clinically important
effect.
What is a Non-inferiority
Trial?
• The required size of non-inferiority
trials is therefore usually larger than
that for superiority trials,
• But:
• The actual calculation is the same as
in superiority trials
What is a Non-inferiority
Trial?
• One should avoid features that might
dilute true differences between
treatments, thereby enhancing the
risk of erroneously concluding noninferiority
What is a Non-inferiority
Trial?
• For superiority trials, intention-totreat (ITT) analysis (analyzing all
patients within their randomized
groups, regardless of whether they
completed allocated treatment) is
recommended
What is a Non-inferiority
Trial?
• In non-inferiority trials, ITT analysis
will often increase the risk of falsely
claiming non-inferiority (type I error),
although not always
What is a Non-inferiority
Trial?
• In non-inferiority and equivalence
trials, non-ITT analyses might be
desirable as a protection from ITT’s
increase of type I error risk (falsely
concluding non-inferiority).
What is a Non-inferiority
Trial?
• Interpreting a non-inferiority trial’s
results depends on where the CI for
the treatment effect lies relative to
both the margin of non-inferiority and
a null effect. The observed treatment
effect is not sufficiently informative.
What is a Non-inferiority
Trial?
• For superiority trials, intention-totreat (ITT) analysis (analyzing all
patients within their randomized
groups, regardless of whether they
completed allocated treatment) is
recommended
Major Methodological Issues for
Equivalence/Non-inferiority Trials
1. Difficult to get consensus on difference
considered equivalence or non-inferior.
2. In superiority trials, lack of rigor in the
conduct of a trial will tend to lead to
equivalence between the comparators
and will cause a conservative bias.
3. Lack of rigor or anything which
causes background noise in an
equivalence/noninferiority trial
will bias in favour of the hypothesis
of interest (no difference).
Important Variables for Defining
Equivalence/non-inferiority
1. Outcome.
2. Prevalence of the disease.
3. Other: cost, invasiveness, ease of use.
Trial Design
1216 ART naive patients
NVP
400 mg od
NVP
200 mg bd
EFV
600 mg od
NVP+ EFV
400 / 800mg od
n=220
n=387
n=400
n=209
Nucleoside backbone: d4T and 3TC
Inclusion criteria:
pVL > 5000 copies/mL
any CD4 cell count
any stage of CDC-classification
Outcome Measures
% of patients with treatment failure at week 48:
• less than 1log10 decline in pVL in first 12 weeks
• 2 consecutive pVL > 50 copies/mL from week 24 onwards
• new CDC-C event or death
• change of allocated treatment
Methods
•
•
•
All analyses Intention-to-Treat (unless stated otherwise)
• all randomised patients
pVL data: missing=failure
Four pre-defined pairwise comparisons
• NVP-bd vs EFV (primary)
• NVP-od vs NVP-bd
• NVP-od vs NVP+EFV
• EFV vs NVP+EFV
Given that the 2NN trial was conducted
as a non-inferiority trial, how does one
go about interpreting the results?
The goal of an inferiority trial is to be
able to rule out differences greater than
the minimally important clinical
difference (MICD).
Translated into a statistical statement,
this means we can say with confidence
(based on the 95% confidence interval)
that the true difference is unlikely to
exceed the difference deemed clinically
important.
Methodological Concerns
With the 2NN Trial
1. Choice of definition of outcome.
2. Background noise.
Concerns in 2NN Study re
non-inferiority
1. patients who never received treatment
(20 in EFV, 10 in NVP-BD).
2. early compliance failures.
3. Question inclusion of changes in d4T or
3TC as evidence of failure.
What Were the Primary
Results of the 2NN Trial?
Treatment Success and Failure
100
patients
ofofpatients
%%
29.1
22.0
75
11.4
18.9
20.0
failure component:
(w hichever comes first)
change Rx
15.3
disease progression
16.3
50
25
34.5
virologic
success
56.4
56.3
62.3
46.9
NVP-od
NVP-bd
EFV
NVP+EFV
0
Success: only significant difference: EFV vs NVP+EFV, p< 0.001
There was no statistical difference
for the primary comparison (NVP vs
EFV) so one cannot say definitively
that one treatment is better than the
other.
The 95% confidence interval (1% in
favour of nevirapine to 13% in favour
of efavirenz) does not allow one to rule
out a clinically important difference (at
least as defined by the investigators)
on the primary efficacy variable.
Conclusions
• NVP and EFV have comparable potency in
suppressing HIV-1 replication
• NVP-od and NVP-bd show comparable efficacy
• Co-administration of NVP and EFV results in higher
treatment failure due to increased toxicity
Comments
• Conclusion slide does not address
primary outcome (combination of
disease progression, virologic failure
and change of therapy)
• Does not address issue of proof of noninferiority
Some Technical Sample Size
issues
• Typically, one wants the sample size to
be large enough so that the bounds of
the confidence interval will exclude
values outside the non-inferiority range
80%-90% of the time
Some Technical Sample Size
issues
• If the outcome is continuous, and one
assumes homogeneity of variance, then
the sample size required for noninferiority under the null hypothesis is
the same as the sample size to detect
the indicated difference under the
alternative hypothesis
Some Technical Sample Size
issues
• If the outcome is dichotomous, then the
sample size required for non-inferiority
under the null hypothesis is not the
same as the sample size to detect the
indicated difference under the
alternative hypothesis because the
variability is a function of p1 and p2
Some Technical Sample Size
issues
• In a non-inferiority trial, the hypotheses
are inverted.
• The null hypothesis is that there is a
difference > m, and the alternative is the
null hypothesis
Some Technical Sample Size
issues
• Generally, the clinical world is
concerned about absolute differences
between treatments, but depending on
the statistical model being used, the null
and alternative may be stated in other
terms eg hazard ratios
Some Technical Sample Size
issues
• The choice of what is considered noninferior in terms of a hazard ratio is
typically driven by what one assumes
the baseline rates are
• Eg if the hazard rates are relatively low,
a somewhat large hazard rate may
translate into a relatively low absolute
difference
Summary of 2NN Study
1. Demonstrated a non-significant 6%
advantage in the primary outcome in
favour of efavirenz over nevirapine bid on
intent-to-treat analysis.
2. Confidence interval around obtained
difference between efavirenz and
nevirapine bid include confidence bounds
to 13% in favour of efavirenz. Cannot
exclude non-inferiority on primary
outcome set out by study criteria.