Continuing Professional Pharmacy Development (CPPD) Program
Title
Putting Evidence in to Practice
David J Woods, Consultant Pharmacist; Managing Editor New Zealand
Medicines Formulary
November 28, 2012
Disclaimer:
PRESENTING AUTHOR HAS NO RELATIONSHIPS TO DISCLOSE
1
Workshop Learning Outcomes
By the end of this session, participants will be able to :
1. Discuss how clinical trial results are reported in scientific papers,
the media, and by the pharmaceutical industry
2. Demonstrate competence in estimating the benefits and risks of
treatment in numerical terms (NNT – number needed to treat;
NNH – number needed to harm)
3. Assess the benefits and risks of treatment in patients at different
levels of baseline risk – the concept of PEER – patient expected
event rate
4. Discuss the application of clinical trial results in to practice – the
concept of generalizability
2
How benefits and harms are expressed
• Clinical trials
• Media
• Advertising
3
Relative Risk
• Risk is: no. events/no. people exposed
• Risk is a RATE or event rate
• 5 heart attacks per 100 risk is 5/100 or 5% or
0.05
• NB event rate can have positive or negative
implications
4
Relative Risk
• Compares the risk (event rates) of two
populations as a ratio
• Event rate in control population:
Control Event Rate or CER
• Event rate with experimental (exposed)
population: Experimental Event Rate or EER
• Relative Risk (RR) = EER/CER
5
Relative Risk – example 1
• Event rate (MI) with drug A = 5/100
EER = 5% or 0.05
• Event rate in non-exposed controls is 10/100
CER = 10% or 0.1
• RR = EER/CER or 0.05/0.1
RR = 0.5 or 50%
• Event rate with A is 50% of that with control
6
Relative Risk example 2
• EER with B is 0.04
• CER is 0.1
• RR = 0.04/0.1
RR = 40 % or 0.04
• Event Rate with B is 40% of that with control
7
Relative Risk Reduction (RRR)
• Often more intuitive to express by how much
risk is reduced compared to control or
background risk
• RRR = CER – EER (Risk Difference)/CER
• Example 2
RRR = 0.1 – 0.04/0.1 = 0.06/0.1
RRR = 0.6 or 60 %
8
Relative Risk Reduction (RRR) 2
• In example 2 the risk of an event is reduced by
60 %
• RRR is also 1 – RR
In this case 1 – 0.4 = 0.6 or 60%
• RRR is often assumed to have a linear
relationship with time;
E.g. RRR of 60% over three years is 20% per
year
9
RR and RRR are only RELATIVE
• Example 1
RR is 0.5 and RRR is 50%
5/100 divided by 10/100
• But;
5/1000 divided by 10/1000 is also 0.5 or 50%
• RR and RRR give no indication of the effect in
absolute terms
10
Absolute Risk Reduction (ARR)
• Risk difference, i.e. difference between CER
and EER
• Example 2
ARR is CER – EER ; 0.1 – 0.04
ARR is 0.06 or 6%
• In absolute terms risk is reduced from 10% to
6%
11
ARR examples
• First case
CER = 10%, EER is 6%
RR 0.6, RRR is 0.4
Can say RR is 60% and risk reduced by 40 %
• ARR (CER – EER) is 4% (large impact on
population ?)
12
ARR examples
• Second case
CER = 1%, EER is 0.6%
RR 0.6, RRR is 0.4
Can also say RR is 60% and risk reduced by
40%
• ARR is 1% - 0.6% = 0.4% (less impact at
population level than first case, but relative
figures the same)
13
Number Needed to Treat
• Number needed to treat to gain one benefit
• E.g. if ARR is 1% need to treat 100 patients to
gain one benefit
• If ARR is 0.1% need to treat 1000 patients to
gain one benefit
• NNT is 100/ARR e.g. 100/1% or 1/0.01
Care with above !
14
Real Example
• Fracture intervention trial
Black et al.Lancet 1996;348:1535-41
• 2027 post-menopausal women treated with
alendronate or placebo
• Reduction in hip fracture was 49%
In advertising this would look impressive !
15
Fracture Intervention trial
• CER is 2.19; EER is 1.08
• RR = 49 %
Can say fractures reduced by 50%
Impressive !
• Absolute Risk Reduction ARR = 1.11%
NNT is 100/1.11 or about 90
More realistic
16
More on NNTs
• Not the perfect solution, e.g. difficult to
compare NNTs from different trials
• Time factor
• Numbers Needed to Harm (NNH) exactly the
same concept
• NNT/NNH ratio should give a benefit to harm
estimate; see www.thennt.com
•
17
18
19
20
Varenicline and CV problems
21
Varenicline and CV problems
• Taking Chantix was associated with an
increased risk of serious heart problems.
These events were reported in 52 people out
of 4,908 (1.06%) who took Chantix, compared
with 27 of the 3,308 participants (0.82%) who
took the placebo.
• ARI = 1.06 – 0.82 = 0.24
NNH = 100/0.24 = approx. 400
22
Other considerations
• Study population
– Already at CV risk
• Benefits of varenicline
– NNT of about 10 cf NNH of 400
23
Benefits and harms in advertising
• RR or RRR never ARR or NNT
• Benefits usually exaggerated
• Harms usually minimised
• Critically appraise
– Do your own calcualtions
25
“Proven to significantly reduce
Vertebral fractures in just
one year”
Reduces new vertebral fractures
by up to 65%
52% risk reduction
65% risk reduction
Benefits and Harms in Clinical Trials
• Usually RR, RRR or Odds ratios
• ARR should be easily accessible
• Some now report NNTs
• Subject to bias
• You often have to do some work from the data
to get a clearer picture
28
Cardiovascular Harm and NSAIDs
DB/FM
27 Mar 2012
Drug
Summary Relative Risk for
Cardiovascular Event (95% CI)
Rofecoxib ≤ 25 mg
1.33 (1.0 – 1.79)
Rofecoxib > 25 mg
2.19 (1.64 – 2.91)
Celecoxib
1.06 (0.91 – 1.23)
Diclofenac
1.40 (1.16 – 1.70)
Naproxen
0.97 (0.87 – 1.07)
Piroxicam
1.06 (0.7 – 1.59)
Ibuprofen
1.07 (0.97 - 1.18)
Meloxicam
1.25 (1.00 – 1.55)
Indomethacin
1.30 (1.07 – 1.60)
29
Problems 1 and 4
• Group discussions
• Report back answers and discuss issues arising
30
Problem 1
• A
RR = 1.9/3.9 = 0.48
RRR = 0.52
ARR 3.9 – 1.9 = 2%; NNT = 100/2 = 50
• Second case
RR = 0.19/0.39 = 0.48
RRR = 0.52
ARR 0.39 – 0.19 = 0.2%; NNT = 100/0.2 = 500
31
Problem 1 B
• ARR = CER – EER = 9.6 – 2.8
ARR = 6.8 %
NNT = 100/6.8 = 15 over 5 years
RRR = ARR/CER
RRR = 6.8/9.6 = 0.7 or 70%
32
Problem 4 a quick appraisal 1
• ARR warfarin cf aspirin (BENEFITS)
20 – 16.7 = 3.3 %
NNT 100/3.3 = 30
For every 30 patients treated with warfarin
instead of aspirin over 4 years one patient will
benefit
33
Problem 4 (Harms)
• Warfarin cf Aspirin (Harms)
• Bleeds; Warfarin 0.62% per year vs 0.17 per
year with aspirin
• Over 4 years 2.48% vs 0.68
ARI 1.8 % over 4 years; NNH 100/1.8 = 55
34
Problem 4 (benefits:harms)
• NNT = 30
• NNH = 55
• Unacceptable benefit:harm ratio
• Discussion:
Statins ?
Aspirin for primary CV prevention ?
35
Consideration of baseline risk – the PEER
• For statins RRR for CV events is about 30 %
across all populations
(NB a generalisation for demonstration)
• If CV risk is 20% over 10 years then this is
reduced to 14 % with statin
• If CV risk is 5% over 10 years then this is
reduced to 3.5 % with statin
36
Compare absolute benefits
• CV risk 20 %
ARR = 20 – 14 = 6%; NNT is 100/6 = 17
• CV risk 5 %
ARR = 5 – 3.5 = 1.5%; NNT is 100/1.5 = 67
• Benefits much greater in patients with higher
baseline risk
37
Aspirin vs Warfarin in non-valvular AF
• Baseline risk can influence drug choice
• Risk of AF and stroke increase with age
• Other factors can affect stroke risk
• RRR for stroke prevention; warfarin vs aspirin
about 40 %
38
39
Baseline stroke risk and treatment choice
See http://www.bpac.org.nz/magazine/2009/february/warfarin.asp
40
What is the PEER
• Patient Expected Event rate
• Baseline risk or CER
• By applying RRR to a patient with known PEER
can assess benefit of treatment in an
individual
• Algorithms available to estimate NNTs from
PEER and RRR
41
Problems 2 and 3
• Influence of baseline risk
• More on benefits vs harms
• Discuss and report back
42
Problem 2 – patient S
• S has PEER of 1% per year or 3 % over three
years
• X has RRR of 33% over three years
• ARR = 3% - 2% = 1% ; NNT = 100
43
Problem 2 – patient A
• A has a PEER of 10% per year or 30 % over 3
years
• X has RRR of 33% over three years
• ARR is 30 – 20% = 10%; NNT = 10
44
Drug X harms
• Severe GI bleed 0.1% per year or 0.3% over
three years
• Drug X increases risk of bleed by three times
to 0.9% over three years
• ARI = 0.9 – 0.3 = 0.6
NNH 100/0.6 = 166
45
Benefit vs Harms
• Patient S
NNT = 100; NNH = 166
• Patient A
NNT = 10 ; NNH =166
Benefits vs Harms more favourable with
Patient A than in patient S
46
Problem 3
• PEER is 10 %; RRR is 50%
• We know that RRR = ARR/CER (PEER)
• Also NNT is 100/ARR or ARR = 100/NNT
• RRR = 100/NNT x PEER
Less confusing to use fractions instead of %
RRR = 1/NNT x PEER
47
Problem 3
• NNT = 1/RRR x PEER (RRR is constant)
• PEER of 10%; NNT = 1/0.5 x 0.1 = 20
• PEER of 5% ; NNT = 1/0.5 x 0.05 = 40
• PEER of 1 %; NNT = 1/0.5 x 0.01 = 200
• Lower the PEER the higher the NNT
48
NNT/PEER nomograms
49
Generalisability/External validity
• How the trial results can be applied in practice
• Consider
– Trial setting
– Selection and exclusion of patients
– Patient characteristics
– The intervention and trial management
– Outcome measures/follow-up
– Adverse effects
50
Problem 5
• Consider aspects of clinical trial design to
assess the effectiveness of a new
antidepressant, i.e. what factors would
increase its external validity ?
• In trials of nicotine dependence what factors
might influence the generalizability of the
results ?
Hint; consider inclusion and exclusion criteria
51
Summary
• Basic understanding of RR, ARR, NNT, NNH
and PEER is vital in order to put evidence in to
practice
• Associated calculations are not difficult
• Baseline risk is important when assessing
treatment benefits and harms
• The generalizability (or lack of) a clinical trial
needs to be recognized
52
References
• Scott IA, Greenberg PB. Cautionary tales and the
interpretation of clinical results. Int Med J 2006;35:611-21
• Barratt A et al. Relative risk reduction, absolute risk reduction,
NNT. CMAJ 2004; 171(4):353-58
• The NNT
www.thennt.com
• Centre for Evidence Based Medicine
www.cebm.net
DB/FM
27 Mar 2012
53