Using Evidence to Guide Drug
Therapy Decisions
Dean Haxby, Pharm.D.
Associate Professor of Pharmacy
Oregon State University, College of
Pharmacy
To receive 1.5 AMA PRA Category 1 Credits™,
you must review this progam and pass the CME
quiz at the end.
Release Date: January 2009 Expiration Date: January 2012
Attachments
• The attachments tab contains documents
that supplement the presentation.
• The slides are available as an attachment to
print out to use as a handout.
• Another attachment provides a listing of
additional resources
Program Funding
This work was made possible by a grant from the
state Attorney General Consumer and Prescriber
Education Program which is funded by the multi-state
settlement of consumer fraud claims regarding the
marketing of the prescription drug Neurontin.
Continuing Education Sponsors
Continuing Medical Education for the following
activity titled “Using Evidence to Guide Drug
Therapy Decisions”, is jointly sponsored by The
University of Texas Southwestern Medical Center
and the Federation of State Medical Board’s
Research and Education Foundation.
CME Information
Program Speaker/Author: Dean Haxby, PharmD,
Course Director:
Program Directors:
Barbara S. Schneidman, MD, MPH
Federation of State Medical Boards Research and Education Foundation, Secretary
Federation of State Medical Boards , Interim President and Chief Executive Officer
David Pass, MD
Director, Health Resources Commission, Oregon Office for Health Policy and Research
Dean Haxby, PharmD
Associate Professor of Pharmacy Practice, Oregon State University College of Pharmacy
Daniel Hartung, PharmD, MPH
Assistant Professor of Pharmacy Practice, Oregon State University College of Pharmacy
Target Audience: This educational activity is intended for health professionals who are involved with medication
prescribing, and those that are involved with committees involved with medication use policies.
Educational Objectives: Upon completion of this activity, participants should be able to: Describe the advantages
and limitations of evidence-based medicine (EBM), and its role in improving prescribing decisions; Describe the
steps in the EBM process; Identify strategies to search for evidence-based medical literature, including useful web
sites; Review key factors that should be appraised with systematic reviews, clinical practice guidelines, and
individual RCTs; Describe the influence different methods of presenting study results can have on decisions; Given
the results of a study calculate ARR and NNT; Discuss how EBM principles can be applied to evaluating new
drugs.
CME Policies
Accreditation: This activity has been planned and implemented in accordance with the
Essential Areas & Policies of the Accreditation Council for Continuing Medical
Education through the joint sponsorship of The University of Texas Southwestern
Medical Center and the Federation of State Medical Boards Research and Education
Foundation. The University of Texas Southwestern Medical Center is accredited by the
ACCME to provide continuing medical education for physicians.
Credit Designation: The University of Texas Southwestern Medical Center designates
this educational activity for a maximum of 1.5 AMA PRA Category 1 Credits™.
Physicians should only claim credit commensurate with the extent of their participation
in the activity.
Conflict of Interest: It is the policy of UT Southwestern Medical Center that
participants in CME activities should be made aware of any affiliation or financial
interest that may affect the authors presentation. Each author has completed and signed a
conflict of interest statement. The faculty members’ relationships will be disclosed in
the course material.
Discussion of Off-Label Use: Because this course is meant to educate physicians with
what is currently in use and what may be available in the future, “off-label” use may be
discussed. Authors have been requested to inform the audience when off-label use is
discussed.
DISCLOSURE TO PARTICIPANTS
It is the policy of the CME Office at The University of Texas Southwestern Medical Center to ensure balance,
independence, objectivity, and scientific rigor in all directly or jointly sponsored educational activities.
Program directors and authors have completed and signed a conflict of interest statement disclosing a financial or
other relationship with a commercial interest related directly or indirectly to the program.
Information and opinion offered by the authors represent their viewpoints. Conclusions drawn by the audience
should be derived from careful consideration of all available scientific information. Products may be discussed in
treatment outside current approved labeling.
FINANCIAL RELATIONSHIP DISCLOSURE
Faculty
David Pass, M.D.
Dean Haxby, Pharm.D
Daniel Hartung, Pharm.D., MPH
Barbara S. Schneidman, MD, MPH
Type of Relationship/Name of Commercial Interest(s)
None
Employment/CareOregon
None
None
Learning Objectives
• Describe the advantages and limitations of
evidence-based medicine (EBM), and its role in
improving prescribing decisions
• Describe the steps in the EBM process
• Identify strategies to search for evidence-based
medical literature, including useful web sites
• Review key factors that should be appraised with
systematic reviews, clinical practice guidelines,
and individual RCTs
Learning Objectives
• Describe the influence different methods of
presenting study results can have on decisions
• Given the results of a study, calculate ARR and
NNT
• Discuss how EBM principles can be applied to
evaluating new drugs.
Problems With Drug Therapy
• Over the past 15 years, drug costs have
escalated at a higher rate than any other area
of health care
• The US has 5% of the worlds population,
but has 50% of drug consumption
• The US ranks very low in measures of
health care quality
• Overuse, underuse and inappropriate use of
medications are an important quality issue
• Many of these problems are preventable
Limitations of the FDA Drug
Approval Process
• FDA review determines if a drug is effective and
has acceptable safety.
• It does not determine place in therapy.
• Value not assessed.
• Often there are many unanswered questions at the
time a drug is approved.
– Limited numbers and types of patients
– Limited duration of studies
– May not know outcomes of greatest interest
• Much of the FDA funding comes from industry
What is Evidence-based
Medicine?
• A process to identify and use the best available
scientific evidence to guide decision making
– Patient care
– Policy
• The scientific evidence is integrated with clinical
judgment and the patients unique circumstances to
provide the best clinical decisions
• The result is predictable improvement in patient
outcomes
EBM Drug Therapy Decisions
• When there is evidence of benefit and
value, do it.
• When there is evidence of no benefit, harm
or poor value, don’t do it.
• When there is insufficient evidence to know
for sure, be conservative.
David Eddy. Formulary 2002;37:52530
Decision Making in Clinical
Practice
• Unfortunately, many decisions are made based on
unreliable “evidence”
• Personal observation or anecdotal experience
– Tends to overestimate efficacy
• Reasoning based on pharmacology, pharmacokinetics or
theory instead of “patient oriented evidence that matters”
• Use of observational studies or case series to draw cause
and effect conclusions
• Expert opinion plays a heavy role in medical decision
making
– Opinions vary
– Unbiased expert opinion using EBM is very useful
– The consensus approach based on uncontrolled clinical experience
risks widespread application of useless or even harmful treatment
Advantages of Using an EBM
Approach
• Guides prescribing decisions to get predictable
improvements in patient outcomes.
• Helps sort through the marketing, opinions, and
theory to get an accurate assessment of the
benefits and risks of various treatments.
• Can help identify and target opportunities to
improve drug therapy.
• Assists with decisions about use of limited
resources.
• Provides incentive to conduct useful research.
Limitations of EBM
• Studies may not answer the question you are
trying to answer
– Evidence may be lacking or of low quality
– Surrogate endpoints
• The body of evidence continues to change
• It can take a lot of resources to conduct clinical
trials or create original high quality evidence
reviews
• It does not make clinical decisions or value
judgments about cost versus benefit
– But it can assist you in doing so
Steps in the EBM Process
• Formulate the question(s) to be answered
• Gather Evidence
– Decide on type of literature
– Conduct search
• Critical appraisal to identify the best evidence
• Evidence summary
– Evaluating the strength of the evidence
– Magnitude benefits and risks (ARR,NNT, NNH)
• Form recommendations/conclusions and apply the
evidence
Step 1: Formulate Clinical
Question(s)
• An important and sometimes difficult step
• Well constructed questions guide the
process
• Helps define what you really want to know
Clinical Questions
• Ask for information about managing
patients
• Contain several important components
(PICOS)
–
–
–
–
1. Population and/or clinical problem/disorder
2. Intervention
3. Comparison intervention (placebo, gold standard)
4. Outcomes
• helps define what is most important
– 5. Setting
Clinical Question Example
• For cigarette smokers, does varenicline
differ in efficacy for promoting long term
smoking cessation than bupropion SR or
NRT in the primary care setting?
• Population: cigarette smokers
• Intervention: varenicline
• Comparator: bupropion, NRT, placebo
• Outcome: smoking cessation at one year
• Setting: Primary Care
Step 2: Gathering Evidence
• When evaluating drug therapy, several
types of literature are typically most useful
– Randomized controlled trials
– Systematic reviews of RCTs
– Evidence-based CPGs
• The goal is to locate the best available
evidence to answer your questions
Evidence Hierarchy
Systematic
Reviews
RCTs
Cohort Studies
Case Control Studies
Case Series
Case Reports
Editorials and Opinions
Lab studies and Animal Research
Strengths of Randomized
Controlled Trials (RCT)
• The foundation for evidence-based evaluation of
drug therapy
• If high quality, can establish benefits and harms of
drug therapy
• Effective randomization minimizes risk of
unexpected factors influencing results
(confounding)
• Fortunately, RCTs are required for FDA approval
Limitations of RCTs
• Experimental design and inclusionexclusion criteria may not reflect general
practice and make it difficult to apply or
generalize results (external validity)
• Outcomes trials in chronic diseases may
take years
• Expensive
• It takes time and effort to review individual
studies once they are published
Systematic Reviews (SRs)
• Usually a good starting place when gathering evidence
about drug therapy
• A SR is an evidenced-based review of the medical
literature up to a certain time point
• Predefined, detailed methods are used to search, screen,
critically evaluate, and summarize the medical research to
answer specific questions
• SRs can be qualitative or quantitative (meta-analyses)
• Meta-analyses combine results of individual studies, but
not all meta-analyses are SRs
• High quality SRs save time and can provide a clearer
picture of the body of evidence than individual studies
Traditional Reviews
• Typically lack the rigor of a systematic
review
• Can be more of an opinion piece rather than
a careful unbiased review of the literature.
• Selective use of literature can be used to
support the authors point of view.
• The medical literature is full of nonsystematic reviews.
Clinical Practice Guidelines
• Usually broader in scope than SR; goal - influence practice
• Are especially useful if they are well developed and free
from bias
– Political, financial or other factors can introduce bias
• Over 2500 guidelines are available
• Vary in quality (evidence vs. consensus opinion) and
recommendations can vary, so need to carefully evaluate
• High quality guidelines have two major components
– A systematic review of the evidence
– Specific recommendations with explicit links to the
evidence and graded for strength of evidence
Searching for Evidence
• This program will focus on a few basic
search strategies or useful sources to locate
the following:
–
–
–
–
RCT’s
Systematic reviews
Clinical practice guidelines
Additional sources of information useful for
evaluating new drugs
Search Criteria
• Decide on the type of literature best suited
to answer your question
• The clinical question components can help
guide the search (PICOS)
– Select search terms
– Screening to make sure the literature is
addressing your question
Searching for Evidence: PubMed
•
•
•
•
A very user friendly approach to Medline
Can access online through various medical libraries
Can also go to pubmed.gov
Enter your search terms based on your clinical question
– Example: varenicline smoking cessation
• Click on limits tab and scroll down to “type of article”
– limit to types of literature you want (RCT, Metaanalysis and/or guidelines)
– other limits can also help target your search
• Can adjust search based on results
PubMed: Searching for
Systematic Reviews
• On the PubMed side bar on the left of the
screen, click on “Clinical Queries”
• Scroll down to “Find Systematic Reviews”
• Enter your search terms based on your
clinical question
– This search retrieves systematic reviews, metaanalyses and other EBM literature
High Quality Systematic Reviews
• Drug Effectiveness Review Project (DERP)
– Contains systematic reviews of various drug classes
– Developed by the OHSU Evidence-based Practice
Center
– http://www.ohsu.edu/ohsuedu/research/policycenter/DE
RP/about/final-products.cfm
– http://www.oregonrx.gov
• Canadian Agency for Drugs & Technologies in
Health
– www.cadth.ca/
• Therapeutics Initiative
– www.ti.ubc.ca
High Quality Systematic
Reviews
• Agency for Health Care Research and Quality Effective
Healthcare Program
– http://effectivehealthcare.ahrq.gov
• Cochrane Collaboration (subscription)
– Can include unpublished data
– http://www.cochrane.org
• Center for Reviews and Dissemination (CRD)
– http://www.crd.york.ac.uk/crdweb
– Database of abstracts of reviews of effects (DARE)
Clinical Practice Guidelines
• National Guidelines Clearinghouse
– www.guideline.gov
• New Zealand Guidelines Group
– www.nzgg.org.nz/
• National Institute for Health and Clinical
Excellence (NICE)
– www.nice.org.uk/
• Veterans Health Administration
– www.oqp.med.va.gov/cpg/cpg.htm
– www.pbm.va.gov (monographs, drug use criteria)
FDA Website: fda.gov
• An important place to look for data regarding new
drugs, but can be a difficult site to navigate
• The advisory panel transcripts can contain a
wealth of information including information on
studies not published, and analyses by fda staff
• Information about unresolved issues and post
marketing requirements may be available
• Drug approval letter and product labeling
Information from Manufacturers
• Package Insert
– A great starting point for a new drug review
– Will indicate how many studies were used in
the review and approval of the drug
• AMCP Dossier Format
– Can be a source of information on unpublished
studies
– Contains an economic analysis
• http://clinicalstudyresults.org
Step 3: Critical Appraisal
• Once you have identified literature that meets your criteria,
it needs to be critically appraised to decide if it is worth
using.
• Critical appraisal is an important step in the EBM process.
• This step helps identify the best evidence to answer our
question.
• Much of the medical literature is not useful or reliable, and
in some cases may even be misleading.
• With critical appraisal, we determine confidence in the
results (internal validity) and if the results are applicable to
our practice (external validity).
Additional Programs on Critical
Appraisal
• In this program, a very brief introduction to
critical appraisal will be provided.
• For a more in-depth review, please see the
following programs that are available from the
same website you accessed this program:
– “Critical Appraisal: Randomized Controlled Trials for
Drug Therapy ”
– “Critical Appraisal: Systematic Reviews and Clinical
Practice Guidelines for Drug Therapy”
Evaluating Individual RCTs
• Internal validity reflects the confidence that
the results are due to the intervention
• Focus on the methods and results sections
• Good studies provide a valid estimate of the
efficacy of an intervention and are usually
easier to review
• Form your own conclusions based on the
above
Items to Evaluate for Internal
Validity of RCTs
• Adequate sample size
with statistical power
• Randomization
methods appropriate
• Comparable groups at
baseline
• Equal co-treatment
• Compliance
• Low dropout rate
• Adequate length of
follow-up
• Blind assessment
• Equal assessment
• Intention-to-treat
analysis
• Post-hoc analysis
Applicability (External Validity)
of RCTs
• How relevant is the study?
– Were clinically meaningful endpoints studied?
– What is the magnitude of the treatment effect and is it
clinically significant?
• How generalizable are the results?
– How comparable were the patients studied to your
practice?
– Does the study reflect real world practice?
• If internal validity unacceptable, the applicability
is irrelevant
Pharmaceutical Industry
Sponsored Studies
• Generally have acceptable internal validity
• Applicability is more often the issue
–
–
–
–
Active controls may not be gold standard
Surrogate outcomes often involved
Dosing regimens not comparable
Population may not reflect population of most
interest
• Publication bias
Items to Assess with Systematic
Reviews
• Evaluate for bias, methods and generalizability
• Is there a clear, clinically relevant question that addresses
what you are looking for, that was defined before hand?
• Were study eligibility criteria defined and appropriate?
• Was the search detailed and exhaustive?
• Were the studies critically appraised by more than one
person?
• Were methods for data synthesis described?
• Were conclusions clear and reflect evidence?
• Was funding disclosed and potential conflict of interest
minimized?
DARE
• Database of Abstracts of Reviews of Effects
• A useful resource for readers of SRs
• Provides concise summaries of published
SRs of healthcare interventions
• Assesses strengths and weaknesses
• http://www.crd.york.ac.uk/crdweb and
click on “DARE” tab
Critically Evaluate Clinical
Practice Guidelines (CPG)
• Studies of guidelines have found that
methodological problems, or potential conflicts of
interest, are common
• Objectives, key questions and applicable
population clearly defined
• Stakeholders involvement (relevant disciplines,
patient views, target user pilot)
• Editorial independence from funding, methods to
manage conflict of interest, bias
CPG Rigor of Methods
• Systematic methods used to search for evidence
• Criteria for selecting evidence described
• Appropriate and clearly defined methods for
formulating recommendations
• Benefits, risks and costs considered
• Explicit link between supporting evidence and
recommendations (grades of recommendations)
• Guideline externally reviewed by unbiased experts
Step 4: Evidence Summary
Rate Confidence in Strength of Evidence
High : Large, well-designed randomized
controlled trials, rigorous systematic
reviews
Small, or not-so-well designed controlled
clinical trials
Non-randomized prospective cohort
studies
Non-randomized case-control studies
Low : Case series
Evidence Summary: Quantify
Results
RR:
% treatment group
% control group
RRR: 1-RR X 100%
ARR: % difference between control and treatment
NNT:
100%
ARR
RR= Relative Risk
ARR = Absolute Risk Reduction/absolute response rate
RRR = Relative Risk Reduction
NNT = Number Needed to Treat
Limitations of Relative Risk
• Example 1: risk of death is 1% Tx and 2% P
– RRR is a 50% reduction
– ARR is 1%
– NNT= 100
• Example 2: risk of death: 25% Tx vs 50%P
– RRR is a 50% decrease
– ARR is 25%
– NNT= 4
• Same RRR, but different clinical implications
Calculating ARR and NNT
• Useful if a statistically significant difference exists
– the confidence interval helps determine precision
– Important to link to the treatment duration
• Can only do this if specific responders are reported
• Can’t do calculations if results are presented as mean
changes for the population.
• Examples:
– Mean change in BP vs. % reaching target BP
– Mean change in HgA1c vs. % below HgA1c of 7
Example Calculation
• 95% of patients with AOM are improved
after 10days with gorillacillin vs. 85% on
placebo (p<.05)
• What is the:
– ARR = ?
– NNT = ?
Example: Amitiza for IBS-C
• “patients receiving Amitiza were nearly twice as
likely to acheive a statistically significant overall
response compared to those given placebo” (from
the monthly prescribing reference)
• Study 1: 13.8% respond to Amitiza vs 7.8% P
• Study 2: 12.1% Amitiza vs. 5.7% P
• Response = responded 2 out of 3 months
• ARR = about 6% will respond 2 out of 3 months
• NNT= For every 16 patients treated for 12 weeks,
one will respond two out of three months
Step 5: Form Conclusions and
Apply the Evidence
• When there is evidence of benefit and
value, do it.
• When there is evidence of no benefit, harm
or poor value, don’t do it.
• When there is insufficient evidence to know
for sure, be conservative.
Eddy DM.Formulary 2002;37:525-30
Do it Examples
•
•
•
•
•
•
Thiazides for hypertension
Aspirin to prevent CVD events
ACEI in CHF
Statins in secondary prevention
Warfarin in atrial fibrillation
Beta-blockers post-MI
Don’t do it Examples
• Estrogen/progestin to prevent CVD in late
postmenopausal women
• Alpha-blockers in hypertension
• Calcium blockers with systolic ventricular
dysfunction
• Zelnorm, Vioxx
• Antibiotics for acute URI
What be conservative means
• With new drugs, burden of proof is on the
manufacturer
• If we use a new treatment without adequate
evidence, there is little incentive to do the
research
• For old drugs, we should not actively
promote without good evidence.
Eddy DM. Formulary 2002;37:52530
Summary: EBM Process
• Formulate the questions to be answered
• Gather Evidence
– Decide on type of literature
– Conduct search
• Critical appraisal to identify the best evidence
• Evidence summary
– Evaluating confidence in the strength of the evidence
– Quantitative benefits and risks (ARR,NNT, NNH)
• Form conclusions and apply the evidence
Conclusions
• Using an evidence-based process results in prescribing
decisions that lead to predictable improvements in patient
care.
• There are numerous sources for high quality systematic
reviews and clinical practice guidelines that provide a
sound framework for drug therapy decisions.
• Prescribing agents that have the best evidence for benefits
vs. harms, and that provide good value can help address
some of the problems associated with prescription drug
use.
Thank you
This work was made possible by a grant from the
state Attorney General Consumer and Prescriber
Education Program which is funded by the multi-state
settlement of consumer fraud claims regarding the
marketing of the prescription drug Neurontin.
CME Instructions
• Please complete the survey, quiz and
program evaluation questions.
• Click the finish button.
• Fill out the form and send via fax, e-mail, or
regular mail to the address on the form to
receive CME credit.
PROPERTIES
On passing, 'Finish' button:
On failing, 'Finish' button:
Allow user to leave quiz:
User may view slides after quiz:
User may attempt quiz:
Goes to URL
Goes to Next Slide
After user has completed quiz
At any time
Unlimited times