Evidence-Based Clinical Practice:
Clinical Practice Guidelines
Cy
h iia
a Jac
us
C y nt
nth
J ac kev iicc iiu
BScPhm, PharmD
Q C a rd i o l o g y, F C S H P, FA H A , F C C P, FAC C
D,, M S c , B C P S - A
AQ
P ro fe s s o r o f P h a r m a c y P ra c t i c e
©2019 Cynthia Jackevicius All Rights Reserved
Reproduction and/or broadcast of this material without the explicit written permission of Dr. Cynthia
Jackevicius is prohibited.
Objectives











Define and describe clinical practice guidelines (CPGs).
Identify the major evidence sources for locating CPGs.
Describe the three elements of the framework analyzed in CPGs.
Evaluate the validity of a CPG, according to the framework described in the JAMA
Users’ Guide on Summarizing the Evidence.
Evaluate the clinical importance of the results from a CPG.
Determine whether a CPG has an explicit and sensible process for identifying,
selecting and combining evidence to create the guideline.
Identify whether a CPGs is current and up to date.
Assess whether a CPG has been subject to peer review and how bias may be
present in developing CPGs.
Describe and interpret the grades and levels of evidence for commonly used CPGs.
Describe the “killer Bs” when determining if CPGs are applicable to your practice
setting.
When given a patient scenario, evaluate the applicability of the CPGs to the specific
patient.
Case
 SP is a 22-month-old, 24.4 lb female who is being seen by
her pediatrician. You are the clinical pharmacist at the
medical practice, and working with the pediatrician that
day. According to her mother, SP has been crying since last
night with constant ear tugging while complaining about
something stuck in her ear. Her fever last night was 100.5 F.
 Her mother also tells you that since she started her new
daycare program a few months ago, SP has gotten sick twice
and is just getting over her last cold. Her temperature in the
office is 100.7 F.
 The pediatrician suspects acute otitis media. Is an antibiotic
appropriate for this patient? P : the child with acute otitis
media , I : antibiotic , C : no antibiotic , C: infection
Steps for EBP
Ask and frame the question
Acquire the evidence
Act
Ask
Appraise the evidence
 Apply and integrate the
evidence with our clinical
expertise and patient values
and circumstances
Act on your findings
Apply
Acquire
Appraise
Ask: PICO
P
I
C
O
What is Our Clinical Question?
 In [population], does [intervention] [versus comparator]
affect [outcome]?
Types of Foreground Questions
Therapy (antibiotic)
Harm
Diagnosis
Prognosis
Search: Pyramid of EBP Resources
Examples
Evidence based textbooks
Guidelines
Evidence based
abstraction journals
Systematic Reviews
Original journal
articles
Summaries
and
Guidelines
PreAppraised
Research
NonPreAppraised
Research
8
What Are Guidelines?
Clinical Practice Guidelines (CPGs)
 Systematically developed statements to help practitioners
and patients make decisions about appropriate care in
specific clinical situations
 Attempt to bridge between producers and consumers of
health care research
 Aim to influence what clinicians to in order to reduce
practice variation, lower costs, and improve health
outcomes of patients
Guideline Characteristics
Often address multiple clinical questions
Usually informed by existing evidence
May include economic or patient
preference considerations
Often attempt to grade evidence and
recommendations
Require strategies for dissemination
Why Do We Need CPGs?
Evidence
Recommendations
Values
Impact of Guidelines
Two big questions:
◦ Do guidelines change what doctors do?
◦ Do guidelines change patient-important outcomes?
One systematic review of studies answers
these questions
◦ 11 studies looked at the effect on patient-important
outcomes
◦ 9 of those studies showed “improvements”
Grimshaw, et al. Lancet. 1993;342:1317-22.
Performance Matters!
Lower Death Rate if Guidelines are Followed
In-Hospital Mortality (%)
7
6
5.9
5.0
5
4.6
4
3.6
3
2
1
0
<65%
65%-75%
75%-80%
>80%
Adherence to Guidelines by the Hospital
Peterson ED. CRUSADE. Annual Scientific Sessions of the AHA; November 17, 2002; Chicago, Ill.
Physician Attitudes
Survey of 201 internal medicine physicians:
◦ 74.7% felt that the guidelines were applicable to over half
their patients
◦ 76.5% agreed that guideline-based performance
measures encourage evidence-based medicine
◦ Over 70% perceive increased bias when committee
members have industry-sponsored research or
presentations
Shea, et al. Am J Med Qual. 2007;22:170-6.
Barriers to Implementation
Knowledge
Lack of Familiarity
Lack of Awareness
Attitudes
Behavior
Lack of Agreement
with Specific
Guidelines
Lack of Outcome
Expectancy
Lack of Agreement
with Guidelines in
General
Lack of Self-Efficacy
External Barriers
Guideline Factors
Environmental
Factors
Lack of
Motivation/Clinical
Inertia
Cabana, et al. JAMA. 1999;282:1458-65.
Finding Guidelines
National Guideline Clearinghouse www.guidelines.gov
◦ Was a reputable source to find multiple guidelines on a
topic
◦ Defunded by the government
◦ Picked up by a commercial entity ECRI Guidelines Trust
ECRI Guidelines Trust https://guidelines.ecri.org/
◦ Set up to replace www.guidelines.gov
◦ Must sign-in to access, making it more difficult to use than
guidelines.gov
◦ Still a work in progress – watch for future developments
Finding Guidelines
Professional Association Websites
◦ ACC/AHA – Cardiology
◦ JNC8, AHA/ACC – Hypertension
◦ NCEPIV, AHA/ACC, NLA – Lipids ◦ ADA – Diabetes
◦ IDSA – Infectious Diseases
Secondary Resources
◦ PubMed with guidelines filter
◦ EMBASE with guidelines filter
Sometimes, you just have to Google!
Activity – Find the Guideline
Treatment of deep vein thrombosis
◦ PubMed
◦ “Chest Guidelines”
◦ Compare the two
ACC/AHA heart failure
Community-acquired pneumonia
Critical Appraisal
Critical Appraisal of CPGs
Is the evidence from the CPG valid?
If valid, is the evidence from the CPG important?
If valid and important, can you apply to the patient?
Appraising
Validity
Steps Involved
Is the evidence valid?
• Do the recommendations consider all relevant
patient groups, management options and
possible outcomes?
• Are there systematic reviews of evidence that
estimate the relative effect of management
options on relevant outcomes?
• Is there an appropriate specification of values
and preferences associated with outcomes?
• Are the guidelines timely and current?
Are There Systematic Reviews?
The systematic review is the ideal method to evaluate
the effect of the different options on relevant
outcomes for the condition in the guideline.
How was the evidence acquired?
Did they do a systematic review?
Is there a summary of the evidence used to make
recommendations?
HOW DO THEY GET THE EVIDENCE FOR
RECOMMENDATION ?
Appropriate Specification of Values
and Preferences?
Who was on the panel? Did they receive
money for the review ???? Any specialist ?
Multiple viewpoints ?
◦ They might have conflicts of interest or other biases.
What were their priorities?
◦ Did they value one outcome over another?
◦ Should state underlying value judgments
Do recommendations consider relevant
patient groups, options, outcomes?
Think of their PICO
◦ Subgroups of patients?
◦ Management options/comparators?
◦ Patient-important outcomes?
Patient already had DVT : looking for treatment not
prevention
Check for therapeutic option
Cover the outcome or not
Example of Values Statement:
CCS Atrial Fibrillation Guidelines
CCS AFib Guidelines 2012
Timeliness
Consensus panels take time to deliberate
Check the dates of the:
◦Guideline itself
◦Evidence search
◦References list
Validity Summary SPOT
S
• Systematic Review was conducted to
obtain evidence
P
• Priorities and values were described
O
• Outcomes, Options, and Patients were
sufficiently broad
T
• Timely and current
Appraising Importance
of
Results
Steps Involved
Is the evidence important?
• What are the key recommendations?
• How strong are the recommendations?
Key Recommendations :
HOW PRACTIAL THEY ARE
What is the clinical importance?
◦ Risks versus benefits of recommendations
◦ Absolute risk versus relative risk reductions
Are they practical? (Ex: not available in US,
too expensive)
◦ Are the interventions described in sufficient detail?
◦ Ease of implementation
Strength of Recommendations
Should consider:
◦ Quality of evidence
◦ Magnitude of benefits
◦ Magnitude of harms
◦ Burden to the patient and the health care system
◦ Costs
Hierarchy of Evidence
High
Systematic reviews of RCTs
Single RCT
Systematic review of observational studies
Single observational study
Unsystematic clinical observations
Low
Example Systems
GRADE
American College of Chest Physicians (ACCP)
American College of Cardiology/American Heart
Association (ACC/AHA)
American Diabetes Association (ADA)
American Academy of Pediatrics/American Academy of
Family Physicians
GRADE Approach
Grades of Recommendation, Assessment,
Development and Evaluation (GRADE)
Working Group
◦ Developed explicit criteria for evaluating evidence and
formulating recommendations
◦ Many organizations and countries involved
Two Components of GRADE
Quality of Evidence
◦ High
◦ Moderate
◦ Low
◦ Very low
Strength of Recommendation
◦ Strong
◦ Weak (conditional)
GRADE: Quality of Evidence
Quality Rating
Definition
High
High confidence that the evidence reflects the true
effect. Future research unlikely to change confidence in
estimate of effect.
Moderate
Moderate confidence that the evidence reflects the true
effect. Future research my change confidence in
estimate of effect.
Low
Low confidence that the evidence reflects true effect.
Future research will likely change both confidence in and
the estimated effect.
Very Low
Very low confidence that the evidence reflects the true
effect. Any estimates are very uncertain.
Hierarchy of Evidence is Too Simplistic
High
Systematic reviews of RCTs
Single RCT
Systematic review of observational studies
Single observational study
Unsystematic clinical observations
Low
Reasons to Downgrade Quality
Reason
What to Assess
Limitation of methodology (bias)
FRISBE of individual studies
Inconsistency of results
Heterogeneity
Indirectness of evidence
PICO
Applicability
Imprecision of results
Wide confidence intervals
Low number of events
Publication bias
Funnel plots
Egger’s, Begg’s
GRADE: Rating Quality of Evidence
*Quality of study moves down 1 or 2 grades. †Quality of study moves up 1 or 2 grades.
Go to kahoot.it
on your browser
Study #1: Rate the Quality
Study #1:
Clinical trial
Randomized
Concealed
Open label : not blinding
Per protocol analysis : bad
Surrogate endpoint : Bad
Wide CI : Bad, imprecise
What is the study quality? Because
High
Moderate
Low
Very low
Study #2: Rate the Quality
Study #2:
Cohort study
Hazard ratio = 3.5 – big magnitude
Dose response gradient – upgrade to
high
Exposure before outcome
Confounding adjusted
using propensity scores
Narrow CI – high
What is the study quality?
High
Moderate
Low
Very low
Two Components of GRADE
Quality of Evidence
◦ High
◦ Moderate
◦ Low
◦ Very low
Strength of Recommendation
◦ Strong
◦ Weak (conditional)
Scenario 1: Recommend or Not??
Benefits
Harms
Scenario 1: GRADE: Strong or Weak?
Benefits
Harms
GRADE: Clinical Implications
Strong Recommendations
◦
◦
◦
◦
◦
For clinicians:
Beneficial outcome: Most individuals should receive intervention
Harmful outcome: Most should not receive intervention
For patients:
Beneficial outcome: Most individuals would want the recommended
intervention; only a small proportion would not
◦ Harmful outcome: Most would not want to receive intervention
Weak Recommendations
◦ For clinicians: Should offer the intervention and include discussion to
make decision (shared decision-making); examine the evidence and
be prepared to weigh individualized pros and cons
◦ For patients: May be split on whether or not they want the
intervention
Factors that Determine
Strength of Recommendation
Quality of the evidence
Balance between desirable & undesirable effects
Values and preferences
Costs (resource allocation)
Scenario 2: Recommend or Not?
Benefits
Harms
Live Longer
Decreased
QOL
Prevent
Heart Attack
Increase
Stroke
Scenario 2: GRADE: Strong or Weak??
Benefits
Harms
Live Longer
Decreased
QOL
Prevent
Heart Attack
Increase
Stroke
Scenario 3: Recommend or Not?
Benefits
Harms
Scenario 3: GRADE: Strong or Weak??
Benefits
Harms
ACC/AHA Rating System (OLD)
SIZE OF TREATMENT EFFECT
PRECISION OF TREATMENT EFFECT
Class I: Benefit >>> Risk
Level A
◦ Procedure SHOULD be performed
Class IIa: Benefit >> Risk
◦ Additional studies needed
◦ It is REASONABLE to perform
Class IIb: Benefit ≥ Risk
◦ Procedure MAY BE CONSIDERED
Class III: Risk ≥ Benefit
◦ Procedure NOT HELPFUL and MAY BE
HARMFUL
◦ Multiple populations evaluated
◦ Data derived from multiple RCTs or metaanalyses
Level B
◦ Limited populations evaluated
◦ Data derived from a single RCT of
nonrandomized studies
Level C
◦ Very limited populations evaluated
◦ Only consensus opinion of experts, case
studies or standards of care
JACC 2009;54:2205-41.
ACC/AHA
Rating
System
(NEW)
ADA Levels
of Evidence
Diabetes Care 2019;42:S1-2.
AAP/AAFP Recommendations
Statement
Definition
Strong
recommendation
Anticipated benefits clearly exceed harms and the
quality of evidence is excellent
Recommendation
Anticipated benefits exceed harms, but the quality of
evidence is not as strong
Option
Either the quality of evidence is suspect or carefully
performed studies have shown little clear advantage
to one approach over another
No recommendation
There is a lack of pertinent published evidence and
that the anticipated balance of benefits and harms is
unclear
Pediatrics 2004;113:1451-65.
Applying to the
Patient
Steps Involved
Can the results be applied to my patient?
• Is the patient the intended target of the guideline?
• Did the guideline make exceptions for your patient
population?
• Do the recommendations fit the current clinical
scenario for your patient?
• Do the recommendations match your patient’s
preferences?
“Killer Bs”
 Burden of illness: Is it too low in the community or is our
patient’s PEER too low?
 Beliefs of individuals and the community: Does it go
against the guideline?
 Bad Bargain: How high is the opportunity cost to
implement? Is it a bad deal?
 Barriers: Geographical, organizational, traditional,
authoritarian, legal, behavioral… too high?
Questions??
Resources
Chapter 26. How to use a patient management
recommendation. Guyatt, ed. User’s Guide to the Medical
Literature. American Medical Association, 2015.
Chapter 28.1. Assessing the strength of recommendations:
The GRADE approach. Guyatt, ed. User’s Guide to the
Medical Literature. American Medical Association, 2015.