Teaching Clinical Effectiveness
Robert Baldor, M.D.
Frank J. Domino, M.D.
Family Medicine and Community Health
University of Massachusetts Medical School
Worcester, MA
Clinical Effectiveness…
…. is the extent to which specific
clinical interventions do what they
are intended to do, i.e. maintain and
improve the health of patients,
securing the greatest possible health
gain from the available resources.
B
To do….
•
•
•
•
the right thing
at the right time
in the right place
with the right result
B
Our Goal
To appreciate the shift to
‘Comparative Effectiveness’
as it relates to teaching
‘Clinical Effectiveness’.
B
By the end of this session, you will..
• Gain an appreciation for the recent emphasis on
‘Comparative Effectiveness‘
• Understand the basic concepts of comparative
effectiveness and the relationship to our current
understanding of evidenced based medicine
• Discover where to find on-line resources to assist
with teaching
• Discuss methods for incorporating comparative
effectiveness into the curriculum for your
students and residents
B
The more things change, the more
they stay the same…
•
•
•
•
HMO’s
Managed care
Capitation
Evidenced-based
medicine
•
•
•
•
Medical Homes
Care management
Global fees
Comparative
effectiveness?
B
Soaring health care costs….
• $2.2 trillion – 16% of GDP in 2007
• Without any changes in federal law, it will rise
to 25 % of GDP in 2025.
B
The American Recovery and
Reinvestment Act (ARRA) of 2009
• Growing concern that we have little solid
evidence of the value of many treatments
• Hope to save money by discouraging the use
of costly, ineffective treatments
B
Comparative Effectiveness Research
• Congress tasked the IOM to recommend
priorities for research to be addressed by CER
and supported by ARRA funds
B
Perspective research…
• Is it better to treat neck pain with surgery or PT & meds?
• What is the best combination of counseling & meds to
treat depression?
• How do drugs compare with surgery to treat intermittent
claudication?
• Is it better to treat CHF by meds alone or by including
home monitoring of BP and weight?
B
A Teaching Perspective
• Your MS III has just seen a patient and
believes they have new onset depression.
“I think he is pretty depressed and needs
medication. On my Psychiatry rotation last
month, citalopram was the SSRI of choice;
they thought it worked better.”
• What would you use for an SSRI and why?
B
CER is done elsewhere…
• Britian, France and others have official bodies
that assess health technologies and compare
the effectiveness, and sometimes the cost, of
different treatments.
B
The CER legislation became a lightning rod …
• Republicans complained that the government
would intrude in a person’s health by enforcing
clinical guidelines.
• Pharmaceutical and medical-device lobbyists fear
the findings will be used to deny coverage for
more expensive treatments and, thus, to ration
care.
B
Rationing ???
• Agreement that researchers should compare the
clinical merits of different treatments.
– Whether they should also consider cost is hotly debated.
• The House Appropriations Committee inadvertently
stoked rationing concerns in a report that stated
‘research comparing different treatments could yield
significant payoffs because less effective, more expensive
treatments will no longer be prescribed’
Sounds like you’re trying to kill Granny to me!
B
Then of course there is Rush…
• Bureaucrats “will monitor
treatments to make sure
your doctor is doing what
the federal government
deems appropriate and
cost-effective”.
What he really meant was to keep
your hands off my drugs!
B
The plan
• The FDA regulates drugs and devices, but the goal is
to establish if a treatment is safe and effective, not if
it was better than the alternatives. The plan is for
– systematic reviews of published scientific studies
– clinical trials of head-to-head comparisons
F
IOM definition
• CER is the generation and synthesis of evidence that
compares the benefits and harms of alternative
methods to prevent, diagnose, treat, and monitor a
clinical condition or to improve the delivery of care.
How is this different from EBM?
F
Your MS III has just seen a patient…
...and believes they have new onset depression
“I think he is pretty depressed and needs
medication. On my Psychiatry rotation last
month, citalopram was the SSRI of choice; they
thought it worked better.”
• What would you use for an SSRI and why?
F
How do you teach “what works better?
Where would YOU look to find this data?
• Pubmed/Clinical Queries?
• In Cochrane?
• In UpToDate?
F
Will CER help us to answer those
questions?
B
IOM prioritization – a political process
• Consulting stakeholders…
• A public session - consumers, advocacy groups,
provider groups, insurers, manufacturers, and
academia.
• A web-based questionnaire received more than
2,600 topics.
B
The selection process:
How an Idea becomes a CER
No
Yes
No
Yes
B
Anybody in this room provide input?
• What’s on your top 10 list?
– For your learners?
Is this different than your list
– as practitioners?
– as patients?
F
Research methodology
• RCTs felt to be the most appropriate
methodology for ½ of the topics
• Prospective observational studies, database
reviews, and systematic literature reviews for
the rest
F
EBM Informational Pyramid
Systematic Review
Evidence Based Review
Randomized Clinical Trial
Cohort Study Prospective
Case Control Study Retrospective
Textbook (Harrison, Uptodate, etc.)
F
CER USE many forms….
1. Systematic literature reviews to summarize evidence,
identify gaps, and generate new ideas for research
2. Randomized controlled trials - considered the gold
standard of evidence, they also have shortcomings
3. Observational research using prospective registries and
large established databases, including EHRs, are used for
cohort studies to understand the natural progression of
disease and the factors that influence clinical outcomes.
F
The Forest Plot
Estimates with 95% confidence intervals
Line of no effect
Kennedy 1997
Locke 1952A
Estimate and confidence
interval for each study
Lopes 1997
Reynolds 1998
Estimate and confidence
for the meta-analysis
Seiberth 1994
Scale (effect measure)
0.2
1.0
Risk ratio
Favours LR
5
Favours control
Direction of effect
Forest Plot: If <> to your Left,
Intervention was Effective at Lowering Risk of Outcome.
F
FOREST PLOT
Meta Analysis: Vitamin D on Falls 400-800 IU per day
F
100 topics, ranked by quartile…..
• Collectively address broad societal needs.
• Conditions with the greatest aggregate effect
on the health of the U.S. population
• Less common conditions that severely affect
individuals in vulnerable subgroups of the
population. (24 of the 100 topics)
B
1st Quartile medical examples……
• Medical Treatments
– low back
– ADHD treatment strategies
– a-fib (surgery v catheter ablation v drugs)
– ACEi/ARBs for CAD
• Medical Diagnostics
– Use of PET, MRI, CT for cancer diagnosing, staging
– Use of genetic and biomarker testing
B
1st Quartile prevention home, community, delivery
• Unintended pregnancies
• Obesity, hypertension, diabetes in at-risk populations
• Primary prevention (exercise and balance training) v clinical
treatments in preventing falls
• Management strategies (e.g., pharmacologic treatment,
social/family support) for dementia in home and institutional
settings.
• Compare the medical home v. usual care in managing those
with chronic disease.
B
4th quartile examples….
• Compare the effectiveness of:
– long-term treatments for acne.
– diagnostic imaging performed by non-radiologists
and radiologists.
B
Back to our student…
• Your MS III has just seen a patient and
believes they have new onset depression.
“I think he is pretty depressed and needs
medication. On my Psychiatry rotation last
month, citalopram was the SSRI of choice;
they thought it worked better.”
• What would you use for an SSRI and why?
• How do you teach “what works better”???
F
Lets answer that question together
• What resources would your learners use…
• What resources would YOU use?
• What kind of a study is found (ie RCT, SR?)
• So what’s the answer?
F
OK lets take a break
• When we come back – you’ll pick a clinical
question related to clinical effectiveness and
search for an answer
Appreciating EBM & CER
1. Pick a clinical question; either one you have on
your mind or one from this list:
•
•
•
•
•
•
•
Treatment for Localized Prostate Cancer
Best Oral Rx for Type II DM
Choosing An Anti Depressant
ACEi or ARB for Hypertension
Best Pain Control in Osteoarthritis
Fracture Prevention in Osteoporosis
Best Management of Gestational Diabetes
Teaching Method for use of CER and
EBM to young (and old) Learners
• Pick a topic of their interest
• Search using conventional methods of
information acquisition
• Find a high level of evidence Systematic
Review on the topic
• Look for a Comparative Effectiveness Review
on the topic
Appreciating EBM & CER
1. Read a summary or review article on your
topic (AFP, eMedicine, etc.)
2. Find a Randomized Controlled Trial using
PubMed Clinical Queries Filters
www.pubmed.gov use CQ*
3. Find a Systematic Review on your question
4. Try to find a Comparative Effectiveness Review
(AHRQ.gov**)
Using Pubmed Clinical Queries
www.pubmed.gov
Pubmed Clinical Queries
Prostate cancer AND local
Appreciating EBM & CER
1. Read a summary or review article on your
topic (AFP, eMedicine, etc.)
2. Find a Randomized Controlled Trial using
PubMed Clinical Queries Filters
www.pubmed.gov use CQ*
3. Find a Systematic Review on your question
4. Try to find a Comparative Effectiveness
Review (AHRQ.gov**)
Comparative Effectiveness Reviews
www.ahrq.gov
After our exercise…
• What’s the curriculum for our students and
residents?
• What are the best methods to teach about
clinical effectiveness?
– Lecture?
– On-line curriculum?
– Workshops?
– Others?