A way forward: better decisions and
better research
Comparing treatments in the new health care
environment
Tim Carey MD MPH
Spring 2013
Support
• NIAMS- National Institute of Arthritis and
Musculoskeletal Disease
• NIH Clinical Translational Science Award to UNC
• NCMHD-National Center for Minority Health and Health
Disparities
• AHRQ-Agency for Healthcare Research and Quality
• Health Resources and Services Administration
• Glaxo Smith Kline Foundation
• Robert Wood Johnson Foundation
• DERP- Drug Effectiveness Review Project
• Dissemination grant supported by the Neurontin Special
Committee
Nothing new
• Clinicians have always compared one treatment with another
• Most conditions have therapeutic options
– Meds vs stent vs bypass surgery for coronary artery disease
– Surgery vs radiation vs surveillance for prostate cancer
– Decompression vs fusion vs exercise for spine disease
– Fluoxetine vs. paroxetine for depression
• Increase in efficacious treatments, and especially expensive
efficacious rx
– Rise in healthcare costs has led to renewed emphasis on
comparative effectiveness and cost-effectiveness
– Direct to consumer advertising
– Information overload for providers
• Increased emphasis on comparing treatments
– Medications with each other
– Procedures with each other
– Procedures compared with medications or physical treatments
(exercise, PT, etc)
One problem
(among many)
• Fractures due to osteoporosis are common, disabling in
the elderly
– Conventional treatment physical therapy, pain control, bonestrengthening medications
• Vertebroplasty was developed in late ‘90’s
– Biologic rationale
• Case series demonstrated marked improvement after
procedure
• > 1,000 procedures in 2007 in NC alone
• 2 randomized trials in 2009 demonstrated minimal, if any
advantage over sham injection, medical regimen
– # of patients across both trials=220
Comparative Effectiveness Research
CER is the generation and synthesis of evidence that
compares the benefits and harms of alternative methods to
prevent, diagnose, treat and monitor a clinical condition or to
improve the delivery of care. The purpose of CER is to
assist consumers, clinicians, purchasers and policy makers
to make informed decisions that will improve health care at
both the individual and population levels.
Institute of Medicine, 2009
Patient Centered Outcomes Research
• Patient-Centered Outcomes Research (PCOR) helps people and
their caregivers communicate and make informed health care
decisions, allowing their voices to be heard in assessing the value of
health care options. This research answers patient-centered
questions such as:
• “Given my personal characteristics, conditions and preferences,
what should I expect will happen to me?”
• What are my options and what are the potential benefits and harms
of those options?”
• What can I do to improve the outcomes that are most important to
me?”
• “How can clinicians and the care delivery systems they work in help
me make the best decisions about my health and healthcare?”
PCORI board-2012
What is being compared?
•
•
•
•
Similar, definable treatments?
Appropriate outcomes that matter to patients?
Are harms being searched for?
Is the comparison treatment the current state of the art
treatment?
• Encompasses comparing systems of care as well as drug A
vs drug B
– Care management
– Self-care, exercise
– Payment issues
– Care integration
• Patient and stakeholder preferences taken into account?
– Relationship to patient-centered outcome research
Structuring CER
•
•
•
•
•
•
Population
Intervention
Comparator
Outcome
Timeframe
Setting
Schematic for patient outcomes
research
Coke vs Pepsi
• Risk of losing perspective- how well does treatment work at all
for the condition
• Is it an interesting question to compare two similar
medications (or procedures)?
– Two statins
– Patent vs generic
– Harm profiles
– Drug vs procedure; invasive vs non-invasive
• What criteria should drive research investment?
– Secondary analysis vs primary data collection
– Large, simple trials
– Which conditions are most important to examine?
– Who are the constituencies? Agencies, manufacturers,
researchers, advocates, patients, the public?
COMPARATIVE EFFECTIVENESS OF SECONDGENERATION ANTIDEPRESSANTS IN THE
PHARMACOLOGIC TREATMENT OF ADULT
DEPRESSION
Do antidepressants differ in efficacy and
effectiveness for the treatment of major
depressive disorder, dysthymia, and
subsyndromal depression?
Included Medications
SSRIs
Other
Citalopram
Escitalopram
Fluoxetine
Fluvoxamine
Paroxetine
Sertraline
Bupropion
Duloxetine
Mirtazapine
Nefazodone
Venlafaxine
Trazodone
Results: Excluded Studies
62 studies excluded because of
poor internal validity
•
•
•
•
High loss to followup
Single blinding
No intention-to-treat analysis
No systematic literature search for
systematic reviews
Major Depressive Disorder:
Body of Evidence
• 72 head-to-head trials (including 3
effectiveness trials) on 16,780 patients
• 18 studies assessed quality of life
• We conducted 4 meta-analyses and 62
adjusted indirect comparisons
– Outcome of interest: response to
treatment
Major Depressive Disorder:
Evidence of Comparative Efficacy
• Overall, no substantial differences in
efficacy
• Statistically significant results from
meta-analyses: modest and likely not
clinically important
• No differences in quality of life
Strength of evidence: moderate
Favors SSRI
Favors SSNRI
SSRI vs. SSNRI
Citalopram vs. Duloxetine
0.76 (0.39, 1.47)
Escitalopram vs. Duloxetine
0.97 (0.71, 1.33)
Fluoxetine vs. Duloxetine
1.12 (0.84, 1.50)
Fluvoxamine vs. Duloxetine
1.59 (0.30, 8.45)
Paroxetine vs. Duloxetine
1.50 (0.88, 2.53)
Favors SNRI
Sertraline vs. Duloxetine
1.27 (0.99, 1.64)
Favors SSRI
SSRI vs. SNRI
Citalopram vs. Mirtazapine
0.78 (0.40, 1.53)
Escitalopram vs. Mirtazapine
1.01 (0.74, 1.37)
Fluoxetine vs. Mirtazapine
0.87 (0.72, 1.06)
Fluvoxamine vs. Mirtazapine
1.64 (0.31, 8.76)
Paroxetine vs. Mirtazapine
1.08 (0.88, 1.33)
Sertraline vs. Mirtazapine
0.92 (0.74, 1.14)
Citalopram vs. Venlafaxine
0.79 (0.41, 1.52)
Escitalopram vs. Venlafaxine
1.02 (0.82, 1.26)
*Fluoxetine vs. Venlafaxine
1.21 (1.01, 1.24)
Fluvoxamine vs. Venlafaxine
1.66 (0.31, 8.81)
Favors SSNRI & SNRI
Favors SNRI
Paroxetine vs. Venlafaxine
1.05 (0.75, 1.49)
Sertraline vs. Venlafaxine
0.88 (0.72, 1.07)
SSNRI & SNRI vs. SNRI
Duloxetine vs. Venlafaxine
1.28 (0.86, 1.91)
Duloxetine vs. Mirtazapine
Mirtazapine vs. Venlafaxine
1.03 (0.79, 1.35)
* Based on meta-analysis of head-to-head trials
0.2
0.5
1
1.01 (0.81, 1.27)
2
5
10
Major Depressive Disorder:
Evidence of Comparative Efficacy
• Although efficacy is similar, secondgeneration antidepressants are not
identical
– Mirtazapine has a significantly faster onset
of action than SSRIs
– Bupropion has less effect on sexual
functioning than SSRIs
– Strength of evidence: moderate
Ongoing Issues for
Clinicians and Patients in depression treatment
• Multiple treatment options may be necessary for
many patients:
– 40% of patients do not achieve clinical
response with initial treatment
– 10% - 15% discontinue treatment because of
adverse events
– Antidepressants differ significantly in dosing
regimens
– Need for treatment of med-refractory patients
• Add medication? Switch medication?
• When to use non-drug therapy such as ECT?
Use of existing data
Its not just claims data anymore
• Many important comparisons can’t or won’t be evaluated
in trials
• Improved methods to control for selection bias
– Propensity score, instrumental variables, etc
• Enhanced methods to link databases
– Disease registry linked with claims
– Patient reported outcomes linked with EHR, etc
• Somewhat improved access to data
– Governance and security issues
– Cost of data acquisition decreasing(?)
• Electronic health records
– Great promise, still a lot to learn
The Weight of the Evidence
Dissemination and implementation
challenges
• Research reports are long, technical and full of
jargon
• Not all research is of equal quality
– Potential for bias in CER research
• Links with health information technology
initiatives promising but still early
• Providers and ‘prompt fatigue’
Translation:
Consumers
Source: ConsumerReportsHealth.org
Applicability
Does one size fit all?
• Occasionally a research study applies to nearly
everyone with a condition
• Most analyses have only limited ability to assess
treatment effectiveness in sub-populations
– Age, gender, ethnicity, income, co-existing conditions,
genetic factors, prior treatment
• Large research networks, hopefully incorporating large
amounts of insurance claims data and/or EHR data are
promising….
• Need to take patient preferences into account when
applying research results
• Need for more information in these areas should not be
an excuse for anything goes
Source: Schumock & Pickard; Am J Health-Syst Pharm 2009
Pragmatic clinical trials
• Prospective comparison of a community, clinical, or
system level intervention and a relevant comparator in
participants who are similar to those affected by the
condition(s) under study and in settings that are similar
to those in which the condition is typically treated.
• Underused, promising to enhance applicability, can
engage stakeholders, PBRN’s
• Significant infrastructure needed to reduce cost:
– Governance and IRB issues
– Informatics
– Methods issues
Next steps for CTSA’s to enhance
pragmatic trials
• Each CTSA to have standing access to communities,
practices and stakeholders for trial development, as well as
implementation and dissemination
• Regulatory relief: Modify informed consent for broad
effectiveness studies. Central or deemed IRB. Contracting
relief as well.
• Learn by doing- work with a set of pragmatic trialsrelationship with the IC’s. Use selected pragmatic trials as
basis for learning.
• Data terminology issues across sites, prioritize the most
important issues, need policies in place to assist the
community, need community input.
• Methods focus in study design and analysis of
treatment/site/patient heterogeneity
Enhancing multi-site studies
• Need for inexpensive, rapid data collection
• Challenges of heterogeneity of
environment across sites
– Heterogeneity of intervention?
• Trialists and PBRN’s may have different
views of ‘trials’
• Timing and type of engagement
– Practice, patient, public communities
Importance:
Why We Need to Identify and Prioritize Research Gaps
from Systematic Reviews
Primary
Research
Systematic
Review
Assessment
of
Gaps
Prioritization
of
New
Research
Funding
Opportunities
Conduct
New
Research
Systematic
Review
Update
• Systematic reviews are the standard for evaluating the current state of
scientific knowledge regarding a specific clinical or policy question.
• Identification and prioritization of research gaps has the potential to lead
to more rapid generation of subsequent research, informed by input
from stakeholders
• Current studies often use only very general terms to describe future
research ‘more and larger studies should be performed.’
• Audiences including researchers, funders, clinicians, advocates, and
patients could use information about prioritized research gaps to
understand areas of uncertainty and more quickly initiate studies.
32
Existing Methods to Identify and Prioritize
Research Gaps
Agency for Healthcare Research and Quality Future Research Needs
– Extract research gaps
from a systematic
review, transforming
them into prioritized
research questions
with aided by diverse
stakeholder groups.
– 7 steps common to
AHRQ FRN projects.
1
• Systematic review is published with EPC-determined
research gaps
2
• Orientation of stakeholders to CER question, FRN
process, and prioritization criteria
3
4
5
6
7
• Elaboration and consolidation of research gaps through
iterative process with stakeholders
• Priority ranking of the research gaps
• Transformation of research gaps into needs
• Refinement and re-ranking of priorities by stakeholders
• Addition of study design considerations
33
Example: AHRQ Future research needs on ADHD
Key Questions from Comparative Effectiveness Review
KQ1
KQ2
KQ3
Among children less than 6 years of age with Attention Deficit Hyperactivity Disorder or
Disruptive Behavior Disorder, what are the effectiveness and adverse event outcomes following
treatment?
Among people 6 years of age or older with Attention Deficit Hyperactivity Disorder, what are the
effectiveness and adverse event outcomes following 12 months or more of any combination of
follow-up or treatment, including, but not limited to, 12 months or more of continuous
treatment?
How do A) underlying prevalence of Attention Deficit Hyperactivity Disorder, and B) rates of
diagnosis (clinical identification) and treatment for Attention Deficit Hyperactivity Disorder vary
by geography, time period, provider type, and sociodemographic characteristics?
20 research gaps from the review mapped to the key questions, presented to a
group of 12 stakeholders, including funders, advocates, clinicians, regulators,
researchers, and policymakers.
After stakeholder input, 29 research gaps. 8 gaps emerged as the top future
research needs after two rounds of prioritization using an online prioritization
tool.
The next two slides show the presentation of one gap from identification to study
design.
34
Example: AHRQ FRN on ADHD
Identify Research Gap:
For children less than 6 years of age with disruptive behavior disorder or ADHD, limited
data are available about the efficacy and effectiveness of psychosocial treatment
programs (e.g., parent training and summer behavior treatment programs), alone or in
combination with pharmacological interventions, compared with other psychosocial
treatment programs, alone or in combination with pharmacological interventions. (KQ 1)
After One Round of Prioritization Apply PICOTS and Develop Research Question:
Population
Intervention
Age < 6 years
Diagnosed with ADHD or
at risk for ADHD or
diagnosed with disruptive
behavior disorder
(including ODD and CD
by DSM)
Psychosocial
interventions alone
(including parent
training and schoolbased interventions)
Comparator
Pharmacological
treatments, alone
or in combination
with psychosocial
treatments
Outcomes
Timeframe/Setting
Outcomes
for children
and
parents*
6 Months/ 1Year
Private clinic,
community clinic
Research Question: For children less than 6 years of age with disruptive behavior
disorder or ADHD, what is the comparative efficacy and effectiveness of specific
psychosocial treatments alone compared with pharmacological treatments alone or in
combination with psychosocial treatments for patient outcomes?
35
Example: AHRQ ADHD FRN
After Second Round of Prioritization Develop Study Design Considerations:
Randomized controlled trials
Randomized trials could be designed to test various components in a 2x2 matrix of
psychosocial treatment variants (parent training, school-based intervention,
combination, or pharmacological).
• Advantages of study design for producing a valid result
Allows isolation of causal inferences related to the intervention being tested.
Multiple-armed trials would allow testing of several hypotheses regarding relative
efficacy of singular or combination treatment components.
• Ability to recruit/availability of data
Common condition in this age group with uncertainty regarding treatment choice;
all arms receive some treatment.
• Resource use, size, and duration
Large sample size (N = 840; n = 210 per treatment arm) needed. Key outcomes
such as school achievement will require follow-up of several years.
• Ethical, legal, and social issues
Vulnerable population, careful informed consent will need to occur.
36
State of the Science
• Multiple groups are currently conducting work in this
area
– Local priorities important to assess
• Sufficient common aspects to serve as a consensus
– Criteria for gaps identification
– Broad social input
– Need for stakeholder training
– Explicit prioritization method - but multiple methods
currently used
– Decisions regarding study design considerations
• Prioritization should never imply discouraging scientific
creativity
• Early evaluation of new technology is critical
37
What about costs?
Cost-effectiveness analysis (CEA)
• CEA integral to European efforts,
especially NICE
• Perspective is critical
• Some organizations forbidden from
conducting CEA under Affordable Care
Act
• Cost savings vs cost-effectiveness
• Risks of back-of-the envelope CEA
Effectiveness and ‘value’
• Deferred future costs
• Value of information analysis
– Models in both planning for and interpreting research
•
•
•
•
MedPAC analyses
Patient-Centered Outcome Research Institute (PCORI)
(Independent Payment Advisory Board-IPAB)
CMS activities
– Coverage with evidence development
– CMS Innovation Center
• State-based initiatives
• Importance of distinguishing between absolute and
comparative savings
Who is sponsoring the work?
The alphabet soup
•
•
•
•
•
•
Patient-Centered Outcomes Research Institute (PCORI)
• Private entity, multi-stakeholder board
• ~$550M budget by 2014
• Strong focus on role of patient/caregiver in research and decisionmaking
FDA
– Regulatory role, not research
NIH
– Historically not involved with CER, interest significantly higher now
– ALLHAT, CATIE, STAR*D, SPORT. More to come?
– CTSA and ‘Type II’ (bench to bedside) translational research
AHRQ
– Effective Health Care Program; EPC’s, DEcIDE, etc
– Education, dissemination, collaboration with PCORI
Drug Effectiveness Review Program: state Medicaid agencies
Industry
– Limited incentive
– “Do you feel lucky”- some potential to game comparisons
– Vast amounts of marketing
Public good, public guardian
• Widespread recognition that current system is
dysfunctional
• Evidence takes too long and is too expensive to
generate
• The presence of research doesn’t mean that it will be
used
• FDA role likely to change
– Avandia, Vioxx, stents, vertebroplasty, etc
• CMS taking increasing role
• State Medicaid programs form consortia, major pressure
to constrain costs, technology utilization
• Loud megaphone of marketing
Challenges
• Substantial, but heterogeneous, current activity
• Need to train additional researchers
– MD, PhD; no consensus yet on core competencies
– Train clinicians, public, administrators in use of
research
• Dissemination of findings into practice
• Relationship to health information technology initiatives
– Need for transparent relationship with vendors
– Information must be combined with ease of
implementation
– Several infrastructure proposals pending to utilize
EHR data for research, form consortia- long term
goals
Resources
• Patient Centered Outcomes Research
Institute www.pcori.org
• Agency for Healthcare Research and
Quality www.ahrq.gov
• Consumer’s Union
http://www.consumerreports.org/health/best-buy-drugs/index.htm
• Cochrane collaboration
– http://www.cochrane.org/
• Drug Effectiveness Review Project DERP
– http://www.ohsu.edu/drugeffectiveness/
Comparative effectiveness research
• (Sort of) new wine
– Interest is predominantly driven by technology
availability, payer interest, rising chronic disease
burden
• New bottle
– Federal and payer interest likely to be great in the
next few years
– Focus on stakeholder engagement and prioritization
– Dissemination and implementation
– Pro-active discussion to avoid political problems
– Critical will be to maintain equipoise
• Some research will find that more expensive
treatments may be a dominant strategy
Thank you and questions?
Go Heels
Beat Duke
IOM Report:
Commissioned by
Congress in the American
Recovery and
Reinvestment Act (ARRA)
of 2009
June 30, 2009
Examples of “Highest Priority”
• Compare the effectiveness of dissemination and
translation techniques to facilitate the use of
CER by patients, clinicians, payers, and others.
• Compare the effectiveness of comprehensive
care coordination programs, such as the medical
home, and usual care in managing children and
adults with severe chronic disease, especially in
populations with known health disparities.
IOM June 30, 2009
Examples of “High Priority”
• Compare the effectiveness of strategies for
enhancing patients’ adherence to medication
regimens.
• Compare the effectiveness of diverse models of
transition support services for adults with
complex health care needs (e.g., the elderly,
homeless, mentally challenged) after hospital
discharge.
IOM June 30, 2009
Activity in North Carolina
• Clinical Trials
– All academic health centers, private sector (Quintiles, etc)
• Drug Effectiveness Review Project (Medicaid consortium)- UNC
• PCORI- limited activity to date, UNC and Duke participating
• Evidence-based Practice Centers (AHRQ)
– RTI-UNC; Duke
• Secondary data analyses: Developing Evidence to Inform Decisions
about Effectiveness (AHRQ DEcIDE network)
– Duke; UNC
• Dissemination activities
– Lineberger Cancer Center, CTSA network at UNC and Duke
• Centers for Evidence and Research on Therapeutics (AHRQ and
FDA CERT)
– Duke