RTI International
Methods for Comparative Effectiveness
and Health Services Research
Kathleen N. Lohr, PhD
Distinguished Fellow (Health Services Research)
RTI International
Research Triangle Park, NC USA
AcademyHealth
Tuesday June 14, 2011
Seattle Washington
RTI International is a trade name of Research Triangle Institute.
www.rti.org
RTI International
CER Methods Research—Gaps and Needs


CER defined well by previous speakers
Need to be clear, though, that CER
Covers more than clinical research
– Involves health services research
– Can go beyond condition-specific health care to health care
delivery and systems
– Includes both generation and synthesis of evidence
–

For CTSAs:
3 “critical needs” with 10 “priority areas” for methods
– Today: One priority area from each of the three buckets
–



Value of information
Treatment heterogeneity
Translation and dissemination
RTI International
Value of Information



Relatively new concept; multiplicity of subparts
Methods research as applied to decisionmaking
about CER in CTSAs
Methods research needed, also, for decisionmaking
about systematic reviews and health technology
assessments: e.g.,
Value of doing a systematic or comparative effectiveness
review on one topic vs. another
– Value of doing an update -- of a review; of a guideline
–

Overarching questions
Value to whom?
– Interactions: CER and systematic review feedback loops?
–
RTI International
Heterogeneity of Treatment Effects
As well illustrated and explained:

HTE = major challenge
For researchers in designing studies, analyzing data
– For investigators in conducting systematic reviews or
health technology assessments
– For clinical professionals and guidelines developers
–

Numerous approaches to addressing the program
and reporting results – but still need to be widely
adopted
RTI International
Heterogeneity

Related concepts and methods challenges:
Clinical heterogeneity: “variability in study population
characteristics, interventions, and outcomes across
studies”*
– Statistical and/or methodologic heterogeneity: causal
relationship (or not) with clinical heterogeneity?
– Effect-measure modification: a intervention-disease
association that differs according to the level of a factor
under investigation
– Applicability (external validity; generalizability): “extent to
which effects observed in published studies are likely to
reflect the expected results when an intervention is applied
to a pre-specified population under ‘real-world’ conditions”†
–
*Gartlehner et al., IJTAHC, forthcoming
†Atkins et al., JCE, 2011
RTI International
Heterogeneity

Any unifying concepts for CER and for guidelines
developers – that call for methods research?
–
PICOTS






Patients/populations
Interventions
Comparators
Outcomes
Time frames
Settings
Net benefits: totality of benefits and harms (risks)
– Information for informing patient care
–



Shared/informed decisionmaking
Individualized (personalized) care
Are “averages” or “subgroup” results ever enough (maybe sometimes?)
RTI International
Translation and Dissemination

Getting stakeholder involvement in research (or
systematic reviews) – only a first, early step
Relatively easy: professional groups, disease advocacy
groups, insurers, payers, health care systems
– Not so easy: Patients and patient advocacy groups
– Consider role of community-based participatory research
–



Moving research findings into practice:
overcoming the “17-year lag”
Conveying the meaning of “uncertain” results, and
helping decisionmakers deal with uncertainty
Reaching patients/families/consumers:
overcoming literacy and numeracy challenges
RTI International
CER Research Design, Statistics, Analysis


AHRQ CER Methods conferences (2006, 2009, 2011)
plus Medical Care supplements (2007*, 2010†)
Broad range of topics
New and expanded designs: randomization techniques;
pragmatic/effectiveness trials; practice-based studies; registry
studies
– Data collection: electronic, computer- or web-based platforms
and hardware; EMRs; distributed data networks; administrative
to primary data (e.g., patient-reported outcomes), and
combinations of these
– Established/newer statistical and analytic methods: propensity
scores to Bayesian techniques; confounding; missing data
– Applications to policy and clinical decisionmaking
–
* AHRQ Effective Health Care program or DEcIDE sites: http://effectivehealthcare.ahrq.gov/index.cfm
† http://journals.lww.com/lww-medicalcare
RTI International
Patient-Centered Care and CER


PCORI and its Methodology Committee is a start
Patient-Reported Outcomes: another challenge for:
Meeting CER (and CTSA) and health services research
agendas
– Conducting Systematic reviews and health technology
assessments
– Compiling clinical practice guidelines
– Using routinely in clinical practice
–

PROs: Concepts, Measures, and Tools
Decades of measurement and instruments
– Newer tools: PROMIS (NIH): http://www.nihpromis.org/
– Translational: from research to practice*
–
*Lohr & Zebrack, QOL Research 2009