Comparative Effectiveness &
Technology Assessment
Call for Papers Session
Comparative Effectiveness Research: The Increasing
Elegance of ‘‘Coke vs. Pepsi’’ Studies
Chair: Tanisha Carino, Ph.D.
Tuesday, June 5 • 9:00 a.m.-10:30 a.m.
ƒ Improving HIV Screening with Nurse Rapid Testing and
Streamlined Counseling
Henry Anaya, Ph.D., Steven M. Asch, M.D., M.P.H., Tuyen
Hoang, Ph.D., Matthew Goetz, M.D., Allen Gifford, M.D.,
Candice Bowman, Ph.D.
Presented By: Henry Anaya, Ph.D., Research Scientist, U.S.
Department of Veteran's Affairs, 11301 Wilshire Boulevard,
111G, Los Angeles, CA 90073, Phone: (310) 478-3711,
Email: henry.anaya@va.gov
Research Objective: Testing for HIV has been shown to be
cost effective in unselected general medical populations, yet
rates of testing among those at risk remain far below optimal,
even among those with regular primary care. The specific
aims of this project are: oTo determine whether nurse-based
referral for traditional HIV testing and counseling will improve
screening rates compared to current testing procedures. oTo
determine whether nurse-based rapid testing with streamlined
counseling improves screening rates more than nurse-based
referral for traditional testing and counseling alone.
Study Design: A parallel-group, controlled study was
conducted in the primary/urgent care clinics of the West Los
Angeles VA. Eligibility was based on same-day appointment;
age (18-65); no prior HIV test in past year; unknown HIV
status. One hundred sixty six patients were randomized to one
of three screening models: Model A: patients urged to discuss
testing with their physician (control). Model B: nurses offered
traditional counseling/testing. Model C; nurses offered
streamlined counseling/rapid testing. Interventions were
performed by nurses in addition to their regular clinic duties.
Population Studied: honorably discharged veterans between
the ages of 18-65 with unknown HIV status
Principle Findings: Model A: 22 patients (40.7%) had test
ordered; Model B: 48 (84.2%) had test ordered; Model C: 51
(92.7%) had test ordered. Of 22 patients in Model A with a
test order, 9 (40.9%) received results; of 48 patients in Model
B with test order, 25 (52.1%) received results; of 51 patients in
Model C with test order, 46 (90.2%) received results.
Conclusions: Results show that both interventional models
will likely result in higher screening rates than traditional HIV
testing models in primary care.
Implications for Policy, Practice or Delivery: HIV rapid
testing has been shown to be an effective means by which to
convey results to patients, which is especially salient given the
approximately 300,000 persons in the US alone who are
unaware of their HIV-positive status. Increased rates of
testing could lead to earlier identification of disease, increased
treatment and reduced morbidity and mortality. Reduced
intensity of counseling might free staff resources. As the VA is
the largest HIV care provider in the US, it would be beneficial
for policymakers to use this project and associated findings as
a model when considering implementing rapid testing on a
regular basis.
Funding Source: VA
ƒ Constructive Technology Assessment of Microarray
Testing in Breast Cancer Treatment
Willem Van Harten, M.D., Ph.D., Jolien M. Bueno-deMesquita, M.D., Valesca P. Rètel, M.Sc., Kim Karsenberg,
M.Sc., Marjan M. Hummel, Ph.D.
Presented By: Willem Van Harten, M.D., Ph.D., Member Ex.
Board of Directors and Prof. Quality Management of Health
Care Technology, Organisation & Management, School of
Management and Governance, The Netherlands Cancer
Institute and University of Twente, Plesmanlaan 121,
Amsterdam, 1066 CX, Netherlands, Phone: 0031 20 5122860,
Fax: 0031 20 6691449, Email: w.v.harten@nki.nl
Research Objective: Performing technology assessment in an
early stage of the controlled implementation of the prognostic
70-gene signature, a genomic test using microrrays, in the
treatment of node-negative breast cancer patients.
Study Design: As the technology was in its earliest stage of
clinical implementation the Constructive Technology
Assessment approach was chosen. This method is related to
theories on Technology Dynamics and was developed in the
field of Public Policy. All aspects as defined by the Institute of
Medicine as well as technology related issues, such as juridical
aspects, were covered. Method used: -Documentation analysis
of internal process dynamics of the organisations before and
after introduction. -Multidisciplinary team interviews before
and after introduction. -Patient consultation recordings and
patient questionnaires. -Descriptive registration and
comparison of prognosis assessment using traditional
clinical. guidelines and the genomic 70-gene microarray test. Scenario drafting and revision.
Population Studied: In this implementation study, 812
eligible patients from 16 Dutch hospitals participated and 425
70-gene microarray test were performed.
Principle Findings: - Introducing and implementing this new
technology in clinical practice took on average about 6 months
per participating hospital.- Especially pathologists had to
change their working routine as they were requested to
process the tissue directly after surgical removal of the tissue.Health care professionals had to take more than average time
than before to properly explain to the patient the pro’s and
con’s of this test.- In 30% of the cases the patients’ prognosis
based on clinical guidelines was discordant with the prognosis
based on the 70-gene microarray test. - In general, patients
were accepting the specialists’ treatment advice. This was
even the case if the patients’ prognoses based on clinical
guidelines was discordant with genomic prognosis based on
the 70-gene microarray test. - Scenario development proved
feasible and provided options in the early stage that were
considered unlikely by professionals, but nevertheless became
reality.
Conclusions: - The implementation of new genomic tests like
the 70-gene microarray test in node-negative breast cancer
patients is a complex process. - Constructed Technology
Assessment is a promising method to analyse technologies in
their early stage of development and implementation that are
introduced in a controlled way. - Health care professionals
have to anticipate to 30% discordance between prognosis
assessed by traditional clinical guidelines versus genomic
testing.
Implications for Policy, Practice or Delivery: - The
controlled introduction of promising genomic tests is feasible.
- Constructive Technology Assessment is a promising broad
assessment method for the implementation of new
technologies in an early stage of their development. - Based
on the experiences of this controlled implementation trial, a
large European randomised trial (MINDACT-trial) was
designed and will start in January 2007.
ƒ Looking for Modifiers of Treatment Effects in the
General Medical Literature: Room for Improvement
Nicole Bloser, M.H.A., M.P.H., Naihua Duan, Ph.D., Diana
Liao, M.P.H., Elizabeth Yakes, M.S., Kiavash Nikkhou, B.S.,
Richard L. Kravitz, M.D., M.S.P.H.
Presented By: Nicole Bloser, M.H.A., M.P.H., Graduate
Student Researcher, University of California, Davis, 2103
Stockton Boulevard, Grange Building, Suite 2224, Sacramento,
CA 95817, Phone: (916) 734-2399, Fax: (916) 734-8731, Email:
nrbloser@ucdavis.edu
Research Objective: Randomized controlled trials (RCTs)
generate average treatment effects, but patients want to know
which treatments will work for them. Individualizing care for
the complex patient requires knowledge of treatment impact
in similar individuals or subgroups, which in turn depends on
identifying moderators of treatment effects (MTEs). In an
effort to avoid the appearance of ‘‘data dredging,’’ clinical
investigators may be missing opportunities to explore MTEs,
thus slowing accrual of evidence for treating ‘‘patients like
me.’’ This study was undertaken to determine current practice
in evaluating MTEs and to elucidate trends.
Study Design: We examined a probability sample of 227
RCTs. Articles were independently reviewed and coded by 2
investigators with adjudication by a third. Studies were
classified as having: a) MTE analysis utilizing a formal test for
heterogeneity or interaction; b) subgroup analysis only,
involving no formal test for heterogeneity or interaction, or c)
no subgroup or MTE analysis. Chi-square tests and multiple
logistic regression analysis were used to identify study
characteristics predictive of MTE reporting.
Population Studied: 227 RCTs published in 5 journals (Ann
Intern Med, BMJ, JAMA, Lancet, and NEJM) during odd
numbered months of 1994, 1999, and 2004.
Principle Findings: Of the 227 RCTs, 101 (44%) performed no
subgroup or MTE analysis, 62 (27%) examined subgroups but
without MTE analysis, and 64 (28%) performed MTE analysis.
MTE analysis gained currency with time (18%, 29%, and 34%
of studies in 1994, 1999, and 2004, respectively). Among the
64 studies reporting MTE analysis, major covariates examined
included age (30%), sex (28%), study site or center (17%), and
race/ethnicity (8%). Using multiple logistic regression to
examine study year, journal, clinical condition, and sample
size, only sample size was a significant predictor of whether
MTE analysis was performed; comparing the top quintile of
studies (median n=1649) to the bottom quintile (median
n=36), the adjusted odds ratio was 4.9 (95% CI 1.6-15.1,
p=.0045). However, MTE analysis was performed less than
half the time (49%) even in the top quintile.
Conclusions: Missed opportunities for MTE analysis abound.
In the face of broad NIH mandates for inclusion of subjects by
race/ethnicity, the low proportion of studies testing
race/ethnicity as a treatment effect moderator is both puzzling
and disappointing.
Implications for Policy, Practice or Delivery: Accepting
Kraemer et al.´s argument (JAMA, 2006) that exploratory
moderator analysis is critical for designing appropriate future
confirmatory studies, standards are needed to assure that
exploratory moderator analysis and reporting become rigorous
and routine. Such standards are essential for developing
practice guidelines that are appropriate to the needs of the
complex patient.
Funding Source: Pfizer Inc.
ƒ Decision Makers' Attitudes Toward Cost Effectiveness
Analysis
Shoshanna Sofaer, Dr.P.H., Stirling Bryan, Ph.D., Taryn
Siegelberg, M.P.A.
Presented By: Shoshanna Sofaer, Dr.P.H., Robert P. Luciano
Professor of Health Care Policy, School of Public Affairs,
Baruch College, One Bernard Baruch Way, Box D901, New
York, NY 10011, Phone: (646) 660-6815,
Email: shoshanna_sofaer@baruch.cuny.edu
Research Objective: While cost-effectiveness analysis (CEA)
is part of the policymaker’s toolkit in making insurance
coverage decisions in many western nations, it is largely
unused in the US. We have little empirical knowledge about
what gives decision-makers pause about using CEA. Our
research sought to understand receptiveness to CEA among
those who influence coverage decisions, and how institutional
constraints, individual values and methodological concerns
shape decision-maker views.
Study Design: This exploratory study collected data through
six structured workshops in California. During these 3-4 hour
workshops/focus groups, participants were asked to take on
the role of ‘‘social decision-maker’’ addressing issues of
concern to the Medicare program. CEA methods were
explained; ethical/normative issues inherent in CEA were
discussed; and participants prioritized 14 treatments for
coverage in response to information from published CEA
studies. At the end of each session, participants removed
their social decision maker ‘‘hat’’ and discussed, from an
organizational perspective, advantages and barriers to CEA. A
pre-group survey addressed knowledge and attitudes to CEA
and gave respondents an opportunity to prioritize the 14
treatments based on effectiveness information alone. A postgroup survey re-asked knowledge and attitude questions
adding other questions about the workshop and CEA. Survey
data were analyzed descriptively at both points in time.
Changes over time were assessed statistically, as were
changes in priorities following the presentation of costeffectiveness data. All sessions were audio-taped, transcribed,
and coded and analyzed using NVivo software.
Population Studied: Participants included senior leaders
(both clinical and non-clinical) from different types of health
insurance plans, private and public sector health care
purchasers, disease management organizations, and state
regulators of managed care plans.
Principle Findings: In the post-workshop survey, over 90% of
participants indicated that CEA should be used as an input
into coverage decisions for Medicare and over 70% said it
should be used in private insurance plans. Participants also
identified reasons to avoid using CEA, including: fears of
negative consumer perceptions; litigation risks; inadequate inhouse expertise; worries about biased studies; concerns about
ethical issues; and a preference for reducing costs by reducing
demand. Many noted that no single entity, particularly in the
private sector, could ‘‘go it alone’’ in explicitly using CEA.
Finally, when provided with cost-effectiveness information on
a variety of condition-treatment pairs, participants changed
their priorities to fund treatments with more favorable cost
effectiveness ratios, and exclude treatments with higher ratios.
Conclusions: Senior California decision-makers believe that
CEA is an effective and promising tool to assist cost
containment.. Some are not confident in their ability to
acquire, assess and apply high quality CEA studies. Cost
effectiveness data was influential in changing hypothetical
decisions about coverage in the Medicare program.
Implications for Policy, Practice or Delivery: There is a need
to identify and motivate a visible policy leader in health care,
such as the Medicare program, to move in the direction of
making CEA studies an input into coverage decisions. Better
promotion of the availability of high-quality CEA studies and
more accessible information about the elements of such
studies, is needed.
Funding Source: California Health Care Foundation
ƒ Using Observational Data to Extend the Results of a
Randomized Controlled Trial: An Application to the HOPE
Trial
Paul Hebert, Ph.D., Mary Ann McLaughlin, M.D., M.P.H., Jodi
M Casabianca, M.S., Anu Lala, M.D.
Presented By: Paul Hebert, Ph.D., Assistant Professor, Health
Policy, Department of Health Policy, Mount Sinai School of
Medicine, One Gustave L Levy Place, Box 1077, New York, NY
10029-6574, Phone: (212) 659-9567, Fax: (212) 423-2998,
Email: paul.hebert@mssm.edu
Research Objective: The purpose of this study was to explore
the use of observational data to extend the results of a
randomized controlled trial (RCT). We considered the Heart
Outcomes PrEvention (HOPE) RCT, which demonstrated
broad clinical benefits of ramipril, an angiotensin converting
enzyme inhibitor (ACEI), for patients with risk factors for
cardiovascular disease.
Study Design: We conducted a retrospective statistical
analysis. We first attempted to recreate the findings of the
HOPE RCT using observational data and instrumental variable
(IV) models, to assess whether IV analyses can address the
selection bias inherent to observational data. With the
success of this analysis, we then applied these same methods
to a sample of non-white patients. Non-white patients
accounted for <5% of HOPE trial participants. We also
assessed whether other ACEIs showed clinical benefits similar
to ramipril. Data came from administrative databases for
dually-eligible Medicare and Medicaid beneficiaries from
California from 1996-99------the period during which HOPE was
conducted. Medicaid claims for prescriptions identified
patients taking antihypertensives in 1997. Diagnosis codes on
Medicaid/Medicare claims identified risk factors in 1996. The
outcome was the composite endpoint of all-cause mortality or
hospitalization for stroke or myocardial infarction (similar to
HOPE) over the 1997-99 period. Instrumental variables were
managed care market penetration in the beneficiary’s county
of residence, and the ratio of ramipril to other ACEI
prescriptions filled in a beneficiary’s ZIP code in 1996. We
estimated linear probability models (LPMs) and IV LPMs with
instruments for ramipril use, both with robust standard errors.
To assess whether similar clinical benefits could be attributed
to other ACEIs, we identified patients who were prescribed
other ACEIs (benezapril, captopril, and enalapril) in 1997 and
estimated multivariate Cox models of the time to the
composite endpoint as a function of the ACEI prescribed.
Population Studied: Dually-eligible Medicaid/Medicare
patients from California being treated with an
antihypertensive.
Principle Findings: We identified 108,209 patients who met
the HOPE inclusion criteria, of whom 4,789 took ramipril in
1997. LPM on 56,549 White patients suggest a risk ratio (RR)
of 0.842 (p<0.001) for ramipril use, which compared favorably
to RR=0.75 in HOPE. IV LPMs models yielded RR=0.463
(p=0.032). Applying these same models to 51,660 non-White
beneficiaries generated RRs of 0.992 (p=0.869) and 1.535
(p=0.127) for ramipril use from LPMs and IV LPMs,
respectively. In Cox models for patients taking any ACEI in
1997, patients taking benezapril (n=2,839; hazard ratio=1.037,
p=0.624) had similar risks of the composite endpoint
compared to patients taking ramipril, while patients taking
enalapril (n=4408; hr=1.17, p=0.018) or captopril (n=1469;
hr=1.3, p<0.001) faired worse.
Conclusions: IV models applied to observational data
matched the results of the HOPE RCT. The same models
suggested no benefit of ramipril for non-white beneficiaries.
Ramipril and benezepril appeared superior to enalapril and
captopril.
Implications for Policy, Practice or Delivery: These results
can help guide prescribing practices and have implications for
proposed formulary policies that restrict access to benezapril
and ramipril. These data also provide some support to the
hypothesis of racial differences in cardiac drug effectiveness.
Funding Source: NIH National Institutes of Diabetes
Digestive and Kidney Disease (NIDDK)