A Randomized Controlled Trial of an Educational
and Motivational Intervention to Enhance
Consumers’ Use of Health Plan and Medical
Group Quality Data
Patrick S. Romano, MD MPH
Julie A. Rainwater, PhD
Jorge A. Garcia, MD MS
Debora A. Paterniti, PhD
Daniel J. Tancredi, MS PhD
Geeta Mahendra, MS
Jason A. Talavera, MD student
AcademyHealth 2006 ARM
June 27, 2006
Seattle, WA
Background to INQUIRE
(INformation about QUality In a Randomized Evaluation)
 Most
previous studies of how consumers use report cards were
conducted in “laboratory” settings, relied entirely on survey data, or
did not randomly allocate participants.
 We planned a prospective study with 3 components:
Focus group discussions of consumer choice and quality of care
A prospective cohort study of factors associated with using a quality
report card and switching health plans/medical groups
A randomized controlled trial of two interventions designed to improve
the use of quality information, under the Health Belief Model.
 Funding
from the US Agency for Healthcare Research and Quality
Health Belief Model
INDIVIDUAL PERCEPTIONS
Readiness To Undertake
Recommended Behaviors
MODIFYING FACTORS
LIKELIHOOD OF ACTION
Demographic Variables:
age, gender, race/ethnicity, SES
Social Psychological Variables:
social networks, group pressure,
acculturation
Perceived Benefits
minus
Perceived Barriers
Perceived Susceptibility to Illness
Perceived Seriousness (Severity)
of Illness
Perceived Threat
of Illness
CUES TO ACTION
Advice from friends, Prompt from MD,
Illness of family member,
Newspaper or magazine article
The Health Belief Model (Becker and Maiman 1975)
Likelihood of Preventive
Behavior
Phase II – Overview of Methods
 Partnership between UC
Davis and Pacific Health Advantage
 Population: 76,000 employees of small businesses (with 2-100
eligible workers) in CA, excluding “guaranteed associations”
 Setting: Open Enrollment 2003; members were offered a choice
of 4 statewide and 4 regional HMOs (each with 3 copayment
levels), 1 PPO (with 3 copayment levels), 1 point-of-service plan
 Study design: Randomized controlled trial
 Unit of randomization: Health insurance brokers (with their
contracted employers and their employees)
 Measures: Observed behavior, post-Open Enrollment survey
Phase II – Control group
 Control
group received “usual care”:
Open Enrollment booklets on program rules and benefit
options were mailed to employers (no quality information)
No information went directly to employees
PacPlan Chooser web site allowed members to compare
plans on cost, features, and quality (overall rating)
Insurance brokers provided limited support
Phase II – Interventions
 Intervention
group 1 received “educational/motivational
treatment”:
A special mailing to each employee, employer, and broker
included a motivational letter (with negative framing), the
California HMO Report Card, and the California HMO Guide
A toll-free telephone line and e-mail address were offered for
counseling and advice (during business hours)
 Intervention
group 2 was delayed
Phase II – Sample design
 Stratified
random sample of brokers with eligible employees
scheduled for Open Enrollment in May-July 2003, after excluding
employers intending to leave (N=1,579 with 26,249 EE’s)
 Excluded 16 brokers with large number of eligible employees (to
increase efficiency)
 Oversampled small brokers (4 strata), brokers for whom at least
40% of EE’s were <39 yrs, and brokers for whom at least 50% of
EE’s had 3 or more HMO options (total 10 sampling strata)
 Brokers allocated in two stages
Phase II – Analytic methods and hypotheses
 All
analyses were (or will be) weighted to account for the cluster
sampling design, using robust methods to correct CIs
 Hypotheses:
Intervention would increase overall switching across health plans and
medical groups
Intervention would promote switching toward “better” health plans and
medical groups, among those who switch
Intervention would enhance perceived threat, enhance self-efficacy,
promote migration from pre-contemplation to contemplation, and
promote use of quality information in decision-making
Phase II – Process results
 292
brokers with 1,835 eligible employees (EE’s) were
randomized to the intervention group
 246 brokers with 1,578 eligible employees (EE’s) were
randomized to the control group
 30.2% of EE’s in the intervention group, and 37.1% of EE’s in
the control group, dropped out of Pacific Health Advantage
 22 intervention group members used the toll-free advice line
 3 intervention group members used the e-mail address
 Broad array of questions and concerns
Phase II – Primary outcome results
(all weighted and nonsignificant)
 9.2%
of intervention group versus 7.0% of control group
switched plans.
 21%
of intervention group switchers versus 35% of
control group switchers moved to a plan with more stars.
 27-28%
in both groups moved to a plan with fewer stars.
Secondary outcome, use of resources
Did you read or review…?
Did you call or contact…?
Ed/Mot
Control
Comparison of health plan benefits
60%
57%
PacPlan Chooser web site
15%
14%
CA HMO Guide (p<0.001)
40%
10%
CA HMO Report Card (p<0.001)
38%
8%
Health plan member services (p=0.02)
7%
9%
OPA, HMO Help Center, Health Rights Hotline
<2%
<2%
Secondary outcome, reason for switch
Stated reason for switching in intervention group vs. control
group, respectively (all p>0.10 unless stated):
Change in geographic coverage of plan (10% vs. 2%, p=0.03)
 Cost (34% vs. 25%)
 Continuity of MD (1% vs. 5%)
 Better network of MDs (8% vs. 7%)
 Concern over poor report card scores (6% vs. 1%, p=0.099)
 Concern over poor access to care (6% vs. 5%)
 Poor service from previous plan (6% vs. 3%)
 Better benefits (5% vs. 3%)
 Other reason (7% vs. 1%, p=0.06)

Secondary outcome,
expected outcome of switch (NS)
Ed/Mot
(N=88)
Control
(N=87)
Better
16%
9%
Same
17%
20%
Worse (p=0.07)
8%
1%
Uncertain or did not respond
59%
70%
Do you expect that quality of care
will be better, the same, or worse
with your new health plan?
Secondary outcome, considered switch
 35%
of intervention group respondents who did not actually
switch “considered” switching
 28% of control group respondents who did not actually
switch “considered” switching (p=0.07)
 Of those who “considered” switching, 31% of intervention
group respondents and 30% of control group respondents
“seriously considered” it (rating=6 on 1-6 scale)
Secondary outcome, reason for considering switch
Stated reason for considering switching in intervention group
vs. control group, respectively:
Cost (69% vs. 74%)
 Continuity of MD (6% vs. 11%)
 Better network of MDs (16% vs. 23%)
 Concern over poor report card scores (15% vs. 7%, p=0.08)
 Concern over poor access to care (17% vs. 17%)
 Poor service from previous plan (10% vs. 10%)
 Better benefits (25% vs. 25%)
 Other reason (14% vs. 7%) – need to review comment fields

Secondary outcome, perceived differences
in quality among plans (NS)
Size of perceived difference
Ed/Mot
Control
Big
47%
48%
Small
31%
30%
None
7%
4%
Don’t know
15%
18%
Secondary outcome, perceived differences
in quality among medical groups (NS)
Size of perceived difference
Ed/Mot
Control
Big
31%
35%
Small
35%
34%
None
7%
5%
Don’t know
27%
27%
Secondary outcome, self-efficacy (NS)
Agree or strongly agree…
Ed/Mot
Control
Confident in my ability to choose a health plan
78%
81%
Confident in my ability to choose a medical group
80%
80%
I felt well informed about my health plan choices
71%
69%
I felt well informed about my medical group choices
66%
65%
I used what I know…to make the best possible
choice for me during Open Enrollment
77%
78%
Secondary outcome,
perceived benefits and barriers (NS)
Agree or strongly agree…
Using the information in…, I was able to choose the
best health plan for my family and me
Looking at the information about health plans was a
waste of time for me (p=0.05)
The materials… helped me better understand my
health plan choices
I guess my health plan has some drawbacks, but
none of the others is really better
Ed/Mot
Control
58%
57%
16%
20%
67%
65%
52%
54%
Secondary outcome,
Difficulty of selecting plan (p=0.003)
How much of a problem, if any, was it
to find a health plan that suited you…?
Ed/Mot
Control
Not a problem (p=0.01)
58%
66%
A small problem
27%
25%
A big problem (p=0.002)
15%
9%
Limitations
 Primary
outcome (actual choice of health plan) may be
difficult to change because of competing concerns (e.g.,
price, convenience) and information from other sources
(e.g., friends and family)
 Analysis of secondary outcomes limited by poor
response to post-OE survey despite two mailings,
financial incentive, and follow-up abbreviated webbased survey (est. 41% excluding ineligibles)
Policy implications
 Educational/motivational interventions
designed to increase
perceived benefits and decrease perceived barriers, with
negative framing, may increase use of quality information but
are unlikely to affect actual choices in the health care market.
 Quality data with negative framing may make decision-making
more difficult for price-sensitive consumers (especially if there is
a perceived cost-quality tradeoff).
 Many other signals affect consumers’ behavior during Open
Enrollment; cost is the dominant factor in the small business
market in the USA.