Disclosure and Acknowledgments
Studying hospital quality using
mixed methods
The research is funded by the Agency for Health
Care Research and Quality (#R01HS10407-01),
and the Donaghue Medical Research
Foundation (#02-102)
Elizabeth H. Bradley, PhD
Yale University
School of Medicine
Conducted in collaboration with Genentech and
the National Registry of Myocardial Infarction
(NRMI) Investigators
2
Purpose of the presentation
Background
Provide an example of using mixed methods
to identify determinants of hospital quality
Highlight the benefits and special challenges
in employing mixed methods
ACC/AHA guidelines recommend betablocker prescription for patients
hospitalized with acute myocardial
infarction (AMI)
However, many patients do not receive
beta-blockers after AMI, and hospitals vary
substantially in rates of beta blocker use
and improvement in those rates over time
3
Objective
To identify success factors in hospitals’
increased rates of beta-blocker use for
patients with AMI… what works?
0.40
0.35
0.30
0.25
0.20
0.15
0.10
0.05
0.00
-0.05
-0.10
-0.15
-0.20
-0.25
-0.30
-0.35
-0.40
-0.45
100
90
80
70
60
50
40
30
20
10
0
-0.50
Number of Hospitals
Improvement in hospital betablocker rates, 1996-1999
4
Change in beta-blocker use
5
6
1
Mixed methods
Qualitative study
Site visits with in-depth interviews (n=45) in 8
hospitals with higher versus medium/lower
performance in beta-blocker rates
• 14 physicians
• 15 nurses
• 11 quality improvement staff
•
5 administrators (senior and mid-level)
Qualitative study
Site visits in higher/lower performers
In-depth interviews with key staff
Quantitative study
Closed-ended survey of random
sample of hospitals
Constant comparative method for data analysis of
qualitative data
7
A taxonomy to classify QI efforts
along key dimensions
8
Hypotheses about “what works”
Goals – content, specificity, sharedness
Admin support – philosophy, resources
Clinical support – physician, nurse, ancillary
Systems design – standing orders, pathways,
reminders, care coordinators, etc.
Data feedback – validity, timeliness
In beta-blocker performance, presence of clinical
champions and administrative support for quality
improvement are more important than systems
design interventions
Data feedback, especially when it is physicianspecific, can be viewed as punitive and can
backfire as an improvement effort
Contextual factors – size, teaching status, system
affil, financial constraints, market and regulatory
pressures, etc.
9
10
Quantitative study
Hospital sample (n=234)
Cross-sectional study of 234 hospitals from those
participating 30+ months during Apr 96-Sept 99 in the
National Registry of Myocardial Infarction (reflects 54.2%
response rate)
Patients: 60,363 treated for AMI in these hospitals during
1998-1999, the years just after beta-blocker
recommendations were widely published
Telephone survey of QI directors at each hospital
Hierarchical models to estimate p (high-rate hospital)
11
Beta-blocker rates
Mean, range
60%;19% - 89%
Urban
83%
Teaching
39%
Annual AMI volume
(median)
137 patients
12
2
QI efforts
QI efforts (continued)
Type of QI effort
Prevalence
Type of QI effort
Standing orders
Clinical pathways
Educational efforts
QI teams
Care coordinators
Reminder forms
Computer support
57%
58%
76%
80%
50%
28%
34%
Data feedback reports
Quarterly reports
Public display of data
Data reports last 6 months
Data are physician-specific
Has physician champion
Prevalence
97%
81%
34%
39%
11%
92%
13
Qualitative study benefits
QI efforts and performance
QI effort
Adj OR
Standing orders
14
p-value
The qualitative study augmented conceptual
background for quantitative study:
2.3
.066
Physician champion
10.5
.001
Admin support [1-5]
2.0
.009
2. Hypotheses about systems interventions,
clinical champions, administrative support, and
data feedback
Data feedback that
is physician-specific
0.1
.001
3. Comprehensible language for survey design
1. Taxonomy with which to characterize QI
efforts, a multifaceted intervention
15
Special challenges of
using mixed methods
16
Why use mixed methods?
Integration the qualitative and quantitative work – benefit
comes from their integration but easy to split them off
The “juicy” ideas in qualitative work can be difficult to
measure (organizational change, culture, etc.) and test in
larger samples
“Grounds” our work, so that we ask the
important questions, have realistic
hypotheses, and use sensible language
Increases the potential that research will be
more easily translated into practice
Qualitative work often slows down and delays the
quantitative work
Publishing mixed methods – a special challenge (length,
reviewers’ tolerance for unknown methods)
17
18
3
Conclusions
Mixed methods studies are particularly
advantageous for some, but certainly not
all, topic areas
Research inquiries that involve multifaceted
interventions, interdisciplinary interactions,
or innovation and organizational change
are good candidates for mixed methods
19
4