Background
Organizational characteristics
and quality of care:
Inside the black box
Elizabeth H. Bradley, PhD
Yale School of Public Health
Health Policy and Administration
Previous work has shown:
Variation by region
Census regions
(Krumholz , AHJ, 2003; Normand JAMA,
1996; Subramanian, J Gen Intern Med, 2002)
Hospital referral regions (small areas)
Substantial variation across hospitals in quality of care
- Processes
- Risk-adjusted outcomes
Application of AMI care
- Beta-blocker rates
- Time to reperfusion therapy
- Risk-adjusted mortality
What contributes to this variation?
However…
These traditional hospital attributes explain
only 6%-12% of the hospital-level variation
in processes and outcomes (Normand, JAMA, 1999;
Schultz, Applied Nurs Rsch, 1999)
(Fisher, Ann Intern Med, 2003; Fisher, HSR 2000)
Variation by types of hospitals
Teaching (Allison, JAMA 2003; Rosenthal, JAMA 1997)
Volume (Theimann, JAMA, 1999; Magid, JAMA 2000)
What distinguishes the top
performing hospitals?
Market factors (competition, managed care
penetration, demand for services)
We have examined top performing
hospitals in time to reperfusion therapy
Few hospitals are meeting guidelines, and
hospitals’ average performance has not
changed dramatically over time.
Regulatory factors (Certificate of Need,
public performance reporting)
Organizational factors (Inside the black box)
1
180
However, some hospitals have
improved
How do top performers do it?
80
100
Door-to-Balloon (mins)
120
140
160
Mixed methods studies
- Qualitative in-depth interviews
- Quantitative hospital survey of efforts
- Chart review data on clinical practices
Q1-1999
Q3-1999
Q1-2000
Q3-2000
Q1-2001
Calendar Quarter
Q3-2001
Q1-2002
Approaches to measurement
Start with open-ended, qualitative
assessment to understand scope and
create a taxonomy, or language for
possible predictor variables
Move to closed-ended, quantitative
measures in order to increase sample
sizes, enhance ability to make statistical
inferences, and improve generalizability
Success Factors
Systems design (structures & processes)
--------------------------Organizational goal
Administrative support
Clinical leadership
Teams
Organizational culture
Sources: Bradley et al., JAMA 2001; Bradley et al., Med Care 2005
Measuring systems design
Measuring structures & processes
Systems design has manifest attributes
- Structures
- Processes
Start with set of key components
hypothesized from qualitative work
The structures are easiest and most objective
internal features to measure
Survey (self-report or external observations)
for quantitative assessment of key
components identified in qualitative work
The processes are more complex because they
are fluid, vary by the people involved, and
include multiple components
2
Key components of systems design
Moving from qualitative to
quantitative measurement
Original data collection (self-reported or
observations by others)
Challenges
- Instrument design, length, and validation
- Pinpointing the time frame
- Measurement can itself change processes
- Issue of multiple respondents
Key components of
organizational environment
Measuring organizational environment
The organizational environment is most
difficult because it is more hidden (i.e.,
culture), and it is a composite of diverse
experiences and perceptions
Moving to quantitative assessment
Measures can be noisy, making it harder to see
their statistical effects (e.g., org culture)
Multiple measures are good (but expensive)
Multiple respondents per organization are good
(but also expensive)
Some black box features defy quantitative
measurement; good to retain mixed methods
•
•
•
•
•
Clear, explicit goals in clinical quality
Administrative support
Clinician “champions”
Collaborative interdepartmental teams
Organization culture
Take Home Messages
The black box is complex; we like to keep it closed
But recent efforts suggest that org context matters
Research can give insight into the org structures,
processes, and environments that foster higher
quality, but we must use a mix of methods and
disciplinary approaches to producing evidence
3