Organizational characteristics
and quality of care:
Inside the black box
Elizabeth H. Bradley, PhD
Yale School of Public Health
Health Policy and Administration
Background
Substantial variation across hospitals in quality of care
- Processes
- Risk-adjusted outcomes
Application of AMI care
- Beta-blocker rates
- Time to reperfusion therapy
- Risk-adjusted mortality
What contributes to this variation?
Previous work has shown:
Variation by region
Census regions (Krumholz , AHJ, 2003; Normand JAMA,
1996; Subramanian, J Gen Intern Med, 2002)
Hospital referral regions (small areas)
(Fisher, Ann Intern Med, 2003; Fisher, HSR 2000)
Variation by types of hospitals
Teaching (Allison, JAMA 2003; Rosenthal, JAMA 1997)
Volume (Theimann, JAMA, 1999; Magid, JAMA 2000)
However…
These traditional hospital attributes explain
only 6%-12% of the hospital-level variation
in processes and outcomes (Normand, JAMA, 1999;
Schultz, Applied Nurs Rsch, 1999)
What distinguishes the top
performing hospitals?
Market factors (competition, managed care
penetration, demand for services)
Regulatory factors (Certificate of Need,
public performance reporting)
Organizational factors (Inside the black box)
We have examined top performing
hospitals in time to reperfusion therapy
Few hospitals are meeting guidelines, and
hospitals’ average performance has not
changed dramatically over time.
80
100
120
140
160
180
However, some hospitals have
improved
Q1-1999
Q3-1999
Q1-2000
Q3-2000
Q1-2001
Calendar Quarter
Q3-2001
Q1-2002
How do top performers do it?
Mixed methods studies
- Qualitative in-depth interviews
- Quantitative hospital survey of efforts
- Chart review data on clinical practices
Approaches to measurement
Start with open-ended, qualitative
assessment to understand scope and
create a taxonomy, or language for
possible predictor variables
Move to closed-ended, quantitative
measures in order to increase sample
sizes, enhance ability to make statistical
inferences, and improve generalizability
Success Factors
Systems design (structures & processes)
--------------------------Organizational goal
Administrative support
Clinical leadership
Teams
Organizational culture
Sources: Bradley et al., JAMA 2001; Bradley et al., Med Care 2005
Measuring systems design
Systems design has manifest attributes
- Structures
- Processes
The structures are easiest and most objective
internal features to measure
The processes are more complex because they
are fluid, vary by the people involved, and
include multiple components
Measuring structures & processes
Start with set of key components
hypothesized from qualitative work
Survey (self-report or external observations)
for quantitative assessment of key
components identified in qualitative work
Key components of systems design
Path #1
phECG
completed and
ED notified
Activate cath
lab, page
interventionalist
Pt arrives in
ED triage
phECG
STEMI?
yes
ED stabilizes pt
and initiates tx
ED transports
to cath lab
Final checklist
PCI
indicated?
no
Do ECG
no
Pt arrives
w/o phECG
STEMI?
yes
Activate cath
lab, page
interventionalist
no
Path #2
Continue ED
work-up
Admit to CCU
yes
Start procedure
Moving from qualitative to
quantitative measurement
Original data collection (self-reported or
observations by others)
Challenges
- Instrument design, length, and validation
- Pinpointing the time frame
- Measurement can itself change processes
- Issue of multiple respondents
Measuring organizational environment
The organizational environment is most
difficult because it is more hidden (i.e.,
culture), and it is a composite of diverse
experiences and perceptions
Key components of
organizational environment
•
•
•
•
•
Clear, explicit goals in clinical quality
Administrative support
Clinician “champions”
Collaborative interdepartmental teams
Organization culture
Moving to quantitative assessment
Measures can be noisy, making it harder to see
their statistical effects (e.g., org culture)
Multiple measures are good (but expensive)
Multiple respondents per organization are good
(but also expensive)
Some black box features defy quantitative
measurement; good to retain mixed methods
Take Home Messages
The black box is complex; we like to keep it closed
But recent efforts suggest that org context matters
Research can give insight into the org structures,
processes, and environments that foster higher
quality, but we must use a mix of methods and
disciplinary approaches to producing evidence