Comparative Rankings of Hospital Quality –
Does the Data Source Matter?
Anne Elixhauser, Ph.D.
Bernard Friedman, Ph.D.
June 26, 2005
AcademyHealth Research Meeting
Background
 Hospitals are being compared based on
readily available data
 Data on Medicare patients are available
from virtually all U.S. hospitals
 Convenient – but do we know how the
Medicare experience reflects hospital
quality overall?
Purpose of Study
 Examine the extent to which information
on Medicare patients can be
extrapolated to the general population
when comparing hospital quality
Methods – Data Source
 2001 Healthcare Cost and Utilization
Project (HCUP) Nationwide Inpatient
Sample (NIS)
 Sample of 986 hospitals from 33 states
– All discharges from each hospital are included
 No weighting for this study – Used NIS as a
convenience sample of hospitals
 Study population: hospital inpatients from
short-term, non-Federal, acute care
hospitals
Methods – Quality Measures
 AHRQ Quality Indicators
– Based on hospital administrative data
– 15 Patient Safety Indicators (PSIs)
 Risk adjusted using gender, age,
comorbidities, and collapsed DRGs
– 12 in-hospital mortality measures from the
Inpatient Quality Indicators (IQIs)
 Risk adjusted using APR-DRGs
Methods – Rankings
For each measure:
– Dropped the 10% of hospitals with the
fewest Medicare cases
– Using Medicare discharges only
 Ranked hospitals and ordered into deciles
– Using all patients (including Medicare)
 Ranked hospitals and ordered into deciles
Comparison of Rankings
 How many hospitals changed from the
highest or lowest rank by at least two
deciles?
 Compared rank using Medicare-only data to
rank using all-patient data
 What % of poorest performing hospitals
increased their ranking?
 What % of best-performing hospitals fell in
their ranking?
Percent of hospitals in lowest decile
that increased rank by at least 2 deciles
Patient Safety Indicators
50
45
40
35
30
25
20
15
10
5
0
s
l
r
sth D RG ulce scue ody orax are ip fxha ge nge p fai DVT epsi ence tion
e
a n ort us re gn b ot h ed c op h orr dera res PE p s is c c era
x
C
m ubit e t o rei um o m ost em op top top osto deh l la
r Fo n e
w
P p h ost os Pos P top nta
et
-lo De c ailu
p
u
P
e
h
s
t
d
F
o x
sto P
a
r
o
t
Po c cid
e
f
P
Ia In
D
A
Percent of hospitals in highest decile
that fell in rank by at least 2 deciles
Patient Safety Indicators
50
45
40
35
30
25
20
15
10
5
0
s
l
r
sth D RG ulce scue ody orax are ip fxha ge nge p fai DVT epsi ence tion
e
a n ort us re gn b ot h ed c op h orr dera res PE p s is c c era
x
C
m ubit e t o rei um o m ost em op top top osto deh l la
r Fo n e
w
P p h ost os Pos P top nta
et
-lo De c ailu
p
u
P
e
h
s
t
d
F
o x
sto P
a
r
o
t
Po c cid
e
f
P
Ia In
D
A
Percent of hospitals in lowest decile
that increased rank by at least 2 deciles
In-hospital Mortality Indicators
50
45
40
35
30
25
20
15
10
5
0
y
nt
air
BG
ep CA iot om eme
r
A
ac
an
AA
Cr repl
Hip
I
AM
e
F
A
e
a
e
CH trok hag ct ur oni PTC
S
orr ip fra neum
m
P
H
he
GI
A
CE
Percent of hospitals in highest decile
that fell in rank by at least 2 deciles
In-hospital Mortality Indicators
50
45
40
35
30
25
20
15
10
5
0
y
nt
air
BG
ep CA iot om eme
r
A
ac
an
AA
Cr repl
Hip
I
AM
e
F
A
e
a
e
CH trok hag ct ur oni PTC
S
orr ip fra neum
m
P
H
he
GI
A
CE
Summary of Findings:
Patient Safety Indicators
Among the top 10% of hospitals (best, or lowest PSI rates):
At least 40% of hospitals fell
to 3rd decile or lower for:
1/3 of hospitals fell to 3rd
decile or lower for:
1/4 of hospitals fell to 3rd
decile or lower for:
Postop hip fracture
Postop physiologic and metabolic derangement
Postop respiratory failure
Wound dehiscence
Anesthesia complications
Death in low mortality DRGs
Foreign body after procedure
Iatrogenic pneumothorax
Infection due to medical care
Postop hemorrhage
Postop PE and DVT
Postop sepsis
Summary of Findings:
In-hospital Mortality Indicators
Among top 10% of hospitals (best, or lowest mortality rates):
38% of hospitals fell to 3rd
decile or lower for:
12-20% of hospitals fell to 3rd
decile or lower for:
Craniotomy
GI hemorrhage
Carotid endarterectomy
Abdominal aortic aneurysm repair
Among bottom 10% of hospitals (worst, or highest mortality rates):
24% of hospitals rose to 8th
decile or higher
21% of hospitals rose to 8th
decile or higher
14% of hospitals rose to 8th
decile or higher
10% of hospitals rose to 8th
decile or higher
CABG
Craniotomy
PTCA
Abdominal aortic aneurysm repair
Limitations
 Findings may be unique to these specific
indicators
 Findings may not hold for cutpoints other
than deciles
 Convenience sample of hospitals – not
nationally representative
– But hospitals are drawn from a sampling
frame that comprises 80% of U.S.
discharges
Conclusions
 When comparing Medicare and all-payer
analyses
– Found loose overlap of the top-ranking and
poorest-ranking hospitals
– Saw largest shifts for PSIs among the topranking hospitals
 Using Medicare-only data may carry a
greater risk of incorrectly labeling a
hospital as a top-ranking performer
 Pre-test ranking approaches