Supplemental Digital Content
1. Creation of crosswalk for postoperative complications
2. Model diagnostic and fit statistics
3. Sensitivity analyses
1
Supplemental Digital Content 1:
Creation of crosswalk for postoperative complications
In order to compare the data sources used in this study, we created a crosswalk that
matches variables in ACS-NSQIP with applicable ICD-9 diagnosis codes in the Medicare data.
This crosswalk was created through careful review and classification of applicable ICD-9 codes
after consulting published literature as well as relevant measures from the Elixhauser
Comorbidity Software,1 Quality Indicators from the Agency for Healthcare Research and
Quality (AHRQ) 2 and the list of Hospital-Acquired Conditions from CMS. 3 We created and
tested multiple variations of some Medicare variables to determine the optimal definition. When
possible, we excluded codes that could represent a preoperative comorbidity rather than a
postoperative complication, as “condition present-on-admission” qualifiers are not reported for
MedPAR diagnosis codes. For example, we only included codes for acute myocardial infarction
(ICD-9 code prefix 410) and excluded codes for old myocardial infarction (ICD-9 code prefix
412). In addition, the acute myocardial infarction ICD-9 code prefix requires a 4th digit indicating
the location of the infarction and a 5th digit indicating whether it is the first episode of care for a
newly diagnosed myocardial infarction (5th digit=1), a subsequent follow-up episode of care (5th
digit=2) or unspecified (5th digit=0). Codes with a 5th digit of 2 were excluded.
2
Table 1. ICD-9 diagnosis codes used to identify 30-day postoperative complications in
Medicare inpatient claims data.
ACS-NSQIP Defined
Postoperative
ICD-9-CM Diagnosis Codes
Complications
Superficial SSI
9985, 99851, 99859
Deep/organ-space SSI
99859
Any SSI
9985, 99851, 99859
Urinary tract infection
1122, 5901*, 5903, 5908*, 5950, 5953, 5990,
99664
Pneumonia
0391, 1124, 1179, 1363, 46619, 480*, 481,
482*, 483*, 4841, 4846, 4847, 485, 486, 4870,
507*, 5130, 5168, 99731, 99739
Sepsis
038*, 78552, 99591, 99592, 9980
Deep venous thrombosis
4511*, 4512, 45181, 4534*, 4538, 4539
Pulmonary embolism
4151*
Venous thromboembolism
4151*, 4511*, 4512, 45181, 4534*, 4538, 4539
Myocardial infarction
410*0, 410*1
We searched for codes included in the index admission or subsequent admissions within
30-days of surgery. For brevity, the * represents all 4th or 5th digits that could designate
an ICD-9-CM code. For example, 4151* = 41511, 41512 and 41519. Any SSI includes
superficial and/or deep and/or organ-space SSI. Venous thromboembolism includes
deep venous thrombosis and/or pulmonary embolism. ACS-NSQIP, American College of
Surgeons National Surgical Quality Improvement Program. SSI, surgical site infection.
3
Supplemental Digital Content 2:
Model diagnostic and fit statistics
The c-statistics and Hosmer-Lemeshow chi-square statistics for the hierarchical
models were estimated by creating standard logistic regression models with the same
covariates and hospital entered as a fixed effect (i.e. dummy variable for each hospital).
Table 2. Model diagnostic and fit statistics
Hosmer-Lemeshow
chi-square statistic
Outcome
C-statistic
AIC
(p-value)
ACS-
ACSMedPAR
NSQIP
ACSMedPAR
NSQIP
1.37
3.32
(0.99)
(0.91)
Deep/Organ-
10.80
3.65
space SSI
(0.21)
(0.89)
6.53
3.32
(0.59)
(0.91)
Urinary tract
3.84
39.18
infection
(0.87)
(<0.001)
28.57
49.35
(0.00)
(<0.001)
30.96
10.07
(<0.001)
(0.26)
Deep venous
5.66
10.52
thrombosis
(0.69)
(0.23)
MedPAR
NSQIP
0.775
0.782
34719
32605
0.789
0.782
26253
31976
0.768
0.782
49524
32605
0.739
0.754
31033
52709
0.818
0.790
31825
41246
0.814
0.803
46606
55397
0.778
0.789
16592
19505
Superficial SSI
Any SSI
Pneumonia
Sepsis
4
Pulmonary
10.58
70.08
embolism
(0.23)
(<0.001)
Venous
3.88
3.58
thromboembolism
(0.87)
(0.89)
Myocardial
7.64
23.11
infarction
(0.47)
(0.00)
28.50
86.34
(<0.001)
(<0.001)
Mortality
0.799
0.921
7838
5276
0.762
0.811
20632
21904
0.819
0.786
6867
23911
0.884
0.848
30994
36179
ACS-NSQIP, American College of Surgeons National Surgical Quality Improvement
Program. MedPAR, Medicare Provider Analysis and Review file. SSI, surgical site
infection.
5
Supplemental Digital Content 3:
Sensitivity analyses
To test the sensitivity of our results, we repeated the analyses with (1) the
inclusion of additional Medicare sources representing outpatient visits; and (2) risk
adjustment using only age and sex for all clinical and claims models. The results of the
sensitivity analyses are reported in the tables below.
6
Table 3. Comparison of unadjusted hospital 30-day postoperative complication rates
between a clinical surgical registry (ACS-NSQIP) and administrative data from Medicare
comprehensive claims (MedPAR and outpatient data sources) for 192 hospitals
Median hospital
Median hospital
Correlation of
Agreement on
percentage of
percentage of
hospitals’
hospitals’ decile rank
patients with
patients with
postoperative
for postoperative
complication using
complication using
complication rates
complication rate
ACS-NSQIP data
Medicare data
between ACS-
between ACS-NSQIP
(interquartile
(interquartile
NSQIP vs.
vs. Medicare
range)
range)
Medicare
(weighted kappa)
3.91%
5.66%
Moderate
Poor
(2.25%-5.27%)
(4.37%-7.16%)
(0.41)
(0.27)
Deep/organ-space
2.78%
5.46%
Moderate
Poor
SSI
(1.69%-3.88%)
(4.25%-7.06%)
(0.45)
(0.29)
6.54%
5.66%
Moderate
Poor
(4.86%-8.52%)
(4.37%-7.16%)
(0.50)
(0.34)
3.49%
11.00%
Poor
Poor
(2.19%-4.56%)
(9.14%-13.08%)
(0.32)
(0.23)
3.56%
8.97%
Poor
Poor
(2.39%-5.30%)
(7.01%-11.30%)
(0.39)
(0.25)
6.22%
12.40%
Moderate
Poor
(4.38%-9.05%)
(9.74%-15.09%)
(0.53)
(0.35)
Deep venous
1.39%
4.52%
Moderate
Moderate
thrombosis
(0.82%-1.94%)
(3.17%-6.24%)
(0.63)
(0.43)
0.54%
1.50%
Moderate
Poor
(0.31%-0.87%)
(1.09%-2.15%)
(0.51)
(0.35)
Venous
1.79%
5.61%
Moderate
Moderate
thromboembolism
(1.29%-2.57%)
(4.18%-7.38%)
(0.64)
(0.44)
Outcomes
Superficial SSI
Any SSI
Urinary tract infection
Pneumonia
Sepsis
Pulmonary embolism
7
0.46%
3.33%
Poor
Poor
(0.20%-0.73%)
(2.43%-4.45%)
(0.30)
(0.19)
Myocardial infarction
Compare to Table 2 in main text. Any SSI includes superficial and/or deep and/or organspace SSI. Venous thromboembolism includes deep venous thrombosis and/or
pulmonary embolism. ACS-NSQIP, American College of Surgeons National Surgical
Quality Improvement Program. MedPAR, Medicare Provider Analysis and Review file.
SSI, surgical site infection.
8
Table 4. Comparison of hospital risk-adjusted statistical outlier classification for 30-day
postoperative complications between a clinical surgical registry (ACS-NSQIP) and
administrative data from Medicare comprehensive claims (MedPAR and outpatient data
sources) for 192 hospitals
Number of
Number of
Agreement on
Number
statistical high
Number of
hospitals
hospitals
statistical low
of
outlier hospitals
outlier status
Number of
outlier hospitals
hospitals
(worse than
classification
hospitals
labeled as a labeled as an
classified
“high outlier”
“outlier” by
as low
by ACS-
ACS-NSQIP
(better than
classified
expected
Outcome
Number of
between
expected
as high
performance)
ACS-NSQIP
performance)
identified by each
and Medicare
outliers by NSQIP and a model and “as
outliers
identified by each
by both
data source
(weighted
both data
“low outlier”
expected” by
data source
data
ACS-
kappa)
sources
by Medicare, Medicare, or
ACSMedicare
sources
NSQIP
Medicare
or vice versa
vice versa
1
67
NSQIP
Poor
Superficial SSI
30
25
6
20
10
2
(0.08)
Deep/organ-space
Poor
20
24
2
13
7
3
1
52
SSI
(0.07)
Poor
Any SSI
30
25
8
18
10
4
1
57
(0.20)
Urinary tract
Poor
25
24
5
14
20
4
4
57
infection
(0.12)
Poor
Pneumonia
33
36
8
17
29
6
5
77
(0.11)
Poor
Sepsis
48
23
9
29
20
6
3
84
(0.12)
Deep venous
Poor
20
39
14
5
19
thrombosis
2
0
51
(0.31)
9
Pulmonary
Poor
4
8
2
0
1
0
0
9
embolism
(0.30)
Venous
Poor
20
37
12
5
18
3
0
50
thromboembolism
(0.30)
Poor
Myocardial infarction
14
29
6
2
9
1
0
40
(0.19)
Compare to Table 3 in main text. Outlier status was determined using the hospital
intercept odds ratio from the applicable hierarchical multivariable logistic regression, as
described in the text. A hospital was considered to be a high outlier (worse than
expected performance) if its odds ratio was >1 (p<0.05) and a low outlier (better than
expected performance) if its odds ratio was <1 (p<0.05). Hospitals with odds ratios not
significantly different than 1 (p0.05) were labeled “as expected” for their given patient
population. ACS-NSQIP, American College of Surgeons National Surgical Quality
Improvement Program. MedPAR, Medicare Provider Analysis and Review file. SSI,
surgical site infection.
10
Table 5. Comparison of hospital risk-adjusted statistical outlier classification for 30-day
postoperative complications between a clinical surgical registry (ACS-NSQIP) and
administrative data from Medicare inpatient claims (MedPAR) for 192 hospitals. All
clinical and claims models include only age and sex for risk adjustment.
Number of
Number of
Agreement on
Number
statistical high
Number of
hospitals
hospitals
statistical low
of
outlier hospitals
outlier status
Number of
outlier hospitals
hospitals
(worse than
classification
hospitals
labeled as a labeled as an
classified
“high outlier”
“outlier” by
as low
by ACS-
ACS-NSQIP
(better than
classified
expected
Outcome
Number of
between
expected
as high
performance)
ACS-NSQIP
performance)
identified by each
and MedPAR
outliers by NSQIP and a model and “as
outliers
identified by each
by both
data source
(weighted
both data
“low outlier”
expected” by
sources
by MedPAR,
MedPAR, or
or vice versa
vice versa
3
69
data source
data
ACS-
kappa)
ACSMedPAR
sources
NSQIP
MedPAR
NSQIP
Poor
Superficial SSI
41
32
11
26
10
6
(0.19)
Deep/organ-space
Poor
33
32
11
15
9
5
2
53
SSI
(0.26)
Poor
Any SSI
48
32
16
27
10
6
1
71
(0.26)
Urinary tract
Poor
32
27
7
22
22
6
2
73
infection
(0.13)
Poor
Pneumonia
42
26
14
27
23
8
4
66
(0.26)
Poor
Sepsis
57
34
17
37
22
10
1
94
(0.21)
Deep venous
Poor
24
21
12
7
6
thrombosis
0
0
34
(0.35)
11
Pulmonary
Poor
7
2
1
0
0
0
0
7
embolism
(0.35)
Venous
Moderate
24
21
13
12
8
3
0
33
thromboembolism
(0.44)
Myocardial
Poor
17
24
5
1
11
1
0
41
infarction
(0.16)
Excellent
Mortality
22
28
21
16
16
12
0
16
0.78
Compare to Table 3 in main text. Outlier status was determined using the hospital
intercept odds ratio from the applicable hierarchical multivariable logistic regression, as
described in the text. A hospital was considered to be a high outlier (worse than
expected performance) if its odds ratio was >1 (p<0.05) and a low outlier (better than
expected performance) if its odds ratio was <1 (p<0.05). Hospitals with odds ratios not
significantly different than 1 (p0.05) were labeled “as expected” for their given patient
population. ACS-NSQIP, American College of Surgeons National Surgical Quality
Improvement Program. MedPAR, Medicare Provider Analysis and Review file. SSI,
surgical site infection.
12
REFERENCES
1.
Elixhauser Comorbidity Software. (Accessed November 18, 2011, at
http://www.hcup-us.ahrq.gov/toolssoftware/comorbidity/comorbidity.jsp.)
2.
Agency for Healthcare Research and Quality (AHRQ) Quality Indicators.
(Accessed November 18, 2011, at
http://www.qualityindicators.ahrq.gov/Modules/iqi_overview.aspx.)
3.
Hospital-Acquired Conditions (HAC) in Acute Inpatient Prospective Payment
System (IPPS) Hospitals. Centers for Medicare & Medicaid Services, 2010. (Accessed
November 18, 2011, at
https://www.cms.gov/hospitalacqcond/downloads/hacfactsheet.pdf.)
13