How Accurate is the
Present on Admission
Indicator?
John S. Hughes, M.D.
Yale School of Medicine
Elizabeth McCullough M.S
Richard Averill M.S.
3M Health Information Systems
John Muldoon M.H.A.
NACHRI
James Gay M.D.
Vanderbilt School of Medicine
Background

The POA indicator has been advocated as a low-cost
way to identify potentially preventable in-hospital
complications

Also used to select conditions used for risk adjustment at
the time of admission

CMS has required coding of the POA indicator for each
secondary diagnosis since October 2007

Payment adjustment for hospital-acquired complications
(HACs) beginning October 2008

The accuracy of hospital complication rates depends on
the accuracy not only of diagnosis coding, but also the
accuracy of POA coding
How Accurate is the POA Indicator?

Purpose – examine the accuracy of POA coding
in California


Data from 387 California hospitals for 2004-2005


California has collected POA data since 1996
Diagnosis & procedure codes, POA, age & gender
Eliminated 42 hospitals (remainder 345):



Fewer than 1,000 admissions per year
Mortality rate > 9%
Average length of stay > 9 days
How Accurate is the POA Indicator?
Methods

We examined:
 Percentage of secondary diagnoses POA for
each hospital
 “False negatives” – diagnoses for elective
surgery patients coded POA but unlikely to
have been POA
 “False positives” – preexisting chronic
conditions that should always be coded POA
but were coded NPOA
“False Negatives”

Two types of false negatives for elective surgery
patients:

Surgical complications (List 1 – 22 codes):
Post-op wound infection
 Post-op hemorrhage
 Post-surgical respiratory failure


Medical complications (List 2 – 30 codes)
Acute MI
 Pneumonia
 Stroke

“False Positives”

Chronic conditions that would have existed
before the hospitalization but were coded
as “not POA”
Hypertension
 COPD
 Diabetes
 Hyperlipidemia
 HIV

Results: Percent secondary diagnoses
coded POA by hospital
Percent
of Secondary
Diagnoses
Coded Coded
POA byPOA
Hospital
Percent
of Secondary
Diagnoses
California
Hospitals
2004-2005
California
2004-2005
50
Range
45
Median
40
42
41
94.1%
37
IQ Range
35
Number of Hospitals
44
80.6 – 100%
36
92 – 96.2%
30
28
25
20
20
17
16
15
10
16
14
12
9
5
6
5
2
0
<86
86
87
88
89
90
91
92
93
94
Percent Coded POA
95
96
97
98
99
100
Results: Percent secondary diagnoses
coded POA by hospital

10% of hospitals had a POA rate > 98%

5% of hospitals had a POA rate > 99.1%

Hospitals with >99% of secondary diagnoses
POA all had at least 60% false negatives
(probable post-op complications coded POA)
Results – False Positives
Percent of Secondary Diagnosis Codes Labeled NPOA That Were
Percent of NPOA Secondary Diagnoses on Preexisting List
Preexisting Conditions – Elective Surgery Patients
140
132
Range
120
0 – 73%
Median:
IQ range:
100
Number of Hospitals
9.6%
80
6.3 – 15.5%
75
60
46
40
31
20
15
12
7
6
4
3
5
1
2
55-59
60-64
0
2
0
0-4
5-9
10-14
15-19
20-24
25-29
30-34
35-39
Percent
40-44
45-49
50-54
65-69
70-74
Results – False Positives


10% of hospitals had false positive rates > 26%
5% of hospitals had > 39% false positives
Results – False Negatives, List 1
Among 322 hospitals with at least 20 secondary diagnoses
from List 1 (surgical complications):

Percentage of List 1 diagnoses labeled POA

Range: 2.5% - 100%

median 16.0%

Inter-quartile range 8.7% – 44.7%

10% of hospitals had > 51% List 1 false negatives

5% of hospitals had > 70% List 1 false negatives
Percent of List 1 Secondary
Diagnoses
Coded POA for Elective
List 1 Pct
POA v SDx POA
Hospitals
w <50of
List
eliminated Diagnoses POA
Surgery Patients vs.
Percent
All1 SDx
Secondary
100.0%
90.0%
Hospitals with <50 List 1 SDx patients eliminated
80.0%
List 1 percent POA
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
80.0%
82.0%
84.0%
86.0%
88.0%
90.0%
92.0%
94.0%
96.0%
Percent SDx POA
Percent of all secondary
diagnoses coded POA
98.0%
100.0%
Percent of List 1 Secondary
Diagnoses
Coded POA for Elective
List 1 Pct
POA v SDx POA
Hospitals
w <50of
List
eliminated Diagnoses POA
Surgery Patients vs.
Percent
All1 SDx
Secondary
100.0%
90.0%
Hospitals with <50 List 1 SDx patients eliminated
80.0%
List 1 percent POA
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
80.0%
82.0%
84.0%
86.0%
88.0%
90.0%
92.0%
94.0%
96.0%
Percent SDx POA
Percent of all secondary
diagnoses coded POA
98.0%
100.0%
Percent of List 1 Secondary
Diagnoses
Coded POA for Elective
List 1 Pct
POA v SDx POA
Hospitals
w <50of
List
eliminated Diagnoses POA
Surgery Patients vs.
Percent
All1 SDx
Secondary
100.0%
90.0%
Hospitals with <50 List 1 SDx patients eliminated
80.0%
List 1 percent POA
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
80.0%
82.0%
84.0%
86.0%
88.0%
90.0%
92.0%
94.0%
96.0%
Percent SDx POA
Percent of all secondary
diagnoses coded POA
98.0%
100.0%
Results – False Negatives, List 2
Among 300 hospitals with at least 40 secondary diagnoses
from List 2 (medical complications):

Percentage of List 2 diagnoses labeled POA 
Range: one hospital at 0%; otherwise 7.1 - 100%

median 35.6%

Inter-quartile range 25.3% – 48.8%

10% of hospitals had > 70% List 2 false negatives

5% of hospitals had > 80% List 2 false negatives
Percent of List 2 Secondary
Diagnoses
POA for Elective Surgery
List 2POA
v Pct SDxCoded
POA
with <50of
List
2 SDx
eliminated (N=105)
PatientsHospitals
vs. Percent
All
Secondary
Diagnoses POA
100.0%
90.0%
Hospitals with <50 List 1 SDx patients eliminated
80.0%
Percent List 2 POA
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
80.0%
82.0%
84.0%
86.0%
88.0%
90.0%
92.0%
94.0%
POA
Percent of all secondarySDx
diagnoses
coded POA
96.0%
98.0%
100.0%
Percent of List 2 Secondary
Diagnoses
POA for Elective Surgery
List 2POA
v Pct SDxCoded
POA
with <50of
List
2 SDx
eliminated (N=105)
PatientsHospitals
vs. Percent
All
Secondary
Diagnoses POA
100.0%
90.0%
Hospitals with <50 List 2 SDx patients eliminated
80.0%
Percent List 2 POA
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
80.0%
82.0%
84.0%
86.0%
88.0%
90.0%
92.0%
94.0%
POA
Percent of all secondarySDx
diagnoses
coded POA
96.0%
98.0%
100.0%
Percent of List 2 Secondary
Diagnoses
POA for Elective Surgery
List 2POA
v Pct SDxCoded
POA
with <50of
List
2 SDx
eliminated (N=105)
PatientsHospitals
vs. Percent
All
Secondary
Diagnoses POA
100.0%
90.0%
Hospitals with <50 List 2 SDx patients eliminated
80.0%
Percent List 2 POA
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
80.0%
82.0%
84.0%
86.0%
88.0%
90.0%
92.0%
94.0%
POA
Percent of all secondarySDx
diagnoses
coded POA
96.0%
98.0%
100.0%
Percent POAListfrom
List
Percent
1 v List
2 - 501orvs
more
List 2 Sdx POA from List 2
295 California hospitals with (295
> 50hospitals)
patients with List 2 secondary diagnoses
100.0%
90.0%
80.0%
Percent List 2 POA
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
Percent List 1 POA
70.0%
80.0%
90.0%
100.0%
How Accurate is the POA Indicator?
Alternative POA Accuracy Screens
If we eliminate hospitals with:
No. removed
1.
> 99% secondary dx POA
- 21 (6.1%)
2.
>98% secondary dx POA
- 38 (11%)
3.
4.
5.
> 99% secondary dx POA, or
List 1 POA > 60%, or List 2 POA > 70%, or
NPOA codes contain > 40% Pre-existing
- 55 (15.9%)
> 98% secondary dx POA, or
List 1 POA > 50%, or List 2 POA > 60%, or
NPOA codes contain > 40% Pre-existing
- 76 (22%)
Above, plus NPOA > 30% pre-existing
- 83 (24%)
How Accurate is the POA Indicator?

Accuracy of POA coding varies widely across
hospitals

Judgments of hospital coding accuracy will
depend on the criteria for adequacy

What levels of false positives and false
negatives are acceptable?

Which hospitals code POA too infrequently,
which too frequently?

POA coding variation could affect
measurements of hospital performance
Conclusions


Hospitals that tend to code post-admission
complications as POA will appear to have lower
complication rates than they deserve, and gain
an unfair advantage in comparisons with other
hospitals.
The methods used to determine the
performance measures for hospitals will need
to be scrutinized as closely as the hospitals
themselves.
Percent NPOA Preexisting v List 1 POA
100.0%
90.0%
80.0%
Pct List 1 Dx POA
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
Pct NPOA Dx Preexisting
70.0%
80.0%
90.0%
100.0%
Percent NPOA Preexisting v List 1 POA
100.0%
90.0%
80.0%
Pct List 1 Dx POA
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
Pct NPOA Dx Preexisting
70.0%
80.0%
90.0%
100.0%
FalsePercent
Positives
– Chronic
Conditions
Coded
Not POA
of Pre-existing
Diagnoses
Coded
as Not as
POA
Distribution of Hospital False Positive Rates
120
100
100
94
Number of Hospitals
80
65
60
40
31
22
20
16
13
15
12
6
5
5
4.5
5.0
3
0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
Percent Not POA
4.0
5.5
>6.0
Pct List 1 Dx (Surgical Compl) Coded POA vs Pct of Secondary Diagnoses POA
(Hospitals with less than 9 List 1 diagnoses excluded)
100.0%
Percent of List 1 Diagnoses Coded POA
90.0%
80.0%
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
80.0%
82.0%
84.0%
86.0%
88.0%
90.0%
92.0%
94.0%
Percent of all Secondary Diagnoses Coded POA
96.0%
98.0%
100.0%
Percent False Positives
(Secondary Diagnoses Coded NPOA that were on the
Percent SDx POA v Pct NPOA SDX on Preexist list
Pre-existing List) vs. Percent of all Secondary Diagnoses Coded POA
80.0%
70.0%
Pct NPOA SDx Preexist
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
80.0%
90.0%
Percent
SDx POA coded POA
Percent of all secondary
diagnoses
California Hospitals, 2004-2005
100.0%
Pct List 1 Dx (Surgical Compl) Coded POA vs Pct of Secondary Diagnoses POA
(Hospitals with less than 9 List 1 diagnoses excluded)
100.0%
Percent of List 1 Diagnoses Coded POA
90.0%
80.0%
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
80.0%
82.0%
84.0%
86.0%
88.0%
90.0%
92.0%
94.0%
Percent of all Secondary Diagnoses Coded POA
96.0%
98.0%
100.0%