An Imputational Model of Medicaid
Underreporting in the National
Health Interview Survey
Jacob Klerman – Abt Associates
Michael Plotzke – Abt Associates
Mike Davern - NORC
June 27, 2010
Acknowledgements
 Funding
— The Robert Wood Johnson Foundation
— Department of Health and Human Services – The Assistant Secretary for
Planning and Evaluation
— Abt Associates Internal Funds
 Data
— National Center for Health Statistics (NHIS)
— Centers for Medicare & Medicaid Services (MSIS)
 Team
— Dean Resnick from U.S. Census Bureau
— Christine Cox, Linda Bilheimer, Robin Cohen, and Chris Moriarity from
NCHS.
1
Outline
Survey
y Data
National Health Interview
Survey (NHIS)
Administrative Data
Medicaid Statistical
Information System (MSIS)
• Annual household survey
• Asks about health insurance at
time of interview
• As per BBA of 1997,
1997 states
provide individual-level monthly
Medicaid enrollment records to
CMS
• Lag: Only available with a lag of
2-3 years
Matching: In its secure data facility and with appropriate safeguards for
human subjects, Census appended Administrative Data to Survey Data
records by its internal PIK/Personal Identification Key
• 2001-2002 MSIS linked to 20012002 NHIS
2
Outline
Survey
y Data
National Health Interview
Survey (NHIS)
Administrative Data
Medicaid Statistical
Information System (MSIS)
• Annual household survey
• Asks about health insurance at
time of interview
• As per BBA of 1997,
1997 states
provide individual-level monthly
Medicaid enrollment records to
CMS
• Lag: Only available with a lag of
2-3 years
Matching: In its secure data facility and with appropriate safeguards for
human subjects, Census appended Administrative Data to Survey Data
records by its internal PIK/Personal Identification Key
• 2001-2002 MSIS linked to 20012002 NHIS
3
Outline
Survey
y Data
National Health Interview
Survey (NHIS)
Administrative Data
Medicaid Statistical
Information System (MSIS)
• Annual household survey
• Asks about health insurance at
time of interview
• As per BBA of 1997,
1997 states
provide individual-level monthly
Medicaid enrollment records to
CMS
• Lag: Only available with a lag of
2-3 years
Matching: In its secure data facility and with appropriate safeguards for
human subjects, Census appended Administrative Data to Survey Data
records by its internal PIK/Personal Identification Key
• 2001-2002 MSIS linked to 20012002 NHIS
4
Outline
Survey
y Data
National Health Interview
Survey (NHIS)
Administrative Data
Medicaid Statistical
Information System (MSIS)
• Annual household survey
• Asks about health insurance at
time of interview
• As per BBA of 1997,
1997 states
provide individual-level monthly
Medicaid enrollment records to
CMS
• Lag: Only available with a lag of
2-3 years
Matching: In its secure data facility and with appropriate safeguards for
human subjects, Census appended Administrative Data to Survey Data
records by its internal PIK/Personal Identification Key
• 2001-2002 MSIS linked to 20012002 NHIS
5
Introduction
 Question:
— Does the NHIS accuratelyy estimate the number of
uninsured?
— Can a prediction model calculate more accurate estimates of
Medicaid enrollment and uninsurance?
 Motivation
M ti ti
— Medicaid Undercount exists in the NHIS and other large
household surveys
– The Undercount can have large impacts on policy makers
and researchers using the NHIS
6
Medicaid Undercount
 NHIS: 22.8 million Medicaid beneficiaries in 2001
 MSIS:
MSIS 31.4
31 4 million
illi Medicaid
M di id beneficiaries
b
fi i i in
i 2001
— Net undercount of 27.3%
 The undercount may impact estimate of the
uninsured
i
d
— Respondents not reporting having insurance who have
Medicaid are incorrectly counted as uninsured
— It is possible to correct NHIS responses by linking the NHIS
respondents to administrative data on Medicaid enrollment
7
Outline
 Linking NHIS data with MSIS administrative data
— Determine reporting
p
g errors of Medicaid enrollment in NHIS
— Form a regression model on linked data to predict survey
response errors for more recent years of NHIS
 Results from linked data
 Discussion and Conclusion
— Imputation model increases estimates of Medicaid enrollees
for 2006/2007 from 31.3
31 3 million to 39
39.1
1 million
— Change in uninsurance estimates is trivial
8
Table 1: Linked MSIS-NHIS Data and Response Errors
(Averaging Counts from 2001 and 2002 Data)
NHIS Response
True MSIS Status
Medicaid
in MSIS
No
M di id
Medicaid
in MSIS
Total
Medicaid
Onlyy
Medicaid
and Other Other Onlyy
16.0
3.5
7.0
2.7
Total
29.2
Row %
54.7%
12.0%
24.0%
9.3%
100.0%
Cell %
5.9%
1.3%
2.6%
1.0%
---
Weighted Count
(millions)
2.0
0.7
202.8
38.0
243.4
Row Percentage
0.8%
0.3%
83.3%
15.6%
100.0%
g
Cell Percentage
0.7%
0.3%
74.4%
13.9%
---
Weighted Count
(millions)
18.0
4.2
209.8
40.7
272.6
Row Percentage
6.6%
1.5%
76.9%
14.9%
Weighted Count
(millions)
Uninsured
Table 1: Linked MSIS-NHIS Data and Response Errors
(Averaging Counts from 2001 and 2002 Data)
NHIS Response
True MSIS Status
Medicaid
in MSIS
No
M di id
Medicaid
in MSIS
Total
Medicaid
Onlyy
Medicaid
and Other Other Onlyy
16.0
3.5
7.0
2.7
Total
29.2
Row %
54.7%
12.0%
24.0%
9.3%
100.0%
Cell %
5.9%
1.3%
2.6%
1.0%
---
Weighted Count
(millions)
2.0
0.7
202.8
38.0
243.4
Row Percentage
0.8%
0.3%
83.3%
15.6%
100.0%
g
Cell Percentage
0.7%
0.3%
74.4%
13.9%
---
Weighted Count
(millions)
18.0
4.2
209.8
40.7
272.6
Row Percentage
6.6%
1.5%
76.9%
14.9%
Weighted Count
(millions)
Uninsured
Table 1: Linked MSIS-NHIS Data and Response Errors
(Averaging Counts from 2001 and 2002 Data)
NHIS Response
True MSIS Status
Medicaid
in MSIS
No
M di id
Medicaid
in MSIS
Total
Medicaid
Onlyy
Medicaid
and Other Other Onlyy
16.0
3.5
7.0
2.7
Total
29.2
Row %
54.7%
12.0%
24.0%
9.3%
100.0%
Cell %
5.9%
1.3%
2.6%
1.0%
---
Weighted Count
(millions)
2.0
0.7
202.8
38.0
243.4
Row Percentage
0.8%
0.3%
83.3%
15.6%
100.0%
g
Cell Percentage
0.7%
0.3%
74.4%
13.9%
---
Weighted Count
(millions)
18.0
4.2
209.8
40.7
272.6
Row Percentage
6.6%
1.5%
76.9%
14.9%
Weighted Count
(millions)
Uninsured
Table 1: Linked MSIS-NHIS Data and Response Errors
(Averaging Counts from 2001 and 2002 Data)
NHIS Response
True MSIS Status
Medicaid
in MSIS
No
M di id
Medicaid
in MSIS
Total
Medicaid
Onlyy
Medicaid
and Other Other Onlyy
16.0
3.5
7.0
2.7
Total
29.2
Row %
54.7%
12.0%
24.0%
9.3%
100.0%
Cell %
5.9%
1.3%
2.6%
1.0%
---
Weighted Count
(millions)
2.0
0.7
202.8
38.0
243.4
Row Percentage
0.8%
0.3%
83.3%
15.6%
100.0%
g
Cell Percentage
0.7%
0.3%
74.4%
13.9%
---
Weighted Count
(millions)
18.0
4.2
209.8
40.7
272.6
Row Percentage
6.6%
1.5%
76.9%
14.9%
Weighted Count
(millions)
Uninsured
Table 1: Linked MSIS-NHIS Data and Response Errors
(Averaging Counts from 2001 and 2002 Data)
NHIS Response
True MSIS Status
Medicaid
in MSIS
No
M di id
Medicaid
in MSIS
Total
Medicaid
Onlyy
Medicaid
and Other Other Onlyy
Uninsured
Total
16.0
3.5
7.0
2.7
29.2
Row %
54.7%
12.0%
24.0%
9.3%
100.0%
Cell %
5.9%
1.3%
2.6%
1.0%
---
Weighted Count
(millions)
2.0
0.7
202.8
38.0
243.4
Row Percentage
0.8%
0.3%
83.3%
15.6%
100.0%
g
Cell Percentage
0.7%
0.3%
74.4%
13.9%
---
Weighted Count
(millions)
18.0
4.2
209.8
40.7
272.6
Row Percentage
6.6%
1.5%
76.9%
14.9%
Weighted Count
(millions)
Prediction Model With Linked Data
 Multivariate Logistic Regression models true
(MSIS) Medicaid enrollment status as a function of
reported (NHIS) Medicaid enrollment status and
other information from the NHIS interview
 Two Models
— First estimates probability of having Medicaid among those
who do not report having Medicaid in the NHIS
— Second estimates probability of not having Medicaid among
those who do report having Medicaid
 Model
M d l covariates
i t
— Receipt of Welfare, Ratio of income to poverty level, age,
receipt
p of SSI, relationship
p to survey
y responder,
p
g
gender of
responder, U.S. Citizen, Education, Race, Health insurance,
and self reported health.
14
Limitations
 We assume conditional on the covariates
response error patterns are unchanged over time
— NHIS did change its Medicaid question over time
— Sharp changes in public insurance coverage and levels of
uninsurance
i
15
Outline
 Linking NHIS data with MSIS administrative data
— Determine reporting
p
g errors of Medicaid enrollment in NHIS
— Form a regression model on linked data to predict survey
response errors for more recent years of NHIS
 Results from linked data
 Discussion and Conclusion
— Imputation model increases estimates of Medicaid enrollees
for 2006/2007 from 31.3
31 3 million to 39
39.1
1 million
— Change in uninsurance estimates is trivial
16
E trapolation to 2006 and 2007 NHIS Data
Extrapolation
Table 2: Comparing Medicaid Enrollment Estimates from Partially
Corrected Imputation
p
Model to the Regular
g
NHIS Estimates
(Calendar Year 2006 and 2007 Average)
Medicaid Enrollment Estimate Medicaid Enrollment Estimate
(NHIS)
(Imputed)
%
Total – Overall
10 6%
10.6%
SE
C
Count
t
0 21% 31,281,000
0.21%
31 281 000
P
Percent
t
13 3%
13.3%
SE
C
Count
t
0 19% 39,066,816
0.19%
39 066 816
17
E trapolation to 2006 and 2007 NHIS Data
Extrapolation
Table 2: Comparing Medicaid Enrollment Estimates from Partially
Corrected Imputation
p
Model to the Regular
g
NHIS Estimates
(Calendar Year 2006 and 2007 Average)
Medicaid Enrollment Estimate Medicaid Enrollment Estimate
(NHIS)
(Imputed)
%
Total – Overall
10 6%
10.6%
SE
C
Count
t
0 21% 31,281,000
0.21%
31 281 000
P
Percent
t
13 3%
13.3%
SE
C
Count
t
0 19% 39,066,816
0.19%
39 066 816
18
Table 3: Comparing Uninsured Rates Based on our Partially
Corrected Imputation Model to the Regular NHIS Estimates
(2006/2007)
NHIS Uninsurance
Rate
NHIS U
Uninsured
i
d
Cases Linked to
MSIS
NHIS Cases not
linked to MSIS
Adjusted
Adj
t d
Uninsurance
Rate
Percent
15.70%
1.20%
1.30%
15.80%
Standard
Error
0.22%
0.03%
0.03%
0.20%
46,296,288
3,571,509
3,863,132
46,587,910
Total
Number
19
Table 3: Comparing Uninsured Rates Based on our Partially
Corrected Imputation Model to the Regular NHIS Estimates
(2006/2007)
NHIS Uninsurance
Rate
NHIS U
Uninsured
i
d
Cases Linked to
MSIS
NHIS Cases not
linked to MSIS
Adjusted
Adj
t d
Uninsurance
Rate
Percent
15.70%
1.20%
1.30%
15.80%
Standard
Error
0.22%
0.03%
0.03%
0.20%
46,296,288
3,571,509
3,863,132
46,587,910
Total
Number
20
Table 3: Comparing Uninsured Rates Based on our Partially
Corrected Imputation Model to the Regular NHIS Estimates
(2006/2007)
NHIS Uninsurance
Rate
NHIS U
Uninsured
i
d
Cases Linked to
MSIS
NHIS Cases not
linked to MSIS
Adjusted
Adj
t d
Uninsurance
Rate
Percent
15.70%
1.20%
1.30%
15.80%
Standard
Error
0.22%
0.03%
0.03%
0.20%
46,296,288
3,571,509
3,863,132
46,587,910
Total
Number
21
Table 3: Comparing Uninsured Rates Based on our Partially
Corrected Imputation Model to the Regular NHIS Estimates
(2006/2007)
NHIS Uninsurance
Rate
NHIS U
Uninsured
i
d
Cases Linked to
MSIS
NHIS Cases not
linked to MSIS
Adjusted
Adj
t d
Uninsurance
Rate
Percent
15.70%
1.20%
1.30%
15.80%
Standard
Error
0.22%
0.03%
0.03%
0.20%
46,296,288
3,571,509
3,863,132
46,587,910
Total
Number
22
Outline
 Linking NHIS data with MSIS administrative data
— Determine reporting
p
g errors of Medicaid enrollment in NHIS
— Form a regression model on linked data to predict survey
response errors for more recent years of NHIS
 Results from linked data
 Discussion and Conclusion
— Imputation model increases estimates of Medicaid enrollees
for 2006/2007 from 31.3
31 3 million to 39
39.1
1 million
— Change in uninsurance estimates is trivial
23
Comparison to CPS
 CPS has a larger undercount (31.4% versus 27.3%)
 False positive error rate
— CPS: 6.1%
— NHIS: 4.8%
 False negative error rate
— CPS: 26.8%
%
— NHIS: 17.8%
 CPS Health insurance question asks about
coverage in previous calendar year, NHIS asks
about coverage at time of interview
 This implies NHIS does a better job of measuring
the concept it claims to be measuring
24
Conclusion
 This paper uses linked data (administrative and
survey) to estimate partially corrected estimates of
the NHIS 2006/2007 for Medicaid enrollment and
uninsurance
 Partially corrected results for uninsurance aren’t
aren t
substantially different than what can be obtained
from the NHIS
 This
Thi result
lt implies
i li that
th t NHIS may be
b a better
b tt tool
t l
to estimate health insurance coverage than CPS
25