A Comparison of Two Sample
Designs for the MEPS-IC
John P. Sommers
Agency for Healthcare Research and Quality
Anne T. Kearney
U. S. Census Bureau
1
Presentation Outline
1.
2.
3.
4.
5.
6.
What is the MEPS-IC?
The Two Private Sector Sample Designs
Purpose of this Study
Measures Used to Compare
Results
Lessons Learned
2
The Medical Expenditure Panel Survey Insurance Component (MEPS-IC)
1. Annual survey of Business
Establishments and Governments
2. Information Collected on Offer Rates,
Enrollments, Costs and Characteristics of
Employer Health Insurance
3
Comparison of Old and New Designs
OLD
•14 strata per state
•Strata boundaries are
employment size classes
•Min sample in 40 states
•31 largest states have
minimum each year
•Average state variance
components
•Optimal allocation using 2
variables
NEW
•15 strata per state
•Strata boundaries are
predicted: % offering and
# enrollees
•Min sample in all states
•Average state variance
components
•Optimal allocation using
3 variables
4
Purpose of this Study
To determine if the new sample design fully
implemented in 2004 improved our
estimates of variances for eight key
variables of interest.
5
Problem: How to Evaluate and Compare
Sample Designs Across Years,
2002 vs. 2004
1. Could not compare standard errors due to
the natural increase in some standard
errors as mean values increase
2. Changes in sample allocation to states:
•
•
2002 had fewer sample units
2002 did not have min sample sizes in all
states
6
Quality Measures
Initial Step
1. We did comparisons over the 31 largest
states since they had similar sample size
before nonresponse in both years
2. These 31 largest states have over 90% of
universe
3. We created pseudo-national level
estimates from these 31 states
7
Quality Measures
1. Relative Standard Error (RSE)
2. Square Root of the Design Effect
3. Unit RSE = Square root of sample size
times RSE
8
Tested Hypothesis
H0  p(QM 2002  QM 2004 )  0.5
H A  p(QM 2002  QM 2004 )  0.5
9
Pseudo National Estimates
Measure
Unit RSE
2004
Root Design
Effect
2002
2004
RSE
Variable
2002
2002
2004
Avg. Family Contribution
2.277 2.062 0.434 0.333 0.0151 0.0144
Avg. Family Premium
0.897 0.745 0.408 0.249 0.0059 0.0052
Avg. Single Contribution* 2.343 2.007 0.489 0.444 0.0155 0.0141
Avg. Single Premium
0.921 0.837 0.514 0.404 0.0061 0.0059
% Employed Where Ins.
Offered
0.505 0.483 0.537 0.353 0.0034 0.0034
% Enrolled Where Ins.
Offered*
1.268 1.061 0.442 0.335 0.0086 0.0074
% of Employees Enrolled
1.358 1.138 0.414 0.304 0.0092 0.0079
% of Establishments That
Offer Health Insurance*
0.980 0.912 1.141 1.021 0.0066 0.0064
10
Average Results of State Estimates
Measure
Avg. Unit RSE
Avg. Root
Design Effect
2004 2002
Avg. RSE
Variable
2002
2004
2002
2004
Avg. Family Contribution
1.975 1.824 0.412 0.358
0.076
0.074
Avg. Family Premium
0.703 0.671 0.368 0.305
0.027
0.028
Avg. Single Contribution* 2.050 1.834 0.474 0.435
0.079
0.075
Average Single Premium
0.744 0.671 0.478 0.385
0.029
0.027
% Employed Where
Insurance Offered
0.453 0.477 0.494 0.411
0.018
0.019
% Enrolled Where
Insurance Offered*
1.149 1.053 0.417 0.345
0.045
0.042
% of All Employees
Enrolled
1.246 1.142 0.395 0.309
0.048
0.046
% of Establishments That
Offer Health Insurance*
0.974 0.902 1.068 0.957
0.038
0.036
11
Average National Results by Firm Size
Measure
Avg. Unit
RSE
2004
Avg. Rt.
Design Effect
2002
2004
Avg. RSE
Variable
2002
2002
2004
Firms with less
than 50 employees
1.510 1.510
1.077 1.143 0.0155 0.0160
Firms with 50 or
more employees
1.071 0.913
0.550 0.430 0.0095 0.0087
12
Average National Results by Industry
Measure
Avg. Unit
RSE
Avg. Rt.
Design Effect
Industry
2002 2004
Ag., Forestry and
Fishing
1.737 2.173 0.615 0.691 0.088 0.131
Construction
1.473 1.397 0.776 0.752 0.038 0.038
2002
2004
Avg. RSE
2002 2004
Fin Svcs / Real Estate 0.944 0.959 0.651 0.468 0.019 0.018
Mfg. and Mining
1.094 0.843 0.506 0.412 0.021 0.016
Other Services
1.630 1.665 0.686 0.714 0.024 0.028
Professional Services
1.305 1.012 0.534 0.386 0.017 0.014
Retail Trade
1.209 1.075 0.881 0.960 0.021 0.021
Utilities and Trans.
1.225 1.187 0.460 0.453 0.045 0.042
Wholesale Trade
1.017 1.018 0.832 0.477 0.029 0.029
13
Lessons Learned
• Targeted and most other estimates
improved at the State and National Level
• Effect of new sample design on estimates
for subpopulations appears to depend upon
the prevalence within the subcategory of
offering insurance to employees
14
John.Sommers@ahrq.hhs.gov
Anne.Theresa.Kearney@census.gov
15