The 2006 American Community Survey
(ACS) Content Test Questions on
International Migration:
Improving Data on the U.S. Foreign-Born
Dean H. Judson
For presentation at the Conference of European
Statisticians, Edinburgh, Scotland, 20-22 November,
2006
CES Recommendations for 2010
Censuses: Migration Section
“To facilitate and improve the
comparability of data at a regional level
through the selection of a core set of
census topics and the harmonization of
definitions and classifications.”
2
Issues with U.S. Census Bureau
International Migration Data
• Difficult to obtain accurate distributions of the
foreign born by U.S. citizenship status (citizen
versus non-citizen)
• Difficult to estimate time spent in the United States by
the foreign born
3
Opportunities for Improvement:
The American Community Survey
• An ongoing nationwide survey that
collects socioeconomic and housing
information and
• replaces the long form component
of the 2010 census
4
Opportunities for Improvement:
The ACS Content Test
• 63,000 housing units
• Two versions of question content
1) a control version and
2) a variant, or test, version
3) Followup tested consistency of responses
• Changes that met data quality criteria will be
implemented in the 2008 ACS, and reflected in
the 2009 data release.
5
Improving Data on International
Migration: Questions on the ACS
• U.S. Citizenship Status
• Year of Entry
Related but Not Discussed Here. . .
• Place of Birth
• Residence One Year Ago
6
Motivation for Changes:
Citizenship Status
• Naturalization: appears to be over-reported in some
Census & survey data (Passel and Clark, 1997)
• Year of Naturalization:
-item would help to reduce reports of naturalization
by non-citizens (by prompting them to examine their
answer)
-Year naturalized could be compared with year
first entered to determine if respondent had been
in country long enough to naturalize
7
Motivation for Changes:
Year of Entry
• Current question does not account for multiple entries
(Redstone and Massey, 2004)
• Will better approximate host country experience by
asking for first & most recent entry
8
Content Test Questions: Citizenship
Control
Variant
9
Content Test Questions: Year of Entry
Control
Variant
10
Summary of Question Changes
Control version:
Test version:
Citizenship
-five categories, including
“Naturalized Citizen”
Citizenship
-five categories, including
“Naturalized Citizen” and
write-in for year naturalized
-American parent(s)
-U.S. Citizen Parents
Year of Entry
-Allows for reporting one entry
Year of Entry
-Allows for reporting more than
one entry (first and most recent)
11
Selection Criteria:
U.S. Citizenship Status
• The percent naturalized in the test
version will be equal to or less than the
percent for the control.
• The percent of non-responses in the
test will be equal to or less than the
percent for the control.
12
Citizenship Status: Naturalized Citizens
25
Percent
20
15
Test
Control
10
5
5.1
5.2
0
Citizen by naturalization
Universe: All nonblank records
13
Citizenship Status Nonresponse Rates
25.0
Percent
20.0
15.0
Test
Control
10.0
9.8
5.0
0.0
2.7
3.0
1
Citizenship Status
2
Year of Naturalization
1: Universe includes all nonblank records
2: Universe includes all test records of naturalized citizens
14
Summary: Citizenship Status
Percent Naturalized (Control versus Test)
No statistical difference
Item Non-response:
-Citizenship Status (Control versus Test)
No statistical difference
-Year of Naturalization
Ten percent non-response for those naturalized
Conclusion:
Asking for year naturalized had no statistically
significant effect on Citizenship item but does have
other uses
15
Selection Criteria:
Year of Entry
• The net difference rate will be lower in the test
version than the control (by period of entry).
• The percent of non-responses for the test version will
be equal to or less than the percent for the control.
(for information purposes only. . .)
• Determine if the year of entry values provided in the
control version reflect a first year of arrival, most
recent year of arrival, or something else.
16
Number of Arrivals: Test versus Follow-up
100
47.7
(4.6)
Percent
80
47.7
(4.6)
87.2
60
Test
60.6
Followup
40
39.4
20
12.8
0
Once
More Than Once
Universe: Test cases of population born outside the U.S.
17
Year Entered:
Control and Test versus Follow-Up
(Year entered matches exactly)
100
Percent
80
60
68.0 70.3
74.0
63.5
66.1 63.5
65.7
60.2
Test (only or most
recent arrival)
Control
40
20
0
2000 or
later
1990 to
1999
1980 to
1989
Before
1980
Universe: All persons born outside the United States.
18
Year Entered:
Control and Test versus Follow-Up
(Year entered matches within two years)
100
94.1
80
Percent
80.5
90.0
83.7
84.1 84.1
87.7 84.7
60
Test (only or most
recent arrival)
Control
40
20
0
2000 or
later
1990 to
1999
1980 to
1989
Before
1980
Universe: All persons born outside the United States.
19
Year of Entry Nonresponse Rates
Universe: Population born outside the United States
100
Percent
80
83.3
60
Test
Control
40
20
23.1
21.7
22.6
0
Year of
First/Only
Arrive
Year
of FirstArrival
or1
Arrivemore
More than
or Year of Entry
than once? 1
Only Arrival
Once?
Year
ofof
Most
Year
Most
Recent Arrival 2
Recent Arrival
1: Includes all nonblank records.
2: Includes all test cases that marked more than one entry.
20
Years Entered for Persons with Multiple Entries:
Follow-up Interviews with Control Group to Check for
Consistency of Year Provided
50
Same Decade
Year to Year
45
44.6
40
Percent
35
32.6
30
30.6
25
25.1
20
15
21.3
18.8
18.0
10
Matches
Decade/Period
of Last Arrival
Matches
Decade/Period
of First Arrival
Matches Both
No Match
Matches Year
of Last Arrival
Matches Year
of First Arrival
Matches Both
0
No Match
9.1
5
Universe: All control records indicating more than one arrival
in reinterview.
21
Summary: Year of Entry
Item Consistency
Number of Entries (Test versus Follow-Up):
Large difference (48 percentage points) in proportion of
respondents indicating single versus multiple entries.
Year Entered (Control and Test versus Follow-up):
Somewhat consistent reporting of exact year and good reporting
within two years.
Year Entered (Control vs. Follow-up, Multiple Entries):
Follow-up reporting indicated that the original response more often
represented the first year of arrival than the most recent year of arrival.
However, exact year matches and same decade reporting was poor,
with a sizeable proportion matching neither first nor last arrival.
22
Summary: Year of Entry (Cont.)
Item Non-response
Year Entered (Control versus Test):
• Only or first arrival
-No statistical difference
-non-response somewhat high for both versions (22 percent).
•
Year of most recent arrival
-very high non-response (83 percent)
Conclusion
The control version performed better, although
follow-up interviewing suggested concerns for the
control.
23
Thoughts on lessons learned
•
•
The purpose was to better represent the hard-toenumerate foreign born
Year of naturalization has analytic value
–
–
–
•
Did no harm to the overall question
Can be used for consistency checking
Appears to be well understood
Year of arrival
–
–
–
Despite successful cognitive testing…
Question form continues to be problematic
Many inconsistent responses
24