FY10 ACS Methods Panel Update
Jennifer Guarino Tancreto
Chief, ACS Data Collection Methods Staff
Decennial Statistical Studies Division
Presentation to COPAFS
March 5, 2010
1
ACS Methods Panel
Vehicle for testing changes to ACS
• Improving current questions/adding new
questions
• Testing new methods to improve costefficiency
2
FY 2010 ACS Methods Panel
• Results of two tests from 2009
• Conduct one test
• Plan for two future tests
3
Tests conducted in FY09
4
Additional Mailing to Mail Nonrespondents
•
Goal: improve mail response
•
Postcard or another questionnaire package
– Mailed ~2 weeks after replacement questionnaire
•
Tested in March 2009
– 1/3 got third questionnaire
– 1/3 got postcard
– 1/3 did not get an additional mailing
5
Current ACS Data Collection Methods
Mail
NR
Phone #?
Yes
Telephone
Follow-Up
NR
Subsample
Personal
Visit
Follow-Up
No
Pre-notice
Letter
Initial
Questionnaire
Reminder
Postcard
NR
Replacement
Questionnaire
6
ACS Data Collection with an
Additional Mailing
Mail
NR
Phone #?
No
Yes
Telephone
Follow-Up
Additional
Mailing
NR
Subsample
Personal
Visit
Follow-Up
NR
7
8
Additional Mailing to Mail Nonrespondents
Response Rate by Treatment
Response Rate
30%
20%
21.3%
22.3%
Postcard
Questionnaire
15.2%
10%
0%
Control
Treatment
9
Multi-lingual Brochure Test
•
Goal: assist non-English speaking households
in responding to ACS
•
Mail brochure in five languages: English,
Spanish, Chinese, Russian, Korean
– Dedicated toll-free number for each language
•
Tested in April through July 2009
– 1/3 got brochure with pre-notice
– 1/3 got brochure with initial questionnaire
– 1/3 did not get brochure
10
11
12
Multi-lingual Brochure Test
• Results imminent
• Study impact on response, specifically
– Response from households that report speaking a
language in the brochure
13
FY10 Testing
14
2010 ACS Content Test
• Similar in design to 2006 ACS content
test
• OMB Interagency Committee for the
ACS provided structure/process for
identifying new and revised content
15
2010 ACS Content Test - New Questions
1. Computer Ownership and Internet
Access (FCC)
2. Parental Place of Birth (Census)
16
2010 ACS Content Test - Revised Questions
1. Food Stamps (USDA)
– Change program name to SNAP
2. Veteran Identification and Period of Service
(VA)
3. Public Assistance Income (HHS)
4. (CATI/CAPI only) Wages/Salary/etc. and
Interest/Dividends/etc. income (HHS)
17
2010 ACS Content Test – Process
• Subcommittees determine preliminary
questions
• Cognitive testing
• Expert Panel
• Field Test
18
2010 ACS Content Test
• Split-panel design
– Control: current questions + version 1 of new questions
– Test: revised questions + version 2 of new questions
• Initial sample of 35,000 housing units per
panel
– Nonresponse follow-up by telephone, pers. visit
– Behavior coding using recordings (CARI)
– Content follow-up reinterview
19
2010 ACS Content Test
• Field test in September – November
2010
• Final changes in ACS production in
January 2013
20
Plans for Future Testing
21
2011 ACS Internet Test
• Evaluate feasibility of Internet option
for ACS and PRCS
– English and Spanish
• Experimentally test ways of offering the
Internet option to maximize Internet
response
22
2011 ACS Internet Test
• Treatments:
–
–
–
–
Prominent Offer (Choice)
Not Prominent Offer
Push Internet
Push Internet (modified mail schedule)
• Control (production): No Internet offer
23
24
25
26
2011 ACS Internet Test
• Two strata: Targeted and Not Targeted
• Fully factorial design of 10 experimental
panels
• Main evaluation measure – response
rates
27
2011 ACS Internet Test
• Designed to present stimuli consistent with
other ACS data collection modes to minimize
potential mode effects
But at the same time…
• Take advantage of the technology to improve
data quality to extent possible
28
29
Open-ended question
30
Mark one
31
Check boxes
32
2011 ACS Internet Test
• Content reinterview (by phone) of INT
respondents
• Follow-up interview about materials and
Internet accessibility
• Test planned for March 2011
• Results expected late 2011
33
ACS Content Reinterview Survey
• Study and monitor response error for ACS
questions
– Help identify items for content testing
– Provide concrete estimates of response error
• Exploring use of Latent Class analysis
• Timing TBD
34
Contact
Jennifer Guarino Tancreto
Jennifer.Guarino.Tancreto@census.gov
301-763-4250
35