Launching Adaptive Design at the US
Census Bureau
Presented by
Michael Thieme, Chief, Center for Adaptive Design
Peter Miller, Chief Scientist, Center for Adaptive Design
Discussion with the
Council of Professional Associations on Federal Statistics
December 7, 2012
U.S. Department of Commerce
Economics and Statistics Administration
U .S. CENSUS BU REAU
Center for Adaptive Design Overview
This new Center works in three areas:
Research
Through research, experimentation, and testing, prove that adaptive design principles
can be effective at increasing survey and census efficiency and timeliness
Outreach and Education
Using results from research and testing, communicate and demonstrate to survey
sponsors and stakeholders that implementing adaptive design is a viable tool for
increasing survey and census efficiency and timeliness
Design and Build
Develop a solution architecture, as well as a strategy for building and deploying the
systems that enable near-real-time cost–error trade-off decisions in data collection
operations
U.S. Department of Commerce
Economics and Statistics Administration
U .S. CENSUS BU REAU
2
CAD Research Activities for 2012-2013
•
1. Assembling and assessing data requisite
for adaptive design
–Auxiliary frame data
–Paradata
–Survey response data
U.S. Department of Commerce
Economics and Statistics Administration
U .S. CENSUS BU REAU
3
CAD Research Activities for 2012-2013
•
2. Test uses of Frame Enhancements and
Paradata in Survey Management
–Propensity models in simulation
–Business rules in simulation
–Business rules in the field
–Testing grounds: National Health
Interview Survey (NHIS), National Survey
of College Graduates (NSCG), others
U.S. Department of Commerce
Economics and Statistics Administration
U .S. CENSUS BU REAU
4
CAD Outreach and Education
•Trainings and follow-up for survey teams
•Development of custom dashboards to display
key survey metrics
•Work with Field on implementation issues
•Publicize activities and lessons
U.S. Department of Commerce
Economics and Statistics Administration
U .S. CENSUS BU REAU
5
Design and Build
• Documenting our current architecture to inform
where Adaptive Design can be applied
• Developing adaptive design overall conceptual
solution architecture, as well as initial baseline
architecture (Spring 2013)
• Target for initial baseline is to have a functioning
system in place for the 2015 American Community
Survey (ACS) and the 2014 Company Organization
Survey/Annual Survey of Manufacturers (COS/ASM)
U.S. Department of Commerce
Economics and Statistics Administration
U .S. CENSUS BU REAU
6
Communication About Adaptive Design
• Need to show that adaptive design is not cutting
corners
• Need to show that it saves money
• Need to show that it does not adversely affect
estimates, time series, or the job of the analyst
• Need to show a comparable product for the money
expended
U.S. Department of Commerce
Economics and Statistics Administration
U .S. CENSUS BU REAU
7
Precursors to Adoption
• Flexibility to give adaptive design a chance to show
whether it works or not
• Openness to consideration of alternative quality
measures (quality of estimates vs. response rate)
• Collaborative strategies for testing adaptive design in
a variety of surveys across government
U.S. Department of Commerce
Economics and Statistics Administration
U .S. CENSUS BU REAU
8
Adaptive Design Challenges
• Legislative and political considerations
• Use of administrative data and paradata
• Alternative contact methods
• Stopping data collection early
• The Decennial Census, the American Community
Survey, etc.
U.S. Department of Commerce
Economics and Statistics Administration
U .S. CENSUS BU REAU
9
Adaptive Design Challenges
• Transition between fixed and adaptive survey designs
at Census:
• Mode-switching based on analysis of paradata and
modeling
• 2015 – American Community Survey (ACS), Company Organization
Survey/Annual Survey of Manufacturers (COS/ASM)
• Maturing and implementing Adaptive Design business
rules (including stopping rules)
• 2016 – ACS, testing for 2020 Census, 2017 Economic Census
• 2017 – Economic Census
• 2020 – Decennial Census
U.S. Department of Commerce
Economics and Statistics Administration
U .S. CENSUS BU REAU
10
Moving Forward
• Take risks now
• Adaptive design is a potential remedy for declining
budgets and rising nonresponse
• Explore the potential so that when budgets decline
we have an understanding of how use it
U.S. Department of Commerce
Economics and Statistics Administration
U .S. CENSUS BU REAU
11
Questions and Discussion
U.S. Department of Commerce
Economics and Statistics Administration
U .S. CENSUS BU REAU
12