Anonymity and
Sensitivity
Fritz Scheuren, Speaker
NORC University of Chicago
February 2013
Scope of USA
Migration Studies
• Emigration from USA to
World
• Immigration from World
to USA
Direct
Indirect
Outline of Talk
• Problem
• Science of Approach
• Engineering of Its
Implementation
• Results So Far!
• Next Steps?
Immigrants Sans Paper
• Physical Border
Protection Worldwide?
• Mostly “Thought” to be
South to North?
• Only An Indirect
Estimate Available
Legal Immigrants
First?
• Over a Million Come
Annually, still Largest
Legal Flow Of Any
Country
• About 150 Million Made
citizens since 1940
Undocumented May
Be Much Smaller?
• Flow is believed to
average Less than Legal
• Stock Estimated,
perhaps, at 12 million
• Flow is Business Cycle
Driven, but usually less
Physical Border
Protection
• Extremely long Borders
with Canada and Mexico
• Mexico Border Not Well
Guarded
• Drug Trafficking a Major
Problem for USA/Mexico
Mainly South to North?
• Mexican Border
Crossers but This Model
is too Simple
• Student and Visitor Visa
Overstays
• Still Often Tied to US
Jobs
Who Comes Legally
Now?
• Traditional Americas and
Western Europeans but
Less legally than earlier
• Eastern Europeans Still
in Large Numbers
• Koreans and Chinese up,
Africa coming up
Science Approach
• Two Card Method
(Handout)
• Two Independent
Replicates
• Difference Estimate of
Number of
Undocumented
Two Card Example
(Green Card)
First Card
• Two options Given -• First option: US
Citizenship or
Undocumented
• Second Option: Green
Card or Other
Two Card Example
(Other than Green)
Second Card
• Two Options Given –
• First Option: US
Citizenship
• Second Option: Green
Card or Other
• Both Cards: Safe Answer
Difference Undocumented
• Estimate Unbiased, if
Question Understood
and Answered Honestly
• But Variance Twice as
Large as Direct Question
• However Bias Eliminated
Engineering Solution
Human Factors Design
Study
• To test Factor
Importance
• Protecting Anonymity to
Gain Unbiased Answers
•
A Credible
Balancing Bar?
Balance of Trust?
• Key is Creating High
Anonymity to Overcome
PERCEIVED level of risk.
• Given the Sensitivity of
the Subject
• Need Data so we tested.
Sensitivity Loss
• The Two Card method
has a Variance Cost and
is a challenge in moving
Univariate analysis to a
Multivariate form
• The Newness remains
for Anonymity form
Anonymity Loss
• Credibility of Promise is
a real concern?
• So Move to a Promise
that CONSTRUCTS
Established Trust?
• Key Improvement!
Results So Far
• About 350 tests of the
Two-Card method and its
variations have been
undertaken.
• The tests show that the
results are robust to
minor wording changes
More Results So Far
• The Results are not
robust to being wordy.
• Potential respondent are
not paid, probably a
mistake.
• If version was lengthy,
more break-offs likely.
Still More Highlights
• Location changes, also
are not robust.
• Intercepts near the
Mexican Border (near
San Diego) were seen as
more risky, so stopped.
Other Changes?
• If scaled up, more
characteristics and
more intercept locations
are definitely needed.
• Respondent based area
probabilities needed to
adjust for self selection.
Interviewer Training
• The extensive paradata
calculated needs to be
turned into a model for
training.
• Intercept Interviews are
changing NORC Team.
• How Measure/Control?
Intercept Scalability
• How replicable are the
Interceptors we chose?
• How replicable and
scalable is this process?
• What controls are there
and how scientific are
we?
Selected Respondent
Comments
• [Interview] was easy,
because none of the
questions are hard and
explained it very well.
• [Glad you] understand
the immigrants and their
education level.
More Respondent
Comments
• He was hesitant at first
and I asked him why.
• He said because people
[might identify him].I
asked him how I was
going to find out. I told
him [many] are similar].
Next Steps Coming?
• Write up Results of this
small NSF grant?
• Expand to another
Undocumented group:
Asians, Russian Jews?
• Take Sting off Hispanics
Portable to Europe?
Your Question!
• In a register based
society is this needed?
• Some evidence says Yes
but little leakage?
• More references
needed? Short list
follows?
Many, Many Thanks!
• Contact Details
Fritz Scheuren
+1-202-320-3446
scheuren@aol.com
References
• Cochran, William G. Sampling
Techniques. New York: Wiley, 1977. Print.
• Dyson, Freeman J. "Is Science Mostly
Driven by Ideas or by Tools?" Science
338.6113 (2012): 1426-427. Print.
• Galison, Peter. Image and Logic: A
Material Culture of Microphysics.
Chicago: University of Chicago, 1997.
References
• GAO. A "Grouped Answers"
Approach to Surveying Foreign-Born
Respondents. Rep. no. GAO-06-775.
N.p.: GAO, 2006. Print.
• Kuhn, Thomas S. The Structure of
Scientific Revolution. Chicago:
University of Chicago, 1996. Print.
References
• Lehoux, Daryn, and Jay Foster. "History
and Philosophy of Science: A Revolution
of Its Own." Science 338.6109 (2012):
885-86. Print.
• Pasic, Peter.” What the Romans Really
Knew,” Science 338:273-274.
• Reynolds, Gregg. Immigrant Experience
Survey Interim Report. , 2013.