Anonymity and Sensitivity Fritz Scheuren, Speaker NORC University of Chicago February 2013 Scope of USA Migration Studies • Emigration from USA to World • Immigration from World to USA Direct Indirect Outline of Talk • Problem • Science of Approach • Engineering of Its Implementation • Results So Far! • Next Steps? Immigrants Sans Paper • Physical Border Protection Worldwide? • Mostly “Thought” to be South to North? • Only An Indirect Estimate Available Legal Immigrants First? • Over a Million Come Annually, still Largest Legal Flow Of Any Country • About 150 Million Made citizens since 1940 Undocumented May Be Much Smaller? • Flow is believed to average Less than Legal • Stock Estimated, perhaps, at 12 million • Flow is Business Cycle Driven, but usually less Physical Border Protection • Extremely long Borders with Canada and Mexico • Mexico Border Not Well Guarded • Drug Trafficking a Major Problem for USA/Mexico Mainly South to North? • Mexican Border Crossers but This Model is too Simple • Student and Visitor Visa Overstays • Still Often Tied to US Jobs Who Comes Legally Now? • Traditional Americas and Western Europeans but Less legally than earlier • Eastern Europeans Still in Large Numbers • Koreans and Chinese up, Africa coming up Science Approach • Two Card Method (Handout) • Two Independent Replicates • Difference Estimate of Number of Undocumented Two Card Example (Green Card) First Card • Two options Given -• First option: US Citizenship or Undocumented • Second Option: Green Card or Other Two Card Example (Other than Green) Second Card • Two Options Given – • First Option: US Citizenship • Second Option: Green Card or Other • Both Cards: Safe Answer Difference Undocumented • Estimate Unbiased, if Question Understood and Answered Honestly • But Variance Twice as Large as Direct Question • However Bias Eliminated Engineering Solution Human Factors Design Study • To test Factor Importance • Protecting Anonymity to Gain Unbiased Answers • A Credible Balancing Bar? Balance of Trust? • Key is Creating High Anonymity to Overcome PERCEIVED level of risk. • Given the Sensitivity of the Subject • Need Data so we tested. Sensitivity Loss • The Two Card method has a Variance Cost and is a challenge in moving Univariate analysis to a Multivariate form • The Newness remains for Anonymity form Anonymity Loss • Credibility of Promise is a real concern? • So Move to a Promise that CONSTRUCTS Established Trust? • Key Improvement! Results So Far • About 350 tests of the Two-Card method and its variations have been undertaken. • The tests show that the results are robust to minor wording changes More Results So Far • The Results are not robust to being wordy. • Potential respondent are not paid, probably a mistake. • If version was lengthy, more break-offs likely. Still More Highlights • Location changes, also are not robust. • Intercepts near the Mexican Border (near San Diego) were seen as more risky, so stopped. Other Changes? • If scaled up, more characteristics and more intercept locations are definitely needed. • Respondent based area probabilities needed to adjust for self selection. Interviewer Training • The extensive paradata calculated needs to be turned into a model for training. • Intercept Interviews are changing NORC Team. • How Measure/Control? Intercept Scalability • How replicable are the Interceptors we chose? • How replicable and scalable is this process? • What controls are there and how scientific are we? Selected Respondent Comments • [Interview] was easy, because none of the questions are hard and explained it very well. • [Glad you] understand the immigrants and their education level. More Respondent Comments • He was hesitant at first and I asked him why. • He said because people [might identify him].I asked him how I was going to find out. I told him [many] are similar]. Next Steps Coming? • Write up Results of this small NSF grant? • Expand to another Undocumented group: Asians, Russian Jews? • Take Sting off Hispanics Portable to Europe? Your Question! • In a register based society is this needed? • Some evidence says Yes but little leakage? • More references needed? Short list follows? Many, Many Thanks! • Contact Details Fritz Scheuren +1-202-320-3446 [email protected] References • Cochran, William G. Sampling Techniques. New York: Wiley, 1977. Print. • Dyson, Freeman J. "Is Science Mostly Driven by Ideas or by Tools?" Science 338.6113 (2012): 1426-427. Print. • Galison, Peter. Image and Logic: A Material Culture of Microphysics. Chicago: University of Chicago, 1997. References • GAO. A "Grouped Answers" Approach to Surveying Foreign-Born Respondents. Rep. no. GAO-06-775. N.p.: GAO, 2006. Print. • Kuhn, Thomas S. The Structure of Scientific Revolution. Chicago: University of Chicago, 1996. Print. References • Lehoux, Daryn, and Jay Foster. "History and Philosophy of Science: A Revolution of Its Own." Science 338.6109 (2012): 885-86. Print. • Pasic, Peter.” What the Romans Really Knew,” Science 338:273-274. • Reynolds, Gregg. Immigrant Experience Survey Interim Report. , 2013.