Technical Issues Associated with the American Community Survey Lisa Neidert NPC Poverty/American Community Survey Workshop June 22-26, 2009 Statistical Notes Reference purposes Do not need to use for presentation on Friday Often ignored for overview talks Often ignored, unless required by reviewer Statistical issues are not ACS-specific Census Bureau is more sensitive to statistical issues in 2008 than they were in 2000, 1990 Confidentiality Standard errors/Margins of Error in ACS, but not census products Resources: http://www.psc.isr.umich.edu/dis/acs/handouts/Compa ss_Appendix.pdf http://www.psc.isr.umich.edu/dis/acs/dataanalysis/Usi ngReplicateWeights.html Technical Appendices Each Compass guide has the same glossary and set of appendices http://www.psc.isr.umich.edu/dis/acs/han douts/Compass_Appendix.pdf Big Picture Appendix 3 – Definitions Standard Errors How to get a SE from MOE Margin of Error Reported with summary tabulations How to change from 90% to 95%; 99% Confidence Interval Coefficient of Variation Recommendations Coefficient of Variation CV = Standard Error/Population Estimate Note: Need to recover SE (Appendix 3) The smaller the CV the more reliable the estimate is Preferable that CV < 10% If using a proportion, the closer the proportion is to 0, the more unstable the CV is Coefficient of Variation . . . 1-year estimate vs 3-year estimate CV will be 70% larger for the 1-year estimate Decision is precision vs currency Use multi-year estimate if single year has CV > 10% If single year CV is > 50 percent, do not use Minimum population thresholds Rule of thumb: Population estimate (weighted) should be: >3,000 for 1-year >1,000 for 3-year >600 for 5-year [From older draft of Appendix 6] Multiyear Estimates When comparing multiyear estimates, use non-overlapping estimates 2005-2007 vs 2008-2010 Do not compare a single year estimate against a multiyear estimate Both should be multiyear estimates The single year estimate could be at the peak or trough of an economic cycle vs the more moderated multiyear estimate Margins of Error for Derived Estimates Summary data provide a margin of error for every cell Researchers often combine cells Geography like NYC community districts comprised of multiple census tracts Cells within a geography Combine several age groups Spreadsheet Illustrate New Derived MOE https://ctools.umich.edu/access/content/ group/34a72eab-daa4-4d14-80e09150727aed6c/Assignments/Technical% 20Appendix_Appendix_5.xls Statistical Significance Appendix 4 ACS vs ACS ACS vs Census Replicate Weights To correct for the fact that the samples are not random Census Bureau’s replicate weights are not handled by statistical packages (as an option, like JK or BRR) SAS program that has the algorithm https://ctools.umich.edu/access/content/gr oup/34a72eab-daa4-4d14-80e09150727aed6c/Technical%20%20Statistical/sas_code_example.txt Census Bureau recommendations for replicate weights Garrett, B. Dale and Michael Starsinic. 2008. “ACS Public Use Microdata Samples of 2005 and 2006 – How to Use Replicate Weights.” Presentation at AAPOR Conference, New Orleans, May 16, 2008. https://ctools.umich.edu/access/content/group/3 4a72eab-daa4-4d14-80e09150727aed6c/Technical%20%20Statistical/How%20to%20Use%20PUMS% 20Replicate%20Weights.ppt Theoretical Basis for Successive Difference Model Fay, R. and Train, G. 1995. “Aspects of Survey and Model-Based Postcensal Estimation of Income and Poverty Characteristics for States and Counties,” Proceedings of the Section on Governmental Statistics, American Statistical Association, pp. 154-159.