SC 516: Survey Methodology
Spring 2011
Wednesday 12:00-2:20 pm
415 McGuinn Hall
Professor: Sara Moorman
Office: 404 McGuinn Hall
Office hours: Mondays 1:00-3:00 pm or by appointment
E-mail: Sara.Moorman.1@bc.edu
Phone: 617-552-4209
About the Course
You’ve heard the saying “Lies, damn lies, and statistics.” Yet statistics comprise much of the
daily news: 47% of voters approve of the president. Two-thirds of adults are obese. More than
50% of marriages end in divorce. In this course, you will become “survey-literate,” learning how
to understand and evaluate such statistics. You’ll also learn how to design, field, and analyze
your own survey, and how to report accurate, comprehensible results. This is a course in social
science research methods, not statistics; no prior background is required.
Required Readings
Texts to purchase:
Groves, Robert M., Floyd J. Fowler Jr., Mick P. Couper, et al. 2009. Survey Methodology,
Second Edition. Hoboken, NJ: John Wiley. ISBN: 9780470465462
(Abbreviated “SM” in the reading list, below.)
Stone, Arthur A., Jaylan S. Turkkan, Christine A. Bachrach, et al. 2000. The Science of SelfReport: Implications for Research and Practice. Mahwah, NJ: Lawrence Erlbaum.
ISBN: 0805829911
(Abbreviated “SSR” in the reading list, below.)
Course reserves online:
Access additional readings marked with an asterisk (*) as .pdf files through the library website
(http://www.bc.edu/libraries/). On the left side of the page under “Find Library Materials” click
on “Course Reserves.” Log in, and search by course number, course name, or instructor’s last
name.
Assessment
Grading scale
A+
none at Boston College
B+
87 – 89%
C+
77 – 79%
D+
67 – 69%
F
below 60%
A
B
C
D
93 – 100%
83 – 86%
73 – 76%
63 – 66%
ABCD-
90 – 92%
80 – 82%
70 – 72%
60 – 62%
SC 516 Survey Methodology
Task
Ethics training
Sample selection
Surveys in the news
Item design
Class presentation
Evaluation of results
2
Due date
February 2
February 16
February 23
March 23
April 27
April 6
May 4
May 11
Percentage of grade
15
10
45 (3 @ 15% each)
10
10
10
Ethics training: Boston College is concerned that its researchers respect and protect the rights
and welfare of human participants in research. As a researcher and student of research methods
and ethics, you should be formally trained and certified. To that end, you’ll complete the online
CITI course on Human Subjects Research by February 2. It will take you a couple of hours (it
can be completed in multiple sessions if you so desire), and can be found here:
https://www.citiprogram.org/Default.asp. I’ll demonstrate the website in class on January 26.
When you finish the tutorial, you’ll print out a certificate of completion, which you’ll hand in to
me as evidence of your work. Worth: 15% of your final grade.
Your survey: In this class, it’s important not only to learn theory, but also to get hands-on
practice designing, fielding, and analyzing your own survey. You’ll do this in a series of stages
as we cover relevant topics in class. By February 16, you’ll have come up with a strategy for
selecting your sample participants. By April 6, you’ll have written the questions for your survey.
Then you’ll go collect data and analyze them, so that on May 4 you can report your findings to
the class, and on May 11 you can turn in a written final product. Details (e.g., length) on each of
these steps will be forthcoming in class. Worth: 40% of your final grade (10% for each stage).
Surveys in the news: Survey research results are reported in the news every day. Sometimes
they’re reported in an accurate and helpful manner; perhaps more often they are misconstrued.
Your task this semester is to become a more savvy consumer of the media. One of each week’s
assigned readings addresses what can happen when numbers from research are employed in
pursuit of political or social agendas. In addition to discussing these readings, you’ll do some
research of your own. First, you’ll find an instance of good or bad survey reporting. You may
find this yourself – media outlets ranging from The New York Times to ESPN report on surveys
– or you can use these watchdog websites to help you identify issues:
http://blogs.wsj.com/numbersguy/; www.numberwatch.co.uk; http://www.stats.org/. Second,
you’ll track down the official, original report on the data, which will be something like an
academic journal article or a governmental report. Third, you’ll find two different news reports
on the story. Finally, you’ll write a paper containing three sections: (a) Summarize—what were
the findings of the survey? (b) Analyze—what did the reports of the survey do right? What did
they do poorly or wrong? (c) Extrapolate—what are the consequences or implications of this
instance of reporting? Devote about one page to each of these goals, and turn in copies of the
original report and the news stories. You’ll complete this assignment three times throughout the
semester, turning in papers on February 23, March 23, and April 27. Worth: 45% of your final
grade (15% each).
SC 516 Survey Methodology
3
Assignment Submission
All assignments are due in hard copy in class, or if you must, be e-mail by 11:59 pm Eastern
time Wednesday. Papers submitted after 11:59 pm on the due date are late and will lose a letter
grade a day. That is, a paper submitted on time is worth, at best, an A. A paper submitted
between midnight and 11:59 pm of the day following the due date is worth, at best, a B. If you
wish to avoid the late penalty, you must make arrangements with me in advance, or you must
provide proof of extenuating circumstances (e.g., doctor’s note, funeral program). If you e-mail
me a paper, I will respond confirming that I received it, can open the attached file, and am
grading it. If you haven’t received confirmation, then I do not have your paper.
SC 516 Survey Methodology
4
Academic Honesty
Your work must be your words and ideas. When writing papers, use quotation marks around
someone else’s exact words and identify whose words they are. If you come across a good idea,
by all means use it in your writing, but be sure to acknowledge whose idea it is. Do not allow
another student to copy your work. Failure to comply will result in (a) automatic failure of the
assignment, and (b) a report to the Dean and the Committee on Academic Integrity. For further
information, please review the College’s policies on academic integrity here:
http://www.bc.edu/offices/stserv/academic/resources/policy.html#integrity
Schedule
January 19: The Survey in Social Science Research
SM Chapters 1 and 2
SSR Chapter 1
*Sokolow, Jayme A. 2000. Lies, Damned Lies, and Statistics: The Use and Abuse of Numbers.
Association of Proposal Management Professionals Journal, Fall, 35-40.
January 26: Ethics
SM Chapter 11
SSR Chapter 2
*Wheeler, Michael. 1976. Lies, Damned Lies, and Statistics: The Manipulation of Public
Opinion in America. Chapter 9, “Political Poll-Vaulting.” Pp. 176-200. Chapter 12,
“Policing the Polls.” Pp. 245-269.
February 2: Sampling
SM Chapter 3
*Best, Joel. 2001. Damned Lies and Statistics: Untangling Numbers from the Media, Politicians,
and Activists. Chapter 2, “Soft Facts.” Pp. 30-61.
*Chang, Linchiat and Jon A. Krosnick. 2009. “National Surveys Via RDD Telephone
Interviewing Versus the Internet.” Public Opinion Quarterly 73(4): 641-678. doi:
10.1093/poq/nfp075
*Michael, Robert T. and Colm A. O'Muircheartaigh. 2009. "Design Priorities and Disciplinary
Perspectives: The Case of the US National Children's Study." Journal of the Royal
Statistical Society 171(2): 465-480. doi: 10.1111/j.1467-985X.2007.00526.x
February 9: Sampling
SM Chapter 4
*Almer, Ennis C. Statistical tricks and traps: An illustrated guide to the misuses of statistics. Part
I, “Sampling.” Pp. 40-45.
*Henderson, Tammy L., Maria Sorois, Angela Chia-Chen Chen, et al. 2009. »After a Disaster:
Lessons in Survey Methodology from Hurricane Katrina. » Population Research and
Policy Review 28(1): 67-92. doi: 10.1007/s11113-008-9114-5
*Kendall, Carl, Ligia Kerr, Rogerio C. Dongim, et al. 2008. “An Empirical Comparison of
Respondent-driven Sampling, Time Location Sampling, and Snowball Sampling for
Behavioral Surveillance in Men Who Have Sex with Men, Fortaleza, Brazil.” AIDS and
Behavior 12: S97-S104. doi: 10.1007/s10461-008-9390-4
SC 516 Survey Methodology
5
*Mlodinow, Leonard. 2008. The Drunkard’s Walk: How Randomness Rules Our Lives. Chapter
2, “The Laws of Truths and Half-Truths.” Pp. 124-145.
February 16: Mode of Response
SM Chapter 5
*Couper, Mick P. 2000. "Review: Web Surveys: A Review of Issues and Approaches."
Public Opinion Quarterly 64(4):464-94.
*De Leeuw, Edith. 2008. “Choosing the method of data collection.” Pp 113-135 in the
International Handbook of Survey Methodology, edited by Edith D. De Leeuw, Joop
Hox, and Don Dillman.
* Goldacre, Ben. Bad Science. Chapter 13, “Bad Stats.” Pp. 239-260.
February 23: Nonparticipation
SM Chapter 6
* Kaplan, Michael. 2006. Chances Are. Chapter 8, “Judging.” Pp. 176-206.
*Keeter, Scott, Courtney Kennedy, April Clark, Trevor Tompson, and Mike Mokrzycki. 2007.
“What’s missing in national landline RDD surveys?” Public Opinion Quarterly 71(5):
772-792. doi: 10.1093/poq/nfm053
*Lynn, Peter. 2008. “The problem of nonresponse.” Pp 35-55 in the International Handbook of
Survey Methodology, edited by Edith D. De Leeuw, Joop Hox, and Don Dillman.
March 2: Cognition
SM Chapters 7 and 8
SSR Chapters 3-6
* Kaplan, Michael. 2006. Chances Are. Chapter 7, “Healing.” Pp. 149-175.
***March 11: Spring Vacation, no class***
March 16: Asking about Events and Behaviors I
SSR Chapters 7, 8, 11-13
*Mlodinow, Leonard. 2008. The Drunkard’s Walk: How Randomness Rules Our Lives. Chapter
7, “Measurement and the Law of Errors.” Pp. 124-145.
March 23: Asking about Events and Behaviors II
SSR Chapters 14-16, 17-19
*Best, Joel. 2004. More Damned Lies and Statistics. Chapter 3, “Scary Numbers.” Pp. 63-90.
March 30: Asking about Opinions I
*Eaton, Asia A., Penny S. Visser, Jon A. Krosnick, and Sowmya Anand. 2009. “Social Power
and Attitude Strength Over the Life Course.” Personality and Social Psychology Bulletin
35(12): 1646-1660. doi: 10.1177/0146167209349114
*Gould, Stephen Jay. 1985. “The Median Isn’t the Message.” Discover.
*Tourangeau, Roger and Mirta Galesic. 2007. “Conceptions of Attitudes and Opinions.” Pp.
141-154 in The Sage Handbook of Public Opinion Research, edited by M. W. Traugott
and W. Donsbach. ISBN: 9781412911771
SC 516 Survey Methodology
6
*Visser, Penny S., Allyson Holbrook, and Jon A. Krosnick. 2007. “Knowledge and Attitudes.”
Pp. 127-140 in The Sage Handbook of Public Opinion Research, edited by M. W.
Traugott and W. Donsbach. ISBN: 9781412911771
April 6: Asking About Opinions II
*Cohen, Jacob. 1994. “The Earth Is Round (p < .05).” American Psychologist 49(12): 997-1003.
*Krosnick, Jon A., Allyson L. Holbrook, Matthew K. Berent, et al. 2002. "The Impact of "No
Opinion" Response Options on Data Quality: Non-Attitude Reduction or an Invitation to
Satisfice?" Public Opinion Quarterly 66(3): 371-403. doi: 10.1086/341394
*Schuman, Howard and Stanley Presser. 1996. “Measuring a Middle Position.” Pp. 161-178 in
Questions and Answers in Attitude Surveys. Thousand Oaks, CA: Sage.
*Schuman, Howard and Stanley Presser. 1996. “The Acquiescence Quagmire.” Pp. 203-230 in
Questions and Answers in Attitude Surveys. Thousand Oaks, CA: Sage.
April 13: Interviewer Effects
SM Chapter 9
*Best, Joel. 2008. Stat-Spotting: A Field Guide to Identifying Dubious Data. Part 2 G and H,
“Packaging: What are they telling us?” and “Debates: What if they disagree?” pp. 64-99.
*Durrant, Gabriele B., Robert M. Groves, Laura Staetsky, and Fiona Steele. 2010. “Effects of
Interviewer Attitudes and Behaviors on Refusal in Household Surveys.” Public Opinion
Quarterly 74(1): 1-36. doi: 10.1093/poq/nfp098
*Gathman, E. Cabell Hankinson, Douglas W. Maynard, and Nora Cate Schaeffer. 2008. “The
Respondents Are All Above Average: Compliments in a Survey Interview.” Research
on Language and Social Interaction 41(3): 271-301. doi: 10.1080/08351810802237867
April 20: No class; work on your survey
April 27 Data Management
SM Chapter 10
*Freese, Jeremy. 2007. “Replication Standards for Quantitative Social Science: Why Not
Sociology?” Sociological Methods and Research 36(2): 153-172. doi:
10.1177/0049124107306659
*Levitt, Steven D. 2005. “What Makes a Perfect Parent?” pp. 147-176 in Freakonomics. ISBN:
006073132
*Smith, Emma. 2008. “Doing Secondary Analysis.” Pp. 61-90 in Using Secondary Data in
Educational and Social Research. ISBN: 0335223575
May 4: Class Presentations