Questions for Standardized Measurement in Surveys Revised Mar 08 NOTE: This list of readings includes most of the readings referenced in my courses as well as many others. The list is not intended to be comprehensive. It provides a starting point to explore several topics related to survey measurement. Bibliography This Bibliography uses the following abbreviations: ASR American Sociological Review JOS Journal of Official Statistics JMR Journal of Marketing Research SMR Sociological Methods and Research JASA Journal of the American Statistical Association POQ Public Opinion Quarterly SM Sociological Methodology General Sources Aday, Lu A. and Llewellyn J. Cornelius. 2006. Designing and Conducting Health Surveys: A Comprehensive Guide, 3rd Edition. New York: Wiley. Belson, William A. 1981. England: Gower. Belson, William A. 1986. The Design and Understanding of Survey Questions. Aldershot, Hants, Validity in Survey Research. Aldershot, Hants, England: Gower. Biemer, Paul P., Robert M. Groves, Lars E. Lyberg, Nancy A. Mathiowetz, and Seymour Sudman. 1991. Measurement Errors in Surveys. New York: Wiley. Bradburn, Norman M., Seymour Sudman, and Associates. 1979. Questionnaire Design. San Francisco: Jossey-Bass. Briggs, Charles L. 1986. Improving Interview Method and Learning How to Ask. New York: Cambridge University Press. Converse, Jean, and Stanley Presser. 1986. Survey Questions: Handcrafting the Standardized Questionnaire. Beverly Hills: Sage. Quantitative Applications in the Social Sciences, v. 63. Converse, Jean, and Howard Schuman. 1974. Conversations at Random. Ann Arbor, Michigan: Wiley. Couper, Mick P., Reginald P. Baker, Jelke Bethlehem, Cynthia Z. F. Clark, Jean Martin, William L. Nicholls II, and James M. O'Reilly, editors. 1998. Computer Assisted Survey Information Collection. New York: John Wiley and Sons. Dillman, Don. 2006. Mail and Internet Surveys: The Tailored Design Method 2007 Update With New Internet, Visual, and Mixed-Mode Guide, 2nd Edition. New York: Wiley. Fowler, Floyd J. 1996. Improving Survey Questions: Design and Evaluation. Sage. Fowler, Floyd J., and Thomas W. Mangione. 1990. Standardized Survey Interviewing: Minimizing 2 Interviewer-Related Error. Newbury Park: Sage. Frey, James H. 1983. Survey Research by Telephone. Newbury Park: Sage. Groves, Robert M., and Robert L. Kahn. 1979. Surveys by Telephone: Personal Interviews. New York: Academic. Groves, Robert M. 1989. A National Comparison with Survey Errors and Survey Costs. New York: Wiley. Groves, Robert M., Don A. Dillman, John L. Eltinge, and Roderick J. A. Little, editors. 2002. Survey Nonresponse. New York: John Wiley and Sons. Groves, Robert M, Paul P. Biemer, Lars E. Lyberg, James T. Massey, William L Nicholls II, and Joseph Waksberg, editors. 1988. Telephone Survey Methodology. New York: Wiley. Hippler, Hans-Jurgen, Norbert Schwarz, and Seymour Sudman. 1987. Survey Methodology. New York: Springer-Verlag. Social Information Processing and Lyberg, Lars, Paul Biemer, Martin Collins, Edith de Leeuw, Cathryn Dippo, Norbert Schwarz, and Dennis Trewin. editors. 1997. Survey Measurement and Process Quality. New York: Wiley. Maynard, Douglas W., Hanneke Houtkoop-Steenstra, Nora Cate Schaeffer, and Johannes van der Zouwen, editors. 2002. Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview. New York: John Wiley and Sons. Mishler, Elliot G. 1986. Research Interviewing: Context and Narrative. Cambridge: Payne, Stanley. 1980. The Art of Asking Questions. Princeton, N.J.: Harvard. Princeton University Press. Presser, Stanley, Jennifer M. Rothgeb, Mick P. Couper, Judith T. Lessler, Elizabeth Martin, Jean Martin, and Eleanor Singer, editors. 2004. Methods for Testing and Evaluating Survey Questionnaires. New York: Wiley. Saris, Willem and Irmtraud Gallhofer. 2007. Design, Evaluation, and Analysis of Questionnaires for Survey Research. New York: Wiley. Schuman, Howard, and Stanley Presser. 1981. Academic. Questions and Answers in Attitude Surveys. Orlando: Schwarz, Norbert, and Seymour Sudman, eds. 1992. New York: Springer-Verlag. Context Effects in Social and Psychological Research. Schwarz, Norbert, and Seymour Sudman, eds.. 1994. Autobiographical Memory and the Validity of Retrospective Reports. New York: Springer-Verlag. Schwarz, Norbert, and Seymour Sudman, eds. 1996. Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research, San Francisco, CA: Jossey-Bass Publishers. 3 Sirken, Monroe G., Douglas J. Herrmann, Susan Schechter, Norbert Schwarz, Judith M. Tanur, and Roger Tourangeau, editors. 1999. Cognition and Survey Research. New York: John Wiley and Sons. Stone, Arthur A., Jaylan S. Turkkan, Christine A. Bachrach, Jared B. Jobe, Howard S. Kurtzman, and Virginia A. Cain, editors. 2000. The Science of Self-Report: Implications for Research and Practice. Mahwah, New Jersey: Lawrence Erlbaum. Sudman, Seymour, Norman M. Bradburn and Norbert Schwarz. 1996. Thinking about Answers: The Application of Cognitive Processes to Survey Methodology. San Francisco: Jossey-Bass Publishers. Sudman, Seymour, and Norman M. Bradburn. 1974. Response Effects in Surveys. Chicago: Sudman, Seymour, and Norman M. Bradburn. 1982. Asking Questions: Questionnaire Design. San Francisco: Jossey-Bass. Tanur, Judith, ed., 1992. Questions about Questions: New York: Russell Sage Foundation. Aldine. A Practical Guide to Inquiries into the Cognitive Bases of Surveys. Tourangeau, Roger, Lance J. Rips, and Kenneth Rasinski. 2000. The Psychology of Survey Response. Cambridge, England: Cambridge University Press. Turner, Charles F., and Elizabeth Martin. 1984. York: Russell Sage. Willis, Gordon. 2005. CA: Sage. Surveying Subjective Phenomena, Volume I and II. New Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, van 't Hof, Sanne. 2006. From Text to Talk: Answers and Their Uptake in Standardised Survey Interviews. Utrecht, The Neterlands: Netherlands Graduate School of LInguistics. Review Papers Dillman, Don A. 1991. "The Design and Administration of Mail Surveys." Annual Review of Sociology 17:225-49. Groves, Robert M. 1990. "Theories and Methods of Telephone Surveys." Annual Review of Sociology 16:221-40. Krosnick, Jon A. 1999. "Survey Research." Annual Review of Psychology 50(537-567). Saris, Willem and Irmtraud Gallhofer. 2007. "Estimation of the Effects of Measurement Characteristics on the Quality of Survey Questions." Survey Research Methods (1):29-43. Schaeffer, Nora Cate and Stanley Presser. 2003. "The Science of Asking Questions." Annual Review of Sociology, 29: 65-88. Download copy at: <http//arjournals.annualreviews.org/eprint/rU4UOoizjrXROhijkRIS/full/10.1146/annurev.soc.29.110702.11011 2> 4 Behaviors and Events Recall and Cognitive Processing Blair, Edward, and Scot Burton. 1987. "Cognitive Processes Used by Survey Respondents to Answer Behavioral Frequency Questions." Journal of Consumer Research 14:280-288. Bostrom, A., and Robinson, J.P. (1992). “Validity and accuracy in estimates of hours at work.” Paper presented at the annual meetings of he American Statistical Association, August, Boston. Bradburn, Norman, Lance J. Rips, and Steven K. Shevell. 1987. "Answering Autobiographical Questions: The Impact of Memory and Inference on Surveys." Science 236:157-161. Bradburn, Norman M., and Catalina Danis. 1984. "Potential Contributions of Cognitive Research to Survey Questionnaire Design." Pp. 101-129 in Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines, edited by Thomas Jabine, Miron Straf, Judith Tanur, and Roger Tourangeau. Washington, D.C.: National Academy Press. Graesser, Arthur C., Zhiqiang Cai, Max M. Louwerse, and Frances Daniel. 2006. "Question Understanding Aid (QUAID): A Web Facility That Tests Question Comprehensibility ." Public Opinion Quarterly 70(1):3-22. Holbrook, Allyson, Young I. Cho, and Timothy Johnson. 2006. "The Impact of Question and Respondent Characteristics on Comprehension and Mapping Difficulties." Public Opin Q 70(4):565-95. Huttenlocher, Janellen, Larry Hedges, and V. Prohaska. 1988. Hierarchical Organization in Ordered Domains: Estimating the Dates of Events." Psychological Review 95: 471-484. Huttenlocher, Janellen, Larry V. Hedges, and Norman M. Bradburn. 1990. "Reports of Elapsed Time: Bounding and Rounding Processes in Estimation." Journal of Experimental Psychology: Learning, Memory, and Cognition 16(2): 196-213. Isen, Alice M. 1984. "Toward Understanding the Role of Affect in Cognition." Pp. 179-236 in Handbook of Social Cognition, Volume 3., edited by Robert Wyer and Thomas Srull. Hillsdale, N.J.: Lawrence Erlbaum. Jabine, Thomas, Miron L. Straf, Judith M. Tanur, and Roger Tourangeau. 1984. Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines. Washington, DC: National Academy Press. Jobe, Jared B., Roger Tourangeau, and Albert F. Smith. 1993. "Contributions of Survey Research to the Understanding of Memory. Applied Cognitive Psychology 7:567-584. Jobe, Jared B., Andrew A. White, Catherine L. Kelley, David J. Mingay, Marcus J. Sanchez, and Elizabeth F. Loftus. 1990. "Recall Strategies and Memory for Health-care Vistis." The Millbank Quarterly 68,2:171-189. Johnson, Marcia K., Carol L. Raye, Alvin Y. Wang, and Thomas H. Taylor. 1979. "Fact and Fantasy: The 5 Roles of Accuracy and Variability in Confusing Imaginations with Perceptual Experiences." Journal of Experimental Psychology: Human Learning and Memory 5:229-240. Johnson, Marcia K., Thomas H. Taylor, and Carol L. Raye. 1977. "Fact and Fantasy: The Effects of Internally Generated Events on the Apparent Frequency of Externally Generated Events." Memory and Cognition 5:116-122. Krosnick, Jon A., Fan Li, and Darrin R. Lehman. 1990. "Conversational Conventions, Order of Information Acquisition, and the Effect of Base Rates and Individuating Information on Social Judgments." Journal of Personality and Social Psychology 59,6:1140-1152. Lessler, Judith, Roger Tourangeau, and W. Salter. 1989. Questionnaire Design in the Cognitive Research Laboratory: Results of an Experimental Prototype. Vital and Health Statistics (DHHS Publication No. PHS 89-1076 - Series 6 - No. 1) Washington, DC: U.S. Government Printing Office. Means, Barbara, Arti Nigam, Marlene Zarrow, Elizabeth F. Loftus, Molla S. Donaldson. 1989. "Autobiographical Memory for Health-Related Events." National Center for Health Statistics. Vital and Health Statistics, Series 6, Cognition and Survey Measurement, No. 2. Menon, Geeta. 1997. "Are the Parts Better Than the Whole? The Effects of Decompositional Questions on Judgments With Frequent Behaviors." Journal of Marketing Reserach 34:335-46. Menon, Geeta and Eric A. Yorkston. 2000. "The Use of Memory and Contextual Cues in the Formation of Behavioral Frequency Judgments." Pp. 63-80 in The Science of Self-Report: Implications for Research and Practice, edited by Arthur A. Stone, Jaylan S. Turkkan, Christine A. Bachrach, Jared B. Jobe, Howard S. Kurtzman, and Virginia S. Cain. Mahwah, New Jersey: Lawrence Erlbaum. Schwarz, Norbert and Julia Bienias. 1990. "What Mediates the Impact of Response Alternatives on Frequency Reports of Mundane Behaviors?" Applied Cognitive Psychology 4(1):61-72. Rose, Robert J. 1980. "Encoding Variability, Levels of Processing, and the Effects of Spacing of Repetitions upon Judgments of Frequency." Memory and Cognition 8:84-93. Rubin, D.C. and A.D. Baddley. 1989. "Telescoping Is Not Time Compression: Autobiographical Events." Memory and Cognition 17: 653-661. A Model of Dating Smith, Albert F. 1991. "Cognitive Processes in Long-Term Dietary Recall." National Center for Health Statistics, Vital and Health Statistics, Series 6, Cognition and Survey Measurement, No. 4. Sudman, Seymour, Richard Warneke, Timothy Johnson, Diane O'Rourke, Andrew Davis. 1994. Cognitive Aspects of Reporting Cancer Prevention Examinations and Tests. Series 6, No. 7, National Center for Health Statistics. Sherman, Steven J., and Eric Corty. 1984. "Cognitive Heuristics." Pp. 189-286 Handbook of Social Cognition, Volume 1., edited by Robert S. Wyer and Thomas K. Srull. Hillsdale, N.J.: Lawrence Erlbaum. Sudman,Seymour and Norman Bradburn. 1974. "Effects of Time and Memory Factors on Response." Pp. 6 67-92 in Response Effects in Surveys. Chicago: Aldine. Tourangeau, Roger. 2000. "Remembering What Happened: Memory Errors and Survey Reports." Pp. 29-48 in The Science of Self-Report: Implications for Research and Practice, edited by Arthur A. Stone, Jaylan S. Turkkan, Christine A. Bachrach, Jared B. Jobe, Howard S. Kurtzman, and Virginia S. Cain. Mahwah, New Jersey: Lawrence Erlbaum. Accuracy of Answers Belli, Robert F., Norbert Schwarz, Eleanor Singer, and Jennifer Talarico. 2000. "Decomposition Can Harm the Accuracy of Behavioral Frequency Reports." Applied Cognitive Psychology 14:295-308. Burton, Scot, and Ed Blair. 1991. "Task Conditions, Response Formulation Processes, and Response Accuracy for Behavioral Frequency Questions in Surveys." POQ 55:50-79. Chu, Adam, Donna Eisenhower, Michael Hay, David Morganstein, John Neter, and Joseph Waksberg. 1992. "Measuring the Recall Error in Self-Reported Fishing and Hunting Activities." JOS 8(1):19-39. Collins, Linda M., John W. Graham, William B. Hansen, and C. Anderson Johnson. 1985. "Agreement Between Retrospective Accounts of Substance Use and Earlier Reported Substance Use." Applied Psychological Measurement 9,3:301-309. Duncan, Greg, and Nancy A. Mathiowetz. 1985. Michigan: Institute for Survey Research. A Validation Study of Economic Survey Data. Ann Arbor, Dykema, Jennifer and Nora Cate Schaeffer. 2000. "Events, Instruments, and Reporting Errors." American Sociological Review 65(4):619-29. Hamilton, Lawrence C. 1981. "Self-Reports of Academic Performance: Well-Behaved." SMR 10,2:165-185. Response Errors are Not Mathiowetz, Nancy A., and Greg J. Duncan. 1988. "Out of Work, Out of Mind: Response Errors in Retrospective Reports of Unemployment." Journal of Business and Economic Statistics 6(2):221-229. Means, Barbara, Gary E. Swan, Jared B. Jobe, and James L. Esposito. 1991. "An Alternative Approach to Obtaining Personal History Data." Pp. 167-184 in P. Biemer, R. Groves, L. Lyberg, N. Mathiowetz, and S. Sudman, eds. Measurement Errors in Surveys. New York: Wiley. Neter, John, and Joseph Waksberg. 1964. "A Study of Response Errors in Expenditures Data from Household Interviews." JASA 59: 18-55. Parry, Hugh J., and Helen M. Crossley. 1950. "Validity of Responses to Survey Questions." 14:61-80. POQ Pearson, Robert W., Michael Ross, and Robyn M. Dawes. 1992. "Personal Recall and the Limits of Retrospective Questions in Surveys." Pp. 65-94 in J. Tanur, ed., Questions about Questions: Inquiries into the Cognitive Bases of Surveys. New York: Russell Sage Foundation. 7 Presser, Stanley. 1984. "Is Inaccuracy on Factual Survey Items Item-Specific or Respondent-Specific?" POQ 344-355. Rodgers, Willard L., Charles Brown, and Greg J. Duncan. 1993. "Errors in Survey Reports of Earnings, Hours Worked, and Hourly Wages." JASA 88(424):1208-1219. Schaeffer, Nora Cate and Jennifer Dykema. 2004. "A Multiple-Method Approach to Improving the Clarity of Closely Related Concepts: Distinguishing Legal and Physical Custody of Children." Pp. 475-502 in Methods for Testing and Evaluating Survey Questionnaires, edited by Stanley Presser, Jennifer M. Rothgeb, Mick P. Couper, Judith T. Lessler, Elizabeth Martin, Jean Martin, and Eleanor Singer. New York: Springer-Verlag. Schaeffer, Nora Cate, Judith A. Seltzer, and Marieka Klawitter. 1991. "Estimating Nonresponse and Response Bias: Resident and Nonresident Parents' Reports About Child Support." Sociological Methods and Research 20,1:30-59. Schaeffer, Nora Cate. 1994. "Errors of Experience: Response Errors in Reports About Child Support and Their Implications for Questionnaire Design." In Autobiographical Memory and the Validity of Retrospective Reports, edited by Norbert Schwarz and Seymour Sudman. New York: Springer-Verlag. Threatening or Socially-desirable Topics Abelson, Robert P., Elizabeth F. Loftus, and Anthony G. Greenwald. 1992. "Attempts to Improve the Accuracy of Self-Reports of Voting." Pp. 138-153 in J. Tanur, ed., Questions about Questions: Inquiries into the Cognitive Bases of Surveys. New York: Russell Sage Foundation. Aquilino, William S. 1994. "Interview Mode Effects in Surveys of Drug and Alcohol Use: Experiment." POQ 58,2:210-240. A Field Aquilino, William S. and L.A. Losciuto. 1990. "Effects of Interview Mode on Self-reported Drug Use." POQ 54:362-395. Belli, Robert F., Michael W. Traugott, Margaret Young, and Katherine McGonagle. in press. "Reducing Vote Overreporting in Surveys." Public Opinion Quarterly Blair, Ed, Seymour Sudman, and Norman M. Bradburn. 1977. "How to Ask Questions about Drinking and Sex: Response Effects in Measuring Consumer Behavior." JMR 14:316-321. Bradburn, Norman M., Seymour Sudman, Ed Blair, and Carol Stocking. 1978. "Question Threat and Response Bias." POQ 42:221-234. DeMaio, Theresa. 1984. "Social Desirability and Survey Measurement: A Review." Pp. 257-281 in C. Turner and E. Martin, eds., Surveying Subjective Phenomena, Vol. 2. New York: Russell Sage Foundation. Gfroerer, Joseph, J. Eyerman, and J. Chromy, eds. 2002. Redesigning an Ongoing National Household Survey: Methodological Issues . Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of 8 Applied Studies. Jobe, Jared B., William F. Pratt, Roger Tourangeau, Alison K. Baldwin, and Kenneth A. Rasinski. 1997. "Effects of Interview Mode on Sensitive Questions in a Fertility Survey." Pp. 311-52 in Survey Measurement and Process Quality, edited by L. Lyberg, P. Biemer, M. Collins, E. de Leeuw, C. Dippo, N. Schwarz, and D. Trewin. N.Y.: Wiley-Interscience. Lara, Diana, Jennifer Strickler, Claudia D. Olavarrieta, and Charlotte Ellerston. 2004. "Measuring Induced Abortion in Mexico: A Comparison of Four Methodologies." Sociological Methods and Research 32(4):529-58. Locander, William, Seymour Sudman, and Norman Bradburn. 1976. "An Investigation of Interview Method, Threat and Response Distortion." JASA 71(354):269-275. Presser, Stanley, and Michael Traugott. 1992. "Little White Lies and Social Science Models: Correlated Response Errors in a Panel Study of Voting." POQ 56:77-86. Schaeffer, Nora Cate. 2000. "Asking Questions About Threatening Topics: A Selective Overview." The Science of Self-Report: Implications for Research and Practice, edited by Arthur A. Stone, Jaylan S. Turkkan, Christine A. Bachrach, Jared B. Jobe, Howard S. Kurtzman, and Virginia A. Cain. Mahwah, New Jersey: Lawrence Erlbaum. Tourangeau, Roger and Smith Tom. W. 1998. "Collecting Sensitive Information With Different Modes of Data Collection." Pp. 431-53 in Computer Assisted Survey Information Collection, edited by Mick P. Couper, Reginald P. Baker, Jelke Bethlehem, Cynthia Z. F. Clark, Jean Martin, William L. Nicholls II, and James M. O'Reilly. New York: John Wiley and Sons. Tourangeau, Roger, Kenneth Rasinski, Jared B. Jobe, Smith Tom W., and William F. Pratt. 1997. "Sources of Error in a Survey on Sexual Behaivor." Journal of Official Statistics 13(4):341-65. Turner, Charles F., Judith T. Lessler, and Joseph C. Gfroerer, eds. 1992. Survey Measurement of Drug Use: Methodological Studies. Developed for the National Institute on Drug Abuse, Division of Epidemiology and Prevention Research by the Research Triangle Institute. Washington, DC: U.S. Government Printing Office. Turner, Charles F., Judith T. Lessler, and James Devore. 1992. "Effects of Mode of Administration and Wording on Reporting of Drug Use ." Pp. 177-220 in Survey Measurement of Drug Use, edited by Charles F. Turner, Judith T. Lessler, and Joseph C. Gfroerer. Rockville, MD: National Institute on Drug Abuse, U.S. Department of Health and Human Services. Turner, Charles F., Judith T. Lessler, Barbara J. George, Michael L. Hubbard, and Michael B. Witt. 1992. "Effects of Mode of Administration and Wording on Data Quality." Pp. 221-43 in Survey Measurement of Drug Use, edited by Charles F. Turner, Judith T. Lessler, and Joseph C. Gfroerer. Rockville, MD: National Institute on Drug Abuse, U.S. Department of Health and Human Services. Warner, S. 1965. "Randomized Response: A Survey Technique for Eliminating Evasive Answer Bias." JASA 60:63-69. 9 Improving Reports Croyle, Robert T., and Elizabeth F. Loftus. 1992. "Improving Episodic Memory Performance of Survey Respondents." Pp. 95-101 in J. Tanur, ed., Questions about Questions: Inquiries into the Cognitive Bases of Surveys. New York: Russell Sage Foundation. Eisenhower, Donna, Nancy A. Mathiowetz, and David Morganstein. 1991. "Recall Error: Sources and Bias Reduction Techniques." Pp. 127-143 in Paul Biemer et al., eds., Measurement Errors in Surveys. New York: Wiley. Kessler, Ronald C. and Elaine Wethington. 1991. "The Reliability of Life Event Reports in a Community Survey." Psychological Medicine 21:723-738. Loftus, Elizabeth F., Kyle D. Smith, Mark R.Klinger, and Judith Fiedler. 1992. "Memory and Mismemory for Health Events." Pp. 102-137 in J. Tanur, ed., Questions about Questions: Inquiries into the Cognitive Bases of Surveys. New York: Russell Sage Foundation. Means, Barbara, Gary E. Swan, Jared B. Jobe, and James L. Esposito. 1991. "An Alternative Approach to Obtaining Personal History Data." Pp. 167-184 in P. Biemer, R. Groves, L. Lyberg, N. Mathiowetz, and S. Sudman, eds., Measurement Errors in Surveys. New York: Wiley. Sudman, Seymour, Adam Finn, and Linda Lannon. 1984. "The Use of Bounded Recall Procedures in Single Interviews." POQ 48:520-524. Filter Questions Knauper, Barbel. 1998. "Filter Questions and Question Interpretation: Opinion Quarterly 62:70-78. Presuppositions at Work." Public Schaeffer, Nora Cate and Dykema, Jennifer. 1994. Open and closed questions about the frequency of events: Replication, extension, paradox. Center for Demography and Ecology, Working Paper 94-18; Response Categories: Relative Frequencies (See also "Category Labels" for "Subjective Questions") Bass, Bernard M. 1968. "How to Succeed in Business According to Business Students and Managers." Journal of Applied Psychology 52:254-262. Bass, Bernard M., Wayne F. Cascio, and Edward J. O'Connor. 1974. "Magnitude Estimation of Expressions of Frequency and Amount." Journal of Applied Psychology 59:313-320. Bradburn, Norman, and Carrie Miles. 1979. "Vague Quantifiers." POQ 43:92-101. Chase, Clinton I. 1969. "Often Is Where You Find It." American Psychologist 24:1043. Dennis, Russell, Cathy Dennis, and Larry Jobe. 1975. "The Meaning of Frequency Words in Different Contexts." Language and Speech 18:343-349. 10 Goocher, Buell E. 1965. "Effects of Attitude and Experience on the Selection of Frequency Adverbs." of Verbal Learning and Verbal Behavior 4:193-195. Journal Hakel, Milton. 1969. "How Often Is Often?" American Psychologist 23:533-534. Newstead, Stephen E., and Janet M. Collis. 1987. "Context and the Interpretation of Quantifiers of Frequency." Ergonomics 30,10:1447-1462. Pace, C. Robert, and Jack Friedlander. 1982. "The Meaning of Response Categories: How Often Is `Occasionally,' `Often,' and `Very Often?'" Research in Higher Education 17,3:267-281. Pepper, Susan. 1981. "Problems in the Quantification of Frequency Expressions." Pp. 25-41 in New Directions for Methodology of Social and Behavioral Sciences, No. 9, edited by Donald Fiske. San Francisco: Jossey-Bass. Pepper, Susan, and Lubomir S. Prytulak. 1974. "Sometimes Frequently Means Seldom: Context Effects in the Interpretation of Quantitative Expressions." Journal of Research in Personality 8:95-101. Pohl, Norval F. 1981. "Scale Considerations in Using Vague Quantifiers." Journal of Experimental Education 49,4:235-240. Schaeffer, Nora Cate. 1992. "Hardly Ever or Constantly? POQ 55,3: 395-423. Group Comparisons Using Vague Quantifiers." Schriesheim, Chester A., and Janet F. Schriesheim. 1978. "The Invariance of Anchor Points Obtained by Magnitude Estimation and Pair-Comparison Treatment of Complete Ranks Scaling Procedures: An Empirical Comparison and Implications for Validity of Measurement." Educational and Psychological Measurement 38:977-983. Simpson, Ray H. 1944. "The Specific Meanings of Certain Terms Indicating Differing Degrees of Frequency." The Quarterly Journal of Speech 30:328-330. Wright, Daniel B., George E. Gaskell and Colm A. O'Muircheartaigh. 1994. "How Much is 'Quite a bit'? Mapping Between Numerical Values and Vague Quantifiers." Applied Cognitive Psychology 8:479-496. Wright, Daniel B., George D. Gaskell, and Colm A. O'Muircheartaigh. 1997. "How Response Alternatives Affect Different Kinds of Behavioural Frequency Questions." British Journal of Social Psychology 36:443-56. Response Categories: Absolute Frequencies (including open vs closed questions) Knauper, Barbel. 1998. “Filter Questions and Question Interpretation: Opinion Quarterly. 62:70-78. Presuppositions at Work.” Public Schaeffer, Nora Cate, and Hong-Wen Charng. 1991. "Two Experiments in Simplifying Response Categories: Intensity and Frequency Categories." Sociological Perspectives 34,2:165-182. 11 Schaeffer, Nora Cate, and Jennifer Dykema. 1994. "Open and Closed Questions about the Frequency of Events: Replication, Extension, Paradox." Center for Demography and Ecology Working Paper 94-18, University of Wisconsin, Madison. Schwarz, Norbert, Hans-Jurgen Hippler, Brigitte. Deutsch, and Fritz Strack. 1985. "Response Scales: Effects of Category Range on Reported Behavior and Comparative Judgments." POQ 49:388-395. Bless, H., G. Bohner, T. Hild, and N. Schwarz. 1992. "Asking Difficult Questions: Task Complexity Increases the Impact of Response Alternatives." European Journal of Social Psychology :309-12. Schwarz, Norbert. 1990. "Assessing Frequency Reports of Mundane Behaviors." Pp. 98-119 in Clyde Hendrick and Margaret S. Clark, editors, Research Methods in Personality and Social Psychology, Volume 11, Review of Personality and Social Psychology. Newbury Park: Sage. Schwarz, Norbert, and Hans-Jurgen Hippler. 1992. "Response Alternatives: The Impact of their Choice and Presentation Order." Pp. 41-56 in Paul Biemer, et al., eds. Measurement Errors in Surveys. New York: Wiley. Schwarz, Norbert, Fritz Strack, Gesine Muller, and Brigitte Chassein. 1988. "The Range of Response Alternatives May Determine the Meaning of the Question: Further Evidence on Informative Functions of Response Alternatives." Social Cognition 6:107-117. Schwarz, Norbert, and Hans-J Hippler. 1987. "What Response Scales May Tell Your Respondents: Informative Functions of Response Alternatives." Pp. 163-178 in Social Information Processing and Survey Methodology, edited by Hans-J. Hippler, Norbert Schwarz, and Seymour Sudman. New York: Springer-Verlag. Reference Period Igou, Eric R., Herbert Bless, and Norbert Schwarz. 2002. "Making Sense of Standardized Survey Questions: The Influence of Reference Periods and Their Repetition." Communication Monographs 69(2):179-87. Schaeffer, Nora Cate and Lina Guzman. "Interpreting Reference Periods." Paper presented at the annual meeting of the American Association for Public Opinion Research, 13-16 May, St. Pete Beach, FL. Winkielman, Piotr, Barbel Knauper, and Norbert Schwarz. 1998. "Looking Back at Anger: Reference Periods Change the Interpretation of Emotion Frequency Questions." Journal of Personality and Social Psychology 75(3):719-28. Event History Belli, Robert F., L. Smith, P. Andreski, and S. Agrawal. 2007. "Methodological Comparisons Between CATI Event History Calendar and Conventional Questionnaire Instruments." Public Opinion Quarterly 71(1):603-22. Belli, Robert F., William L. Shay, and Frank P. Stafford. 2001. "Event History Calendars and Question List Surveys: A Direct Comparison of Interviewing Methods." Public Opinion Quarterly 65(1):45-74. 12 Belli, Robert F. 1998. "The Structure of Autobiographical Memory and the Event History Calendar: Potential Improvements in the Quality of Retrospective Reports in Surveys." Memory 6(4):383-406. Freedman, Deborah, Arland Thornton, Donald Camburn, Duane Alwin, and Linda Young-DeMarco. 1988. "The Life History Calendar: A Technique for Collecting Retrospective Data." Pp. 37-68 in Clifford C. Clogg, ed., SM 1988. Washington, D.C.: American Sociological Association. Don’t Know Poe, Gail S., Isadore Seeman, Joseph McLaughlin, Eric Mehl, and Michael Dietz. 1988. "'Don't Know' Boxes in Factual Questions in a Mail Questionnaire." POQ 52:212-222. Sanchez, Maria E. and Giovanna Morchio. 1992. "Probing 'Don't Know' Answers: Effects on Survey Estimates and Variable Relationships." POQ 56,4:454-474. Subjective Phenomena Recall and Related Issues Markus, Gregory B. 1986. "Stability and Change in Political Attitudes: Observed, Recalled, and 'Explained.'" Political Behavior 8,1:21-44. Krosnick, Jon A. 1991. "Response Strategies for Coping with the Cognitive Demands of Attitude Measures in Surveys." Applied Cognitive Psychology 5:213-236. Ross, Michael, and Michael Conway. 1986. "Remembering One's Own Past: The Construction of Personal Histories." Pp. 122-144 in Handbook of Motivation and Cognition: Foundations of Social Behavior, edited by Richard M. Sorrentino and E. Tory Higgens. Chichester, England: Wiley. Smith, Tom W. 1984. "Recalling Attitudes: An Analysis of Retrospective Questions on the 1982 GSS." POQ 48:639-649. Open and Closed Questions Converse, Jean. 1984. "Strong Arguments and Weak Evidence: the 1940's." POQ 48:267-282. The Open/Closed Questioning Controversy of Kane, Emily, and Howard Schuman. 1991. "Open Survey Questions as Measures of Personal Concern with Issues: A Reanalysis of Stouffer's Communism, Conformity, and Civil Liberties." Pp. 81-96 in Peter Marsden, editor, SM 1991. Oxford: Basil. Schuman, Howard, and Stanley Presser. 1981. "Open versus Closed Questions." Pp. 79-112 in Questions and Answers in Attitude Surveys. New York: Academic Press. Schuman, Howard, and Jacqueline Scott. 1987. "Problems in the Use of Survey Questions to Measure 13 Public Opinion." Science 236:957-959. Response Order Effects Blunch, Niels J. 1984. "Position Bias in Multiple-Choice Questions." JMR 21:216-220. Allyson L Holbrook, Jon A Krosnick, David Moore, and Roger Tourangeau. 2007. "RESPONSE ORDER EFFECTS IN DICHOTOMOUS CATEGORICAL QUESTIONS PRESENTED ORALLY: THE IMPACT OF QUESTION AND RESPONDENT ATTRIBUTES." Public Opinion Quarterly 71(3):325. Krosnick, Jon A., and Duane F. Alwin. 1987. "An Evaluation of a Cognitive Theory of Response-Order Effects." POQ 51(2):201-219. McClendon, McKee J. 1986. "Response-order Effects for Dichotomous Questions." Social Science Quarterly 67:205-211. Schwarz, Norbert, Hans-Jurgen Hippler, and Elisabeth Noelle-Neumann. 1992. "A Cognitive Model of Response-Order Effects in Survey Measurement." Pp. 187-201 in Norbert Schwarz and Seymour Sudman, eds., Context Effects in Social and Psychological Research. New York: Springer-Verlag. Question Order Effects Schuman, Howard. 1992. "Context Effects: State of the Past/State of the Art." Pp. 5-22 in Norbert Schwarz and Seymour Sudman, eds., Context Effects in Social and Psychological Research. New York: Springer-Verlag. Tourangeau, Roger. 1992. "Context Effects on Responses to Attitude Question: Attitudes as Memory Structures." Pp. 35-48 in Norbert Schwarz and Seymour Sudman, eds., Context Effects in Social and Psychological Research. New York: Springer-Verlag. Tourangeau, Roger, Kenneth A. Rasinski, and Norman Bradburn. 1991. "Measuring Happiness in Surveys: A Test of the Subtraction Hypothesis." Public Opinion Quarterly 55,2:255-266. Tourangeau, Roger, Kenneth A. Rasinski, Norman Bradburn, and Roy D'Andrade. 1989. "Carryover Effects in Attitude Surveys." Public Opinion Quarterly 53m 4:495-525. Acquiescence and Response Sets Bachman, Jerald G., and Patrick M. O'Malley. 1984. "Yea-Saying, Nay-Saying and Going to Extremes: Black-White Differences in Response Styles." Public Opinion Quarterly 48:491-509. McClendon, McKee. 1991. "Acquiescence and Recency Response-Order Effects in Interview Surveys." SMR 20,1:60-103. 14 Schuman, Howard, and Stanley Presser. 1981. "The Acquiescence Quagmire." Pp. 203-230 in Questions and Answers in Attitude Surveys. New York: Academic Press. Watson, Dorothy. 1992. "Correcting for Acquiescent Response Bias in the Absence of a Balanced Scale: An Application to Class Consciousness." SMR 21,1:52-88. Response Dimensions Diener, Ed, Ed Sandvik, and William Pavot. 1989. "Happiness is the Frequency, Not Intensity, of Positive Versus Negative Affect." Forthcoming in Subjective Well-Being, edited by Fritz Strack, M. Argyle, and Norbert Schwarz. New York: Pergamon. Schaeffer, Nora Cate. 1989. "The Frequency and Intensity of Parental Conflict: Dimensions." Journal of Marriage and the Family, 51,3 (August):759-766. Choosing Response Rankings Alwin, Duane, and Jon Krosnick. 1985. "The Measurement of Values in Surveys: Ratings and Rankings." POQ 49:535-552. A Comparison of Magnitude Estimation Lodge, Milton. 1981. Magnitude Scaling: Quantitative Measurement of Opinions. Sage University Paper series on Quantitative Applications in the Social Sciences, 07-025. Beverly Hills: Sage. Lodge, Milton, David Cross, Bernard Tursky, Joseph Tannenhaus, and Richard Reeder. 1976. "The Psychophysical Scaling of Political Support in the `Real World.' " Political Methodology 3:159-182. Schaeffer, Nora Cate, and Norman M. Bradburn. 1989. "Respondent Behavior in Magnitude Estimation." Journal of the American Statistical Association 84,406:402-413. Wegener, Bernd, ed. 1982. Social Attitudes and Psychophysical Measurement. Hillsdale, N.J.: Lawrence Erlbaum. Question Balance Bishop, George F., R.W. Oldendick, and A.J. Tuchfarber. 1982. "Effects of Presenting One versus Two Sides of an Issue in Survey Questions." POQ 46:69-85. Number of Categories Alwin, Duane F. 1997. "Feeling Thermometers Versus 7-Point Scales: Which Are Better?" Sociological Methods & Research 25(3):318-40. 15 Alwin, Duane. 1992. "Information Transmission in the Survey Interview: Number of Response Categories and the Reliability of Attitude Measurement." Pp. 83-118 in Sociological Methodology, edited by Peter Marsden. Washington, DC: American Statistical Association. Cox, Eli P. 1980. "The Optimal Number of Response Alternatives for a Scale: A Review." 17:407-422. JMR Givon, Moshe M., and Zur Shapira. 1984. "Response to Rating Scales: A Theoretical Model and Its Application to the Number of Categories Problem." JMR 21:410-419. Komorita, S.S., and William K. Graham. 1965. "Number of Scale Points and the Reliability of Scales." Educational and Psychological Measurement 25,4:987-995. Matell, Michael S., and Jacob Jacoby. 1971. "Is There an Optimal Number of Alternatives for Likert Scale Items? Study I: Reliability and Validity." Educational and Psychological Measurement 31:657-674. Miller, George A. 1956. "The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information." The Psychological Review 63,2:81-97. Category Labels Alwin, Duane F., and Jon A. Krosnick. 1991. "The Reliability of Survey Attitude Measurement: Influence of Question and Respondent Attributes." SMR 20,1: 139-181. The Bartram, Peter, and David Yelding. 1973. "The Development of an Empirical Method of Selecting Phrases Used in Verbal Rating Scales: A Report on a Recent Experiment." Journal of the Market Research Society 15,3:151-156. Bendig, A. W., and J. B. Hughes II. 1953. "Effect of Verbal Anchoring and Number of Rating-Scale Categories Upon Transmitted Information." Journal of Experimental Psychology. 46:87-90. Cliff, Norman. 1959. "Adverbs as Multipliers." Psychological Review 66:27-44. Dixon, Paul N., Mackie Bobo, and Richard A. Stevick. 1984. "Response Differences and Preferences for All-Category-Defined and End-Defined Likert Formats." Educational and Psychological Measurement 44:61-66. Dobson, Keith S., and Kerry J. Mothersill. 1979. "Equidistant Category Labels for Construction of Likert-Type Scales." Perceptual and Motor Skills 49:575-580. Jones, Lyle V., and L. L. Thurstone. 1955. "The Psychophysics of Semantics: An Experimental Investigation." The Journal of Applied Psychology 39:31-36 Klockars, A. J. and M. Yamagishi. 1988. "The Influence of Labels and Positions in Rating Scales." Journal of Educational Measurement 25:85-96. 16 Krosnick, Jon A. and Matthew A. Berent. 1993. "Comparisons of Party Identification and Policy Preferences: The Impact of Survey Question Format." American Journal of Political Science 37,3 (August):941-964. Krosnick, Jon A. and Leandre R. Fabrigar. 1997. "Designing Rating Scales for Effective Measurement in Surveys." Pp. 141-64 in Survey Measurement and Process Quality, edited by L. Lyberg, P. Biemer, M. Collins, E. de Leeuw, C. Dippo, N. Schwarz, and D. Trewin. N.Y.: Wiley-Interscience. Lichtenstein, Sarah, and Robert Newman. 1967. "Empirical Scaling of Common Verbal Phrases Associated with Numerical Probabilities." Psychonomic Science 9,10:562-564. Mittelstaedt, Robert A. 1971. "Semantic Properties of Selected Evaluative Adjectives." Mosier, Charles I. 1941. "A Psychometric Study of Meaning." JMR 8:236-7. The Journal of Social Psychology 13:123-140. Murphy, Kevin R., and Joseph I. Constans. 1987. "Behavioral Anchors as a Source of Bias in Ratings." Journal of Applied Psychology 72:573-577. O'Muircheartaigh, Colm A., George Gaskell, and Daniel B. Wright. 1995. "Weighing Anchors: Verbal and Numeric Labels for Response Scales." Journal of Official Statistics 11(3):295-307. O'Muircheartaigh, Colm A., George D. Gaskell, and Daniel B. Wright. Frequency Questions." Public Opinion Quarterly 57(4):552-65. 1993. "Intensifiers in Behavioral Schaeffer, Nora Cate, and Hong-Wen Charng. 1991. "Two Experiments in Simplifying Response Categories: Intensity and Frequency Categories." Sociological Perspectives 34,2:165-182. Scherpenzeel, Annette C. and Saris Willem E. 1997. "The Validity and Reliablity of Survey Questions: A Meta-Analysis of MTMM Studies." Sociological Methods and Research 25(3):341-83. Spector, Paul E. 1976. "Choosing Response Categories for Summated Rating Scales." Journal of Applied Psychology 61:374-375. Wilson, Barbara F., Barbara M. Altman, Karen Whitaker, and Mario Callegaro. 2004. "How Good Is Good? Comparing Numerical Ratings of Response Options for Two Versions of the Self-Assessed Health Status Question." Journal of Official Statistics 20(2):379-92. Wright, Daniel B., George D. Gaskell, and Colm A. O'Muircheartaigh. 1997. "How Response Alternatives Affect Different Kinds of Behavioural Frequency Questions." British Journal of Social Psychology 36:443-56. Middle Categories Edwards, Allen L. 1946. "A Critique of 'Neutral' Items in Attitude Scales Constructed by the Method of Equal Appearing Intervals." Psychological Bulletin 53:159-169. Guy, Rebecca F., and Melissa Norvell. 1977. "The Neutral Point on a Likert Scale." The Journal of 17 Psychology 95:199-204. Kaplan, Kalman J. 1972. "On the Ambivalence-Indifference Problem in Attitude Theory and Measurement: A Suggested Modification of the Semantic Differential Technique." Psychological Bulletin 77,5:361-372. Klopfer, Frederick J., and Theodore M. Madden. 1980. "The Middlemost Choice on Attitude Items: Ambivalence, Neutrality, or Uncertainty?" Personality and Social Psychology Bulletin 6:91-101. O'Muricheartaigh, Colm, Krosnick, Jon, and Helic, Armin. 1999. Middle alternatives, acquiescence, and the quality of questionnaire data. Paper presented at the annual meetings of the American Association for Public Opinion Research, May, St. Pete Beach, FL. Schuman, Howard, and Stanley Presser. 1981. "Measuring a Middle Position." Pp. 161-178 in Questions and Answers in Attitude Surveys. New York: Academic Press. Filters and Don’t Know Bishop, George F., Robert W. Oldendick, and Alfred Tuchfarber. 1984. "What Must My Interest in Politics Be If I Just Told You 'I Don't Know'?" POQ 48:510-524. Coombs, Clyde H., and Lolagene C. Coombs. 1976. "'Don't Know: Item Ambiguity or Respondent Uncertainty?" POQ 40:497-514. Duncan, Otis Dudley, and Magnus Stenbeck. 1988. "No Opinion or Not Sure?" POQ 52:513-525. Krosnick, Jon A., Allyson L. Holbrook, Matthew K. Berent, Richard T. Carson, W. M. Hanneman, Raymond J. Kopp, Robert C. Mitchell, Stanley Presser, Paul A. Ruud, V. K. Smith, Wendy R. Moody, Melanie C. Green, and Michael Conaway. 2002. "The Impact of "No Opinion" Response Options on Data Quality: Non-Attitude Reduction or an Invitation to Satisfice?" Public Opinion Quarterly 66(3):371-403 Leigh, James H., and Claude R. Martin, Jr. 1987. "'Don't Know' Item Nonresponse in a Telephone Survey: Effects of Question Form and Respondent Characteristics." JMR 24:418-24. McClendon, McKee J. and Duane Alwin. 1993. "No-Opinion Filters and Attitude Measurement Reliability." SMR 21,4:438-464 Sterngold, Arthur; Warland, Rex H., and Herrmann, Robert O. Do surveys overstate public concern? Public Opinion Quarterly. 1999; 58(2):255-263. Schuman, Howard, and Stanley Presser. 1981. "The Assessment of No Opinion." Pp. 113-146 in Questions and Answers in Attitude Surveys. New York: Academic Press. Social Characteristics Kormendi, Eszter. !988. "The Quality of Income Information in Self-Administered and Telephone Surveys." Pp. 321-340 in R. Groves et al., ed., Telephone Survey Methodology. New York: Wiley. 18 Martin, Elizabeth, Theresa DeMaio, Pamela Campanelli. Race and Hispanic Origin." POQ 54:551-566. 1990. "Context Effects for Census Measures of Moore, Jeffrey C., Linda L. Stinson, and Jr. E. J. Welniak. 2000. "Income Measurement Error Is Surveys: Review." Journal of Official Statistics 16(4):331-62. Smith, Tom W. 1992. "Changing Racial Labels: 'African-American.'" POQ 56,4: 496-514. A From 'Colored' to 'Negro' to 'Black' to Sudman, Seymour, and Norman M. Bradburn. 1982. "Using Standard Demographic Terms." Pp. 174-206 in Asking Questions: A Practical Guide to Questionnaire Design. San Francisco: Jossey-Bass. Mode of Administration Comparisons (not elsewhere classified) DeLeeuw, Edith, and Johannes van der Zouwen. 1988. "Data Quality in Telephone and Face-to-Face Surveys: A Comparative Meta-analysis." Pp. 283-300 in R. Groves, et al., eds., Telephone Survey Methodology New York: Wiley. Groves, Robert. 1978. "On the Mode of Administering a Questionnaire and Responses to Open-ended Items." Social Science Research 7:257-71. Groves, Robert. 1979. "Actors and Questions in Telephone and Personal Interview Surveys." 43:190-205. POQ Jordan, Lawrence A., Alfred C. Marcus, and Leo G. Reeder. 1980. "Response Styles in Telephone and Household Interviewing: A Field Experiment." POQ 44,2:210-222. Miller, Peter V. 1984. "Alternative Question Forms for Attitude Scale Questions." Quarterly 48:766-778. Public Opinion Sykes, Wendy and Collins Martin. 1988. "Effects of Mode of Interview: Experiments in the UK." Pp. 301-20 in Telephone Survey Methodology, edited by Robert M. Groves, Paul P. Biemer, Lars E. Lyberg, James T. Massey, William L. Nicholls II, and Joseph Waksberg. New York: Wiley. Computer Assisted Methods House, Carol C., and William L. Nicholls II. 1988. "Questionnaire Design for CATI: Design Objectives and Methods." Pp. 421-436 in Robert M. Groves et al., editors, Telephone Survey Methodology. New York: Wiley. Weeks, Michael. 1992. "Computer-Assisted Survey Information Collection: and Their Implications for Survey Operations." JOS 8(4) 445-465. A Review of CASIC Methods Proxy Respondents Blair, Johnny, Geeta Menon, and Barbara Bickart. 1991. "Measurement Effects in Self vs. Proxy Response 19 to Survey Questions: An Information-Processing Perspective." Pp. 145-166 in Measurement Errors in Surveys, P. Biemer et al., eds. New York: Wiley. Mathiowetz, Nancy, and Robert M. Groves. 1985. "The Effects of Respondent Rules on Health Survey Reports." American Journal of Public Health 75(6):639-644. Moore, J. 1988. "Self-Proxy Response Status and Survey Response Quality: JOS 4(2) 155-172. A Review of the Literature." Self-Administered Paper and Web Instruments Christian, Leah M., Don A. Dillman, and Jolene D. Smyth. 2007. "Helping Respondents Get It Right the First Time: the Influence of Words, Symbols, and Graphics in Web Surveys." Public Opinion Quarterly 71(1):113. Dillman, Don A., Jon R. Clark, and James B. Treat. 1994. "Influence of 13 Design Factors on Completion Rates to Decennial Census Questionnaires." Paper presented at the Annual Research Conference of the U.S. Bureau of the Census. Arlington, VA, March 1994. Dillman, Don A., Michael D. Sinclair, and Jon R. Clark. 1993. "Effects of Questionnaire Length, Respondent-Friendly Design, and a Difficult Question on Response Rates for Occupant-Addressed Census Mail Surveys. " POQ 57:289-304. Dillman, Don A., Eleanor Singer, Jon R. Clark, and James B. Treat. 1996. "Effects of Benefits Appleals, Mandatory Appeals, and Variations in Statement of Confidentiality on Completion Rates for Census Questionnaires." POQ 60,3 (Fall): 376-389. Faas, Thorsten and Harald Schoen. 2006. "Putting a Questionnaire on the Web Is Not Enough - A Comparison of Online and Offline Surveys Conducted in the Context of the German Federal Election 2002." Journal of Official Statistics 22(2):177-90. Peytchev, Andy, Mick P. Couper, Sean E. McCabe, and Scott D. Crawford. 2006. "Web Survey Design: Paging Versus Scrolling." Public Opinion Quarterly 70(4):596-607. Jenkins, Cleo R. and Don A. Dillman. 1997. "Toward a Theory of Self-Administered Questionnaire Design." Pp. 165-198 in Lars Lyberg et al., eds., Survey Measurement and Process Quality. New York: John Wiley and Sons. Smyth, Jolene D., Dillman, Don A., Christian, Leah Melani, and McBride, Mallory. 2006. "Open-Ended Questions in Web Surveys: Can Increasing the Size of Answer-Boxes and Providing Extra Verbal Instructions Improve Response Quality?" American Association for Public Opinion Research, Montreal, Quebec, Canada. Smyth, Jolene D., Don A. Dillman, Leah M. Christian, and Michael J. Stern. 2006. "Comparing Check-All and Forced Choice Question Formats in Web Surveys." Public Opinion Quarterly 70(1):66-77. Thomas, Randall K. and Jonathan D. Klein. 2006. "Merely Incidental?: Effects of Response Format on Self-Reported Behavior." Journal of Official Statistics 22(2):221-44. 20 Tourangeau, Roger, Mick P. Couper, and Frederick Conrad. 2004. "Spacing, Position, and Order: Interpretive Heuristics for Visual Features of Survey Questions." Public Opinion Quarterly 68(3):368-93. Tourangeau, Roger, Mick P. Couper, and Frederick Conrad. 2007. "Color, Labels, and Interpretive Heuristics for Response Scales." Public Opinion Quarterly 71:91-112. Walston, Jill T., Robert W. Lissitz, and Lawrence M. Rudner. 2006. "The Influence of Web-Based Questionnaire Presentation Variations on Survey Cooperation and Perceptions of Survey Quality." Journal of Official Statistics 22(2):271-91. Question Development and Testing Beatty, Paul C. and Gordon B. Willis. 2007. "Research Synthesis: The Practice of Cognitive Interviewing." Public Opin Q 71(2):287-311. Bischoping, Katherine. 1989. "An Evaluation of Interviewer Debriefing in Survey Pretests." Pp. 15-29 in Charles Cannell, Lois Oksenberg, Graham Kalton, Katherine Bischoping, and Floyd Fowler, New Techniques for Pretesting Survey Questions: Final Report. Report supported by Grant Number HS 05616 from the National Center for Health Services Research and Health Care Technology Assessment. Brenner, Michael. 1985. "Intensive Interviewing." Pp. 147-162 in Michael Brenner, Jennifer Brown, and David Canter, editors, The Research Interview: Uses and Approaches. London: Academic. Calder, Bobby J. 1977. "Focus Groups and the Nature of Qualitative Marketing Research." Marketing Research. 14: 353-364. Journal of Cannell, Charles F., Lois Oksenberg, Graham Kalton, Katherine Bischoping, and Floyd J. Fowler. 1989. New Techniques for Pretesting Survey Questions. Final Report, Grant Number HS 05616, National Center for Health Services Research and Health Care Technology Assessment. Copeland, Kennon R. and Jennifer M. Rothgeb. 1990. Testing Alternative Questionnaires for the Current Population Survey. Pp. 63-71 in the American Statistical Association 1990 Proceedings of the Section on Survey Methods Research. Alexandria, VA: American Statistical Association. DeMaio, Theresa, Nancy A. Mathiowetz, Jennifer Rothgeb, Mary E. Beach, and Sharon Durant. 1993. Protocol for Pretesting Demographic Surveys at the Census Bureau. Washington D.C.: Bureau of the Census. DeMaio, Theresa. 1983. Approaches to Developing Questionnaires. Statistical Policy Working Paper 10. Washington, D.C.: U.S. Government Printing Office. DeMaio, Theresa J. and Jennifer M. Rothgeb. 1996. "Cognitive Interviewing Techniques: In the Lab and in the Field." Pp. 177-196 in Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research, edited by Norbert Schwarz and Seymour Sudman. San Francisco, CA: Jossey-Bass Publishers. 21 Desvousges, William H., and James H. Frey. 1989. "Integrating Focus Groups and Surveys: from Environmental Risk Studies. Journal of Official Statistics 5,4:349-363. Examples Dornburg, Courtney C. and Mark A. McDaniel. 2006. "The Cognitive Interview Enhances Long-Term Free Recall of Older Adults." Psychology and Aging 21(1):196-200. Forsyth, Barbara, Judith T. Lessler, and Michael Hubbard. 1992. "Cognitive Evaluation of the Questionnaire." Pp. 13-52 in Survey Measurement of Drug Use, edited by Charles F. Turner, Judith T. Lessler, and Joseph C. Gfroerer. Rockville, MD: National Institute on Drug Abuse, U.S. Department of Health and Human Services. Fowler, Floyd J. 1992. "How Unclear Terms Affect Survey Data." POQ 56:218-231. Fowler Jr., Floyd Jackson and Charles F. Cannell. 1996. "Using Behavioral Coding to Identify Cognitive Problems with Survey Questions." Pp. 15-36 in Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research, edited by Norbert Schwarz and Seymour Sudman. San Francisco, CA: Jossey-Bass Publishers. Goldman, Alfred, and Susan S. McDonald. Prentice-Hall. 1987. The Group-Depth Interview. Englewood Cliffs, N.J.: Hess, Jennifer, Eleanor Singer, and John Bushery. 1999. “Predicting Test-Retest Reliability from Behavior Coding.” International Journal of Public Opinion Research. 11,4: 346-360. Hess, Jennifer and Eleanor Singer. 1995. “The Role of the Respondent in Questionnaire Debriefing.” Paper presented at the annual meetings of the American Association for Public Opinion Research, May, Ft. Lauderdale, FL. Hunt, Shelby D., Richard D. Sparkman, Jr., and James B. Wilcox. 1982. "The Pretest in Survey Research: Issues and Preliminary Findings." JMR 19: 269-273. Joseph, Jill G., Carol-Ann Emmons, Ronald C. Kessler, Camille B. Wortman, Kerth O'Brien, William T. Hocker, and Catherine Schaefer. 1984. "Coping with the Threat of AIDS." American Psychologist 39(11):1297-1302. Lessler, Judith T. and Barbara H. Forsyth. 1996. "A Coding System for Appraising Questionnaires." Pp. 259-292 in Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research, edited by Norbert Schwarz and Seymour Sudman. San Francisco, CA: Jossey-Bass Publishers. McCracken, Grant. 1988. The Long Interview. Newbury Park: Sage. McKennell, Aubrey. 1977. "Attitude Scale Construction." Pp. 183-220 in The Analysis of Survey Data, Volume I, Exploring Data Structures, edited by Colm O'Muircheartaigh and Clive Payne. New York: Wiley. Merton, Robert K., Marjorie Fiske, Patricia Kendall. 1990. Glencoe, Ill.: The Free Press. The Focused Interview, second edition. Morgan, David L. 1996. "Focus Groups." Annual Review of Sociology 22:129-152. 22 Morgan, David, and Margaret Spanish. Qualitative Sociology 7(3): 253-270. 1984. "Focus Groups: A New Tool for Qualitative Research. Morton-Williams, Jean. 1978. "Unstructured Design Work." Pp. 10-26 in Gerald Hoinville and Roger Jowell, Survey Research Practice. London: Heinemann Educational Books. Morton-Williams, Jean. 1979. "The Use of 'Verbal Interaction Coding' for Evaluating a Questionnaire." Quality and Quantity 13:59-75. Morton-Williams, Jean, and Wendy Sykes. 1984. "The Use of Interaction Coding and Follow-up Interviews to Investigate Comprehension of Survey Questions." Journal of the Market Research Society 26:109-127. Nelson, Dawn D. 1985. "Informal Testing as a Means of Questionnaire Development." Statistics 1(2):79-188. Journal of Official Ongena, Yfke P. and Wil Dijkstra. 2006. "Methods of Behavior Coding of Survey Interviews." Journal of Official Statistics 22(3):419-51. Oksenberg, Lois, Charles F. Cannell, and Graham Kalton. 1991. "New Strategies for Pretesting Survey Questions." Journal of Official Statistics 7(3):349-365 Oksenberg, Lois, Charles Cannell, and Graham Kalton. 1989. "New Methods for Pretesting Survey Questionnaires." Pp. 30-62 in Charles Cannell, Lois Oksenberg, Graham Kalton, Katherine Bischoping, and Floyd Fowler, New Techniques for Pretesting Survey Questions: Final Report. Report supported by Grant Number HS 05616 from the National Center for Health Services Research and Health Care Technology Assessment. Polivka, Anne E. and Jennifer M. Rothgeb. 1993. "Redesigning the CPS Questionnaire." Monthly Labor Review 116, 9:10-28. Presser, Stanley, and Johnny Blair. 1994. "Survey Pretesting: Do Different Methods Produce Different Results?" Pp. 73-104 in Sociological Methodology 1994, Peter V. Marsden, editor. Oxford: Basil Blackwell. Rothgeb, Jennifer, Anne E. Polivka, Kathleen P. Creighton, Sharon Cohany. 1991. "Development of the Proposed Revised Current Population Survey." Proceedings of the Section on Survey Research Methods of the American Statistical Association. Schaeffer, Nora Cate, and Elizabeth Thomson. 1992. "The Discovery of Grounded Uncertainty: Developing Standardized Questions about Strength of Fertility Motivation." Pp. 37-82 in Sociological Methodology 1992, edited by Peter Marsden. Washington, D.C.: American Statistical Association. Schuman, Howard. 1966. "The Random Probe: A Technique for Evaluating the Validity of Closed Questions." ASR 31:218-222. Spradley, James P. 1979. The Ethnographic Interview. New York: Holt, Rinehart, and Winston. 23 Sudman, Seymour, Richard Warneke, Timothy Johnson, Diane O'Rourke, Andrew Davis. 1994. Cognitive Aspects of Reporting Cancer Prevention Examinations and Tests. Series 6, No. 7, National Center for Health Statistics. Turner, Charles F., Judith T. Lessler, Barbara George, Michael Hubbard, and Michael Witt. 1992. "Effects of Decomposition of Complex Concepts." Pp. 221-44 in Survey Measurement of Drug Use . eds. ed., edited by Charles F. Turner, Judith T. Lessler, and Joseph C. Gfroerer. Rockville, MD: National Institute on Drug Abuse, U.S. Department of Health and Human Services. Weller, Susan C., and A. Kimball Romney. 1988. Methods, Vol. 10. Newbury Park, CA: Sage. Systematic Data Collection. Qualitative Research Willis, Gordon B., Patricia Royston, and Deborah Bercini. 1991. "The Use of Verbal Report Methods in the Development and Testing of Survey Questionnaires." Applied Cognitive Psychology 5:251-67. Interviewing Cannell, Charles F., Peter Miller, and Lois Oksenberg. 1981. "Research on Interviewing Techniques." 1981:389-437. San Francisco: Jossey-Bass. Collins, Martin. 1980. "Interviewer Variability: Research Society 22,2:77-95. A Review of the Problem." SM Journal of the Market Conrad, Frederick G. and Michael F. Schober. 2000. "Clarifying Question Meaning in a Household Telephone Survey." Public Opinion Quarterly 64:1-28. Djikstra, Wil. 1987. "Interviewing Style and Respondent Behavior: An Experimental Study of the Survey Interview." SMR 16:309-334. Dijkstra, Wil and Yfke P. Ongena. 2006. "Question-Answer Sequences in Survey Interviews." Quality and Quantity 40:983-1011. Mangione, Thomas W., Floyd J. Fowler, and Thomas A. Louis. Interviewer Effects." JOS 8,3: 293-308. 1992. "Question Characteristics and Maynard, Douglas W. and Nora Cate Schaeffer. 2006. "Standardization-in-Interaction: The Survey Interview." Pp. 9-27 in Talk and Interaction in Social Research Methods , edited by Paul Drew, Geoffrey Raymond, and Darin Weinberg. London: Sage. Maynard, Douglas W. and Nora Cate Schaeffer. 2002. "Standardization and Its Discontents: Standardization, Interaction, and the Survey Interview." Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview, edited by Douglas W. Maynard, Hanneke Houtkoop-Steenstra, Johannes van der Zouwen, and Nora Cate Schaeffer. New York: Wiley. Schaeffer, Nora Cate. 1991. "Conversation with a Purpose or Conversation? Interaction in the Standardized Interview." Pp. 367-391 in Measurement Errors in Surveys, edited by Paul P. Biemer, Robert M. 24 Groves, Lars E. Lyberg, Nancy A. Mathiowetz, and Seymour Sudman. New York: Wiley. Schaeffer, Nora Cate and Douglas W. Maynard. 1996. "From Paradigm to Prototype and Back Again: Interactive Aspects of Cognitive Processing in Standardized Survey Interviews." Pp. 65-88 in Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research, edited by Norbert Schwarz and Seymour Sudman. San Francisco, CA: Jossey-Bass Publishers. Schaeffer, Nora Cate and Douglas W. Maynard. 2008. "The Contemporary Standardized Survey Interview for Social Research." Pp. 31-57 in Envisioning the Survey Interview of the Future, edited by Frederick G. Conrad and Michael J. Schober. Hoboken, NJ: Wiley. Schober, Michael F. and Frederick G. Conrad.1997. "Does Conversational Interviewing Reduce Survey Measurement Error?" Public Opinion Quarterly 61:576-602. Schober, Michael F., Frederick G. Conrad, and Scott S. Fricker. 2000. "When and How Should Survey Interviewers Clarify Question Meaning?" Proceedings of the American Statistical Association Section on Survey Research Methods ( ). Alexandria, VA: American Statistical Association. Suchman, Lucy, and Brigitte Jordan. 1990. "Interactional Troubles in Face-to-Face Survey Interviews." JASA 85(409):232-241. (Also, comments and rejoinder.) Questionnaire Review and Formatting Jabine, Thomas B. 1985. "Flow Charts: Questionnaires." JOS 1,2:189-207. A Tool for Developing and Understanding Survey Sanchez, Maria E. 1992. "Effect of Questionnaire Design on the Quality of Survey Data." 206-217. POQ 56,2: Sudman, Seymour, and Norman Bradburn. 1982. "Order of the Questionnaire" and "Format of the Questionnaire." Pp.207-260 in Asking Questions. San Francisco: Jossey-Bass.