Survey Development Tips
Here are a few key things to remember when developing surveys (Adapted from Dillman, D. A., Smyth,
J.D. and Christian, L. M. (2008). Mail, Internet, and Mixed-Mode Surveys: The Tailored Design Method, 3rd
Ed. New York: John Wiley and Sons.):
1) Ask exactly as many questions as you need to learn what you want to know. Surveys that are too
long lead respondents to quit midway through or give hurried answers. Also, only ask as many
questions as you wish to analyze later.
2) Keep the question format the same for similar questions. People get attuned to patterns and if
you change the patterns, people might not answer the questions correctly.
3) Unless there is an excellent reason to put demographic questions at the beginning of your
survey (for instance, for branching questions), they should be put at the end. Some people will
not answer demographic questions and if those questions are up front, they might not answer
any of your questions.
4) When asking for demographic status, only ask questions that are pertinent to your evaluation
needs. For instance, if you survey faculty members who attended a workshop you hosted, you
might want to know what department they are in (e.g., education, chemistry), and how long
they have been teaching, but it is not relevant to know their religion.
5) Ask questions that will help you identify patterns in the responses. For instance, if you are
surveying students, you might ask when they entered the university to learn if students in
different class years give different responses.
6) Be sure that closed-ended response options are mutually exclusive. For instance, age categories
might be: <18, 18-34, 35-54, 55+.
7) When developing closed-ended questions with a scaled response, make sure the scale is
balanced around an actual or theoretical middle such as: “strongly disagree, mildly disagree,
neither agree nor disagree, mildly agree, strongly agree.” “Not applicable” or “don’t know”
should not be within the scale as a midpoint, but rather at the end and not considered as part of
any scale.
8) Be sure your questions are asked in a way that it is clear what the respondent is expected to do
and what type of answer is expected. A question like “What is the best thing about going to
school here?” might yield answers about weekend parties when what you really want to know is
“What is the most important factor in your choice of a major?”
9) Be cautious when using “check all that apply” questions, particularly if the list is long. With a
check all that apply format, respondents are more likely to check options at the top of the list
than at the bottom or to decide after awhile that they have checked “enough” of the response
options. Instead, ask each question with a yes or no option next to each response choice.
10) Ask questions that don’t require too much thought or calculation on the part of the respondent.
A question like “How many visits did you make to the library this past semester?” requires too
much work (hopefully) for respondents to spend the effort to give you an accurate answer. If
you want to know how many times people do something, use a frame of reference (e.g., last
week, last month) that is appropriate for what you want to know and easy enough for the
respondents to calculate the right response quickly.
11) Avoid “double barreled” questions that ask two questions which may have different answers but
allow only one answer choice. For instance “Please rate the quality of the food and service in the
cafeteria on a scale of 1 to 5 where 1 is poor and 5 is excellent” is a double barreled question
where the food may be excellent and the service poor or vice-versa, but the respondent is not
given the opportunity to separate these features.
12) Avoid leading questions such as “Since we all know that the department has lost its best
instructors to other institutions in the past few years, how would you rate the quality of
teaching in the department?”i
Material adapted from: Brawner, C., Litzler, E., and Slattery, M. Evaluation As A Tool For Improvement: Building
Evaluation Capacity in the forthcoming L. Barker and J. M. Cohoon eds. Reforming Computer Science at the
Undergraduate Level: NCWIT Extension Services. Funded by the National Center for Women and Information
Technology and NSF HRD-1203148/1203198/1203174/1203179.