Do’s and Don’ts of
Good Survey Question Design
In a world where there are surveys everywhere it's easy to think writing a survey would be a cinch, but writing a good
survey question can be daunting. A good survey question is one that your respondents understand and are able to
answer accurately. Here are the rules of good survey question design.
Don’ts:
1. Don't ask double barreled questions
Do you like rap or country music?
a. Yes
b. No
Explanation: A question is double barreled if it asks two questions at the same time. What if a respondent likes country,
but not rap? What if you later want to compare rap fans with country fans? You can’t do this if you ask more than one
question at a time. The solution here is to split up the double barreled question into two separate questions.
2. Don't use unclear scales
How often do you act a fool?
a. Frequently
c. A lot
d. Sometimes
e. Occasionally
f. Never
Explanation: A scale is a list of response options that have a meaningful order. So if you ask someone to rate their
happiness from 1 to 10 you are using a scale. The scale above is unclear because what is the difference between
“frequently” and “a lot”? Or for that matter, what’s the difference between “sometimes” and “occasionally”? If you ask 10
people you might get 10 different answers. A good survey question asks questions that anyone can understand the same
way. Instead of the scale used above we could use, “always, sometimes, or never” and we would do away with all the
confusion.
3. Don’t use scales that overlap
How many times a week do you call your mother?
a. 1-2 times a week
b. 2-3 times a week
c. 3 or more times a week
Explanation: What would a respondent answer if they call there mom exactly 2 times a week? If your scales overlap you
will baffle your respondents and collect bad data.
4. Don’t ask questions that can’t be answered
What was the exact date of your first kiss?
Explanation: If you ask questions that are too specific or too obscure your respondents will have no way of answering it
accurately. If you want to know when someone had their first kiss or something equally memorable, ask them what year it
happened. You could also ask what year in school were you when you were first kissed. This is something that many
people could answer accurately.
5. Don’t ask leading questions
How much do you like Justin Bieber?
a. I love him
b. Words can not describe how much
c. I would give my first born to touch him
d. I tattooed his face on my body
Explanation: Good survey questions use language that is as unbiased as possible. Asking “how much do you like...”
starts with the assumption that you, at the very least, like it a little bit. A much better question would ask a respondent to
rate the opinion of Justin Bieber on a scale of 1 to 10 where 10 equals “you like him as much as possible” and 1 equals
“you don’t like him at all”
Do’s:
1. Do use simple clear language
In general you should write questions that an 8th grader could understand. Avoid buzz words, jargon, and words
that are not commonly used when writing survey questions. If you have to use potentially confusing words follow
it up with an explanation, “and by cohabiting I mean you have lived with another non-relative adult in a romantic
relationship.”
2. Do think about social desirability
If you ask the question, “Do you use heroin?” on a survey you’ll find that almost everyone says no. If you ask,
“Are you a good parent?” everyone says that they are. However, we know that some people bang heroin and
some people are crummy parents. What’s going on here is what social scientists call social desirability. Survey
respondents are far more likely to lie if the question asked has an answer that is widely thought of as good or
desirable. So if you want to ask controversial questions you need to keep this in mind. Lastly, if you have to ask
a controversial question ask it as close to the end of the survey as possible. That way if your respondent decides
to stop taking the survey when they read the controversial question you will have as much data collected as
possible.
3. Do allow respondents to not answer
You have to allow your respondent to either opt out of answering a question or to say “Don’t Know”. For multiple
choice questions all you need to do is include a response option called “Don’t Know” as well as one that says
“Refuse” or “Would rather not answer”.