Do’s and Don’ts of Good Survey Question Design In a world where there are surveys everywhere it's easy to think writing a survey would be a cinch, but writing a good survey question can be daunting. A good survey question is one that your respondents understand and are able to answer accurately. Here are the rules of good survey question design. Don’ts: 1. Don't ask double barreled questions Do you like rap or country music? a. Yes b. No Explanation: A question is double barreled if it asks two questions at the same time. What if a respondent likes country, but not rap? What if you later want to compare rap fans with country fans? You can’t do this if you ask more than one question at a time. The solution here is to split up the double barreled question into two separate questions. 2. Don't use unclear scales How often do you act a fool? a. Frequently c. A lot d. Sometimes e. Occasionally f. Never Explanation: A scale is a list of response options that have a meaningful order. So if you ask someone to rate their happiness from 1 to 10 you are using a scale. The scale above is unclear because what is the difference between “frequently” and “a lot”? Or for that matter, what’s the difference between “sometimes” and “occasionally”? If you ask 10 people you might get 10 different answers. A good survey question asks questions that anyone can understand the same way. Instead of the scale used above we could use, “always, sometimes, or never” and we would do away with all the confusion. 3. Don’t use scales that overlap How many times a week do you call your mother? a. 1-2 times a week b. 2-3 times a week c. 3 or more times a week Explanation: What would a respondent answer if they call there mom exactly 2 times a week? If your scales overlap you will baffle your respondents and collect bad data. 4. Don’t ask questions that can’t be answered What was the exact date of your first kiss? Explanation: If you ask questions that are too specific or too obscure your respondents will have no way of answering it accurately. If you want to know when someone had their first kiss or something equally memorable, ask them what year it happened. You could also ask what year in school were you when you were first kissed. This is something that many people could answer accurately. 5. Don’t ask leading questions How much do you like Justin Bieber? a. I love him b. Words can not describe how much c. I would give my first born to touch him d. I tattooed his face on my body Explanation: Good survey questions use language that is as unbiased as possible. Asking “how much do you like...” starts with the assumption that you, at the very least, like it a little bit. A much better question would ask a respondent to rate the opinion of Justin Bieber on a scale of 1 to 10 where 10 equals “you like him as much as possible” and 1 equals “you don’t like him at all” Do’s: 1. Do use simple clear language In general you should write questions that an 8th grader could understand. Avoid buzz words, jargon, and words that are not commonly used when writing survey questions. If you have to use potentially confusing words follow it up with an explanation, “and by cohabiting I mean you have lived with another non-relative adult in a romantic relationship.” 2. Do think about social desirability If you ask the question, “Do you use heroin?” on a survey you’ll find that almost everyone says no. If you ask, “Are you a good parent?” everyone says that they are. However, we know that some people bang heroin and some people are crummy parents. What’s going on here is what social scientists call social desirability. Survey respondents are far more likely to lie if the question asked has an answer that is widely thought of as good or desirable. So if you want to ask controversial questions you need to keep this in mind. Lastly, if you have to ask a controversial question ask it as close to the end of the survey as possible. That way if your respondent decides to stop taking the survey when they read the controversial question you will have as much data collected as possible. 3. Do allow respondents to not answer You have to allow your respondent to either opt out of answering a question or to say “Don’t Know”. For multiple choice questions all you need to do is include a response option called “Don’t Know” as well as one that says “Refuse” or “Would rather not answer”.