Survey Terms and Survey Design Important terms The presentation of a [survey] to a few individuals, who are either from the intended priority population or from a very similar population (McKenzie, Pinger & Kotecki, 2008, p 617). Pilot testing: Anonymity: Confidentiality: an ethical practice of providing research participants with enough information about a study, especially its potential risks, to enable them to make an informed decision about whether to participate (Singleton and Straits, 2005, p 564). Informed Consent: Surveys are a great way to collect information from a large group of people. They are typically more structured than interviews, helping to reach more people but in less depth. These are generalities of surveys, as there are many different types of surveys that researchers may choose. Surveys can take various forms, each with advantages and disadvantages. Some surveys are read aloud to respondents, either in person or over the phone. Others are self-administered, meaning that the respondent of the survey fills out the survey form by him or herself. Self-administered surveys can be taken with paper and pencil or online, while still other types of surveys combine different survey types to collect information for the same project (Singleton & Straits, 2005, p 244). Online surveys hold the advantage that they can be given to many people with a relatively small cost. It is also easier for respondents to return surveys that are online. They can also provide more options than paper surveys in the design and adaptability of the survey. However, online surveys do not always have a strong response rate, and have limitations in that individuals who do not have access to the Internet are left out of the sample (Singleton & Straits, 2005, p 244). Pretesting or Pilot-testing It is important to pretest (or pilot) a survey before giving it to the entire sample to make sure that all questions are clear. Give your survey to a few individuals who have the same characteristics as the rest of the sample, but who are not in the sample itself. (Ex. A few students from a different school, class, or grade, but who would be likely to have similar responses). This allows a researcher to determine if it is necessary to make revisions to the survey. For example, if those pretesting the survey have trouble understanding or answering the survey questions, or if the survey is not meeting its designed purpose, the researcher may want to edit the survey or its questions so that the intended sample does not have the same trouble (Singleton & Straits, 2005, p 248). Anonymity and Confidentiality In order for the survey administrator to get the most accurate results, survey responses should be anonymous. This means that the survey administrator, or anyone else who sees the survey, should not be able to determine the identity of the survey respondent. While this usually means leaving the name and email address off the survey, it also means not asking questions that are so specific that someone would be able to identify the participant by the responses they provide. Anonymity is also practiced in order to fully respect the privacy of the respondent. Researchers must make sure that survey responses are confidential when identifying information is used. This may mean removing names from surveys as soon as possible, not using real names when reporting results and keeping surveys in locked cabinets (Singleton & Straits, 2005, p 527). Since the surveys in this course will remain anonymous, we won’t have to worry about confidentiality, but it is a very important part of research. Informed Consent Informed consent is “an ethical practice of providing research participants with enough information about a study, especially its potential risks, to enable them to make an informed decision about whether to participate” (Singleton and Straits, 2005, p 564). Basically this means participants should know what they are going to be asked and what purpose the survey will serve. While there will be few potential risks from our surveys, it is still important that participants voluntarily take surveys. Informed consent and anonymity are both typically addressed in the cover letter. Cover letter Many surveys include a cover letter or description of the survey and its purpose. Cover letters may encourage the respondent to answer survey questions by describing the purpose of the survey and how this information will be used. Researchers also provide their contact information for follow up. The cover letter is also used to describe any incentives, or rewards, for taking the survey. For the purposes of this course, the cover letter must also state that survey answers will remain anonymous. Tips and Pointers for Designing a SelfAdministered survey Important terms A double barreled question is a single question that actually asks two different things at once (Singleton and Straits, 2005). Double Barreled Question: A leading question “suggests a possible answer or makes some responses seem [better] than others” (Singleton and Straits, 2005, p 565). Leading Question: Surveys should have a clear focus The first step to designing a survey is deciding exactly what you want to know from the survey (what research question do you hope to answer from the survey?). This research question should serve as your guide to creating each survey question. If a particular question does not help you to answer your research question, do not include it in the survey. For our purposes, we suggest asking questions about potential risk factors for the problem we are studying and finding out whether these risk factors are true for your selected population. For example, if you are studying injury during car accidents, and your key consideration is seat belt use, ask questions ask questions about using a seatbelt, not about injury. Your questions should also be tailored to the ecological level that you have chosen. If you have chosen to analyze at the interpersonal level, ask questions about how the participant's family and/or friends. If you are studying the institutional level, ask questions about the policies or habits of the individuals in that institution. You do not need to ask questions about the individual’s seatbelt use, unless you are focusing on the intrapersonal level. When possible, surveys should be short A survey likely will not hold a respondents attention for more than a few minutes. For this course, surveys should be between 5 and 15 questions long. Questions should be concise Each question should be short and clear-cut. If a question needs too much clarification, it may need to be separated into more than one question. Closed-ended questions versus Open-ended questions (free response) Our surveys will be using closed-ended questions. Note that closed-ended questions, in which respondents choose from a set of listed responses, are easier to analyze and are often easier for the respondent. However, open-ended questions, or freeresponse questions, are often used in research because they provide a greater depth of information (Singleton and Straigts, 2005, p 266). Open-ended questions are typically used in interviews, but do appear in many surveys as well. Stay Away from Double Barreled Questions A double barreled question is a single question that actually asks two different things at once (Singleton and Straits, 2005, p 284). For example asking “Why do you exercise and eat right?” is actually asking two different things. People may exercise because they think its fun but eat right because they like the way healthy food tastes. Instead, the question should be separated into two different questions; “Why do you exercise?” and “Why do you eat right?” Stay Away from Leading Questions A leading question “suggests a possible answer or makes some responses seem [better] than others” (Singleton and Straits, 2005, p 565). In the example above, respondents are asked “Why do you exercise”? However, some respondents may not exercise at all. Which option would a person choose below if they never exercised? Why do you exercise? A. I exercise because it is fun B. I exercise to become healthy C. I exercise to stay healthy D. I exercise because I feel good afterward. It may be better to ask if a person exercises, and then ask them why they exercise. (See example). Do you exercise? A. Yes B. No Skip this question if you answered “B. No” to the previous question. If you answered “A. Yes,” why do you exercise? A. I exercise because it is fun B. I exercise to become healthy C. I exercise to stay healthy D. I exercise because I feel good afterward. Alternatively, you can combine the two questions, as long as you include an “I do not exercise” is a response option. (See example) If you exercise, why do you exercise? A. I exercise because it is fun B. I exercise to become healthy C. I exercise to stay healthy D. I exercise because I feel good afterward E. I do not exercise Another type of leading question makes the respondent think that they should answer a question in a particular way. In the following example, the use of the word “love” makes individuals think that they should love math class. Even the responses to this question are bias because love is used as an option. How much do you love math class? A. I love math class more than most people B. I love math class the same as most people C. I love math class less than most people However, asking people how they feel about math class, instead of how much do they love math class, allows them to come up with their own feelings about the course. How do you feel about math class? A. I like it B. Its alright C. I don’t like it Opening question The first question in a survey or interview should be easy to answer and nonthreatening. It should also grab the survey takers interest. For example, you might not want to ask personal questions about a respondent’s weight as a first question in a health survey. Save more complicated or uncomfortable questions for the latter parts of the survey. Flow of questions Survey questions should flow from one to the next. For example, if you are asking questions about eating habits, do not skip around from fruits, to vegetables, to meats, and back to fruits. Try to ask all questions about the same subject before moving on to a different subject. This will help to keep the respondents’ focus and increase the quality of the research (Singleton and Straigts, 2005, p 279). Response Options Multiple Choice (See example question 1) Multiple Choice answers should be used when there are a few options that you would like your respondent to choose from. These options may include “Yes”, “No” “I Don’t Know” or may include options like “Cats, Dogs, Fish, Rodents, No Pets”. Try not to list too many options for this type of response. Choose from a list (see example question 2) The “Choose from a list” option allows individuals to pick an answer from a variety choices. Use this option when respondents should only select one answer, but when you would like them to have many options to choose from. Check boxes (See example question 3) Check boxes allow respondents to enter more than one response at once. If you would like respondents to be able to say that they like both Math, Science, and P.E., use check boxes. However, if you want to know which subject is their favorite (favorite implies only one response is possible), pick the “Choose from a list” option. Scale (See example question 4) A “Scale” allows a researcher to enter the two end points of a 5 point rating scale. In research, survey designers usually present a statement and then ask respondents to select how much they agree with the statement that follows. As a result, these two points are most often “strongly agree” and “strongly disagree”. Thus, if a respondent chooses 1, they strongly disagree with the statement. If they chose 3, they neither agree nor disagree. If they choose 5, they strongly agree with the statement. Feel free to try different end point options for this type of question. Example Survey