Survey Terms and Survey Design
Important terms
The presentation of a [survey] to a few individuals, who are either from the
intended priority population or from a very similar population (McKenzie, Pinger & Kotecki, 2008, p
617).
Pilot testing:
Anonymity:
Confidentiality:
an ethical practice of providing research participants with enough
information about a study, especially its potential risks, to enable them to make an informed
decision about whether to participate (Singleton and Straits, 2005, p 564).
Informed Consent:
Surveys are a great way to collect information from a large group of people. They
are typically more structured than interviews, helping to reach more people but in
less depth. These are generalities of surveys, as there are many different types of
surveys that researchers may choose.
Surveys can take various forms, each with advantages and disadvantages. Some
surveys are read aloud to respondents, either in person or over the phone. Others
are self-administered, meaning that the respondent of the survey fills out the survey
form by him or herself. Self-administered surveys can be taken with paper and pencil
or online, while still other types of surveys combine different survey types to collect
information for the same project (Singleton & Straits, 2005, p 244). Online surveys
hold the advantage that they can be given to many people with a relatively small
cost. It is also easier for respondents to return surveys that are online. They can also
provide more options than paper surveys in the design and adaptability of the
survey. However, online surveys do not always have a strong response rate, and
have limitations in that individuals who do not have access to the Internet are left
out of the sample (Singleton & Straits, 2005, p 244).
Pretesting or Pilot-testing
It is important to pretest (or pilot) a survey before giving it to the entire sample to
make sure that all questions are clear. Give your survey to a few individuals who
have the same characteristics as the rest of the sample, but who are not in the
sample itself. (Ex. A few students from a different school, class, or grade, but who
would be likely to have similar responses). This allows a researcher to determine if it
is necessary to make revisions to the survey. For example, if those pretesting the
survey have trouble understanding or answering the survey questions, or if the
survey is not meeting its designed purpose, the researcher may want to edit the
survey or its questions so that the intended sample does not have the same trouble
(Singleton & Straits, 2005, p 248).
Anonymity and Confidentiality
In order for the survey administrator to get the most accurate results, survey
responses should be anonymous. This means that the survey administrator, or
anyone else who sees the survey, should not be able to determine the identity of the
survey respondent. While this usually means leaving the name and email address off
the survey, it also means not asking questions that are so specific that someone
would be able to identify the participant by the responses they provide. Anonymity is
also practiced in order to fully respect the privacy of the respondent.
Researchers must make sure that survey responses are confidential when identifying
information is used. This may mean removing names from surveys as soon as
possible, not using real names when reporting results and keeping surveys in locked
cabinets (Singleton & Straits, 2005, p 527). Since the surveys in this course will
remain anonymous, we won’t have to worry about confidentiality, but it is a very
important part of research.
Informed Consent
Informed consent is “an ethical practice of providing research participants with
enough information about a study, especially its potential risks, to enable them to
make an informed decision about whether to participate” (Singleton and Straits,
2005, p 564). Basically this means participants should know what they are going to
be asked and what purpose the survey will serve. While there will be few potential
risks from our surveys, it is still important that participants voluntarily take surveys.
Informed consent and anonymity are both typically addressed in the cover letter.
Cover letter
Many surveys include a cover letter or description of the survey and its purpose.
Cover letters may encourage the respondent to answer survey questions by
describing the purpose of the survey and how this information will be used.
Researchers also provide their contact information for follow up. The cover letter is
also used to describe any incentives, or rewards, for taking the survey. For the
purposes of this course, the cover letter must also state that survey answers will
remain anonymous.
Tips and Pointers for Designing a SelfAdministered survey
Important terms
A double barreled question is a single question that actually
asks two different things at once (Singleton and Straits, 2005).
Double Barreled Question:
A leading question “suggests a possible answer or makes some
responses seem [better] than others” (Singleton and Straits, 2005, p 565).
Leading Question:
Surveys should have a clear focus
The first step to designing a survey is deciding exactly what you want to know from
the survey (what research question do you hope to answer from the survey?). This
research question should serve as your guide to creating each survey question. If a
particular question does not help you to answer your research question, do not
include it in the survey.
For our purposes, we suggest asking questions about potential risk factors for the problem we are
studying and finding out whether these risk factors are true for your selected population. For
example, if you are studying injury during car accidents, and your key consideration is seat belt
use, ask questions ask questions about using a seatbelt, not about injury. Your questions should
also be tailored to the ecological level that you have chosen. If you have chosen to analyze at the
interpersonal level, ask questions about how the participant's family and/or friends. If you are
studying the institutional level, ask questions about the policies or habits of the individuals in that
institution. You do not need to ask questions about the individual’s seatbelt use, unless you are
focusing on the intrapersonal level.
When possible, surveys should be short
A survey likely will not hold a respondents attention for more than a few minutes.
For this course, surveys should be between 5 and 15 questions long.
Questions should be concise
Each question should be short and clear-cut. If a question needs too much
clarification, it may need to be separated into more than one question.
Closed-ended questions versus Open-ended questions (free response)
Our surveys will be using closed-ended questions. Note that closed-ended questions,
in which respondents choose from a set of listed responses, are easier to analyze and
are often easier for the respondent. However, open-ended questions, or freeresponse questions, are often used in research because they provide a greater depth
of information (Singleton and Straigts, 2005, p 266). Open-ended questions are
typically used in interviews, but do appear in many surveys as well.
Stay Away from Double Barreled Questions
A double barreled question is a single question that actually asks two different things
at once (Singleton and Straits, 2005, p 284). For example asking “Why do you
exercise and eat right?” is actually asking two different things. People may exercise
because they think its fun but eat right because they like the way healthy food
tastes. Instead, the question should be separated into two different questions; “Why
do you exercise?” and “Why do you eat right?”
Stay Away from Leading Questions
A leading question “suggests a possible answer or makes some responses seem
[better] than others” (Singleton and Straits, 2005, p 565). In the example above,
respondents are asked “Why do you exercise”? However, some respondents may not
exercise at all. Which option would a person choose below if they never exercised?
Why do you exercise?
A.
I exercise because it is fun
B.
I exercise to become healthy
C.
I exercise to stay healthy
D.
I exercise because I feel good afterward.
It may be better to ask if a person exercises, and then ask them why they exercise.
(See example).
Do you exercise?
A.
Yes
B.
No
Skip this question if you answered “B. No” to the previous question. If you
answered “A. Yes,” why do you exercise?
A.
I exercise because it is fun
B.
I exercise to become healthy
C.
I exercise to stay healthy
D.
I exercise because I feel good afterward.
Alternatively, you can combine the two questions, as long as you include an “I do not
exercise” is a response option. (See example)
If you exercise, why do you exercise?
A.
I exercise because it is fun
B.
I exercise to become healthy
C.
I exercise to stay healthy
D.
I exercise because I feel good afterward
E.
I do not exercise
Another type of leading question makes the respondent think that they should
answer a question in a particular way. In the following example, the use of the word
“love” makes individuals think that they should love math class. Even the responses
to this question are bias because love is used as an option.
How much do you love math class?
A.
I love math class more than most people
B.
I love math class the same as most people
C.
I love math class less than most people
However, asking people how they feel about math class, instead of how much do
they love math class, allows them to come up with their own feelings about the
course.
How do you feel about math class?
A.
I like it
B.
Its alright
C.
I don’t like it
Opening question
The first question in a survey or interview should be easy to answer and
nonthreatening. It should also grab the survey takers interest. For example, you
might not want to ask personal questions about a respondent’s weight as a first
question in a health survey. Save more complicated or uncomfortable questions for
the latter parts of the survey.
Flow of questions
Survey questions should flow from one to the next. For example, if you are asking
questions about eating habits, do not skip around from fruits, to vegetables, to
meats, and back to fruits. Try to ask all questions about the same subject before
moving on to a different subject. This will help to keep the respondents’ focus and
increase the quality of the research (Singleton and Straigts, 2005, p 279).
Response Options
Multiple Choice (See example question 1)
Multiple Choice answers should be used when there are a few options that you would
like your respondent to choose from. These options may include “Yes”, “No” “I Don’t
Know” or may include options like “Cats, Dogs, Fish, Rodents, No Pets”. Try not to
list too many options for this type of response.
Choose from a list (see example question 2)
The “Choose from a list” option allows individuals to pick an answer from a variety
choices. Use this option when respondents should only select one answer, but when
you would like them to have many options to choose from.
Check boxes (See example question 3)
Check boxes allow respondents to enter more than one response at once. If you
would like respondents to be able to say that they like both Math, Science, and P.E.,
use check boxes. However, if you want to know which subject is their favorite
(favorite implies only one response is possible), pick the “Choose from a list” option.
Scale (See example question 4)
A “Scale” allows a researcher to enter the two end points of a 5 point rating scale. In
research, survey designers usually present a statement and then ask respondents to
select how much they agree with the statement that follows. As a result, these two
points are most often “strongly agree” and “strongly disagree”. Thus, if a respondent
chooses 1, they strongly disagree with the statement. If they chose 3, they neither
agree nor disagree. If they choose 5, they strongly agree with the statement. Feel
free to try different end point options for this type of question.
Example Survey