Questionnaire Design
“It is not every question
that deserves an answer.”
Publius Syrus
(Roman, 1st century B.C.)
Ask a Sensitive Question, Get a
Sensitive Answer
• Survey researchers believe
that the responses that
people give are valid.
• Care must be taken with
sensitive questions.
• Researchers must take care
in asking relevant questions
in ways that produce the
most truthful results.
Basic Considerations in
Questionnaire Design
• Questionnaire design is one of the most
critical stages in the survey research
process.
– A questionnaire (survey) is only as good
as the questions it asks — ask a bad
question, get bad results.
– The questions must meet the basic criteria
of relevance and accuracy.
What Should Be Asked?
• Questionnaire Relevancy
– All information collected should address a research question
that helps the decision maker in solving a current marketing
problem.
• Questionnaire Accuracy
– The information is valid; it faithfully represents reality.
• Questionnaires should use simple, understandable, unbiased,
unambiguous, & nonirritating words.
• Questionnaire design should facilitate recall & motivate
respondents to cooperate.
• Proper question wording & sequencing to avoid confusion &
biased answers.
Major Decisions in Questionnaire
Design
What should be asked?
How should each question be phrased?
In what sequence should the questions be
arranged?
What questionnaire layout will best serve
the research objectives?
How should the questionnaire be pretested?
Does the questionnaire need to be revised?
Phrasing Questions
• Open-ended questions
–Aka essay questions, short-answer
questions
• Fixed-alternative questions
–Aka closed or closed-ended questions
Open-Ended Response Questions
– Pose some problem & ask respondents to answer in their
own words.
– Advantages:
• Particularly beneficial in exploratory research, especially when
the range of responses is not known.
• Identify which words & phrases people spontaneously give.
• Valuable at the beginning of an interview.
– Disadvantages:
• High cost of administering open-ended response questions.
• The possibility that interviewer bias will influence the answer.
• Bias introduced by articulate individuals’ longer answers.
Example of Open-Ended Response
Question
Fixed-alternative Questions
– Questions in which respondents are given specific,
limited-alternative responses & asked to choose the
one closest to their own viewpoint.
– Advantages:
•
•
•
•
Require less interviewer skill
Take less time to answer
Are easier for the respondent to answer
Provides comparability of answers
– Disadvantages:
• Researcher may be unaware of all potential responses
• Tendency of respondents to choose more prestigious or
socially acceptable alternative
Example of a Fixed-Alternative
Question
Types of Fixed-Alternative Questions
• Simple-dichotomy (dichotomous) Question
– Requires the respondent to choose one of two alternatives (e.g., yes or no).
– Example:
Did you make any calls with your home (landline) phone during the 7 days?
_____ Yes
_____ No
• Determinant-Choice (multiple-choice) Question
– Requires the respondent to choose one response from among multiple
alternatives (e.g., A, B, or C).
– Example:
Types of Fixed-Alternative Questions
(con’t.)
• Frequency-determination Question
– Asks for an answer about general frequency of occurrence (e.g., often,
occasionally, or never).
• Checklist Question
– Allows the respondent to provide multiple answers to a single question by
checking off items.
• Scale
– Likert, Semantic Differential, Stapel, etc.
Phrasing Questions for Self-Administered,
Telephone, & Personal Interview Surveys
• Influences on Question Phrasing
– Means of data collection — telephone
interview, personal interview, self-administered
questionnaire — will influence question format
& question phrasing.
• Questions for mail, Internet, & telephone surveys
must be less complex than those used in personal
interviews.
• Questionnaires for telephone & personal interviews
should be written in a conversational style.
Best Question Formats Vary by the Interview Medium
Guidelines For Avoiding Mistakes
• Simpler is better
• Avoid leading & loaded questions
– Leading question – directs respondents to an answer you want them to give
– Loaded question – suggests a socially desirable answer or is emotionally
charged.
Guidelines For Avoiding Mistakes
(con’t.)
• “Avoid” ambiguity: Be as specific as possible
– Probably /Definitely; Sometimes/Always, etc.
• Avoid double-barreled items
– Double-barreled question – may induce bias because it covers two or more
issues at once.
– Do you think the President is responsible for the federal government shutdown and the currently rising gasoline prices?
Yes
No
• Avoid making assumptions
– Given Macy’s skill-level at gift wrapping, …….
• All-inclusive response alternatives
• Avoid taxing respondent’s memory
Avoid Common Wording Mistakes in
Questionnaire Design
Order Bias
• Question Sequence
– Order bias
• Bias caused by the influence of earlier questions in a questionnaire or by an
answer’s position in a set of answers.
– Funnel technique
• Asking general questions before specific questions in order to obtain
unbiased responses.
• Randomized Presentations
– Used in electronic questionnaires, but rarely used in printed
questionnaires due to coding difficulties.
• Randomized Response Techniques
– Randomly assigning respondents to answer either the question of
interest (embarrassing) or a mundane & unembarrassing question.
Survey Flow
• Survey flow
– The ordering of questions through a survey.
• Breakoff
– A respondent who stops answering questions before reaching the
end of the survey.
• Filter question
– A question that screens out respondents who are not qualified to
answer a second (or follow-up) question.
• Branching
– Directing respondents to alternative portions of the questionnaire
based on their response to a filter question.
Survey Flow for Eurocar Tour de France Sponsorship
Telephone Questionnaire with Skip Questions
Survey Technology
• Physical Features
– Heat map question
• A graphical question that tracks the parts of an image or advertisement
that most capture a respondent’s attention.
– Status Bar
• A visual indicator that tells the respondent what portion of the survey he
or she has completed.
• Prompting
– Informs the respondent that he or she has skipped an item or
provided implausible information.
• Piping Software
– Allows question answers to be inserted into later questions.
Tracking Points of Interest Using a Heat Map
Illustration of Status Bar & Prompts
Pretesting & Revising Questionnaires
• Pretesting Process
– Seeks to determine whether respondents have
any difficulty understanding the questionnaire
– whether there are any ambiguous or biased
questions.
• Preliminary Tabulation
– A tabulation of results of a pretest to help
determine whether questionnaire will meet the
objectives of the research.
I Give Up!
 Large portion of respondents give up
before finishing & abandon the survey
(break-offs).
 Guidelines
 Visually appealing & easy to read
 Fewer questions per page (no more than 20)
 4 pages maximum for consumers
 6 pages maximum for business leaders
 Question order (funnel vs. reverse-funnel)
 Sensitive questions & open-ended questions
encourage break-offs &/or item non-response
 Sophisticated samples increase response rate
 Make information requests legitimate
 Pretesting is important (pretest EVERYTHING)
Questionnaire Reproduction
(if mail survey)
Professional appearance
Booklet format for long
questionnaires
Place directions as close to
questions as possible
Expect reproduction errors