Questionnaire
Design
Purposes of the Questionnaire
A questionnaire ensures standardization and
comparability of the data across interviews,
increases speed and accuracy of recording
Facilitates data processing
Also allows the researcher to collect the relevant
information necessary to address the management
decision problem
Designing the Questionnaire
Logical Steps to Develop a Good Questionnaire
 Plan what to measure
 Formulate questions to obtain the needed information
 Decide on the wording of questions
 Decide on the order and layout of the questionnaire
 Using a sample, test the questionnaire for omissions and
ambiguity
 Correct the problems (pretest again, if necessary)
The Major Decisions in
Questionnaire Design
1. Content - What should be asked?
2. Wording - How should each question be phrased?
3. Sequence - In what order should the questions be
presented?
4. Layout - What layout will best serve the research
objectives?
The most difficult step is specifying exactly what
information is to be collected from each respondent
Content: Questions must meet 5
requirements
1. Are the questions relevant. Do they pertain to the
research problem
2. Are the questions accurate. Do they accurately depict the
attitudes, behaviors, etc. intended to investigate
3. Do respondents have the necessary information?
• Qualify respondents
4. Do respondents understand and interpret the question
correctly
5. Will respondents give the information?
How should the questions be asked?
Format: How much freedom do we give respondents in
answering questions
Open-ended questions
– eg. People look for different things in a job. What would you prefer most in a
job?
Some key advantages
 Wide range of responses can be obtained
 Lack of influence. Don't channel respondents thinking
 Particularly useful as introduction to survey or topic
 When it’s important to measure the salience of an issue
 When too many possible responses to be listed or unknown
Open-ended questions
Key disadvantages
Ability and/or willingness of respondent to answer
 Interviewer’s ability to record answers quickly or summarize
accurately & probe effectively
 Interviewer’s attitude influences response
 Time consuming (interview sessions, tabulation,
classification, assignment, validation)
 Difficulty in coding
 Require respondents to be articulate
 Respondents may miss important points
 Non-response
Closed-ended questions (Fixed-alternative questions)
People look for different things in a job. What would you prefer
most in a job?
____ Work that pays well
____ Work that gives a sense of accomplishment
____ Work where you make most decisions by yourself
____ Work that is steady with little chance of being laid off.
Advantages
Ease of understanding
 Requires less effort on part of interviewer
 Ease of tabulation & analysis
 Less error prone
 Less time consuming
 Answers directly comparable from respondent to respondent
Closed-ended questions (Fixed-alternative questions)
Disadvantages
 Middle/Neutral categories often selected inappropriately
(ignorance, safety)
 Less opportunity for self-expression or subtle qualifications
 Less involving for respondents
Closed-ended questions
Order of response categories
 Can have major impact on results
Key tradeoff
Want to get respondent to address issues your
research is concerned with (Forced response) and at
same time give respondent opportunity to honestly opt
out of question (i.e., Don’t Know, No Answer, Neither
Agree nor Disagree) so as not to dilute data collected
Question Wording
Can have major impact on how respondent
interprets question
Things to avoid
 Avoid Complexity: use simple, direct, conversational
language
 Avoid leading questions -- that suggest or imply certain answers
 Avoid loaded questions -- that suggest social desirability, or are
emotionally charged
 Avoid ambiguity and vagueness: be as specific as possible
 Avoid long-worded questions
More things to avoid
Avoid Ambiguity: Words such as “often”,
“occasionally”, “usually”, “regularly”, “frequently”,
“many”, should be used with caution. If these words have
to be used, their meaning should be explained properly.
 Avoid double-barreled items. questions that refer to
two or more issues within the same question Where
respondent may agree with only 1 part of multipart
statement. E.g. do you think Nike offers better pricing
and variety than other brands
 Avoid making implicit assumptions
Avoid burdensome questions - that may tax the
respondent’s memory
Surveys are more than a collection of unambiguous questions
How questions are specified and put together will influence
the sample’s willingness to participate & the responses they
provide
Sequence & Layout Decisions
Initial stages
 Need to gain & maintain respondent’s cooperation
 Make questionnaire simple for interviewer to administer
The opening questions should always be interesting, simple, and
easy to answer.
Beyond initial stages
 General questions should be asked before more specific ones
 Demographic questions should come at the end.
 Use multiple questions instead of one
 Similar questions together
– consistent mindset for respondents
Develop a logical flow
 Use transitions between sections
Distinguish question and responses
CAPS or BOLD or Underline versus lower case or
unformatted
Distinguishing question and
responses:
LAYOUT A:
Do you agree, disagree or have no opinion that this company has:
• A good vacation policy - disagree/not sure/agree.
• Good management feedback - disagree/not sure/agree.
• Good medical insurance - disagree/not sure/agree.
LAYOUT B:
Does this company have:
•
Disagree Not Sure Agree
• A good vacation policy
1
2
3
• Good feedback
1
2
3
• Good medical insurance
1
2
3
–Filter and pivot questions should be used as
necessary.
– A FILTER question is one that screens out
respondents who are not qualified to answer a
second question.
–A PIVOT question is a type of filter question that
is used to determine what version of a second
question to ask.
Developing a logical flow
• If the questionnaire deals with several topics,
complete questions on a single topic before
moving on to a new topic
• If topics are related, ask questions on related
topics before asking questions about unrelated
topics
• If you ask questions about behaviors over some
time period, follow chronological order backward
in time
• When changing topics, use some transitional
phrase
QUESTIONNAIRE LAYOUT
The layout and physical attractiveness of
a questionnaire are important aspects
 Questionnaires should be designed to
appear as short as possible
 Questionnaires should not appear
overcrowded
 Questionnaires in booklet form are often
recommended
Pretesting and Correcting Problems
Purpose of pretest: To ensure that the questionnaire meets the
expectations in terms of the information that will be obtained
Missing important variables
Pretesting Specific Questions For
• Variation
• Meaning
• Task difficulty
• Respondent interest and attention
• Ambiguous, ill-defined, loaded, double-barreled questions
Pretesting the Questionnaire
•Flow of the questionnaire
•Skip patterns
•Length
Respondent Interest and Attention
A QUESTIONNAIRE
IS ONLY AS GOOD
AS THE QUESTIONS
IT ASKS