Designing the Questionnaire
Designing the Questionnaire
Logical Steps to Develop a Good
Questionnaire
• Recall the research objective, the research questions and
hypotheses
• Identify the variables to be measured
• Formulate items in the questionnaire
• Order and wording of questions and the layout of the
questionnaire
• Test for omissions and ambiguity
• Correct the problems (pretest again, if necessary)
Exercise – Identify the variables
1. To understand if my ad campaign was liked.
2. To understand how much students know about
credit
3. To understand the relative importance of price
and image in in my brand’s overall sales
4. To understand which brand of cell phone
enjoys top of mind recall
5. To understand if lower prices could lead to
higher sales
Questions must meet 4
requirements
 You must ask the right questions
 Respondents must understand the questions
 Respondents must know the answers
 Respondents must be willing to tell you those
answers.
From Variables to Survey
questions
1.
2.
3.
4.
5.
To understand if my ad
campaign was liked.
To understand how much
students know about credit
To understand the relative
importance of the price and
image in in my brand’s
overall sales
To understand which brand
of cell phone enjoys top of
mind recall
To understand if lower prices
could lead to higher sales
• “I liked the ad campaign” (SD –
SA)
• “How much do you know about
credit” (Nothing – Quite a lot)
• “How important is 1. ‘price’
(Not imp – very Imp) and 2.
‘image’ (Not Imp – Very Imp) in
your decision to buy my brand”
• “Tell us some names of cell
phone brands you can think of”
1._____ 2. _____ 3. _____
• “If the price of X was reduced
by 10%, how likely would you
be to buy X” (Not likely – Very
likely)
Types of Questions
1. Open-response question:
People look for different things in a job.
What would you prefer most in a job?
2. Closed-response question:
People look for different things in a job.
What would you prefer most in a job?
–
–
–
–
Work that pays well
Work that gives a sense of accomplishment
Work where you make most decisions by yourself
Work that is steady with little chance of being laid off.
Open Ended Questions
• Advantages
– Gain insight into the problem
– Too many options to list
– When verbatim responses are desired to give the flavor of the problem
– When behavior to be measured is sensitive or disapproved
– Interviewer / questionnaire structure influence can be minimal
• Disadvantages
– Inarticulate respondents
– Interviewer’s skill in recording quickly and summarizing accurately
– Time consuming, subjective judgments while tabulating, adds to cost
Closed-response Questions
Two Basic Formats for Closed Ended or
Structured Questions
• Choice from a list of responses
• Appropriate single-choice rating on a scale
Closed-response Questions
What type of fast-food restaurant do you visit most
often?
Burger
Chicken
Seafood
Don’t know
Mexican
Pizza
Chinese
Other (please specify)
Closed-response Questions
What is your overall satisfaction with McDonalds
Hamburgers?
Very satisfied
Quite Satisfied

Very satisfied
Somewhat satisfied

7
6

5
4
3
Not at all satisfied

2
1
Not at all satisfied
Closed-response Questions
• Advantages
– Easy to understand, quick responses possible
– Micro-differences in responses can be captured
– Easier tabulation and analysis
– Answers are directly comparable from respondent to respondent
• Disadvantages
– Neutral category may attract more responses than warranted
– Information between categories may be lost (extreme case –
dichotomous categories)
Issues in Questionnaire Design
• Mutually exclusive choices
How many long-distance calls do you make in a
week?
 5 and less
 5-10
 10 and more
Issues in Questionnaire Design
• Order of response categories
“I pay my bills on time”
•
•
•
•
•
Never
Often
Always
Sometimes
Rarely
Issues in Questionnaire Design
• Meanings of response labels
“I pay my bills on time”
•
•
•
•
•
Never
Often
Always
Sometimes
Frequently
Issues in Questionnaire Design
• Range of response categories
How many long-distance calls do you make in a
week?
 less than 5
 less than 10
 5-10
or
 10-20
 More than 10.
 More than 20.
Issues in Questionnaire Design
• Respondent uncertainty
– Should respondents be provided with a
Don’t know or No opinion option?
– When it is important to differentiate between
ambivalence and ignorance, both options
should be provided
Issues in Questionnaire Design
• Question Wording - Vocabulary
– Simple, easy to understand, commonly
used language
– Avoid technical words and jargon
(unless sample is technically qualified)
– Words meaning something else in
different languages and cultures (e.g.
Nova meaning “no go” in Spanish; “mist
stick’ meaning manure in German, etc.)
Question Wording
• Ambiguity
How many times per month do you visit a
fast-food restaurant?
 Never
 Occasionally
 Sometimes
 Often
Question Wording
• Are any questions "double-barreled”?
Are you satisfied with the price and the service of
Taco Bell?
Question Wording
• Are any questions loaded or leading?
1) Don’t you think, because its so greasy, fast-food
is one of the worst types of food? …leading
question
2) Do you prefer a hamburger that is grilled on a
hot stainless-steel grill or cooked by passing the
raw meat through an open gas flame?...loaded
question
• Both skew responses in the desired direction
• Questions which threaten respondent selfesteem e.g. occupation question produces more
“executives”
Issues in Questionnaire Design
• Question Wording
– vocabulary
– “double-barreled” questions
– leading or loaded questions
– Instructions
• Complicated or lengthy instructions confuse
and bias respondents
Question Wording
• Is the question applicable to all
respondents?
Why do you like fast-food?
Assumes respondents like fast foods
Better strategy would be to ask a filter question
first.
Question Wording
• Question length
– Should be short
– Longer questions confuse and fatigue respondents
• Sensitive questions
– Questions on information perceived to be
embarrassing, like personal income, criminal
activities, alcoholism, smoking, drugs habits, social
desirability issues, etc.
– Creativity rules (assurances of confidentiality,
anonymity, slipping it in sideways, open-ended
questions, asking in third person, etc.)
Sequence And Layout
Decisions
• Opening questions – easy and non-threatening
• Flow – smooth and logical – avoid jumps
• Broad to specific
• Critical questions – placed in the middle
• Appealing and interesting
• Order bias – the possibility that subsequent responses
are influenced by preceding responses e.g. fewer people
will say that their taxes are too high after being asked
whether govt. spending should be increased in certain
areas.
• Demographic questions - last
Pretesting and Correcting
Problems
• Pretesting Specific Questions For
• Meaning
• Task difficulty
• Respondent interest and attention
• Pretesting the overall Questionnaire
• Flow of the questionnaire
• Skip patterns
• Length
• Put yourself in the respondent’s shoes and answer
the questionnaire.
Examples – spot the problems in
the questions
Why did you purchase a Bose home theatre system?
 Great self-expressive benefits
 High quality
 Good service
 Great bass to treble ratio
 Better warranty
 Looks good
What is your annual income
Examples – spot the problems in
the questions
Don’t you think Budweiser makes the best commercials
Yes
No
Which of the following restaurants do you visit frequently?
 Burger King
 Pizza Hut
 Subway
 KFC
 McDonalds
Rate how much you like the quality and image of Dove
Hate it 1
2
3
4
5
Love it
Examples – spot the problems
in the questions
Do you agree that, since fast-food restaurants produce a
disproportionate amount of waste, they should be subject to an
additional environmental tax?
 Yes
 No
How often do you watch “American Idol”
Always
Never
Examples – spot the problems
in the questions
How often do you eat fast food?
 Daily
 Every second day
 Once a week
 Every two weeks
What do you think about the super bowl
Love it
Like it
Hate it
Examples – spot the problems
in the questions
• “I read the Wall Street Journal”
– Frequently
– Sometimes
– Often
– Never
– Always