The Questionnaire
Development Process
MKTG 3342
Fall 2008
Professor Edward Fox
Questionnaire
Definition – A set of questions designed to
generate the information necessary to meet
research objectives
 Characteristics






Elicits information from respondents
Results can be tabulated
Standardized across respondents
Understandable to respondents
A good questionnaire must:



provide the necessary information
be considerate of respondents
meet coding and data processing requirements
The Questionnaire Design Process
Implementation
Determine Data
Collection Methods
Determine Question
Response Format
Prepare Final Copy
Determine Survey
Objectives and
Constraints
Decide on Question
Wording
Pretest and Revise
Obtain Approval
from all Relevant
Parties
Evaluate the
Questionnaire and
Layout
Establish
Questionnaire Flow
and Layout
Step 1: Determine Survey Objectives
Translates research objectives into
information requirements
 Survey objectives should be spelled out as
clearly and precisely as possible

 Rule 1 - Let the research objectives
dictate what questions to ask
 Rule 2 - Avoid unnecessary questions
Step 2: Determine the Data Collection
Method

We previously discussed the variety of ways
that survey data can be gathered
Personal
 Telephone
 Electronic
 Mail
 Self administered


Each method has a different impact on
survey design
Step 3: Determine the Question
Response Format
Open-Ended Questions
 Open-ended
questions are those in which the
respondent can reply in his/her own words
 Example of an open-ended question:
What are the advantages, if any, of
ordering from a mail order catalog
compared with buying at a nearby
retail outlet?
Open-Ended Questions
Open-ended questions are useful:
to interpret closed-ended responses
when there are too many alternatives
to list
 when alternatives vary between
respondents or are not clear
 when intensity of feeling is important
 when verbatim response is helpful
 for some sensitive issues (e.g. illness)


Open-Ended Questions
Advantages
 More information
 Wide range of responses
 Maybe more accurate description
Limitations
 Respondent articulateness can lead to
error/variation
 Time consuming for respondent
 Coding and analysis is difficult
Step 3: Determine the Question
Response Format
Closed-Ended Questions
A
closed-ended question is one that requires
the respondent to make a selection from a list
of options
 The primary advantage of closed-ended
questions is avoiding the problems of openended questions (such as subjectivity, coding,
etc.)
 May be dichotomous or multiple choice
Closed-Ended Questions
 Types
 Dichotomous
– Respondent chooses
between two alternatives
 Multiple-Choice – Respondent selects from
among several alternatives
 Example of a multiple-choice question:
Who do you think will win the World Series?
A. Tampa Bay Devil Rays
B. Boston Red Sox
C. Philadelphia Phillies
D. Los Angeles Dodgers
Step 3: Determine the Question
Response Format
Scaled-Response Questions
 Question
format that permits measurement of
the “intensity” of a respondent’s answers
 Example of a scaled-response question:
Now that you have tried the new sugar-free
Slurpee, would you say that you would…
(Check one)
definitely buy it
probably buy it
might or might not buy it
probably not buy it
definitely not buy it
Closed-Ended Questions
Advantages
 Easier to answer
 Reduce measurement error by overcoming
respondent articulateness and possible
interviewer bias
 Easy to code and analyze
 More likely to respond for some personal data
(e.g. income, age)
Disadvantages
 Information is lost
 Answers may not be precise
 Not able to develop alternatives
Step 4: Question Wording
 Rule 3: Use simple words
 e.g., not “marital status” but whether
“married,” “single” or “divorced”
 Rule 4: Use unambiguous words (words
that are easily and clearly understood)
e.g. not “where do you usually buy school
supplies?”
but “from which retailer(s) did you buy school
supplies this year?”
“where” and “usually” are often ambiguous

Question Wording
 Rule 5: Avoid “double-barreled”
questions (two questions in one)
 e.g.,
“What is your opinion about the price
and convenience of DART?”
 Rule 6: Avoid leading questions
(questions that imply an answer)
 e.g.
“Do you agree that the government
should cut taxes so that we can lead better
lives?”
Question Wording
 Rule 7: Avoid “loaded” questions
(framing questions so that they are
leading or emotionally charged)

e.g., “Do you think chemical companies are
doing everything possible to control
pollution?”
 Rule 8: Avoid implicit assumptions or
alternatives – State them clearly

e.g., “Do you favor a law mandating
methanol-free gasoline” [if it means an incras
in gas prices]
Question Wording
 Rule 9: Avoid burdensome questions
(questions that tax memory or skills)
 e.g.,
“How many different brands of breakfast
cereal have you bought in the last twelve
months?”
 Rule 10: Clarify references (how the
should respondent answer)
 e.g.,
“State your annual income” [personal
income or household income?]
Question Wording
 Rule 11: Alternatives should be
mutually exclusive,collectively
exhaustive, and have reasonable
range
e.g., What is your annual household income?
__ $0 - $10,000
__ $10,000 - $15,000
__ $15,000 - $20,000
__ $40,000 - $60,000
What is wrong with this question and how
would you correct it?
Question Wording – Scaled
Responses
 Rule 12: Use appropriate phrases
e.g.,
B. Are you: (1) amazingly happy, (2)
middling happy or (3) kind of unhappy
(check one)
A. Are you: (1) very happy, (2) happy or (3)
unhappy (check one)
 Rule 13: Avoid order bias
e.g., should you start with (1) as “very
happy” or (1) as “unhappy?”
Question Wording - Sensitive /
Embarrassing Questions

Questions about borrowing money, personal
hygiene, sexual activity, criminal history, etc.
must be phrased carefully to minimize
measurement error

Suggestions/Techniques - Not Rules

Ask non-threateningly and mix with other questions
e.g., How many times each day do you brush your
teeth?

Frame question to prompt more honest responses
e.g., Many people find it difficult to brush their teeth
twice a day. How often do you brush your teeth?
Step 5: Questionnaire Flow and
Layout
Use screening
questions to
identify qualified
respondents
Position sensitive,
threatening, and
demographic
questions as the
end
After identifying
qualified respondent,
begin with a question
that piques the
respondent’s interest
Insert “prompters”
at strategic points
Ask general
questions first
Ask questions that
require “work” in
the middle of the
questionnaire
Step 5: Questionnaire Flow and Layout
 Rule 14: Opening questions should be
simple and interesting – begin with openended question if warm-up is necessary
 Rule 15: Use “funnel approach” – ask
most general questions first, then
increasingly specific questions
 Rule 16: Questions should flow smoothly
and logically
Step 5: Questionnaire Flow and Layout
 Rule 17: Ask “screener” questions up front
to assess respondent qualifications
 Rule 18: Ask sensitive questions near the
end
 Rule 19: Ask for demographic information
at the end
 Rule 20: The questionnaire’s appearance
should be attractive and professional
Step 6: Evaluate the Questionnaire

Is each question necessary?

Is the questionnaire too long?

Will the questions provide the desired information
to accomplish the research objectives?

Is the layout attractive and professional?
Step 8: Pretest and Revise
 Rule 21: Always pretest your questionnaire
Pretesting can be done by giving the
questionnaire to a few friends, “experts”, and
potential respondents (may be 5-10 persons
total), asking them to fill out, and obtaining
their feedback
Step 8: Pretest and Revise
Pretest the questionnaire for:
 wording of individual questions
 flow of the questionnaire
 skip patterns
 length
 respondent interest and attention
Step 9: Prepare Final Copy

Even the final copy phase does not allow
the researcher to relax.
 Precise
typing instructions, spacing,
numbering, and precoding must be set up,
monitored, and proofread,
before …
Step 10: Implement the Survey