CHAPTER 11
DESIGNING DATA
COLLECTION FORMS
The Questionnaire
Development Process
Implementation
Determine Data
Collection Methods
Determine Question
Response Format
Prepare Final Copy
Determine Survey
Objectives and
Constraints
Decide Question
Wording
Pretest and Revise
Obtain Approval
from all Relevant
Parties
Evaluate the
Questionnaire and
Layout
Describe
Questionnaire Flow
and Layout
Determine Objectives,
Resources, and Constraints
Survey (information) objectives
should be spelled out as clearly and
precisely as possible.
 If this step is completed in a
thorough fashion, the rest of the
process will follow more smoothly
and efficiently.

Determine the Data
Collection Method
Chapter 9 discussed the variety of
ways that survey data can be
gathered, such as in-person,
telephone, mail, internet or selfadministration.
 Each method will have an impact on
survey design.

Determine the Question
Response Format

Open-Ended Questions


Open-ended questions are those in which the
respondent can reply in his or her own words.
Closed-Ended Questions


A closed-ended question is one that requires
the
1 respondent to make a selection from a list
of options.
The primary advantage of closed-ended
questions is simply the avoidance of many of
the problems (such as subjectivity) of openended questions.
Determine the Question
Response Format(cont.)

Dichotomous Questions

This simplest form of a closed-ended
question is the dichotomous choice.
• Example of a dichotomous question:
• Would you rather live in a small town or a city?
• A. Small Town
1
• B. City
2
Determine the Question
Response Format(cont.)

Multiple-Choice Questions

Multiple-Choice Questions permit the
respondent to select from several
alternatives.
• Example of a multiple-choice question:
•Who do you think will win the World Series?
•A. New York Yankees
•B. St. Louis Cardinals
•C. Los Angeles Dodgers
•D. Atlanta Braves
Determine the Question
Response Format (cont.)

Scaled-Response Questions:

Question format that permits the
measurement of the “intensity” of a
respondents’ answers.
• Example of a scaled-response question:
•. Now that you have read a sample of the Wall Street
Journal, would you say that you would…(Check one):
Definitely buy it
Probably buy it
Might or might not buy it
Probably will not buy it
Definitely will not buy it
Decide the Question
Wording

Four Guidelines in Selecting Wording
The wording must be clear.
 Select words so as to avoid biasing the
respondent.
 Consider the ability of the respondent
to answer the question.
 Consider the willingness of the
respondent to answer the question.

Establish Questionnaire
Flow and Layout
After the questions have been
properly formulated, the next step is
to sequence them and develop a
layout for the questionnaire.
 The general guidelines concerning
questionnaire flow are included in
the next slide.

How a Questionnaire
Should be Organized
Use the screener
questions to
identify qualified
respondents
Ask questions that
require “work” in
the middle of the
QuestionnaireTransition questions
After obtaining a
qualified respondent,
begin with a question
that obtains a
respondent’s interestWarm-up questions
Insert “prompters”
at strategic points
Ask general
questions first.
Use skip questions
If it is necessary
Position sensitive,
threatening, and
Classification or
demographic
questions at the
end
Evaluate the Questionnaire

Issues to Consider in Reviewing the
Design of the Questionnaire:
• Is the Question Necessary?
• Is the Questionnaire Too Long?
• Will the Questions Provide the Desired Information to Accomplish the
Research Objectives?
• Appearances of Mail and Self-Administered Questionnaires
• Avoid a Cluttered Look
• Allow Plenty of Space for Open-Ended Responses
• Consider Color-Coding the Questionnaires
• Instructions Printed Within The Questionnaire Should be in Capital
Letters
Obtain Approval of All
Relevant Parties
At this point in the questionnaire
design process, the first draft of the
questionnaire has been completed.
 Approval for going forward with the
questionnaire is the objective of this
step.

Pretest and Revise
When final managerial approval has
been obtained, the questionnaire
must be pre-tested.
 The primary purpose of a pretest is
to make certain that the
questionnaire gives the respondent
clear, understandable questions that
will evoke clear, understandable
responses.

Prepare Final Copy

Even the final copy phase does not
allow the researcher to relax.

Precise typing instructions, spacing,
numbering, and pre-coding must be set
up, monitored, and proofread.
Implementing the Survey

Supervisor’s Instructions:



Inform them of the nature of the study, start
and completion dates, quotas, reporting
times, equipment and facility requirements,
sampling instructions, number of interviewers
required, and validation procedures.
Interviewer’s Instructions
Cover many of the same points as
supervisor’s instructions but are geared to the
actual interview.
Implementing the Survey
(cont.)

Call Record Sheets
Interviewers’ logs listing the number
and results of a contact.
 Call record sheets are used to measure
the efficiency of the interviewers.


Visual Aids and Other Supplements

Many studies use visual aids to
facilitate the interviewing process.
Implementing the Survey
(cont.)

Field Management Companies

Field management companies such as Qfact,
On-Line Communications, and Direct
Resource generally provide questionnaire
formatting, screener writing, development of
instructional and peripheral materials,
shipping departments, field auditing, and all
coordination of data collection, coding, and
tab services required for the project.
Questionnaire Challenges
The Difficulty of Finding a Qualified Respondent in a Central
Location Telephone Interview
1. Failed Attempts
- Busy
- No Answer
- Answering Machine
- Business Number
- Phone/Language Problem
- Discontinued Line
2. Cooperation Problems
- Respondent Not at Home
- Refused to be Interviewed
Questionnaire Challenges
The Difficulty of Finding a Qualified Respondent in a Central
Location Telephone Interview
3. Screener Determines Respondent Not Eligible
- Failed security test (works for marketing research firm,
advertising agency, or the client)
- Doesn’t use the product
- Demographic disqualification (wrong gender, age, etc.)
- Quota filled (survey has quota of 500 users of Tide and 500
users of other clothing washing powder. Interviewer already
has 500 Tide users; the current respondent uses Tide.
4. Respondent Terminates During Interview
5. Completed interview
Questionnaire Design Checklist
Step 1. Specify The Information Needed
Step 2. Type of Interviewing Method
Step 3. Individual Question Content
Step 4. Overcome Inability and Unwillingness to Answer
Step 5. Choose Question Structure
Step 6. Choose Question Wording
Step 7. Determine the Order of Questions
Step 8. Form and Layout
Step 9. Reproduce the Questionnaire
Step 10. Pretest
Questionnaire Design Checklist
Step 1.
Specify The Information Needed:
•
Ensure that the information obtained fully addresses all the components of the problem.
•
Review components of the problem and the approach, particularly the research questions,
hypotheses, and characteristics that influence the research design.
Step 2.
Individual Question Content:
•
Is the question necessary?
•
Are several questions needed instead of one to obtain the required information in an
unambiguous manner?
•
Do not use double-barreled questions
•
Question should be focused on a single issue.
•
Question should be brief.
•
Question should be interpreted the same way by all responded.
•
Question should use the respondent’s core vocabulary.
Step 3. Overcoming Inability and Unwillingness to Answer:
•Is the respondent informed?
•If respondents are not likely to be informed, filter questions that
measure familiarity, product use, and past experience should be asked
before questions about the topics themselves.
•Can the respondent remember?
•Avoid errors of omission, telescoping and creation.
•Questions which do not provide the respondent with cues can
underestimate the actual occurrence of an event.
•Can the respondent articulate?
Step 3 Overcoming Inability and Unwillingness to Answer (cont.):
•Minimize the effort required of the respondents.
•Is the context in which the questions are asked appropriate?
•Make the request for information seem legitimate.
•If the information is sensitive:
•Place sensitive topics at the end of the questionnaire.
•Preface the question with a statement that the behavior of interest is common.
•Ask the question using the third-person technique.
•Hide the question in a group of other questions which respondents are willing
to answer.
•Provide response categories rather than asking for specific figures.
Step 4 What Should Not be in Question Wording:
•Question should not assume criteria that are not obvious.
•Question should not be beyond the respondent’s ability or experiences.
•Question should not use a specific example to represent a general case.
•Question should not ask the respondent to recall specific when only generalist will
be remembered.
•Question should not require the respondent to guess a generalization.
•Question should not ask for details that cannot be related.
•Question should not use words that overstate the condition.
•Question should not ambiguous wording.
•Question should not be double barreled.
•Question should not lead the respondent to particular answer.
•Question should not have “loaded” wording or phrasing.
Step 5. Choosing Question Structure:
•Open-ended questions are useful in exploratory research and as opening
questions.
•Use structured questions whenever possible.
•In multiple-choice questions, the response alternatives should include the
set of all possible choices and should be mutually exclusive.
•In a dichotomous question, if a substantial proportion of the respondents
can be expected to be neutral, include a neutral alternative.
•Consider the use of the split ballot technique to reduce order bias in
dichotomous and multiple-choice questions.
•If the response alternatives are numerous, consider using more than one
question to reduce the information processing demands on the
respondents.
Step 6. Choosing Question Wording:
•Define the issue in terms of who, what, when, where, why, and way (the six
Ws).
•Use ordinary words. Words should match the vocabulary level of the
respondents.
•Avoid ambiguous words: usually, normally, frequently, often, regularly,
occasionally, sometimes, etc.
•Avoid leading questions that clue the respondent to what the answer should
be.
•Avoid implicit alternatives that are not explicitly expressed in the options.
•Avoid implicit assumptions.
•Respondent should not have to make generalizations or compute estimates.
•Use positive and negative statements.
Step 7. Determine the Order of Questions:
•The opening questions should be interesting, simple, and non-threatening.
•Qualifying questions should serve as the opening questions.
•Basic information should be obtained first, followed by classification, and,
finally, identification information.
•Difficult, sensitive, or complex questions should be placed late in the
sequence.
• General questions should precede the specific questions.
•Questions should be asked in a logical order.
•Branching questions should be designed carefully to cover all possible
contingencies.
•The question being branched should be placed as close as possible to the
question causing the branching, and (2) the branching questions should be
ordered so that the respondents cannot anticipate what additional
information will be required.
Step 8. Form and Layout:
•
Divide a questionnaire into several parts.
•
Questions in each part should be numbered.
•
The questionnaire should be pre-coded.
•
The questionnaires themselves should be numbered serially.
Step 9.
Reproduction of the Questionnaire:
•
The questionnaire should have a professional appearance.
•
Booklet format should be used for long questionnaires.
•
Each question should be reproduced on a single page (or double-page spread).
•
Vertical response columns should be used.
•
Grids are useful when there are a number of related questions which
set of response categories.
use the same
•
The tendency to crowd questions to make the questionnaire look
be avoided.
shorter should
•
Directions or instructions for individual questions should be placed as close
questions as possible.
to the
Step 10. Pre-testing::
•Pre-testing should be done always.
•All aspects of the questionnaire should be tested, including question content,
wording, sequence, form and layout, question difficulty, and instructions.
•The respondents in the pretest should be similar to those who will be included in
the actual survey.
•Begin the pretest by using personal interviews.
•Pretest should also be conducted by mail or telephone if those methods are to be
used in the actual survey.
•A variety of interviewers should be used for pretests.
•The pretest sample size is small, varying from 15 to 30 respondents for the initial
testing.
•Use protocol analysis and debriefing to identify problems.
•After each significant revision of the questionnaire, another pretest should be
conducted, using a different sample of respondents.
•The responses obtained from the pretest should be coded and analyzed.