Chapter 4
Composing
Questions
McGraw-Hill/Irwin
© 2004 by The McGraw-Hill Companies, Inc. All rights reserved.
Guidelines for Expressing
Questions
• FOCUS
• Every question on a questionnaire should focus
directly on a single, specific issue or topic
• BREVITY
• Short questions are less subject to error on the part
of both interviewer and respondents
• CLARITY
• Demands that virtually everyone interprets the
question in exactly the same way
4-2
Guidelines for Expressing
Questions
• VOCABULARY
• Use words in the core vocabulary of virtually all
respondents
• Limit vocabulary to words the least sophisticated
respondent will know
• GRAMMAR
• Use simple sentences where possible, complex only
when necessary
• Break compound and compound/complex into
simple sentences
• Change long, dependent clauses to words or short
phrases where possible
4-3
Instrumentation Bias and
Error
• Unstated Criteria
• State criteria in the question if it’s at all unclear to
respondents regarding how to respond
• Inapplicable Questions
• The questions must be applicable to all
respondents, based on their own situation
• Example Containment
• Don’t use actual response alternatives as examples
or respondents will over-select them
4-4
Instrumentation Bias and
Error
• Over-Demanding Recall
• Don’t assume respondents will recall their behavior
or feelings over an extended period
• Over-Generalizations
• Seek generalizations only if they represent policies,
strategies, or habitual behavior pattern
• Over-Specificity
• Don't ask for a precise answer unless respondents
will be able to express it exactly
4-5
Instrumentation Bias and
Error
• Over-Emphasis
• Avoid dramatic terms and lean toward
understatement, rather than overstatement
• Ambiguity of Wording
• Every word or phrase must have a plain, common
meaning for everyone in the sample
• Double-Barreled Questions
• Split or modify compound questions, especially
those asking an action and a reason for it
4-6
Instrumentation Bias and
Error
• Leading Questions
• Avoid leading words such as “Don’t you . . .” or in
favor of “Do you . . .” or “Would you . . .”
• Loaded Questions
• Never use questions asking a preference or opinion
and including a socially desirable reason
4-7
Avoiding Instrumentation
Bias
1.
State the criteria for response clearly
2.
Each item must apply to all respondents presented
with it
3.
Use descriptions rather than examples of things being
rated
4.
Don’t tax respondents memories beyond what they
can recall
5.
Don’t ask for generalizations if specifics are needed
and obtainable
6.
Don’t make questions more specific than respondents
are able to answer
4-8
Avoiding Instrumentation
Bias
7.
Avoid being over-emphatic by using less dramatic terms
8.
Use commonly recognized terms to avoid ambiguity of
meaning
9.
Express items and questions in a non-threatening way
10. Split so-called “double-barreled” questions into two or
more items
11. Express items inclined toward both “yes” and “no” to
avoid yea- and nay-saying
12. Rewrite items beginning with a “leading” phrase
13. Never “load” questions by including a powerful reason
to respond in a certain way
4-9
Sources of Response Bias
• Social Desirability
• Response based on what's perceived as socially
acceptable or respectable
• Acquiescence
• Response based on perception of what would be
desirable to the sponsor
• Yea- and Nay-Saying
• Response influenced by a global tendency toward
positive or negative answers
4-10
Sources of Response Bias
• Prestige
• Response intended to enhance the image of the
respondent in the eyes of the others
• Threat
• Response influenced by anxiety or fear instilled by
the question or interviewer
• Hostility
• Response arises from feelings of anger or
resentment engendered by the response task
4-11
Sources of Response Bias
• Auspices
• Response dictated by the image or opinion of the
sponsor, rather than the actual question
• Mental Set
• Cognitions or perceptions based on previous items
influence response to later ones
• Order
• The sequence in which a series is listed affects the
responses to the items
• Extremity
• Clarity of extremes and ambiguity of mid-range
options encourage extreme responses
4-12
Question Formats
Compared
• Unstructured
• Doesn’t clearly show the dimensions respondents
are supposed to use
• Structured
• They clearly show the dimensions respondents are
supposed to use
4-13
Question Formats
Compared
• Unstructured
• Data aren’t directly comparable from one
respondent to the next
• Structured
• Data are directly comparable from one respondent
to the next
4-14
Question Formats
Compared
• Unstructured
• Verbatim responses are time-consuming and
difficult to record
• Structured
• Precoded response alternatives are quick and easy
to record
4-15
Question Formats
Compared
• Unstructured
• Response task is time-consuming and difficult for
respondents
• Structured
• Response task is quick and easy for all respondents
4-16
Question Formats
Compared
• Unstructured
• Relatively quick and easy for the researcher to
compose
• Structured
• Very time-consuming and difficult for the researcher
to compose
4-17
Composing Categorical
Questions
• A categorical item asks a question and is followed by a
series of alternative answers
• The list of categories must be all-inclusive, include
every possible response
• Use mutually exclusive categories with a unique
association with any possible answer
4-18
Composing Categorical
Questions
• There should be more variance in the meaning
between categories than within them
• The alternatives used should cluster together
responses that are similar to one another
4-19
Response Category Size and
Number
• The number of alternatives for an item may range from
two to about six or eight categories
• The categories shouldn't be more "fine-grained" than
required by the sponsors
4-20
Response Category Size and
Number
• Too many, narrow categories are better than too few,
broad ones
• Too many, narrow categories can readily be combined
into fewer, broader ones after data collection
• Too few, overly broad categories can't be split into
more, smaller ones after data collection
4-21
End of
Chapter 4
McGraw-Hill/Irwin
© 2004 by The McGraw-Hill Companies, Inc. All rights reserved.