Questionnaire Design: Part 1
Slide 1
In addition to all the issues surrounding the creation of good questions, there also are many
issues surrounding the creation of good questionnaires. The goal of this lecture is to discuss the
issues related to overall questionnaire design.
Slide 2
It’s important to recognize that questionnaires can be written to study a large range of problems.
The example shown on this slide—from a study conducted by one of my professors at Purdue—
concerns willingness to donate human body parts, either before or after death, and the extent
that people might be price sensitive. In other words, would some people be willing to donate
certain body parts before death in return for money. Although this sounds somewhat macabre,
at the time of the study there was some interest in establishing a national organ bank, and the
issue was how to stock it. If people were price sensitive, then that suggests one alternative for
stocking human body parts. As such, it’s a marketing problem; hence the questionnaire. Be
sensitive to the huge range of research problems that can be addressed via survey research.
Slide 3
It may seem that designing a good questionnaire is a relatively simple and straightforward task.
Although a good questionnaire looks as if it was easy to design, in fact it’s the result of long and
painstaking work, often by several researchers. Unlike that old joke about a camel being a horse
designed by a committee, a good questionnaire typically requires several authors because each
author will be blind to certain limitations or idiosyncrasies in formatting and wording. Several
people working singly and jointly typically can eliminate many such shortcomings.
Slide 4
As with the design of good questions, there are only guidelines, rather than immutable rules, for
designing good questionnaires. This lecture is meant to provide a critical but non-exhaustive set
of those guidelines.
Slide 5
Here are a few basic decisions that researchers must make before designing questionnaires.
After determining the scope of their study, researchers must consider the specifics of each
question and how it should be phrased. They must consider the sequence in which the
questions should be arranged (which I’ll discuss in more detail later). They must select a
questionnaire layout suitable to the research objectives. Finally, they must revise their
questionnaires based on their pretest results.
Slide 6 (No Audio)
Slide 7
Screener questions are designed so that only qualified respondents participate in surveys.
Here’s a rather extensive example of the screener questionnaire for someone participating in a
study of soft drink consumption. Notice that the questionnaire will screen out people who are
knowledgeable of the research domain and the researcher. Such people will tend to be non-
Page | 1
representative of the general population. The screener also attempts to qualify respondents, in
that it makes little sense for this study to survey people who don’t consume soft drinks.
Slide 8 (No Audio)
Slide 9
In contrast to screener questions, which are meant to ensure that only qualified respondents
participate in a study, filter questions are designed to ensure that respondents receive the
correct questionnaire out of multiple questionnaires they might receive. In this example, there
are three types of consumers the researcher wishes to query. Each group can provide useful
but different information. From people who are aware of the product but have never tried it, a
researcher can learn the reasons they are aware of the product but have chosen not to use it.
From people who have tried the product but are no longer users, a researcher can learn the
reasons they stopped using the brand. From current users, a researcher can learn the best
ways to retain these people as customers by discovering what they like and dislike about the
brand. The filter question for this study ensures the appropriate questionnaire is administered to
aware non-triers, current users, or triers who’ve rejected the product.
Slide 10 (No Audio)
Slide 11
Researchers use skip patterns to qualify respondents for answering subsequent questions
based on their answers to one or more previous questions. Here’s an example of a skip pattern
for a crème rinse or hair conditioner product. “Do you usually use a crème rinse or hair
conditioner on your child’s hair? If ‘no’, then skip to question 5; if ‘yes’, then answer question 4b.
Asking non-users how often they use a crème rinse or hair conditioner is nonsensical.
Slide 12
Here’s another example of a skip pattern: “Have you heard of Brand X? If ‘yes’, answer
questions ‘A’ and ‘B’ below; if ‘no’, then skip to the next question.” Asking people who’ve never
heard of Brand X how frequently they used it in the last 30 days or 6 months is nonsensical.
Slide 13
It’s relatively easy to follow skip patterns properly for computer-assisted telephone interviews.
The computers can be programmed so that only the proper questions are asked. In other words,
skip patterns always are followed correctly. Skip patterns are a bigger problem with selfadministered questionnaires. Despite efforts to ensure that respondents read instructions and
only answer relevant questions, they often answer questions that do not pertain to themselves.
For a self-administered questionnaire, multiple visual elements can increase the likelihood that
respondents see the need to skip questions that don’t pertain.
Slide 14
All questionnaires—even self-administered ones—simulate conversations between researchers
and respondents. As a conversation, questionnaires should have a logical flow. In addition, it’s
best that conversation runs from general to specific. Starting respondents out with very specific
questions can be somewhat jarring and may not appropriately prepare them, from a memory
perspective, to answer questions properly. Starting them with more general questions—to warm
them up and prepare them to answer more specific questions—is preferred. Also, it’s best to
Page | 2
ask sensitive questions towards the end of a questionnaire. At that point, respondents have
bought into cooperating, so they’re unlikely to either break off the interview or refuse to answer
those questions. If sensitive questions appear toward the beginning of a questionnaire, then
respondents may become suspicious and discontinue the interview process before completion.
Slide 15
Here’s an example of going from general to specific questions, which sometimes is referred to
as a funnel organization. A funnel is broad at the top and narrow at the bottom, which is
analogous to going from general to specific. The first question reads “Have you heard or seen
any advertisements for automobiles recently? If yes, then answer question #2, which reads
“Were any of these advertisements for foreign automobiles?” If yes, then answer question #3,
which reads “Were any of the advertisements on television?” If yes, then answer question #4,
which reads “Were any of these advertisements for a certain kind of car?” Finally, if yes, then
answer question #5, which reads “What did the advertisement say?” This question sequence
moves from a general question about auto ads to a specific question about the content of a
television ad for a specific auto.
Slide 16
The top of this slide provides a sample question that’s far too complex to begin a questionnaire.
Asking respondents, right off the bat, to think about all the things that makes their community a
pleasant place to live is far too complex. It’s best to ease respondents into an interview. We
want to ask simpler questions initially and then work towards more complex questions. As an
alternative to the first version of the question, we might consider the set of questions towards
the bottom of this slide: “Think about this community and how would you rate it as a place to
live; excellent, good, fair, or poor?” That’s relatively straightforward and easy to answer. The
second question, “How long have you lived in the community?” to some extent qualifies
respondents to either continue on or to terminate the interview.
Slide 17
How should we organize questionnaires? This slide provides a good summary. We’d start with
screener or filter questions to ensure that respondents were qualified and that the correct
questionnaire was administered to the correct respondents (if there are multiple questionnaires).
The first several questions should be warm-up questions that begin respondents thinking about
the problem area. The first-third questions—referred to as transitional questions—should be
somewhat simple and straightforward, the next set of questions should be the most difficult and
complex, and the last section should contain the personal questions typically used for
classification purposes (such as age, income, occupation, marital status, number of children in
household, and home ownership). People typically are reluctant to answer personal questions
because of privacy concerns, so they should be saved for last, when trust has been developed.
Slide 18
There are numerous layout considerations when designing questionnaires. None of these is
likely to boost response rates, but they may be important depending on research needs.

Using a booklet, instead of a series of sheets stapled in the upper left-hand corner, may
make questionnaires appear more professional.
Page | 3

More questions can be included per page in a booklet format than in the stapled-in-theupper-left-hand-corner format. As a result, the inbound and outbound postage for selfadministered mail questionnaires may be reduced.

Color coding may help with organizing multiple versions of a questionnaire. As noted in a
different lecture, eliminating order bias affects requires multiple question versions with
items sequenced differently. By having questionnaires color coded, there’s less of a
chance of entering the wrong number into the database. (For the same question, a ‘3’
response on the blue questionnaire and a ‘5’ response on the yellow questionnaire are
the same as a ‘1’ response on the white questionnaire; all three responses would be
coded identically in the database.)

Numbering questionnaire items is somewhat arbitrary. I recommend not numbering
items because numbering them provides respondents an additional reason for
terminating the interview process before it begins: they believe the questionnaire is too
long and will take too much time and effort to complete.

The number of questions per page is an issue. It’s important that questionnaires be
aesthetically appealing and also readable for older people whose vision may be worse
than 20/20. The trade off here is questions per page versus number of pages. More
pages make questionnaires appear longer, which may discourage some respondents.
Conversely, fewer pages with a smaller typeface may make questionnaires unreadable
to people with poor vision (especially the elderly); as a result, they’ll opt not to respond.

Pre-coding questionnaires is an issue. I recommend pre-coding questionnaires because
they ease computer data entry efforts.
Slide 19
Including clear and concise instructions is important to ensure that respondents properly answer
questions, especially to self-administered questionnaires. It’s often difficult, when researchers
receive a questionnaire from a respondent, to know if that respondent followed the instructions
and answered the questions as intended. The clearer and less ambiguous the instructions, the
more likely that occurred.
Slide 20
Because respondents often are less than highly motivated regarding the research problem, it’s
best to put instructions exactly where needed as opposed to providing a lengthy list of
instructions to introduce the questionnaire. If an instruction is needed on page 5 but it appears
on page 1, then it’s less likely that respondents will attend to it.
Slide 21
To make questionnaires more conversational, special instructions should be included as part of
the question and not as a free standing entity. Here’s an example. At the top of the slide, the
original question reads “How many months have you worked at your current job? Fill in blank
number of months. Then, please be as specific as possible in answering the next question
including any area of specialization….” The revised version reads “How many months have you
worked in your current job? Provide the number of months.” Then, “What kind of work do you do
in your current job? Please be as specific as possible in answering. Include any areas of
specialization….” By including the instruction as part of the question, the questionnaire becomes
more conversational, which encourages respondents to offer fuller answers.
Page | 4
Slide 22
This slide illustrates why questionnaire organization, as well as the instructions and the path by
which respondents proceed through a questionnaire, should be as clear as possible. The top of
this slide shows something like the type of questionnaire fielded by government entities. It’s
difficult to follow, difficult to read, and definitely unappealing. In contrast, the bottom of this slide
shows a superior form of the same question.
Slide 23
This slide illustrates that response problems can be reduced through careful questionnaire
design. Although the question set is relatively complex, the arrows help to indicate what
responses go where and upon what those responses should be based. For example, question
‘5A’ is a percent, question ‘6’ requires a number of hours, and questions ‘7’ and ‘8’ also ask for
percents. The way this questionnaire is constructed, it’s clear what number goes where. It’s
easy, given the size of the boxes, for respondents to write those numbers in clearly so that the
person responsible for entering this data into a computer will be able to read those numbers. It’s
also possible to provide relatively detailed questions so that respondents can sense what
number belongs in each cell on the right-hand side of the questionnaire.
Slide 24 (No Audio)
Slide 25
The next four slides address typeface and blank space issues. This slide shows that it’s
important to maintain simplicity, regularity, and a consistent figurative ground format, to ease
respondents’ efforts. The problematic example at the top shows so many inconsistently used
bolded and specialty characters that it’s a chore to read these three questions. In contrast, the
revision at the bottom shows a consistent typeface and clean appearance, which makes the
questions easy to read and answer.
Slide 26
Another useful guideline for typeface and blank space is to increase the size and brightness of
the visual elements to emphasize the order for reading questionnaires. The top of this slide
shows a poor example in which both the instructions ‘start here’ and the question itself are in
small font and not bolded, but the choices and the ‘please specify’ are in far larger fonts and
bolded. The alternative at the bottom of this slide—in which the instruction ‘start here’ is all
uppercase and bolded, the question is bolded and in a smaller but upper- and lower-case font,
and the choices are not bolded and in a still smaller font—is superior.
Slide 27
A third rule of thumb for typeface and blank space is to use a different bolding or case or italics
or underline to highlight key words in questions to be certain that respondents noticed those
words, but not to use so many highlighting techniques that it becomes visually distracting. The
most problematic example appears at the top of the slide, with all uppercase for ‘last three
weeks’, underline for ‘experience pain’, and italics for ‘call a doctor’. That’s annoying to read.
Alternatively, but still a problem because of continuing excessive underlines and emphasis, is
the second example. The best revision is at the bottom of the slide, which stresses the key
notion that respondents limit their response to the ‘last three weeks’. Emphasis or highlights
should be used consistently, only when they are important, and in a way that will not annoy
respondents.
Page | 5
Slide 28
A fourth typeface and blank space guideline is to change spacing and elements similarity to
identify appropriate groupings of visual elements. The problematic example at the top of the
slide is so cluttered that it’s difficult to read; also, some of the choices are denoted by numbers
and other are denoted by symbols, like a diamond or a heart shape. This is a poor way to
structure two questions. Instead, the revised version is numbered consistently, the questions
are bolded similarly, and the choices (for question #1) and words following the blank line (for
question #2) are in the same non-bolded upper- and lower-case. By keeping similar elements
formatted similarly, respondents’ task is simplified and they will tend to see all questions and
respond to them appropriately.
Slide 29 (No Audio)
Slide 30
The placement of answer categories for questions on either on the left- or right-hand side is
arbitrary. However, that placement should be consistent throughout a questionnaire.
Slide 31
This formatting issue is a problem with many surveys. At the top of the slide, there’s a question
and a series of four choices, and then there’s another question and a series of the same four
choices. Questionnaires with Likert-type attitude items tend to use the same 6- or 7-point
‘strongly agree’ to ‘strongly disagree’ scale. Rather than format like the top question #7 and #8,
a better format is shown at the bottom of the slide. In the revised version, there’s this initial
question: “Do you consider each of the following to be a serious problem, a moderate problem,
small problem, or not a problem in this community?” Under this question is a list of possible
problem areas, the degree to which each may be a problem, and an instruction to circle one
answer for each possible problem. (On the left-hand side a list of issues—a lack of rental
housing, poor road and street repair—and then to the right is the extent to which that issue is a
problem—serious, moderate, small, and not a problem.) This revised format shows a far more
compact way to present the questions that appear at the top of this slide.
Slide 32
To avoid confusing respondents, it’s best to run all scales in one direction—for example, from
negative to positive—throughout a questionnaire. That way, they’re more likely to provide an
answer consistent with their true opinion.
Slide 33
I’ve been guilty of not doing this in some of my questionnaires. On many occasions, I’ve used
the ‘check all that apply’ question format. The problem with this format is that it’s impossible to
know if people meant not to check a choice or if they meant to but failed to check it or didn’t see
it. Positive confirmation whether or not people intended to check or not to check a choice is
preferred; hence, the three revisions to the question at the top of the slide.
Slide 34
For questions with many response options, self-administered mail questionnaires work better
than telephone questionnaires. If a telephone interviewer reads a response list with more than
four or five choices, then it’s likely that respondents will forget the initial choices by the time
Page | 6
they’ve heard the last one. In contrast, all the choices are readily available with a selfadministered mail questionnaire, so it’s far easier to pick among a larger set of options.
Slide 35
Because it may be necessary to ensure that a self-administered mail questionnaire was
answered by the intended respondent, and that certification process typically occurs by
telephone, uni-mode formatting (suitable for either mail or telephone query) of questions is
preferred. This slide shows the traditional mail and telephone formats for a question, the latter
including more extensive instructions and both ‘no opinion’ and ‘refuse to answer’ choices. The
bottom of the slide shows a uni-mode formatted version in which ‘refused’ is not an option and
the choices are embedded in the question.
Slide 36
As I mentioned at the beginning of this lecture on questionnaire format, I recommend using precoded questionnaires because it will ease subsequent data entry into a computerized data
base. Here’s one example of pre-coding. For questions 1B and 1C, respondents are instructed
to check a box, but there are numbers assigned immediately to the right of those boxes.
Slide 37
The problem with failing to pre-code questions is that it increases the time needed to enter data
into the computer. For example, in question 3A, the numbers to the right of each cell make
entering the correct code easy. Yes, the data entry person could think “The respondent checked
‘to improve my game’, which is a 4,” but that requires additional thought that will slow the data
entry process, especially for lengthy questionnaires and many respondents. Each additional
moment needed to reflect on the number that corresponds with a checked box will greatly
increase data entry time and effort.
Slide 38 (No Audio)
Slide 39
What appears on this and the next slide are two more examples of what I mean by pre-coding a
questionnaire.
Slide 40 to Slide 43 (No Audio)
Page | 7