Organizing the Questions
1.
Questionnaire Sections Common to (Nearly All) Surveys
2.
What questions should the questionnaire begin with?
3.
Grouping Questions into Sections (and Subsections)
4.
Questionnaire Length and Respondent Burden
Questionnaire Sections Common to
(Nearly All) Surveys

Introduction

Respondent Selection

Substantive Questions

Background Questions

Post-Interview Questions
“The survey introduction serves multiple
purposes. It provides a compact preface to
the survey, telling the participants the
subject, purpose, sponsorship, and a few
other details. It gives the prospective
respondent sufficient information about the
study to satisfy the needs of informed
consent. Most of all, it elicits participation.”
(Czaja & Blair [2005], p. 88)
Introduction
1.
2.
3.
Provides Information About the Study (subject, purpose, etc.)
Allows for Informed Consent
“Elicits Participation” (recruitment)

Phrasing Needs to Be Time Efficient & Concise
(i.e., for both written & oral)

Want Respondents to Feel that the Survey Project is important,
serious, etc. and therefore worthy of their time

Relationship Between Overall Rates of Participation, Possible
Response Bias, Low Survey Validity and/or Reliability
(Czaja & Blair [2005], p. 88-89)
Respondents have these kinds of questions
about surveys…
“What is the study about?”
“Who is conducting it?”
“Who is the sponsor?”
“Why is the study important?”
“What will be done with the study results?”
“Why is the study pertinent to me?”
“Why was I selected?”
This is NOT a comprehensive list; many additional questions may be
asked about the project, especially if the questions are of a sensitive
nature.
(Czaja & Blair [2005], p. 90)
Substantive Questions
“This is the heart of the questionnaire, accounting for
the majority of the data and, hence, the majority of
our effort and costs.” (Czaja & Blair [2005], p.86)

Content-related questions, questions of substance,
etc.

Questions most directly related to our research
questions & scientific hypotheses
Example:
Agricultural Injury Survey
Substantive Questions START: p. 3
http://www.cdc.gov/niosh/childag/pdfs/appendixfull.pdf
Background Questions


Demographic questions
E.g., sex, gender, sexual orientation, age, marital
status, race, nationality, ethnicity, income, employment
status, place of residence, etc.
Reasons for Inclusion…
1. Related to Our Research Question and/or Scientific Hypothesis
2.
Help Determine the Representativeness of Our Sample
3.
Useful for Stratification of the Data File and/or for Developing
Weights, Coefficients, etc. During Data Analysis
(Czaja & Blair [2005], p. 86)
Example:
CDC Mass Casualties Event Survey
Demographics; pp. 1-2
http://www.bt.cdc.gov/masscasualties/pdf/mhsurvey-instrument.pdf
Post-Interview Questions
Intended for Respondents, Interviewers, or Both
Will frequently ask about…

Difficulty and/or Lack of Clarity of Items on the Questionnaire

Reason/s Why Certain Items Were Not Answered
Why the Respondent Hesitated Before Answering)

Person/s to Contact (e.g., should respondents move, become very
ill, or otherwise drop out of contact)
(Czaja & Blair [2005], pp. 86-87)
(or
Example:
HINTS 2005
(Health Information National Trends Survey)
Debriefing Questions; p. 53
http://hints.cancer.gov/instrument.jsp
Importance of First Impressions…*
“…we do have a lot of evidence that
most refusals occur at the introduction
or during the very first questions…it
may be helpful to think of the
respondent as making an initial
tentative decision to participate (or at
least to keep listening)…”
(Czaja & Blair [2005], p. 94)
1st Question Should Be…

Relevant to the Central Topic

Easy to Answer

Interesting

Applicable To and Answerable By Most Respondents

Closed Format
(Czaja & Blair [2005], pp. 94-95)
Avoid Asking Sensitive Questions Early
If they must be asked then…

“Ask as few sensitive questions as absolutely necessary to
establish the respondent’s membership in the target population.”

“Make clear to respondents why it is necessary to ask such
questions.”

“If costs permit, consider inserting an opening nonsensitive
‘buffer’ question or two before the screening item(s) to establish
some rapport with the respondent.”
(Czaja & Blair [2005], p. 96)
Sensitive Issues
What strategies can we employ?
Fowler (2002; pp.98-100) offers this advise:
1.
“Minimize a sense of judgement; maximize the
importance of accuracy.”
2.
“Use self-administered data collection procedures.”
3.
“Confidentiality and anonymity.”
Question Order or Groupings
Some Relevant Issues Include…
Avoidance of Survey Termination
Internal Logic
Smooth Progression or Flow Through the Instrument
Respondent Sense of the Ease & Speed of Completion
(Czaja & Blair [2005], pp. 95-96)
Questionnaire Length and Respondent Burden

Research on Specifics of Respondent Motivation is Lacking

Respondents Need Some Reason/s to Participate in a Survey
E.g., Financial Compensation; Interesting Study; Important Topic;
Potential for Action Taken Following the Project (Dillman [2000];
Groves [1989])

Survey Length (or its perception) is Very Important
(Czaja & Blair [2005], pp. 99-100)
Guidelines for Formatting Such That Survey Length:
Difficulty Appear Low

“Limit instruments to six to eight pages.”

“Precode response categories by assigning a number to each
possible answer for the respondent to circle.”

“Space the categories so that it is easy to circle one response
without touching an adjoining one; arrange the categories
vertically, one under another, rather than horizontally spread
across the page.”
(Czaja & Blair [2005], pp. 100-101)
Guidelines for Formatting Such That Survey Length:
Difficulty Appear Low

“Provide simple instructions of no more than two sentences
describing how to answer questions; for example, ‘Please
circle the number of your answer unless otherwise instructed.’”

“Use a different typeface for questions, response categories, and
transitions or section headings.”

“Whenever possible, use arrows to indicate skip instructions.”
(Czaja & Blair [2005], pp. 100-101)
Example:
NAMCS 2008
(National Ambulatory Medical Care Survey)
Extremely Short – More of a FORM than a Survey
http://www.cdc.gov/nchs/data/ahcd/NAMCS_30_2008.pdf