CHAPTER 9
Survey Research
Basics
 In Survey Research, the researcher
selects a sample of respondents and
administers a standardized
questionnaire.
 Main problem:
 How to prepare a questionnaire and how
best to administer it to respondents.
 Surveys are social telescopes, best method
for doing structural research or describing
populations too large to observe.
Attitudes and Dispositions
 Surveys are good guides for
measuring attitudes and dispositions
of a large population.
 The accuracy of surveys or ‘polls’ is in
the representativeness of the sample
and whether questions are formatted
properly or not.
Guidelines for asking Questions
 Operationalization that we talked about
earlier involves asking people questions
to measure what actually indicates the
concept you are measuring.
 The Questionnaire is the instrument (like
a thermometer) designed to MEASURE
accurately/scientifically what we desire
to measure- it generates data that can
then be analyzed.
Question Format
 There are two types of questions
 Open-ended and Closed-ended
 In open-ended questions, the respondent is asked
to provide his or her own answers to the question
(requires coding before processing)
 In closed-ended questions, the respondent is
asked to select an answer from among a list
provided by the researcher (often just need to be
fed into the computer, but a disadvantage is that
they might miss important aspects of responses if
the respondent is restricted by researcher choices)
Closed-Ended Questions
 An example is the Likert Scale
 A format in which respondents are asked
to strongly agree, agree, disagree,
strongly disagree etc- it measures the
extent to which respondents hold a
particular attitude.
 Closed-ended questions have two
structural requirements: The choices
should be 1) Exhaustive 2) Mutually
Exclusive.
 1. Exhaustive choices means that
your choices should include ALL
possible responses that might be
expected.
 2. Mutually exclusive implies that only
one choice can be logically selected
by the respondent, one choice
excludes the possibility of the other
choices.
I. Making Questionnaire
Items Clear.
 Questions and possible responses should be 1)
worded clearly and 2) should be unambiguous in
meaning.
 In other words, the questions and response
categories (if closed-ended) should be clear
enough so as not to elicit further questions from
the respondent.
 E.g. if you are measuring unemployment which you
have conceptualized as being an active search for work
by someone currently not employed, in the past 4
weeks, you have to clearly define what you mean by
the “past 4 weeks” when the period starts and when it
ends. What active search means and what ‘not
employed’ means
II. Avoid Double Barreled
Questions
 Double barreled questions are those
that require one answer to a question
that actually has multiple parts:
 E.g. Should the U.S. abandon its space
program AND spend that money on
healthcare.
 Whenever AND comes in the question beware of
the double barreled question. Also the second part
is LEADING the person to answer in a specific way
by giving you a choice of what is to be done with
the money, so it is choosing your response to the
first part.
III. Respondents Must be
Competent to Answer.
 Asking a 40 year old to remember specific
details about their childhood - the person will
probably not remember accurately, i.e. he or
she is not competent to answer your
question.
IV. Respondents must be Willing to Answer
 People are reluctant to reveal viewpoints that
they think might be a minority opinion or
might be unpopular. In those circumstances
what can you do to get them to give an
honest response?
V.
Questions should be relevant
Know your audience or you might get responses that are
irrelvant.
VI. Short Items are the best
Can be read quickly and answered easily.
VII. Avoid negative items – these can confuse the
respondent, people tend to read over the word “not.”
VIII. Avoid biased items and loaded terms:
Bias in this case refers to a question that encourages a
particular response, e.g. “Don’t you agree that…..?” is a
leading question.
Loaded terms would seek through the avenue of “social
desirability” a certain response from the respondent, e.g.
based on the congruence of a particular response with a
well known culturally positive value.
Questionnaire Construction
The format of a questionnaire is as important as
the wording of the questions.
 The questions should be spread out and
uncluttered.
 Closed-ended questions can be constructed in
numerical form for easy processing
 E.g. Have you ever been a victim of racial
discrimination? 1. Yes 2. No 3. Don’t Know
 Contingency Questions: Answer to one determines if
the other questions are relevant to you.
 E.g. If married answer questions 13-20, otherwise go to
question 21.
 Matrix Questions:
 Same set of answer categories for multiple
questions- saves space and ensures ease of
answering by the respondent.
e.g.
SA
 Q1
 Q2
 Q3
A
D
SD
Uncertain
Ordering of questions
 Earlier questions might affect later answers
 Put the most important questions to your study
first.
Questionnaire instructions:
 Help the respondent make sense of the questionnaire
 Let them know if you want a single answer because
sometimes more than one answer might appeal to
them
 Some questions require special instructions, make
sure those are provided, like instructions that go with
contingency questions.
 Pretest the Questionnaire
 To see how respondents feel and any
errors that stand out
 Also use ‘cognitive interviewing’- ask the
test respondents for their comments on
the questionnaire itself.
 Self Administered Questionnaires- mailed to
respondents, monitor returns (devise a way
to check who returned), send follow up
mailings, only to non respondents, after a
reasonable amount of time.
Response Rate
 Most important in surveys- affects the
representativeness of the sample
 Calculated as :
 The number of people participating in a
survey (divided by) The number of people
selected in the sample
 Also called the completion rate or the return
rate.
Interview Surveys
 Researchers send interviewers to ask
questions orally and record
respondents’ answers




Higher response rate
Decreases the number of ‘don’t knows’
Interviewer can clarify matters
Can observe as well as record
 Disadvantage- the Hawthorne effect.
Guidelines for Survey
interviewing
 Dress somewhat similar to the people you
will be interviewing- dress indicates
“attitudes and orientations.”
 Be familiar with questions and question
specifications should be prepared
beforehand if possible.
 Be familiar with probing techniques (i.e.
requesting the respondent for clarification)
 Interviewer should be trained beforehand in
the above, and in administrative guidelines
and procedures for recording the answers.
 Other survey methods
 Telephone surveys, through random digit dialing
(RDD), not everyone has a land line or a cell
phone- what does that do to the sample
representativeness?
Response rates of telephone interviews: Around
60% (1996 figures)- what does this do to
sample representativeness?
The GSS (General Social Survey) uses personal
interviews and their response rate is around
70%.
Strengths and Weakness of
Surveys
 Make large samples possible. Indispensible
for structural analysis.
 Objectivity and reliability- given
standardized questions, ease of analysis.
Weakness:
1. Decontextual at times, not too good for micro
analysis, misses depth that can create validity
problems
2. Standarization means things get lost in the
‘translation.’
Secondary Data Analysis
 Data processed and collected by one
researcher are reanalyzed for a different
purpose by another. E.g. Sources of
secondary data 1)the GSS (general social
survey) by the NORC (National Opinion
Research Center) at the University of
Chicago. 2) The Roper Center for Public
Opinion Research, at the University of
Connecticut.
 Ethical Issues with Survey Analysis:
 No harm psychological or otherwise (Job,
relationships etc.) should come to the respondents,
since they reveal info about themselves, it can cause
harm.
In class Assignment
 Using the variables that you will
examine in your term paper,
construct a six question survey with
closed ended questions that will help
you answer your “why” question.
 Questions need to be clear, non-leading,
concise and numbered for quantitative
analysis.