SURVEY RESEARCH
• Babbie (2011, p.269): “Surveys are a very old
research technique. In the Old Testament, for
example, we find the following:”
• After the plague the Lord said to Moses and to
El-eazar the son of Aaron, the priest, “Take a
census of all the congregation of the people of
Isreal, from twenty old and upward.” (Numbers
26:1-2).
Types of Surveys
•
•
•
•
Telephone
Face-to-face
Mail
Internet (web/email)
• Administered
• Self-administered
Key Concerns/Issues
•
•
•
•
•
Sampling issues
Response rate
Clarity of concepts/constructs/questions
Non-attitudes (door-stop answers)
social desirability bias
Guidelines for Asking Questions
•
•
•
•
•
Respondents must be willing to answer.
Questions should be relevant.
Short items are best.
Avoid negative items.
Avoid biased items and terms.
Guidelines for Questionnaire
Construction
• Be aware of issues with ordering items.
• Include instructions for the questionnaire.
• Pretest all or part of the questionnaire.
Question/Response Wording
•
•
•
•
•
•
Likert Scales
Thermometer ratings
Agree/Disagree statements.
Ordered responses.
Open responses.
Check all that apply.
– open-ended: allows subjects more leeway,
flexibility
– “What is your primary ethnic/cultural
background?”
– requires a content analysis of responses
– close-ended ties respondents’ hands somewhat
– easy to code the data
– always include an “other_____________”
category
Phrasing questions
• Avoid evaluative language
– Phrasing of questions should not imply approval,
disapproval
– Follow-ups should not suggest surprise, liking, disliking,
etc.
• Bad example: Do you think the Democratic
dominated Congress should lift the harsh restrictions
on stem cell research?
• Bad example: Despite its poor track record in crisis
intervention, do you think the United Nations should
intervene in Darfur, Sudan?
Acceptable Response Rates
• 50% - adequate for analysis and reporting
• 60% - good
• 70% - very good
• This is a rule of thumb that may change
depending on time period, method, and
the book you are looking at
Other Issues
•
•
•
•
•
Pre-testing
Follow-up mailings or telephone calls
Training of interviewers
CATI
Legal issues
– Minors
– Autodialing of cell phone numbers
10
Guidelines for Survey
Interviewing
• Dress in a similar manner to the people
who will be interviewed.
• Study and become familiar with the
questionnaire.
• Follow question wording exactly.
• Record responses exactly.
• Probe for responses when necessary.
additional considerations
• gender of researcher/respondent can
influence responses
• ethnicity of researcher/respondent can
influence responses
• medium can influence responses
– face-to-face interview, telephone survey, mail
survey
– anonymous versus non-anonymous
questionnaire
• fatigue: minimize length of surveys
Strengths of Survey Research
• Useful in describing the characteristics of
a large population.
• Make large samples feasible.
• Flexible - many questions can be asked on
a given topic.
Weaknesses of Survey
Research
• Can seldom deal with the context of social
life.
• Inflexible in some ways.
• Subject to artificiality.
• Weak on validity.
Survey Problems
• Literary Digest presidential survey of 1936;
post cards. Over 2 million respondents.
• Predicted Alf Landon over FDR
• Survey predicted the outcomes of the
1916, 1920, 1924, 1928, and 1932
• What went wrong?
– Based on telephone and automobile
registration
– Response rate
Survey Problems
• Hite Sexuality Survey – 70% of women
married 5 years or more are having sex
outside of marriage (4,500 completed
surveys out of 100,000).
• Recorded phone surveys.
• Push & Entertainment polling: political
telemarketing masquerading as a poll
Bush campaign in South Carolina asked the
following:
"John McCain calls the campaign finance
system corrupt, but as chairman of the
Senate Commerce Committee, he raises
money and travels on the private jets of
corporations with legislative proposals
before his committee. In view of this, are you
much more likely to vote for him, somewhat
more likely to vote for him, somewhat more
likely to vote against him or much more likely
to vote against him?"
Video
Political Polling C-SPAN Video
http://www.c-spanvideo.org/program/54099-1