MSS 905 Methods of
Missiological Research
Survey Research
Survey Research
• The most widely used data gathering
technique in the social sciences
• Can be well made and appropriate, but
can also be poorly constructed and used
inappropriately
• Produces information that is inherently
statistical in nature
• Appropriate for self-reported beliefs or
behaviors (see box 10.1, p. 273); not
“why?” questions
Types of Questionnaires
• See Adler and Clark (2008), Chapter 9, p.
232-239
Good Question Writing
• Good surveys are smooth, nonthreatening, unambiguous, clear
• Question writing is more an art than just a
science
10 Things to Avoid
1. Avoid jargon, slang and abbreviations
•
Use the language of TV and newspapers
2. Avoid ambiguity, confusion and
vagueness
•
•
•
“What is your income?”
Monthly? Annual? Before or after taxes?
From salary or all sources?
Accurate response categories
10 Things to Avoid
3. Avoid emotional language and
prestige bias
•
•
Some words carry emotional baggage,
“What do you think about a policy to pay
murderous terrorists who threaten to
steal the freedoms of peace-loving
citizens?”
Prestige bias: associating a statement
with a person or group that enjoy
prestige, “Most doctors say that cigarette
smoke causes lung disease for those
near a smoker. Do you agree?”
10 Things to Avoid
•
“Do you support the president’s policy
regarding Uzbekistan?” (very few have
heard of Uzbekistan and will respond on
the basis of their attitude toward the
president)
4. Avoid double-barreled questions
•
•
Two or more questions in the same
question
“Do you have pension and health
insurance benefits at your job?”
10 Things to Avoid
5. Avoid leading (“loaded”) questions
•
•
•
•
Suggesting the answer in the way you
pose the question
“You go to church, don’t you?”
“Should the church spend even more
tithing income on interior decorating?”
“Should the church allocate more of
tithing income to save lost souls in our
community?”
10 Things to Avoid
6. Avoid asking questions that are
beyond people’s capabilities
•
•
•
People can’t always recall past details
“How did you feel about your Sunday
School teacher when you were six years
old?”
Asking church visitors, “What percentage
of our tithing income should we be
investing in our building fund?”
10 Things to Avoid
7. Avoid false premises
•
•
You cannot begin with a premise which is
debatable, and then ask someone a
question about it
“Our sanctuary is already open too early
on a Sunday morning. Do you want it to
be open an hour earlier or an hour later?”
8. Avoid asking about future, distant
intentions
•
•
“If we opened a branch church in the
north, would you attend it?”
Stay in the present
10 Things to Avoid
9. Avoid double negatives
•
If you disagree with this question,
“Students should not be required to take
a comprehensive exam at the end of the
course”, then you don’t agree they
should not
10. Avoid overlapping or unbalanced
response categories
•
•
Categories should be mutually exclusive
“Are you working or unemployed?”
10 Things to Avoid
•
Unbalanced: “Which of the Democratic
candidates do you support, Hillary
Clinton or one of the others?”
Some Tips
1. Use a Likert-scale to gain responses
2. I prefer a 5-pointer
3. “Anchor” the two extreme positions (1
and 5) with clear opposites
•
•
“To a great extent” vs “Not at all”
“Strongly agree” vs “Strongly disagree”
4. In the question, make a clear statement
to which the person needs to respond
5. Mix the response directions
Getting Good Answers
1. Be aware of social desirability bias
– Respondents give the socially desirable
response (politically, ethically, etc.)
2. Avoid questions that test knowledge
– Large majority cannot answer elementary
geography questions or id important political
documents
– Respondents do not want to appear ignorant
Topic
% Very
Uneasy
Getting Good Answers
3. Make use of contingency questions
– “if – then”
– Saves time of those who do not qualify
4. Open-ended versus closed questions
– Open is unstructured, respondents can give
any answer they wish
– Closed restricts the options of the
respondent
– Both have benefits and disadvantages (box
10.5)
Types of Surveys: Mail and Selfadministered
• Cheap
• No need for
presence of
researcher
• Can be completed
when convenient
• Anonymity
• Avoid interviewer
bias
• Low response rates
• Cannot control
conditions when
questionnaire is
completed
• No one to clarify
• Someone else may
complete
• Incomplete questions
• Ill-suited for illiterate
Types of Surveys: Telephone
Interviews
• 95% can be reached
• Can be done long
distance as well
• High response rates
• Cheaper than face-toface, same benefits
• Computer-assisted
telephone interview
(CATI) & Interactive
Voice Response (IVR)
• Higher cost
• Limited interview
length
• Not all have
telephones
• Calling at
inconvenient times
• Open-ended
questions are difficult
to use
Types of Surveys: Face-to Face
Interviews
• Highest response
• Highest cost
rates
• Training, travel,
• Permit longest format
supervision,
personnel deployment
• Interviewer can
observe non-verbals • Interviewer bias
potential
• Can use visual aids
• All types of
questions, complex
issues, extensive
probing possibilities
Types of Surveys: Web Surveys
• By 2004 75% of US
homes connected to
Internet
• Very fast, inexpensive
• Flexible: visual or
audio inputs
• Cuts out the “data
entry” step (and
costs)
• International potential
• Could exclude older,
less-educated, lower
SES and rural people
• Sampling problems are
formidable
• Privacy and security
• Design issues: question
format, drop-down
menus, progress
indicators, move back