DIE 4564 Research Methods
Qualitative Research- Part 2
Tools and Techniques
Chapter Outline
•
•
•
•
•
•
•
•
•
•
•
Topics Appropriate for Survey Research
Guidelines for Asking Questions
Questionnaire Construction
Self-Administered Questionnaires
Interview Surveys
Telephone Surveys
Online Surveys
Comparison of the Different Survey Methods
Strengths and Weaknesses of Survey Research
Secondary Analysis
Ethics and Survey Research
Topics Appropriate for Survey
Research
•
Descriptive, exploratory, and explanatory
•
Units of analysis = respondents

•
Respondents – a person who provides data for
analysis by responding to a survey questionnaire.
Large samples, original data, measuring attitudes
and orientations
Guidelines for Asking Questions
•
Questionnaire - a document containing questions
and other types of items designed to solicit
information appropriate for analysis.

Open-Ended Questions – questions for which the
respondent is asked to provide his/her own answers.

Closed-Ended Questions – survey questions in which
the respondent is asked to select an answer from a list
provided by the researcher.
Guidelines for Asking Questions
•
•
•
•
•
•
•
•
•
Select appropriate question forms.
Make items clear.
Avoid double-barreled questions.
Respondents must be competent to answer.
Respondents must be willing to answer.
Questions should be relevant.
Short items are best
Avoid negative items.
Avoid biased items and terms.
Guidelines for Asking Questions
•
Bias – that quality of measurement device
that tends to result in a misrepresentation of
what is being measured in a particular
direction.
Guidelines for Asking Questions
•
Review Question
 Whenever we ask people for information,
they answer through a filter of what will
make them look good. This filter is called
_________.
Guidelines for Asking Questions
•
Review Question
 Whenever we ask people for information,
they answer through a filter of what will
make them look good. This filter is called
social desirability.
Questionnaire Construction
•
General Questionnaire Format



Uncluttered
One question per line
Consistent format
Questionnaire Construction
•
Formats for Respondents
Questionnaire Construction
•
Contingency Question – a survey question intended for only
some respondents, determined by their responses to some
other question.
Questionnaire Construction
•
Contingency Question – a survey question
intended for only some respondents, determined
by their responses to some other question.
Questionnaire Construction
•
Matrix Questions
Questionnaire Construction
•
Ordering Items in a Questionnaire
 Appearance
 Open-Ended or Closed-Ended First?
Questionnaire Construction
•
Questionnaire Instructions
 Introductory comments and clear
instructions
•
Pre-testing the Questionnaire
Questionnaire Construction
•
A Composite Illustration
Questionnaire Construction
•
Review Question
 Which of the below is not a characteristic of
matrix questions?
A. They use space efficiently when asking
multiple questions.
B. They must all have the same response
categories.
C. They are less valid measures of concepts.
D. They will speed the response time for
respondents.
Questionnaire Construction
•
Review Question
 Which of the below is not a characteristic
of matrix questions?
The answer is C. Matrix questions use
space efficiently when asking multiple
questions, have the same response
categories, and will speed the response
time for respondents. They are not less
valid measures of concepts.
Self-Administered Questionnaires
Questionnaires in which respondents are
asked to complete the questionnaire by
themselves
• Mail Distribution and Return

•
•
Why do people not return questionnaires?
Monitoring Returns
Follow-Up Mailings
Self-Administered Questionnaires
•
Response Rate – the number of people
participating in a survey divided by the
number selected in the sample.
 Ideal = higher than 70%
•
Other kinds of rates:
 Cooperation rates
 Refusal rates
 Contact rates
Self-Administered Questionnaires
•
Compensation for Respondents
 Effect on response rates
•
A Case Study
Self-Administered Questionnaires
•
Review Question
 The proportion of all potentially eligible
cases in which the respondent refuses to
be interviewed or breaks off an interview is
called the ________ rate.
Self-Administered Questionnaires
•
Review Question
 The proportion of all potentially eligible
cases in which the respondent refuses to
be interviewed or breaks off an interview is
called the refusal rate.
Interview Surveys
•
Interview – a data-collection encounter in which
one person (interviewer) asks questions of another
(respondent).
•
The Role of the Survey Interviewer
Interview Surveys
•
Guidelines for Survey Interviewing
 Appearance and demeanor
 Familiarity with the questionnaire
 Following question wording exactly
 Recording responses exactly
 Probing for responses
Probe – a technique employed in
interviewing to solicit a more complete
answer to a question.
Interview Surveys
•
Coordinate and Control
 Training
General guidelines
How to handle difficult situations
Practice interviews
Interview Surveys
•
Review Question
 Which of the below is not one of the
advantages of using a survey interviewer?
A. Increased response rates
B. Decreased number of “don’t knows”
and “no answers”
C. Better understanding of questions
D. Increased refusal rate
Interview Surveys
•
Review Question
 Which of the below is not one of the advantages
of using a survey interviewer?
The correct answer is D. The use of survey
interviewers increases response rates,
decreases “don’t knows” and “no answers,”
and assures the respondents better
understand the questions. Using a survey
interviewer does not increase the refusal rate.
Telephone Surveys
•
Advantages


•
95.5% of households have a telephone
Time and money
Disadvantages


Unlisted phone numbers
Cell phones
Telephone Surveys
•
Random-Digit Dialing (RDD) – a sampling
technique in which random numbers are selected
from within the range of numbers assigned to
active telephones.
Telephone Surveys
•
Computer-Assisted Telephone Interviewing
(CATI) – a data-collection technique in which
a telephone-survey questionnaire is stored in
a computer, permitting the interviewer to
read the questions from the monitor and
enter the answers on the computer keyboard
Telephone Surveys
•
Response Rates in Interview Surveys
Telephone Surveys
•
Review Question
 This type of telephone survey is more
cost-effective because it cuts out the labor
costs of hiring interviewers and it
automatically prepares data for analysis:
_______.
Telephone Surveys
•
Review Question
 This type of telephone survey is more
cost-effective because it cuts out the labor
costs of hiring interviewers and it
automatically prepares data for analysis:
CATI.
Online Surveys
•
•
•
•
•
•
•
•
DO use consistent wording.
DO use simple language.
DON’T force excessive scrolling.
DO offer to share select result with respondents.
DO plan time and day of initial mailing.
DO be aware of technical limitations.
DO test incentives, rewards, and prizes.
DO limit studies to less than 15 minutes.
Online Surveys
•
Review Question

True or False: The relative youth of online surveys
makes them fertile ground for innovation and
experimentation.
Online Surveys
•
Review Question
 True: The relative youth of online surveys
makes them fertile ground for innovation and
experimentation.
Comparison of the Different Survey
Methods

Self-Administered Questionnaires





Interview Surveys




Fewer incomplete questionnaires
More effective for complicated questionnaires
Face-to-face is more intimate
Telephone Surveys


Cheaper and faster than face-to-face interviews
National is the same cost as local mailings
Requires small staff
More willingness to answer controversial items
Cheaper and more time efficient
Online Surveys

Available software and websites
Examples of Methods for
Collecting Data
Comparison of the Different Survey
Methods
One advantage of _______ surveys is that
they work better for questionnaires with
complicated contingency questions.
A. Self-administered
B. Interview
C. Telephone
D. Online
Comparison of the Different Survey
Methods
 The
correct answer is B. Interview
surveys are better for complicated
questionnaires.
Recruiting Methods
Once a data collection method has been selected,
the next step is to decide on an effective method
for recruiting members of the sample population to
be participants in the study.
The best method for initiating contact with potential
participants is often related to the intended data
collection method.
Recruiting Methods
Participation rates will likely be higher if:
• Recruits understand the importance and value of
the research project
• Researchers provide multiple invitations and
opportunities to participate, and make participation
as easy as possible
• Incentives (such as small gifts) are offered
Data Recording Methods
A decision must also be made about how
responses will be recorded and when they will be
entered into a computer database. There are two
basic options:
• Record the responses on paper and to enter
them into a computer database later
• Have interviewers or participants enter
responses directly into a database
Methods for Collecting and
Recording Survey Data
Training Interviewers
The interview process should be the same for all
participants in a study, whether they are being
interviewed in-person or by telephone interview.
Uniformity is easiest to accomplish when all
interviewers attend training sessions where they
have an opportunity to practice their interview
skills.
Strengths and Weaknesses of
Survey Research

Strengths
 Useful in describing large populations
 Surveys are flexible
 Standardized questions
Strengths and Weaknesses of
Survey Research

Weaknesses
 Round pegs in square holes
 Seldom deal with context of social life
 Inflexible
 Artificial
 Weak on validity
Strengths and Weaknesses of
Survey Research
By presenting all subjects with a standardized
stimulus, survey research goes a long way
toward eliminating _______ in observations.
A.
B.
C.
D.
Invalidity
Unreliability
Validity
Reliability
Strengths and Weaknesses of
Survey Research

The correct answer is B. Because all
respondents react to the same, standardized
stimulus, survey research reduces
unreliability.
Secondary Analysis
•
Secondary Analysis – a form of research in which
the data collected and processed by one
researcher are reanalyzed by another.

•
General Social Survey
Secondary analysis provides social researchers
with an important option for “collecting” data
cheaply and easily but at a potential cost in validity.
Secondary Analysis
Which of the below is not an advantage of
secondary analysis?
A. It’s cheaper than original surveys.
B. It’s faster than original surveys.
C. It has greater validity.
D. It has the enhanced possibility of metaanalysis.
Secondary Analysis
•
Review Question
The correct answer is C. Secondary
analysis often has problems with
validity.
Ethics and Survey Research
•
•
Surveys often ask for private information,
and researchers must keep such information
confidential.
Because asking questions can cause
psychological discomfort or harm to
respondents, the researcher should minimize
this risk.
Chapter Summary
•
•
•
•
•
•
Bias
Closed-ended questions
Computer-assisted telephone interviewing
(CATI)
Contingency question
Interview
Open-ended questions
Quick Quiz
(Babbie text)
Chapter 9 Quiz
When is survey research the best method
available?
A. when collecting original data
B. when describing a population too large to
observe directly
C. when measuring attitudes
D. all of the above
Chapter 9 Quiz
Answer: D.
Survey research the best method available
when collecting original data, when describing
a population too large to observe directly, and
when measuring attitudes.
Chapter 9 Quiz
_____ questions have a respondent select an
answer from a list provided.
A.
B.
C.
D.
Open-ended
Pretest
Experimental
Closed-ended
Chapter 9 Quiz
Answer: D.
Closed-ended questions have a respondent
select an answer from among a list provided.
Chapter 9 Quiz
As a general rule, a questionnaire should be
A.
B.
C.
D.
spread out.
uncluttered.
relevant.
all of the above
Chapter 9 Quiz
ANSWER: D.
As a general rule, a questionnaire should be
spread out, uncluttered, and relevant.
Chapter 9 Quiz
Which of these are among the many advantages that
underlie the growing popularity of telephone surveys?
A. money
B. time
C. convenience
D. all of the above choices
E. none of the above choices
Chapter 9 Quiz
ANSWER: D.
Money, time, and convenience are among the
many advantages that underlie the growing
popularity of telephone surveys.
Chapter 9 Quiz
Which is not an advantage of survey research?
A.
B.
C.
D.
increased validity
increased reliability
increased generalizability
increased flexibility in analysis
Chapter 9 Quiz
ANSWER: A.
Increased validity is not an advantage of
survey research.
Chapter 9 Quiz
The major problem with secondary analysis
pertains to
A.
B.
C.
D.
E.
theory.
hypotheses.
validity.
sampling.
empirical generalization.
Chapter 9 Quiz
ANSWER: C.
The major problem with secondary analysis
pertains to validity.