Survey Research
Slide 1
Data Collection Design:
Survey Research
T. Ramayah
School of Management, Universiti Sains Malaysia
ramayah@usm.my, http://www.ramayah.com
Slide 2
Learning Objectives
Understand . . .
1. The process for selecting the appropriate and optimal survey approach.
2. Factors that affect participation in survey research.
3. Sources of error in communication studies and how to minimize them.
4. Major advantages and disadvantages of the three survey methods.
5. Why an organization might outsource a communication study.
6. Ethical issues with survey research and their solutions.
Slide 3
“We saw that our customers required help beyond the data sets they had
and that they could benefit from a wider opinion. So we built
SurveyMonkey Audience, and we’ve now got 4 million users who
signed up to take surveys. Our clients can choose the demographic they
want to hear from, and we can provide that sample.”
David Goldberg
CEO
SurveyMonkey
Slide 4
Data Collection Approach
Slide 5
Communication Approach
•
The communication approach involves surveying people,
through the use of a questionnaire or a personal interview.
•
People often make the mistake of seeing survey research in
business as the only appropriate data collection design.
Rather, it should be seen as a strong complement to
observation research.
•
When is it useful:
•
Observation cannot give you the information needed
•
Articulation of ideas
•
Speed
•
Cost
Slide 6
Social Media Analytics vs. Survey
If social media is the temperature
check, surveys are the taste test to
validate that the meal (kale salad) is
cooked properly.
Slide 7
Selecting a Communication Data Collection
Approach
Slide 8
Communication Approach
Strengths
• Versatility
• Efficiency
• Geographic coverage
Weaknesses
• Error
• Inaccessible populations
Slide 9
Sources of Error
Error
Sources
Instrument
Participant
Interviewer
Situation
Slide 10
1. Instrument
• Wording of the question
• Interpretation of some words
• Jargons
Slide 11
2. Participant
• Possess the information being
targeted by the investigative questions
• Understand his or her role in the
survey as the provider of accurate
information
• Must have adequate motivation to
cooperate (see the next slide)
Slide 12
Participant Motivation
Slide 13
3. Interviewer Error
1. Sampling Error
▪
Failure to secure full participant cooperation
2. Data Entry Error
▪
Failure to record answers accurately and completely
3. Process Errors
▪
Failure to consistently execute interview procedures
▪
Failure to establish appropriate interview environment
▪
Physical presence bias
▪
Inappropriate influencing
▪
Falsification of individual answers or whole interviews.
Slide 14
4. Situation
• When data was collected
• Where the data was collected
Slide 15
Response Terms
Noncontact rate
Refusal rate
Incidence rate
Slide 16
Response Terms
1.
The noncontact rate is a ratio of potential but unreached contacts to all potential
contacts. A contact may be unreachable due to no answer, busy signal, answering
machine or voice mail, and disconnects).
2.
The refusal rate refers to the ratio of contacted participants who decline the
interview to all potential contacts.
3.
The incidence rate refers to the ratio of contacted people who actually qualify for
the survey to all contacts. Total number of qualified respondents divided by the total
number of respondents who were screened for the study (qualified plus nonqualified).
Slide 17
Communication Approaches
SelfAdministered
Survey
Telephone
Survey
Survey via
Personal
Interview
Slide 18
Surveys
Slide 19
Self-Administered Surveys
Mobile service
provider
Mailed/Faxed
Messenger App.
Delivery
Modes
Courier
Computer delivered
Intercept
Slide 20
Self-Administered Surveys – When?
Costs
Sample
Accessibility
Topic Coverage
Systematic
Anonymity
Time
Constraints
Slide 21
Advantages of Self-Administered Study
• Access inaccessible participants
• Rapid data collection
• Incentives for higher response rates
• Visuals possible
• Lowest-cost
• Multiple sampling possible
• Geographic coverage
• Minimal staff needed
• Perceived anonymity
• Reflection time
• Question complexity
Slide 22
Disadvantages of Self-Administered Study
•
Low response rates in some modes
•
Skewed responses by extremists
•
No interviewer intervention
•
Participant anxiety possible
•
Need for low-distraction environment
•
Security
•
Requires accurate list
•
Cannot be too long
•
Directions necessary
•
Cannot be too complex
Slide 23
Designing Questionnaires (TDM Method)
• The TDM method refers to Don
Easy to read
Dillman’s Total Design Method.
Offer clear directions
• He proposes that surveys be
based upon social exchange
Include personalization
theory.
• One suggestion that flows from
Notify in advance
social exchange theory is that the
burden to participants should be
Encourage response
minimized.
Slide 24
Improving Response Rates
•
•
•
•
•
•
•
Advance notification
Reminders
Return directions and devices
Monetary incentives
Deadlines
Promise of anonymity
Appeal for participation
Slide 25
Options for Web-based Surveys
Fee-Based
Service
Surveying
Software
Slide 26
Surveying Software
Slide 27
Advantages of Surveying Software
• Questionnaire design in word processing environment
• Question and scale libraries
• Automated publishing to the Web
• Real-time viewing of incoming data
• Rapid transmission of results
• Flexible analysis and reporting mechanisms
Slide 28
The Web as a Survey Research Venue
Advantages
Disadvantages
•
•
•
•
•
•
Cost savings
• Recruitment
Short turnaround
• Coverage
Use of visual stimuli
• Difficulty developing probability
Access to participants
samples
Perception of anonymity
• Technical skill
Access to data and experiences
• System compatibility issues
otherwise unavailable
• Possible self-selection bias
Slide 29
Telephone Survey
Traditional
CATI systems
Computeradministered
Slide 30
Advantages of the Telephone Survey
• Lower costs than personal interview
• Wide geographic coverage
• Fewer interviewers
• Reduced interviewer bias
• Fast completion time
• Random Dialing
• CATI
Slide 31
Disadvantages of the Telephone Survey
• Lower response rate
iPhone
• Early termination
• Higher costs if geographically dispersed sample
• Limited Interview length
• Inaccessible populations
• Limited complexity of scales
Voice-over IP
32
Slide 32
Survey via Personal Interview
• A survey via personal interview is a
quantitative study using an individual depth
interview (IDI) between a trained
interviewer and a single participant taking
anywhere from 20 minutes to an hour or
more.
Pre-scheduled
Intercept
• This structured interview uses a
measurement instrument similar to a
questionnaire which specifies the questions
and the order of the questions.
Slide 33
Personal Interview Survey
Advantages
Disadvantages
• Good cooperation rates
• Interviewer can probe and
• High costs
explain
• Visual aids possible
• Illiterate participants can be
reached
• Interviewer can prescreen
• CAPI possible
• Need for highly trained
interviewers
• Time consuming
• Labor-intensive
• Some unwilling to invite strangers
into homes
• Interviewer bias possible
Slide 34
Outsourcing
Data
Analysis
Sampling
via Panels
Survey
Tasks
Omnibus
• Individual questions in omnibus studies are used to capture everything from
people’s feeling about the rise in gasoline prices to the power of a celebrity
spokesperson in an advertising campaign or the latest teenage fashion trend.
• Questions of more than one company are fielded in the same survey, but
where the results are separated
Survey Fielding
Ethical Issues
Deception
Confidentiality
Quality
Safety
Slide 36
Ethical Issues
1. Deception occurs when participants are told only part of the truth or when truth is fully
compromised. Survey research requires an estimate of time commitment for the participant. If
researchers aren’t careful in questionnaire design, surveys are often longer than they need to be.
If a researcher fails to correctly estimate time, he or she may falsely gain a participant’s
cooperation.
2. The right to confidentiality—to protect their identity and specific responses from being
attached to their identity. Researchers address these rights by prescreening participants to
ensure willingness, prescheduling survey experiences, limiting time requirements, restricting
access to participant identification, restricting access to completed data instruments, and
revealing participant information only with signed consent.
Slide 37
Ethical Issues
3. Quality relates to avoidance of error, which translates to strict adherence to questionnaire
structure and instructions. It also relates to not asking unnecessary questions to achieve the
research objective or making claims about data that cannot be supported by the survey
protocols employed. Interviewers have the responsibility to not falsify survey completions.
4. Safety refers to not putting the interviewer in harm’s way due to the environment where the
interview takes place (e.g., night interview, high crime location) or the people they must
interview. Researchers, who are insensitive to interviewers’ fears of bodily harm, face potential
legal consequences as well as research quality issues (falsified measurement instruments). The
solution involves choosing a different interviewer or a different survey method.
Slide 38
Ethical Issues and their Solutions
• Adequately pretest the survey to determine a solid estimate of the time
needed for completion. Underestimates of completion time also lead to
higher drop-out rates that, for any given survey, cause poorer-quality data.
Informed
Consent
In the broader sense, such underestimates discourage people from
participating in surveys, increasing the likelihood of nonresponse error.
• To protect participants from the consequences of deception, researchers
should explain the benefits of the study (through introducing themselves or
Pretest
their company, explaining participants’ rights and protections, and
obtaining informed consent—a full disclosure of research procedures).
• Researcher solutions to quality issues relate to interviewer training and to
Training
following appropriate industry standards for measurement—including
selection and development of measurement scales and measurement
instruments—and for data analysis.
Slide 39
Informed Consent
Slide 40
Institutional Review Board (IRB) Process
(JEPEM)
Purpose
IRB Actions
Applications
Process
Revision
Initial Review
Continuing
Review
Prepare IRB
Materials
Slide 41
What is important in this chapter?
• Types of communication
• Sources of error
• Ethical Issues and Solution
Slide 42
Thank you for listening
Slide 43
43