Survey Research
Survey data
• Survey data are obtained when individuals
respond to questions asked by interviewers
or when the individual responds to
questions that he has read
• Quantitative and qualitative information
• Census or sample?
Survey Sampling
• Sample Survey
– Sample of the population
(e.g., sample of Alberta CEOs)
• Census Survey
– Complete population
(e.g., all CEOs in Lethbridge)
Functions
• Descriptive
• Causal (limited function)
• Exploratory (limited function)
Information Provided by Surveys
• respondent’s knowledge of facts
– how many times a month do you buy cookies?
• respondent’s attitudes
– “chocolate chip is my favorite variety of cookie.”
• May describe processes undertaken by respondent
– Stages in the purchase decision, for example
• usually self-report data
• The type of information gathered depends
on a survey’s objectives.
• Surveys typically have multiple objectives
Classifications
• Method of Communication
– Telephone
– Mail
– Personal interview
• Degree of Structure
– Unstructured
– structured
• Degree of Disguise
Limitations:
Structure and disguise
are not clear categories;
most surveys are hybrids
• Temporal classifications
– Cross sectional studies
– Longitudinal studies
• Trend Studies
– Samples general population at each point; complete turnover in
who is actually sampled
• Cohort studies (tracking studies)
– Samples from one group over time; e.g., sample of a graduating
class
• Panel studies
– sample cohort, same specific respondents each sample
Method of Communication
•
•
•
•
Media
Personal Interviews
Telephone Interviews
Self administered Questionnaires
Media
• Human interactive
• Electronic interactive: researcher and
respondent interact using digital technology
• Non-interactive media
Personal interview
• Direct communication between business
and consumers in which interviewers ask
respondents questions in face to face
situations
• Key advantages:
– Opportunities for feedback
– Probing for complex answers
– Length of interview
Personal interview
• Direct communication between business
and consumers in which interviewers ask
respondents questions in face to face
situations
Key advantages:
Personal interview
•
•
•
•
•
•
Opportunities for feedback
Probing for complex answers
Length of interview
Completion rates
Ability to use props
High participation/low refusal rate
(especially if using call back)
Key disadvantages:
•
•
•
•
Anonymity of respondent
Interviewer bias
Interviewer cheating
cost
Types of personal interviews
• Door to door interviews:
– Low refusal rates
– Provide a more representative sample
– Provide a less representative sample (underrepresentation of certain groups)
Types of personal interviews
• Mall intercept:
–
–
–
–
–
–
Lower cost
Provide a larger sample
Useful if targeting particular population
Increased use of visual aids
Higher refusal rate
Provide a less representative sample
Telephone interviews
• Considered to be the primary method of
survey research
• Why:
– representative samples
– Technology
– Perception of anonymity
Key advantages:
Telephone interview
• Central location - increased quality of
supervision
• Computer assisted telephone interviewing data entry
• Speed
• Cost
•
•
•
•
•
•
Anonymity- absence of face to face contact
Less interviewer cheating
Less researcher bias
Cooperation???
Representative samples (RDD)
Call backs easier
Key disadvantages:
• Cooperation
• Lack of visual media
• Limited duration
Mail and self-administered
questionnaires
• A very popular method of data collection
Types of Mail Surveys
– Ad Hoc Mail Surveys:
• Questionnaires for a particular project sent to
selected names and addresses with no prior contact
by the researcher.
– Mail Panels:
• Precontacted and screened participants who are
periodically sent questionnaires.
• A mail panel is a type of longitudinal study. A
longitudinal study in one in which the same
respondents are resampled over time.
Key advantages:
Self-administered
• Geographical flexibility
• Cost
• Respondent convenience - accuracy and
reflection
• Anonymity
• Standardized questions - increase validity
Key disadvantages:
Self-administered
•
•
•
•
Time
Length (6 pages)
Interviewers absence
Response rates
Tactics Employed to Increase
Response Rate
• Advance postcard or telephone call alerting respondent of survey
• Follow-up postcard or phone call (coding/keying)
• Monetary incentives (nickel, dime, quarter, half-dollar)
• Premiums (pencil, pen, keychain, etc.)
• Postage stamps rather than metered envelopes
• Self-addressed, stamped return envelope
• Assurances regarding length
• Promise of contributions to favorite charity
• Entry into drawing for prize
• Emotional appeals
• Affiliation with universities or research institutions
• Multiple mailings of the questionnaire
• Bids for sympathy
• Reminder that respondent participated in previous study
• Interesting questions; well laid out
• Ask the right people
• A cover letter
Cover Letter
• introduces who you are
• purpose and importance of study
• relevance of survey to respondent
– especially if they receive several survey’s a
week, need to be convinced survey results
matter to them
• why and how they were selected
– why their input is crucial
– e.g., “As a respected member of the
Trustees, you have a unique perspective
on....”)
• big name sponsor, if any
– e.g., University of Lethbridge, Heart
Foundation, etc.
• give due date
– make reasonable but specific
• contact number if they have any
questions
– include email these days too
• end by thanking them for completing
survey
• explain direct incentive, if any (optional)
Not just the mail...
•
•
•
•
At point of service
Fax surveys (respond by fax; deliver by fax)
Email surveys
Computerized questionnaires (computer
interactive)
• Internet surveys
Factors Determining Choice of
Particular Survey Method
• Sampling Precision Required
• Budget Available
• The Need to Expose the Respondent to
Various Stimuli
• Quality of Data Required
• Length of the Questionnaire
• Necessity of Having Respondent Perform
Certain Specialized Tasks
• Incident Rate
– Percentage of people or households in the
general population that fit the qualifications to
be sampled.
• Degree of Structure of the Questionnaire
Error
Types of Error in Survey
Research
• Random Sampling Error (Random error)
– Error that results from chance variation
– Impact can be decreased by increasing sample
size and through statistical estimation
(confidence interval) or “rule of thumb”
• Systematic Error (non sampling error)
– Error that results for the research design or
execution.
Total Error
Random Error
Systematic Error
Respondent error
Administrative error
Non response error
Response bias
Deliberate Falsification
Acquiescence
Extremity bias
Unconscious misrepresentation
Interviewer bias
Auspices bias
social desirability
Data Processing
Sample selection
Interviewer cheating
Interviewer error
Types of Systematic Error
1. Administrative Error
– Error that results from improper execution.
• Data Processing Error
– Quality of data depends on quality of data
entry.
– Use of verification procedures can minimize
• Sample Selection Error
– Systematic error resulting from improper
sampling techniques either in design or
execution.
• Interviewer Error
– Data recorded incorrectly (error or selective
perception).
• Interviewer Cheating
– Mitigate by random checks
2. Respondent Error
– Humans interviewing humans...
• Non-response error
– Statistical difference between a survey that
includes only those who responded and a survey
that also includes those who failed to respond.
– Non-respondent: person not contacted or who
refuses to participate
– Self selection bias: extreme positions represented
• Response bias
– Errors that result from tendency to answer in “a
certain direction”.
– Conscious or unconscious misrepresentation
•
Types:
– 1. Deliberate falsification (why?)
• Why would people deliberately falsify data
–
–
–
–
–
Appear to be what they are not
Don’t trust confidentiality
Protect
To end the interviewer quicker
“Average man effects”
• Types of response bias continued:
– 1. Deliberate falsification
– 2. Unconscious misrepresentation
• Reasons for unconscious misrepresentation:
– Question format
– Question content
– Misunderstanding of question leading to biased
answer
– Lack of time to consider answer fully
– Communication or semantic confusion
– other
Types of response bias
• Acquiescence bias: individuals have a
tendency to agree or disagree with all
questions or to indicate a positive/negative
connotation
• Extremity bias: results for response styles
varying from person to person; some people
tend to use extremes when responding to
questions
Types of response bias continued...
• Interviewer bias: Bias in the responses of the
subject due to the influence of the interviewer
• Auspices bias: respondents being influenced
by the organization conducting the study
• Social desirability bias: caused by
respondents’ desire, either consciously or
unconsciously to gain prestige or to appear in a
different social role