Chapter Five
Survey Research:
The Profound Impact of
the Internet
Chapter Five
The Popularity of Survey Research
Why
How
Chapter Five
Who
Survey Research Error
Total Survey
Error
Random
Unintentional vs. Intentional
Respondent vs. Researcher
Measurement
Processing
Response Bias
Non-response Bias
Measurement Instrument Bias
Interviewer
Surrogate Information
Chapter Five
Systematic
Bias
Sample
Design
Selection
Population Specification
Frame
Survey Research Error
Key Definitions
Random Error or Random Sampling Error:
• Error resulting from chance variation.
Chance Variation:
• The difference between the sample value and the true
value of the population mean.
Systematic Error or Bias:
• Error that results from problems or flaws in the execution of
the research design; Sometimes called non-sampling error.
Sample Design Error:
• Systematic error that results from an error in the sample
design or sampling procedures.
Chapter Five
Survey Research Error
Key Definitions
Population Specification Error:
• Error resulting from incorrectly defining the
population or universe from which a sample
is chosen.
Selection Error:
• Error that results from incomplete or improper sampling procedures or not
following appropriate procedures.
Measurement Error:
• Systematic error that results from a variation between the information
being sought and what is actually obtained by the measurement process.
Surrogate Information Error:
• Error that results from a discrepancy between the information needed to solve
a problem and that sought by the researcher.
Chapter Five
Survey Research Error
Key Definitions
Interviewer Error or Interviewer Bias:
• Error that results from the interviewer’s influencing consciously or unconsciously - the answers of the
respondent.
Measurement Instrument Error:
• Error that results from the design of the questionnaire or
measurement instrument; also known as questionnaire bias.
Processing Error:
• Error that results from the incorrect transfer of information
from a survey document to a computer.
Chapter Five
Survey Research Error
Key Definitions
Non-response Bias:
• Error that results from a systematic difference between
those who do and those who do not respond to the
measurement instrument.
Refusal Rate:
• Percentage of persons contacted who refuse to participate in a survey.
Response Bias:
• Error that results from the tendency of people to answer a question incorrectly through
wither deliberate falsification or unconscious misrepresentation.
Sampling Frame:
• The list of the population elements or members from which units to be sampled are
selected.
Frame Error:
• Error that results from incorrectly defining the population or universe from which a
sample is chosen.
Chapter Five
Types of Surveys
• Mail & Mail Panels
• E-Mail & Internet
• Mall Intercept
• Interviews
• Telephone Surveys
• Door to Door
• Computer Assisted
• Self Administered
• Executive Interviews
• Longitudinal Study
Methods to Increase
Response Rates:
• Pre-Test (to make it the best survey it can be)
• Appeals (greater good, etc.)
• Deadlines (not too long or short)
• Incentives (money, giveaways, etc)
• Confidentiality / Anonymity
• Sponsorship (careful - the name could hurt)
• Cover Letter (explaining survey purpose)
• Personalization (tailor to respondents)
• Preliminary Notification / Follow-up
• Interesting Topic/Questions
• Return Postage/Outgoing Types
• Questionnaire (size, length, color)
Chapter Five
Types of Surveys
Door to Door:
• Interviews conducted face to face with customers in their homes.
• pros - direct customer contact, can demonstrate products, responses rates still good.
• cons - expensive, bad image, liability, safety issues.
Executive Interviews:
• Industry equivalent of door-to-door interviewing.
• pros - talking with decision maker is valuable, can have complicate questioning.
• cons - hard to get decision maker, limited sample size, not reflective of the population.
Mall Intercept Interviews:
• Face-to-face interviews conducted by intercepting people at high traffic locations.
• pros - direct customer contact, can test products, human interaction.
• cons - expensive, not always representative of the population, bad image.
Chapter Five
Types of Surveys
Central Location Telephone Interviews:
• Interviews conducted by calling respondents from a central location.
• pros - good segmentation, can verify respondent as the right person.
• cons - cost, response rates slipping, need to have short surveys.
Computer Assisted Telephone Interviews (CATI):
• Central-location telephone interviews in which interviewers enter
respondents’ answers directly into a computer.
• pros - data can be directly entered into computer, can have
complicated skip patterns, data entry errors rare.
• cons - computer glitches, need interviewer training, the equipment can
be expensive.
Cell Phone Surveys:
• cons – Generally not permitted by law
Chapter Five
Types of Surveys
Self Administered Questionnaires:
• Questionnaires filled out by respondents with no interviewer present.
• pros - respondents can take their time, good response rates.
• cons - respondents often take a long time, researcher can overcomplicate the
survey since respondents have more time.
Mail Surveys:
• Ad hoc / cross sectional and mail panels / longitudinal.
• pros - not too expensive, can get targeted mailing lists, longer questionnaires.
• cons - low response rates, mailing list often out of date, not sure who
completed the survey, manual data entry common.
E-mail - Internet:
• pros - cheap, direct access, can get quick information, can download results can
contact hard-to-reach people, good responses rates for web.
• cons - response rates low for e-mail , can’t verify responder, security issues, not
always representative of the population.
Chapter Five
Types of Surveys
Cross Sectional and Longitudinal
Cross Sectional Surveys:
• Are one time snapshots of population
• Use the same survey & different sample
Longitudinal Surveys:
• Identify market trends
• Use the same survey & same sample
• Use tracking panels & groups
Chapter Five
Survey Research on the Internet
• Low costs & no geographic barriers;
• Can be executed quickly / real time results;
• Ready personalization;
• Good for generating fresh ideas / brainstorming;
• Can enhance other data collection methods;
• Generally higher response rates;
• Simplified panel management;
• High profitability for research firms.
• Information security real and perceived;
• Not everyone has easy access to the Internet;
• Often not representative of the general population;
• Connecting speeds can slow down certain survey applications;
• Anyone can take the survey if not password protected;
• Can be hard to track who responded and who did not respond.
Chapter Five
Determination of the Survey Method
• Sampling Precision;
• Budget - Cost / Benefit Analysis;
• Requirements for Respondent Reactions;
• Quality of Data Desired/Required;
• Desired Questionnaire Length;
• Incidence and Cooperation Rates;
• Questionnaire Structure;
• Time Available to Complete the Survey;
• Desired Response Rate;
• How the Data Will be Used.
Chapter Five