Surveys
What is a Survey?
Why Survey?




Get lots of structured information from lots
of people.
Simplify and standardize data collection.
Use statistics to make predictions.
Find out about things that can not be
observed. Such as Thoughts, Emotions,
Intentions, Attitudes
The
Need to Know Why
There generally is a critical need to have some
idea about why people do or do not do something.
• Likes, dislikes, attitudes, behaviors, influences

The Need to Know How
• Also it is often necessary to understand the
process consumers go through before taking
some action.

The Need to Know Who
• Information on age, income, occupation, marital
status, stage in the family life cycle, education,
and other lifestyle factors is necessary to the
identification and definition of market segments.
Survey Methods
Personal
Interview
Door-to-Door (in-home), Mall-Intercept, Purchase-Intercept,
Executive (office)
Telephone-Interview
•People Computer assisted (CATI)
Self-Administered
•Mail Survey
•Fax
•One-time vs. Panels
Computer
Direct Computer Interviewing
Interactive voice response technology-Automated Telephone
Surveys
Internet
A Classification of Survey Methods
SURVEY
METHODS
TELEPHONE
TRADITIONAL
TELEPHONE
FACE TO FACE
CATI
IN HOME
MAIL
POSTAL
SURVEY
MALL
INTERCEPT
CAPI
COMPUTER
POSTAL
PANEL
FAX
E-MAIL
INTERNET
DIRECT COMPUTER
INTERVIEW
Personal Interviews
Personal Interviews
Advantages

Can arouse and keep interest

Can build rapport and enlist cooperation

Ask complex questions

Can use visual and other aids

Clarify misunderstandings

High degree of flexibility

Probe for more complete answers

Do not need an explicit or current list of households or
individuals
Personal Interviews
Disadvantages:

Bias of Interviewer

Response Bias
• Embarrassing/personal questions

Time Requirements

Cost Per Completed Interview Is High.

A trained staff of interviewers that is geographically
near the sample is needed.
 The total data collection period is likely to be longer
than for most procedures.
Telephone Interviewing
What are the Important Aspects of Telephone
Interviewing?

Selecting telephone numbers
• Pre specified list
• A directory
• Random dialing procedure
• Random digit dialing
• Systematic random digit dialing (SRDD)

The introduction
 When to call
 Call reports
Telephone Interviewing
Advantages:

Central location, under supervision, at own hours

More interviews can be conducted in a given time
• Travelling time is saved
 More hours of the day are productive

Repeated call backs at lower cost

Lower administrative
completed interview

Less sample bias

costs/
Lower
Better access to certain populations
 Shorter data collection periods.
cost
per
Telephone Interviewing
Limitations and Disadvantages:

Inability to employ visual aids or complex tasks

Can't be longer than 5-10 min. or they get boring

Amount of data that can be collected is relatively less

A capable interviewer essential

Sample bias -- Not all people have phones, or are not
listed

Nonresponse associated with RDD sampling is higher
than with interviews
 Possibly less appropriate for personal or sensitive
questions if no prior contact
Increasing Phone Survey Response

Call at a convenient time (Weekdays 7-9
PM, Sunday afternoon)

Have a nice Pleasant introduction

Emphasize you are not selling anything.

State how long it will take.

Keep the survey short
Self-Administered Types of Survey Research
Direct Mail Survey
Respondent
Reads
Survey
Questions
and Records
Answers
Without
Assistance
Questionnaire is distributed to and
returned from respondents via the
postal service.
Mail Panel Survey
Selected group of individuals that have
made an advance agreement to
participate in a series of direct mail
surveys.
Drop Off Survey
Questionnaires are left with respondent
to be completed at a later time and
returned to the researcher.
Mail Surveys

Requires a broad identification of the individuals
to be sampled before data collection begins
• Ad Hoc Mail Surveys (cold):
• Questionnaires for a particular project sent to
selected names and addresses with no prior contact
by the researcher.
• Mail Panels (warm):
• Pre-contacted and screened participants who are
periodically sent questionnaires.
• A mail panel is a type of longitudinal study. A
longitudinal study in one in which the same
respondents are re-sampled over time.
Mail Surveys
Some Decisions That Need to Be Taken







Type of Return Envelope
Postage
Method of Addressing
Cover Letter
The Questionnaire Length, Layout, Color,
Format Etc
Method of Notification
Incentive to Be Given
Mail Surveys
Advantages

Relatively low cost

Reliable answers as no inhibiting intermediary

Survey answered at respondents discretion

Can be accomplished with minimal staff and facilities.

Provides access to widely dispersed samples.

Respondents have time to give thoughtful answers,
look up records, or consult others.
Mail Surveys
Disadvantages

No control over whom the respondent consults before
answering the questions

The identity of the respondent is inadequately controlled

The speed of the response can't be monitored

No control on the order in which the questions are exposed
or answered

Especially careful questionnaire design is needed.

Open questions usually are not useful.

Good reading and writing skills are needed by respondents.

The interviewer is not present to exercise quality control
with respect to answering all questions, meeting questions
objectives, or the quality of answers provided.
Mail Surveys
Disadvantages (Contd.):

The respondent may not clearly understand the
question and no opportunity to clarify

No long questionnaires

Need for good up-to date mailing list

Response rate is generally poor

Number of problems such as obsolescence,
omissions, duplications, etc
 Ineffective as a way of enlisting cooperation.
Factors Affecting the Response Rate

Perceived amount of work required, and the
length of the questionnaire

Intrinsic interest in the topic

Characteristics of the sample

Credibility of the sponsoring organization

Level of induced motivation
Tactics Employed to Increase Mail
Survey Response Rate
• Advance postcard or telephone call alerting respondent of
survey.
• Follow-up postcard or phone call.
• Monetary incentives (how much?).
• Premiums (pencil, pen, keychain, coupons, etc.).
• Postage stamps rather than metered envelopes.
• Self-addressed, stamped return envelope.
• Personalized address and well-written cover letter.
• Personally signed cover letter.
Tactics Employed to Increase Mail
Survey Response Rate
• Entry into drawing for prize.
• Emotional appeals.
• Affiliation with institutions or reputed organizations.
• Multiple mailings of the questionnaire.
• Bids for sympathy.
• Offer to share information from the survey.
• Reminder that respondent participated in previous
study.
• Promise of contributions to favorite charity.
Drop-off questionnaires
The
interviewer can explain the study, answer
questions, and designate a respondent.
Response
rates tend to be like those of personal
interview studies.
There
is more opportunity to give thoughtful answers
and consult records.
Costs
about as much as personal interviews.
A field
staff is required.
Fax Surveys
Advantages









Relatively low cost
Can be accomplished with minimal staff and
facilities
Provides access to widely dispersed samples.
Respondents have time to give thoughtful answers.
Local faxes are free.
Administrative costs are fixed.
It is fast.
List management is easy.
Can send and receive by computer.
Fax Surveys
Disadvantages

Higher fixed costs for computer/fax equipment,
multiple phone lines.
 Cost varies by time on line, time of day, distance, and
telephone carrier.
 Generally limited to organizational populations.
 Loss of anonymity.
Internet Surveys
Internet Samples
To realize the importance
of the marketing research
interviewer.
•Unrestricted
• Open to any Internet user.
•Screened
• adjust for unrepresentitiveness of the self-selected
respondents by imposing quotas based on some desired
sample characteristics
Recruited
• to target populations in surveys that require more
control of the sample
Internet Surveys
Advantages:
The
advantages of interviewer administration (In
contrast to mail surveys).
smaller staff
needed,
High-speed,
Instantaneous
Cost
efficient,
Automatic
data entry
Multimedia
Easy
data access
stimuli
to update
Ability
to reach a lot of people
Internet Surveys
Advantages:

Potential for longitudinal studies

Surveys can be unobtrusively included with a general
site

pre-screening of respondents possible

tracking

No geographic boundaries

supervision and quality control potentially better.

better response rate from a list sample than from mail

use a branching or skip pattern
Internet Surveys
Disadvantages:
•Internet users are not representative of the
population as a whole (strong sample bias)
•Strong selection bias for respondents who are not
pre-screened
•security/privacy issues
• unrestricted: anyone can complete the questionnaire
• fully self-selecting
E-mail Questionnaires
• The
questionnaire is prepared like a simple
E-mail message, and is sent to a list of known
E-mail addresses. The respondent fills in the
answers, and E-mails the form plus replies
back to the research organization
Increasing Response: Internet/E-mail Survey
Almost all the actions listed for mail survey
should hold for E-Mail/Internet survey with
some modifications.
 The questionnaire should be fairly short.
 No need for advance notification
 Have short, pleasant introduction
 Monetary incentives not given. But sharing
information would be nice.
 Reminder e-mail would be useful.
Criteria for Determining Choice Of Particular Survey Methods
Sample Criteria: Ability to reach and get responses
from the desired sample (sample control)


right type of people
“adequate” sample size.
Information Criteria: Ability to get the desired
information from respondents.




Need to Expose Respondents to Various Stimuli or Perform Certain
Specialized Tasks e.g. Taste tests, product concept and prototype tests, etc.
Length of Questionnaire
Degree of Structure of the Questionnaire
Control social desirability
Administrative Criteria:




time for data collection and analysis
Interviewer control
Error control
Budget
Factors Determining the Choice of
Survey Method.
Personal:
Use for long, complex questionnaires where respondent is
important and budgets are high.
Mail:
Use when you have a mailing list, somewhat long / complex
questionnaire, and budgets are low.
Phone:
Use to reach a large number of people quickly and you have a
short questionnaire that can be easily understood.
E-Mail/Internet:.
Use when target audience is educated, topic interesting, short
questionnaire, have e-mail list, representativeness not a major
issue
Comparative Evaluation







Sample control
Use of physical stimuli
Diversity of questions
Length
Perceived anonymity
Potential for interviewer bias
Field force control
 Speed
 Cost
Personal Phone



























Mail
Advantages & Disadvantages of Survey Methods
Personal
Mail/Self Report Telephone
Speed of data
collection
Moderate to fast
No control over
return of
questionnaire
Very fast
Geographic
flexibility
Limited to
moderate
High
Good
Respondent
cooperation
Excellent except
in shopping malls
Moderate –poorly Good
designed
questionnaires
have poor
response rates
Versatility of
questioning
Very versatile
Highly
standardized
format
Moderate
Advantages & Disadvantages of Survey Methods
Personal
Mail/Self Report
Telephone
Questionnaire
length
Long
Varies depending
on incentive
Moderate/Short
Respondent
misunderstanding
Lowest
Highest
Moderate
Interviewer
influence
High
None
Moderate
Supervision of
interviewers
Moderate
Not applicable
High
Anonymity of
respondent
Low in face-toface situations
High
Moderate
Advantages & Disadvantages of Survey Methods
Personal
Mail/Self Report
Telephone
Ease of callback or Difficult
follow-up
Easy, but takes
time
Easy
Cost
Highest
Lowest
Low to moderate
Special features
Visual materials
may be shown or
demonstrated;
extended probing
possible
Respondents may
answer questions
at own
convenience; has
time to reflect on
answers
Field and
supervision of
data collection are
simplified; quite
adaptable to
computer
technology
Note: These reflect typical situations. For example, an elaborate mail survey may be
far more expensive than a short interview, but this is generally not the case.
Potential Problems with Surveys
How accurate are the results?

Did you ask the right questions in the right way?
 Did you ask the right people?
 Did they tell you the truth, the whole truth, and
nothing but the truth?
• Will they?
• Can they? Do they have the knowledge, opinions, attitudes, or facts
required. Do they understand the questions?

Structure causes a loss of data richness.
 Have the Respondents/Interviewers understood and
correctly recorded the responses

Lack of control causes time and response problems.
Overview of the Types of Errors in
Survey Research Methods
Survey research errors can be classified as
either:

Random
Sampling Error

statistically measured
difference between
the actual sampled
results and the
estimated true
population results.
error because of
chance variation


Nonsampling Error
(Systematic Error)

results from mistakes or
problems in the research design
or from flaws in the execution of
the sample design
Causes your data to be
misleading or incomplete in some
systematic way
If you are aware of the problem
you may be able to deal with it.


Systematic Sample Design Error

Frame Error: The list from which you draw your
sample is not what you think it is—some on it don’t
belong and/or some who belong are not on it.
 Population Specification Error: incorrect definition
of the universe or population from which the sample
is to be selected.You left people out of the study.
 Selection Error: You include or exclude people in
the sample so that it is not random. Use of
incomplete or improper sampling procedures or
when appropriate selection procedures are not
properly followed.
Systematic Measurement Error

Surrogate Information Error
• Acquisition of the wrong data because wrong question was
asked.

Interviewer Error
• interviewer may, consciously or unconsciously, influence
respondents to give untrue or inaccurate answers.

Instrument Bias
misunderstood or leading ?s.

Processing Error
• sloppy data input

Non-response Bias
• differences between the “did” vs. “did not” answer question

Response Bias
a failure to tell the truth., consciously or unconsciously
Your company has just launched a
new brand of pancake syrup. In
order to improve sales the marketing
department would like to know why
consumers buy the brands and sizes
they do. How would you go about
collecting this information?
Calgary Transit would like to
conduct a ridership survey. Funds
are limited but they need a
relatively large sample. How
would you suggest they gather the
information?
Your company makes the new flip and fold device.
You want to find out what consumers think about it
and how much they are willing to pay. What sort of
survey technique do you use and why. Who do you
target in your survey
OBSERVATIONAL RESEARCH
Watching what people do
•
The information must be observable
Helpful conditions:
– the behavior is repetitive and of short duration
Approaches to observational research
• Natural Versus Contrived Situations
• Open Versus Disguised Observation
• Structured Versus Unstructured
• Human Versus Machine Observers
What does gathering data through
Observation entail
What are the advantages of observational
data collection as opposed to surveys?
What are the relative disadvantages of
observational data collection as opposed to
surveys?
Main Observational Research Methods

Direct Observation
– Shopper Patterns and Behavior

Contrived Observation
• Mystery Shopper

Content Analysis
• Analyzing written material into meaningful units, using carefully
applied rules

Physical Trace Measures
• “Garbology”

Ethnographic Research

The researcher becomes Immersed in or part of the group

Behavior (Emotion) Recording Devices

Toothpaste manufactures have found consistently that if
they ask for detailed information on the frequency with
which people brush their teeth, and then make minimal
assumptions as to the quantity of toothpaste used on each
occasion, as well as spillage and failure to squeeze the tube
empty, the result is a serious overstatement of toothpaste
consumption.
• How would you explain this phenomenon?
• Would it be possible to design a study to overcome these
problems and obtain more accurate estimates of
consumption?
Advantages:
–We see what people actually do
–May avoid interviewer bias
Disadvantages:
• No information on motives attitudes or
intentions
• Time-consuming and expensive
Machine observation
Traffic Counters
• Time and flow in retail stores
Behavior Measurement
• People Reader: reading habits
Physiological Measurement
• EEG: electroencephalogram
• GSR: galvanic skin response
• Pupilometer: pupil dilation
Scanner Based
• Store scanners read the UPC codes on products
and produce instantaneous information on sales