Surveys
What is a Survey?
Why Survey?




Get lots of structured information from lots of
people.
Simplify and standardize data collection.
Use statistics to make predictions.
Find out about things that can not be observed.
Such as Thoughts, Emotions, Opinions,
Intentions, Attitudes
The
Need to Know Why
why people do or do not do something.
• Likes, dislikes, attitudes, behaviors, influences
The
Need to Know How
• the process consumers go through before taking
some action.
• Decision making process
The
Need to Know Who
• Information on age, income, occupation, marital status,
stage in the family life cycle, education, and other lifestyle
factors is necessary to the identification and definition of
market segments.
Survey Methods
Personal
Interview
Door-to-Door (in-home), Mall-Intercept, Purchase-Intercept,
Executive (office)
Telephone-Interview
•People Computer assisted (CATI)
Self-Administered
•Mail Survey
•Fax
•One-time vs. Panels
Computer
Direct Computer Interviewing
Interactive voice response technology-Automated Telephone
Surveys
Internet
A Classification of Survey Methods
SURVEY
METHODS
TELEPHONE
TRADITIONAL
TELEPHONE
FACE TO FACE
CATI
IN HOME
MAIL
POSTAL
SURVEY
MALL
INTERCEPT
CAPI
COMPUTER
POSTAL
PANEL
FAX
E-MAIL
INTERNET
DIRECT COMPUTER
INTERVIEW
Personal Interviews
Personal Interviews
Advantages

Can arouse and keep interest

Can build rapport and enlist cooperation

Ask complex questions

Immediate feedback

Can use visual and other aids

Clarify misunderstandings

High degree of flexibility

Probe for more complete answers

Do not need an explicit or current list of households or
individuals
Personal Interviews
Disadvantages:

Bias of Interviewer

Response Bias
• Embarrassing/personal questions

Time Requirements

Cost Per Completed Interview Is High.

High refusal rate

A trained staff of interviewers that is geographically near the
sample is needed.
The total data collection period is likely to be longer than for most
procedures.
High selection error – non-representative sample


Telephone Interviewing
What are the Important Aspects of Telephone
Interviewing?

Selecting telephone numbers
• Pre specified list
• A directory
• Random dialing procedure
• Random digit dialing
• Systematic random digit dialing (SRDD)

The introduction
 When to call
 Call reports
Telephone Interviewing
Advantages:

Central location, under supervision, at own hours

More interviews can be conducted in a given time
• Travelling time is saved
 More hours of the day are productive

Repeated call backs at lower cost

Lower administrative costs/ Lower cost per
completed interview

Less sample bias

Better access to certain populations
 Shorter data collection periods.
Telephone Interviewing
Limitations and Disadvantages:

Inability to employ visual aids or complex tasks

Can't be longer than 5-10 min. or they get boring

Amount of data that can be collected is relatively less

A capable interviewer essential

Sample bias -- Not all people have phones, or are not listed

Nonresponse associated with RDD sampling is higher than with
interviews
Possibly less appropriate for personal or sensitive questions if
no prior contact
National Do Not Call List 1-866-580-DNCL (866-580-3625)
http://www.LNNTE-DNCL.gc.ca


Increasing Phone Survey Response

Call at a convenient time (Weekdays 7-9
PM, Sunday afternoon)

Have a nice Pleasant introduction

Emphasize you are not selling anything.

State how long it will take.

Keep the survey short
Self-Administered Types of Survey Research
Direct Mail Survey
Respondent
Reads
Survey
Questions
and Records
Answers
Without
Assistance
Questionnaire is distributed to and
returned from respondents via the
postal service.
Mail Panel Survey
Selected group of individuals that have
made an advance agreement to
participate in a series of direct mail
surveys.
Drop Off Survey
Questionnaires are left with respondent
to be completed at a later time and
returned to the researcher.
Mail Surveys

Requires a broad identification of the individuals
to be sampled before data collection begins
• Ad Hoc Mail Surveys (cold):
• Questionnaires for a particular project sent to
selected names and addresses with no prior contact
by the researcher.
• Mail Panels (warm):
• Pre-contacted and screened participants who are
periodically sent questionnaires.
• A mail panel is a type of longitudinal study. A
longitudinal study in one in which the same
respondents are re-sampled over time.
Mail Surveys
Some Decisions That Need to Be Taken







Type of Return Envelope
Postage
Method of Addressing
Cover Letter
The Questionnaire Length, Layout, Color,
Format Etc
Method of Notification
Incentive to Be Given
Mail Surveys
Advantages

Relatively low cost

Reliable answers as no inhibiting intermediary

Survey answered at respondents discretion

Can be accomplished with minimal staff and facilities.

Provides access to widely dispersed samples.

Respondents have time to give thoughtful answers,
look up records, or consult others.
Mail Surveys
Disadvantages

No control over whom the respondent consults before
answering the questions

The identity of the respondent is inadequately controlled

The speed of the response can't be monitored

No control on the order in which the questions are exposed
or answered

Open questions usually are not useful.

Good reading and writing skills are needed by respondents.

The interviewer is not present to exercise quality control
with respect to answering all questions, meeting questions
objectives, or the quality of answers provided.

High non-response rate
Mail Surveys
Disadvantages (Contd.):

The respondent may not clearly understand the
question and no opportunity to clarify

No long questionnaires

Need for good up-to date mailing list

Response rate is generally poor

Number of problems such as obsolescence,
omissions, duplications, etc
 Ineffective as a way of enlisting cooperation.
Factors Affecting the Response Rate

Perceived amount of work required, and the
length of the questionnaire

Intrinsic interest in the topic

Characteristics of the sample

Credibility of the sponsoring organization

Level of induced motivation
Tactics Employed to Increase Mail
Survey Response Rate
• Advance postcard or telephone call alerting respondent of
survey.
• Follow-up postcard or phone call.
• Monetary incentives (how much?).
• Premiums (pencil, pen, keychain, coupons, etc.).
• Postage stamps rather than metered envelopes.
• Self-addressed, stamped return envelope.
• Personalized address and well-written cover letter.
• Personally signed cover letter.
Tactics Employed to Increase Mail
Survey Response Rate
• Entry into drawing for prize.
• Emotional appeals.
• Affiliation with institutions or reputed organizations.
• Multiple mailings of the questionnaire.
• Bids for sympathy.
• Offer to share information from the survey.
• Promise of contributions to favorite charity.
Drop-off questionnaires
The
interviewer can explain the study, answer
questions, and designate a respondent.
Response
rates tend to be like those of personal
interview studies.
There
is more opportunity to give thoughtful
answers and consult records.
Costs
about as much as personal interviews.
A field
staff is required.
Fax Surveys
Advantages









Relatively low cost
Can be accomplished with minimal staff and
facilities
Provides access to widely dispersed samples.
Respondents have time to give thoughtful answers.
Local faxes are free.
Administrative costs are fixed.
It is fast.
List management is easy.
Can send and receive by computer.
Fax Surveys
Disadvantages

Higher fixed costs for computer/fax equipment,
multiple phone lines.
 Cost varies by time on line, time of day, distance,
and telephone carrier.
 Generally limited to organizational populations.
 Loss of anonymity.
Internet Surveys
Internet Samples
To realize the importance
of the marketing research
interviewer.
•Unrestricted
• Open to any Internet user.
•Screened
• adjust for unrepresentitiveness of the self-selected
respondents by imposing quotas based on some desired
sample characteristics
Recruited
• to target populations in
surveys that require more
control of the sample
•Greenfield
Internet Surveys
Advantages:
The
advantages of interviewer administration (In contrast to
mail surveys).
smaller staff
needed,
High-speed,
Instantaneous
Cost
data access – real time reporting
efficient
Automatic
data entry
Multimedia stimuli
Easy
to update
Ability
to reach a lot of people, and often those hard to reach
Internet Surveys
Advantages:

Potential for longitudinal studies

Surveys can be unobtrusively included with a general
site

pre-screening of respondents possible

tracking

Easy to personalize

No geographic boundaries

supervision and quality control potentially better.

better response rate from a list sample than from mail

use a branching or skip pattern
Internet Surveys
Disadvantages:
Internet
users are not representative of the population
as a whole (strong sample bias)
Strong
selection bias for respondents who are not prescreened
security/privacy
issues

unrestricted: anyone can complete the questionnaire

fully self-selecting
•Free Trial Survey Software
•Free Online Surveys
•eSurveypro
•Question Pro
•Keysurvey
•Checkbox survey
E-mail Questionnaires
• The
questionnaire is prepared like a simple
E-mail message, and is sent to a list of known
E-mail addresses. The respondent fills in the
answers, and E-mails the form plus replies
back to the research organization
Increasing Response: Internet/E-mail Survey
Almost all the actions listed for mail survey
should hold for E-Mail/Internet survey with
some modifications.
 The questionnaire should be fairly short.
 No need for advance notification
 Have short, pleasant introduction
 Monetary incentives not given. But sharing
information would be nice.
 Reminder e-mail would be useful.
Criteria for Determining Choice Of Particular Survey Methods
Sample Criteria: Ability to reach and get responses
from the desired sample (sample control)


right type of people
“adequate” sample size.
Information Criteria: Ability to get the desired
information from respondents.




Need to Expose Respondents to Various Stimuli or Perform Certain
Specialized Tasks e.g. Taste tests, product concept and prototype tests, etc.
Length of Questionnaire
Degree of Structure of the Questionnaire
Control social desirability
Administrative Criteria:




time for data collection and analysis
Interviewer control
Error control
Budget
Factors Determining the Choice of
Survey Method.
Personal:
Use for long, complex questionnaires where respondent is
important and budgets are high.
Mail:
Use when you have a mailing list, somewhat long / complex
questionnaire, and budgets are low.
Phone:
Use to reach a large number of people quickly and you have a
short questionnaire that can be easily understood.
E-Mail/Internet:.
Use when target audience is educated, topic interesting, short
questionnaire, have e-mail list, representativeness not a major
issue
Comparative Evaluation
Personal









Phone
Mail
Internet
Sample control
Use of physical stimuli
Diversity of questions
Length
Perceived anonymity
Potential for interviewer bias
























Field force control
Speed
Cost












Advantages & Disadvantages of Survey Methods
Personal
Mail/Self Report
Telephone
Internet
Speed of
data
collection
Moderate to fast No control over
return of
questionnaire
Very fast
Fastest
Geographic
flexibility
Limited to
moderate
Good
Best
Respondent
cooperation
Excellent except Moderate –poorly Good
in shopping
designed
malls
questionnaires
have poor response
rates
Best
Versatility of Very versatile
questioning
High
Highly
standardized
format
Moderate
Good
Advantages & Disadvantages of Survey Methods
Personal
Mail/Self Report
Telephone
Internet
Long
Varies depending
on incentive
Moderate/
Short
Moderate
Respondent
Lowest
misunderstanding
Highest
Moderate
Moderate
Interviewer
influence
High
None
Moderate
None
Supervision of
interviewers
Moderate
Not applicable
High
None
Anonymity of
respondent
Low in face- High
to-face
situations
Moderate
Low
Questionnaire
length
Advantages & Disadvantages of Survey Methods
Personal
Mail/Self Report
Telephone
Internet
Ease of
callback or
follow-up
Difficult
Easy, but takes
time
Easy
Easy
Cost
Highest
Low
Low to
moderate
Lowest
Special
features
Visual materials
may be shown
or
demonstrated;
extended
probing
possible
Respondents may
answer questions
at own
convenience; has
time to reflect on
answers
Field and
supervision of
data collection
are simplified;
quite
adaptable to
computer
technology
Data
collection
automatic
Skip patterns
automatic
Note: These reflect typical situations. For example, an elaborate mail survey may be
far more expensive than a short interview, but this is generally not the case.
Potential Problems with Surveys
How accurate are the results?

Did you ask the right questions in the right way?
 Did you ask the right people?
 Did they tell you the truth, the whole truth, and
nothing but the truth?
• Will they?
• Can they? Do they have the knowledge, opinions, attitudes, or facts
required. Do they understand the questions?

Structure causes a loss of data richness.
 Have the Respondents/Interviewers understood and
correctly recorded the responses

Lack of control causes time and response problems.
Overview of the Types of Errors in
Survey Research Methods





Survey research errors can be classified as
Random Sampling Error either:
 Nonsampling Error
statistically measured
difference between the actual
sampled results and the
estimated true population
results.
error because of chance
variation
Reduce by increasing sample
size
Range can be estimated with a
certain level of confidence



(Systematic Error)
results from mistakes or
problems in the research
design or its execution
Causes data to be misleading
or incomplete in some
systematic way from true
population parameter
If you are aware of the
problem you may be able to
deal with it.
Systematic Sample Design Error

Frame Error: The list from which you draw your sample is
not what you think it is—some on it don’t belong and/or
some who belong are not on it – sample is not representative
of population (e.g. telephone directory)
 Population Specification Error: incorrect definition of the
universe or population from which the sample is to be
selected. You left people out of the study.

Selection Error: You include or exclude people in the
sample so that it is not random. Use of incomplete or
improper sampling procedures or when appropriate
selection procedures are not properly followed (e.g. nonprobability samples such as intercept surveys)
Systematic Measurement Error
•Difference between the information sought and that obtained

Surrogate Information Error
• Acquisition of the wrong data because wrong question was asked. –
failure to define problem, e.g. New Coke

Interviewer Error
• interviewer may, consciously or unconsciously, influence respondents to
give untrue or inaccurate answers. – training problem

Instrument Bias
misunderstood or leading questions – questionnaire design

Processing Error
• sloppy data input

Non-response Bias
• differences between the “did” vs. “did not” answer question

Response Bias
a failure to tell the truth, consciously or unconsciously
Your company has just launched a
new brand of pancake syrup. In
order to improve sales the marketing
department would like to know why
consumers buy the brands and sizes
they do. How would you go about
collecting this information?
Calgary Transit would like to
conduct a ridership survey. Funds
are limited but they need a
relatively large sample. How
would you suggest they gather the
information?
Your company makes the new flip and fold device.
You want to find out what consumers think about it
and how much they are willing to pay. What sort of
survey technique do you use and why. Who do you
target in your survey
OBSERVATIONAL RESEARCH
Watching what people do
The information must be observable
Helpful if the behavior is repetitive and of short duration
Approaches to observational research
 Natural Versus Contrived Situations
 Open Versus Disguised Observation
 Structured Versus Unstructured
 Human Versus Machine Observers
What does gathering data through
Observation entail
What are the advantages of observational
data collection as opposed to surveys?
What are the relative disadvantages of
observational data collection as opposed to
surveys?
•Main Observational Research Methods
Direct Observation
Shopper Patterns and Behavior
Contrived Observation
Mystery Shopper
Content Analysis
Analyzing written material into meaningful units, using
carefully applied rules
Physical Trace Measures
“Garbology”
Ethnographic Research
The researcher becomes Immersed in or part of the group
Behavior (Emotion) Recording Devices
•Toothpaste manufactures have found consistently that if they ask for
detailed information on the frequency with which people brush their
teeth, and then make minimal assumptions as to the quantity of
toothpaste used on each occasion, as well as spillage and failure to
squeeze the tube empty, the result is a serious overstatement of
toothpaste consumption.
•How would you explain this phenomenon?
•Would it be possible to design a study to overcome these
problems and obtain more accurate estimates of consumption?
Advantages:
We see what people actually do
May avoid interviewer bias
Disadvantages:
No information on motives attitudes or
intentions
Time-consuming and expensive
•Machine observation
Traffic Counters
Time and flow in retail stores
Behavior Measurement
People Reader: reading habits
Physiological Measurement
EEG: electroencephalogram
GSR: galvanic skin response
Pupilometer: pupil dilation
Scanner Based
Store scanners read the UPC codes on products and
produce instantaneous information on sales