Survey Data Collection Methods
Slide 1
The goal of this lecture is to summarize the ways in which survey data is collected and to
provide some detail about the relative advantages and disadvantages of these different data
collection methods.
Slide 2
I hope it’ll be apparent that there is no best way to collect survey information. Each method has
its own advantages and disadvantages, which make it more or less suitable for a particular
research effort.
Slide 3
Here’s a list in the different ways in which we communicate with respondents. One way is
personal interviews, which can be conducted either door to door, within office, or in shopping
malls by intercepting mall patrons. Alternatively, we can contact respondents by telephone. We
also can contact respondents via self administered questionnaires, which can be delivered
either by conventional mail or via the internet (as web-based or e-mail-based questionnaires).
Slide 4
Here are some questions you should consider when choosing a data collection method. First,
what’s your available budget? Personal interviews, for example, are far more expensive per
completed interview than self-administered mail or internet surveys. If your budget is limited,
then you might steer away from personal interviews.
Second, how quickly do you need the data? Telephone interviews are relatively quick.
Turnaround time for telephone interviews, as you may recall from the recent elections, can be
24 hours or less. Alternatively, a self-administered mail questionnaire can take several months.
Third, how complex is the structure or length of your questionnaire? Clearly, telephone
questionnaires must be limited to 10-15 minutes unless the interviewer has pre-contacted the
respondent and scheduled a time for a subsequent, far longer, telephone call. In contrast, a selfadministered mail or internet questionnaire can be far more complex and include many more
questions. If there’s a need to expose respondents to stimuli—for example a test ad—that type
of exposure is impossible via telephone interview but easy via internet-based data collection
procedures. In testing a video commercial, online respondents with reasonably modern PCs and
internet connections easily could preview and respond to it.
Fourth, how important is collecting a large and representative sample? Different data collection
methods will lend themselves to greater or lesser sampling precision. For example, if an
academic researcher mailed a questionnaire to the Marketing Department at NMSU and asked
my department head to forward that questionnaire to the person who teaches marketing
research, I might or might not receive that questionnaire because several faculty members and
doctoral students teach marketing research. Alternatively, that questionnaire could be
addressed to someone who rarely teaches marketing research. Either way, the returned
questionnaire would reveal little about the way marketing research is taught at NMSU. If I were
to send a questionnaire to a VP of marketing at some corporation, there’s no way to guarantee
the VP, as opposed to an administrative assistant completed that questionnaire. If sampling
Page | 1
precision is important, then self-administered mail questionnaires are a poor choice but personal
interviews are an excellent choice.
Fifth, how important are data quality and quantity? Personal interviews tend to produce the
highest quality data because interviewers can use thoughtful follow-up probing questions to
ensure that respondents’ answers are as complete as possible. In contrast, self-administered
questionnaires produce data of far lower quality.
Sixth, will respondents be interested in the topic? If respondents are highly motivated, then they
will respond thoughtfully to online or self-administered mail questionnaires. Alternatively, if the
topic is of lesser interest, then strong encouragement may be necessary to get our questions
answered. In this case, a personal or telephone interview may be more suitable.
Finally, is incidence rate an issue? If a small percent of the population is qualified to respond,
then a low incidence rate may bias our results. In such cases, mail surveys are less suitable
than personal interviews.
Slide 5
The next nine slides summarize the relative advantages and disadvantages of door-to-door and
mall-intercept interviews.
Slide 6 to Slide 14 (No Audio)
Slide 15
Telephone surveys typically entail a telephone bank with interviewers manning each telephone
and a supervisor who eavesdrops randomly on interviews to ensure they’re being conducted
properly. The interviewers rely on one of two approaches: paper-and-pencil or computer
assisted. Paper-and-pencil administration means the interviewer reads through the interview
and records the respondent’s answers. Eventually, those answers are entered into a computer
database for analysis. A somewhat more sophisticated approach—one in which data is
available for intermediate analysis—is the computer-assisted telephone interview. This type
differs from the first type in that interviewers use a computer for survey administration and data
entry. Because the survey software can be programmed to ensure proper skip patterns and to
avoid impossible answers (for example, a ‘6’ on a 1-to-5 scale), administrative errors are
minimized.
A third way to conduct a telephone survey, one that I’m sure you’ve experienced and learned to
dread, is the computerized and voice-activated interview. In essence, respondents are forced to
speak to a machine, which is unlikely to garner thoughtful responses due to the impersonal
nature of the interview.
Slide 16
The next four slides summarize the relative advantages and disadvantages of telephone
surveys.
Slide 17 to Slide 19 (No Audio)
Slide 20
Self-administered questionnaires come in many forms. Paper-and-pencil questionnaires can be
delivered by mail or dropped off for later pick up at a scheduled time. This latter approach is
Page | 2
common in international research because many Easterners and others are reluctant to respond
to questionnaires unless there’s a visible sponsor. Paper questionnaires also can be delivered
as magazine or newspaper inserts or as faxes. Alternatively, self-administered questionnaires
can be delivered electronically: by e-mail, an internet website, or an interactive kiosk in a place
like a shopping mall.
Slide 21
Mail surveys come in two forms. In the one-shot (cross-sectional) survey, potential respondents
receive a questionnaire with a cover letter in the mail and are asked to respond to that single
questionnaire. Alternatively—and I’ve personally participated in several such studies ‘for the
experience’—respondents can join a mail panel, in which case they will receive mail surveys in
a routine and ongoing basis.
Slide 22
The next four slides cover the relative advantages and disadvantages of mail surveys.
Slide 23 to Slide 25 (No Audio)
Slide 26
Internet surveys are based on self-administered questionnaires posted on a website.
Respondents provide answers to questions that are displayed online by highlighting a phrase,
clicking an icon, or keying in an answer.
Slide 27
Here is a screen capture that lists the first six questions of an internet survey for the U.S. Mint.
Slide 28
The next four slides summarize the relative advantages and disadvantages of internet surveys.
Slide 29 to Slide 31 (No Audio)
Slide 32
The main limitation of internet surveys is lack of sample representativeness. More than any
other way to administer a survey, internet surveys are subject to self selection bias, in the sense
that people who opt into an internet survey typically are far more enthusiastic about the topic in
question, are the types of people who are far more willing to provide their opinions, et cetera.
Internet surveys tend to rely on relatively non-representative samples of the population, which is
true for more than interest reasons; it’s also true because some people lack internet access. A
person without internet access cannot, by definition, respond to an internet survey. Even for
people who have PC’s and internet access, often there are issues regarding the power of those
PCs and users’ sophistication. If, for example, a researcher is interested in testing alternative ad
executions and wishes to show respondents alternative ads, and those ads are either radio or
television ads, then it will be necessary for respondents to receive streaming media content. For
potential respondents with a narrow-band 56K modem and primitive PC, receiving streaming
video is problematic. In fact, it’s likely that unsophisticated computers won’t have installed the
software needed to view streaming media. To boost cooperation rate, which is a more ethical
approach than spamming large numbers of folks, we often send potential respondents a query
e-mail asking them to opt into our study. If they do, then they’re given specific instructions or a
Page | 3
URL to visit. When we use an opt-in approach, cooperation is far better, the response rate is far
higher, and sample representativeness tends to improve.
Slides 33 and 34 (No Audio)
Slide 35
These final five slides summarize the relative advantages and disadvantages of different data
collection methods. I believe that each summary is useful in its own right.
Slide 36 to 40 (No Audio)
Page | 4