Response Rate
Slide 1
The goal of this lecture is to alert you to the problems associated with ever decreasing response
rates, to talk about response rates to different types of surveys, especially telephone and mail
surveys, to suggest some ways to increase response rates for those types of surveys, and to
provide you with a basic background about the costs of conducting such surveys.
Slide 2
Unfortunately, people no longer believe in polls. They’re inundated with the results of polls;
seemingly, polls are conducted regularly and their results reported in the mass media. Requests
to participate in surveys are a daily affair. Ultimately, people burn out on survey participation or
survey results. As a result, people are no longer inclined to participate, nor are they willing to be
serious about their answers if they do participate. Once response rates decline, the
representativeness of survey results will decline as well. Declining response rates are a major
concern to marketing researchers, as it decreases the quality of their research while boosting
the cost of data collection. This has major implications on the quality of marketing managers’
decisions.
Slide 3
Here’s another example of the current general disregard for the value of surveys.
Slide 4
This slide indicates the potential problems associated with non-response error. In A, there’s no
non-response error; the response rate for the planned sample is 100%. In B, the response rate
is less than 100% but there’s no non-response error because there’s nothing systematic about
the people or sampling units that responded. The problem comes with C, where the response
rate is less than 100% response rate but there’s something systematically different about
respondents relative to non-respondents.
Slide 5
As a result of this declining interest, it’s important for researchers to maximize response rates
and hence the representativeness of their samples. It’s critical that they be able to develop and
convince respondents of the good reasons to participate in a survey.
Slide 6
Unfortunately, many potential respondents have encountered unscrupulous marketers who
pretended to conduct research, when in fact they were doing something entirely different, like
selling. As a result, response rates to telephone interviews have dropped markedly in the last 20
to 30 years.
Slide 7
Even under the best of circumstances, completing a telephone interview is difficult. This exhibit
shows all the possible outcomes to an attempted telephone interview. Only one of these
outcomes—the one in the bottom left-hand corner—shows completed interview useful to a
marketing researcher. All these other possible outcomes are unusable.
Page | 1
Slide 8
More than 20 years ago, completing telephone interviews was problematic. Now, with
answering machines, caller ID, and the like, it’s become very trying. Even in 1983, the results of
a study that involved more than 250,000 first call attempts showed that only 8.4% of those call
attempts resulted in a successfully completed telephone survey. That is an extraordinarily low
success rate.
Slide 9
When dealing with telephone interviews, there’s only two ways to cope with first call attempts
that don’t result in either a refusal or a successfully completed interview. One way is to call back
the non-answering potential respondent at a different day and time. Calling back at the same
day and time, given the regularity of people’s lifestyles and schedules, is likely to result in
another non-answer. Instead, calling back at various days and times in the hopes of catching
that person, assuming they’re interested in participating in a survey, would be better. Typically,
three call back attempts is the conventional number for telephone surveys. After three attempts,
the researcher will delete that potential respondent from further consideration.
Another strategy is to substitute other households for the ‘not at homes.’ As a result, the
planned sample and the final sample will differ, as shown here in figure B, but if the final sample
is within the final sampling frame, then the only issue is to determine if some types of
respondents have been oversampled and other types undersampled. If that has occurred, the
sample bias has been introduced.
Slide 10
As I mentioned in the previous lecture on sample design, there’s an issue associated with
unlisted telephone numbers. An ever larger portion of households are unlisted. Nationally, that
now comes to 27.6%, but in many cities the unlisted rate is over 50%. That rate is especially
bad in California, with many cities over 50%. In L.A., 56% of households have an unlisted
number.
Slide 11 (No Audio)
Slide 12
Here’s a summary of some recent Canadian data indicating the problem with unlisted phone
numbers is not limited to the U.S. In some areas, the unlisted rate approaches or even exceeds
50% of households.
Slide 13
Why the big fuss about unlisted households? Why do marketing researchers worry about such
things? It’s because listed and unlisted households differ from one another. Listings increase
with age. Over 50% of households with a person between the ages of 18 and 34 are unlisted,
whereas roughly 40% of households with a person between the ages of 35 and 54 are unlisted.
Among people who moved in the past two years, 59% chose not to list their numbers. Unlisted
households tend disproportionally to unmarrieds and renters. Nonetheless, there is a fair body
of literature that suggests listed and unlisted households differ meaningfully from one another;
therefore, telephone directories are unacceptable sample frames for telephone interviews.
Page | 2
Slide 14
Although survey researchers have explored many ways to increase response rates to mail
surveys, here are some accepted ways to boost those response rates.

Be certain that the cover letter is sales oriented, provides respondents with a good
reason to respond to the survey, and mentions a token of appreciation. (If that token is
money, then it could go to the respondent or a charity. Just mail the incentive in advance
to encourage participation.

Be certain that the questions are interesting; after all, a 20-page survey about laundry
detergent is unlikely to receive a high response rate.

Follow ups are critical; second and third mailings, reminder postcards, and letters that
alert respondents that they’ll receive a questionnaire shortly, are all useful in boosting
response rates.

Sponsorship is important. If the sponsor is well known--perhaps a prestigious university-then response rates tend to be higher.
Slide 15
Here are the ways in which Bicycling Magazine attempted to increase its response rate to its
semi-annual survey of bicycle dealers throughout the U.S. The magazine used an alert letter
that indicated a questionnaire was coming. Five days later, respondents received a
questionnaire packet that included a cover letter, a questionnaire, a $1 bill, and a stamped
return envelope. A second packet—which contained a reminder letter, a questionnaire, and a
stamped return envelop—was mailed five days after the initial packet was mailed. A week later,
a follow up postcard, reminding respondents of the opportunity to participate, was sent. One
week after that, a second reminder postcard was mailed. By using this procedure, the magazine
was able to achieve a 68% response rate, which is excellent for a mail survey. Unfortunately,
this was in 1987, when response rates were higher. Although an excellent procedure for
boosting response rate, Bicycling Magazine would be unlikely to achieve the same results
today.
Slide 16
The lower the response rate, the greater the cost per completed interview. Different surveys are
associated with different costs. This can be seen in the next several slides.
Slide 17
As an example of the different costs associated with the different methods, central telephone
surveys have telephone charges, in addition to any charges associated with computer recording
of responses and computer administration of surveys. In contrast, the cost of mall intercept
interviews includes location rental, necessary equipment for respondent stimuli, supplies, and
product storage. The costs for personal or door-to-door interviewing include interviewer travel
time, and mileage.
Slide 18
The next two slides illustrate the increased costs associated with reduced incidence rate and
increased length of interview. For example, with a 5% incidence rate and an hour-long
Page | 3
telephone interview, the cost is $78.25 per completed interview. In contrast, with a 100%
incidence rate (meaning the percent of the population qualified to respond to a survey) and a 5minute telephone interview is $4.05 per completed interview. That’s a twenty fold increase for
the longer interview! In attempting to budget for phone interviews, it’s important to consider the
incidence rate and the length of the interview.
Slide 19
This table suggests that the same relative cost structure exists for mall intercept interviews as
well. When the incidence rate is 5% for an hour-long interview, the cost is $73.20; in contrast,
when the incidence rate is 100% for a 5- minute interview, the cost is $4.55, which is roughly the
same 20 to 1 ratio. Whether it’s a telephone or mall-intercept interview, the length of the
interview and the incidence rate are critical to cost estimates and subsequent cost realization.
Slide 20
The next three slides address the issue of estimating the cost of fielding a telephone survey.
The assumptions on this first slide are as follows: 300 respondents, 225 phone numbers can be
dialed per eight-hour shift (which assumes 6 ½ productive hours), an incidence rate of 65%, a
cooperation rate of 60%, and a 15-minute interview. The interviewer takes three additional
minutes to double check the completed survey and to prepare for administering the next
questionnaire. The interviewer costs $215.00 per work day.
Slide 21 (No Audio)
Slide 22
Given the assumptions from the first slide and the calculations from the second and third slide,
the estimated field costs are $3715.20. That number is unimportant. What is important is that
these are the sorts of assumptions and calculations that one must make in order to properly
budget for a telephone survey.
Slide 23
This slide shows a budget sheet for a mail questionnaire. The direct expenses include planner
salary, preparation costs for copying, layout, and photography, printing costs for paper,
envelopes, folding, collating, stapling, inserting, addressing, sorting, postage—both outbound
and return--and premiums (which could be a pencil or a promise to provide a summary of the
results). The indirect expenses are office space, office supplies, secretarial assistance, and
utilities. This budget sheet could be more detailed, but the point of showing it is to indicate the
type of data one must consider when trying to budget for a mail questionnaire.
Page | 4