CHAPTER 5
RESEARCH DESIGN
LEARNING OBJECTIVES

To understand what research design is, why it is significant, and the types

To learn how exploratory research design may be used and the methods to conduct
exploratory research

To know the fundamental questions addressed by descriptive research and the two
major types of descriptive research

To know the different types and uses of panels in marketing research

To explain what is meant by causal research, experiments and experimental design

To know the different types of test marketing and how to select test market cities
CHAPTER OUTLINE
PROPER RESEARCH DESIGN IS ESSENTIAL TO VALID RESEARCH
RECOMMENDATIONS
The Significance of Research Design
THREE TYPES OF RESEARCH DESIGNS
Research Design: A Caution
Exploratory Research
Uses of Exploratory Research
Gain Background Information
Define Terms
Clarify Problems and Hypotheses
Establish Research Priorities
Methods of Conducting Exploratory Research
Secondary Data Analysis
Experience Surveys
Case Analysis
Focus Groups
Projective Techniques
Descriptive Research
Classification of Descriptive Research Studies
Causal Research
EXPERIMENTS
60
Chapter 5: Research Design
Experimental Design
After-Only Design
One-Group, Before-After Design
Before-After with Control Group
How Valid Are Experiments?
Types of Experiments
TEST MARKETING
Types of Test Markets
Consumer Versus Industrial Test Markets
"Lead Country" Test Markets
Selecting Test Market Cities
Pros and Cons of Test Marketing
KEY TERMS
Research design
Exploratory research
Secondary data analysis
Experience surveys
Case analysis
Focus groups
Projective techniques
Descriptive research
Cross-sectional studies
Sample surveys
Longitudinal studies
Panels
Continuous panels
Discontinuous panels
Omnibus panels
Brand-switching studies
Market tracking studies
Causality
Experiment
Independent variables
Dependent variables
Extraneous variables
Experimental design
Pretest
Posttest
"True" experimental design
After-only design
Quasi-experimental designs
One-group, before-after design
Control group
61
Chapter 5: Research Design
Experimental group
Before-after with control group
Internal Validity
External validity
Laboratory experiments
Field experiments
Test marketing
Standard test market
Controlled test markets
Electronic test markets
Simulated test markets
Lead country test market
Representativeness
Degree of isolation
Ability to control distribution and promotion
ACTIVE LEARNING EXERCISES
We just used the UWF/Listener Group® panel as an example of a discontinuous
panel. Go to the website at www.uwf.edu/panel.
This exercise requires students to visit an online panel web site and learn about how it
operates. The question answers are:
Read what the web site has to say about what a panel is…
Panels
Panels are simply groups of people who have agreed to answer survey questions
periodically. Many of the top research firms, such as ACNielsen, use panels to collect
research information.
The UWF/Listener Group Panel
This is a panel of male and female heads of households in the Pensacola MSA (Escambia
and Santa Rosa counties). The panel is constructed in such a way as to be representative
of all heads of households in the two county area. There are approximately 800 persons in
the panel.
…how often panel members may be contacted…
How often are panel members surveyed? Panel members are contacted no more than
once a month.
…and how the panel protects the privacy of its panel members…
62
Chapter 5: Research Design
How private is the information gathered by the panel? Although panel managers must
have the panel member's telephone number (or email address) in order to contact the
panel member, no other information is known about the identity of the panel member
except for the panel member's first name. The first name is gathered simply to allow the
survey to be directed to the correct member of the household. Any information reported
by the panel member is never associated with the member's telephone number. Rather,
data will be reported in the aggregate and/or reported by broad demographic categories
such as by income group, zip code, or county of residence.
… take a look at the kinds of information the panel has collected by looking at some of
the survey results…
Recent survey results are posted for inspection.
For the UWF/Listener Group panel, how representative are the results and how can the
panel members determine if the results are representative?
The UWF/Listener Group Panel and Representativeness
Panel is still a sample, a subset of a larger population, to which one wishes to
make inferences. If data derived from the sample are to be inferred to the
population, the question of representativeness arises. Is the UWF/Listener Group
Panel representative?
Heads of Households with Telephones
First, the Panel was constructed so as to recruit heads of households having a
telephone. Heads of household are often used in survey research because
household heads are considered to be knowledgeable of a wide variety of
decisions, have a broad base of experience, and so on, which makes them better
qualified to answer a broad variety of survey questions. This is particularly
important for panels whose members may be asked a variety of questions from
one panel administration to another. It is important to note, therefore, that the
UWF/Listener Group Panel does not represent the general population, but rather
heads of households in the Pensacola having a telephone. (Approximately 98
percent of the households in the MSA have a telephone).
How is representativeness achieved? The sample plan , not sample size, allows
one to achieve sample representativeness. Essentially, sample plans describe how
elements, or sample units, are drawn from the population. For example, we could
“draw” elements by standing on a street corner and asking passersby if they
would participate in the panel. This is a type of sample plan (often referred to as
the “man on the street” interview). However, this type of plan, part of a number of
plans known as “non-probability” sample plans, does NOT guarantee
representativeness. Rather, “probability” sample plans guarantee
representativeness. A probability sampling plan is one in which every member of
the population (defined as heads of households with telephones) has a chance of
63
Chapter 5: Research Design
being drawn into the sample. The UWF/Listener Group Panel was recruited using
a probability sample plan known in the marketing research industry as a “RDD
Sample.” In this sample plan, computer programs generate lists of randomly
selected telephone numbers. The numbers are then screened to eliminate not
working (or unassigned) blocks of numbers. By randomly generating numbers, we
are assured that everyone in the defined population has a chance of being included
in the sample. The RDD sample for this project was supplied by Survey
Sampling, Inc.http://www.surveysampling.com/ssi_home.html, a major sampling
firm in the U.S. So, in summary, the UWF/Listener Group Panel is a
representative sample of heads of households with telephones in the Pensacola
MSA.
(There is more on this topic on the website.)
TEACHING SUGGESTIONS
1. (The following will require giving an assignment to students and having them bring to
class their findings.) Although students have not been fully exposed to exploratory
research methods, they can perform three of them to some extent. Divide the class up
into three groups and assign each group one of the following types of exploratory
research: (1) secondary data analysis, (2) experience surveys, and (3) case analysis.
Here are some possible “assignment” situations. It is necessary to keep the situations
local.

The university is experiencing a decline (increase) in enrollment and wonders
why. (Chose “decline” or “increase” to fit the situation of your institution.
Alternatively, consider using a particular major or a college as it suits your
situation and students’ abilities.)

The university bookstore is considering going to an “all rent” policy for
textbooks. (Some universities have this policy in place. Note: the experience
survey will pertain to students’ and instructors’ experiences with buying and
selling back textbooks.)

The director of your “student life” office is concerned about the high use and
possible abuse of credit cards by college students. Under consideration is a selfhelp program for students to learn about credit card abuse and money
management.
Have the groups present their findings and use class discussion to generate
hypotheses.
2. Focus groups are covered in detail in chapter 8, and projective techniques are
described in greater detail in that chapter as well. Resist the temptation to tell all
about these methods to students at this time unless you have something else to present
for your qualitative research techniques coverage later in the course.
64
Chapter 5: Research Design
3. Illustrate the result of a cross-sectional descriptive study by presenting findings from
such a study to the class. If you have used team projects in the past, you have many
from which to pick. Alternatively, you may use a consulting report, or even the
descriptive findings from an academic study may suffice. Use the tables and figures
to illustrate the descriptive nature of the findings. For instance, show how the
demographic profile describes the sample. If there is product use information, use it
to show how it describes the types of product usage.
4. Some students may not readily understand the value of a longitudinal panel. An
analogy is that a cross-sectional survey is like a snapshot; whereas a longitudinal
panel is like a video.
5. A tangential discussion can be generated by asking students the implications of
attrition with an omnibus panel versus a longitudinal panel. In the former case, the
lost panel member can be replaced with someone whose profile is identical or very
similar. That is, if a female elderly sole survivor living on the West Coast drops out,
you would replace her with an elderly sole survivor who lives on the West Coast.
However, with a drop out in a longitudinal panel, the string of observations over time
is broken, so attrition is a much more serious problem.
6. You can have fun with causality because humans have the inborn tendency to make
causal attributions without using good experimental design. Here are some examples.

It has been documented that human births increase as stork populations grow, so
storks bring babies. (Actually, rain causes crops to grow, so agrarian people
expand their families, while water affords more protection for stork nests so more
storks are sighted.)

Lizards sometimes jump out of fires, so medieval people believed that lizards
were created by fire. (Actually, lizards jump out of logs when lit on fire, because
they hide in logs and the fire forces them to leave their hiding places.)

The sun revolves around the earth because one can see it rising and setting. It was
believed for centuries that the sun revolved around the earth because of the
fallacious assumption that the earth stood still. (In truth, the sun’s gravity causes
the earth to revolve around it while the earth spins on its axis.)
7. The after-only experimental design can be set up as a convenient straw man to
demonstrate the effects of not controlling other independent and extraneous variables.
Ask students to identify the independent variable, dependent variable, and causal
statement in one or more of the following examples.

A dry cleaning company institutes a “frequent cleaner” plan where $1 is taken off
the price of a dry cleaning order for every 10 times a customer uses the dry
cleaning company. Sales at the end of three months are $10,000.
65
Chapter 5: Research Design

A movie theater adds more parking spaces in its lot, and 3,000 tickets are sold in a
month.

Goodyear adds a line of ultra-high performance radials and sells 200,000 of them
in six months.
As an after-only design, the causal statement is that “X caused O.” That is, the
frequent cleaner plan caused $10,000 in sales; the parking spaces caused 3,000 tickets
to be sold, and that addition of the radials caused Goodyear to sell 200,000 of them.
However, students will generate a great number of other possible reasons why the
sales were as described. Other independent variables may be at work, such as an
especially popular movie being shown during the month. Extraneous variables are
unaccounted for as well as competitors’ actions or changes in customers.
8. The descriptions of internal and external validity in the chapter do not utilize terms
used in more comprehensive treatments. For internal validity, the terms are history,
maturation, instrumentation, pretest effect, equivalence, and mortality, while for
external validity, they are sample representativeness, artificiality, and generalizability.
Some instructors may want to introduce these terms to students when reviewing the
two types of validity.
9. The two experimental designs without control groups can be used to illustrate the role
of a control group. The after-only and one-group, before-after designs suffer from
internal validity problems, namely extraneous effects and changes in subjects in both
cases, and measure error and subjects guessing for the before-after design. The way
to overcome these problems is to let them happen to both the experimental and the
control group. That is, you must have a control group to “capture” the internal
validity problem, and then subtract it out. The description of the before-after with
control group and the causal statement formula where E = (O2 - O1) - (O4 - O3) shows
how the control groups O3 and O4 do the subtracting out.
10. There are experimental designs that are more complex and which are not covered in
the chapter. Some instructors may want to expand students’ knowledge by bringing
these into the classroom.
11. Students can relate to a taste test as an example of a laboratory experiment, and they
can be questioned on why, for example, when Diet Pepsi does a blind taste test
against Diet Coke, findings show a slight preference for Diet Pepsi. However, Diet
Coke sales are more than those for Diet Pepsi. The artificiality of the blind taste test
can be exploited. There are extraneous and independent variables affecting brand
loyalty, plus who does a blind taste test of two colas to decide which one to buy?
12. Consider doing an abbreviated taste test in class. You can select some students to be
the administrators, and two or three to be the taste testers. One approach is to tell the
administrators to read up on taste tests and to design and implement one themselves.
66
Chapter 5: Research Design
Another is to bring the materials to class and select the administrators and tasters
from class, but you will need to orient the administrators on how to conduct the test.
Here are the instructions for three colas with six subjects. It is best to use Dixie cupsized cups as subjects must take several drinks. It is fun to use national brands like
Coke or Pepsi along with private brands like Sam’s Choice or Dr. Thunder.
Instructions for Conducting an In-Class Taste Test
1. Pour the colas into identical large containers suitable for storage and
decanting. Assign the three colas letters (A, B, C).
2. Select subjects at random.
3. Assign the specific order of the taste tests to each subject:
ABC, ACB, BCA, BAC, CAB, CBA
4. Subjects drink bottled water.
5. Pour the first letter cola for each subject; each tastes the cola and rates it.
6. Subjects drink bottled water.
7. Pour the second letter cola for each subject; each tastes the cola and rates it.
8. Subjects drink bottled water.
9. Pour the third letter cola for each subject; each tastes and rates the cola.
10. Subjects identify 1st choice with respect to best taste rating.
For class enjoyment: Let the subjects guess what cola brands they have tasted,
then divulge the brands to them and the class.
13. There are four types of test markets described: standard, controlled, electronic, and
simulated. Ask students who uses each type and why. One factor is the distribution
system of the company wishing a test market; another is the type of product being
tested; a third factor is the type of information desired.

Standard test markets fit companies with fully developed distribution systems that
are marketing convenience goods such as grocery products, personal care items,
or over-the-counter drugs. This is a “do-it-yourself” approach with the
information need being primarily to track the test product’s sales.

Controlled test markets fit companies without fully developed distribution
systems. The test market company establishes the retail structure during the test.
Convenience goods marketers are a good fit. There is more competitive sales
67
Chapter 5: Research Design
information available because all brands are tracked, so the information needs are
more specific to what brand is gaining or losing during the test.

Electronic test markets are limited to those companies testing products that are
sold in the stores where the electronic tracking system is in place (grocery and
drug stores primarily). Because electronic test market customers are in a
database, the information would profile the buyers versus nonbuyers and provide
market segmentation implications.

The simulated test market is suited to a new product where the innovation is tried
and the repurchase rate estimated by mathematical model. The distribution
system may be nonexistent, or the cost of other marketing variables needed in the
test (e.g., training salespeople to demonstrate it, or a large-scale educational
advertising campaign) may be prohibitive. There is an implicit evaluative process
and adoption decision on the part of the consumer, and its outcome is the primary
information need.
14. Principles of marketing textbooks sometimes list 10 to 20+ U.S. test market cities.
One exercise is to take these listings over the past 10 years and see which cities have
been added or deleted from the lists. If one of these is known to students due to its
proximity to the university, students may be able to comment on why it was dropped
or added.
15. The representativeness issue is paramount in selecting a test market city. Although
the cities cited in some textbooks (Tulsa, Oklahoma; Charleston, West Virginia;
Lexington-Fayette, Kentucky; Wichita, Kansas; Rockford, Illinois; Omaha, Nebraska;
and Midland, Texas) are well known for being similar, in many respects, to the
“average U.S. city,” they do not fit the target market definitions of all marketers. Ask
students to indicate products, services, or brands whose target markets are not
“average Americans” and who would be ill-advised to use these cities in their market
tests. Here are some examples of product markets that are not comprised of average
Americans:
In-line skates
Fly fishing equipment
Jazz music CDs
White water rafting adventures
Your students can probably generate a much better list of nonqualifiers.
ANSWERS TO END-OF-CHAPTER QUESTIONS
1. How would you match research designs with various research objectives?
Review question. To answer this question, students can use Table 5.1.
68
Chapter 5: Research Design
This is done in Table 5.1, repeated below.
Research Objective
To gain background information,
to define terms, to clarify problems and
hypotheses, to establish research priorities
Appropriate Design
Exploratory
To describe and measure marketing phenomena
at a point in time
Descriptive
To determine causality, to make “if-then”
statements
Causal
2. Give some examples illustrating the uses of exploratory research.
Review question. Students must demonstrate an understanding of exploratory
research.
Exploratory research is unstructured, informal research that is undertaken to gain
background information about the general nature of the research problem. It is
usually conducted when the researcher does not know much about the problem and
needs additional information or desires new or more recent information. The
purposes of exploratory research are to (1) gain background information, (2) define
terms, (3) clarify problems and hypotheses, and (4) establish research priorities.
Any number of examples exists. The types described in the chapter are: (1)
secondary data analysis, (2) experience surveys, (3) case analysis, (4) focus groups,
and (5) projective techniques.
For class discussion, ask students what types of exploratory research they would
undertake if they were managers of a food service center on campus, and the school
newspaper just published a blistering editorial on the poor quality of food and service
in their unit. Some exploratory research methods include:
Research method
Example
Secondary data analysis
stories.”
Reading about college food service “success
Experience surveys
Discussing the matter with student leaders.
Asking the newspaper editor for more
information.
Asking cashiers what comments students
have made to them.
69
Chapter 5: Research Design
Case analysis
in the food
Observing what students do and do not eat
service center.
Focus group
Inviting students to participate in a open
meeting on how the food service can do
better.
Projective techniques
Giving students a sheet of paper and asking
them to complete the statement, “This food
service center would be a better place if...”
3. What type of research design answers the questions of who, what, where, when, and
how?
Review question. To answer this question, students will need to realize that these
questions are basic elements of descriptive research
This is part of the definition of descriptive research.
 Who are our customers?
 What brands do they buy and in what quantities?
 Where do they buy?
 When do they buy?
 How do they buy?
4. What are the differences between longitudinal studies and cross-sectional studies?
Review question. Students must distinguish between these two types of studies.
By definition, cross-sectional studies measure a population at only one point in time,
while longitudinal studies repeatedly measure the same population over a period of
time. One might make the analogy that a cross-sectional study is a snapshot and a
longitudinal study is a video.
5. In what situation would a continuous panel be more suitable than a discontinuous
panel? Under what situation would a discontinuous panel be more suitable than a
continuous panel?
Review question. This question tests students’ comprehension of these two panel
types.
Continuous panels ask panel members the same questions on each panel
measurement. Discontinuous panels vary questions from one panel measurement to
the next. Unlike the continuous panel, discontinuous panels may be used for a variety
of purposes because the information collected by an discontinuous panel varies from
one panel measurement to the next. Continuous panels are noted in the chapter as
70
Chapter 5: Research Design
suited to brand switching and market tracking studies, while discontinuous panels are
of the “snapshot” variety. That is, comparisons are made between groups (say
MasterCard owners versus Discover Card owners) existing within the entire
discontinuous panel.
6. Explain why studies of “if-then” variety are considered to be causal studies.
Review question. Students must relate the concept of manipulating an independent
variable.
Causality may be thought of as understanding a phenomenon in terms of conditional
statements of the form “If X, then Y.” These “if-then” statements, then, become ways
of manipulating independent variables of interest.
7. What is the objective of good experimental design? Explain why certain designs are
called “quasi- experimental” design.
Review question. This question tests students’ understanding of experimental design.
An experimental design is a procedure for devising an experimental setting such that
a change in a dependent variable may be attributed solely to the change in an
independent variable. Experimental designs are procedures that allow experimenters
to control for the effects on a dependent variable by an extraneous variable. In this
way, the experimenter is assured that any change in the dependent variable was due
only to the change in the independent variable.
A “true” experimental design is one that truly isolates the effects of the independent
variable on the dependent variable while controlling for effects of any extraneous
variables. Designs that do not properly control for the effects of extraneous variables
on the dependent variable are known as quasi-experimental designs. In the quasiexperimental design diagrams there is no measure of E, the “experimental effect,” on
the dependent variable due solely to the independent variable. This is true in all
quasi-experimental designs.
8. Explain the two types of validity in experimentation and also explain why different
types of experiments are better suited for addressing one type of validity versus
another.
Review question. Students are required to discuss internal and external validity.
An experiment is valid if (a) the observed change in the dependent variable is, in fact,
due to the independent variable, and (b) if the results of the experiment apply to the
“real world,” outside the experimental setting. Two forms of validity are used to
assess the validity of an experiment: internal and external. Internal validity is
concerned with the extent to which the change in the dependent variable was actually
due to the independent variable. External validity refers to the extent that the
71
Chapter 5: Research Design
relationship observed between the independent and dependent variable during the
experiment is generalizable to the “real world.” Also, refer to Table 5.5.
We distinguished between laboratory and field experiments. Laboratory experiments
are those in which the independent variable is manipulated and measures of the
dependent variable are taken in a contrived, artificial setting for the purpose of
controlling the many possible extraneous variables that may affect the dependent
variable. So lab experiments are better suited for addressing internal validity. Field
experiments are those in which the independent variables are manipulated and the
measurements of the dependent variable are made on test units in their natural setting.
Field experiments are better suited for addressing external validity.
9. Distinguish among the various types of test marketing.
Review question. In order to answer this question, students must review the various
types of test markets described in the chapter.
Test marketing is the phrase commonly used to indicate an experiment, study, or test
that is conducted in a field setting. The chapter classifies test markets into four types:
standard, controlled, electronic, and simulated. The standard test market is one in
which the firm tests the product and/or marketing mix variables through the
company’s normal distribution channels. Controlled test markets are conducted by
outside research firms who guarantee distribution of the product through pre-specified
types and numbers of distributors. Electronic test markets are those in which a panel
of consumers has agreed to carry an identification card that they present when buying
goods and services. Simulated test markets (STMs) are those in which some limited
amount of data on consumer response to a new product are fed into a model
containing certain assumptions regarding planned marketing programs, which
generates likely product sales volume. There are also consumer versus industrial test
markets. In consumer tests, multiple versions of a more-or-less finished product are
tested by consumers. In industrial test markets, the key technology is presented to
selected industrial users who offer feedback on desired features and product
performance levels.
10. Think of a past job that you have held. List three areas in which you, or some other
person in the organization, could have benefited by having information generated by
research. What would be the most appropriate research design for each of the three
areas of research you have listed?
Application question. This exercise asks students to relate how information could
help an organization with which they are familiar.
The purpose of this question is to have students relate research design to something
with which they are familiar. Students’ work experiences will vary greatly and may
prove interesting as a basis for class discussion.
72
Chapter 5: Research Design
11. Design an experiment. Select an independent variable and a dependent variable.
What are some possible extraneous variables which may cause problems? Explain
how you would control for the effects these variables may have on your dependent
variable. Is your experiment a valid experiment?
Application question. Students must apply experimental design concepts.
The purpose is to have students apply an experimental design and thus pick out
independent, dependent, and extraneous variables as well as assess the internal and
external validity of the experiment. Class discussion can be generated by having a
student “present” his or her design to the class.
12. The Maximum Company has invented an extra-strength instant coffee brand to be
called “Max-Caff,” and positioned to be stronger tasting than any competing brands.
Design a taste-test experiment that compares Max-Caff to the two leading instant
coffee brands to determine which brand consumers consider to taste the strongest.
Identify and diagram your experiment. Indicate how the experiment is to be
conducted, and assess the internal and external validity of your experiment.
Application question. A taste test is an experiment, so students must
demonstrate comprehension of experimental design concepts.
There is no need to perform a pretest as there is no baseline to taste tests.
Max-Caff is to be compared to the leading brands, A and B (Maxwell House
and Folger’s, for instance). With a taste test, the design can be within subjects
or across subjects. For the within design, the same subject tastes all three
coffees and rates the taste of each. Typically, a neutral stimulus such as a
soda cracker or a drink of water is administered between the randomly ordered
stimuli. The panel design would look like the following:
X1
O1
N
X2
O2
N
X3
O3
Where the Xis are the randomly ordered stimuli, the Ois are the measurements,
and the Ns are the neutralizers.
With an across subjects design, three groups are used, each with its own
stimulus. Subjects are assigned randomly to each group, and the after-only
design diagram looks like this:
X1
X2
X3
O1
O2
O3
No control group is used as it does not make sense to have subjects rate the
strength of a nonstimulus.
73
Chapter 5: Research Design
Internal validity is addressed in the following ways.

Extraneous factors – eliminated if the panel design is random and if the three
groups take place in the same circumstances and time frame

Changes in subjects – obviated by the neutral stimulus in the panel and the short
time frame in the three-group design

Measure error – use the same measure

Subjects guessing – could happen but will be random and cancel out

Equivalent groups – use random selection or matching criteria

Drop-out – should be monitored
External validity is an issue as the taste test is conducted in an artificial setting
(perhaps at a shopping mall or some other facility), and the subjects are told
that they are to rate the taste which means that they will be especially vigilant
to the taste. Also, the long-term taste aspects of the coffees are not tested, so
if Max-Caff “stays with” the taster, this factor is not captured by the design.
Of course, the sample should be representative of coffee drinkers.
13. Coca-Cola markets PowerAde as a sports drink that competes with Gatorade.
Competition for sports drinks is fierce where they are sold in the coolers of
convenience stores. Coca-Cola is thinking about using a special holder that fits in a
standard cooler but moves PowerAde to eye level and makes it more conspicuous
than Gatorade. Design an experiment that determines whether or not the special
holder increases the sales of PowerAde in convenience stores. Identify and diagram
your experiment. Indicate how the experiment is to be conducted, and assess the
internal and external validity of your experiment
Application question. This question describes an experimental design that
students must recognize and analyze.
There is one treatment (the cooler holder for PowerAde) and one control
situation (the current stocking of PowerAde without the cooler holder). A
before-after with control group or an after-only with control group design
would be satisfactory. The after-only with control group design looks like the
following:
X1
O1
O2
The groups are randomly assigned convenience stores such as Circle Ks or 711s, and the Os are store sales, and the effect of the PowerAde cooler holder is
74
Chapter 5: Research Design
O1:O2, and if the cooler holder average PowerAde sales are greater than the
control group average PowerAde sales, the experiment has shown that the
cooler holder has caused the greater sales
Internal validity is addressed in the following ways:


Extraneous factors – eliminated groups take place in the same circumstances and
time frame
Changes in subjects – if groups are matched or random, changes will be the same
for all

Measure error – use the same measure – sales of PowerAde

Subjects guessing – no pretest, so no problem

Equivalent groups – use random selection or matching criteria

Drop-out – probably no problem, but should be monitored
External validity is taken care of if the groups are representative of all
convenience stores.
The test is realistic and generalizable.
14. SplitScreen is a marketing research company that tests television advertisements.
SplitScreen has an agreement with a cable television company in a medium-sized city
in Iowa. The cable company can send up to four different television ads
simultaneously to different households. SplitScreen also has agreements with the
three largest grocery store chains, which will provide scanner data to SplitScreen.
About 25 percent of the residents have SplitScreen scan cards that are scanned when
items are bought at the grocery store and which allows SplitScreen to identify who
bought what grocery products. For allowing SplitScreen access to their television
hook-ups and their grocery purchases information, residents receive bonus points
that can be used to buy products in a special points catalog. Identify and diagram
the true experimental designs possible using the SplitScreen system. Assess the
internal and external validity of SplitScreen’s system.
Application question. This question introduces students to scanner data experimental
design considerations and possibilities.
Virtually any true experimental design is possible with the Split Screen system, but
the most powerful ones would be before-after with control group or after-only design.
The system allows for up to four groups – either four experimental groups or three
experimental groups and one control group. By combining or not using groups,
fewer groups can be used for a less complex design.
75
Chapter 5: Research Design
All internal validity problems are under control assuming that groups are equivalent,
drop-outs are replaced, commercials are seen at the same time, stores remain
constant, etc. The external validity may concern some students who will point out
the artificiality of the scanner cards, bonus points, and the use of a medium-sized city
in Iowa. If Split Screen recruits households and does not use them right away and
thus lets them settle back into their normal shopping patterns after what might be a
period of unusual shopping because they feel “watched,” the artificiality will be
minimized. The medium-sized city in Iowa is consistent with good market test
selection criteria.
CASE SOLUTIONS
Case 5.1 Quality Research Associates
Case Objective
This case presents students with six different situations where they must determine the
appropriate research design.
Answers to Case Questions
1. What research design do you think Sam Fulkerson should select for each of his clients?
2. For each research design you specify in question 1, describe the reason(s) you selected
it.
The answers are provided in the table below.
Client
Hamptons Bank
Research Design
Descriptive research
Wesleyan College
Exploratory research
M&M Mars candy
bars
M&M Mars cereal
Exploratory research
McBride’s Markets
Causal research
Phillips, Yarbrough
and Rogers
Descriptive research
Causal research
76
Reason(s)
Need to measure the bank’s image with
a cross-sectional survey
Early stage in this decision; need to
clarify opinions (probably with
focus groups)
Will require an extensive literature
search with secondary data
Need to conduct a taste test type study
but out of the laboratory
A “what-if” situation. The “if” is
variations of ad copy and layout,
and the “what” is some measure of
attention
The “more evidence” required by retail
chain buyers would be satisfied with
a large survey. A pretest of the two
Chapter 5: Research Design
promotional campaign themes could
be embedded in this survey.
CASE 5.2 The Hobbit’s Choice: Research Design
Case Objective
This case requires students to identify appropriate research designs for the Hobbit’s
Choice Restaurant marketing research project.
Answers to Case Questions
1. What type of research design did Cory Rogers use to be prepared for the second
meeting with his client, Jeff Dean? What are some methods that he likely used?
Given that he was armed with financial and performance information about restaurants,
Cory most certainly used exploratory research in the form of secondary data. He
probably used an Internet search engine or, if available to him, he used a library-type data
base system of some sort. The case notes that he has ordered a marketing research report
on upscale restaurants, and this is secondary data.
2. Consider the second paragraph above which refers to the critical issues in the case.
Would exploratory research be the most appropriate research design to choose, given the
research objectives that would be generated to provide the information necessary to
resolve these issues? Explain why or why not.
The critical issues are: “a. determining demand, b. determining the best choice of
restaurant design and operating characteristics, c. determining where to locate the
restaurant, and d. determining how to promote The Hobbit’s Choice Restaurant.” These
issues are all in the realm of primary demand: he needs specific information about each
one.
Students might mistakenly recommend focus groups; however, although some useful
information would be gained, the generalizability of this information would be very low.
3. If you elected not to use exploratory research design in question 2, which of the two
remaining research designs would be more appropriate - descriptive or causal? Select
one and provide the rational for you choice.
Descriptive research is definitely called for as we need answers to who, what, when, how,
and so on. So, this requires a survey. Jeff needs information on how to design the
restaurant, where to locate it, how to promote it, and so forth.
77
Chapter 5: Research Design
4. How does selection of the research design you chose in question 3 aid Cory Rogers in
planning the research project for The Hobbit’s Choice Restaurant?
The choice of a research design such as descriptive research has a number of implicit
decisions. For example, the objectives must be set, questionnaire designed, data
collection method decided, sample plan identified, sample size determined, and so on.
So, when the basic research design is decided, the researcher knows what decisions must
be made (and in what order if he/she refers to the 11-step process for marketing research
described in this textbook.)
78