Chapter Eight
Experimentation
in Marketing
Research
Eddie Bauer's Electronic Windows
• Eddie Bauer, a leading tri-channel specialty
retailer was looking for a way to draw more
shoppers into their stores
• Indiana University students conducted an instore advertising experiment using electronic
window posters (images displayed on
plasma screens) on 3 selected stores
Copyright © Houghton Mifflin Company. All rights reserved.
8|2
Electronic Windows (Cont’d)
• RESULTS
– The number of passersby who entered the
control stores went up 7 percent
– Sales soared 56 percent compared to the
weeks before the installation of digital
windows
Copyright © Houghton Mifflin Company. All rights reserved.
8|3
Advertising Experiment
• Will replacing commercial A with commercial
B lead to a marked increase in consumer
preference for a company’s brand?
Copyright © Houghton Mifflin Company. All rights reserved.
8|4
Pricing Experiment
• Can a company improve the profitability of its
fashion clothing line by increasing its price by
10 percent?
Copyright © Houghton Mifflin Company. All rights reserved.
8|5
Sales Productivity Experiment
• Will an increase in the average number of
sales calls per customer from six to eight per
year significantly improve sales?
Copyright © Houghton Mifflin Company. All rights reserved.
8|6
Shelf Space Experiment
• Will decreasing the shelf space allocated to
brand X detergent by 25 percent significantly
lower its sales?
Copyright © Houghton Mifflin Company. All rights reserved.
8|7
Direct Mail Experiment
• Will it be worthwhile to mail last year's donors
an attractive (but expensive) brochure
describing the company’s activities and
soliciting higher contributions for this year?
Copyright © Houghton Mifflin Company. All rights reserved.
8|8
Experiment
• An experiment is a procedure in which a
company manipulates one (or sometimes
more than one) independent or cause
variable and collects data on the dependent
or effect variable while controlling for other
variables that may influence the dependent
variable
Copyright © Houghton Mifflin Company. All rights reserved.
8|9
Descriptive Research
• This research asks consumers whether they
would buy more of a product if its price were
lowered
• Descriptive survey data will merely suggest
causation
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 10
Experimental Research
• Manipulates the independent variable or
variables before measuring the effect on the
dependent variable
– The effect of price changes on sales volume of
a particular product can be examined by
actually varying the price of the product
• The very basis of experimental research lies
in the manipulation of independent variables
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 11
Conditions For Inferring Causality
• Temporal ordering of variables
– X  Y not Y  X
• Evidence of association
– X and Y are related ; presence of X 
presence of Y; absence of X  absence of Y
• Control of other causal factors
– X  Y, Z  Y
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 12
Laboratory vs. Field Experiments
• A laboratory experiment is a research study
conducted in a contrived setting in which the effect of
all, or nearly all, influential but irrelevant independent
variables is kept to a minimum
• A field experiment is a research study conducted in
a natural setting in which the experimenter
manipulates one or more independent variables
under conditions controlled as carefully as the
situation will permit
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 13
Internal Validity
• Internal validity is the extent to which
observed results are solely due to the
experimental manipulation
• Laboratory experiments are generally high
on internal validity
• Field experiments are generally low on
internal validity
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 14
External Validity
• External validity is the extent to which
observed results are likely to hold beyond the
experimental setting
• Laboratory experiments are generally low on
external validity
• Field experiments are generally high on
external validity
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 15
Deciding Which Type of
Experiment to Use
• Practical Considerations
–
–
–
–
Time
Cost
Exposure to competition
Nature of the manipulation
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 16
Test Marketing
• Dunkin Donuts and Baskin-Robbins are now
offered in “combo” stores
• KaBloom is testing kiosk flower sales in a
variety of locations
• Utilities companies are experimenting with
providing Internet services via existing power
lines
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 17
McDonald's Tests McPizza
• McDonald's test-marketed McPizza to
strengthen the after-4pm adult market
– Introduced McPizza with heavy advertising,
emphasizing speedy service for pizza
– McPizza received favorable nods in some test
markets and had partial rollout nationally
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 18
McDonald's Tests McPizza (Cont’d)
• Pizza Hut, a leading competitor, reacted
aggressively to McDonald's move by running
a buy-one-get-one-free promotion wherever
McPizza was introduced
• The sales performance of McPizza did not
meet management's expectations
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 19
Simulated Test Marketing
www.harrisinteractive.com/advantages/marketingsciences.asp
• Step #1 Pre-recruitment
• Step #2 Background: habits and practices
• Step #3 Exposure to real advertising in a
competitive context
• Step #4 Simulated store purchase
• Step #5 Post: purchase inquiry
• Step #6 Respondents take product home for
usage
• Step #7 Post: usage evaluation
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 20
Virtual Test Markets-- Ray Burke, Professor of
Business Administration at Indiana University
• Created a virtual store to determine how
products catch a consumer's eye
• Computer 3-D graphics create a feeling of
being in a store, walking past shelves of
grocery items just as in a real store
• Consumers can pick items off the virtual
shelves to examine them as in real store and
can select items they would buy
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 21
Virtual Test Markets
• Virtual simulated marketing tests will enable
companies to examine consumers' reactions
to new products, product line extensions,
prices, packaging, and merchandising
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 22
Scanner Data Analysis
• Electronic scanners at the checkouts capture
the product sales
• Marketers of packaged goods conduct sophisticated
field experiments
• The data from the stores are transmitted
electronically to central computers for analysis and
interpretation
• Information Resources Inc. (IRI) and ACNielsen offer
marketers a variety of services through
their information system called
BehaviorScan and Scantrack
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 23
Web-Based Experiments
• Web-based experimentation will enable
companies to test a wide range of possible
marketing mix changes and statistically
model consumer responses to these changes
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 24
Web-based Experiments Conducted to Test
the Effectiveness of Banner Advertising
• Random people were selected while visiting
the company’s website
• The questionnaire asked them to complete a
short questionnaire while on the site
• Participants are shown random test banner
ads
• Participants fill out a second survey,
answering questions about the impact of the
banner ads on their impressions of the brand
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 25
Web-based Experiments Conducted to Test
the Effectiveness of Banner Advertising (Cont’d)
• What are the independent variables?
• What is the dependent variable?
• What are some validity threats?
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 26
Internal Validity
• The presence of any condition or occurrence
(other than the independent variable
manipulation) that can offer a compete
explanation for the experimental results is a
threat to internal validity
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 27
Threats to Internal Validity
• History - external events
• Maturation - physiological or psychological
changes that occur over time
• Pretesting - Early responses impact later
responses
• Instrument Variation - questionairre changes
• Selection - groups differ on characteristics
• Mortality - participants drop put
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 28
Threats To External Validity
• External validity of experimental results relates to
their generalizability
• The various internal validity threats also indirectly
affect external validity
• Biases that stand in the way of generalizing
experimental results:
– Reactive bias
– Pretest-manipulation interaction bias
– Non-representative-sample bias
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 29
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 30
Hyundai India’s Experiment
• Hyundai wanted to test and increase its
brand awareness in rural India
• Hyundai attempted to identify and establish
relationships with village leaders
– The leaders are the only ones offered a test
drive of the vehicle
– A van with a video screen is brought to the
village
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 31
Hyundai India’s Experiment (Cont’d)
• Advertisements are shown to the assembled
villagers via the vans’ screens
• In some cases, the leaders take this
opportunity to announce they have decided to
purchase the vehicle
• These types of field experiments are not
conventional field experiments, they are
intended to gauge the effectiveness of a
novel marketing technique
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 32
Hyundai India’s Experiment (Cont’d)
• What do you think of the Hyundai
experiment?
• What causal inference is implied in this
scenario?
• What is your evaluation of the validity of the
implied causal inference?
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 33
Pre-Experimental Designs
• Pre-experimental designs exert little or no
control over the influence of extraneous
factors
• These studies are not much better than
descriptive studies when it comes to making
causal inferences
• Pre-experimental: emphasizes the fact that
these studies are more exploratory than
conclusive as far as causal inferences are
concerned
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 34
Reasons for Studying
Pre-Experimental Designs
• Studies employing pre-experimental designs
often form the basis for causal inferences in
the real world, and we need to be aware of
their pitfalls to avoid interpreting their findings
at face value
• Comparisons with pre-experimental designs
can help highlight the merits of true
experimental designs
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 35
One-Group, After-Only Design
• Situation A. A company introduces a new brand of
margarine in four test market areas and employs a
unique and revolutionary promotional campaign for it
– The brand captures at least a 10 percent share in each
market within two months after introduction
– The company's management concludes that the
revolutionary promotional campaign played a major
role in the market share achieved by the brand
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 36
One-Group, After-Only Design (Cont’d)
• Situation B. The president of the United
States makes a television speech soliciting
public support for legislation favoring prayer
in public schools
– A telephone survey of those who viewed the
presidential speech indicates that 70 percent
favor such legislation
– The president's speech is therefore
considered to have had a significant impact on
the U.S. public
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 37
One-Group, After-Only Design (Cont’d)
• Casual inference from a one-group, after-only
design cannot be trusted entirely
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 38
True Experimental Designs
• The presence of one or more control groups
• The random assignment of units to various
experimental and control groups
• Random assignment distributes the sample
units chosen for a study to various groups on
a strictly objective basis so that the group
compositions can be equivalent before an
experiment is started
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 39
2004 Ford F-150 Launch
• Ford launched the redesigned F-150 with the biggest
communication blitz in its history
• 90% of the spending went to television, and it was
estimated that males 25 – 49 saw the F-150 ad 30
times during the 60 day launch
• The Internet was also extensively used for this
campaign
• An online marketing research companies was hired
to track the effectiveness of this campaign
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 40
2004 Ford F-150 Launch (Cont’d)
• The researchers used a variety of
experimental designs to isolate the effects of
the various media
• The campaign was shown to be highly
successful, and overall brand recall rose 26
percent over the course of the campaign
Copyright © Houghton Mifflin Company. All rights reserved.
8 | 41
Experimental Designs
EG
EG
O1
EG
CG
EG
CG
O1
O3
Copyright © Houghton Mifflin Company. All rights reserved.
X
O
X
O2
X
O1
O2
X
O2
O3
8 | 42