Chapter 9
Experiments
McGraw-Hill/Irwin
Copyright © 2011 by The McGraw-Hill Companies, Inc. All Rights Reserved.
Learning Objectives
Understand . . .
• Uses for experimentation.
• Advantages and disadvantages of the
experimental method.
• Seven steps of a well-planned experiment.
• Internal and external validity with experimental
research designs.
• Three types of experimental designs and the
variations of each.
9-2
Experiments Challenge Perceptions
“There is no such thing as a failed
experiment, only experiments with
unexpected outcomes.”
Richard Buckminster Fuller,
engineer and architect
9-3
PulsePoint:
Research Revelation
45
The percent of smartphone users who
check their e-mail before they get
dressed.
9-4
Causal Evidence
Agreement between
IVs and DVs
Time order of occurrence
Extraneous variables
did not influence DVs
9-5
Causal Evidence?
9-6
Evaluation of Experiments
Advantages
• Ability to manipulate IV
• Use of control group
• Control of extraneous
variables
• Replication possible
• Field experiments
possible
Disadvantages
• Artificiality of labs
• Non-representative
sample
• Expense
• Focus on present and
immediate future
• Ethical limitations
9-7
Experimentation in the Research
Process
9-8
Conducting an Experiment
Specify treatment variables
Specify treatment levels
Control environment
Choose experimental design
Select and assign participants
Pilot-test, revise, and test
Collect data
Analyze data
9-9
Experiment: Placement of
Benefits Module
9-10
Selecting and Assigning
Participants
Random
assignment
Matching
9-11
Random Assignment
9-12
Quota Matrix Example
9-13
Measurement Options
Observation
Physiological
measures
Paper-andpencil tests
Options
Scaling
techniques
Selfadministered
instruments
9-14
Validity in Experimentation
Internal
External
9-15
Threats to Internal Validity
Maturation
Experimental
mortality
History
Threats
Statistical
regression
Testing
Instrumentation
Selection
9-16
Additional Threats to Internal
Validity
Diffusion of treatment
Compensatory equalization
Compensatory rivalry
Resentful disadvantaged
Local history
9-17
Threats to External Validity
Reactivity of
testing on X
Interaction of
selection and X
Other
reactive factors
9-18
Experiments Challenge Perceptions
“We need to keep an open mind and approach
life as a series of experiments. We need to
observe the experiments happening around us
and create new ones. Instead of accepting the
world as we think it is, we need to keep testing it
to find out what it is and what works.”
Jerry Wind
Wharton School of Business,
University of Pennsylvania
9-19
Experimental Research Designs
Pre-experiments
True experiments
Field experiments
9-20
After-Only Case Study
X
Pre-experiment
O
9-21
One Group Pretest-Posttest
O1
Pre-experiment
X
O2
9-22
Static Group Comparison
X
O1
O2
Pre-experiment
9-23
Pretest-Posttest Control Group
Design
R
R
True experiment
O1
O3
X
O2
O4
9-24
Posttest-Only
with Control Group
R
R
True experiment
X
O1
O2
9-25
Nonequivalent Control Group
Design
O1
O3
Field experiment
X
O2
O4
9-26
Separate Sample
Pretest-Posttest
R
R
Field experiment
O1
(X)
X
O2
9-27
Group Time Series Design
R
R
Field experiment
O1 O2 O3 X O4 O5 O6
O7 O8 O9
O10 O11 O12
9-28
Job Enrichment
Quasi-Experiment
9-29
Experiment:
Refining Store Design
9-30
Experiment:
The Right Size of Flavor
9-31
Key Terms
• Blind
• Control group
• Controlled test
market
• Dependent variable
• Double-blind
• Environmental
control
• Experiment
• Experimental
treatment
• External validity
• Field experiment
• Hypothesis
• Independent variable
• Internal validity
9-32
Key Terms
•
•
•
•
•
Matching
Operationalized
Quota matrix
Random assignment
Replication
• Test market
–
–
–
–
Electronic test market
Simulated test market
Standard test market
Virtual test market
• Treatment levels
• Web-enabled test
market
9-33
Appendix 9b
Test Markets
9-34
Test Market Selection
Over-testing
Control of
distribution
Representative
Criteria
Multiple
locations
Isolation
Media
coverage
9-35
Types of Test Markets
Standard
Controlled
Electronic
Simulated
Virtual
Web-enabled
9-36
Test Market Cities
9-37