Business
Research Methods
William G. Zikmund
Chapter 12:
Experimental Research

Experiment
• A research investigation in which conditions are controlled
• One independent variable is manipulated
(sometimes more than one)
• Its effect on a dependent variable is measured
• To test a hypothesis

Basic Issues of Experimental
Design
• Manipulation of the Independent Variable
• Selection of Dependent Variable
• Assignment of Subjects (or other Test
Units)
• Control Over Extraneous Variables

The experimenter has some degree of control over the independent variable.
The variable is independent because its value can be manipulated by the experimenter to whatever he or she wishes it to be.

Experiment Treatment
Alternative manipulations of the independent variable being investigated

Independent Variable
• The experimenter controls independent variable.
• The variable’s value can be manipulated by the experimenters to whatever they wish it to be.

Manipulation of Independent
Variable
• Classificatory Vs. continuous variables
• Experimental and control groups
• Treatment levels
• More than one independent variable

Experimental Treatments
• The alternative manipulations of the independent variable being investigated

Dependent Variable
• Its value is expected to be dependent on the experimenter’s manipulation
• Criterion or standard by which the results are judged

Dependent Variable
• Selection
– e.g... sales volume, awareness, recall,
• Measurement

Test Units
• Subjects or entities whose response to the experimental treatment are measured or observed.

Two Types of Experimental Error
• Constant errors
• Random errors

Field versus
Laboratory Experiments

Controlling Extraneous Variables
• Elimination of extraneous variables
• Constancy of conditions
• Order of presentation
• Blinding
• Random assignment

How May an Experimenter control for
Extraneous Variation?
• Eliminate Extraneous Variables
• Hold Conditions Constant
• Randomization
• Matching Subjects

Establishing Control

Demand Characteristics
• Experimental procedures that intentionally hint to subjects something about the experimenter’s hypothesis

Demand Characteristics
• Guinea pig effect
• Hawthorne effect

Field Vs. Laboratory Experiment

Laboratory Experiment
Artificial-Low Realism
Few Extraneous
Variables
High control
Low Cost
Short Duration
Subjects Aware of
Participation
Field Experiment
Natural-High Realism
Many Extraneous
Variables
Low control
High Cost
Long Duration
Subjects Unaware of
Participation

Control Groups
Isolate extraneous variation

When does an Experiment have
Internal Validity?
Internal Validity - The ability of an experiment to answer the question whether the experimental treatment was the sole cause of changes in a dependent variable
Did the manipulation do what it was supposed to do?

Factors Influencing Internal
Validity
• History
• Maturation
• Testing
• Instrumentation
• Selection
• Mortality

Isolating Extraneous Variation with a Control Group
• History Effects
• Maturation Effects
• Mortality Effects

Type of Extraneous Variable
History - Specific events in the environment between the Before and After measurement that are beyond the experimenter’s control
Maturation - Subjects change during the course of the experiment
Testing - The Before measure alerts or sensitizes subject to nature of experiment or second measure.
Example
A major employer closes its plant in test market area
Subjects become tired
Questionnaire about the traditional role of women triggers enhanced awareness of women in an experiment.

Instrument - Changes in instrument result in response bias
New questions about women are interpreted differently from earlier questions.
Selection Sample selection error because of differential selection comparison groups
Control group and experimental group is self-selected group based on preference for soft drinks
Mortality - Sample attrition; some subjects withdraw from experiment
Subjects in one group of a hair dying study marry rich widows and move to Florida

How can Internal Validity
Increase?

Increasing Internal Validity
• Control group
• Random assignment
•
Pretesting and posttesting
•
Posttest only

What are the Different Basic
Experimental Designs?

Quasi-Experimental Designs
• One Shot Design (After Only)
• One Group Pretest-Posttest
• Static Group Design

One Shot Design (After Only)
X O
1

One Group Pretest-Posttest
O
1
X O
2

Static Group Design
Experimental Group X O
1
Control Group O
2

Three Good Experimental Designs
• Pretest - Posttest Control Group Design
• Posttest Only Control Group
• Solomon Four Group Design

Pretest-Posttest Control Group Design
Experimental Group
Control Group
R O
1
X O
2
R O
3
X O
4

Posttest Only Control Group
Experimental Group R
Control Group R
X O
1
O
2

One-Shot Design
Internal Validity Problems
• History
– weak
• Maturation
– weak
• Testing
– not relevant
• Instrumentation
– not relevant
• Selection
– weak
• Mortality
– weak

One-Group Pretest-Posttest
Internal Validity Problems
• History
– weak
• Maturation
– weak
• Testing
– weak
• Instrumentation
– weak
• Selection
– controlled
• Mortality
– controlled

Static-Group Design
Internal Validity Problems
• History
– controlled
• Maturation
– possible source of concern
• Testing
– controlled
• Instrumentation
– controlled
• Selection
– weak
• Mortality
– weak

Pretest-Posttest Control
Internal Validity Problems
• History
– controlled
• Maturation
– controlled
• Testing
– controlled
• Instrumentation
– controlled
• Selection
– controlled
• Mortality
– controlled

Solomon Four-Group Design
Internal Validity Problems
• History
– controlled
• Maturation
– controlled
• Testing
– controlled
• Instrumentation
– controlled
• Selection
– controlled
• Mortality
– controlled

Posttest-Only Control
Internal Validity Problems
• History
– controlled
• Maturation
– controlled
• Testing
– controlled
• Instrumentation
– controlled
• Selection
– controlled
• Mortality
– controlled

Solomon Four Group Design
Experimental Group 1:
Control Group 1:
Experimental Group 2:
Control Group 2:
R O
1
X O
2
R O
3
O
4
R X O
5
R X O
6

Advanced Experimental Designs are
More Complex
• Completely randomized
• Randomized block design
• Latin square
• Factorial

Completely Randomized Design
• An experimental design that uses a random process to assign subjects (test units) and treatments to investigate the effects of only one independent variable.

Completely Randomized Designs
Average minutes shopper spends in store
Control: no music
16
Experimental treatment: slow music
Experimental treatment: fast music
18 12

Independent Variable A
Level 1 Level 2 Level 3
Group A Group B Group C

Completely Randomized Design
With a pretest posttest
Group A R O
1
Group A
Group A
R O
3
R O
5
X
1
X
2
X
3
O
2
O
4
O
6

Completely Randomized Design
With a posttest
Group A
Group B
Group C
R X
1
R X
2
R X
3
O
1
O
2
O
3

Randomized Block Design
• An extension of the completely randomized design in which a single extraneous variable that might affect test units’ response to the treatment has been identified and the effects of this variable are isolated by blocking out its effects.

Randomized Block Design
Independent Variables
Control: no music
Experimental treatment slow music
Experimental treatment: fast music
Mornings and afternoons
Evening hours

Factorial Design
• An experiment that investigates the interaction of two or more variables on a single dependent variable.

No Music cart signs
Grocery cart signs
Independent Variable 1
No Music Slow Music Fast Music

Factorial Design -- Roller Skates
Price
$25
$30
$35
Package Design
Red Gold
Cell 1
Cell 2
Cell 3
Cell 4
Cell 5
Cell 6

• Main effect
• The influence of a single independent variable on a dependent variable.
Effects
• Interaction effect
• The influence on a dependent variable by combinations of two or more independent variables.

Men
Women
2 x 2 Factorial Design
Ad A Ad B
65
>
Main Effects of Gender
65
70 60
Main Effects of Ad

100
90
80
70
60
50
40
30
20
10
Ad A
Interaction Between Gender and
Advertising Copy
Ad B

Independent Variable 1
Level 1 Level 2
Level 1 Group A
Group B
Level 2
Group C
Group D

2 x 2 Factorial with a Pretest
Posttest
Group A
Group B
Group C
Group D
R
R
R
R
O
1
O
3
O
5
O
7
X
11
X
21
X
12
X
22
O
2
O
4
O
6
O
8

2 x 2 Factorial Design with a
Posttest Measure
Group A
Group B
Group C
Group D
R
R
R
R
X
11
X
21
X
12
X
22
O
1
O
2
O
3
O
4

A Test Market Experiment on Pricing
Test Market A, B, or C
Test Market D, E, or F
Test Market G, H, or I
Test Market J, K, or L
Sales in Units (thousands)
Regular Price
$.99
Reduced Price
$.89
130
118
87
84
145
143
120
131
Mean
Grand Mean
X
1
=104.75
X=119.58
X
2
=134.75
Cents-Off Coupon
Regular Price
153
129
96
99
X
1
=119.25

Latin Square Design
• A balanced, two-way classification scheme that attempts to control or block out the effect of two or more extraneous factors by restricting randomization with respect to the row and column effects.

Order of Usage
1 2 3
1 A B C
2 B C A
3 C A B

TEST MARKETING
Not just trying something out
Controlled experimentation
But scientific testing

Test Marketing
Not just trying something out
Controlled experimentation
But scientific testing

Test Marketing
• An experimental procedure that provides an opportunity to test a new product or a new marketing plan under realistic market conditions to measure sales or profit potential.

Functions of
Test Marketing
ESTIMATE
OUTCOMES
IDENTIFY AND
CORRECT
WEAKNESSES
IN PLANS

A Lengthy and Costly Procedure
$$$$$
Loss of
Secrecy
When not to Test?
How Long
Should a
Test Last?

Popular Test Markets
• Pittsfield,
Massachusetts
• Charlotte, North
Carolina
• Columbus, Ohio
• Little Rock, Arkansas
• Evansville, Indiana
• Cedar Rapids, Iowa
• Eau Claire,Wisconsin
• Wichita, Kansas
• Tulsa, Oklahoma
• Omaha, Nebraska
• Grand Junction.
Colorado
• Wichita Falls, Texas
• Odessa-Midland,
Texas

Selecting a Test Market
• Population size
• Demographic composition
• Lifestyle considerations
• Competitive situation
• Media
• Self-contained trading area
• Overused markets - secrecy

Control Method of Test
Marketing
• Small city
• Low chance of being detected
• Distribution is forced (guaranteed)

The Advantages of Using the
Control Method of Test Marketing
• Reduced costs
• Shorter time period needed for reading test market results
• Increased secrecy from competitors
• No distraction of company salespeople from regular product lines

Some Problems Estimating Sales
Volume
• Over-attention
• Unrealistic store conditions
• Reading competitive environment incorrectly
• Incorrect volume forecasts
– Adjusted data
– Penetration and repeat purchase rate
• Time lapse

High Tech Test Markets
Electric
Test
Markets
Simulated
Test
Markets
Virtual-reality
Simulated
Test Markets