Essentials of
Marketing Research
Chapter 9:
Experimental Research
Causal Research
• Used to
– to determine causality
– to make “if-then” statements
• Method
– Experimentation
Causal Research
•establish cause and effect relationship
•to make “if-then” statements
•Brand switching & Market tracking studies.
•involves experiments, e.g., pretest, posttest, control
groups
•problem: multiple causes and effects
•problem: hard to isolate
Education
Income
Happiness
IDENTIFYING CAUSALITY
A causal relationship is impossible to prove.
Evidence of causality:
1. The appropriate causal order of events
2. Concomitant variation--two phenomena
vary together
3. An absence of alternative plausible
explanations
EXPERIMENT
• A RESEARCH INVESTIGATION IN WHICH
CONDITIONS ARE CONTROLLED
• ONE INDEPENDENT VARIABLE IS
MANIPULATED (SOMETIMES MORE THAN
ONE)
• ITS EFFECT ON A DEPENDENT VARIABLE IS
MEASURED
• TO TEST A HYPOTHESIS
Basic Issues of Experimental Design
•Selection and Manipulation of the Independent Variable
•Selection and measurement of Dependent Variable
•Assignment of Subjects (or other Test Units)
•Control Over Extraneous Variables
Experiment treatment
• Alternative manipulations of the
independent variable being investigated
• The experimenter controls independent
variable.
• The variable’s value can be manipulated
by the experimenters to whatever they
wish it to be.
DEPENDENT VARIABLE
• Its value is expected to be dependent on
the experimenter’s manipulation
• Criterion or standard by which the results
are judged
• Selection- e.g., sales volume, awareness,
recall
• Measurement
TEST UNITS - subjects or
entities whose response to the
experimental treatment are
measured or observed.
FIELD VERSUS LABORATORY
EXPERIMENTS
Laboratory Experiment
Field Experiment
Artificial-Low Realism
Natural-High Realism
Few Extraneous
Variables
Many Extraneous
Variables
High control
Low control
Low Cost
High Cost
Short Duration
Long Duration
Subjects Aware of
Participation
Subjects Unaware of
Participation
Controlling Extraneous Variables
•Elimination of Extraneous Variables
•Order of Presentation
•Blinding (subjects knowledge about whether they are in the control or the
experiment group)
•Random Assignment
•Hold Conditions Constant
•Matching Subjects
DEMAND CHARACTERISTICS
EXPERIMENTAL PROCEDURES THAT
INTENTIONALLY HINT TO SUBJECTS
SOMETHING ABOUT THE
EXPERIMENTER’S HYPOTHESIS
e.g., Guinea Pigs ->
Establishing Control
Control Groups
Isolates Extraneous Variation
WHAT ARE THE DIFFERENT
BASIC EXPERIMENTAL
DESIGNS?
Experiments
• Manipulating an independent variable to see
how it affects a dependent variable while
controlling the effects of extraneous
variables
• Independent variable (IV):
– researcher has some control
– wishes to manipulate
• Dependent variable (DV):
– little or no control, but of interest
Symbols of Experimental Design
O = measurement/observation of DV
X = manipulation of or change in the IV
R = random assignment of subjects into
experimental or manipulated and control
groups
E = experimental effect (change in DV due to
the IV)
Pretest and posttest
Quasi-Experimental Designs
• One Shot Design (After Only)
• One Group Pretest-Posttest
• Static Group Design
Quasi-Experimental Designs
• Do not properly control for the effects of
extraneous variable on DV
• After Only : X -> O1
• One Group, Before and After
O1 X O2
Static Group Design
Experimental Group
Control Group
X
O1
O2
True Experimental Designs
• Pretest - Posttest Control Group Design
• Solomon Four Group Design
Pretest-Posttest Control Group
Experimental Design
• Design:
Experimental
Control
(R)
(R)
O1
O3
X
O2
O4
• where E = (O2-O1) - (O4-O3)
• controls for most threats to internal validity
• note that there are other experimental
designs; this design is the basic one
Four Group Design
Experimental
Control
Experimental
Control
(R)
(R)
(R)
(R)
O2 - O1 = P+E+I+U
O5 - 0.5( O1 + O3 ) = E+U
O1
O3
X
O4
X
O2
O5
O6
O4 - O3 =P+U
O6 - 0.5( O1 + O3 ) = U
E-Impact of Experiment; P-Pre-measurement;
I-Interaction between E and M; U-Uncontrollable variables.
TEST MARKETING
TEST MARKETING IS AN EXPERIMENTAL
PROCEDURE THAT PROVIDES AN
OPPORTUNITY TO TEST A NEW PRODUCT
OR A NEW MARKETING PLAN UNDER
REALISTIC MARKET CONDITIONS TO
MEASURE SALES OR PROFIT POTENTIAL.
SELECTING A TEST MARKET
•
•
•
•
•
•
•
POPULATION SIZE
DEMOGRAPHIC COMPOSITION
LIFESTYLE CONSIDERATIONS
COMPETITIVE SITUATION
MEDIA
SELF-CONTAINED TRADING AREA
SECRECY
CONTROL METHOD
OF TEST MARKETING
• SMALL CITY
• LOW CHANCE OF BEING DETECTED
• DISTRIBUTION IS FORCED
(GUARANTEED)
ADVANCED EXPERIMENTAL DESIGNS
ARE MORE COMPLEX
•
•
•
•
COMPLETELY RANDOMIZED
RANDOMIZED BLOCK DESIGN
LATIN SQUARE
FACTORIAL
Complex Experimental Designs
Control:
no music
Average minutes
shopper spends
in store
16
Experimental
treatment:
slow music
18
Copyright © 1999 by Harcourt Brace & Company
All Rights Reserved
Experimental
treatment:
fast music
12
RANDOMIZED BLOCK DESIGN
Independent Variable
Control:
no music
Experimental
treatment:
slow music
Morning and
afternoons
Evening
hours
Copyright © 1999 by Harcourt Brace & Company
All Rights Reserved
Experimental
treatment:
fast music
Factorial Design
Independent Variable 1
No Music
No grocery
cart signs
Grocery cart
signs
Slow Music
Fast Music
2 x 2 Factorial Design
Ad A
Ad B
Men
65
Main Effects
> of Gender
Women
65
70
60
Main Effects of Ad
100
90
Interaction Between Gender and
Advertising Copy
80
70
60
50
40
30
20
10
Ad A
Ad B
Factorial Design -- Roller Skates
Package Design
Price
Red
Gold
$25
$30
$35
Cell 1
Cell 2
Cell 3
Cell 4
Cell 5
Cell 6
How Valid Are Experiments?
• Internal validity: extent to which the change in
the DV was due to the IV
• External validity: extent to which the results
can be applied to the outside real world.
• Internal validity is a prerequisite for the external
validity.
INTERNAL VALIDITY?
• INTERNAL VALIDITY - THE ABILITY
OF AN EXPERIMENT TO ANSWER THE
QUESTION WHETHER THE
EXPERIMENTAL TREATMENT WAS
THE SOLE CAUSE OF CHANGES IN A
DEPENDENT VARIABLE
• DID THE MANIPULATION DO WHAT IT
WAS SUPPOSED TO DO?
Factors Influencing Internal Validity
• History
• Maturation
• Testing
• Instrumentation
• Selection
• Mortality
•Statistical Regression
Threats to Internal Validity
• History - specific, interfering (external) events occur between the first and
second measurement of subjects; Factors: time differences, type of event, level
of control;
• Maturation - (internal) change happens in respondents over time (e.g., fatigue,
getting hungry, boredom, age, learning)
• Testing - effects of taking a test upon the scores of a 2nd test
• Instrumentation - differences in measuring instruments, scorers or observers
(give inconsistent measurements)
• Statistical Regression - second scores regress toward the mean when groups
are selected on the basis of extreme scores.
• Selection - respondents are not assigned randomly to groups; each member of
sample should have equal chance of being assigned to an experimental
condition.
• Mortality - differential loss of respondents in the comparison groups
Type of Extraneous Variable
Example
History - specific events in the
Environment between the Before
and After Measurement That Are
beyond the Experimenter’s Control
A Major Employer
Closes Its Plant in
Test Market Area
Maturation - subjects Change
during the Course of the Experiment
Subjects Become
Tired
Testing - The Before Measure Alerts
or Sensitizes Subject to Nature of
Experiment or Second Measure.
Questionnaire
about the Traditional
Role of Women
Triggers Enhanced
Awareness of Women
in an Experiment.
Copyright © 1999 by Harcourt Brace & Company
All Rights Reserved
Instrument - Changes in
Instrument Result in Response Bias
Selection - Sample Selection
Error Because of Differential
Selection Comparison Groups
Mortality - Sample Attrition; some
Subjects Withdraw from
Experiment
New Questions about
Women are Interpreted
Differently from Earlier
Questions.
Control Group and
Experimental Group Is
Self-Selected Group
Based on Preference for
Soft Drinks
Subjects in One Group
of a Hair Dying Study
Marry Rich Widows and
move to Florida
Increasing Internal Validity
• Control Group
• Random Assignment
• Pretesting and Posttesting
• Posttest Only
External Validity
• Extent to which the relationship between
the IV and DV is generalizable to the real
world
• Threats:
– non-representative sample
– too artificial a setting
– generalizability -other factors making the
experiment unrealistic
Types of Test Marketing
•
•
•
•
•
•
standard
controlled
electronic
simulated (STM)
consumer vs. industrial
lead country
Some key decisions in Experiment
•Classificatory Vs. Continuous Variables
•Experimental and Control Groups
•Treatment Levels
•More Than One Independent Variable