Experimental Research
An Overview
Professor Gary Merlo
Chapter 11
1
The Nature of Experiments
• Purpose of experimental research is to allow the
investigator to control the research situation so
that causal relationships among variables may be
evaluated.
• In experiment one variable, (the independent
variable) is manipulated and its effect upon
another variable (the dependent) is measured.
Other variables that may have influence upon the
dependent are eliminated or controlled.
2
Experiment
• A research investigation in which conditions are
controlled
• One independent variable is manipulated
(sometimes more than one)
• Its effect on a dependent variable is measured
• To test a hypothesis
Basic Issues of Experimental Design
Manipulation of the Independent Variable
Selection of Dependent Variable
Assignment of Subjects (or other Test Units)
Control Over Extraneous Variables
Unit Pricing Illustration
• Purpose of unit pricing is to avoid confusion
due to price calculations.
• Some evidence that unit pricing has failed
to change consumer’s purchasing habits.
• Current form of unit price display is a
separate shelf tag for each item.
• This type of display may not facilitate price
comparisons.
5
Unit Pricing Illustration
(continued)
• A survey may result in bias
– Who would admit they did not use unit pricing when
shopping.
• Randomly assign different experimental conditions to
stores, this would minimize the differences inherent in the
stores.
• Results show, a change in the presentation of unit price
information caused a change in average price paid.
• Controlled for effect that store image could cause by using
only on chain store in the experiment.
6
Results of Unit Pricing
Experiment
Average Price Paid per Unit
Treatment
Five
Weeks
before
During
Experiment Experiment
Shelf tag treatment (stores 1 and 2)
$
List format treatment (stores 3 and 4) $
Total all stores
$
0.65 $
0.65 $
0.65 $
Decrease in price regardless of method of pricing
However, greater decrease using list
0.616
0.60
0.608
7
Basic Issues
•
Decisions that must be made about four basic elements of an experiment:
– A. Manipulation of Independent Variable (IV)
• Experimenter has some degree of control over IV
• IV is hypothesized to be the causal influence
• Experimental treatments are the alternative manipulations of
independent variable being investigated
• Experimental and Control Groups:
– In simplest form only two values of the independent variable are
manipulated.
– By holding control group at zero (unchanged) can increase
measurement of independent variable on experimental group.
– May want more than two groups in experiment to get better
understanding of relationships.
– May want to investigate relationships on changes on more than
one independent variable at the same time.
8
Basic Issues
(continued)
• B. Selection of Measurement of Dependent
Variable
– Dependent variable is the criterion or standard by
which the results are judged.
– Careful consideration of an appropriate dependent
variable is required.
– For example: in a test market the time period for the
effects to become evident should be carefully
considered. (Buy a loser once, may not retry)
9
Basic Issues
(continued)
• C. Selection and Assignment of Test Units
– Test units are the subjects or entities whose responses to the
experimental treatment are measured or observed.
– People are the most common test units
• Sample selection error may occur because of the procedure
utilized to assign subjects or test units to either the
experimental group or the control group.
• Random sampling error occurs if repetitions of the basic
experiment sometimes favor on experimental condition and
sometimes the other by chance.
• However, random is still the best method of assignment.
10
Basic Issues
(continued)
• C. Selection and Assignment of Test Units (continued)
– Matching respondents on the basis of pertinent background information is
another technique for controlling assignment errors.
• Examples: age, income, education (whatever is hypothesized to influence the
dependent variable)
– Repeated measure of same subjects eliminates possibility of differences in
subjects but potentially the exposure may cause other problems.
11
Basic Issues
(continued)
• D. Control over Extraneous Variable
– Need to understand the types of experimental error.
– Errors in research can be classified into two basic categories.
• Random error (similar to sampling error)
• Constant error (extraneous variables or conditions of administration
are allowed to have an influence on dependent variables.
– Example: Subjects in experimental group are always administered the
treatment in the morning, testing fuel efficiency using different size car
engines.
– Time of day in this case is the constant error.
• Uncontrolled extraneous variable.
– Constant error distorts the results in a particular direction, results are
erroneous difference masks the results of the experiment.
12
Basic Issues
(continued)
• D. Control over Extraneous Variable
– Demand Characteristics
• Design characteristics that unintentionally hint to subjects
about the experimenter’s hypothesis.
• If participants recognize experimenter’s expectation (demand),
they are likely to act in a manner consistent with the
experimental treatment.
• Demand characteristic can be person administering
experiment, body language.
• Guinea pig effect - subjects change normal behavior or
attitudes in order to cooperate.
13
Basic Issues
(continued)
• D. Control over Extraneous Variable
– Hawthorne Effect
• Researchers were attempting to study the effects on
productivity of various working conditions.
• Regardless of treatment, productivity increased.
• Demonstrates that people behave differently when they know
they are part of an experiment.
• Often disguise purpose of experiment to eliminate this effect.
14
Basic Issues
(continued)
• D. Control over Extraneous Variable
– Establishing Control
• If brand image and preference may influence outcome of taste
test, package experiment in plain packages.
• If eliminating of extraneous variable is not possible, maintain
constant conditions.
– May require several tests and take average to determine results.
• If the order presented could influence results, counterbalancing
attempts to limit this effect.
15
Basic Issues
(continued)
• D. Control over Extraneous Variable
• Blinding
– Control’s subjects knowledge of whether or not
they have been given a particular treatment.
• Double-blind designs
– Neither the experimenter nor the respondent
know who receives the experiment.
– Used when researcher might influence
outcome.
16
Ethical Issues in Experimentation
• Experimental subjects should be fully
informed and receive accurate information.
• However, researchers intentionally use
deception to avoid respondents knowing the
brand.
• In such situations it is necessary to debrief
subject.
17
Fundamental Questions in
Experimentation
• Basic experimental designs a single
independent variable is manipulated.
• Simultaneous manipulation of the several
independent variable is referred to as
factorial design.
18
Fundamental Questions in
Experimentation (continued)
• Laboratory experiment
– Almost complete control
over the research setting.
– Tachistoscope allows
researcher to experiment
with the visual impact of
advertising, packaging, etc.
by controlling the amount
of time a visual image is
exposed to a subject.
19
Fundamental Questions in
Experimentation (continued)
• Field Experiment
– Natural setting
– Used to tune a marketing strategy
• Controlled store test
– Products put into a stores in a number of small
cities or into selected supermarket chains.
– Hybrid between laboratory and field
experiment.
20
Laboratory Experiment
Field Experiment
Artificial-Low Realism
Natural-High Realism
Few Extraneous
Variables
Many Extraneous
Variables
High control
Low control
Low Cost
High Cost
Short Duration
Long Duration
Subjects Aware of
Participation
Subjects Unaware of
Participation
Copyright © 2000 by Harcourt, Inc. All rights reserved.
Issues of
Experimental Validity
• Internal Validity
– Refers to the question of whether the experimental
treatment was the sole cause of observed changes in the
dependent variable.
• Extraneous Variables that may jeopardize internal
validity.
– History
• Specific events in the external environment between the first
and second measurements that are beyond the experimenter’s
control.
22
Issues of Experimental
Validity continued)
• Cohort Effects
– Change in dependent variable that occurs
because the members of one experimental
group experienced different historical situations
than members of the other experimental groups.
– Maturation
• Change in people over time, function of time rather
than function of independent variable. (hunger,
tired)
23
Issues of Experimental
Validity continued)
• Testing Effects
– Pretesting effects because the initial measurement or
test alerts respondents to the nature of the experiment
and respondents may act differently than they would
have if no pretest measures were taken.
– If identical measuring instrument is used more than
once there may be a testing effect. May use alternate
form of testing for second test.
• May result in instrument effect
24
Issues of Experimental
Validity continued)
• Instrument decay or variation
– Selection: sample bias resulting in differential
selection of respondents for comparison groups.
– Mortality: (sample attrition that takes place
over time)
25
Issues of Experimental
Validity continued)
• External Validity
– Ability of the research to generalize the results of the experiment to
the marketplace.
• Must have internal validity to have external
– Example: Student surrogates used a subjects
• Trade-offs between internal and external validity
– Natural experiments tend to have greater external
validity.
– Laboratory experiments have higher internal validity.
26
Classification of Experimental
Designs
• Various types of experimental designs. If only one
variable is manipulated, basic experimental design.
• Symbolism for Diagramming Experimental
Designs
– X = exposure of a group to an experiment treatment
– O = observation of measurement
• 01 = first observation
• 02= second observation
• R = random assignment of test units
27
Three Examples of QuasiExperimental Design
• Not true experiments because they do not
adequately control for problems associated
with loss of external or internal validity.
28
X
01
Subjects participate because of voluntary self-selection or
arbitrary assignment.
Lacks control of extraneous influences.
Under certain circumstances may be only alternative.
29
One-Shot Design
Internal Validity Problems
• History
– weak
• Instrumentation
– not relevant
• Maturation
• Selection
– weak
– weak
• Testing
– not relevant
• Mortality
– weak
30
Copyright © 2000 by Harcourt, Inc. All rights reserved.
01
02
X
Measurement before and after but no control group.
If change over time, lacks control.
Lacks control of history
02
-
01
= Experiment
No control for mortality
Lacks control for testing
Frequently used in marketing because of the time and cost
31
One-Group Pretest-Posttest
Internal Validity Problems
• History
• Instrumentation
– weak
– weak
• Maturation
• Selection
– weak
• Testing
– weak
– controlled
• Mortality
– controlled
32
Copyright © 2000 by Harcourt, Inc. All rights reserved.
Experimental Group:
X
Control Group:
01
02
Lack of assurance that two groups were equal during start
Mortality is not controlled
=
01
02
Results of Experiment
Used frequently when before measurement is not possible
33
Static-Group Design
Internal Validity Problems
• History
– controlled
• Maturation
– possible source of
concern
• Testing
• Instrumentation
– controlled
• Selection
– weak
• Mortality
– weak
– controlled
34
Copyright © 2000 by Harcourt, Inc. All rights reserved.
Improved Experimental Designs
Experimental Group:
R
Control Group:
R
01 X 0 2
03
04
Effect of the treatment is:
(02 - 01) - (04 - 03)
Effect of extraneous variables is same on both groups
Example, both groups receive pretest, same for history, maturation,
testing effects, instrument decay, and regression effects.
However, interactive testing effect not controlled.
35
Experimental Group: R
Control Group:
R
X
01
02
The effect of the experimental group is
01 - 02
Used when a pretest is difficult or impossible
With posttest measurement, the effects of testing and instrumen
decay are eliminated.
Assumptions about extraneous variables are made the same.
36
Experimental Group I:
Control Group I:
Experimental Group II:
R
R
R
Control Group II:
R
01 X
03
X
02
04
05
06
Controls for interactive testing effect
02 - 01 Estimates Results of experiment
04 - 03 Estimates testing effect because of repeated measure
05 - 06 Measurement without pretest
(02 - 01 ) - (04 - 03) - (05 - 06 ) = Maximum Control of
Extraneous Variables
37
Compromise Designs
• Often in marketing research true
experimentation is not possible.
• Often must sacrifice random assignment
38
01
02
03
X
04 05 06
Vulnerable to historical changes - population, attitudes,
economic patterns, etc.
Advantage is to determine temporary from permanent
changes.
39
X
1
2
3
4
5
6
Time 40
Degree of Change
X
1
2
3
4
5
6
Time 41
Degree of Change
X
1
2
3
4
5
6
Time 42
Degree of Change
X
1
2
3
4
5
6
Time 43
Degree of Change
• End of Chapter 11
44