Experimental Designs
1
Experimental Designs
I.
After-Only Design
Experimental variable introduced
“After” means
Experimental
Group
yes
yes (X1)
X1 = effect of experimental variable
e.g., spend $1 million on advertising in the beginning of the year and
sales are $20 million, claiming $20 sales per $1 of adv.
Note that even if there is a correlation between sales and advertising
expenditures, correlation does not prove causality. We need an
experiment to demonstrate that advertising  sales.
II.
Before-and-After Design
Before means
Experimental variable
After means
Experimental
Group
yes (X1)
yes
yes (X2)
Effect of experimental variation = (X1 - X2)
e.g.,
687322417
1 of 5
Experimental Designs
2
Experimental errors:
 History - A change in the environment external to the experiment that
can cause before and after measurements to differ. It’s an uncontrolled
variable. The longer the time period between measurements the greater
the danger.
 Maturation - changes occurring with the passage of time on test
subjects. These changes may be biological or psychological. E.g., older,
tired, thinner, more educated, …
 Instrument variation - changes in the measuring instrument, e.g.,
different interviewers. Measurement error = random error - bias (bias
tends to be in one direction).
 Selection bias - results from the way in which test subjects were selected
or assigned to different groups. Researcher should: [1] select subjects
randomly [2] assign to groups randomly (or, at least, by matching).
 Mortality - loss of subjects due to moving away, disinterest, death, etc.
Mainly refers to differential loss of subjects from various groups, i.e.,
attrition rate for one group differs from that of another.
 Testing effect - occurs when the process of measurement has an
influence on what is being measured.
2 types: [a] Main testing effect - occurs when the first observation
affects the second observation. (The subj. is now familiar with the
questionnaire and may respond differently the second time as they are
now “experts.”) Affects internal validity. [b] Interactive testing effect situation in which the prior measurement affects the subject’s reaction to
the experimental variable. E.g., an initial question about shampoo may
make one more aware of the shampoo market and the subj. notices a new
brand of shampoo (the external variable). Affects external validity.
Experimenter Effects - Rosenthal suggests that experimenters, througfh
subtle and unintentional actions communicate their expectations, desires,
687322417
2 of 5
Experimental Designs
3
and biases to their subjects and thus influence the outcome of the
experiment. (see Rosenthal, Experimenter Effects in Behavioral Research)
Demand Artifacts - can cause subjects to perceive, interpret, and act upon
what they believe is expected or desired of them in the experiment.
Hawthorne Effect - distortion in behavior that occurs when ppl know they
are subjects in a study. The best kind of study is an unobtrusive study.
Placebo Effect - … - but, a double-blind design doesn’t always work either:
In one study (NYT 8/17/93) 78% of patients and 97% of doctors could tell
who had the real drug because of side effects.
687322417
3 of 5
Experimental Designs
4
Experimental Designs (continued)
III.
Before-and-After with control Group Design
“Before” means
Experimental variable
“After” means
Experimental
Group
yes (X1)
yes
yes (X2)
Control
Group
yes (Y1)
no
yes (Y2)
Effect of experimental variable = (X1 - X2) - (Y1 - Y2)
Problems: [a] selection must be random [b] mortality.
IV.
After-only with Control Group
(most widely used design in marketing)
Experimental variable
“After” means
Experimental
Group
yes
yes (X1)
Control
Group
no
yes (Y1)
Effect of experimental variable = (X1 - Y1)
e.g., Ad 1 uses celebrity; Ad 2 doesn’t. Split run w/ 2 types of
(coded) coupons.
‘celebrity’ ad
‘control’ ad
Experimental variable
yes
no
% redeemed
40%
2%
V.
Ex Post Facto Design [same as ===== surveys???]
This is a variation of After-only With Control Group design. Control group
and experimental group are chosen after the experimental variable is
introduced instead of before. Prevents subjects from knowing whey are
being tested. E.g., [a]compare ppl who watch violent programs with ppl
who do not watch violent programs on a violence-prone measure [b] short
vs. tall ppl on life expectancy [c] bottle-fed vs. breast-fed.
687322417
4 of 5
Experimental Designs
5
Problem: “differential selection”
This design is only effective if we have an objective way of determining
whether subjects were exposed to experimental variable (e.g., married vs.
unmarried; car owners vs. non-owners, etc.)
VI.
Factorial Design
687322417
5 of 5