More About Factorial Design
• Suppose experiment tests whether taking a
game theory class causes a person to get more
papers published
• We cannot simply examine whether people
who took a game theory class had more
papers published due to possible effect of selfselection into game theory class.
• Therefore, we should randomly assign people
to game theory class.
Does Game Theory Lead to More
Articles?
• DV: Number of papers published 4 years later
• IV: Took game theory class or not
• Two groups:
1. Take game theory class
2. Did not take game theory class
Does Class on Experiments Lead to
More Articles?
• We also may want to determine whether
taking a class on experiments helps a person
get more articles published.
• Now we randomize whether a person takes a
class on experiments
2 x 2 Factorial Design
Class Taken
Game
Theory
No Game
Theory
Experiments
No Experiments
Now we Have 4 groups in the experiment, randomly assigned.
Each cell should have equal numbers of subjects
2 x 2 x 3 Factorial
• Now we randomly assign whether a person
studies international relations, public policy,
or economics.
• This is now a 2 x 2 x 3 factorial design with 12
groups
Economics
Game
Theory
No Game
Theory
Game
Theory
No Game
Theory
Game
Theory
No Game
Theory
Experiments
No Experiments
Public Policy
Experiments
No Experiments
International
Relations
Experiments
No Experiments
Power Analysis: How Many Subjects Needed?
• Example: G*Power (free!)
• Hypothesize the magnitude of the effect (such
as one additional paper published)
• Calculate sample size needed for two
independent samples difference in means
(average number of articles published for each
of two groups)
• Answer will be number of subjects needed in
each group
Natural Experiments
Exploiting Randomization When It
Occurs
Natural Experiments
• Experimenter does not pre-plan experiment
• Experimenter finds a naturally-occurring
random assignment mechanism that serves as
a treatment
Why Randomization?
• Randomization allows comparison across
groups by controlling for differences in other
variables.
• The expected values of other variables are the
same between across randomly-assigned
treatment groups.
• If other variables have equal expected values
across treatments, then differences in
outcomes must be due to treatment variable
Dartmouth Roommate “Experiment”
• First-year roommates at Dartmouth College
assigned randomly within “like-groups” based
on gender and whether student
– Smokes
– Listen to music while studying
– Stay up late
– Are neat or messy
• Assignment random within 25=32 different
groups
Treatment=Roommate
• In this case, the treatment variable is a
student’s roommate
• The experiment does not have a true “control
group” but many different treatments in the
form of roommates
• Also in this case, the roommates are
“treatments” for each other
First Step: Is Assignment Truly
Random?
• Are there any variables that determine one’s
assigned roommate other than
– Gender
– Smoking
– Late night habits
– Cleanliness
– Listening to music
DV’s and IV’s
• The DV in this case is a student’s GPA
• The most important IV is the roommate’s GPA
• Other IV’s included as control variables are a
student’s own academic index (entrance exam
scores and high school grades) and
roommate’s academic index
Statistical Model
Conclusions
• Roommates influence each other’s
performance in school
– Top performers reinforce each other
– Bottom performers reinforce each other
• Roommates also influence choice of social
group such as fraternity or sorority
• Roommates do not influence each other’s
choice of major
Other Examples of Natural Experiments