Field Experiments
Typical Format
• Researcher manipulates something in the real
world, exposing randomly-assigned groups of
people to different treatments
• Possibly the most fruitful area of research in
social sciences, but also greatest ethical
challenges since the experiments can be so
powerful and have real-world effects
Field Experiments in Practice
• Most medical clinical trials are field
experiments
• Since the experiment takes place outside of
the laboratory, researchers cannot completely
control randomization
• Some subjects selected for treatment may
refuse, others may switch from control to
treatment or from treatment to control
– Example: Dartmouth Spine Study
As Treated vs Intent to Treat
• As treated (AT) analysis compares subjects
given the treatment they received.
– Does not consider treatment group subject
assigned, only which they received
• Intent to Treat (ITT) compares subjects in the
groups to which they were randomized,
regardless of whether they received or
adhered to the allocated treatment.
– Required in medical trials
Intent to Treat: Adherence
• Problem 1: Adherence: Subjects who did not
adhere to treatment or switched to control
should be included
– Analyze data in group to which subject allocated,
not what they actually received
Intent to Treat: Loss to Follow-Up
• Problem 2: Loss to Follow-up: Subjects who
did not complete trial or could not be
contacted to provide data should be included
– Cannot be deleted from analysis since they are
not missing due to random factors
– Missing data problem
Applications in Social Sciences
• Many field experiments manipulate the
information that people receive
• Information is a less powerful and shorterterm manipulation, making the ethical issues
less significant
The Effects of Choice and Enhanced Personal
Responsibility for the Aged:
A Field Experiment in an Institutional Setting
Ellen Langer and Judith Rodin
Journal of Personality and Social Psychology
1976, 34(2)
Research Question
Objective helplessness as well as feelings of
helplessness and hopelessness may contribute
to psychological withdrawal, physical disease,
and death.
Objective control and feelings of mastery may
contribute to physical health and personal
efficacy.
Study Design
• Conducted in nursing home for generally
upper income residents
• Four floors in the home.
• Two floors selected for study based on
similarity in the residents’ physical and
psychological health and prior socioeconomic
status.
Randomization?
• Residents assigned to floors based on
availability
• Residents on two different floors exposed to
different “treatments”
Treatment: A Meeting on Lounge of Each Floor
Both floors held meeting directed by the same
staff member, who read from two different
scripts
Script 1: Responsibility-Induced Message
[Treatment]
Script 2: No Responsibility [Control]
Treatment Group: Responsibility Message
“Many of you don't realize the influence you have
over your own lives here. Take a minute to think of
the decisions you can and should be making.”
“You should be deciding how you want your rooms
to be arranged—whether you want it to be as it is or
whether you want the staff to help you rearrange the
furniture. It’s your life and you can make of it
whatever you want.”
Treatment Group (continued)
“If you are unsatisfied with anything here, you
have the influence to change it. It's your
responsibility to make your complaints known,
to tell us what you would like to change, to tell
us what you would like.”
Treatment Group (continued)
“Also, I wanted to take this opportunity to give
you each a present from the Arden House. The
plants are yours to keep and take care of as
you'd like.”
[A box of small plants was passed around, and
patients were given two decisions to make:
first, whether or not they wanted a plant at
all, and second, to choose which one they
wanted. All residents did select a plant.]
Treatment Group (continued)
“One last thing, I wanted to tell you that we're
showing a movie two nights next week,
Thursday and Friday. You should decide which
night you’d like to go, if you choose to see it at
all.”
Control Group
“I was surprised to learn that many of you
don't know about the things that are available
to you; that many of you don’t realize all
you're allowed to do here.”
“We want your rooms to be as nice as they
can be, and we've tried to make them that
way for you. We want you to be happy here.”
Control Group
“If you have any complaints or suggestions
about anything, let a nurse know what they
are. Let us know how we can best help you.
You should feel that you have free access to
anyone on the staff, and we will do the best
we can to provide individualized attention and
time for you.”
Control Group
“Also, I wanted to take this opportunity to give
you each a present from the Arden House. The
plants are yours to keep. The nurses will water
and care for them for you.”
[The nurse walked around with a box of plants
and each patient was handed one.]
Control Group
“One last thing, I wanted to tell you that we're
showing a movie next week on Thursday and
Friday. We'll let you know later which day
you're scheduled to see it.
Dependent Variables
Pre-post within and between subjects design
Questionnaire given to each subject one week
before and three weeks after meetings”
How much control do you have over your life?
(0=none, 8=total)
How happy are you?
How active do you feel?
Alertness rated by research assistant
Dependent Variables
• Nurses rated each resident on
–
–
–
–
–
–
Happiness
Alertness
Dependence
Sociability
Activity
Eating and Sleeping Habits
• Nurses’ ratings blind (did not know whether
resident was in treatment or control group)
Dependent Variables
• Residents could enter competition to guess
number of jelly beans in a jar
• Tenth day after treatment measured how far
residents moved in their wheelchairs during
one day
• Note: Many dependent variables measured
– Often guarantees at least something turns up
significant
Results
Behavioral Results
• Responsibility group
– More likely to attend movie
– More likely to enter jelly-bean guessing contest
• No evidence of more wheelchair movement
by responsibility group (floors too clean to
measure movement?)
Conclusions
• 71% of control group residents more
debilitated over 3-week study
• 93% of treatment group showed improvement
• Having personal control over one’s
environment and taking care of a plant made
subjects happier, healthier, more active
Concerns?
• Would be interesting to see in a 2x2 design
whether taking care of the plant had an effect
separate from message received
The Effects of Canvassing, Telephone Calls,
and Direct Mail on Voter Turnout: A Field
Experiment
ALAN S. GERBER and DONALD P. GREEN
American Political Science Review
Vol. 94(3): 2000
Field Experiment on Voter Turnout
• Conducted in city of New Haven, Connecticut
• Test effect of encouraging people to vote by
– canvassing (appearing at a person’s house),
– telephone call
– direct mail (up to three)
• 2 x 2 x 4 factorial design
• Canvassing x telephone call x direct mail
Problem
The probability that a given person in the
experiment votes is:
Y=a + b1X1 + b2X2 + e
Y=1 if subject votes
X1=1 if subject is difficult to contact
X2=1 if subject is actually contacted, 0 otherwise
Cannot assume b1=0
Creates correlation between X1 and e
Solution: Instrumental Variables
• Use the intent to treat as an instrumental
variable
– Uncorrelated with regression error due to random
assignment to intent to treat group
– Correlated with endogenous variable since being
contacted during experiment is a function of being
in the intent to treat goup
Field Experiments of Political
Participation (Green, Gerber, Shachar)
• Contact a random sample of registered voters
either by mail or face-to-face in 1998
– We know whether the subjects voted in 1996
• Encourage people to vote
• Check polling data to see if they voted in the
next election (1998 and 1999)
Results
Natural Experiment (Lassen)
• Question: Do more informed people vote, or
do people vote become more informed?
– What is the causal relationship between
information and voting?
• Research Design: Find a natural experiment in
Copenhagen, Denmark, where 4 out of 15
districts carried out a pilot project on
decentralization (PCD) before a vote on
whether to decentralize