4.2 - Experiments
Observational Studies
measures variables of interest without attempting to
influence the responses.
sample surveys
watching animals in nature
you provide no influence on responses
just notices relationship, doesn’t imply causation
Experiment
deliberately imposing some treatment(s) on
individuals to measure their responses.
does the treatment cause a change in the
responses?
helps determine a cause and effect relationship
Lurking Variable
a variable that is not the explanatory or the response
variable in a study but may influence the response
variable.
example: you provide data for the relationship between the number of dinners
you eat with your family and your GPA. The number of dinners you eat with your
family might not be the only reason for a higher GPA.
•
A lurking variable could be the interest your parents have in your education therefore your parents involvement is creating a higher GPA, not necessarily the
number of dinners.
Confounding
occurs when 2 variables are associated in such a way
that their effects on a response variable cannot be
distinguished from each other.
By designing effectively, you can prevent lurking
variable from becoming confounding variables.
Observational studies often fail due to confounding of
explanatory variables and lurking variables.
Observational or Experiment?
1.
Does reducing screen brightness increase battery life in laptop computers? To
find out, researchers obtained 30 new laptops of the same brand. They chose 15
at random and adjusted their screens to the brightest setting. The other 15
laptops were left at the default setting - moderate brightness. Researchers then
measured how long each machine’s battery lasted.
2.
A study of child care enrolled 1364 infants and followed them through their sixth
year in school. Later, the researchers published an article in which they stated
that “the more time children spent in child care from birth to age four-and-a-half,
the more adults tended to rate them, both at age four-and-a-half and at
kindergarten, as less likely to get along with others, as more assertive ,as
disobedient, and as aggressive.”
Let’s look at the child care study some
more:
•
•
•
What are the explanatory and response variables?
Does this study show that child care causes children
to be more aggressive? Explain
Are their any lurking variables? Are they confounded?
Effects of binge drinking
•
A common definition of “binge drinking” is 5 or more
drinks at one sitting for men and 4 or more for
women. An observational study finds that students
who binge drink have lower average GPA than those
who don’t. Identify a lurking variable that may be
confounded with the effects of binge drinking. Explain
how confounding might occur.
•
Could a lurking variable be “controlled” so that it is not
confounded?
Parts of an Experiment
1. Explanatory Variable (aka Factors) - we can have more than one
2. Response Variable - what we are measuring as a result of the
experiment
3. Treatment - the specific condition applied to individuals in an
experiment. Could be 1 or many in one experiment.
4. Experimental Units - the smallest collection of individuals to
which treatments are applied. (when they are humans they are
called subjects)
5. Levels - when there is more than one treatment option due to
multiple factors in an experiment
Identify the experimental units, the explanatory and
response variables, and treatments in the following
experiments.
•
A study published in the New England Journal of
Medicine in March 2010 compared 2 medicines to
treat head lice: an oral medication (ivermectin) and a
topical lotion (malathion). Researchers studied 812
people in 376 households in 7 areas around the
world. Of the 185 households randomly assigned to
ivermectin, 171 were free from head lice after 2
weeks compared with only 151 of the 191 households
randomly assigned to malathion.
•
Identify the experimental units, the explanatory and
response variables, and treatments in the following
experiments.
Does adding fertilizer affect the productivity of tomato
plants? How about the amount of water? To answer
these questions, a gardener plants 24 similar tomato
plants in identical pots in his greenhouse. He will add
fertilizer to the soil in half the pots. Also, he will water
8 of the plants with 0.5 gallons of water per day, 8 of
the plants with 1 gallon of water per day, and the
remaining 8 plants with 1.5 gallons of water per day.
At the end of three months, he will record the total
weight of tomatoes produced by each plant.
A Template to Design an
Experiment
Explanatory
Variables/Factors
Response
Variable
Subjects/Experim
ental units
Treatment(s)
more will be added later
This will be written out
and later a design can
be drawn
Ch 4.2 - Experiments
Day 5
Randomized Design
•
•
To help control lurking
variables, an
experiment needs
comparisons
This helps
confounding from
occurring
•
Examples of a Completely Randomized
Design
30 students volunteer to be
subjects in a caffeine experiment.
On 30 identical slips of paper there
are 15 A’s and 15 B’s. They are
mixed in a hat and each student
selects one slip of paper. Students
who receive A drink the cola with
caffeine and students who receive
B drink the cola without caffeine. At
the end of an hour we will ask the
students if they still feel energized
and compare the results.
A health organization wants to know if a lowcarb or a low-fat diet is more effective for longterm weight loss. The organization decides to
conduct an experiment to compare these two
diet plans with a control group that is only
provided with a brochure about healthy eating.
Ninety volunteers agree to participate in the
study.
Assign a number 00 to 90 to all subjects in
alphabetical order by last name. Go to a line in
Table D and read two-digit groups from left to
right (throwing out any repeated digits or digits
larger than 90). The first 30 digits will have the
brochure, the next 30 digits will do the lowcarb diet, and the remaining 30 will have the
low-fat diet. At the end of the year the total
weight loss of each group will be compared.
Control Groups
•
•
These are important to provide a baseline for
comparing the effects of the treatments
aka placebo groups (more will be discussed about
placebos tomorrow)
Three Principles of Experimental Design
1. Control
Control Groups
Control lurking variables by creating
groups with the only difference
being the treatments
2. Random Assignment
3. Replication
impersonal chance to assign
experimental units to treatments
create roughly equal groups by
balancing the effects of lurking
variables that can’t be controlled
use enough experimental units in
each group so differences from the
treatments can be distinguished
from just chance differences
between the groups
Don’t get lost in the vocabulary - remember the main
goal is to create “large” treatment groups with no
systematic differences between them other than the
treatment!
A Template to Design an
Experiment
Explanatory
Variables/Factors
Response
Variable
Keep in mind 3 Principles:
1. Control
2. Random Assignment
3. Replication
Sample
Random Assignment of
experimental
units/subjects
Assign
Treatment(s)
State what results will
be measured and
compared
more will be added later
What conclusions can
be drawn?
What went wrong?
•
Will cash bonuses speed the return to work of
unemployed people? A state department of labor
notes that last year 68% of people who filed claims for
unemployment insurance found a new job within 15
weeks. As an experiment, this year the state offers
$500 to people filing unemployment claims if they find
a job within 15 weeks. The percent who do so
increases to 77%. What flaw does this design have?
Is it impossible to say whether the bonus really
caused the increase?
Design a completely randomized
experiment.
• 150 students are willing to serve as subjects to study
the effects of repeated exposure to an advertising
message. The answer depends on both the length of
the ad and on how often it is repeated. The different
lengths that will be used are 30 second ads and 90
second ads. The commercial will either be shown 1,
3, or 5 times during the program. After viewing, all the
subjects answered questions about their recall of the
ad, their attitude toward the camera ad, and their
intention to purchase it.
Ch 4.2 - Experiments
Day 6
Placebo Effect
•
a response to a dummy
treatment
• subjects do not know
they are receiving a
placebo because the
effects are so strong
How to Control the Placebo
Effect
•
•
•
Double Blind Experiments - neither the subject nor
those who interact with them know who has what
treatment
Single-Blind Experiments - when the subject knows
what treatment they receive, but those measuring the
responses do not.
Double Blind is the best... WHY?
A Template to Design an
Experiment
Explanatory
Variables/Factors
Response
Variable
Keep in mind 3 Principles:
1. Control
2. Random Assignment
3. Replication
Sample
Random Assignment of
experimental
units/subjects
Assign
Treatment(s)
State if there is any
blinding, if so why
more will be added later
State what results will
be measured and
compared
What conclusions can
be drawn?
and now, for two more infamous stat
words:
• Statistically Significant
•
“an observed effect so large that it would rarely occur
by chance”
•
How can we measure if an experiment’s results were
statistically significant?
Can random just be “unlucky?”
•
•
•
NO!
If you think there is going to be a difference between
the way the experimental units will react to the
treatment, then you need to create a design to control
those lurking variables.
i.e. With the distracted driving activity, let’s say that
you feel women will forget more anyways because
they are worse drivers. Then you will need to design
your experiment to have women and men in both
groups, but still randomly assigned!
Blocking
•
Used mainly when there are strata already built into
the population where a stratified random sample will
be appropriate. Blocking is a form of control in the
experimental design.
1. Block - the group of experimental units that are
known before the experiment to be similar in some
way that is expected to affect the response to the
treatments.
2. Randomized Block Design - random assignment of
experimental units to treatments that is carried out
separately within each block
Stratified Random Sample or
Randomized Block Design?
Stratified Random Randomized Block
Sample
Design
Controls lurking
variables
Forms similar
groups
Done when taking
the sample from
the population
Done when
assigning units to
the treatments
Examples of a Randomized Block Design
•
Anne is an avid baker who would like to compare two different chocolate chip
cookie recipes (A and B). So she recruits 10 volunteer taste testers to rate each
type of cookie on a scale of 1 (very bad) to 10 (very good). She will make 10 of
each type of cookie, for a total of 20. Each cookie tray will hold only 10 cookies,
so she will use two trays to bake them at the same time in the same oven, one
sheet on the lower rack and one of the upper rack. She thinks that the cookies
will bake differently depending on which rack they are on, we will use the 10
locations on the lower-rack cookie sheet as 1 block and the 10 locations on the
upper-rack cookie sheet as the 2nd block. On each sheet Anne will randomly
place 5 of each type of cookie. This way each type of cookie will have 5 on the
lower rack and 5 on the upper rack, balancing out the effect of rack location.
•
What are the experimental units?
•
What are the treatments?
•
Draw a diagram to represent this situation.
Matched Pairs
•
•
•
•
a type of blocking
block size is always 2 because only two treatments
every experimental unit will receive both treatments,
in a random order or the experimental units are paired
as closely as possible then each treatment is
assigned randomly to each unit
results are compared from each individual and then
as a whole group
A Template to Design an
Experiment
Explanatory
Variables/Factors
Response
Variable
Keep in mind 3 Principles:
1. Control
2. Random Assignment
3. Replication
Sample
Random Assignment of
experimental
units/subjects
Assign
Treatment(s)
State if there is any
blinding, if so why
State if there is any
blocking, if so why
State what results will
be measured and
compared
What conclusions can
be drawn?