Experimental Design
Section 4.2A
• Objective:
To learn about Observational Studies and Completely
Randomized Experimental Design
• Purpose:
To see a possible association from an observational
study or a possible cause and effect from an
experiment.
Observational Study
• Observes individuals and measures
variables of interest but does not
attempt to influence the responses.
• Goal: To describe some group or
situation, to compare groups, or
to examine relationships
between variables.
Observational Studies
• You can only see an association in an
observational study.
• You CANNOT see a cause and effect.
Experiment
• Deliberately imposes some
treatment on individuals to measure
their responses.
• Goal: To understand the response to a
change
Experiments can answer
questions like:
“Does aspirin reduce the chance of a
heart attack?”
Or
“Can yoga help dogs live longer?”
Does Taking Hormones Reduce Heart
Attack Risk after Menopause?
• Should women take hormones such as estrogen after menopause,
when natural production of these hormones ends? In 1992, several
major medical organizations said “yes.” Women who took
hormones seemed to reduce their risk of a heart attack by 35% to
50%. The risks of taking hormones appeared small compared to the
benefits.
Confounding Variable!
• This came from a number of observational studies that compared
women who were taking hormones with others who were not. But
the women who chose to take hormones were richer and better
educated and saw doctors more often than women who didn’t take
hormones. Because the women who took hormones did many other
things to maintain their health, it isn’t surprising that they had fewer
heart attacks.
• After doing real experiments…they found that taking hormones did
not reduce the risk of heart attacks. They are now out of favor.
Want Good Data???
•If you really want to get
convincing data on the link
between hormone
replacement and heart attacks,
we should do an experiment!
•Why? – Because we can show
cause and effect.
Observational Study
vs.
Experiment
• Observational studies of the effect of
one variable on another often fail
because of confounding between the
explanatory variable and one of
more lurking variables.
Explanatory and Response
Variables.
• Response Variable: It is what we
measure at the end of our study.
Measure at the End; Outcome of a Study
• Explanatory Variable: Any factor
that can influence the response
variable or help to explain it.
“Treatments”
Also Called
“Factors”
Lurking Variables
• A variable that is not among the
explanatory or response variables in a
study but that may influence the
response variable.
• Hormone Replacement Study
• The effect of taking hormones was
mixed up with the characteristics of
women. These characteristics were
lurking variables.
Example: “An apple a day keeps the doctor away!”
Confounding
Cannot pull 2 variables apart.
• This occurs when two variables are
associated in such a way that their
effects on a response variable cannot
be distinguished from each other.
Example: Single Sex Schools
Typically have smaller class size……..
But in the final analysis (as it were), it really doesn't matter what
we call them. The important thing for students to recognize is that
variables other than those named in the study may be closely
associated with one or several variables.
Example
Professor wanted to see if his style had an effect
on students in New York. He taught the fall &
spring classes identically – same text, syllabus,
etc. - but in the fall he used a subdued
demeanor and in the spring he used expansive
gestures & lectured with more enthusiasm.
The fall class rated him an average teacher.
The spring class rated him excellent and
praised him on everything.
Can we conclude that the teacher’s style had
a role in the students’ evaluations?
NO!
Confounding variables
Fall ends cold & bleak
Spring is bright & cheerful.
The student’s happiness could have been
affected by the season & reflected it in their
evaluations.
Observational or Experimental
• Does reducing screen brightness increase battery life in
laptop computers? To find out, researchers obtained 30
new laptops of the same brand. They chose 15 of the
computers at random and adjusted their screens to the
brightest setting. The other 15 laptop screens were left
at the default setting – moderate brightness.
Researchers then measured how long each machine’s
battery lasted.
• Experiment – treatment was imposed.
A treatment was received and a cause and effect can be seen!
Does eating dinner with their families improve students’
academic performance? According to an ABC News
article, “Teenagers who eat with their families at least five
times a week are more likely to get better grades in
school.” This finding was based on a sample survey
conducted by researchers at Columbia University.
• Is this observational or experimental?
• If observational, what are the lurking variables that may be
confounded?
• If experimental, what is the treatment?
• What are the explanatory and response variables?
Factors
• The explanatory variable…many
experiments study the joint effects of
several factors.
Treatment
• A specific condition applied to the
individuals in an experiment.
• If more than one factor, then the
treatment is a combination of specific
values of these variables.
Experimental Units
• The smallest collection of individuals
to which treatments are applied.
• When the units are human beings,
they are often called subjects.
• The Randomized Comparative Experiment
Group 1
Experimental
Units
Experiments
Definition:
In a completely randomized design, the treatments are
assigned to all the experimental units completely by chance.
Some experiments may include a control group that receives
an inactive treatment or an existing baseline treatment.
Treatment
1
Compare
Results
Random
Assignment
Group 2
Treatment
2
A study published in the New England Journal of Medicine
(March 11, 2010) compared two medicines to treat head
lice: an oral medication called ivermectin & a topical lotion
containing malathion. Researchers studied 812 people in
376 households in seven areas around the world. Of the
185 randomly assigned to ivermectin, 171 were free from
head lice after two weeks compared to only 151 of the 191
households randomly assigned to malathion.
• Experimental Units
• Explanatory Variable
• Treatments
• Response Variable
Does adding fertilizer affect the productivity of tomato
plants? How about the amount of water given to the
plants? To answer these questions, a gardener plants 24
similar tomato plants in identical pots in his greenhouse.
He will add fertilizer to the soil in half of the pots. Also, he
will water 8 of the plants with 0.5 gallons of water per day,
8 of the plants with 1 gallon of water per day and the
remaining 8 plants with 1.5 gallons of water per day. At the
end of three months he will record the total weight of
tomatoes produced on each plant.
• Experimental Units
• Explanatory Variable
• Treatments
• Response Variable
Example: What’s the effect of repeated
exposure to an ad?
An experimenter used undergraduate students as subjects. All
subjects viewed a 40 min. t.v program with ads for a camera.
Some saw a 30-sec commercial – others a 90-sec commercial.
It was shown either 1, 3, or 5 times during the program.
Experimental Unit:
Factors:
Treatments:
Response Variable:
2 Factors: length of commercial
#times it played
6 treatments
Response: Subjects answer questions about their recall of the ad,
their attitude toward the camera & intent to purchase.
Many students regularly consume caffeine to
help them stay alert. Thus, it seems plausible
that taking caffeine might increase an
individual’s pulse rate. Is this true? One way to
investigate this is to have volunteers measure
their pulse rates, drink some cola with caffeine,
measure their pulses again after 10 minutes
and calculate the increase in pulse rate.
• What is wrong with this study?
Good Experiments
• Use a comparative design to compare
two or more treatments.
• Randomly assign experimental units
to the treatments.
• Be specific about how you are
randomly placing them
Completely Randomized
Design
*All experimental units are allocated at
random among all treatments.
Ex: Does talking on a hands-free cell phone
distract drivers?
Group 1
(20 students)
Treatment 1
(Drive)
Random
Compare
brake time
allocation
Group 2
Treatment 2
(20 students)
(Drive & Talk)
Put the 40 names in a hat, mix them up, and draw 20. This
will form the experimental group and the remaining 20 will
make up the control group.
A health organization wants to know if a low-carb or low-fat
diet is more effective for long-term weight loss. The
organization decides to conduct an experiment to compare
these two diet plans with a control group that is only
provided with brochure about healthy eating. Ninety
volunteers agree to participate in the study for one year.
Principles of Experimental Design
Control – the effects of lurking variables on the
response, most simply by comparing two or more
treatments.
Replication – Apply treatment to a number of
subjects & repeat the experiment.
Randomization – Equalize the effects of unknown or uncontrollable sources of
variation – used to assign treatments.
Example
“Gastric Freezing” is used to treat ulcers in the upper intestine.
Patient swallows a deflated balloon with tubes attached, then
a refrigerated liquid is pumped through the balloon for an
hour. This reduces the acid and relieves the pain.
Design: Gastric freezing →Observed pain relief
Placebo – dummy treatment
Placebo Effect – Respond favorable to any
treatment, even a placebo.
Control Group – The group of patients who
receive the sham treatment.
Gastric Freezing Revisited
•
•
•
•
Used 2 groups
Control group used (fluid was room temp)
34% of treatment group improved
38% of control group improved
They were not statistically different and thus gastric freezing has
been abandoned.
Blinding
* Single-blind – Subjects do not know what
treatment they received
(controls for personal beliefs)
• Double-blind – neither the subject nor the
individual measuring the
response know which treatment
was received.
Example
Subjects were told the experiment was
about 3-D spatial perception and were
assigned to draw a model of a horse.
While they were busy drawing, a loud
noise and then groaning were heard
coming from the room next door. The real
purpose of the experiment was to see how
people reacted to the apparent disaster.
Something to think about…..
• Use appropriate ethics when
setting up an experiment.
• Placebos only need to be used on
people, not animals.
Homework
• Page 253 ( 45, 47, 57, 65)
• Worksheet – Observational vs Experimental