Chapter 13: Experiments and
Observational Studies
AP Statistics
Observational Studies
Observational Studies: Researchers observe. They
don’t assign choices or manipulate anything
(unlike experiments).
We just use an existing situation (or data), neither
choosing who or what treatments.
Example: A recent study showed that men who
have had a heart attack, have a greater chance of
having a second heart attack if a certain protein is
present in their blood.
Observational Studies
• They are not based on random samples, nor do
they randomly impose treatments. The results
cannot be generalized, nor can they show causeand-effect.
• They, however, are not worthless.
• They can show us trends and possible
relationships—even if we can’t show cause-andeffect.
• They can show us variables related to certain
outcomes
Types of Observational Studies
Retrospective: Subjects are selected and then
their previous conditions and behaviors are
determined
restricted to small part of population
Prone to errors—looking at historical data
Usually focus on estimating differences between
groups or associations between variables
Types of Observational Studies
Prospective: Study where subjects are followed
to observe future outcomes.
Focus is on estimating differences among groups
that might appear as groups are followed
Because no treatment is applied, it is NOT and
experiment.
Randomized, Comparative
Experiments
• Only method by which we can prove cause-andeffect.
• We want to see if learning math on a computer is
better than learning it in a traditional
classroom—randomly assign half of a group of
students to classroom where the content was
only taught on computer and the other half to a
classroom where the content was never taught
on the computer, then we would compare the
results.
Randomized, Comparative
Experiments
Comparative just means we are
comparing the results at the end of
the experiment.
Randomized, Comparative
Experiments
Also called a
“factor”
Each factor has levels—values that
the experimenter chooses for the
factors
Randomized, Comparative
Experiments
An experiment is designed to test the claim that
those people who sleep less than 8 hours a night
have a decreased ability to remember
information. The experimenter has obtained 50
subjects and has randomly placed them in two
groups. All subjects will be given a memory test
as a baseline. One group will be required to sleep
at least 8 hours for one night and the other
groups will be prevented from sleeping 8 hours a
night. The next day, each group will be given a
test of memory and differences in the test will be
recorded.
Randomized, Comparative
Experiments
Important Concepts
• The experimenter actively and deliberately
manipulates the factors to control the details
of the possible treatments.
• The subjects are assigned to the treatments
randomly.
Four Principles of Experimental Design
• Control
• Randomization
• Replicate
• Block
Control
• We want to control sources of variation other
than the factors we are testing by making
conditions as similar as possible for all treatment
groups.
– We control a factor by assigning subjects to different
factor levels because we want to see how the
response will change at those different levels.
– We control other sources of variation to prevent them
from changing and affecting the response variable.
Control
• Controlling extraneous sources of variation
reduces the variability of the responses,
making it easier to detect differences among
the treatment groups.
• Making generalizations from the experiment
to other levels of the controlled factor can be
risky.
Randomize
• Allows us to equalize the effects of unknown
or uncontrollable sources of variation
– Doesn’t eliminate those effect of these sources,
but it spreads them out across all treatment
levels, so that they “even out” and can be looked
past.
– If not randomized, you will not be able to draw
conclusions from the experiment
– “control what you can, randomize the rest”
Replicate
• 1st type: We need to repeat the experiment,
applying the treatment to a number of
subjects. If we don’t assess the variation, it is
not complete. The outcome of an experiment
on a single subject is an anecdote—not an
experiment
Replicate
• 2nd type: Occurs when our experimental units
(subjects) are not representative of the
population of interest. We will need to repeat
the experiment with a different experimental
units. Replication of an entire experiment with
the controlled sources of variation at different
levels is an essential step in science. If your
subjects are from an Intro to Psychology class,
you can’t generalize the results—so you will need
to replicate the experiment
Block
• Sometimes random assignment to treatments
from our subjects is not the way to go.
• Sometimes we need to block. This is when we
group experimental subjects that are known
before the experiment to be similar in some
way that is expected to affect the response to
the treatments.
• The randomization comes within the blocks—
where we assign treatments in each block
Logic to Experimental Design
• Randomization produces groups of subjects
that should be similar in all respects before we
apply treatments
• Comparative design ensures that influences
other than the experimental treatment
operate equally on all groups.
• Therefore, differences in the response variable
must be due to the effects of the treatments
Experimental Diagram
Diagram of a randomized comparative experiment.
An experiment that was designed to test the effectiveness of the drug hydroxyurea for
treating sickle cell anemia. There were 299 adult patients who had at least three
episodes of pain from sickle cell anemia in the past year.
The factor is:
The response variable is:
What are the treatments levels?
Experiment to test the effectiveness of see what treatment may reduce the number
of repeat offenders.