AP STATISTICS
LESSON 5 - 2
DESIGNING EXPERIMENTS
ESSENTIAL QUESTION:
How is a good experiment
designed?
Objectives:
To understand and use the statistical vocabulary
involved in designing experiments.
To design experiments that come as close as
possible to eliminating lurking variables and
bias.
Experimental Units, Subjects,
Treatment
A study is an experiment when we actually do
something to people, animals, or objects in order
to observe the response.
The individuals on which the experiment is done
are the experimental units.
When the units are human beings, they are
called subjects.
A specific experimental condition applied to the
units is called a treatment.
Additional Vocabulary
The explanatory variables in an experiment are
often called factors.
In such an experiment, each treatment is formed
by combining a specific value (often called a
level) of each of the factors.
A placebo is a dummy pill that creates a situation
so the experimenter can see what the effect of
the subject will be just by being treated.
Experiments
In principle, experiments can give good
evidence for causation.
Experiments have the advantage of allowing us
to study the specific factors we are interested in,
while controlling the effects of lurking variables.
Experiments also allow us to study the combined
effects of several factors. The interaction of
several factors can produce effects that could
not be predicted from looking at the effects of
each factor alone.
Outlining a designed Experiment
Units  Treatment  Observed response
Page 290
Example 5.9
THE PHYSICIAN’S HEALTH STUDY
Comparative Experiments
Placebo effect (a dummy treatment) – Many
patients respond favorably to any treatment,
even a sugar pill.
Control group – The group that receives the
sham treatment (placebo).
We rely on the controlled environment of the
laboratory to protect us from lurking variables.
Control is the first basic principle of statistical
design of experiments. Comparison of several
treatments in the same environment is the
simplest form of control.
Control
Without control, experimental results in
medicine and the behavioral sciences can
be dominated by such influences as the
details of the experimental arrangement,
the selection of subjects, and the placebo
effect. The result is often bias, systematic
favoritism toward one outcome.
Randomization
The design of an experiment first describes the
response variable or variables, the factors
(explanatory variables), and the layout of the
treatment, with comparison as the leading
principle.
Experimenters often attempt to match the patients
in one group with similarities in the control group
with elaborate balancing acts.
The statistician’s remedy is to rely on chance to
make an assignment that does not depend on any
characteristic of the experimental units and does
not rely of the judgment of the experimenter in
any way.
Randomized Comparative
Experiments
Randomization produces groups of units that
should be similar in all respects before the
treatments are applied.
Comparative design ensures that influences
other than the response variable operates
equally for both groups.
Therefore, differences in response variable must
be due either to the variable factors or to the
play of chance in the random assignment of
units.
Principles of Experimental
Design
The basic principles of statistical design of
experiments are:
1. Control the effects of lurking variables on the
response, most simply by comparing two or
more treatments.
2. Randomize – use impersonal chance to assign
experimental units to treatments.
3. Replicate each treatment on many units to
reduce chance variation in the results.
Statistical Significance
An observed effect so large that it would rarely
occur by chance is called statistically significant.
You will see the phrase “ statistically significant”
in reports of investigations in many fields of
study. It tells you that the investigators found
good evidence for the effect they were seeking.
When all experimental units are allocated at
random among all treatments the experimental
design is completely randomized.
Cautions About Experimentation
The logic of a randomized comparative
experiment depends on our ability to treat all the
experimental units identically in every way
except for the actual treatments being
compared.
Many behavioral science experiments use as
subjects students who know they are subjects in
an experiment. That’s not a realistic setting
(lack of realism).
Matched Pairs Design
Match pairs designs compare just two treatments.
Completely randomized designs are the simplest
statistical designs for experiments. However,
completely randomized designs are often inferior
to more elaborate statistical designs. In
particular, matching the subjects in various ways
can produce more precise results than simple
randomization.
Blocking Designs
Matched pairs is an example of blocking design.
A block is a group of experimental units of
subjects that are known before the experiment to
be similar in some way that is expected to affect
the response to the treatment. In a block
design, the random assignment of units to
treatments is carried out separately within each
block.
Double – Blind Experiment
In a double-blind experiment neither the
subjects nor the people who have contact
with them know which treatment a subject
received.