AP Statistics Section 5.2 B
More on Experiments
As a researcher conducts an
experiment, he/she is hoping to see a
difference in the responses that is so
large that the difference is unlikely to
be due to mere chance variation. An
observed effect so large that it would
rarely occur by chance variation is
called __________________
statistically significant
When all experimental units are
allocated at random among all
treatments, the experiments is said
to have a completely randomized
design.
Completely randomized designs
are the simplest statistical designs
for experiments. However,
completely randomized designs are
inferior to more elaborate
statistical designs.
A block is a group of experimental
units that are known before the
experiment to be similar in some
way that is expected to
systematically affect the response
to the treatments.
In a block design, the random
assignment of units to treatments
is carried out separately within
each block.
* Blocks control the effects of some
outside variables by bringing those
variables into the experiment to form
the blocks.
* Unless the link between the blocking
variable and the response variable is
obvious, you must justify the reason
for blocking on any given variable.
Example: The progress of a type of cancer differs in
women and men. Design an experiment to compare
three therapies for this cancer.
randomly
assign
Why no control group in this experiment?
Unethical to not treat patients with cancer
Blocks allow us to draw separate
conclusions about each block, for
example, men and women in the
previous experiment. Blocking also
allows more precise overall
conclusions because the systematic
differences between men and women
can be removed when we study the
overall effects of the three therapies.
Example: The soil type and fertility of farmland differ by
location (hilly, arid and shaded). Because of this, a test of the
effect of 2 tillage types and 3 pesticide application schedules
on soybean yields uses small fields as blocks. Design an
experiment to test the effects of the six treatments.
Randomly assign T1P1, T1P2, T1P3, T2P1, etc. to the
plots within each block.
Again, no control group because tillage and some type of
pesticide are both necessary.
A wise experimenter will form blocks
based on the most important
____________
unavoidable sources of variability
among the experimental units.
______________will
then average out
randomization
the remaining variation and allow an
unbiased comparison of the
treatments.
Example: Design an experiment to compare old
and new waterproofing treatments on the boots
of 20 volunteers.
Match subjects with similar
lifestyles.
Randomly assign which
subjects gets boots with the
new or old treatment.
Compare the condition of
the boots after a certain
length of time.
Each subject will get a pair of
boots where one boot has the
new treatment and the other
boot the old treatment.
Randomly assign which boot
gets which treatment.
The simplest use of blocking is a
matched pairs design, which
_____________________________
compares just two treatments.
Subjects are matched in pairs,
essentially forming a block of two units
that are
_____________________________
similar in some important way.
The idea is that matched subjects are
more similar than unmatched subjects,
so that comparing responses within a
number of pairs is more efficient than
comparing the responses of groups of
randomly assigned subjects.
Randomization remains important,
however.
Example: An experiment to
compare two advertisements for
the same product might use pairs
of subjects with the same _____,
sex
age and _______.
_____
income
One common variation of the
matched pairs design imposes both
treatments on the same subjects,
so that each subject serves as his
or her own control.
Example: Consider our earlier experiment
(Section 5.2 A) concerning the effect of listening
to classical music while reading an unfamiliar
piece of literature on retention. Previously, we
simply randomly divided 40 participants into
two groups and had one listen to classical music
while reading and the other did not listen to
classical music. What lurking variables could
affect the response variable?
reading ability, IQ, etc.
Design a matched pairs experiment that will
better determine the effects of listening to
classical music while reading on retention. How
does randomization play a part?
Each subject will read an unfamiliar piece of
literature while not listening to music and a
second unfamiliar piece of literature while
listening to classical music.
Randomly assign which subjects do the music
first or second.
Compare the results.
Example: We often see football players on the sidelines
inhaling oxygen. Does oxygen help speed recovery
from intense exercise?
Design a completely randomized experiment.
What confounding variables are not accounted for?
Athlete’s weight and conditioning
Design a blocked approach for conducting this
experiment.
Divide the subjects into two blocks by position –
linemen and skill position.
Within each block, randomly assign subjects to the
two treatment groups.
Compare the recovery times within each block and
overall.
Design a matched pairs approach for conducting this
experiment.
Each subject will do intense exercise and have oxygen
afterward as well as do intense exercise with no
oxygen afterwards. Randomly assign which subjects
do which first and give sufficient time between
exercise periods.
Randomly assign which subjects do which first and
give sufficient time between exercise periods.
Compare the recovery times.
Cautions about
Experimentation
Earlier in this section we discussed the
role of a placebo in experimental
design. What would be one important
factor when using a placebo in an
experiment?
The group receiving the placebo
cannot know it is getting the placebo
instead of the drug.
BLIND experiment
In fact, a researcher may accidentally introduce
bias into an experiment if he is aware of which
group received the placebo. To avoid this
possibility we could use a double-blind
experiment:
neither the subjects nor the researcher
knows which treatment a subject receives.
Many, perhaps most, experiments have some
weaknesses in detail. The environment of an
experiment can influence the outcomes in
unexpected ways. The most serious potential
weakness of experiments is the lack of realism –
the subjects treatments or settings do not
realistically represent the conditions we wish
to study.
CAUTION: Most experimenters
want to generalize their
conclusions to some setting wider
than that of the original
experiment. Statistical analysis of
an experiment cannot tell us how
far the results will generalize to
other settings.