4.4 Designing Experiments to Reduce Variability
In this section we look at the different types of experimental designs.
Completely Randomized Design






Treatments are assigned randomly to all _____________________ or experimental units are assigned
randomly to all treatments.
Can compare any number of treatments.
An _________ number of experimental units for each treatment is not necessary.
The control group can either be an older treatment (if the new treatment is being compared to a
treatment whose effects have been previously experimentally established) or a ___________.
Best if the _________________ are the only systematic differences in the experiment.
Relies on _______________________ to control for the effects of lurking variables. The effects of
other factors tend to average out between the treatment and control group, which reduces
confounding.
Group 1
Treatment 1
Experimental
Units
Randomly allocate
units to groups
Group 2
Treatment 2
Compare
Group n
Control (placebo)
Steps:
1. Number the available experimental units from 1 to n.
2. If you have three treatments, for example, use a random number table (or your calculator) to pick
n
integers at random from 1 to n, discarding any repetitions. These units will be given the first
3
treatment.
3. Repeat step 2 to pick the units for the second treatment.
4. The remaining units will get the third treatment.
1
4.4 Designing Experiments to Reduce Variability
Randomized Block Design





The subjects are divided into blocks, such that the variability within each block is less than the
variability between blocks. For this reason, blocks are to experiments, as strata are to samples.
The purpose of both blocks and strata is to group similar units together so that the variation within
each group is as small as possible – the more similar the groups the more effective the blocks or strata
will be.
The blocking variable is chosen on the likelihood that it is related to the response variable.
Blocks can be any size; in fact the “Matched Pairs Design” below is a special type of randomized
block design with a block size of two.
Although blocks can be created by sorting units into groups, you can also get a block by reusing
subjects in each of several time slots (repeated measures), or by subdividing larger blocks of material
into smaller units.
Blocks are another form of control. However, don’t confuse blocking and randomization. You
randomize to control for the variables you don’t know about, and you block to control for the
variables you know might influence the response variable.
Treatment 1
Block 1
Treatment 2
Random
allocation
Compare
Treatment 3
Subjects
Treatment 1
Block 2
Treatment 2
Compare
Treatment 3
Steps:
1. Sort the experimental units into blocks of similar (homogenous) units. The units in each block should
be alike enough that you expect them to have a similar response to any treatment.
2. The treatment is then randomly assigned to the subjects within each block to reduce confounding.
2
4.4 Designing Experiments to Reduce Variability
Matched Pairs Design (a special case of a block design)



A special type of randomized block design, which compares just two treatments.
The subjects are matched in pairs by some characteristic(s) which may cause confounding (for
example, height, race, age, etc.). The idea is that matched subjects are more similar than unmatched
subjects; therefore the differences in responses are due to the differences in the treatment received.
Randomization is still important – the two treatments are assigned to the two subjects are random.
Treatment 1
Pair 1
Compare
Random
allocation
Treatment 2
Experimental
units
Pair subjects by
some
characteristic, such
as gender.
Treatment 1
Pair n
Compare
Treatment 2
Steps:
1. Sort the experimental units into pairs of similar (homogenous) units. The two units in each pair
should be alike enough that you expect them to have a similar response to any treatment.
2. Randomly decide which unit in each pair is assigned each treatment. For example, you could flip a
coin: heads means first unit gets first treatment and tails means first unit gets second treatment. The
other unit will get the other treatment.
3
4.4 Designing Experiments to Reduce Variability
Repeated Measures Design (a special case of a block design)



Another special type of randomized block design, where each unit is assigned all the treatments in
random order.
The main advantage of this design is that it reduces variation due to differences of subjects across
different treatments.
Another advantage is you can conduct experiments with less experimental units.
Subject 1
Subject 2
Subject n
Treatment 1
Treatment 2
Treatment 3
Treatment 3
Treatment 2
Compare
Treatment 3
Treatment 1
Compare
Treatment 2
Treatments
assigned in
random order
Treatment 1
Compare
Example 1: To test a new drug for asthma, both the new drug and the standard treatment, in random
order, will be administered to each subject in the study.
(a) What kind of design is this?
(b) An observant statistician cries, “No, no! Use two similar subjects in each pair, randomized to each
treatment.” What kind of design is this?
(c) Which design do you prefer, and why?
4
4.4 Designing Experiments to Reduce Variability
Example 2: For each experiment, describe the within-treatment variability that might obscure any
difference between treatments. Then describe an experimental design that includes blocking, and define a
response variable.
(a) To determine whether studying with the radio on helps or hurts the ability to memorize, there will be
two treatments: listening to the radio and not listening to the radio. The subjects available are all
seniors in the same school.
(b) To determine whether adding MSG to soup makes customers eat more soup, a large restaurant will
assign two treatments: adding MSG to the soup and not adding MSG. The subjects available are all
customers during one evening.
5
4.4 Designing Experiments to Reduce Variability
Example 3: Once a person has been convicted of drunk driving, one purpose of court-mandated treatment
or punishment is to prevent future offenses of the same kind. Suggest three different treatments that a
court might require. Then outline the design of an experiment to compare their effectiveness. Be sure to
specify the response variable(s) you will measure.
Solution:
Three possible treatments are:
(1)
(2)
(3)
The response variable would be:
Design outline (diagram):
In summary I want to remind you that the purposes of sampling and experimental design are quite
different. Sample surveys are used to establish the parameters of fixed, well-defined populations.
Experiments are used to establish cause and effect by comparing treatments.
The table below summarizes the differences between a survey and an experiment.
What you
examine
Ultimate
Goal
Sample
Survey
Population
Experiment
Treatments
Describe some
characteristic
of the
population
See whether
different
treatments
cause different
results
6
Role of
How you
Randomization Control
Variation
Take a random
Stratify
sample from the
population
Threats to
Inference
Assign
treatments at
random to
available units
Confounding
4.4 Designing Experiments to Reduce Variability
Blocks
Sampling bias,
response bias