Chapter 2
Experimental
Design
Definitions:
1) Observational study observe outcomes without
imposing any treatment
2) Experiment - actively impose
some treatment in order to
observe the response
I’ve developed a new rabbit food, Hippity
Hop.
Makes fur
soft &
shiny!
Rabbit Food
Increases
energy!
100% of daily
vitamins &
essential oils!
Can I just make these claims?
NO
What must I do to make these
claims?
Do an experiment
Who (what) should I test this
on? Rabbits
What do I test?
The type of food
3)Experimental unit – the single
individual (person, animal,
plant, etc.) to which the
different treatments are
assigned
4) Factor – is the explanatory
variable
5) Level – a specific value for
the factor
6) Response variable – what you
measure
7) Treatment – a specific
experimental condition applied to
the units
I plan to test my new rabbit
food.
What are my experimental
units? Rabbits
What is my factor?
Type of food
What is the response variable?
How well they grow
I’ll use my pet
rabbit, Lucky!
Hippity Hop
Since Lucky’s coat is shinier &
he has more energy, then
Hippity Hop is a better rabbit
food!
8) Control group – a group that
is used to compare the factor
against; can be a placebo or the
“old” or current item
9) Placebo – a “dummy”
treatment that can have no
physical effect
Old Food
Hippity Hop
Now I’ll use Lucky & my
friend’s rabbit, Flash.
Lucky gets Hippity Hop
food & Flash gets the
old rabbit food.
WOW! Lucky is bigger &
shinier so Hippity Hop is
better!
Old Food
Hippity Hop
The first five rabbits
that I catch will get
Hippity Hop food and
the remaining five will
get the old food.
The Hippity Hop rabbits have
scored higher so it’s the better
food!
Old Food
5
73
98
Hippity Hop
Number
from
–
Placethe
therabbits
numbers
in a1hat.
The
first
five numbers
10.
The
remaining
rabbitspulled
get
from the
be the
the hat
old will
food.
rabbits that get Hippity Hop
food.
6
2
4the rabbits & found
I evaluated
5
9
10
that
Hop
1 the 3rabbits eating 7Hippity
8
are better than
the old food!
10) blinding - method used so
that units do not know which
treatment they are getting
11) double blind - neither the
units nor the evaluator know
which treatment a subject
received
Rabbit Food
Hippity Hop
Rabbit Food
makes fur soft
and shiny, &
increases
energy for ALL
types of
rabbits!
Can I
make this
claim?
Principles of Experimental
Design
• Control of effects of extraneous
variables on the response – by
comparing treatment groups to a
control group (placebo or “old”)
• Replication of the experiment on
many subjects to quantify the
natural variation in the experiment
• Randomization – the use of chance
to assign subjects to treatments
The ONLY way to show cause &
effect is with a well-designed,
well-controlled experiment!
The ONLY way to show cause &
effect is with a well-designed,
well-controlled experiment!!
The ONLY way to show cause
& effect is with a welldesigned, well-controlled
experiment!!!
Example 1: A farm-product manufacturer wants
to determine if the yield of a crop is different
when the soil is treated with three different
types of fertilizers. Fifteen similar plots of
land are planted with the same type of seed
but are fertilized differently. At the end of
the growing season, the mean yield from the
sample plots is compared.
Experimental units?
Plots of land
Factors?
Type of fertilizer
Levels?
Fertilizer types A, B, & C
Response variable?
Yield of crop
How many treatments?
3
Example 2: A consumer group wants to
test cake pans to see which works the
best (bakes evenly). It will test aluminum,
glass, and plastic pans in both gas and
electric ovens.
Experiment units?Cake
Factors?
Levels?
batter
Two factors - type of pan & type of oven
Type of pan has 3 levels (aluminum, glass, & plastic
& type of oven has 2 levels (electric & gas)
Response variable? How
evenly the cake bakes
Number of treatments?
6
Example 3: A farm-product manufacturer
wants to determine if the yield of a crop is
different when the soil is treated with three
different types of fertilizers. Fifteen similar
plots of land are planted with the same type
of seed but are fertilized differently. At the
end of the growing season, the mean yield
from the sample plots is compared.
Why is the same type of seed used on all 15
plots? It is part of the controls in the experiment.
What are other potential extraneous
variables? Type of soil, amount of water, etc.
Does this experiment have a placebo? Explain
NO – a placebo is not needed in this experiment
Experiment Designs
• Completely randomized – all
experimental units are
allocated at random among all
treatments
explanatory
Treatment group 1
response
Treatment group 2
variable
variable
Treatment group 3
Treatment A
Treatment B
Treatment C
Treatment D
Randomly assign
experimental units to
treatments
Completely randomized design
explanatory
Group1
varaible
Group2
Random assignment
• Randomized block – units are
blocked into groups and then
randomly assigned to
treatments
Treatment 1
Treatment 2
Treatment 3
Treatment 1
Treatment 2
Treatment 3
response
varaible
Treatment A
Treatment B
Treatment A
Treatment B
Put
into homogeneous
Randomly
assign
groups
experimental
units to
treatments
Randomized block design
•Matched pairs - a special
type of block design
– match up experimental units
according to similar
characteristics & randomly assign
one to one treatment & the other
automatically gets the 2nd
treatment
– have each unit do both
treatments in random order
– the assignment of treatments is
dependent
Treatment A
Treatment B
experimental
Next,Pair
randomly
assign
units
according
to
one unit
from
a pair to
specific
Treatment
A. The
characteristics.
other
unit gets
Treatment B.
This is one way to do a matched
pairs design – another way is to have
the individual unit do both
treatments (as in a taste test).
12) Confounding variable – the
effect of the confounding
variable on the response cannot
be separated from the effects of
the explanatory variable (factor)
Suppose we wish to test a new
deodorant against one currently on
the market.
Treatment A
Treatment A
Treatment B
One group is
assigned to
Treatment B
treatment A & the
other group to
treatment B.
Confounding does NOT
occur in a completely
randomized
Treatment
& groupdesign!
are confounded
Example 4: An article from USA
Today reports the number of victims
of violent crimes per 1000 people. 51
victims have never been married, 42
are divorced or separated, 13 are
married, and 8 are widowed.
Is this an experiment? Why or why
not? No, no treatment was imposed on people.
What is a potential confounding
Age – younger people are more at risk
variable? to be victims of violent crimes
Example 5: Four new word-processing
programs are to be compared by measuring
the speed with which standard tasks can
be completed. One hundred volunteers are
randomly assigned to one of the four
programs and their speeds are measured.
Is this an experiment? Why or why not?
Yes, a treatment is imposed.
What type of design is this?
Completely randomized
Factors? Levels? one factor: word-processing
program with 4 levels
Response variable? speed
Example 5: Four new word-processing
programs are to be compared by
measuring the speed with which
standard tasks can be completed.
One hundred volunteers are randomly
do the
a block
designedYou
to could
one of
four programs
design where each person
and theiruses
speeds
are measured.
each program
in
random order.
Is there a potential confounding
variable?
Can this design
NO, completely randomized
designs have no confounding
be improved?
Explain.
Example 6: Suppose that the manufacturer
wants to test a new fertilizer against the
current one on the market. Ten 2-acre plots of
land scattered throughout the county are used.
Each plot is subdivided into two subplots, one
of which is treated with the current fertilizer,
and the other with the new fertilizer. Wheat
is planted and the crop yields are measured.
What type of design is
this? Why use this
method?
When does
randomization occur?
Matched - pairs
design
Randomly assigned
treatment to first acre
of each two-acre plot
Randomization reduces bias by
spreading any uncontrolled
Is there another way
confounding variables evenly
to reduce variability?
throughout the treatment groups.
Blocking also helps reduce
variability.
Variability is controlled by sample
size. Larger samples produce
statistics with less variability.
High bias & high variability
Low bias & high variability
High bias & low variability
Low bias & low variability