The Lady Tasting Coffee:
A Case Study in Experimental Design
The History of Experimentation
Experimentation characterizes modern science.
Galileo (1564-1642) reportedly dropped balls of
various masses from the Leaning Tower of Pisa.
Assuming the story of Galileo’s Pisa experiment
is true: How many balls did he drop? How
many times did he repeat the comparison?
What were his independent and dependent
variables? How did he measure the time to
impact? We don’t know the answers to these
questions…
Take Home Message: Experimental design was
haphazard prior to the 1920’s.
Ronald Aylmer Fisher
• Considered by some scientists to be the father of
modern statistics .
• Poor eyesight; did a lot of math in his head without
paper or pencil.
• In 1919, he began working as a statistician at the
Rothamsted Agricultural Experiment Station in the
United Kingdom.
• Published many papers and wrote several books on
experimental design and evolution.
Ronald Aylmer Fisher
1890-1962
http://www-groups.dcs.st-and.ac.uk/~history/Mathematicians/Fisher.html
At Rothamsted, Fisher recognized problems with
some of the agricultural experiments
Same field, same treatment, but
plant performance is uneven...
Thin
Growth
Thick
Growth
Fisher’s Solution:
Replicate and
randomize to spread
variation evenly
among treatments.
Source of Picture: http://www.ipm.iastate.edu/ipm/icm/files/images/uneven-corn-VS6.jpg
Lessons Learned at Rothamsted
Experiments at Rothamsted prior to Fisher
generally involved two fields (containing
hundreds of plants), each receiving a
treatment.
Example: two levels of nitrogen (N) fertilizer
Field with
High N
Field with
Low N
Problem: So much variability exists within a field
itself that it is difficult or impossible to tease
out the effect the treatment.
Fisher’s Solution at Rothamsted
– Old Problematic Design: One large field receiving high
nitrogen (N), one large field receiving low nitrogen (N).
(Today this design is sometimes called “pseudoreplication” if
the experimenter attempts to say that the sample size is the
number of plants.)
– New Improved Design: Many small plots, randomly
receiving high N or low N; plots can also be blocked to
help tease out the variation due to location and local
conditions.
Hurlbert, S. H. (1984). Pseudoreplication and the design of ecological field experiments. Ecological monographs 54(2): 187-211.
Examples of Correct & Incorrect Ways
to Randomize Treatments
Correct Ways:
• Use a random
number table.
• Pick treatments
from a hat.
• Flip a coin.
Incorrect Ways:
• Haphazardly decide which experimental
units should receive which treatments.
(Problem: too tempting for experimenter to bias.)
• Use a net to grab the goldfish in an
ecology study. (Problem: might pick just the
easiest to catch, sickly animals.)
• Alternate treatments (every other one).
(Problem: that’s systematic, not random; who knows
what other factors vary in the same systematic way.)
• Assign people to drug study on the basis
of their last name. (Problem: could be related to
a person’s ancestry.)
Fisher, Randomization, Replication & Blocking
• No replication (or pseudoreplication) (Rothamsted, pre-Fisher):
Field with
High N
Field with
Low N
• Replicated with complete randomization:
Field broken
up into
smaller plots
Treatments are applied to plots
rather than to an entire field;
this improves replication &
interspersion of treatments.
• Replicated, randomized and blocked design:
Field broken
up into
smaller plots
& plots are
grouped.
Dashed rectangle
is a block
Plots are blocked by
location or other
condition; treatments
are applied randomly to
plots within blocks.
Another of Fisher’s Contributions to Statistics:
The Analysis of Variance (ANOVA)
Allows scientists to mathematically partition variation
among different sources (treatments, blocks, plots, for
example).
Some of Fisher’s contributions to the field of statistics grew out of
his experience with spatial agricultural experiments at
Rothamsted.
From: Sokal, Robert R., & F.James Rohlf, Biometry: The Principles and
Practice of Statistics in Biological Research, San Francisco: W.H. Freeman.
Why do these two
plants differ in
growth? Is it
because of block,
treatment, or
extraneous
variation within
plots?
At Rothamsted, Fisher saw firsthand that the purpose of good experimental design is
not to eliminate variation entirely, but rather to try to ensure that extraneous
variation is spread evenly among treatments. In the case of ANOVA, the
experimental design can enable the variation to be partitioned mathematically
during analysis.
Variation in growth of plants can be partitioned into different sources of variation:
1. Variation in soil moisture, texture, etc. within a plot.
2. Variation between treatments (high N and low N).
3. Variation in soil moisture, texture, sunlight, etc., among blocks.
The Design of Experiments (1935)
One of the first chapters of this textbook written by Fisher
is the essay, “Mathematics of a Lady Tasting Tea.”
A lady tasting tea
Can she tell whether the milk was added before or after the tea?
Afternoon tea during study abroad experience by University of Pittsburgh at Bradford students at the
University of Sussex in Brighton, Great Britain. Copyright © Janelle Elmquist. Used with permission.
So, you think statistics is boring . . .
Statisticians and the history of
statistics are far from boring.
Other interesting trivia on Fisher:
-Charming but had a terrible temper
(and a big ego)
-Smoked a pipe & argued
professionally in the 1950’s that
smoking did not cause cancer
-Supported eugenics
Picture taken from:
Parascandola, M. (2004). "Two approaches to etiology:
the debate over smoking and lung cancer in the 1950s."
Endeavour 28(2): 81-86.
Take Home Messages
• The 1920’s was a rich time for the development of concepts of
modern experimental design.
• Fisher was one of a number of statisticians who greatly
affected the development of modern statistics.
• Fisher’s experience at Rothamsted Agricultural Experiment
Station influenced his vision of experimental design and
helped him develop the concept of ANOVA .
• Fisher’s essay on a lady tasting tea eloquently outlines some
important issues in experimental design.
To learn more, read the biographies of
statisticians as you learn their techniques
The Student’s t-test
Student is the pseudonym of William Sealy Gosset, a
contemporary of Fisher who worked for Guiness, the Irish
brewery.
Other techniques
Many statistical techniques are named after interesting
historical people:
Bayes, Bernoulli, Cochran, Cox, Kolmogorov, Mann, Pearson, Smirnov, Tukey, Whitney,
Wilcoxon to name just a few
You are more likely to remember specific statistical
techniques if you know about the people who created
them. Don’t be afraid to look at the original works
published by these famous statisticians.
Examples of statistical techniques or tests named after people important in the
history of statistics. Names below include: Cochran, Cox, Friedman, Gosset,
Kolmogorov, Kruskal, Mann, Smirnov, Spearman, Wallace, Whitney, & Wilcoxon.
Recommended Reading
•
•
Salsburg, D. 2002. The Lady Tasting Tea: How Statistics Revolutionized Science in
the Twentieth Century. Henry Holt and Company, NY.
Stigler, S. M. 1999. Statistics on the Table: The History of Statistical Concepts and
Methods. Harvard University Press, Cambridge, MA.