Class 03. The Lady Tasting Tea
R A Fisher
1890-1962
Rothamstead Experimental Station
Circa 1917
Dr B. Muriel Bristol
An Algologist
R A Fisher
1890-1962
Rothamstead Experimental Station
Circa 1917
The Experiment
• Ten pairs of cups (20 cups)
• Each pair has one MT the other TM
• The assignment of MT or TM to cups?
– is done randomly
• Double Blind
– Neither Dr B Muriel Bristol nor Sir R A Fisher know
the assignment pattern used.
Most Police Lineup’s are not double
blind…but they should be.
The use of so-called double-blind, sequential
lineups in criminal cases minimizes mistaken
eyewitness identifications, according to a
report released today by the American
Judicature Society.
“Double-blind” lineups must be conducted by
an officer without knowledge of the suspect's
identity.
http://www.texastribune.org/texas-dept-criminal-justice/innocence-projectof-texas/report-police-lineup-protocol-can-be-improved/
Double-Blind Placebo controlled is the
“gold standard” for medical
The Placebo
Effect.
experiments.
• In the 1950s, surgery for angina was internal mammary
artery ligation.
• In a 1955 experiment, half the patents were cut and sewed
up again.
The Placebo
– IDENTICAL OUTCOMES (immediate relief lasting 3
months…identical ekgs)
Effect.
• A 1993 experiment involving arthroscopic knee surgery for
arthritic knees.
– A was everything. B was everything but cartilage removal. C
was sham surgery.
– All three got pretty much identical results.
From the book, Predictably Irrational
Our experiment will be randomized,
double-blind, but there is no need for a
placebo.
If she guesses, how many of the
ten will she get correct?
How many will she have to get
correct in order to convince you
she’s not guessing?
H0: The Null Hypothesis
EMBS 9.1
• She is guessing
Independent trials with probability P=1/2 on each
Ha: The alternative Hypothesis
• She’s skillful
P > 1/2
A one-sided
alternative.
Examples of Null Hypotheses
•
•
•
•
•
•
He is innocent
The drug and placebo are equally effective
There is no such thing as a “hot hand” in sports.
The mean heights of Men and Woman are equal.
Smoking is unrelated to cancer.
There is no relationship between NFL outcomes
and Presidential election outcomes.
Classical Statistics
EMBS p 367
• Develop and state H0 and Ha.
• Specify the level of significance
– α=0.05 is most common
• Identify the test statistic, design and run the experiment, calculate
the test statistic
– 10 paired cups, double blind
– Test statistic is number of correct
• Calculate the p-value: the probability of observing a test statistic as
“extreme” as the one calculated if H0 is true.
– P(#correct ≥ 8 given she’s guessing) = .055
– P(#correct ≥ 9 given she’s guessing) = .011
– P(10 correct given she’s guessing) = .001
• Reject H0 if p-value is ≤ α
Volunteer?
Who can tell Coke from Pepsi?
The language of classical statistics
• If the p-value comes out smaller than α = 0.05
– The result is statistically significant at the 0.05 level
– We reject the null hypothesis.
– There is only a small probability this result happened
by chance.
– There is strong evidence to conclude that Ha is true.
(EMBS p 39)
Language you should not use.
• If the p-value comes out smaller than α = 0.05
– There is a greater than 0.95 probability Ha is true.
The language of classical statistics
• If the p-valueClassical
comes
out smaller than α = 0.05
Statistics Assumes H0 is true
– The result and
is statistically
significant
atabout
the 0.05 level
makes probability
statements
testnull
statistics.
Classical statistics does
– We reject the
hypothesis.
NOT make probability statements about
– There is only a smalleither
probability
H0 or Ha. this result happened
by chance.
– There is strong evidence to conclude that Ha is true.
(EMBS p 39)
Language you should not use.
• If the p-value comes out smaller than α = 0.05
– There is a greater than 0.95 probability Ha is true.
Two Types of “errors”
EMBS 9.2
• Type I. Rejecting H0 when it is true.
– (getting fooled by a guesser)
• Type II. Failing to reject H0 when Ha is true.
– (someone with skill not being about to convince you she’s
skillful).
• We focus on Type I because we can control it (by our
selection of α)
• We give less concern to Type II because it is difficult to
measure (how skillful is skillful?)
• The only way to make BOTH less likely, is to increase n.
Monday: The Normal Distribution
(a lecture, notes handed out,
assignment due on Wed)
Case: Wunderdog Sports Picks