The Texas Sharpshooter’s Fallacy
There once was a sharpshooter who was challenged by a
skeptic to demonstrate his skill.
“I’m your huckleberry,” said the sharpshooter.
Then he drew his pistol and emptied it into the side of a
barn from a distance of 200 feet. He strolled up to the barn
and located the area where the bullets clustered most
tightly. He drew a circle around it and marked a bull’s eye in
the center of the cluster of bullet holes.
“See that,” he said. “Right where I was aiming.”
Should the skeptic be impressed with the sharpshooter’s
aim?
What Are the Chances?
• Abraham Lincoln and
John F. Kennedy were
both presidents of the
United States
• They were elected 100
years apart.
• Both were assassinated
• Lincoln had a secretary
named Kennedy
• Kennedy had a secretary
named Lincoln.
What Are the Chances?
• Both were shot and killed by
assassins who were known by
three names with 15 letters.
• They were both killed on a Friday
while sitting next to their wives.
• Lincoln was killed in the Ford
Theater
• Kennedy was killed in a Lincoln
made by Ford.
• Both men were succeeded by a
man named Johnson – Andrew
for Lincoln and Lyndon for
Kennedy. Andrew Johnson was
born in 1808. Lyndon Johnson
was born in 1908.
What Are the Chances?
• Kennedy was Catholic. Lincoln
was born Baptist.
• Kennedy was killed with a rifle,
Lincoln with a pistol.
• Kennedy was shot in Texas,
Lincoln in Washington D.C.
• Kennedy had lustrous auburn
hair, while Lincoln wore a stove
pipe hat.
• The more we look, the more of
such differences we would find,
but when you draw the bull’s-eye
around the similarities – whoa.
The Texas Sharpshooter’s Fallacy
• The Texas Sharpshooter’s fallacy concerns the
tendency for people to attach undue significance to
artifacts of randomness such as clustering, streaks,
and coincidences.
• “This can’t just be random. I mean, what are the
chances?” Asking this question under many
circumstances amounts to using the same data to test
your hypothesis that you used to generate the
hypothesis.
What Are the Chances?
• If you mean, “how likely would it be to be dealt a hand with
neither any matched cards, nor a straight, nor a flush?”
then the chance is about 1 in 2.
• If you mean the chance of getting this particular hand, then
the probability is 1 in about 2.6 million.
The Texas Sharpshooter’s Fallacy
• “There must be something serious at work
here. I mean, what are the chances!!?”
• Often this sort of astonishment is as foolish as
picking up your cards at the poker table after
being dealt every hand and exclaiming, “My
gosh! There is only a 1 in 2.6 million chance
that I would get this hand!”
The Texas Sharpshooter’s Fallacy
• The Sharpshooter’s Fallacy is at work when are
astonished by the occurrence of extremely low
probability events of this sort.
• If you are being dealt 5 cards, there is a 100%
chance that one of these extremely unlikely 1 in 2.6
million events will occur.
• Like the Texas Sharpshooter, someone announces
that a target has been hit at which no one previously
knew we were even aiming.
The Texas Sharpshooter’s Fallacy
• One of the reasons scientists form a hypothesis and then try
to disprove it with new research is to avoid the Texas
Sharpshooter Fallacy. Epidemiologists are especially wary of it
as they study the factors which lead to the spread of disease.
• If you look at a map of the United States with dots assigned
to where cancer rates are highest, you will notice areas of
clumping. It looks like you have a pretty good indication of
where the groundwater must be poisoned, or high-voltage
power lines are bombarding people with damaging energy
fields, or where cell phone towers are frying people’s organs,
or where nuclear bombs must have been tested.
• A map like that is a lot like the side of the sharpshooter’s
barn, and presuming there must be a cause for cancer
clusters is the same as drawing bull’s-eyes around them.
The Sharpshooter’s Fallacy
The fallacy is characterized by a lack of specific hypothesis prior to the
gathering of data, or the formulation of a hypothesis only after data has
already been gathered and examined (a practice also known as “Data
Snooping”).
What we used to say…
“What are the chances of THAT?!!”
“Whoa!”
“Oh, my gosh! That just CAN’T be a coincidence.”
What we will (I hope) say from now on…
“That seems interesting, but we should collect new data. To think we can
confirm our hypothesis from the same data we used to formulate our
hypothesis would be to make an error analogous to a sharpshooter marking
the location of the target after the shot has already been taken.”