Statistics as Evidence
Arguing by the numbers
impressing with statistics





approximately one in five children ages
10-17 who use the Internet frequently
have received a sexual solicitation or
approach over the Internet.
the mortality rate for avian flu is more
than 50 percent.
The average L.A. driver spends 136 hours
per year in rush hour traffic
A typical child has seen 4,286 acts of
violence on TV by age 18.
Bill Gates makes more money than the
poorest 100 million Americans
But are the numbers really that
impressive?

An article in the New England Journal of
Medicine stated “a person who purchases a
handgun is 57 times more likely to commit
suicide within a week of buying the weapon
than the general population as a whole.”
 The study said suicide is the leading cause of
death among gun buyers, in the first year after
a weapon is purchased.
 The statistic is misleading, however, because
buying a handgun doesn’t make one more
likely to commit suicide, wanting to commit
suicide makes one more likely to buy a
handgun.
 People who are bent on committing suicide
probably buy more knives, rope, sleeping pills,
and razor blades, too.
Are the numbers accurate?
•
•
Kevin & Bean, KROQ 10/9/08:
“Tigers have killed kill 300,000
people in India.”
Wikipedia: “Although humans
are not regular prey for it, the
tiger has killed more people
than any other cat. Between
1800 and 1900, it is estimated
that tigers had killed over
300,000 people in India alone”
•
•
•
Actually, about 300,000 people
live in the Sundarban area of
Bangladesh, where Royal
Bengal tigers are found
Tigers are responsible for about
10 deaths a year. (Reuters, Jan
23, 2008, reported by Enamul
Haque)
Valmik Thapar, a leading expert
on tigers in India, put deaths
due to tiger attacks at about 80
people annually. (The
Statesman, New Delhi, July 16,
2001)
Are the numbers accurate?
• On average
men think
about sex
every 7
seconds.
• 54% of men think about
sex everyday or several
times a day, 43% a few
times per month or a few
times per week, and 4%
less than once a month
(Kinsey Report from
KinseyInstitute.org)
watch out for “unknowable”
statistics
 example: The average person tells 13
lies per week
 example: There are at least 30 million
victims of modern day slavery in the
world today (US Trafficking in Persons
Report).
 example: “in up to 98% of the cases,
simply brandishing a gun is sufficient
to stop a crime.” John Lott Jr., Wall
street Journal July,30, 2001
comparing non-comparable units
(apples versus oranges)
A study conducted by the dean of
admission for Harvard University
concluded that Harvard students who
took prep courses scored lower than
students who didn’t take prep courses.
The study concluded “the coaching
industry is playing on parental anxiety.”
– However, the comparison is not based
non-equivalent samples of students.
Students who didn’t even bother taking
a prep course might be much stronger
academically than students who opted to
take such a course.
No
coaching
coached
difference
VSAT
649
611
+38
MSA
T
685
660
+25
watch out for unrepresentative
samples
 Non-random samples:
 voting on the American Idol show
 phone calls or emails to a politician about a particular bill
 testimonials on infomercials
 talk radio callers
 To be truly valid a sample must be:
 random
 representative
 sufficient
questionable extrapolations
 Using a limited sample (limited in time or
number) and extrapolating the percentages
nationwide or globally
 example: According to the National Self Defense
Survey in 1994, the rate of Defensive Gun Uses can be
projected nationwide to approximately 2.5 million per
year -- one Defensive Gun Use every 13 seconds.
 http://www.pulpless.com/gunclock/framedex.html
questionable extrapolation
• Babe Ruth would have hit 1,051
home runs if he had the same
number of at bats as Hank Aaron.
• Ruth hit .08502 home runs per at
bat, Aaron had 12,364 at bats
keep the numbers in perspective
 Compare “scary” statistics to other, well-established health
risks.
 “mad cow disease” or the “flesh eating virus” sound scary,
but the odds are much greater of dying of dog bites, drowning
in the tub, or being struck by lightning.
 Chicken pox kills 100 people per year in America.
 Flu kills about 35,000 people annually in the U.S.
Keep the numbers in perspective
 School killings
 Total school-related
 Compared to drownings
and car crashes
violent deaths, August 1,  In 2003, 782 children ages
2005, July 31, 2006:
0 to 14 years died from
drowning (CDC 2005).
 Shooting: 15
Suicides: 1
 In the United States during
Murder-Suicide: 4
2004, 1,638 children ages
14 years and younger died
 Fight-Related: 0
as occupants in motor
 Stabbing: 3
vehicle crashes
 Other: 4
 Total = 27
Correct interpretation?
• Does owning a gun make it more likely you’ll be
killed?get you killed?
• gun owners are 2.7 times more likely to be murdered than
non-owners. (Arthur Kellermann et. al., "Gun Ownership as a Risk
Factor for Homicide in the Home," The New England Journal of
Medicine, October 7, 1993, pp. 1084-1091)
• Does this prove it is safer not to have guns in the house?
• It may be true that guns cause murders, but it also might
be true that having a greater risk of being murdered causes
people to own guns.
bias in gathering and reporting
statistics
•
•
•
Cathie Adams, president of the Texas Eagle Forum,
stated on CNN on April 21, 2005 that: “research
shows that children in same-sex couple homes are 11
times more likely to be abused sexually…It is a
proven fact and that was a research study done in the
state of Illinois”
But Illinois doesn’t keep track of whether households
are same sex or not. It's possible that much of what the
study calls “homosexual abuse” occurred in
heterosexual households.
The study also confuses pedophilia with
homosexuality. If the sexual abuse was male-male or
female-female, the study presumed the abuser was gay
or lesbian.
not defining the units to be counted
clearly
• example: statistics on “juvenile violence”
• the legal ages of juveniles vary from state to
state, as do the ages at which juveniles may be
tried as adults.
• example: comparing “unemployment”
rates in two countries
• The definitions of unemployed may not be the
same. In the U.S., people who are underemployed, or having stopped looking for work
aren’t counted.
• example: statistics on “child abductions”
• can refer to kidnappings by strangers or
parental custody disputes
Not defining the units clearly
•
•
•
•
Report: 1 in 50 U.S. children face homelessness
“OVER 1.5 MILLION OF OUR NATION’S CHILDREN GO TO
SLEEP WITHOUT A HOME EACH YEAR.”
The report, by the National Center on Family Homelessness, analyzed
data from 2005-06 and found that more than 1.5 million children were
without a home.
By “each year,” they mean “at least one night a year.” But what does
“without a home” mean?
•
•
“Sharing the housing of other persons due to loss of housing, economic
hardship, or a similar reason (sometimes referred to as doubled-up);
• Living in motels, hotels, trailer parks, or camping grounds due to lack of
alternative accommodations”
if you did any of the above for one night out of a year, you are counted
by the National Center on Family Homelessness as “homeless.”
carefully considering statistical
measures
 Failure to report the margin of error (a.k.a “sampling error”)
 example: +/- 5% in a national poll of 1,200 respondents
 Making conclusions within the margin of error
 If candidate A is preferred by 43% of voters, and candidate B is
preferred by 46% of voters, all one can say is that they are tied in a
statistical dead heat.
 Not stating whether the results are statistically significant or what
the threshold for statistical significance is.
 example: p < .05
misleading statistics
• Some studies have reported that
90% of students are bullied at
school.
• But the results depend on what the
researcher defines as “bullying.” Does
bullying included physical and verbal
abuse? If bullying means “people being
mean to you,” then it is surprising that
10% of students haven’t been bullied!
Misleading statistics
• Comparing the
total number of
complaints by
airline
passengers,
without
considering the
total numbers
of passengers
per airline.
Airline
Total Complaints JanOct 2008
American
1,105
Continental
456
Delta
1,120
Northwest
362
Southwest
224
United
1,045
US Airways
957
misleading graphs and charts
•
This CDC graph,
comparing causes of death
is misleading, because it
leaves out deaths from
heart disease, cancer, and
stroke. The omission
tends to make smoking
look much more harmful
by comparison.
media slanting: What to watch and
listen for








Remember: news media are after ratings, not necessarily
the truth.
Watch for bias during the opening or lead-in.
Ask how representative or generalizable case studies are.
Don’t let pictures overwhelm the facts.
Beware of quoted sources with an agenda.
Beware of “black hats” or “white hats.”
When statistics are presented is the underlying
methodology explained?
Watch out for rhetorical questions with no follow-up
proof.