Statistics as Evidence
Arguing by the numbers
impressing with statistics





Approximately one in five children ages
10-17 who use the Internet frequently
have received a sexual solicitation or
approach over the Internet.
The mortality rate for avian flu is more
than 50 percent.
The average L.A. driver spends 136
hours per year in rush hour traffic
A typical child has seen 4,286 acts of
violence on TV by age 18.
Bill Gates makes more money than the
poorest 100 million Americans
But are the numbers really that impressive?




An article in the New England Journal of Medicine stated “a
person who purchases a handgun is 57 times more likely to
commit suicide within a week of buying the weapon than the
general population as a whole.”
The study said suicide is the leading cause of death among
gun buyers, in the first year after a weapon is purchased.
The statistic is misleading, however, because buying a
handgun doesn’t make one more likely to commit suicide,
wanting to commit suicide makes one more likely to buy a
handgun.
People who are bent on committing suicide probably buy
more knives, rope, sleeping pills, and razor blades, too.
watch out for “unknowable” statistics



example: The average person tells 13 lies per week
example: “in up to 98% of the cases, simply brandishing a gun
is sufficient to stop a crime.” John Lott Jr., Wall street Journal
July,30, 2001
example: Judy Garland sang “over the Rainbow” 1,476 times.”
Esquire, Feb. 1964
comparing non-comparable units (apples
versus oranges)

A study conducted by the dean of admission for Harvard
University concluded that Harvard students who took prep
courses scored lower than students who didn’t take prep
courses. The study concluded “the coaching industry is
playing on parental anxiety.”

However, the comparison is based on non-equivalent samples
of students. Students who didn’t even bother taking a prep
course might be much stronger academically than students
who opted to take such a course.
No
coaching
coached
difference
VSAT
649
611
+38
MSAT
685
660
+25
watch out for unrepresentative samples
 Non-random samples:
 voting on the American Idol show
phone calls or emails to a politician about a particular
bill
 testimonials on infomercials
 talk radio callers
 To be truly valid a sample must be:
 random
 representative
 sufficient

questionable extrapolations
 Using a limited sample (limited in time or
number) and extrapolating the percentages
nationwide or globally


example: According to the National Self Defense
Survey in 1994, the rate of Defensive Gun Uses
can be projected nationwide to approximately 2.5
million per year -- one Defensive Gun Use every
13 seconds.
http://www.pulpless.com/gunclock/framedex.html
keep the numbers in perspective
 Compare “scary” statistics to other, well-established
health risks.



“mad cow disease” or the “flesh eating virus” sound
scary, but the odds are much greater of dying of
dog bites, drowning in the tub, or being struck by
lightning.
Chicken pox kills 100 people per year in America.
Flu kills about 35,000 people annually in the U.S.
Keep the numbers in perspective







School killings
Total school-related violent
deaths, August 1, 2005,
July 31, 2006:
Shooting: 15
Suicides: 1
Murder-Suicide: 4
Fight-Related: 0
Stabbing: 3
Other: 4
Total = 27



Compared to drownings
and car crashes
In 2003, 782 children ages
0 to 14 years died from
drowning (CDC 2005).
In the United States during
2004, 1,638 children ages
14 years and younger died
as occupants in motor
vehicle crashes
misleading statistics

Some studies have reported that 90% of
students are bullied at school.

But the results depend on what the researcher defines
as “bullying.” Does bullying included physical and
verbal abuse? If bullying means “people being mean to
you,” then it is surprising that 10% of students haven’t
been bullied!
misleading graphs and charts

This CDC graph,
comparing causes of
death is misleading,
because it leaves out
deaths from heart
disease, cancer, and
stroke. The omission
tends to make smoking
look much more
harmful by comparison.
not defining the units to be counted clearly


example: statistics on “juvenile violence”
 the legal ages of juveniles vary from state to
state, as do the ages at which juveniles may be
tried as adults.
example: statistics on “child abductions”
 can refer to kidnappings by strangers or
parental custody disputes
carefully considering statistical
Failure to report the margin of error ( “sampling error”)
measures

example: +/- 5% in a national poll of 1,200 respondents
 Making conclusions within the margin of error
 If candidate A is preferred by 43% of voters, and
candidate B is preferred by 46% of voters, all one can
say is that they are tied in a statistical dead heat.
 Not stating whether the results are statistically
significant or what the threshold for statistical
significance is.
 example: p < .05

media slanting: What to watch and listen
for








Remember: news media are after ratings, not necessarily
the truth.
Watch for bias during the opening or lead-in.
Ask how representative or generalizable case studies are.
Don’t let pictures overwhelm the facts.
Beware of quoted sources with an agenda.
Beware of “black hats” or “white hats.”
When statistics are presented is the underlying
methodology explained?
Watch out for rhetorical questions with no follow-up proof.