Section 6.1 ~
The Role of Probability in Statistics:
Statistical Significance
Introduction to Probability and Statistics
Ms. Young
Sec. 6.1
Objective
Understand the concept of statistical
significance and the essential role that
probability plays in defining it.
Sec. 6.1
Statistical Significance

A set of measurements or observations are considered to be
statistically significant if they probably DID NOT occur by
chance


Ex. ~ Tossing a coin 100 times and getting 80 heads and 20 tails
would be statistically significant because it probably did not occur
by chance
Example 1:

Determine whether each scenario is statistically significant or not

A detective in Detroit finds that 25 of the 62 guns used in crimes
during the past week were sold by the same gun shop.

This finding is statistically significant. Because there are many gun shops in
the Detroit area, having 25 out of 62 guns come from the same shop seems
unlikely to have occurred by chance.
Sec. 6.1
Example 1 Cont’d…

In terms of the global average temperature, five of the years between
1990 and 1999 were the five hottest years in the 20th century.
Having the five hottest years in 1990–1999 is statistically significant
 By chance alone, any particular year in a century would have a 5 in 100, or 1
in 20, chance of being one of the five hottest years. Having five of those
years come in the same decade is very unlikely to have occurred by chance
alone
 This statistical significance suggests that the world may be warming up


The team with the worst win-loss record in basketball wins one game
against the defending league champions.

This one win is not statistically significant because although we expect a
team with a poor win-loss record to lose most of its games, we also expect it
to win occasionally, even against the defending league champions
Sec. 6.1
Example 2

A researcher conducts a double-blind experiment that tests whether a
new herbal formula is effective in preventing colds. During a threemonth period, the 100 randomly selected people in a treatment group
take the herbal formula while the 100 randomly selected people in a
control group take a placebo. The results show that 30 people in
the treatment group get colds, compared to 32 people in the
control group. Can we conclude that the new herbal formula is
effective in preventing colds?
Whether a person gets a cold during any three-month period depends on
many unpredictable factors. Therefore, we should not expect the number of
people with colds in any two groups of 100 people to be exactly the same.
 In this case, the difference between 30 people getting colds in the
treatment group and 32 people getting colds in the control group seems
small enough to be explainable by chance.
 So the difference is not statistically significant, and we should not conclude
that the treatment is effective.

Sec. 6.1
Quantifying Statistical Significance


Determining if something is statistically significant can be obvious in
some cases (i.e, 80 heads vs. 20 tails), but how do you decide if
something is statistically significant if the numbers are closer (i.e., 55
heads vs. 45 tails)?
Probability is used to quantify statistical significance by determining
the likelihood that a result may have occurred by chance

.05 level of significance: if the probability that something DID occur by
chance is less than or equal to .05, or 5%, then it is statistically significant
at the .05 level


.01 level of significance: if the probability that something DID occur by
chance is less than or equal to .01, or 1%, then it is statistically significant at
the .01 level


In other words, if the probability that something did occur by chance is small
(5%), then the probability that it did not occur by chance is big (95%), which
means it is statistically significant because it probably did not occur by chance
In other words, if the probability that something did occur by chance is small (1%),
then the probability that it did not occur by chance is big (99%), which means it is
statistically significant because it probably did not occur by chance
Something that is significant at the .01 level is also significant at the .05
level (since 1% is less than 5%), but something significant at the .05 level is
not necessarily significant at the .01 level (since something could be
significant at the .05 level if it’s under 5%, but doesn’t have to be as low as
1%)
Sec. 6.1
Example 3

In the test of the Salk polio vaccine (see Section 1.1), 33 of the
200,000 children in the treatment group got paralytic polio, while 115
of the 200,000 in the control group got paralytic polio. Calculations
show that the probability of this difference between the groups
occurring by chance is less than 0.01. Describe the implications of this
result.

The results are significant at the .01 level. This means there is a 1% chance
or less that the results occurred by chance, therefore the results probably
did not occur by chance which means that there is good reason to believe
that the treatment works.