The Language of
Statistical Decision
Making
Lecture 2
Section 1.3
Mon, Sep 5, 2005
Statistical Significance

Statistically significant – The difference between
the claim of the null hypothesis and the data is
large enough that it cannot reasonably be
attributed to chance.
Possible Errors

We might reject H0 when it is true.


We might accept H0 when it is false.


This is a Type I error.
This is a Type II error.
See Making Intelligent Errors, by Walter
Williams.
Decisions and Errors
State of Nature
H0 true
H0 false
Accept H0
Correct
Type II
Error
Reject H0
Type I
Error
Correct
Decision
Example

Consider the hypotheses:
H0: A category 5 hurricane will not hit New Orleans
in the next 25 years.
 H1: A category 5 hurricane will hit New Orleans in
the next 25 years.




What are the negative consequences of each
type of error?
Which type of error is more serious?
Which should get the benefit of the doubt?
Example

We should probably reverse the roles:
H0: A category 5 hurricane will hit New Orleans in
the next 25 years.
 H1: A category 5 hurricane will not hit New Orleans
in the next 25 years.

Example

Consider the hypotheses:
H0: An asteroid will not hit New Orleans in the next
25 years.
 H1: An asteroid will hit New Orleans in the next 25
years.




What are the negative consequences of each
type of error?
Which type of error is more serious?
Which should get the benefit of the doubt?
Example

These are fine the way they are:
H0: An asteroid will not hit New Orleans in the next
25 years.
 H1: An asteroid will hit New Orleans in the next 25
years.

Significance Level





Significance Level – The likelihood of rejecting
H0 when it is true, i.e., the likelihood of
committing a Type I error.
 – The likelihood of a Type I error.
 – The likelihood of a Type II error.
That is,  is the significance level.
The quantity 1 –  is called the power of the test.
Significance Level



The value of  is determined by our criteria for
rejecting the null hypothesis (which we haven’t
talked about yet).
If we demand overwhelming evidence against
H0 before rejecting it, then  will be small.
If we demand little evidence against it, then 
will be large.
Significance Level



Generally speaking, as  increases,  decreases.
That is, if we demand overwhelming evidence
against H0 (i.e., for H1) before rejecting it, then
 will be large.
If we demand little evidence against H0, then 
will be small.
Let’s Do It!



Let’s do it! 1.5, p. 13 – Which Error is Worse?
Let’s do it! 1.6, p. 14 – Testing a New Drug.
Porn can make you blind.