STAT 3130
Statistical Methods II
Session 6
Statistical Power
STAT3130 – Statistical Power
Ho TRUE
Reject Ho
FTR Ho
Ho FALSE
Type 1 Error (alpha)
Power (1-beta)
1-alpha
Type 2 Error (beta)
When we engage in Hypothesis Testing, we have focused
on the p-value and the alpha value…recall what these
actually mean.
STAT3130 – Statistical Power
Ho TRUE
Reject Ho
FTR Ho
Ho FALSE
Type 1 Error (alpha)
Power (1-beta)
1-alpha
Type 2 Error (beta)
A second calculation of interest is the Power of the test –
or 1- the probability of committing a Type 2 error. In
words, the Power of the test is “The probability of correctly
rejecting the Null Hypothesis when it should be rejected”.
STAT3130 – Statistical Power
In other words, it is the probability that if a true difference exists, it
will be discovered. It is sometimes referred to as the “sensitivity” of
the test.
Statistical Power is heavily used in medicine, clinical psychology
and biology.
Typically, a test must have a Statistical Power of 80% or greater to
be considered valid. Why would that be?
STAT3130 – Statistical Power
Power is a function of three factors:
1.
Effect size – i.e., the difference between the two groups or
measurements. As the effect size goes up, the power increases.
2.
Alpha – As the chance of finding an incorrect significant effect is
reduced (Type I error), the probability of correctly finding an effect
is also reduced. Typically, alpha is set to be .01 (most conservative
and lowers power), .05 or .10 (most risk tolerant and increases
power). As alpha goes down, the power increases.
3.
Sample Size - Increased sample sizes will always produce greater
power. But, increasing the sample size can also produce “too
much” power – smaller and smaller effects will be found to be
significant until at a large enough sample size, any effect is
considered to be significant. As sample size goes up, power
increases.
STAT3130 – Statistical Power
When conducting a one tailed test, the power calculation is
executed as:
1-() = 1-P[z<z- (| 0 - a|/ (s/SQRT(n))]
A two tailed test is conducted similarly, with the alpha value associated
with the z score divided by two.
STAT3130 – Statistical Power
Example…
Lets say that we have a treatment group taking a cholesterol
lowering drug. Lets say that the average decrease in cholesterol
after taking the drug is 2 points. (n=100 and s = 10). What do you
conclude.
1-(2) = 1-P[z<1.645- (| 2|/ (10/SQRT(100))]
Now…what if the drop is 3 points. What happens?
STAT3130 – Statistical Power
Lets build some power curves…
Power versus effect size
Power versus sample size
STAT3130 – Statistical Power
If you are starting an experiment from scratch,
how do you determine the appropriate sample
size?
n= 2*(Z +Z )2/Δ2
A two tailed test is conducted similarly, with the alpha value
associated with the z score divided by two.
(Or…you can just use SAS).