Power, Power Curves and
Sample Size
1
Planning Reliable and
Efficient Tests
For any job, you need a tool that offers
the right amount of power for the task at
hand.
You wouldn't use a telescope to examine a
stamp collection, or a handheld magnifying
glass to search for new galaxies, because
neither would provide you with meaningful
observations. To complicate matters, if
detecting a galaxy really was your goal, the
cost of gaining the necessary power might
be more than you can afford.
2
Planning Reliable and
Efficient Tests
Anyone using statistical tests faces
the same issues. You must consider
the precision you need to meet your
goals (should your test detect subtle
effects or massive shifts?), and
balance it against the cost of
sampling your population (are you
testing toothpicks or jet engines?).
3
Planning Reliable and
Efficient Tests
You also want the confidence in your
results that's appropriate for your
situation (testing seat belts demands
a greater degree of certainty than
testing shampoo). We measure this
certainty with statistical power – the
probability your test will detect an
effect that truly exists.
4
Planning Reliable and
Efficient Tests
Minitab's Power and Sample Size
tools, with Power Curves, help you
balance these issues that may
compete for your limited resources.
Here are three examples of how a
quick Power and Sample Size test can
help you save time and money getting
results you can trust.
5
Don't Leave Success to
Chance
A paper clip manufacturer wants to
detect significant changes in clip
length. They sample thousands of
clips because it is cheap and quick to
do. But this huge sample makes the
test too sensitive: the broken line
(next slide) shows it will sound the
alarm if the average length differs by
a trivial amount (0.05).
6
Don't Leave Success to
Chance
Power Curve for 1-Sample t Test
1.0
Sample
Size
100
2000
0.8
A ssumptions
A lpha
0.05
S tDev
1
A lternativ e
>
Power
0.6
0.4
0.2
0.0
0.00
0.05
0.10
0.15
Difference
0.20
0.25
0.30
7
Don't Leave Success to
Chance
This Power Curve shows they are
wasting resources on excessive
precision. A sample size of just 100
will detect meaningful differences
(0.25) without “crying wolf” at every
negligible blip.
8
Don't Leave Success to
Chance
9
Don't Leave Success to
Chance
An aerospace company is designing an
experiment to test a new rocket.
Each rocket is very expensive, so it is
critical to test no more than
necessary.
10
Don't Leave Success to
Chance
Power Curve for 2-Level Factorial Design
1.0
Reps,
Ctr Pts Per Blk
6, 0
0.8
A ssumptions
A lpha
0.05
S tDev
1
# F actors
3
# C orner P ts
8
# Blocks
none
# Terms O mitted
0
Power
0.6
0.4
0.2
0.0
-1.0
-0.5
0.0
Effect
0.5
1.0
11
Don't Leave Success to
Chance
This Power Curve confirms an
experiment with 6 replicates will give
researchers the power they need
without spending more than they
must.
12
Don't Leave Success to
Chance
13
Don't Leave Success to
Chance
“We've always done it this way.”
That's why a lumber company would
sample 10 beams to test whether
their strength meets the target.
14
Don't Leave Success to
Chance
Power Curve for 1-Sample t Test
1.0
Sample
Size
10
34
0.8
A ssumptions
A lpha
0.05
S tD ev
1
A lternativ e N ot =
Power
0.6
0.4
0.2
0.0
-1.0
-0.5
0.0
Difference
0.5
1.0
15
Don't Leave Success to
Chance
According to the Power Curve, this
small sample size made their test
incapable of detecting important
effects. They must sample 34 beams
to detect meaningful differences
(0.50).
16
Don't Leave Success to
Chance
17
See the Big Picture
A power analysis helps you weigh your
resources against your demands, and
quantifies a test's ability to answer
your question. It can expose design
problems, like the lumber company's
insufficient sample size. It can also
reveal design solutions you hadn't
considered.
18
See the Big Picture
Take for instance the packaging plant
of a snack company. Customers
complain that the company's pretzel
bags are sealed with glue that's too
strong, so researchers use One-Way
ANOVA to compare their current
glue with three potential
replacements.
19
See the Big Picture
Differences in seal strength less than
10 are undetectable to most people,
so their test only needs to detect a
difference of 10. A power value of
80% is acceptable, but 90% is ideal.
What sample size meets their needs?
20
See the Big Picture
30 samples of each glue ensure the
test detects a difference of 10 with
90% power.
21
See the Big Picture
Or, they could detect the same difference with 23
samples and 80% power. If this represents
considerable savings, the researchers may consider
using the smaller sample.
22
See the Big Picture
The Power Curve illustrates this
information, but it also charts every
other combination of power and
difference for a given sample size.
23
See the Big Picture
Power Curve for One-way ANOVA
1.0
Sample
Size
23
30
0.8
A ssumptions
A lpha
0.05
S tDev
10
# Lev els
4
Power
0.6
0.4
0.2
0.0
0
5
10
Maximum Difference
15
20
24
See the Big Picture
The solid line indicates researchers
can attain 90% power with just 23
samples if they are willing to seek a
difference of 12 instead of 10. This
might just be the ideal choice.
25
See the Big Picture
26
How to Create Power
Curves in Minitab
Performing power analyses with Power
Curves couldn't be simpler. You supply
the factors you know, and Minitab
calculates the one you omit.
27
How to Create Power
Curves in Minitab
Suppose a trainer wants to compare
two training courses for forklift
operators.
She will use a two-sample t-test to
compare the average scores that
operators from each course earned
on the final exam.
28
How to Create Power
Curves in Minitab
She knows she must detect a
difference of 5 in either direction
between the two courses with 80%
power, and historical data suggest a
standard deviation of 5.
But how many participants must she
sample from each course?
29
How to Create Power
Curves in Minitab
Choose
Stat > Power and Sample Size
> 2-Sample t
In Differences, type -5, 5
In Power values, type 0.80
In Standard deviation, type 5
Click OK
30
How to Create Power
Curves in Minitab
31
How to Create Power
Curves in Minitab
Power Curve for 2-Sample t Test
1.0
Sample
Size
17
0.8
A ssumptions
A lpha
0.05
S tD ev
5
A lternativ e N ot =
Power
0.6
0.4
0.2
0.0
-6
-4
-2
0
Difference
2
4
6
32
How to Create Power
Curves in Minitab
According to this Power Curve,
sampling 17 participants from each
class enables her test to detect the
difference she seeks with 80%
power.
33
How to Create Power
Curves in Minitab
34
Putting Power Curves to
Use
Without knowing the power of your test,
it's hard to know if you can trust your
results: your test could be too weak to
answer your question, or too strong for
your needs. Minitab's Power Curves
(available for many common statistical
procedures) help you balance your
resources against your goals and design a
test you can trust that costs no more than
necessary.
35
Putting Power Curves to
Use
Power Curves graph the dynamic
relationships that define power,
revealing the big picture and ensuring
no option escapes your consideration.
And perhaps most importantly, they
make power analysis an easier and
more accessible part of every
project. Empower your test. Trust
your results.
36