The Essentials of 2-Level Design of Experiments
Part I: The Essentials of Full Factorial Designs
Developed by Don Edwards, John Grego and James Lynch
Center for Reliability and Quality Sciences
Department of Statistics
University of South Carolina
803-777-7800
Part I. Full Factorial Designs
 24
Designs
– Introduction
– Analysis Tools
– Example
– Violin Exercise
 2k
Designs
24 Designs
U-Do-It - Violin Exercise
24 Designs
U-Do-It - Violin Exercise
How to Play the Violin in 176 Easy Steps1,2

A very scientifically-inclined violinist was interested in
determining what factors affect the loudness of her
instrument. She believed these might include:
–
–
–
–


A:
B:
C:
D:
Pressure (Lo,Hi)
Bow placement (near,far)
Bow Angle (Lo,Hi)
Bow Speed (Lo,Hi)
The precise definition of factor levels is not shown, but
they were very rigidly defined and controlled in the
experiment.
Eleven replicates of the full 24 were performed, in
completely randomized order. Analyze the data!
1176=11x16
2Data courtesy of Carla Padgett
24 Designs
U-Do-It - Violin Exercise Report Form
A
Pressure
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1
B
C
D
Bow
Bow Angle
Bow Speed
Placement
-1
-1
-1
-1
-1
-1
1
-1
-1
1
-1
-1
-1
1
-1
-1
1
-1
1
1
-1
1
1
-1
-1
-1
1
-1
-1
1
1
-1
1
1
-1
1
-1
1
1
-1
1
1
1
1
1
1
1
1
Y
Loudness
(Decibels)
69.3
75.3
75.9
79.3
67.4
74.9
74.4
78.8
73.4
77.5
78.8
81.6
72.3
78.8
78.1
81.5

Responses are Averages
of 11 Independent
Replicates
– All 176 trials were
randomly ordered

Analyze and Interpret the
Data
24 Designs
U-Do-It Solution - Violin Exercise Signs Table
Main Effects
Actual
Orde r
Sum
Divisor
Effe ct
y
69.3
75.3
75.9
79.3
67.4
74.9
74.4
78.8
73.4
77.5
78.8
81.6
72.3
78.8
78.1
81.5
A
B
C
D
AB
-1
-1
-1
-1
1
1
-1
-1
-1
-1
-1
1
-1
-1
-1
1
1
-1
-1
1
-1
-1
1
-1
1
1
-1
1
-1
-1
-1
1
1
-1
-1
1
1
1
-1
1
-1
-1
-1
1
1
1
-1
-1
1
-1
-1
1
-1
1
-1
1
1
-1
1
1
-1
-1
1
1
1
1
-1
1
1
-1
-1
1
1
1
-1
1
1
1
1
1
1217.3 38.1 39.5 -4.9 26.7 -10.1
16
8
8
8
8
8
76.1 4.8 4.9 -.6 3.34 -1.3
Interaction Effects
AC AD
1
1
-1
-1
1
1
-1
-1
-1
1
1
-1
-1
1
1
-1
1
-1
-1
1
1
-1
-1
1
-1
-1
1
1
-1
-1
1
1
5.5 -4.5
8
8
.7 -.6
BC BD
1
1
1
1
-1
-1
-1
-1
-1
1
-1
1
1
-1
1
-1
1
-1
1
-1
-1
1
-1
1
-1
-1
-1
-1
1
1
1
1
-.7 -3.5
8
8
-.1 -.4
CD ABC ABD ACD BCD ABCD
1
-1
-1
-1
-1
1
1
1
1
1
-1
-1
1
1
1
-1
1
-1
1
-1
-1
1
1
1
-1
1
-1
1
1
-1
-1
-1
1
-1
1
1
-1
-1
1
1
-1
1
-1
1
-1
-1
-1
-1
-1
-1
1
1
1
-1
-1
1
-1
-1
1
1
-1
1
-1
1
-1
1
-1
-1
1
-1
-1
-1
1
1
1
-1
-1
1
1
-1
-1
1
-1
-1
1
-1
-1
-1
1
-1
1
1
1
1
1
1
3.7 -2.3
1.3
.5 -1.3
-1.3
8
8
8
8
8
8
.5
-.3
.2
0
-.2
-.2
U-Do-It Exercise
Note:
Hi C
Lo C
U-Do-It Solution - Violin Exercise Cube Plot
y
69.3
75.3
75.9
79.3
67.4
74.9
74.4
78.8
73.4
77.5
78.8
81.6
72.3
78.8
78.1
81.5
A
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1
B
-1
-1
1
1
-1
-1
1
1
-1
-1
1
1
-1
-1
1
1
C
-1
-1
-1
-1
1
1
1
1
-1
-1
-1
-1
1
1
1
1
D
-1
-1
-1
-1
-1
-1
-1
-1
1
1
1
1
1
1
1
1
81.5
81.6
78.1
78.8
74.4
75.9
78.8
79.3
+
78.8
77.5
72.3
73.4
B
+

Factors
–
–
–
–
A: Pressure (Lo,Hi)
B: Bow Placement
(near,far)
C: Bow Angle (Lo,Hi)
D: Bow Speed (Lo,Hi)
D
_
67.4
69.3
_
74.9
75.3
A
+
_
24 Designs
U-Do-It Solution - Violin Exercise Effects Normal Probability Plot

Factors
– A: Pressure (Lo,Hi)
– B: Bow Placement
(near,far)
– C: Bow Angle (Lo,Hi)
– D: Bow Speed (Lo,Hi)
B
A
D
AB
-1
0
1
Ordered Effects: -1.3 -0.6 -0.6 -0.4 -0.3
-0.2 -0.2 -0.1 0.0 0.2 0.5 0.7 3.34 4.8 4.9
2
3
Effects
4
5
24 Designs
U-Do-It Solution - Violin Exercise Interpretation

The interaction between A and B is so weak
that it is probably ignorable and will not be
included initially. This simplifies the analysis
since, when there are no interactions, the
observed changes in the response will be the
sum of the individual changes in the main
effects, i.e, the main effects are additive.
24 Designs
U-Do-It Solution - Violin Exercise Interpretation

When the AB interaction is ignored, we expect
– A loudness increase of 3.3 decibels when increasing
bow speed from Lo to Hi.
– A loudness increase of about 5 decibels when
changing the bow placement from “near” to “far”.
– A loudness increase of 4.8 decibels when changing
pressure from Lo to Hi.
– The loudness seems unaffected by the angle factor;
this “non-effect” is in itself interesting and useful.
U-Do-It Exercise
U-Do-It Solution
Violin Exercise: Including the AB Interaction

We now include the AB interaction for
comparison purposes. Since the
interaction is so weak, it does not
appreciably change the analysis
U-Do-It Exercise
U-Do-It Solution - Violin Exercise AB Interaction Table
y
69.3
75.3
75.9
79.3
67.4
74.9
74.4
78.8
73.4
77.5
78.8
81.6
72.3
78.8
78.1
81.5
A
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1
B
-1
-1
1
1
-1
-1
1
1
-1
-1
1
1
-1
-1
1
1
B: Bow Plac em ent
1
1
A:
Pres sure
2
69.3
67.4
73.4
72.3
282.4
A1B1 =
282.4/4 = 70.60
75.3
74.9
77.5
78.8
306.5
A2B1 =
306.5/4 = 76.625
2
75.9
74.4
78.8
78.1
307.2
A1B2 =
307.2/4 = 76.80
79.3
78.8
81.6
81.5
321.2
A2B2 =
321.2/4 = 80.30
U-Do-It Exercise
U-Do-It Solution - Violin Exercise AB Interaction Table/Plot
B: Bow Pla ce ment
1
1
A:
Pres sure
2
69.3
67.4
73.4
72.3
282.4
A1B1 =
282.4/4 = 70.60
75.3
74.9
77.5
78.8
306.5
A2B1 =
306.5/4 = 76.625
2
75.9
74.4
78.8
78.1
307.2
A1B2 =
307.2/4 = 76.80
79.3
78.8
81.6
81.5
321.2
A2B2 =
321.2/4 = 80.30
24 Designs
U-Do-It Solution - Violin Exercise Interpretation

If We Include the AB Interaction, We
Expect
– Loudness to increase 3.3 when bowing
speed, D, increases from Lo to Hi.
– Since the lines in the AB interaction are
nearly parallel, the effect of the
interaction is weak. This is reflected in
our estimates of the EMR.
24 Designs
U-Do-It Solution - Violin Exercise EMR
Let us calculate the EMR if we want the
response to be the quietest.
 If We Don’t Include the AB Interaction,

–
–
Set A, B and D at their Lo setting, -1.
EMR = 76.1 - (4.8+4.9+3.34)/2 = 69.56
y
1217.3
16
76.1

A
-1
-1
38.1
8
4.8
B
-1
-1
39.5
8
4.9
C
-1
1
-4.9
8
-.6
D
-1
-1
26.7
8
3.34
AB
1
1
-10.1
8
-1.3
If We Include the AB Interaction,
–
–
Set D at its Lo setting, -1. The AB Interaction
Table and Plot show that A and B still should be
set Lo, -1. Note that when A and B are both -1,
AB is +1.
EMR = 76.1 - (4.8+4.9+3.34)/2 +(-1.3)/2
=68.93
y
1217.3
16
76.1
A
-1
-1
38.1
8
4.8
B
-1
-1
39.5
8
4.9
C
-1
1
-4.9
8
-.6
D
-1
-1
26.7
8
3.34
AB
1
1
-10.1
8
-1.3
y
69.3
75.3
75.9
79.3
67.4
74.9
74.4
78.8
73.4
77.5
78.8
81.6
72.3
78.8
78.1
81.5
A
B
C
D
AB
-1
-1
-1
-1
1
1
-1
-1
-1
-1
-1
1
-1
-1
-1
1
1
-1
-1
1
-1
-1
1
-1
1
1
-1
1
-1
-1
-1
1
1
-1
-1
1
1
1
-1
1
-1
-1
-1
1
1
1
-1
-1
1
-1
-1
1
-1
1
-1
1
1
-1
1
1
-1
-1
1
1
1
1
-1
1
1
-1
-1
1
1
1
-1
1
1
1
1
1
1217.3 38.1 39.5 -4.9 26.7 -10.1
16
8
8
8
8
8
76.1 4.8 4.9 -.6 3.34 -1.3
2k Designs
Introduction


Suppose the effects of k factors, each having two levels,
are to be investigated.
How many runs (recipes) will there be with no replication?
– 2k runs

How may effects are you estimating?
– There will be 2k-1 columns in the Signs Table
– Each column will be estimating an Effect
 k main effects, A, B, C,...
 k(k-1)/2 two-way interactions, AB, AC, AD,...
 k(k-1)(k-2)/3! three-way interactions
 .
.
.
 k (k-1)-way interactions
 one k-way interaction
2k Designs
Analysis Tools

Signs Table to Estimate Effects
– 2k-1 columns of signs; first k estimate the k main effects and remaining 2k-k -1
estimate interactions
 2k
- 1 Effects Normal Probability Plots to
Determine Statistically Significant Effects
 Interaction Tables/Plots to Analyze TwoWay Interactions
 EMR Computed as Before
2k Designs
Concluding Comments
Know How to Design, Analyze and
Interpret Full Factorial Two-Level Designs
 This means that

– The design is orthogonal
– The run order is totally randomized
2k Designs
Orthogonality

(Hard to Explain) If a Design is
Orthogonal, Each Factor’s “Effect” can be
Estimated Without Interference From the
Others...
2k Designs
Orthogonality - Checking Orthogonality
1. Use the -1 and 1 Design Matrix.
2. Pick Any Pair of Columns
3. Create a New Column by Multiplying These
Two, Row by Row.
4. Sum the New Column; If the Sum is Zero, the
Two Columns/Factors Are Orthogonal.
5. If Every Pair of Columns is Orthogonal, the
Design is Orthogonal.
2k Designs
Randomization

It is Highly Recommended That the Trials
be Carried Out in a Randomized Order!!!
2k Designs
Randomization devices
Slips of Paper in a Bowl
 Multi-Sided Die
 Coin Flips
 Table of Random Digits
 Pseudo-Random Numbers on a Computer

2k Designs
Randomization - Why randomize order?
It’s MAE’s fault...
M=A + E
(M easured response)=
(A ctual effect of factor combination)
+ (E verything else-”random error”)

2k Designs
Randomization - Beware the convenient sample!

Randomize Run Order to Protect Against
the Unknown Factors Which are not Either
–
–

varied as experimental factors, or
fixed as background effects.
Try Hard to Determine What These
Unknown Factors Are!
2k Designs
Randomization - Instructions for Operators
Having Randomized the Run Order,
Present the Operator With Easy-to-Follow
Instructions.
 Tell Him/Her Not to "Help" by Rearranging
the Order for Convenience!

2k Designs
Randomization - Partial Randomization

In Certain Situations It May Not Be Possible to Totally
Randomize All the Runs
– e.g., it may be too costly to completely randomize the temperatures
of a series of ovens while one may be able to totally randomize the
other factor levels

This Leads to Blocks of Runs Within Which The Factor
Settings Can Be Totally Randomized
– The Analysis of Blocked Designs Will Be Discussed in a Later Module

Remember An Important Goal of a DOE is to Get Good
Data
– Randomization Protects Us From Background Sources of Variation Of
Which We May Not Be Aware
– Blocking Allows Us to Include Known But Hard to Control Sources So
That We Estimate Their Effect. We Can Then Remove Their Effect
and Analyze the other Factor Effects