Chapter 6
Chapter 6
1
Design & Analysis of Experiments
8E 2012 Montgomery
3
“-” and “+” denote the low and
high levels of a factor,
respectively
• Low and high are arbitrary
terms
• Geometrically, the four runs
form the corners of a square
• Factors can be quantitative or
qualitative, although their
treatment in the final model
will be different
The Simplest Case: The 22
Design & Analysis of Experiments
8E 2012 Montgomery
• Text reference, Chapter 6
• Special case of the general factorial design; k factors,
all at two levels
• The two levels are usually called low and high (they
could be either quantitative or qualitative)
• Very widely used in industrial experimentation
• Form a basic “building block” for other very useful
experimental designs (DNA)
• Special (short-cut) methods for analysis
• We will make use of Design-Expert
Design of Engineering Experiments
Part 5 – The 2k Factorial Design
Chemical Process Example
Design & Analysis of Experiments
8E 2012 Montgomery
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
A = reactant concentration, B = catalyst amount,
y = recovery
Chapter 6
4
2
Chapter 6
Model
Model
Model
Model
Error
Error
6.15809
7.95671
Design & Analysis of Experiments
8E 2012 Montgomery
Lenth's ME
Lenth's SME
Term
Effect
SumSqr
% Contribution
Intercept
A
8.33333
208.333
64.4995
B
-5
75
23.2198
AB
1.66667
8.33333
2.57998
Lack Of Fit 0
0
P Error
31.3333
9.70072
Estimation of Factor Effects
Form Tentative Model
Design & Analysis of Experiments
8E 2012 Montgomery
Statistical testing (ANOVA)
Refine the model
Analyze residuals (graphical)
Interpret results
Chapter 6
•
•
•
•
– With replication, use full model
– With an unreplicated design, use normal probability
plots
• Estimate factor effects
• Formulate model
Analysis Procedure for a
Factorial Design
7
5
Design-Expert analysis
Practical interpretation?
Statistical Testing - ANOVA
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
8
6
The effect estimates are:
A
= 8.33, B = -5.00, AB = 1.67
See textbook, pg. 235-236 for
manual calculations
Design & Analysis of Experiments
8E 2012 Montgomery
ab (1) a b
2n
2n
1
2 n [ ab (1) a b]
ab b a (1)
2n
2n
1
2 n [ ab b a (1)]
yB yB ab a b (1)
2n
2n
1
2 n [ ab a b (1)]
y A y A
The F-test for the “model” source is testing the significance of the
overall model; that is, is either A, B, or AB or some combination of
these effects important?
Chapter 6
AB
B
A
Estimation of Factor Effects
Design & Analysis of Experiments
8E 2012 Montgomery
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
Residuals and Diagnostic Checking
Chapter 6
11
9
Design-Expert output, full model
Chapter 6
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
The Response Surface
Design & Analysis of Experiments
8E 2012 Montgomery
Design-Expert output, edited
or reduced model
12
10
Design & Analysis of Experiments
8E 2012 Montgomery
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
A = gap, B = Flow, C = Power, y = Etch Rate
An Example of a 23 Factorial Design
Chapter 6
The 23 Factorial Design
15
13
Chapter 6
Chapter 6
yC yC C
Analysis
done via
computer
Design & Analysis of Experiments
8E 2012 Montgomery
Table of – and + Signs for the 23 Factorial Design (pg. 218)
Design & Analysis of Experiments
8E 2012 Montgomery
yB yB B
etc, etc, ...
y A y A A
Effects in The 23 Factorial Design
16
14
AB
AB 2C
AC
Design & Analysis of Experiments
8E 2012 Montgomery
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
ANOVA Summary – Full Model
Chapter 6
• Orthogonal design
• Orthogonality is an important property shared by all factorial designs
AB u BC
Au B
• Except for column I, every column has an equal number of + and –
signs
• The sum of the product of signs in any two columns is zero
• Multiplying any column by I leaves that column unchanged (identity
element)
• The product of any two columns yields a column in the table:
Properties of the Table
19
17
Design & Analysis of Experiments
8E 2012 Montgomery
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
Model Coefficients – Full Model
Chapter 6
Estimation of Factor Effects
20
18
1
1
20857.75 /12
5.314 u105 /15
5.106 u105
5.314 u105
SS E / df E
SST / dfT
SS Model
SST
0.9509
21
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
23
• R2 for prediction (based on PRESS)
PRESS
37080.44
2
RPred
1
1
0.9302
SST
5.314 u 105
2
RAdj
R
2
0.9608
Model Summary Statistics for Reduced Model
Design & Analysis of Experiments
8E 2012 Montgomery
• R2 and adjusted R2
Chapter 6
Refine Model – Remove Nonsignificant Factors
Model Summary Statistics
Design & Analysis of Experiments
8E 2012 Montgomery
V ( Eˆ )
n2
k
V2
MS E
n2k
2252.56
2(8)
11.87
Chapter 6
E
Design & Analysis of Experiments
8E 2012 Montgomery
E
Eˆ tD / 2,df se( Eˆ ) d E d Eˆ tD / 2,df se( Eˆ )
• Confidence interval on model coefficients
se( Eˆ )
• Standard error of model coefficients (full
model)
Chapter 6
Model Coefficients – Reduced Model
24
22
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
Design & Analysis of Experiments
8E 2012 Montgomery
Cube Plot of Ranges
What do the
large ranges
when gap and
power are at the
high level tell
you?
Chapter 6
The Regression Model
27
25
Chapter 6
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
Design & Analysis of Experiments
8E 2012 Montgomery
28
26
Cube plots are
often useful visual
displays of
experimental
results
Model Interpretation
Spacing of Factor Levels in the
Unreplicated 2k Factorial Designs
Design & Analysis of Experiments
8E 2012 Montgomery
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
More aggressive spacing is usually best
If the factors are spaced too closely, it increases the chances
that the noise will overwhelm the signal in the data
Chapter 6
1 k factor interaction
§k ·
¨ ¸ two-factor interactions
© 2¹
§k ·
¨ ¸ three-factor interactions
© 3¹
• Section 6-4, pg. 253, Table 6-9, pg. 25
• There will be k main effects, and
The General 2k Factorial Design
31
29
Design & Analysis of Experiments
8E 2012 Montgomery
30
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
32
– Pooling high-order interactions to estimate error
– Normal probability plotting of effects (Daniels, 1959)
– Other methods…see text
• Potential solutions to this problem
– Replication admits an estimate of “pure error” (a better
phrase is an internal estimate of error)
– With no replication, fitting the full model results in zero
degrees of freedom for error
• Lack of replication causes potential problems in
statistical testing
Unreplicated 2k Factorial Designs
Chapter 6
• These are 2k factorial designs with one
observation at each corner of the “cube”
• An unreplicated 2k factorial design is also
sometimes called a “single replicate” of the 2k
• These designs are very widely used
• Risks…if there is only one observation at each
corner, is there a chance of unusual response
observations spoiling the results?
• Modeling “noise”?
6.5 Unreplicated 2k Factorial Designs
Design
Chapter 6
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
The Resin Plant Experiment
Design & Analysis of Experiments
8E 2012 Montgomery
35
33
• A 24 factorial was used to investigate the
effects of four factors on the filtration rate of a
resin
• The factors are A = temperature, B = pressure,
C = mole ratio, D= stirring rate
• Experiment was performed in a pilot plant
Example of an Unreplicated
2k
Chapter 6
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
Design & Analysis of Experiments
8E 2012 Montgomery
The Resin Plant Experiment
36
34
Design & Analysis of Experiments
8E 2012 Montgomery
37
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
39
Design Projection: ANOVA Summary for
the Model as a 23 in Factors A, C, and D
Chapter 6
Estimates of the Effects
Chapter 6
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
The Regression Model
Design & Analysis of Experiments
8E 2012 Montgomery
The Half-Normal Probability Plot of Effects
40
38
Design & Analysis of Experiments
8E 2012 Montgomery
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
With concentration at either the low or high level, high temperature and
high stirring rate results in high filtration rates
Model Interpretation – Response
Surface Plots
Chapter 6
Model Residuals are Satisfactory
43
41
Chapter 6
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
Design & Analysis of Experiments
8E 2012 Montgomery
Model Interpretation – Main Effects
and Interactions
44
42
Dealing with Outliers
Design & Analysis of Experiments
8E 2012 Montgomery
45
Design & Analysis of Experiments
8E 2012 Montgomery
47
Replace with an estimate
Make the highest-order interaction zero
In this case, estimate cd such that ABCD = 0
Analyze only the data you have
Now the design isn’t orthogonal
Consequences?
Chapter 6
•
•
•
•
•
•
Chapter 6
Chapter 6
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
Design & Analysis of Experiments
8E 2012 Montgomery
48
46
Outliers: suppose that cd = 375 (instead of 75)
Chapter 6
Chapter 6
DESIGN-EXPERT Plot
adv._rate
1.69
4.70
P re d ic te d
7.70
10.71
13.71
R e s id ua ls vs . P re d ic te d
Design & Analysis of Experiments
8E 2012 Montgomery
- 1.963755
- 0.826255
0.311255
1.448755
2.586255
Residual Plots
Design & Analysis of Experiments
8E 2012 Montgomery
A = drill load, B = flow, C = speed, D = type of mud,
y = advance rate of the drill
The Drilling Experiment
Example 6.3
R e s id u a ls
51
49
Residual Plots
Design & Analysis of Experiments
8E 2012 Montgomery
50
yO
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
– Stabilize variance
– Induce at least approximate normality
– Simplify the model
• Transformations are typically performed to
y*
52
• The residual plots indicate that there are problems
with the equality of variance assumption
• The usual approach to this problem is to employ a
transformation on the response
• Power family transformations are widely used
Chapter 6
Normal Probability Plot of Effects –
The Drilling Experiment
Chapter 6
Recommend transform:
Log
(Lambda = 0)
Lambda
Current = 1
Best = -0.23
Low C.I. = -0.79
High C.I. = 0.32
DESIGN-EXPERT Plot
adv._rate
Chapter 6
1.05
2.50
3.95
5.40
6.85
-3
-2
1
Lam bda
0
2
3
Design & Analysis of Experiments
8E 2012 Montgomery
-1
B o x-C o x P lo t fo r P o w e r T ra ns fo rm
53
55
If unity is included in the
confidence interval, no
transformation would be
needed
The procedure provides a
confidence interval on
the transformation
parameter lambda
A log transformation is
recommended
The Box-Cox Method
Design & Analysis of Experiments
8E 2012 Montgomery
• Empirical selection of lambda
• Prior (theoretical) knowledge or experience can
often suggest the form of a transformation
• Analytical selection of lambda…the Box-Cox
(1964) method (simultaneously estimates the
model parameters and the transformation
parameter lambda)
• Box-Cox method implemented in Design-Expert
Selecting a Transformation
L n (R e s id u a lS S )
Chapter 6
Chapter 6
54
Design & Analysis of Experiments
8E 2012 Montgomery
56
What happened to the
interactions?
No indication of large
interaction effects
Three main effects are
large
Effect Estimates Following the
Log Transformation
Design & Analysis of Experiments
8E 2012 Montgomery
(15.1)
The Log Advance Rate Model
Design & Analysis of Experiments
8E 2012 Montgomery
57
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
59
• Is the log model “better”?
• We would generally prefer a simpler model
in a transformed scale to a more
complicated model in the original metric
• What happened to the interactions?
• Sometimes transformations provide insight
into the underlying mechanism
Chapter 6
ANOVA Following the Log Transformation
Other Examples of
Unreplicated 2k Designs
Design & Analysis of Experiments
8E 2012 Montgomery
58
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
– Replicates versus repeat (or duplicate) observations?
– Modeling within-run variability
60
• The oxidation furnace experiment (Example 6.5, pg.
245)
– Two factors affect the mean number of defects
– A third factor affects variability
– Residual plots were useful in identifying the dispersion
effect
• The sidewall panel experiment (Example 6.4, pg. 274)
Chapter 6
Following the Log Transformation
Design & Analysis of Experiments
8E 2012 Montgomery
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
For an individual contrast, compare to the margin of error
Overview of Lenth’s method
Chapter 6
– Using DOX in marketing and marketing
research, a growing application
– Analysis is with the JMP screening platform
• Example 6.6, Credit Card Marketing, page
278
63
61
Chapter 6
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
Design & Analysis of Experiments
8E 2012 Montgomery
• Conditional inference charts (pg. 264)
– Analytical method for testing effects, uses an estimate
of error formed by pooling small contrasts
– Some adjustment to the critical values in the original
method can be helpful
– Probably most useful as a supplement to the normal
probability plot
• Lenth’s method (see text, pg. 262)
Other Analysis Methods for
Unreplicated 2k Designs
64
62
Design & Analysis of Experiments
8E 2012 Montgomery
E 0 E1 x1 E 2 x2 E12 x1 x2 H
Chapter 6
y = Xȕ İ, \
(1)
ª (1) º
«a»
« »,;
«b»
« »
¬ ab ¼
Design & Analysis of Experiments
8E 2012 Montgomery
ª1 1 1 1 º
«1 1 1 1»
«
» ,ȕ
«1 1 1 1»
«
»
¬1 1 1 1 ¼
E 0 E1 (1) E 2 (1) E12 (1)(1) H1
a E 0 E1 (1) E 2 (1) E12 (1)(1) H 2
b E 0 E1 (1) E 2 (1) E12 (1)(1) H 3
ab E 0 E1 (1) E 2 (1) E12 (1)(1) H 4
y
ª E0 º
«E »
« 1 » ,İ
« E2 »
« »
¬ E12 ¼
ª H1 º
«H »
« 2»
«H 3 »
« »
¬H 4 ¼
The four
observations
from a 22 design
The model parameter estimates in a 2k design (and the effect estimates) are
least squares estimates. For example, for a 22 design the model is
The 2k design and design optimality
Chapter 6
JMP has an excellent implementation of Lenth’s method
in the screening platform
Lenth’s method is a nice supplement to the normal
probability plot of effects
Simulation was used to find these adjusted multipliers
Suggested because the original method makes too many
type I errors, especially for small designs (few contrasts)
Adjusted multipliers for Lenth’s method
67
65
0º
0 »»
0»
»
4¼
1
ª4
«0
«
«0
«
¬0
0
4
0
0
0
0
4
0
Design & Analysis of Experiments
8E 2012 Montgomery
ª(1) a b ab º
« a ab b (1) »
«
»
«b ab a (1) »
«
»
¬ (1) a b ab ¼
ª (1) a b ab º
«
»
4
«
»
(1)
a
b
ab
ª
º
a ab b (1) »
«
«
»
»
1 « a ab b (1) » «
4
I4
«
»
4 «b ab a (1) » « b ab a (1) »
«
»
4
»
¬ (1) a b ab ¼ «
« (1) a b ab »
«
»
¬
¼
4
;c;-1 ;c\
Chapter 6
ª Eˆ0 º
« »
« Eˆ1 »
« ˆ »
« E2 »
«ˆ »
¬ E12 ¼
ȕˆ
66
68
The regression
coefficient estimates
are exactly half of the
‘usual” effect estimates
The XcX matrix is
diagonal –
consequences of an
orthogonal design
The “usual” contrasts
Design & Analysis of Experiments
8E 2012 Montgomery
The least squares estimate of ȕ is
Chapter 6
I
Design & Analysis of Experiments
8E 2012 Montgomery
Chapter 6
4V 2
9
Design & Analysis of Experiments
8E 2012 Montgomery
71
A = area of design space = 22
1
1
V 2 (1 x12 x22 x12 x22 )dx1dx2
³
³
4 1 1 4
1 1
1
V [ yˆ ( x1 , x2 )dx1dx2
A ³1 ³1
1 1
Average prediction variance
Chapter 6
What about predicting the response?
Notice that these results depend on both the design that you
have chosen and the model
69
4
V [ yˆ ( x1 , x2 )] V 2
|(XcX) | 256
(1 x12 x22 x12 x22 )
x2
0 is
r1, x2
r1
Chapter 6
Min StdErr Mean: 0.500
Max StdErr Mean: 1.000
Cuboidal
radius = 1
Points = 10000
Design-Expert® Software
Chapter 6
0.00
0.50
0.75
Fraction of Design Space
0.25
FDS Graph
Design & Analysis of Experiments
8E 2012 Montgomery
0.000
0.250
0.500
0.750
1.000
Design & Analysis of Experiments
8E 2012 Montgomery
1.00
72
70
4
What about average prediction variance over the design space?
V [ yˆ ( x1 , x2 )]
V2
The prediction variance when x1
4
The maximum prediction variance occurs when x1
Maximum possible
value for a four-run
design
4
V [ yˆ ( x1 , x2 )]
Minimum possible
value for a four-run
design
V2
V [ yˆ ( x1 , x2 )] V 2 xc(XcX)-1 x
xc [1, x1 , x2 , x1 x2 ]
V2
V ( Eˆ ) V 2 (diagonal element of (XcX)1 )
The matrix XcX has interesting and useful properties:
StdErr Mean
Addition of Center Points
to a 2k Designs
Design & Analysis of Experiments
8E 2012 Montgomery
73
Chapter 6
k
k
k
k
i 1 j !i
k
i 1
i 1 j !i
i 1
75
E 0 ¦ E i xi ¦¦ E ij xi x j ¦ E ii xi2 H
k
i 1
E 0 ¦ E i xi ¦¦ E ij xi x j H
Design & Analysis of Experiments
8E 2012 Montgomery
Second-order model y
k
First-order model (interaction) y
• Based on the idea of replicating some of the
runs in a factorial design
• Runs at the center provide an estimate of
error and allow the experimenter to
distinguish between two possible models:
Chapter 6
• The design produces regression model coefficients that
have the smallest variances (D-optimal design)
• The design results in minimizing the maximum
variance of the predicted response over the design space
(G-optimal design)
• The design results in minimizing the average variance
of the predicted response over the design space (Ioptimal design)
For the 22 and in general the 2k
Chapter 6
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
Design & Analysis of Experiments
8E 2012 Montgomery
76
74
• These results give us some assurance that
these designs are “good” designs in some
general ways
• Factorial designs typically share some (most)
of these properties
• There are excellent computer routines for
finding optimal designs (JMP is outstanding)
Optimal Designs
k
Design & Analysis of Experiments
8E 2012 Montgomery
Design & Analysis of Experiments
8E 2012 Montgomery
Chapter 6
Chapter 6
79
77
This sum of squares has a
single degree of freedom
SS Pure Quad
nF nC ( yF yC ) 2
nF nC
i 1
H1 : ¦ E ii z 0
k
i 1
H 0 : ¦ E ii
0
yC Ÿ no "curvature"
The hypotheses are:
yF
Chapter 6
Chapter 6
nC
4
Design & Analysis of Experiments
8E 2012 Montgomery
ANOVA for Example 6.7
80
78
Design-Expert
provides the analysis,
including the F-test
for pure quadratic
curvature
Usually between 3
and 6 center points
will work well
Design & Analysis of Experiments
8E 2012 Montgomery
Refer to the original experiment
shown in Table 6.10. Suppose that
four center points are added to this
experiment, and at the points x1=x2
=x3=x4=0 the four observed
filtration rates were 73, 75, 66, and
69. The average of these four center
points is 70.75, and the average of
the 16 factorial runs is 70.06.
Since are very similar, we suspect
that there is no strong curvature
present.
Example 6.7, Pg. 286
Design & Analysis of Experiments
8E 2012 Montgomery
81
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
83
Center Points and Qualitative Factors
Chapter 6
If curvature is significant, augment the design with axial runs to
create a central composite design. The CCD is a very effective design
for fitting a second-order response surface model
Chapter 6
Design & Analysis of Experiments
8E 2012 Montgomery
• Use current operating conditions as the center
point
• Check for “abnormal” conditions during the
time the experiment was conducted
• Check for time trends
• Use center points as the first few runs when there
is little or no information available about the
magnitude of error
• Center points and qualitative factors?
Practical Use of Center Points (pg. 289)
82