Design of Experiments (DOE)
ME 470
Fall 2013 – Day 2
We will use statistics to make good design decisions!
We may be forced to run experiments to
characterize our system. We will use valid
statistical tools such as Linear Regression,
DOE, and Robust Design methods to help us
make those characterizations.
DOE is a powerful tool for analyzing and predicting
system behavior.
At the end of the DOE module, students should be able to
perform the following actions:
• Explain the advantages of designed experiments (DOE) over
one-factor-at-a-time
• Define key terminology used in experimental design
• Analyze experimental data using the DOE techniques
introduced
• Make good design decisions!!!
Be prepared to define the terms below.





Factor - A controllable experimental variable thought to influence
response (in the case of the Frisbee thrower: angle, motor speed, tire
pressure)
Response - The outcome or result; what you are measuring (distance
Frisbee goes)
Levels - Specific value of the factor (15 degrees vs. 30 degrees)
Interaction - Factors may not be independent, therefore combinations
of factors may be important. Note that these interactions can easily be
missed in a straight “hold all other variables constant” scientific
approach. If you have interaction effects you can NOT find the
global optimum using the “OFAT” (one factor at a time) approach!
Replicate – performance of the basic experiment
There are six suggested steps in DOE.
1. Statement of the Problem
2. Selection of Response Variable
3. Choice of Factors and Levels


Factors are the potential design parameters, such as angle or tire
pressure
Levels are the range of values for the factors, 15 degrees or 30
degrees
4. Choice of Design



screening tests
response prediction
factor interaction
5. Perform Experiment
6. Data Analysis
23 Factorial Design Example
#1. Problem Statement: A soft drink bottler is interested in obtaining more
uniform heights in the bottles produced by his manufacturing process. The
filling machine theoretically fills each bottle to the correct target height, but in
practice, there is variation around this target, and the bottler would like to
understand better the sources of this variability and eventually reduce it.
#2. Selection of Response Variable: Variation of height of liquid from target
#3. Choice of Factors: The process engineer can control three variables during
the filling process:
(A) Percent Carbonation
(B) Operating Pressure
(C) Line Speed
Pressure and speed are easy to control, but the percent carbonation is more
difficult to control during actual manufacturing because it varies with product
temperature. It can be controlled in a lab setting.
23 Factorial Design Example
#3. Choice of Levels – Each test will be performed for both
high and low levels
#4. Choice of Design – Interaction effects
#5. Perform Experiment
Determine what tests are required using tabular data or
Minitab
Determine the order in which the tests should be
performed
Stat>DOE>Factorial>Create Factorial Design
Full
Factorial
3
factors
Number of
replicates
Enter
Information
Ask for
random
runs
Pareto Chart of the Standardized Effects
(response is Deviation from Target, Alpha = .05)
2.306
F actor
A
B
C
A
N ame
% C arbonation
P ressure
Line S peed
B
Term
C
AB
ABC
BC
AC
0
1
2
3
4
5
Standardized Effect
6
7
8
The Pareto Chart shows the significant effects. Anything to the right of the red line
is significant at a (1-a) level. In our case a =0.05, so we are looking for significant
effects at the 0.95 or 95% confidence level. So what is significant here?
Residual Plots for Deviation from Target
Normal Probability Plot of the Residuals
Residuals Versus the Fitted Values
99
1.0
Residual
Percent
90
50
10
0.0
-0.5
-1.0
1
-1
0
Residual
1
-2
Histogram of the Residuals
6.0
1.0
4.5
0.5
3.0
1.5
0.0
0
2
Fitted Value
4
6
Residuals Versus the Order of the Data
Residual
Frequency
0.5
0.0
-0.5
-1.0
-1.0
-0.5
0.0
Residual
0.5
1.0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16
Observation Order
Estimated Effects and Coefficients for Deviation from Target (coded units)
Term
Constant
%Carbonation
Pressure
Line Speed
%Carbonation*Pressure
%Carbonation*Line Speed
Pressure*Line Speed
%Carb*Press*Line Speed
Effect
3.0000
2.2500
1.7500
0.7500
0.2500
0.5000
0.5000
Coef
1.0000
1.5000
1.1250
0.8750
0.3750
0.1250
0.2500
0.2500
SE Coef
0.1976
0.1976
0.1976
0.1976
0.1976
0.1976
0.1976
0.1976
T
5.06
7.59
5.69
4.43
1.90
0.63
1.26
1.26
P
0.001
0.000
0.000
0.002
0.094
0.545
0.242
0.242
S = 0.790569 PRESS = 20
R-Sq = 93.59% R-Sq(pred) = 74.36% R-Sq(adj) = 87.98%
We could construct an equation from this to predict Deviation from Target.
Deviation = 1.00 + 1.50*(%Carbonation) +1.125*(Pressure) + 0.875*(Line Speed) +
0.375*(%Carbonation*Pressure) + 0.125*(%Carbonation*Line Speed) +
0.250*(Pressure*Line Speed) + 0.250*(%Carbonation*Pressure*Line Speed)
We can actually get a better model, which we will discuss in a few slides.
Main Effects Plot (data means) for Deviation from Target
Pressure
%Carbonation
Mean of Deviation from Target
2
1
0
12
10
Line Speed
2
1
0
200
250
25
30
Estimated Effects and Coefficients for Deviation from Target (coded units)
Term
Constant
%Carbonation
Pressure
Line Speed
%Carbonation*Pressure
%Carbonation*Line Speed
Pressure*Line Speed
%Carb*Press*Line Speed
Effect
3.0000
2.2500
1.7500
0.7500
0.2500
0.5000
0.5000
Coef
1.0000
1.5000
1.1250
0.8750
0.3750
0.1250
0.2500
0.2500
SE Coef
0.1976
0.1976
0.1976
0.1976
0.1976
0.1976
0.1976
0.1976
T
5.06
7.59
5.69
4.43
1.90
0.63
1.26
1.26
P
0.001
0.000
0.000
0.002
0.094
0.545
0.242
0.242
S = 0.790569 PRESS = 20
R-Sq = 93.59% R-Sq(pred) = 74.36% R-Sq(adj) = 87.98%
The statisticians at Cummins suggest that you remove all terms that have a p value
greater than 0.2. This allows you to have more data to estimate the values of the
coefficients.
Here is the final model from Minitab
with the appropriate terms.
Estimated Effects and Coefficients for Deviation from Target (coded units)
Term
Constant
%Carbonation
Pressure
Line Speed
%Carbon*Press
Effect
3.0000
2.2500
1.7500
0.7500
Coef
1.0000
1.5000
1.1250
0.8750
0.3750
SE Coef
0.2030
0.2030
0.2030
0.2030
0.2030
T
4.93
7.39
5.54
4.31
1.85
P
0.000
0.000
0.000
0.001
0.092
S = 0.811844 PRESS = 15.3388
R-Sq = 90.71% R-Sq(pred) = 80.33% R-Sq(adj) = 87.33%
Deviation from Target = 1.000 + 1.5*(%Carbonation) + 1.125*(Pressure) +
0.875*(Line Speed) + 0.375*(%Carbonation*Pressure)
Estimated Effects and Coefficients for Deviation from Target (coded units).
The term coded units means that the equation uses a -1 for the low value and a
+1 for the high value of the data.
Deviation from Target = 1.000 + 1.5*(%Carbonation) + 1.125*(Pressure) + 0.875*(Line
Speed) + 0.375*(%Carbonation*Pressure)
Let’s check this for %Carbonation = 10, Pressure = 30 psi, and Line Speed = 200 BPM
%Carbonation is at its low value, so it gets a -1. Pressure is at its high value, so it gets +1,
Line Speed is at its low value, so it gets a -1.
Deviation from Target = 1.000 + 1.5*(-1) + 1.125*(1)+ 0.875*(-1)+
Deviation from Target = -0.625 tenths of an inch
How does this compare with the actual runs at those settings?
0.375*(-1*-1)
>Stat>DOE>Factorial>Response Optimizer
Now that we have our model, we can play with it to find items of interest.
Select C8 Deviation
(tenths of an inch)
Optimal
High
D
Cur
1.0000 Low
Composite
Desirability
1.0000
Deviatio
Targ: 0.0
y = 0.0
d = 1.0000
%Carbona
12.0
[10.0]
10.0
Pressure
30.0
[29.3713]
25.0
Line Spe
250.0
[223.2464]
200.0
The engineer wants
the higher line speed
and decides to put the
target slightly
negative. Why??
NEVER GIVE THIS SETTING TO
PRODUCTION UNTIL YOU HAVE
VERIFIED THE MODEL!!!
VOC
Lawn
Mower
Example
Not too Noisy
System Spec
Noise Level < 75 db
Engine Noise
Blade Assy Noise
Combustion Noise
Blade Speed
Muffler Noise
Blade Area
Muffler Volume
Hole Area
Diameter
Blade Width
Blade Length
Grass Height
Blade to Hsg
Clearance
Let’s revisit the Frisbee Thrower and see what the
data shows us.
Pareto Chart of the Standardized Effects
(response is Distance (ft), Alpha = 0.05)
2.306
F actor
A
B
C
C
N ame
S peed %
Tire P ressure (psi)
A ngle (Degrees)
A
AC
Residual Plots for Distance (ft)
B
Normal Probability Plot
ABC
Versus Fits
99
5.0
1
2
3
4
5
6
Standardized Effect
7
8
9
Residual
0
50
10
1
2.5
0.0
-2.5
-5.0
-5.0
-2.5
0.0
Residual
2.5
5.0
40
50
Histogram
5.0
1.5
2.5
1.0
0.5
0.0
60
Fitted Value
70
Versus Order
2.0
Residual
BC
Percent
90
Frequency
Term
AB
0.0
-2.5
-5.0
-4
-2
0
Residual
2
4
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16
Observation Order
Factorial Fit: Distance (ft versus Speed %, Tire
Pressure, Angle (Degrees)
Estimated Effects and Coefficients for Distance (ft) (coded units)
Term
Constant
Speed %
Tire Pressure (psi)
Angle (Degrees)
Speed %*Tire Pressure (psi)
Speed %*Angle (Degrees)
Tire Pressure (psi)*Angle (Degrees)
Speed %*Tire Pressure (psi)*
Angle (Degrees)
S = 3.84171
R-Sq = 92.02%
PRESS = 472.28
R-Sq(pred) = 68.09%
Effect
6.000
-3.125
-15.650
-5.225
-4.400
0.075
1.775
Coef
51.775
3.000
-1.562
-7.825
-2.612
-2.200
0.037
0.887
SE Coef
0.9604
0.9604
0.9604
0.9604
0.9604
0.9604
0.9604
0.9604
R-Sq(adj) = 85.04%
T
53.91
3.12
-1.63
-8.15
-2.72
-2.29
0.04
0.92
P
0.000
0.014
0.142
0.000
0.026
0.051
0.970
0.382
Following recommended procedures, we achieved a reduced model.
Pareto Chart of the Standardized Effects
(response is Distance (ft), Alpha = 0.05)
2.228
F actor
A
B
C
C
N ame
S peed %
Tire P ressure (psi)
A ngle (Degrees)
AB
Residual Plots for Distance (ft)
Normal Probability Plot
AC
Versus Fits
99
5.0
Percent
0
1
2
3
4
5
6
Standardized Effect
50
10
8
7
1
-8
2.5
0.0
-2.5
9
-4
0
Residual
4
-5.0
8
40
50
60
Fitted Value
Histogram
5.0
3
2
1
0
70
Versus Order
4
Residual
B
Residual
90
Frequency
Term
A
2.5
0.0
-2.5
-4
-2
0
2
Residual
4
6
-5.0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16
Observation Order
Let’s look at both main effects and interaction effects.
STAT >DOE>FACTORIAL>FACTORIAL PLOTS
Main Effects Plot for Distance (ft)
Data Means
Speed %
60
Tire Pressure (psi)
55
50
Mean
45
50
100
30
45
Angle (Degrees)
60
55
50
45
Interaction Plot for Distance (ft)
15
Data Means
30
30
45
15
30
60
Speed %
Speed %
50
100
50
40
60
T ir e P r essur e ( psi)
50
40
A ngle ( Degr ees)
Tire
Pressure
(psi)
30
45