Statistics and DOE
ME 470
Spring 2012
We will use statistics to make
good design decisions!
We will categorize populations by the mean,
standard deviation, and use control charts to
determine if a process is in control.
We may be forced to run experiments to
characterize our system. We will use valid
statistical tools such as Linear Regression,
DOE, and Robust Design methods to help us
make those characterizations.
What is DOE
Sometimes we would like to model or optimize a
design or process, but the “first principles”
analytic approach (Conservation laws, etc.) is
too challenging.
Empirical (experimental) models are the best
next choice. The organized approach using DOE
provides a good modeling tool for optimization
of designs and processes
Example
Suppose you are trying to get faster cycle time on
an injection molder and are looking for the best
settings for several temperatures and pressures.
The transient heat transfer and transient fluid
mechanics (with changing viscosity) problem is
challenging. Instead we change the inputs in an
organized way, measure the outputs, and develop
an experimental model that can be optimized.
Here is an email that made my day!
I'm working on a project that is nearing data collection. The study is focused on
Thumb-Tip force resulting from muscle/tendon force. We're working with cadaveric
specimens so this is awesome lab work. Some of the relationships are expected to be
nonlinear so we're looking 10 levels of loading for each tendon. We also wish to
document first order and possible second order interactions between tendons if they are
significant.
Last year with the human powered vehicle team we used minitab to create a test
procedure for testing power output resulting from chain ring shape, crank length, and
rider. There were 3 chain ring shapes, 3 different crank lengths. If possible, in this
current study I would like to run a preliminary factorial experiments to determine
which interactions are significant before exhaustively testing every combination at
every level of loading. If such a method is appropriate it could save us a lot of
time. Could you suggest a reference that I might be able to find at the library or on
amazon?
DOE Objectives
At the end of this module, the participant will be
able to :




Understand the basic concepts and advantages of
designed experiments
Understand key terminology used in experimental
design
Use different techniques to deal with noise in an
experiment
Make good design decisions!!!
Statistical Design of Experiments

Chapter 13 of your text. Your text only looks at DOE in
the context of Robust Design. This is one of several
important uses of DOE:




Determining Significant Factors
Constructing a Model of System Performance
Design and Analysis of Experiments, Montgomery and
Hines is an excellent reference.
Look at the Frisbee Thrower from Catapult
What Experiments Can Do


Characterize a Process/Product
 determines which X’s most affect the Y’s
 includes controllable and uncontrollable X’s
 identifies critical X’s and noise variables
 identifies those variables that need to be carefully
controlled
 provides direction for controlling X’s rather than control
charting the Y’s
Optimize a Process/Product
 determines where the critical X’s should be set
 determines “real” specification limits
 provides direction for “robust” designs
Definition of Terms





Factor - A controllable experimental variable thought to influence
response (example air flow rate, or in the case of the Frisbee thrower:
angle, tire speed, tire pressure)
Response - The outcome or result; what you are measuring (cycle time
to produce one bottle, distance Frisbee goes)
Levels - Specific value of the factor (fast flow vs. slow flow, 15
degrees vs. 30 degrees)
Interaction - Factors may not be independent, therefore combinations
of factors may be important. Note that these interactions can easily be
missed in a straight “hold all other variables constant” scientific
approach. If you have interaction effects you can NOT find the
global optimum using the “OFAT” (one factor at a time) approach!
Replicate – performance of the basic experiment
How Can DOE Help?





run a relatively small number of tests to isolate the
most important factors (screening test).
determine if any of the factors interact (combined
effects are as important as individual effects) and
the level of interaction.
predict response for any combination of factors
using only empirical results
optimize using only empirical results
determine the design space for simulation models
Trial and Error

Problem : Current gas mileage is 23 mpg. Want 30
mpg. But what is the best we can get?
We might do the following :







Change brand of gas
Change octane rating
Drive slower
Tune-up car
Wash and wax car
Buy new tires
Change tire pressure
Terminology :
response variable / dependent variable - what is being measured/optimized
ex. gas mileage
factor / main effect / independent variable - a controlled variable being
studied at 2 or more levels during the experiment ex. brand of gas
From the previous example, here were some potential factors
to investigate (factors can be either continuous or discrete) :
brand of gas (discrete)
octane rating (continuous)
driving speed (continuous)
tires (discrete)
tire pressure (continuous)
Terminology :
factor levels / main effect levels - the values or settings of
the variable being manipulated
ex. levels for brand of gas are Shell & Texaco
ex. levels for speed are 55 & 60
ex. levels for octane are 85 & 90
One Factor at a Time
Problem: Gas mileage is 23 mpg
Baseline =>




Speed
55
60
55
55
Octane
85
85
90
85
Tire Pressure
30
30
30
35
Y = MPG
23
24
22
20
Can you explain the results?
How many more runs would you need to figure out the
best combination of variables?
If there were more variables, how many runs would it
take to get an optimized solution?
What if there is a specific combination of two or more
variables that leads to the best mpg?
Full Factorial Experiment
Problem: Gas Mileage is 23 mpg
Speed
55
60
55
60
55
60
55
60
Octane
85
85
90
90
85
85
90
90
Tire Pressure
30
30
30
30
35
35
35
35
Y = MPG
23
24
22
28
20
21
27
25
OFAT Runs
What conclusion do you make now?
Full Factorial Experiment



Full Factorial experiment consists of all possible combinations of the
levels of the factors
Design Matrix is the complete specification of the experimental test runs,
as seen in the example below
Treatment Combination is a specific test run set-up, consisting of a
specific combination of the factor levels
Speed
55
60
55
60
55
60
55
60
Octane
85
85
90
90
85
85
90
90
Tire Pressure
30
30
30
30
35
35
35
35
Design
Matrix
A treatment
combination
What makes up an experiment?
Response Variable(s)
 Factors
 Randomization
 Repetition and Replication

Response Variable




The variable that is measured and the object of the
characterization or optimization (the Y)
Defining the response variable can be difficult
Often selected due to ease of measurement
Some questions to ask :
 How will the results be quantified/analyzed?
 How good is the measurement system?
 What are the baseline mean and standard deviation?
 How big of a change do we care about?
 Are there several response variables of interest?
Factor




A variable which is controlled or varied in a systematic way
during the experiment (the X)
Tested at 2 or more levels to observe its effect on the
response variable(s) (Ys)
Some questions to ask :
 what are reasonable ranges to ensure a change in Y?
 knowledge of relationship, i.e. linear or quadratic, etc?
Examples
 material, supplier, EGR rate, injection timing
 can you think of others?
Randomization


Randomization can be done in several ways :
 run the treatment combinations in random order
 assign experimental units to treatment combinations
randomly
 an experimental unit is the entity to which a specific
treatment combination is applied
Advantage of randomization is to “average out” the effects of
extraneous factors (called noise) that may be present but were
not controlled or measured during the experiment
 spread the effect of the noise across all runs
 these extraneous factors (noise) cause unexplained
variation in the response variable(s)
Repetition and Replication




Repetition : Running several samples during one
experimental setup (short-term variability)
Replication : Repeating the entire experiment (long-term
variability)
You can use both in the same experiment
Repetition and Replication provide an estimate of the
experimental error
 this estimate will be used to determine whether observed
differences are statistically significant
Pressure :
HHHH LLLL HHHH LLLL HHHH LLLL
Temp:
HHLL HHLL HHLL HHLL HHLL HHLL
130
Yield
120
Repetition
110
100
In d e x
Test Sequence
Replication
1st
Replicate
Pressure :
HHHH LLLL HHHH LLLL HHHH LLLL
Temp:
HHLL HHLL HHLL HHLL HHLL HHLL
130
Yield
120
2nd
110
Replicate
100
In d e x
Test Sequence
3rd
Replicate
Steps in DOE
1. Statement of the Problem
2. Selection of Response Variable
3. Choice of Factors and Levels


Factors are the potential design parameters, such as angle or tire
pressure
Levels are the range of values for the factors, 15 degrees or 30
degrees
4. Choice of Design



screening tests
response prediction
factor interaction
5. Perform Experiment
6. Data Analysis
23 Factorial Design Example


Problem Statement: A soft drink bottler is interested in obtaining more
uniform heights in the bottles produced by his manufacturing process.
The filling machine theoretically fills each bottle to the correct target
height, but in practice, there is variation around this target, and the
bottler would like to understand better the sources of this variability
and eventually reduce it.
Choice of Factors: The process engineer can control three variables
during the filling process:




(A) Percent Carbonation
(B) Operating Pressure
(C) Line Speed
Pressure and speed are easy to control, but the percent carbonation is
more difficult to control during actual manufacturing because it varies
with product temperature.
23 Factorial Design Example




Choice of Levels – Each test will be performed for both
high and low levels
Selection of Response Variable – Variation of height of
liquid from target
Choice of Design – Interaction effects
Perform Experiment
 Determine what tests are required using tabular data
 Determine the order in which the tests should be
performed
Determine which experiments
should be performed
Run
1
2
3
4
5
6
7
8
A(% C)
B(Pressure)
C(Speed)
Treatment
combinations
+
+
+
+
+
+
+
+
+
+
+
+
(1)
a
b
ab
c
ac
bc
abc
Determine Order of Experiments






Decided to run two replicates
Requires 16 tests
Put 16 numbers in a hat and draw out the numbers
in a random order
Assume that the number 7 is pulled out first, then
run test 7 first. (% C low, Pressure high, line speed
high)
What happens when you draw a 10?
Minitab can do this for you automatically!!
Stat>DOE>Factorial>Create Factorial Design
Full Factorial
Number of
Factors
Number of
Replicates
Data for the Fill Height Problem
(Average deviation from target in tenths of an inch)
Operating Pressure (B)
25 psi
Line speed (C)
30 psi
Line Speed (C)
(A) Percent
Carbonation
200
(bpm)
250
(bpm)
200
(bpm)
250
(bpm)
10
-3
-1
0
1
-1
0
2
1
-1
0
2
3
1
1
6
5
12
Enter Information
Ask for
random runs
Data Gathered on Each Run
DOE Run
1
2
3
4
5
6
7
8
(1)
(a)
(b)
(ab)
(c)
(ac)
(bc)
(abc)
A
+
+
+
+
B
+
+
+
+
C
+
+
+
+
Low, Low, Low
Data
-3,-1
0,1
-1,0
2,3
-1,0
2,1
1,1
6,5
Algebraic Signs for Calculating
I now know how this one
Effects
is done!
Treatment
Combination
Factorial Effect
(1)
A
B
AB
C
AC
BC
ABC
(1)
+
-
-
+
-
+
+
-
a
+
+
-
-
-
-
+
+
b
+
-
+
-
-
+
-
+
ab
+
+
+
+
-
-
-
-
c
+
-
-
+
+
-
-
+
ac
+
+
-
-
+
+
-
-
bc
+
-
+
-
+
-
+
-
abc
+
+
+
+
+
+
+
+
Calculate Contrast for A
Procedure
1.
2.
3.
Add all runs where A
is positive.
Subtract all runs
where A is negative.
The difference is
called the contrast.
Contrast   AHigh   ALow
Definition of terms in formulas
A,B,C are factors.
 a, b, c are the levels of factors A, B, C. In
our example we are only testing high and
low, so a = b = c = 2.
 n = the number of replicates which happens
to be 2 in our example

The effect is significant if Fo is greater
than the value from the table.
ANOVA TABLE
Source of
variation
A
Sum of
squares
Degrees of
freedom
(contrastA ) 2
SSA 
8n
SSA
DOFA
MS A
MS E
Num = DOFA
Den = DOFMSE
SSB
DOFB
MS B
MS E
Num = DOFB
Den = DOFMSE
(c-1)
MSC 
SSC
DOFC
MS C
MS E
Num = DOFC
Den = DOFMSE
(contrastAB ) 2
8n
(a-1)(b-1)
MS AB 
SSAB
DOFAB
MS AB
MS E
Num = DOFAB
Den = DOFMSE
SSAC 
(contrastAC ) 2
8n
(a-1)(c-1)
MS AC 
SSAC
DOFAC
MS AC
MS E
Num = DOFAC
Den = DOFMSE
SSBC 
(contrastBC ) 2
8n
(b-1)(c-1)
MS BC 
SSABC 
(contrastABC ) 2
8n
(a-1)(b-1)(c-1)
MS ABC 
abc(n-1)
MS E 
(contrastB )
8n
C
SSC 
(contrastC ) 2
8n
AB
SSAB 
AC
BC
Total
Who cares?
MS B 
SSB 
Error
MS A 
Fo
(b-1)
2
B
ABC
(a-1)
Mean
Square
SSE  SST  SSEffects
SS
T
abcn-1
SSBC
DOFBC
SSABC
DOFABC
SSE
DOFE
MS BC
MS E
Num = DOFBC
Den = DOFMSE
MS ABC
MSE
Num = DOFABC
Den = DOFMSE
a
b
c
n
SST   y
i 1 j 1 k 1 l 1
a
b
c
2
ijkl
2
...
y

abcn
n
y...   yijkl
i 1 j 1 k 1 l 1
In English, y… is the sum of all data points. So SST is the (sum of
the square of each data point) - (sum of all data points)2/(abcn)
Carbonation Example
Source of
variation
Sum of
squares
Degrees of
freedom
Mean
Square
Fo
Who cares?
% Carbon.
(A)
36.00
1
36.00
57.60
5.32 for
a=.05 (95%)
Pressure (B)
20.25
1
20.25
32.40
11.26 for
99%
Line speed ©
12.25
1
12.25
19.60
AB
2.25
1
2.25
3.6
AC
0.25
1
0.25
0.4
BC
1.00
1
1.00
1.6
ABC
1.00
1
1.00
1.6
Error
5.00
8
0.625
Total
78.00
15
I don’t want to do all of the math
• Can I get Minitab to do it for me?
>Stat>ANOVA>General Linear Model
Select the response by double clicking
Select model
terms by double
clicking or by
typing label,
c5*c6
>Stat>DOE>Factorial>Analyze Factorial Design
Select “Deviation”
As Response
Select the Graphs tab to get
the next screen
Pareto Chart of the Standardized Effects
(response is Deviation from Target, Alpha = .05)
2.306
F actor
A
B
C
A
N ame
% C arbonation
P ressure
Line S peed
B
Term
C
AB
ABC
BC
AC
0
1
2
3
4
5
Standardized Effect
6
7
8
The Pareto Chart shows the significant effects. Anything to the right of the red line
is significant at a (1-a) level. In our case a 0.05, so we are looking for significant
effects at the 0.95 or 95% confidence level. So what is significant here?
Residual Plots for Deviation from Target
Normal Probability Plot of the Residuals
Residuals Versus the Fitted Values
99
1.0
Residual
Percent
90
50
10
0.0
-0.5
-1.0
1
-1
0
Residual
1
-2
Histogram of the Residuals
6.0
1.0
4.5
0.5
3.0
1.5
0.0
0
2
Fitted Value
4
6
Residuals Versus the Order of the Data
Residual
Frequency
0.5
0.0
-0.5
-1.0
-1.0
-0.5
0.0
Residual
0.5
1.0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16
Observation Order
Estimated Effects and Coefficients for Deviation from Target (coded units)
Term
Constant
%Carbonation
Pressure
Line Speed
%Carbonation*Pressure
%Carbonation*Line Speed
Pressure*Line Speed
%Carb*Press*Line Speed
Effect
3.0000
2.2500
1.7500
0.7500
0.2500
0.5000
0.5000
Coef
1.0000
1.5000
1.1250
0.8750
0.3750
0.1250
0.2500
0.2500
SE Coef
0.1976
0.1976
0.1976
0.1976
0.1976
0.1976
0.1976
0.1976
T
5.06
7.59
5.69
4.43
1.90
0.63
1.26
1.26
P
0.001
0.000
0.000
0.002
0.094
0.545
0.242
0.242
S = 0.790569 PRESS = 20
R-Sq = 93.59% R-Sq(pred) = 74.36% R-Sq(adj) = 87.98%
We could construct an equation from this to predict Deviation from Target.
Deviation = 1.00 + 1.50*(%Carbonation) +1.125*(Pressure) + 0.875*(Line Speed) +
0.375*(%Carbonation*Pressure) + 0.125*(%Carbonation*Line Speed) +
0.250*(Pressure*Line Speed) + 0.250*(%Carbonation*Pressure*Line Speed)
We can actually get a better model, which we will discuss in a few slides.
>Stat>DOE>Factorial>Factorial Plots
Go to set up
Main Effects Plot (data means) for Deviation from Target
Pressure
%Carbonation
Mean of Deviation from Target
2
1
0
12
10
Line Speed
2
1
0
200
250
25
30
Practical Application
Carbonation has a large effect, so try to
control the temperature more precisely
 There is less deviation at low pressure, so
use the low pressure
 Although the slower line speed yields
slightly less deviation, the process engineers
decided to go ahead with the higher line
speed - WHY???

We can also use Minitab to construct a
predictive model!!
Estimated Effects and Coefficients for Deviation from Target (coded units)
Term
Constant
%Carbonation
Pressure
Line Speed
%Carbonation*Pressure
%Carbonation*Line Speed
Pressure*Line Speed
%Carb*Press*Line Speed
Effect
3.0000
2.2500
1.7500
0.7500
0.2500
0.5000
0.5000
Coef
1.0000
1.5000
1.1250
0.8750
0.3750
0.1250
0.2500
0.2500
SE Coef
0.1976
0.1976
0.1976
0.1976
0.1976
0.1976
0.1976
0.1976
T
5.06
7.59
5.69
4.43
1.90
0.63
1.26
1.26
P
0.001
0.000
0.000
0.002
0.094
0.545
0.242
0.242
It is recommended to delete
items with P > 0.200
S = 0.790569 PRESS = 20
R-Sq = 93.59% R-Sq(pred) = 74.36% R-Sq(adj) = 87.98%
>Stat>DOE>Factorial>Analyze Factorial Design
Select this arrow to remove
the 3-way interaction term.
Estimated Effects and Coefficients for Deviation from Target (coded units)
Term
Constant
%Carbonation
Pressure
Line Speed
%Carbonation*Pressure
%Carbonation*Line Speed
Pressure*Line Speed
Effect
3.0000
2.2500
1.7500
0.7500
0.2500
0.5000
Coef
1.0000
1.5000
1.1250
0.8750
0.3750
0.1250
0.2500
SE Coef
0.2041
0.2041
0.2041
0.2041
0.2041
0.2041
0.2041
T
4.90
7.35
5.51
4.29
1.84
0.61
1.22
P
0.001
0.000
0.000
0.002
0.099
0.555
0.252
S = 0.816497 PRESS = 18.9630
R-Sq = 92.31% R-Sq(pred) = 75.69% R-Sq(adj) = 87.18%
Next term to remove
Here is the final model from Minitab with the
appropriate terms.
Estimated Effects and Coefficients for Deviation from Target (coded units)
Term
Constant
%Carbonation
Pressure
Line Speed
%Carbon*Press
Effect
3.0000
2.2500
1.7500
0.7500
Coef
1.0000
1.5000
1.1250
0.8750
0.3750
SE Coef
0.2030
0.2030
0.2030
0.2030
0.2030
T
4.93
7.39
5.54
4.31
1.85
P
0.000
0.000
0.000
0.001
0.092
S = 0.811844 PRESS = 15.3388
R-Sq = 90.71% R-Sq(pred) = 80.33% R-Sq(adj) = 87.33%
Deviation from Target = 1.000 + 1.5*(%Carbonation) + 1.125*(Pressure) +
0.875*(Line Speed) + 0.375*(%Carbonation*Pressure)
Estimated Effects and Coefficients for Deviation from Target (coded units).
The term coded units means that the equation uses a -1 for the low value and a
+1 for the high value of the data.
Deviation from Target = 1.000 + 1.5*(%Carbonation) + 1.125*(Pressure) + 0.875*(Line
Speed) + 0.375*(%Carbonation*Pressure)
Let’s check this for %Carbonation = 10, Pressure = 30 psi, and Line Speed = 200 BPM
%Carbonation is at its low value, so it gets a -1. Pressure is at its high value, so it gets +1,
Line Speed is at its low value, so it gets a -1.
Deviation from Target = 1.000 + 1.5*(-1) + 1.125*(1)+ 0.875*(-1)+
Deviation from Target = -0.625 tenths of an inch
How does this compare with the actual runs at those settings?
0.375*(-1*-1)
The engineer wants the
higher line speed and
decides to put the target
slightly negative. Why??
NEVER GIVE THIS SETTING TO
PRODUCTION UNTIL YOU HAVE
VERIFIED THE MODEL!
VOC
Lawn
Mower
Example
Not too Noisy
System Spec
Noise Level < 75 db
Engine Noise
Blade Assy Noise
Combustion Noise
Blade Speed
Muffler Noise
Blade Area
Muffler Volume
Hole Area
Diameter
Blade Width
Blade Length
Grass Height
Blade to Hsg
Clearance