BLACK BELT – WEEK 4
“A successful
person is one
who can lay a
firm foundation
with the bricks
that others
throw at him or
her.”
- David Brinkley © 2014 PACCAR Inc
2
The Difference Between
Green Belt and Black Belt
Weeks 3 and 4
– Multiple X Statistical
Techniques
• Multiple ANOVA
• Multiple Regression
– Logistic Regression
– Experimentation
• Full Factorial DOE
• Fractional Factorial
• Attribute DOE
– Advanced SPC
• X-Bar / Subgroups
• Attribute Charts
© 2014 PACCAR Inc
– Leadership Skills
3
Black Belt Week 4
AM
Introduction / Review
Design of Experiments
Project Reviews
Fractional Factorials
Project Reviews
Simulation
Team Project
Team Project
COMPETITION
Team Presentations
Project Reviews
Full Factorials
Team Project
Project Reviews
Attribute DOE
Team Project
Team Project
(EVENING WORK)
Exam Preparation
FINAL EXAM
Exam Review
Congratulations!
LUNCH
PM
This week will be very intense… and very rewarding!
Due to the focus on DOE and the Team Case Study,
students typically consider this week the BEST part
of the entire Black Belt class!
© 2014 PACCAR Inc
4
The Tiger Trophy
•Enthusiastic participation
•Strong positive attitude
•Punctual
•Team oriented
•Helps others learn
•Tenacious problem solver
•Understands material in
Theory and in Practice
•Achieves noticeable project
progress using the methods
© 2014 PACCAR Inc
5
TEAM = Together Everyone Achieves More!
© 2014 PACCAR Inc
6
Exercise:
Reviewing Week 3
Week 4 Class Project:
Catapult
Design of Experiments
The Model
PROCESS
Uncontrollable Inputs
© 2014 PACCAR Inc
10
Outputs
(Continuous Y)
Controllable Inputs
Our Y is Continuous!
Most of this week of training will be dealing
with a continuous Y.
Later we will briefly discuss experimentation
with a discrete Y.
As with most situations,
a continuous scale Y is more useful.
© 2014 PACCAR Inc
11
Ways of Learning About a Process
• Empirically: Observance of naturally occurring informative event
(“Passive“ Multi-vari Studies)
– If you’re lucky, an informative event might happen while
you’re watching
• Experimentally: Create informative events
• Experimental Design (DOE)
– Proactively manipulates input variables so their effect on the
output variables can be studied
– Invites an informative event to occur
– Experiments, if done correctly, are efficient and powerful
© 2014 PACCAR Inc
12
What Is An Experimental Design ?
• A systematic series of tests in which various Input Variables
(X’s) are directly and purposefully manipulated. The effects
on the Output Variables (Y’s) are observed.
• Purpose: To determine which X’s most affect Y. To
determine where to set the influential X‘s to center Y on the
target and minimize the variability of Y.
• A well designed experiment eliminates all possible causes
except the one that you are testing. If an effect occurs on
the Y, then it can be tied directly to the KPIV’s you have
directly manipulated and not to some other variable.
© 2014 PACCAR Inc
13
Uses of Experiments
•
Characterizing a Process
– Determining which X’s most affect the Y’s
– Identifies those variables that need to be carefully
controlled in the Process
•
Optimizing a Process
– Determining where the critical Inputs should be set
•
Product Design
– Aids in understanding X’s early in the design process
– Provides direction for “robust” designs
© 2014 PACCAR Inc
14
The Trial & Error Approach
• Problem: Increase gas mileage from 20 mpg to 30 mpg.
• We might try:
– Change brand of gas
– Change octane rating
– Drive Slower
– Tune-up Car
– Wash and wax car
– Buy new tires
– Change Tire Pressure
• What if something works?
• What if it doesn’t?
© 2014 PACCAR Inc
15
The One-Factor-At-A-Time Approach
Problem: Gas mileage is 20 mpg
Speed
55
65
65
65
Octane
85
85
91
85
Tire Pressure Miles per Gallon
30
23
30
29
30
23
35
24
How many more runs would you need to figure out the best configuration
of variables? How can you explain the above results?
If there were more variables, how long would it take to get a good
solution?
What if there’s a specific combination of two or more variables
(interactions) that leads to the best mileage?
© 2014 PACCAR Inc
16
Full Factorial Experiment
Problem: Gas Mileage is 20 mpg
Speed
55
65
55
65
55
65
55
65
Octane
85
85
91
91
85
85
91
91
Tire Pressure Miles per Gallon
30
23
30
29
30
37
30
23
35
37
35
24
35
30
35
36
OFAT Runs
What conclusion do you make now? Is this approach better? Why?
© 2014 PACCAR Inc
17
DOE Terminology
The Basic Model
c1
Inputs
Controllable factors
c2
cp
Output
Process
z1
z2
zq
Uncontrollable factors
• Factors: A factor is one of the controlled or uncontrolled variables
being studied in the experiment.
– A factor may be quantitative, e.g., temperature in degrees,
time in seconds.
– A factor may also be qualitative, e.g., different machines,
different operator, clean or no clean.
© 2014 PACCAR Inc
19
Pressure:
HHHH LLLL HHHH LLLL HHHH LLLL
Temp:
HHLL HHLL HHLL HHLL HHLL HHLL
130
Yield
120
Levels
Factors
110
100
In d e x
Test Sequence
• Levels: The “levels” of a factor are the values of the factor being examined in
the experiment.
– For quantitative factors, each chosen value becomes a level
• For example, if the experiment is to be conducted at two different
temperatures, then the factor “ temperature has two “levels”
– In a qualitative factor, clean or no clean has two “levels”.
© 2014 PACCAR Inc
20
Common Terms
• Design (Layout): Complete specification of experimental test runs
including blocking, randomization, replications, repetitions, and
assignment of factor-level combinations to experimental units.
Run Temperature Pressure
© 2014 PACCAR Inc
1
Hi
Hi
2
Hi
Hi
3
Lo
Hi
4
Lo
Hi
5
Hi
Lo
6
Hi
Lo
7
Lo
Lo
8
Lo
Lo
21
Design Layout
Common Terms
• Treatment Combination: An experimental run using a set-up
with specific levels of each Input variable. Also known as a
Cell. This is also a test run.
Treatment
Combination
or Test Run
© 2014 PACCAR Inc
Run Temperature Pressure
1
Hi
Hi
2
Hi
Hi
3
Lo
Hi
4
Lo
Hi
5
Hi
Lo
6
Hi
Lo
7
Lo
Lo
8
Lo
Lo
22
Common Terms
• k1 x k2 x k3 ... Factorial:
Description of the basic design.
The number of treatment
combinations is the product
Run Temperature Pressure
– A 2 x 3 x 3 design has three
inputs, one with 2 levels and
two with 3 levels. This design
has 18 treatment
combinations
– A 2 x 2 design has two
inputs, each with 2 levels.
This has 4 treatment
combinations.
© 2014 PACCAR Inc
23
1
Hi
Hi
2
Hi
Hi
3
Lo
Hi
4
Lo
Hi
5
Hi
Lo
6
Hi
Lo
7
Lo
Lo
8
Lo
Lo
Common Terms
• Repetition: Running several samples during one
experimental setup run (multiple observations at the
same treatment combination)
• Replication: Replicating the entire experiment in a time
sequence
• You can use both in the same experiment.
© 2014 PACCAR Inc
24
Common Terms
• Balanced Design: A design with an equal number of
experimental units (observations) for each treatment
combination or experimental run.
Run Temperature Pressure
2
This is a
balanced design
because we have
2 repetitions of
each treatment
combination
2
2
2
© 2014 PACCAR Inc
25
1
Hi
Hi
2
Hi
Hi
3
Lo
Hi
4
Lo
Hi
5
Hi
Lo
6
Hi
Lo
7
Lo
Lo
8
Lo
Lo
Common Terms
• Unbalanced Design: A design with an unequal number of
units (observations) for each treatment combination.
Run Temperature Pressure
This is an
unbalanced
design because
we have 2
repetitions of
each treatment
combination
except for the
last one.
2
2
2
1
© 2014 PACCAR Inc
1
Hi
Hi
2
Hi
Hi
3
Lo
Hi
4
Lo
Hi
5
Hi
Lo
6
Hi
Lo
7
Lo
Lo
26
Common Terms
• Test Run: A single combination of factor levels
that yields one or more observations of the
output variable
• Main Effect: The average change in Y when a
single Factor (X) is changed from one level to
another level
• Interaction: The combined effect of two
factors that is over and above the singular
effect of each factor
© 2014 PACCAR Inc
27
Design Families
One Factor At A Time
Fractional Factorials
Screening Designs
Characterization
Studies
Full Factorials
Full Factorials
Response Surface Methods
Optimization
Studies
© 2014 PACCAR Inc
28
Design Families
• Screening Designs:
– To isolate the “vital few” from the trivial many. Investigates a
relatively large number of factors with a small number of
experimental runs. Generally fractional factorial designs.
• Characterization Designs:
– To identify the key leverage variables. Investigates more complex
relationships among a small number of factors (2-6). Generally
full factorial experiments or higher resolution fractional factorial
designs.
• Optimization Designs:
– To define the optimal operating windows for key leverage input
variables. Full factorial and response surface designs.
© 2014 PACCAR Inc
29
DOE Basic Steps
Basic Steps to Experimentation
1.
DEFINE THE PROBLEM !!!
2.
Establish the objective
3.
Select the Output variables
4.
Select the Input variables
5.
Choose the variable levels
6.
Select the Experimental Design
7.
Collect Data
8.
Analyze Data
9.
Draw Statistical Conclusions
10.
Replicate results
11.
Draw Practical Solutions
12.
Implement solutions
© 2014 PACCAR Inc
31
Example Objectives
•
To determine the effects of material variation on product
reliability
•
To determine sources of variation in a critical process
• To determine the impact of operator variation on the product
•
To determine cause-effect relationships between process inputs
and product characteristics
Usually stated in terms of the effects of inputs on outputs
© 2014 PACCAR Inc
32
Factor Selection
• Which factors (KPIV’s) do we include?
• We can use the following sources:
• Process Map
• Cause and Effects Matrix
• FMEA
• Multi-vari Study Results
• Brainstorming (For a simple process)
• Literature Review
• Engineering Knowledge
• Operator Experience
• Scientific Theory
• Customer/Supplier Input
© 2014 PACCAR Inc
33
Level Selection
• Once factors are selected, the levels of the factor must be determined
• Beware of Too Close/Too Wide issues
• It can be helpful to begin with the normal operating range
• Objective: Determine vital few Inputs from a large number of variables
(Screening)
• Examples of settings:
– Qualitative:
• Method A Vs B
• Machine 1 Vs Machine 2
– Quantitative
• 5 minutes Vs. 15 minutes
• 30 psi Vs. 60 psi
© 2014 PACCAR Inc
34
General Advice
• Make sure you have tied potential business results to your
project
• The best time to design an experiment is after the previous
one is finished
• Don’t try to answer all the questions in one study.
Rely on a sequence of studies
• Use two-level designs early
• Spend less than 25% of budget on the first experiment
• Always verify results in a follow-on study
• Be ready for changes
• A final report is a must!! - The reader should be able to
replicate the experiment from your report
© 2014 PACCAR Inc
35
Final Report Basic Framework
• State the objective of the experiment
• State the time frame of the experiment
• State the names of the participants (points of contact)
• What was the “Y”? (Response)
• What were the results of the MSA on the Y? (proof the data is valid)
Provide enough information
so that someone who was
not there can
run the experiment
• What were the X’s? (Inputs/Factors)
• What were the levels for each X?
• How many replications were done?
• For results clarify…
– Significant factors and interactions (statistically & epsilon square)?
– What was the level of error?
• Describe the method of validation (how did you confirm?)
© 2014 PACCAR Inc
36
Be Proactive
Move Fast, Hit Hard
• DOE is a proactive tool
• There is no such thing as a bad experiment - only poorly
designed and executed ones
• Not every experiment will produce earth shattering
discoveries
– Something will always be learned
– New data prompts asking new questions and
generates follow-on studies
• Validate! Don’t blindly follow the outcome. Confirm!
© 2014 PACCAR Inc
37
Full Factorial DOE
Objectives
After this Module, You Should be Able to…
• Describe the overall concepts of 2k Factorials
• Create standard order designs
• Design and Analyze 2k Factorials
– Using ANOVA
– Using Effects Plots
– Graphs and Residual Plots
• Use Center Points in your designs
© 2014 PACCAR Inc
39
Families of Designs
One Factor At A Time
Fractional Factorials
Screening Designs
We’re starting here
because it’s the
least difficult
Characterization
Studies
Full Factorials
Full Factorials
Response Surface Methods
Optimization
Studies
© 2014 PACCAR Inc
40
The 2-Level Design
The 2k Factorial DOE
2k factorials refer to k factors, each with 2 levels.
A 22 factorial is a 2x2 factorial. This design has two factors
with two levels and can be done in 2x2 or 4 runs.
Likewise a 23 factorial includes 3 factors, each with two
levels. This experiment can be done in 2x2x2 or 8 runs.
• Require relatively few runs per factor studied
• Can be the basis for more complex designs
• Good for early investigations - can look at a large
number of factors with relatively few runs
• Lend themselves well to sequential studies
• Analysis is fairly easy
© 2014 PACCAR Inc
42
Standard Order of 2k Designs
The design matrix for 2k factorials are usually shown is
standard order. The low level of a factor is designed with a “-”
or -1 and the high level is designated with a “+” or 1.
An example of a design matrix for a 22 Factorial would look
like this:
Temp
Conc
A 23 Factorial Looks like this:
The 22 is imbedded inside the 23
© 2014 PACCAR Inc
43
-1
1
-1
1
-1
-1
1
1
Temp
-1
1
-1
1
-1
1
-1
1
Conc
-1
-1
1
1
-1
-1
1
1
Catalyst
-1
-1
-1
-1
1
1
1
1
EXERCISE
• Create a 24 Factorial Design Matrix
• What are the minimum number of runs
needed?
Not by hand!
Use MINITAB!!!
© 2014 PACCAR Inc
44
Create the Matrix in MINITAB
Stat > DOE > Factorial > Create Factorial Design
Generate the Matrix in MINITAB
24 Factorial Design Matrix
1. Select the correct number of Factors
2. Click on the Designs button
3. Select the Full Factorial Design
© 2014 PACCAR Inc
46
Building the Matrix
Select Factors to assign names
and levels to various factors
DeselectitRandomize
Runs to randomize the runs. However, in
Typically,
is good practice
options
while
class
BB
class
we in
will
keep the data in standard (non-randomized)
format so we can learn the DOE concepts as a class.
© 2014 PACCAR Inc
47
24 Matrix
Note the patterns…
•
1st column is alternating -1
&1
•
2nd column is 2 of one type
then 2 of another
•
3rd column is 4 of one type
then 4 of another
•
4th column is 8 of one then
8 of the other.
•
Remember the patterns and
you can make your own
matrix quickly.
© 2014 PACCAR Inc
48
DOE Roadmap
DOE Roadmap
(Page 1)
• Define objective of the experiment
• Design the experiment
– Stat > DOE > Factorial > Create Factorial Design
• Do the experiment
– Collect output data and add into Minitab
• Analyze the results for STATISTICAL significance
– Stat > DOE > Factorial > Analyze Factorial Design
– Identify KPIVs from Effects Plots and p values
– Reduce model if necessary and re-run
DOE Roadmap
(Page 2)
• Analyze the results for PRACTICAL significance (if needed)
– Go into Results and turn on Expanded Tables
– Re-run the analysis by clicking “OK”
– % Contribution will identify dominant KPIVs
• Optimize the process
– Stat > DOE > Factorial > Factorial Plots
– Stat > Tables > Descriptive Statistics (Cross Tab)
– Select best input settings to meet requirements
• Investigate further (2nd DOE, Center Points, Blocking, etc.)
DOE Example
Example – 23 Factorial
• This example relates two quantitative Input Variables
(Temperature and Concentration) and one qualitative Input
Variable (Catalyst) to the Output (Yield)
Factor
Low Level (-1)
High Level (1)
Temperature
160o C
180o C
Concentration
20%
40%
Catalyst
Brand A
Brand B
• What are the KPIVs affecting Yield?
• What are the optimal settings to maximize Yield?
© 2014 PACCAR Inc
53
DOE Roadmap
(Page 1)
• Define objective of the experiment
• Design the experiment
– Stat > DOE > Factorial > Create Factorial Design
• Do the experiment
– Collect output data and add into Minitab
• Analyze the results for STATISTICAL significance
– Stat > DOE > Factorial > Analyze Factorial Design
– Identify KPIVs from Effects Plots and p values
– Reduce model if necessary and re-run
Stat > DOE > Factorial > Create
Factorial Design…
© 2014 PACCAR Inc
55
Assigning Factors to the Experiment
•
You can type in the Name of the Factor
•
Designated Variable vs. Attribute data
•
Use Coded or Uncoded Units for Low High
© 2014 PACCAR Inc
56
Don’t forget to
deselect ‘Randomize
runs’ while you are in
class
Design Matrix
This is the design matrix. You run the experiment relative to the
RunOrder (1,2,3….8) If we had randomized the runs the
StdOrder column would be different than the RunOrder column.
DOE Roadmap
(Page 1)
• Define objective of the experiment
• Design the experiment
– Stat > DOE > Factorial > Create Factorial Design
• Do the experiment
– Collect output data and add into Minitab
• Analyze the results for STATISTICAL significance
– Stat > DOE > Factorial > Analyze Factorial Design
– Identify KPIVs from Effects Plots and p values
– Reduce model if necessary and re-run
Collecting Output Data
Here is the output
data from the
experiment.
Add this as a new
column in your
Minitab file
Notice:
The Y value (60) is the result
when Temp was set to low,
Conc was set to low and
Catalyst was set to low.
Yield
60
72
54
68
52
83
45
80
© 2014 PACCAR Inc
60
DOE Roadmap
(Page 1)
• Define objective of the experiment
• Design the experiment
– Stat > DOE > Factorial > Create Factorial Design
• Do the experiment
– Collect output data and add into Minitab
• Analyze the results for STATISTICAL significance
– Stat > DOE > Factorial > Analyze Factorial Design
– Identify KPIVs from Effects Plots and p values
– Reduce model if necessary and re-run
Stat > DOE > Factorial > Analyze Factorial Design
© 2014 PACCAR Inc
62
Stat > DOE > Factorial > Analyze Factorial Design
The typical plots to analyze are
the Pareto and the Normal Plots.
You can include all terms or only
model terms
© 2014 PACCAR Inc
63
Stat > DOE > Factorial > Analyze Factorial Design
Go into Options to select the
alpha risk level.
A conservative risk level is 0.10
For this example, let’s use
an Alpha of 0.05
© 2014 PACCAR Inc
64
DOE Analysis Results
There aren’t any p-values because
we only have one data point per treatment combination.
If we “reduce the model” by eliminating non-important
main effects and interactions, we might get p-values
Pareto Effects Plot
This indicates that A and the AC
interaction are important
However, we should first reduce the model
to eliminate non-significant factors
Normal Effects Plot
+
This indicates that the Effects
associated with A (Temp) and the A*C
(Temperature * Catalyst) Interaction
could be important.
Reducing the Model
• One at a time, eliminate:
– The high order interactions (3-way or higher)
– The least important factors (highest p-value)
• Continue reducing the model until only the
significant factors are remaining
© 2014 PACCAR Inc
68
Reducing the Model
• Remove the 3 way
interaction first.
• Review the updated
Pareto
ABC Factor is
gone
© 2014 PACCAR Inc
69
Review the Session Window
• Notice the P values have arrived!
• Find the highest P value, remove the
term, re-evaluate and so on…
Reducing the Model
•
•
Notice the last two terms
look close to the line
You must always verify
with P values before
assuming the model has
been completely
reduced
© 2014 PACCAR Inc
71
Reducing the Model
•
Only one last term to
remove
© 2014 PACCAR Inc
72
Reducing the Model
Error Message from
Minitab!
Leave in terms that are
a part of interactions
© 2014 PACCAR Inc
73
Final Reduced Model
• This is only a graphical
representation of the
reduced model.
• Always verify with P values!
Mathematical Model
•
•
© 2014 PACCAR Inc
75
Review the final P values
Review the R squared
values
Mathematical Model
These “Coefficients” can be used
to develop a mathematical
equation (next slide)
Here is the “Effect” on the Output
when each Input is changed from
the low to high level
© 2014 PACCAR Inc
76
Mathematical Model
We can use the Coefficients from the analysis to
derive the following mathematical model:
Yield = 64.250 + 11.500(Temp) - 2.500(Conc)
+ 0.750(Cat) + 5.000(Temp*Cat)
Important Note: Since our DOE analysis was run using
“coded units” (-1 and +1), the above equation is only valid
when we use -1 and +1 as Input values.
When you use “uncoded” units Minitab will provide different
coefficients in the regression equation.
© 2014 PACCAR Inc
77
DOE Roadmap
(Page 2)
• Analyze the results for PRACTICAL significance (if needed)
– Go into Results and turn on Expanded Tables
– Re-run the analysis by clicking “OK”
– % Contribution will identify dominant KPIVs
• Optimize the process
– Stat > DOE > Factorial > Factorial Plots
– Stat > Tables > Descriptive Statistics (Cross Tab)
– Select best input settings to meet requirements
• Investigate further (2nd DOE, Center Points, Blocking, etc.)
Showing Significance
If the basic DOE analysis does not provide enough information, we
can expand the analysis to find practical significance.
Go into Results and
change the Display of
Results to “Expanded
tables”
Session Window now
includes a Practical
Significance column
DOE Roadmap
(Page 2)
• Analyze the results for PRACTICAL significance (if needed)
– Go into Results and turn on Expanded Tables
– Re-run the analysis by clicking “OK”
– % Contribution will identify dominant KPIVs
• Optimize the process
– Stat > DOE > Factorial > Factorial Plots
– Stat > Tables > Descriptive Statistics (Cross Tab)
– Select best input settings to meet requirements
• Investigate further (2nd DOE, Center Points, Blocking, etc.)
Optimize the Process
Stat > DOE > Factorial > Factorial Plots
© 2014 PACCAR Inc
82
Optimize the Process
Stat > DOE > Factorial > Factorial Plots
Plot the Main Effects Plot and
Interaction Plot for all factors
83
Analysis
We can use the interaction plot for analyzing the
Temperature by Catalyst Interaction. What do
we select if we want to higher Yield?
If the Temp is set at (-1)
then the best Catalyst is (-1)
If the Temp is set at (1)
then the best Catalyst is (1)
If you can’t select the Temp you’ll need to be flexible on Catalyst.
Analysis
Steeper the slope of the line, the greater the impact of that X on the Y.
Analysis
Stat > DOE > Factorial > Cube Plot
© 2014 PACCAR Inc
86
Select “Data means” to see
only the factors levels in
our analyzed design
Analysis
Indicates the Highest Mean response
when Temp = 1, Catalyst = 1 and
Conc = -1
• Select best settings
• Use Cross Tab to evaluate Standard Deviation
© 2014 PACCAR Inc
88
Review Variation Over the Three Factors
Stat > Tables > Descriptive Statistics
Review Variation Over the Three Factors
Review the Tables for Metrics
With the small amount of data we
have we don’t see the standard
deviations across treatments
© 2014 PACCAR Inc
90
DOE Exercises
EXERCISE
• Use the data for the 2x2x2x2 Factorial
• Response is “Convert” (higher is better).
• Inputs are Cat-Charg, Temp, Press, Conc
• Analyze the data and be prepared to state your
conclusions.
Open file:
BHH325.mtw
in the provided files
• Use Alpha = 0.10
© 2014 PACCAR Inc
92
DOE Roadmap
(Page 1)
• Define objective of the experiment
• Design the experiment
– Stat > DOE > Factorial > Create Factorial Design
• Do the experiment
– Collect output data and add into Minitab
• Analyze the results for STATISTICAL significance
– Stat > DOE > Factorial > Analyze Factorial Design
– Identify KPIVs from Effects Plots and p values
– Reduce model if necessary and re-run
DOE Roadmap
(Page 2)
• Analyze the results for PRACTICAL significance (if needed)
– Go into Results and turn on Expanded Tables
– Re-run the analysis by clicking “OK”
– % Contribution will identify dominant KPIVs
• Optimize the process
– Stat > DOE > Factorial > Factorial Plots
– Stat > Tables > Descriptive Statistics (Cross Tab)
– Select best input settings to meet requirements
• Investigate further (2nd DOE, Center Points, Blocking, etc.)
Define the Factorial Design
© 2014 PACCAR Inc
95
Effects Plot
Use the normal plot
to check borderline
Pareto results.
Is factor C significant?
© 2014 PACCAR Inc
96
Reduced Model - Visual
Interpret Large Effects
(Highest Order Interactions First)
We see the Temperature x Concentration interaction is
important (BD), so we go to Stats > DOE > Factorial > Factorial Plots
and create the interaction graph:
© 2014 PACCAR Inc
98
Investigate Appropriate Graphs
Main Effects Plots are very handy for investigating all main effects
very quickly.
© 2014 PACCAR Inc
99
Practical Significance
Go to Stat > DOE > Analyze Design and go to the
Results tab
Change the Tables to “Expanded Tables”
© 2014 PACCAR Inc
100
Practical Significance
We see that Temp is by far the strongest
factor in this experiment.
© 2014 PACCAR Inc
101
In-Class EXERCISE
• Objective: Working with a PACCAR supplier we want to determine the effect of
Moisture, Temperature and Crimp Condition on the “Dyeability” of carpeting for
the cab.
Open
MINITAB File
Carpet2.mtw in
provided files
• Output: Dyeability - Higher is better
• Inputs:
– Moisture (Dry, Wet)
– Process Temperature (195, 205)
– Fiber Crimp (Low, High)
• Design: 2x2x2 with two replicates
• Procedure:
– Analyze the Data completely
• Analysis of Interactions and Main Effects
• Follow-up Graphical Analysis
Alpha risk = 0.05
• Diagnostics
• Practical Significance
– Be prepared to present conclusions - “Neighbor Teams”
© 2014 PACCAR Inc
102
Inferential Statistics
Notice the associated p-values for the Main Effects are < 0.05.
The Moisture X Crimp interaction is close (0.085) so we might investigate
that interaction by reducing the model.
© 2014 PACCAR Inc
103
Reduced Model - Diagnosis
This graph reaffirms that Moisture and Temp are active
effects and the Main Effect for Crimp is small in comparison.
However, Crimp is involved in an interaction with Moisture so
we investigate the interaction plot.
© 2014 PACCAR Inc
104
Reduced Model - Diagnosis
This graph indicates that the combination of Moisture at level 1
and Crimp at level 2 yields the best dyeability.
© 2014 PACCAR Inc
105
Practice!!!
You are Working with a supplier to develop a better dye process
for upholstery shade. Use the data below for a 23 full factorial.
• Factors affecting shade:
Shade
target value
is 220
• Material quality,
• Oxidation Temperature,
• Oven Pressure
Material Quality
L
H
Oxy Temp
• Oven Pressure
Oxy Temp
L
H
L
H
L
189
195
228
200
H
218
238
259
241
© 2014 PACCAR Inc
106
DOE Using Center Points
Adding Center Points
• There is always a risk in 2-level designs of missing a curvilinear
relationship by only including two levels of the Input Variable.
• The addition of Center Points is an efficient way to test for curvature
without adding a large number of experimental runs.
EXAMPLE...
• A Belt wants to improve the yield of a process. There are two inputs of
interest: Time and Temperature.
• Normal factor range is 150 - 160 for temp & 30 - 40 for time.
• The Belt decides to conduct the experiment using a 2x2 design (Time x
Temp), but will add five center points to estimate experimental error
and curvature.
• Revised Inputs:
– Temp: 150, 155 and 160
– Time: 30, 35 and 40.
© 2014 PACCAR Inc
108
Creating Center Points in MINITAB
Stat > DOE > Factorial > Create Factorial Design
© 2014 PACCAR Inc
109
Why are we using
the actual values instead
of coded units?
© 2014 PACCAR Inc
110
Deselect the
Randomized Runs
for class use only
© 2014 PACCAR Inc
111
The experiment was carried out with the following results:
Yield
39.3
40.0
40.9
41.5
Center
Points
40.3
40.5
40.7
40.2
40.6
Hey! We’ve got P-values!
(The center points Provided Enough Information Despite Only One Replicate)
The center points are not “active” (T-value) nor are they statistically significant.
This is a linear system.
© 2014 PACCAR Inc
113
Cube Plot with Center Point
Let’s Try:
Yield increases
Yield increases
© 2014 PACCAR Inc
114
YIELD2
39.9
40.0
40.9
41.5
42.5
42.3
42.2
42.6
42.4
Look What Happened to Curvature
This is a non-linear system. Curvature exists!
© 2014 PACCAR Inc
115
Note the Uphill-Downhill from Corner-to-Corner
Yield decreases
Peak
Yield increases
© 2014 PACCAR Inc
116
Review the Practical Significance
In the Results tab choose the “Expanded table”
Curvature has a large contribution number.
© 2014 PACCAR Inc
117
Center Point Exercise
Practice!!!
•
You are assisting in the development of a new process for using adhesive to secure a
component in a cab.
•
Data has been provided to you in preparation for a 23 full factorial - 2 replicates.
– A = Amount used (0.25” or 0.75” blob)
– B = Application pattern (4 or 6 point pattern)
– C = Surface Roughness (Low or High roughness)
– Response = lbs. of pull required to remove
•
Run
A
B
C
Responses
1
-
-
-
14
15
2
+
-
-
15
15
3
-
+
-
14
14
4
+
+
-
14
13
5
-
-
+
19
22
6
+
-
+
18
20
7
-
+
+
20
21
8
+
+
+
18
19
© 2014 PACCAR Inc
119
Add center points
•
More Practice!!!
Now, data has been provided to you in preparation for experimental analysis using a 23
full factorial w/ 2 center points - 2 replicates.
– A = Amount used (0.25”, 0.5” or 0.75” diameter blob)
– B = Application pattern (4, 5, 6 point pattern)
– C = Surface Roughness (Low, Med, High)
– Response = lbs. of pull required to remove
•
Run
A
B
C
Responses
1
-
-
-
14
15
2
+
-
-
15
15
3
-
+
-
14
14
4
+
+
-
14
13
5
-
-
+
19
22
6
+
-
+
18
20
7
-
+
+
20
21
8
+
+
+
18
19
9
0
0
0
18
19
© 2014 PACCAR Inc
120
Same data but...
Look where the
centerpoints are...
DOE Using Blocking
Blocking with 2k Factorials
Suppose we wanted to run a 2x2x2 factorial. The factors are A, B &
C. We would like to run the experiment under as homogeneous
conditions as possible. But we find that we need two bags of
catalyst to run the entire experiment. The challenge is to run the
experiment while minimizing the effect of the Noise variable, Catalyst.
Recall our basic design matrix:
Run
A
B
C
1
-1
-1
-1
2
1
-1
-1
3
-1
1
-1
4
1
1
-1
5
-1
-1
1
6
1
-1
1
7
-1
1
1
8
1
1
1
© 2014 PACCAR Inc
It’s easy to see that if we ran the first
four runs with Bag 1 of the Catalyst
and the 2nd four runs with Bag 2, we
would “confound” Factor C.
We could not separate the Effect of C
from the Effect of Catalyst.
122
Confounding and Blocking
We must figure out a way to “spread” the Catalyst Effect across the
experiment to neutralize the Catalyst Effect. Let’s review the
expanded Design Matrix that shows the contrasts for all the
Experimental Interactions.
Run
1
2
3
4
5
6
7
8
A
-1
1
-1
1
-1
1
-1
1
B
-1
-1
1
1
-1
-1
1
1
C
-1
-1
-1
-1
1
1
1
1
A*B
1
-1
-1
1
1
-1
-1
1
A*C
1
-1
1
-1
-1
1
-1
1
B*C
1
1
-1
-1
-1
-1
1
1
A*B*C
-1
1
1
-1
1
-1
-1
1
Generally, the 3-way Interactions in experiments are not significant
or important. We can use that contrast vector to define our
blocking Strategy. The new design would look like:
© 2014 PACCAR Inc
123
Confounding
and
Blocking
Run
A
B
C
A*B
A*C
B*C A*B*C Block
1
2
3
4
5
6
7
8
-1
1
-1
1
-1
1
-1
1
-1
-1
1
1
-1
-1
1
1
-1
-1
-1
-1
1
1
1
1
1
-1
-1
1
1
-1
-1
1
1
-1
1
-1
-1
1
-1
1
1
1
-1
-1
-1
-1
1
1
-1
1
1
-1
1
-1
-1
1
I
II
II
I
II
I
I
II
So we would run runs 1, 4, 6 and 7 with Bag 1 and 2, 3, 5 and 8 with Bag 2.
This experiment would not allow us to test the 3-way interaction, but would allow us to
investigate the Main Effects and 2-way Interactions without worrying about the
Catalyst Effects.
Note: In an actual
experiment, you would
randomize the runs within
each Block. MINITAB will do
that for you.
© 2014 PACCAR Inc
124
Run
1
4
6
7
A
-1
1
1
-1
B
-1
1
-1
1
C
-1
-1
1
1
Block
I
I
I
I
2
3
5
8
1
-1
-1
1
-1
1
-1
1
-1
-1
1
1
II
II
II
II
Why Do We Block? When Do We Block?
• Blocking is a way to deal with noise factors that you can not
control but do not want to impact your results.
• Such factors as
– Day
– Shift
– Equipment/Machine
– Location/Plant
– Etc.
• When you block, you remove the effect of those factors and
focus on the chosen items of interest.
© 2014 PACCAR Inc
125
In-Class EXERCISE
• Objective: To perform the analysis of a Four-Factor Experiment with two
blocks.
• Problem: A Belt is working with a supplier who is interested in
maximizing the fuel filtration rate for a diesel engine. The experiment
will take 16 runs but only 8 runs can be completed in a day. Two days are
necessary for the completion of the entire experiment.
• Output: Filtration Rate (gal/h) - higher is better
• Inputs:
– Temperature (T)
– Pressure (P)
– F-Concentration (F)
– Stirring Rate (S)
• Procedure:
– Design the experiment with 4 Factors in 16 runs with 2 Blocks.
– Analyze the data using the provided data.
– Use Alpha = 0.10
© 2014 PACCAR Inc
126
Create Factorial Design
Blocks = DAY
Put in the
Output
information
© 2014 PACCAR Inc
127
Filtrate
71
48
68
65
43
104
86
70
45
65
60
80
100
45
75
96
Telling MINITAB About a Block
Stat > DOE > Factorial > Define Custom…
© 2014 PACCAR Inc
128
Analyze the Design
Review the
Graphs for
Visual
Significance
•
© 2014 PACCAR Inc
129
Review the Session Window
No P-values!
We need to reduce
the model
Looking for P-Values & T-Values
• Since we have no replications we
won’t get p-values or T-values.
• We can generate these by asking
MINITAB to only analyze 2-way
interactions or less.
• Stat > DOE > Factorial > Analyze
Factorial…
– Select Terms
– Set “Include terms…” to 2
• Run the analysis again
© 2014 PACCAR Inc
131
Review the Session Window
We now have
P-values!
Review to see if
we should reduce
the model further
If “Block” were
significant then
something
unexpected
happened and we
need to investigate.
Final Reduced Model
Final reduced
model is
without
Pressure factor
Diagnostics
The residuals look good, so it looks like our
model fits the data.
© 2014 PACCAR Inc
134
Diagnostics
If we want a high Filtrate Output we set the
factors: Temp (1), F Conc (1), Stir Rate (1)
© 2014 PACCAR Inc
135
Interaction Plots
There are 3 Significant Main Effects (Temp, F Conc, Stir), but each of
these is involved with one of the Two 2-way interactions.
Notice Minitab does not display interactions that are not significant.
© 2014 PACCAR Inc
136
Analysis - Practical Significance
In the Results tab choose the “Expanded table”
Temperature has the highest Practical Significance.
If you can’t change everything investigate if you can
control (change) Temperature.
© 2014 PACCAR Inc
137
Power and Sample Size
DOE
Sample Size & 2k Factorials
• One observation per treatment combination
– Usually low statistical power
– Use normal plots instead of F-tests
– Can create full factorials by dropping unimportant
factors out.
• More than one observation per treatment combination
– Better estimates of error
– Better statistical power
– Can still run reduced models
– F-tests and normal plots can be used.
© 2014 PACCAR Inc
139
2-Level Factorial Sample Size
• A full factorial DOE (23 = 8 runs) is contemplated to
investigate the sensitivity of thermostat motion to changes
in the chosen factors
• A motion of 0.001 inch affects functional performance, and
the design team wants to detect an effect of this size or
greater
• Process variation in thermostat motion is considerable. SPC
data indicates that the standard deviation is about 0.001 inch
(although the process is stable)
• The alpha risk that is deemed acceptable (risk of saying the
two levels of a factor are significant when they really aren‘t)
is 0.05
• Due to time constraints, only two replicates (16 runs total)
can be performed
© 2014 PACCAR Inc
140
2-Level Factorial Sample Size
Continued
•
What is the probability of seeing an effect of 0.001 inch
when such an effect exists? (Power)
•
From the information given, we have:
– Significance level, a = 0.05
– Number of replicates = 2
– Effect Size, d = 0.001 inch
– Standard Deviation, s = 0.001 inch
–
Alternate Hypothesis (Ha) is that a factor has an
effect upon the thermostat motion
© 2014 PACCAR Inc
141
Minitab Exercise
Stat > Power and Sample Size > 2 Level Factorial…
2-Level Factorial Sample Size
Stat> Power and Sample Size> 2 Level Factorial…
Power is what
we want, so we
leave it blank.
© 2014 PACCAR Inc
143
2-Level Factorial Sample Size
How many
replicates will be
needed to
get Power
above .90?
If only 16 runs
are allowed
what else can
you do?
© 2014 PACCAR Inc
144
2-Level Factorial Sample Size
How many replications are needed for a Power of
.90 or greater IF we change the effect (signal)
from .001 to .002?
How many replications are needed for a Power of
.90 or greater IF the effect is .001 BUT the sigma is
.0001 (less variation)?
© 2014 PACCAR Inc
145
Sample Size Rules of Thumb
2 8
2
Here are general
recommendations
for minimum
sample sizes:
2  16
3
2  16 (32 recommended)
4
2  32 (64 recommended)
5
© 2014 PACCAR Inc
146
EXERCISE
• You are planning to use a 24 full factorial to compare
the impact of 4 factors on the average Y.
• 4 factors - 16 corner points
• The effect you seek is 5
• You would like a Power of 0.90
• Past data shows that the standard deviations are equal
at 3
• How many replicates are needed?
• Replicates for an effect of 3 (less signal)?
• Effect reset to 5, but standard deviation of 2 (less
noise)?
• Standard deviation reset to 3, but Power set to .95?
© 2014 PACCAR Inc
147
Full Factorial Exercises
Easy Practice
•
In celebration of your team’s success with a Six Sigma project you decide to serve
popcorn and soft drinks. Being a hard core BB, you set up a 23 full factorial to maximize
popcorn yield.
•
Run
Brand Ratio Batch
Response (cup yield)
1
-
-
-
6.25
2
+
-
-
8
3
-
+
-
6
4
+
+
-
9.5
5
-
-
+
8
6
+
-
+
15
7
-
+
+
9
8
+
+
+
17
•
Calculate the main effects and interactions (if any).
•
Note: “Ratio” is the ratio of popcorn to oil (low, high). “Batch” is the cup size of
popcorn (1/3, 2/3).
© 2014 PACCAR Inc
149
Not So Easy Practice!
•
Water leaks in the doors have been a real hassle for PACCAR. You come to the rescue
and design a 23 full factorial with 3 replicates total. The response variable is the amount
of water from a leak. NOTE: Assume less leaking is better AND that our leak-detector
has passed an MSA (green light).
•
Run
A
B
C
Responses
1
-
-
-
2
3
4
2
+
-
-
7
4
9
3
-
+
-
1
6
2
4
+
+
-
7
7
6
5
-
-
+
4
2
1
6
+
-
+
3
3
1
7
-
+
+
2
8
3
8
+
+
+
5
6
4
Factors:
A = Tool (A, B)
•
B = Door seal material (A, B)
C = Adjustment sequence (1, 2)
© 2014 PACCAR Inc
150
.
More Practice!!
• A comparison is done of truck profits (in dollars) as a function of
pricing and advertising.
• Which factors are significant with alpha = 0.05?
• Which interactions are significant?
• What do you recommend for an action plan?
Advertising
With
Without
9800
6000
10600
5300
6200
3900
7100
4300
PRICING
W Discount
WO Discount
© 2014 PACCAR Inc
151
Enough Already !!!!!
• Objective: What effect do the
following factors have on assembly
time (Y) in seconds.
– Build Paper Format BP (A, B)
– Assembly Preparation AP (old, new)
– Fastener Type FT (Quik, Zapp)
• Analyze and report…
© 2014 PACCAR Inc
152
Y
131.4
132.4
141.6
135.0
125.9
120.2
128.7
130.7
Fraction Factorial DOE
Families of Designs
These will be the
focus of this module
Screening Designs
Characterization
Studies
Full Factorials
Full Factorials
Response Surface Methods
Optimization
Studies
© 2014 PACCAR Inc
One Factor At A Time
Fractional Factorials
154
Why do Fractional Factorial Experiments?
• As the number of factors increases, so do the number of runs
– 2x2 Factorial = 4 runs
– 2x2x2 Factorial = 8 runs
– 2x2x2x2 Factorial = 16 runs
– 2x2x2x2x2x2x2 Factorial = 128 runs!
• IF the experimenter can assume higher order interactions are
negligible, it is possible to do a fraction of the full factorial and
still get good estimates of low-order interactions
• The major use of Fractional Factorials is for screening
– A relatively large number of Factors can be evaluated in a
relatively small number of runs
Factorial Experiments
Successful factorials are based on:
• The Sparsity of Effects Principle
–
Systems are usually driven by Main Effects and Low-order
interactions
• The Projective Property
–
Fractional Factorials can represent full-factorials once some
effects demonstrate weakness (lack of impact -- TOSS ‘EM!)
• Sequential Experimentation
–
Fractional Factorials can be combined into more powerful
designs
–
Half-Fractions can be “folded over” into a full factorial
–
By eliminating uninteresting Input Variables, fractions can
become full factorials
© 2014 PACCAR Inc
156
Half-Fraction
Recall that the table below is the expanded representation of a
23 Factorial design
Factor D
A
-1
1
-1
1
-1
1
-1
1
B
-1
-1
1
1
-1
-1
1
1
C
-1
-1
-1
-1
1
1
1
1
AXB
1
-1
-1
1
1
-1
-1
1
AXC
1
-1
1
-1
-1
1
-1
1
BXC
1
1
-1
-1
-1
-1
1
1
AXBXC
-1
1
1
-1
1
-1
-1
1
Suppose we wanted to investigate 4 Input Variables but can not
afford extra runs…
Since all the contrasts are independent (orthogonal) we can
assign any interaction as the contrast to represent the fourth
variable…
Usually we select the highest order interaction and replace it with
the additional factor…
In this case, when we replace the AxBxC Interaction with Factor
D, we say the ABC was aliased with D.
© 2014 PACCAR Inc
157
Half-Fraction
The new design matrix looks like this:
A
-1
1
-1
1
-1
1
-1
1
B
-1
-1
1
1
-1
-1
1
1
C
-1
-1
-1
-1
1
1
1
1
D
-1
1
1
-1
1
-1
-1
1
This is a Half-fraction of a 24 design
Instead of 16 runs, we only need 8 runs (HALF) to evaluate 4
factors
This is considered a “Resolution IV Design” (see next slide)
© 2014 PACCAR Inc
158
Half Fractions
We would call this a half-fraction since a full 24 Factorial would
take 16 runs to complete. Here we can estimate 4 factors in 8 runs
(half of 16).
But there is a cost: We lost the higher order interaction. When
assessing what we have to lose, we use the concept of Resolution.
• Resolution III Designs
– No main effects are aliased with other Main Effects
– Main Effects aliased with two-factor interactions
• Resolution IV Designs
– No Main Effect aliased with other Main Effects or with twofactor interactions
– Two-factor interactions aliased with other two-factor
interactions
• Resolution V Designs - We will focus on these
– Main Effects okay, Two-factor interactions aliased with 3-factor
interactions
Notation
The general notation to designate a fractional factorial design is:
kp
2R
k is the number of factors to be investigated
2k-p is the number of runs
R is the resolution
Example:
The designation below means four factors will
be investigated in 23 = 8 runs. This design is
resolution IV.
41
2 IV
© 2014 PACCAR Inc
160
Fractional Factorials and MINITAB
• Notice that for a 5 factor experiment we have two fractional
factorial designs available
• Remember the aliasing for a Resolution III design untangling a Resolution III is beyond this course
HEY! It’s RED for a reason!
© 2014 PACCAR Inc
161
Fractional Factorial
Example
In-Class Exercise (Follow Along)
• You are working with a PACCAR supplier on improving their welding process
• Two components in an assembly must be welded to such a degree that the
pull-force required to separate them must exceed 1100 lbs. Currently this is not
the case and the PACCAR Belt and Supplier Belt are teamed to address the
issue.
• After completing the D & M stages, they are in the latter stages of Analyze. To
learn more a screening DOE is selected.
• With pull-force (PF) as the response, the factors are:
-1
+1
– Heat (H)
80
95
– Pressure (P)
5.0
7.5
– Weld Time (WT)
2.5
3.0
– Hold Time (HT)
1.0
2.0
– Squeeze Time (ST) 4.5
5.5
© 2014 PACCAR Inc
163
Build the matrix
in MINITAB
for a 25-1 fractional
Preparing the Experiment
• The process has been established to be stable at 1100 lbs with a
standard deviation of 61 lbs.
• Alpha risk = .10
• Response of Pull-Force measured with a calibrated Dillon Force
Gage (P/T = 8.4, %R&R = 9.7)
Noise Variables:
Method of Control:
– Condition of can & bracket
Clean before welding
– Condition of electrode
Use fresh
– Operator pacing
Record temp of electrodes
before each run
– Ambient Temp
Record
© 2014 PACCAR Inc
164
DOE Roadmap
(Page 1)
• Define objective of the experiment
• Design the experiment
– Stat > DOE > Factorial > Create Factorial Design
• Do the experiment
– Collect output data and add into Minitab
• Analyze the results for STATISTICAL significance
– Stat > DOE > Factorial > Analyze Factorial Design
– Identify KPIVs from Effects Plots and p values
– Reduce model if necessary and re-run
Design Options
Stat > DOE > Factorial > Create…
This table shows three options: Two fractional designs and
the full factorial design
© 2014 PACCAR Inc
166
Experimental Design
© 2014 PACCAR Inc
167
Experimental Design
© 2014 PACCAR Inc
168
DOE Roadmap
(Page 1)
• Define objective of the experiment
• Design the experiment
– Stat > DOE > Factorial > Create Factorial Design
• Do the experiment
– Collect output data and add into Minitab
• Analyze the results for STATISTICAL significance
– Stat > DOE > Factorial > Analyze Factorial Design
– Identify KPIVs from Effects Plots and p values
– Reduce model if necessary and re-run
Response Data Pull-Force
PF
1194
871
764
1463
1205
1256
616
1384
1152
1398
533
1382
1170
920
776
1410
© 2014 PACCAR Inc
170
Enter the
experiment data
into the
Spreadsheet then
build a Pareto
and Normal plot
DOE Roadmap
(Page 1)
• Define objective of the experiment
• Design the experiment
– Stat > DOE > Factorial > Create Factorial Design
• Do the experiment
– Collect output data and add into Minitab
• Analyze the results for STATISTICAL significance
– Stat > DOE > Factorial > Analyze Factorial Design
– Identify KPIVs from Effects Plots and p values
– Reduce model if necessary and re-run
Initial Analysis Results
Review the
Graphs for
Visual
Significance
•
© 2014 PACCAR Inc
172
Review the Session Window
No P-values!
We need to reduce
the model
Reduced Model
Final reduced
model is
without WT and
HT factors
Reduced Model Pareto
Diagnosis
If we want a high Pull Force Output we set the
factors: H (95), P (5.0), ST (5.5)
Diagnosis
There are 3 Significant Main Effects (H, P, ST), but each of these is
involved with one of the Two 2-way interactions.
This is visual display of the interactions. Always verify with P-values
DOE Roadmap
(Page 2)
• Analyze the results for PRACTICAL significance (if needed)
– Go into Results and turn on Expanded Tables
– Re-run the analysis by clicking “OK”
– % Contribution will identify dominant KPIVs
• Optimize the process
– Stat > DOE > Factorial > Factorial Plots
– Stat > Tables > Descriptive Statistics (Cross Tab)
– Select best input settings to meet requirements
• Investigate further (2nd DOE, Center Points, Blocking, etc.)
Practical Significance
In the Results tab choose the “Expanded table”
Remember: ALL this information with half the runs of a 25
© 2014 PACCAR Inc
179
Reduced Model Diagnostics
© 2014 PACCAR Inc
180
DOE Roadmap
(Page 2)
• Analyze the results for PRACTICAL significance (if needed)
– Go into Results and turn on Expanded Tables
– Re-run the analysis by clicking “OK”
– % Contribution will identify dominant KPIVs
• Optimize the process
– Stat > DOE > Factorial > Factorial Plots
– Stat > Tables > Descriptive Statistics (Cross Tab)
– Select best input settings to meet requirements
• Investigate further (2nd DOE, Center Points, Blocking, etc.)
Re-Run DOE on the Reduced Model
(WT & HT Removed)
•
© 2014 PACCAR Inc
•
•
Recall the original specification requirement
for Pull Force “must exceed 1100 lbs.”
Notice your options
Which
182 settings do you choose?
Review Variation Over the Three Factors
Stat > Tables > Descriptive Statistics
Review Variation Over the Three Factors
Review the Tables for Metrics
© 2014 PACCAR Inc
184
Review Variation Over the Three Factors
The best mean AND the
least Std Dev in Pull-force
results from settings of:
H at 95
P at 7.5
ST at 4.5
ALL this information with half the runs of a 25
© 2014 PACCAR Inc
185
DOE Roadmap
(Page 2)
• Analyze the results for PRACTICAL significance (if needed)
– Go into Results and turn on Expanded Tables
– Re-run the analysis by clicking “OK”
– % Contribution will identify dominant KPIVs
• Optimize the process
– Stat > DOE > Factorial > Factorial Plots
– Stat > Tables > Descriptive Statistics (Cross Tab)
– Select best input settings to meet requirements
• Investigate further (2nd DOE, Center Points, Blocking, etc.)
Other Fractional Factorial Design Considerations
• Fractional Factorials can be blocked and use center points, just as in
full factorials
• Fractional Factorials can be “folded over” to add to the design
– Example: A half-fraction folded over can become a full factorial
with two blocks. Folding over is just change the signs of the
original fraction and rerunning the experiment.
• Using sequential assembly of fractions, effects can be isolated from
other confounded effects
• Reduced fractions can become full factorials with replication
• These are advanced topics to be covered in future courses.
© 2014 PACCAR Inc
187
Fractional Factorial
Exercise
EXERCISE
•
Your project has 6 possible key variables.
•
A DOE would be a great way to clarify the situation.
•
You decide to conduct a fractional factorial (26-1) because you just
won’t get the time or money to run a full factorial.
•
After Week 3 you’re still not sure this is a value-added method but you
dive in and hope for the best.
•
The good news is that the situation is a Resolution V+!
•
The factors: Assembly Technique, Operator Experience, Build Paper
Layout, Tools, Line Speed, Number of Operators
•
The data is laid out in “standard order” (no randomization).
•
The response: Cycle-time (32 data points shown at right)
•
Standard questions at this point…INCLUDING
•
What would you recommend be done next?
© 2014 PACCAR Inc
189
CT
2
4
2
3
1
4
1
5
2
4
2
5
1
3
2
3
1
4
2
5
1
4
2
3
1
4
2
5
1
4
2
5
Attribute DOE
Objectives
• To review the difference between Defects and Defectives
• To introduce methods for appropriate DOE analysis when
using attribute response variables
• To demonstrate how data transformation reduces the
number of samples per treatment combination
• Demonstrate these ideas through examples
© 2014 PACCAR Inc
191
Attribute Data - A Review
• DEFECTIVE: “Bad or Good” Classification
•Defective counts take on integer values of 0, 1, 2, …,
n
•Number of Defectives can not exceed the number of
units measured (n)
• DEFECT: “Number Count”
•Theoretically no limit to the number of defects
•Defect counts take on integer values of 0, 1, 2, …..,
with no upper limit
•1 or more “defects” makes an item “defective”
© 2014 PACCAR Inc
192
Standard DOE Philosophy
• Efficient way of learning about KPIV’s and Noise
• DOE design is based on current knowledge (Full vs.
Fractional strategy)
• The effect of the factors on the response allows for
process optimization
 Key Assumption
– Analysis of factorial designs assumes the response is
measured on a continuous scale and that it has
constant variance
• Factor levels are set such that variation is induced
© 2014 PACCAR Inc
193
Attribute DOE
for Defectives
Data Transformation - Defectives
• Defective data has a “Binomial distribution” rather than
a Normal distribution
• There must be a chance (p) of getting “Bad” units and a
chance (1-p) of getting “Good” units
• For a sample size of n and a proportion defective of p:
Mean () = np
Variance (s2 ) = np (1- p)
• For a sample of 100 where 20% are defective, what is
the mean and variance?
© 2014 PACCAR Inc
195
Percent Defective As An Attribute (Discrete)
• Percentage defective is calculated from discrete data but the
format (decimal) gives the impression it is continuous.
• Because it looks continuous many feel they can avoid the
large sample sizes needed for discrete data
• However, percent defective can cause problems…
– When the sample is too small to detect the percentage.
•A 1% defect rate will probably NOT be seen in a sample
of 10 items
•A 1% defect rate might be seen in a sample of 100
•More would be recommended
© 2014 PACCAR Inc
196
More Issues…
• A subgroup is a “packet” of
data from which we can
calculate our percent defective.
• If the subgroup is small
(< 100) the variance is not
constant across subgroups
Variance
• Ex: For n=20 the variance of
the data will change as “p”
changes - see the graph
• The Freeman-Tukey
transformation gives a great
option - small samples & stable
variance
a - Untransformed data
c - Freeman Tukey data
© 2014 PACCAR Inc
197
Error Term or Variance Term
• ANOVA and Error
– ANOVA uses changes in the variance (error) as an
indication of a difference between samples.
– Variance is expected to be constant
• Heteroskedasticity
– a data sample or data-generating process in which the
errors are drawn from different distributions for different
values of the independent variables.
•variance of income across individuals is systematically
higher for higher income individuals
© 2014 PACCAR Inc
198
Variance
Variance Functions for Sample Sizes of N = 20 and N = 50
for Binomial Distributed Proportions
a & d : No Transformation used (variation changes as p changes)
b & e : Arcsin Square Root Transformation
c & f : Freeman - Tukey Modification
© 2014 PACCAR Inc
199
The Freeman-Tukey Transformation
(Open Freeman - Tukey.xls)
• We can defend against false signals using a
transformation
• We transform the Y data that was calculated as a
proportion defective (not percent) during the
experiment
sin 1
Y
Np
 sin 1
N 1
2
Np  1
N 1
• This will minimize the inherent variation shown in the
graphs on the previous page and allow us to identify
the variation due to the KPIVs
© 2014 PACCAR Inc
200
Example: Transactional Attribute DOE
• The finance team wants to see what causes
people to default on their loan.
• Three factors were felt to be possible
contributors.
• Response: Percentage of defaulted loans.
• Twenty customers are assessed for each
treatment combination.
• We “talk” to management in percentages but
run the experiment as proportions.
• Only 1 replicate is run for each set up.
© 2014 PACCAR Inc
201
Example: Transactional Attribute DOE
• Run the DOE at the
right
Proportion
1. Without
transformation
2. With transformation
• How do they
compare and why?
A
B
C
-1
-1
-1
0.05
1
-1
-1
0.20
-1
1
-1
0.10
1
1
-1
0.35
-1
-1
1
0.15
1
-1
1
0.25
-1
1
1
0.05
1
1
1
0.15
NOTE: For the transformation, N = 20
© 2014 PACCAR Inc
202
Default
Example: Manufacturing Attribute DOE
• A supplier of integrated circuits for a new PACCAR fuel control
was experiencing an increase in wire bonds “popping off” (20%) each pop-off breaks the circuit and causes the unit to fail
• A fractional factorial experiment was performed to try and
determine the KPIV’s related to the problem
– Five factors and the response of “% bonds popped off” were
evaluated
– Product was available to allow for 16 experimental runs
– 10 IC’s, each with 100 wire bonds were used at each treatment
combination (10 x 100 = 1000 possible pop-offs)
• The design matrix and “raw data” are on the next page
© 2014 PACCAR Inc
203
Example: Transactional Attribute DOE
0.10
A
B
C
D
E
P
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1
-1
-1
1
1
-1
-1
1
1
-1
-1
1
1
-1
-1
1
1
-1
-1
-1
-1
1
1
1
1
-1
-1
-1
-1
1
1
1
1
-1
-1
-1
-1
-1
-1
-1
-1
1
1
1
1
1
1
1
1
1
-1
-1
1
-1
1
1
-1
-1
1
1
-1
1
-1
-1
1
0.10
0.30
0.29
0.13
0.00
0.00
0.09
0.00
0.34
0.24
0.30
0.21
0.30
0.14
0.05
0.20
0.30
0.29
0.13
0.00
0.00
0.09
0.00
0.34
0.24
0.30
0.21
0.30
0.14
25-1 half-fraction factorial with 1 replicate.
NOTE: Y is expressed as a proportion not a percent!
© 2014 PACCAR Inc
204
0.05
0.20
Example: Transactional Attribute DOE
• Analyze both the “raw” data (p) and the transformed data
(Freeman-Tukey)
NOTE: N =1000
Stat > DOE > Factorial > Analyze
Select all factors!
• What are your analytical conclusions?
– What factors are important?
– What factors will you keep in your model?
• Are your conclusions different for the transformed data?
• Why?
© 2014 PACCAR Inc
205
Attribute DOE
for Defects
Moving From Defectives to Defects
• The previous examples were proportions of items which
either had a trait or did not.
– Good Vs. Bad
– Pass Vs. Fail
– Not Late Vs. Late
• Another possibility is the number of defects that make an
item defective
– Scratches on surfaces
– Typos on documents
• Although both use an attribute scale of measurement they
are not exactly the same
© 2014 PACCAR Inc
207
Dealing with DEFECTS
(As Opposed to Defectives)
• Assume…
• Probability of getting a defect in any subgroup is constant
from subgroup-to-subgroup
• Chance of a defect in any subgroup is independent of
getting a defect in any other subgroup
Defect Rate

Lambda again!
Poisson Distribution
Mean () = 
© 2014 PACCAR Inc
Variance (s2 ) = 
208
Dealing with DEFECTS
Freeman - Tukey Transformation
c  c 1
Y
2
See the other tab of Freeman-Tukey.xls
It’s marked “Defect”.
© 2014 PACCAR Inc
209
Variance
Variance Functions for Poisson Distributed Counts
© 2014 PACCAR Inc
a : No Transformation used
b : Square Root of Counts Transformation
c : Freeman - Tukey Modification
210
Transactional Attribute DOE
• Dealers are trying to decide
the best way to display their
wares.
Lighting Location Facing Inquiries
-1
-1
-1
58
1
-1
-1
44
-1
1
-1
60
1
1
-1
65
– Lighting
-1
-1
1
36
– Location on the floor
1
-1
1
41
-1
1
1
50
1
1
1
46
• 3 conditions are investigated
to determine if they make a
difference
– Facing relative to the
entrance
• Analyze the results
• What do you recommend?
© 2014 PACCAR Inc
211
This Is All Cool, BUT There Is A Price
• To get the same Power…
• The sample sizes required for Attribute DOE are
much larger than with a Variables DOE.
© 2014 PACCAR Inc
212
MINITAB Sample Size Calculation
• We can use MINITAB’s Power and Sample Size
calculation methods to determine sample size
for proportions
• Since we are dealing with two ‘samples’ (our
reference distribution and the new distribution),
we will use the 2-Proportions function
• Stat> Power and Sample Size> 2 Proportions
© 2014 PACCAR Inc
213
MINITAB Sample Size Calculation
Let’s first evaluate the sample
size required with varying
power values
Our original proportion (1) is
20% (0.20)
Our ‘new’ proportion (2) is
10% (0.10)
We need 266 data points
per treatment combo
for a Power of .90
© 2014 PACCAR Inc
214
Bottom-Line on Attribute DOE
• If you can avoid it, do so.
– Select instead a continuous scale response DOE
• If you must do it, choose sufficiently large sample sizes.
– This is typically 100+ per treatment combination
• Once done, transform the response using the appropriate
tab on the Freeman-Tukey.xls spreadsheet
– Especially important if your samples are small (< 100)
Attribute DOEs are another example of discrete data
looking easy initially but ending up to be a real pain.
© 2014 PACCAR Inc
215
Simulation and Modeling
Wrap Up of Activities
Class Project Debriefs:
Catapult
Exam Preparation
Closing Comments
Think about what you’ve accomplished in
the past four months…
Work Expectations
• Functional Black Belt
– Help develop and lead cross-functional process improvement teams
within their own functional work area
– Apply Six Sigma tools to improve process capability and eliminate
defects
– Transfer best practices, techniques, procedures and tools to peers
and other process improvement teams
– Work with cross-functional teams to recommend and implement
process improvement plans
– Provide analysis and feedback to senior management on project
activities, improvements, and savings
– Demonstrate successful application of new methodologies to
improve product and process quality
© 2014 PACCAR Inc
224
Work Expectations
• Dedicated Black Belt
– Member of Corporate Quality Department, focused on Six Sigma
– Help develop and lead cross-functional process improvement teams
– Mentor and support Black/Green Belts
– Transfer projects to other PACCAR locations where appropriate with
assistance of MBB
– Transfer best practices, techniques, procedures and tools to peers and
other process improvement teams
– Work with cross-functional teams to recommend and implement
process improvement plans
– Provide analysis and feedback to senior management on project
activities, improvements, and savings
– Demonstrate successful application of new methodologies to improve
product and process quality
© 2014 PACCAR Inc
225
Certification
“Trained Black Belt”
versus
“PACCAR Certified Black Belt”
• Successful completion of BB training
• Successful closure of 2 BB-level projects
• Successful co-leadership of HIKE or large Kaizen
• Successful completion of BB Certification Exam
© 2014 PACCAR Inc
226
Our mission is to develop highly skilled leaders
who will continuously improve business
processes, products and services to contribute
to the customer’s and the company’s success!
© 2014 PACCAR Inc
227
Press On… with Persistence and Determination!
“Nothing in the world will take the place of persistence.
Talent will not; nothing is more common than the
unsuccessful person with talent.
Genius will not; unrewarded genius is almost a proverb.
Education will not; the world is full of educated derelicts.
Persistence and determination alone are omnipotent.
The slogan “press on” has solved and always will solve the
problems of the human race.”
Calvin Coolidge
Thank you Black Belts!