Adventures in industry
Sue Lewis
Southampton Statistical Sciences Research Institute
University of Southampton
sml@maths.soton.ac.uk
Outline
• Experiments on many factors
- with Jaguar Cars
- using two-stage group screening
- to find the important factors
• Experiments on assembled mechanical products
- where values of factors cannot be set
- with Hosiden Besson, Sauer Danfoss, Goodrich
• Software for implementing the methods
Cold Start Optimisation
Factors Affecting Performance
Control (or design) factors – can be set by the engineers
Noise factors - cannot be controlled in use
eg ambient temperature
- can be controlled in an experiment
Aim: find the control factor settings that
• Optimise the performance (engine starts - resistance)
• Minimize variability in performance
- due to the varying noise factors
- Deming, Taguchi
Want to detect
control x noise interactions
Response
High
Noise
Low
Low
Control
High
Also main effects and control x control interactions
For conventional factorial designs
large number of factors  large number of runs
Classical Solution
• Run an experiment to estimate only main effects
- identify the important factors
• For the important factors, run an experiment
- to estimate both main effects and interactions
Disadvantage: could miss factors that interact with noise
Response
High
Noise
Low
Low
Control
High
Grouping factors
• Arrange the factors in groups
• Label the factor levels
high - larger response anticipated
low
- smaller response anticipated
• For each group define a new grouped factor with two levels
high - all factors in group high
low - all factors in group low
• Experiment on the grouped factors
Two Stage Group Screening
Stage 1: perform an experiment on the grouped factors
to decide which groups are important
- estimate main effects and/or interactions
Stage 2: dismantle those groups found to be
important and experiment on their individual factors
- estimate both main effects and interactions
Gathering Information from Experts
Opinions on
• Factors that might be included in the experiment
- and their levels
• The likely importance of each factor
• The direction of each main effect
• Any insights/experience on interactions
Local brainstorming – but experts often at different sites
Web-based System (GISEL)
• Gathers opinions/suggestions on factors and their levels
- via a dynamic questionnaire
- with free form comments
• Keeps a record of opinions, experiments and results
• Guides factor groupings via software that
- explores the resources needed for various strategies and
factor groupings
- estimates the risk of missing important factors through
simulation of experiments
Factors under Consideration
Summary of Opinions on Air to Fuel Ratio
Making a decision on groupings
Assess possible grouping strategies
- resource required
- risk of missing an important factor
Individual factors are classified as
Very likely to be active
Less likely to be active
Not worth including
Probabilities assigned
eg 0.7 and 0.2
Ten Factors for the Experiment
Control – very likely
Plug type*
Plug gap*
Air fuel ratio
Injection timing
Noise
Temperature
Injector tip leakage
Control – less likely
Spark during crank
Spark time during run-up
Higher idle speed
Idle flare
* hard-to-change: grouped together
Investigation of different groupings
Plan for the First Stage (10 factors)
Control factors:
Group 1: Plug type* & Plug gap*
Group 2: Air to fuel ratio & Injection timing
Group 3: Spark time during crank & During run-up
Group 4: Higher idle speed & Idle flare
Noise factors:
Group 5: Injector tip leakage
Group 6: Temperature
Design:
Half-replicate (I=123456) in 4 sessions of 8 runs
Results of First Stage Experiment
Included large interactions
(Afr & Injection timing) x Temperature
(Higher idle speed & Idle flare) x Injector tip leakage
- both grouped control x noise interactions
 6 factors to investigate at the Second Stage Experiment
Second Stage Experiment
Design
• Half-replicate in 32 runs (I = ABCDEF)
- for the individual factors
- could have been smaller
Preliminary findings include
• Air to Fuel Ratio x Temperature is large
• Possible three factor interaction
Experiments on assembled products
Acoustic sounder
Hosiden Besson
front case
armature
Aim: mean sound output
close to target
with reduced variation
magnet
diaphragm
Gear pump
gear pack
Aim: reduce mean leakage and variation in leakage
- under varying pressure and speed
Possible approaches
• Factorial experiments
- set factors to values specified in the design
Obtain parts with required factor values by
- making special components
- measuring large samples and using components with
required factor values
For our examples: too slow and costly
• Disassembly/reassembly experiments (Shainin)
In our examples: cannot reuse components
Our Approach
• Take a sample of each kind of component from
production
• Measure the relevant component variables
• Assemble the components to form a set of products for
testing
– to maximise information on the factors of interest
Factors
• Directly measurable on a component
- eg permeability of the armature in the sounder
• Formed or derived as a function of measured quantities
on two or more components
- eg gaps between components in the assembled product
- cannot be handled by conventional designs
• Factors that can be set
- eg the skill of the operator in making certain adjustments
during the manufacture of the sounder
To design the experiment
-must decide which set of products to assemble
• There is a huge number of possibilities
Eg For 4 components (pump gear pack) and sufficient parts
to assemble 12 products
- the number of possibilities is ~ 12x1035
• Needs a non-standard search algorithm that
- finds an efficient set of assemblies
- allows for the non-reuse of components
- accommodates conventional factors
Finding a design
Use a specially developed search algorithm with
- a low order polynomial to describe the response
- a design chosen for accurate estimation of the
coefficients of the model (D-optimality)
Software (DEAP) has been developed that
- assists with product and component definition
- provides access to the design algorithm
Software to Implement the Methods
(DEAP)
Software to Implement the Methods
(DEAP)
Results from the studies
The most important factors for improving the product
performance were:
For the sounder : the pip height and skill of operator
For the pump: positioning of the cover and the alignment of gears
1
.
0
P
r
e
s
s
u
r
e
1
1
0
8
0
5
0
0
.
8
MeanLkge
0
.
6
0
.
4
0
.
1
0
.
1
0
.
3
E
n
d
f
l
o
a
t
Conclusions
• Tools and methods developed in collaboration with
industry for two kinds of experiments
- large numbers of factors
- assembled products
• Software at the beta testing stage
- freely available
Some related references
Atkinson, A.C. and Donev, A.N. (1992) Optimum Experimental Designs. Oxford:
Oxford University Press.
Dean, A.M. and Lewis, S.M. (2002) Comparison of group screening strategies for
factorial experiments. Computational Statistics and Data Analysis, 39, 287297.
Deming, W.E. (1986) Out of the Crisis. Cambridge: C.U.P.
Dupplaw, D., Brunson, D., Vine, A.E., Please, C.P., Lewis, S.M., Dean, A.M.,
Keane, A.J. and Tindall, M.J. (2004) A web-based knowledge elicitation
system (GISEL) for planning and assessing group screening experiments for
product development. To appear in J. of Computing and Information Science
in Engineering (ASME).
Harville, D. A. (1974) Nearly optimal allocation of experimental units using
observed covariate values. Technometrics 16, 589-599.
Some related references
O’Neill, J.C., Borror, C.M., Eastman, P.Y., Fradkin, D.G., James,
M.P., Marks, A.P. and Montgomery, D.C. (2000) Optimal
assignment of samples to treatments for robust design. Qual. Rel.
Eng. Int. 16, 417-421.
Lewis, S.M. and Dean, A.M. (2001) Detection of Interactions in
Experiments with large numbers of factors (with discussion). J.
Roy. Statist. Soc. B, 63, 633-672.
Sexton, C.J., Lewis, S.M. and Please, C.P. (2001) Experiments for
derived factors with application to hydraulic gear pumps J. Roy.
Statist. Soc. C, 50, 155-170.
Shainin, R.D. (1993) Strategies for technical problem solving. Qual.
Eng., 433-448.
Taguchi, G. (1987) System of Experimental Design. New York: Kraus.