FTC 2005 Abstracts
Thursday, 9:15 to 10: 00 am
Logarithmic SPC Session
An SPC Control Chart Procedure Based on Censored Lognormal Observations
Uwe Koehn
Koehn Statistical Consulting LLC
Purpose: To present an SPC control chart procedure based on censored lognormal
observations
Abstract: A bicycle manufacturer was interested in testing whether or not the lifetimes of
purchased bicycle frames were in statistical control. Based on prior data, it was
determined that the lifetimes had a lognormal distribution. Censoring the observations
was desirable since time till failure can be very long, especially when the process is in
control or better. A procedure using censoring was developed and its properties
explored. Graphical techniques, a corresponding method for spread, and sequential
methods will be discussed. The procedure is seen to compare very favorably to using
non-censored data. From a practical standpoint, the procedure using censoring is better
not only from a time standpoint, but usually because it better models the data.
Robust Parameter Design Session
Process Optimization through Robust Parameter Design in the Presence of
Categorical Noise Variables
Timothy J. Robinson,
University of Wyoming
William A. Brenneman
William R. Myers,
The Procter & Gamble Company
Purpose: To demonstrate the use of response surface methods for robust design
categorical noise factors are present and when some control factors have levels which
are not equally desirable due to cost and/or time issues.
Abstract: When categorical noise variables are present in the Robust Parameter Design
(RPD) context, it is possible to reduce process variance by not only manipulating the
levels of the control factors but also by adjusting the proportions associated with the
levels of the categorical noise factor(s). When no adjustment factors exist or when the
adjustment factors are unable to bring the process mean close to target, a popular
approach to determining optimal operating conditions is to find the levels of the control
factors which minimize the estimated mean squared error of the response. Although this
approach is effective, engineers may have a difficult time translating mean squared error
into quality. We propose the use of a parts per million (PPM) defective objective function
in the case of categorical noise variables in the RPD setting. Furthermore, we point out
that in many situations the levels of the control factors are not equally desirable due to
cost and/or time issues. We have termed these types factors non-uniform control
factors. We propose the use of desirability functions to determine optimal operating
conditions in the RPD with categorical noise factor setting when non-uniform control
factors are present and illustrate this methodology with an example from industry.
Business Process Modeling Session
Business Process Characterization Using Categorical Data Models
Cathy Lawsom
General Dynamics
Douglas Montgomery
Arizona State University
Purpose: The purpose of this paper is to demonstrate how logistic regression is
used to model a complex business process.
Abstract: Variation exists in all processes especially business processes where critical
input and process variables may be controlled by human intervention. Significant returns
may be realized by identifying and removing sources of variation from business
processes. Because business processes tend to be heavily dependent on human
interaction, they can be difficult to characterize and model. This research develops a
methodology for synthesizing the qualitative information about the performance of a
business process and transforming it into specifically defined categorical data that can
be used for statistical modeling and optimization. The process under investigation is the
identification and pursuit of new business opportunities for a Department of Defense
(DoD) prime contractor. This process is heavily dependent on the people who obtain
information about potential opportunities and make decisions about whether to pursue
an identified opportunity. This research explores methods for taking the demographic,
anecdotal and qualitative data associated with particular business opportunities and
creating categorical data sets that can be statistically modeled. This research illustrates
how binary logistic regression was used to analyze these data and establish significant
relationships between these key process attributes and the process outcome which is
either the win or loss of the opportunity.
Thursday, 10:30 to 12:00 am
Mulitvariate SPC Session
Using Nonparametric Methods to Lower False Alarm Rates in Multivariate
Statistical Process Control
Luis A Beltran
Linda Malone
University of Central Florida
Purpose: To devise a systematic distribution-free approach by extending current
developments and focusing on the dimensionality reduction using Principal Component
Analysis (PCA) without restricting the technique or techniques to normality requirements.
Abstract: Although there has been progress in the area of multivariate SPC, there are
numerous limitations as well as unanswered questions with the current techniques.
MSPC charts plotting Hotelling’s T 2 require the normality assumption for the joint
distribution among the process variables, which is not feasible in many industrial
settings. The motivation to investigate nonparametric techniques for multivariate data in
quality control is that fewer restrictive assumptions of the data are imposed; as such the
assumption of normality is not a requirement. The nonparametric approach is also less
sensitive to outliers, hence more robust and prone to fewer false alarms. In this
research, the goal will be to create a systematic distribution-free approach by extending
current developments and focusing on the dimensionality reduction using PCA without
restricting the technique or techniques to normality requirements. The proposed
technique is different from current approaches in that it creates a unified distribution-free
approach to non-normal multivariate quality control. The proposed technique will be
better based on its ease of use and robustness to outliers in MSPC. By making the
approach simple to use in an industrial setting, recommendations on the process could
be obtained efficiently, resulting in a cost savings and consequently improving quality.
Statistical Monitoring of Dose-Response Quality Profiles from High-Throughput
Screening
James D. Williams
General Electric
Jeffrey B. Birch,
William H. Woodall,
Virginia Tech
Purpose: To present a statistical monitoring procedure for researchers in
pharmaceutical and chemical companies to evaluate the quality of their bioassay test
procedures over time.
Abstract: In pharmaceutical drug discovery and agricultural crop product discovery, in
vitro bioassay experiments are used to identify promising compounds for further
research. The reproducibility and accuracy of the bioassay is crucial to be able to
correctly distinguish between active and inactive compounds. In the case of agricultural
product discovery, a replicated dose response of commercial crop protection products is
assayed and used to monitor test quality. The activity of these compounds on the test
organisms, the weeds, insects, or fungi, is characterized by a dose-response curve
measured from the bioassay. These curves are used to monitor the quality of the
bioassays. If undesirable conditions in the bioassay arise, such as equipment failure or
problems with the test organisms, then a bioassay monitoring procedure is needed to
quickly detect such issues. In this paper we illustrate a proposed nonlinear profile
monitoring method to monitor the within variability of multiple assays, the adequacy of
the dose-response model chosen, and the estimated dose-response curves for aberrant
cases. We illustrate these methods with in vitro bioassay data collected over one year
from DuPont Crop Protection.
Topics In DOE Session
Bayesian Analysis of Data from Split-Plot Designs
Steven G. Gilmour
University of London
Peter Goos
Universiteit Antwerpen
Purpose: In this talk, a reliable method to analyze data from split-plot experiments will
be presented.
Abstract: Often, industrial experiments involve one or more hard-to-change variables
which are not reset for every run of the experiment. The resulting experimental designs
are of the split-plot type and fall in the category of multi-stratum designs (see, for
example, Trinca and Gilmour (2001)). A proper classical statistical analysis requires the
use of generalized least squares estimation and inference procedures and, hence, the
estimation of the variance components in the statistical model under investigation.
In most split-plot or multi-stratum designs utilized in practice, the hard-to-change
variables are reset a small number of times, such that estimation of the variance
components corresponding to the whole plots stratum of the experiment is either
impossible or inefficient. As a consequence, these variance components are estimated
by most packages to be zero (see, for example, Goos, Langhans and Vandebroek
(2004)) and the generalized least squares inferences collapse to ordinary least squares
ones. The resulting statistical analysis may lead to erroneous decisions regarding the
significance of certain effects in the model.
In the presentation, it will be shown that this problem can be avoided by incorporating
the researcher’s prior beliefs regarding the magnitude of the variance components in a
Bayesian analysis of the data. This approach guarantees a more correct analysis of the
data as it will only produce ordinary instead of generalized least squares results if the
data contain enough information to contradict the researcher’s prior. A split-plot
experiment conducted at the University of Reading to identify the factors influencing the
aroma of freeze-dried coffee will be used as an illustration.
How will the status quo be changed?
There is still a lot of research on the topic of analyzing industrial split-plot and other
multi-stratum experiments. The research described here shows that it is essential to
incorporate prior knowledge regarding the variance components in the analysis.
Otherwise, the statistical analysis of the data may be flawed.
Adapting Second Order Designs for Specific Needs: a Case Study
James R Simpson.
FAMU-FSU
Drew Landman
Old Dominion University
Rupert Giroux
FAMU-FSU
Purpose: To develop an experiment and analysis method for efficiently characterizing
and calibrating large load strain gauge balances for detecting aerodynamic forces and
moments on aircraft models in wind tunnel testing
Abstract: Strain gauge balances can be used to capture aerodynamic forces and
moments on aircraft and other vehicles tested in wind tunnels. These balances must be
calibrated periodically using static load testing. The calibration testing procedure at
NASA was originally developed during the 1940’s and is based on a modified one-factorat-a-time method that is time intensive and does not provide a statistically rigorous
estimate of model quality. An approach using experimental design is proposed to
characterize the relationships between applied load (force) and response voltages for
each of the six aerodynamic forces and moments. A second order experimental design
based on the structure of the Box Behnken design was constructed to suit the unique
requirements of the calibration operation. Monte Carlo simulation was used to compare
the proposed design’s performance potential relative to existing designs prior to
conducting a set of actual experiments. Lessons were learned in constructing
nonstandard designs for second order models and in leveraging simulation effectively
prior to live testing. The new calibration process will require significantly fewer tests to
achieve the same or improved precision in characterization.
Six Sigma Session
Six Sigma beyond the Factory Floor
Ron Snee
Tunnell Consulting
Abstract: Six Sigma is a process-focused, statistically-based approach to business
improvement that companies as diverse as Motorola, Honeywell, General Electric,
DuPont, 3M, American Express, Bank of America and Commonwealth Health
Corporation have used to produce millions of dollars in bottom-line improvements.
Initially Six Sigma initiatives were focused on improving the performance of
manufacturing processes. Today there is a growing awareness that additional gains in
efficiency and operational performance can be achieved by widening the scope of Six
Sigma beyond the factory floor to include the improvement of non-manufacturing,
administrative and service functions. The economic advantage of such efforts are
potentially very significant because (a) the administrative component of modern
manufacturing is large and has a great influence on overall economic performance, and
(b) the service industry is in a modern society well over two-thirds of the entire economy.
This presentation will discuss how Six Sigma can be used to improve processes beyond
the factory floor including dealing with the “We’re different” barrier, appropriate
roadmaps and common technical challenges for the methods and tools used. Several
illustrative examples will be presented
Some Trends in Six Sigma Education
Douglas Montgomery
Arizona State University
Thursday, 2:00 to 3:30 pm
TECHNOMETRICS Session
Control Charts and the Efficient Allocation of Sampling Resources
Marion R Reynolds, Jr.
Virginia Tech
Zachary G. Stoumbos
Rutgers University
Abstract: Control charts for monitoring the process mean  and process standard
deviation  are often based on samples of n > 1 observations, but in many applications,
individual observations are used (n = 1). In this paper, we investigate the question of
whether it is better, from the perspective of statistical performance, to use n = 1 or n > 1.
We assume that the sampling rate in terms of the number of observations per unit time
is fixed, so using n = 1 means that samples can be taken more frequently than when
n > 1. The best choice for n depends on the type of control chart being used, so we
consider Shewhart, exponentially weighted moving average (EWMA), and cumulative
sum (CUSUM) charts. For each type of control chart, a combination of two charts is
investigated; one chart designed to monitor , and the other designed to monitor .
Most control chart comparisons in the literature assume that a special cause produces a
sustained shift in a process parameter that lasts until the shift is detected. We also
consider transient shifts in process parameters, which are of a short duration, and drifts
in which a parameter moves away from its in-control value at a constant rate. Control
chart combinations are evaluated using the expected detection time for the various types
of process changes, and a quadratic loss function. When a signal is generated, it is
important to know which parameters have changed, so the ability of control chart
combinations to correctly indicate the type of parameter change is also evaluated. Our
overall conclusion is that it is best to take samples of n = 1 observations and use an
EWMA or CUSUM chart combination. The Shewhart chart combination with the best
overall performance is based on n > 1, but this combination is inferior to the EWMA and
CUSUM chart combinations on almost all performance characteristics (the exception
being simplicity). This conclusion seems to contradict the conventional wisdom about
some of the advantages and disadvantages of EWMA and CUSUM charts, relative to
Shewhart charts.
The Inertial Properties of Quality Control Charts
William H. Woodall
Virginia Tech
Mahmoud A.Mahmoud
Cairo University
Abstract: Many types of control charts have an ability to detect process changes that
can weaken over time depending on the past data observed. This is often referred to as
the “inertia problem.” We propose a new measure of inertia, the signal resistance, to be
the largest standardized deviation from target not leading to an immediate out-of-control
signal. We calculate the signal resistance values for several types of univariate and
multivariate charts. Our conclusions support the recommendation that Shewhart limits
should be used with EWMA charts, especially when the smoothing parameter is small.
DOE for Computer Simulation Session
Application of Design of Experiments in Computer Simulation Studies
Shu Yamada
Hiroe Tsubaki
University of Tsukuba
Purpose: This paper presents an approach of design of experiments in computer
simulation with some case studies in automobile industry.
Abstract: In recent days, computer simulation has been applied in many fields, such as
Computer Aided Engineering in manufacturing industry and so forth. In order to apply
computer simulation effectively, we need to consider the following two points: (1)
Exploring a model for computer simulation, (2) Effective application of simulation based
on the explored model. As regard (1), once a tentative model is derived based on
knowledge in the field, it is necessary to examine validity of the model. At this
examination, design of experiments plays an important role. After exploring a computer
model, the next stage is (2), such as optimization of the response by utilizing computer
simulation. This paper presents an approach of design of experiments in computer
simulation in terms of (1) and (2) with some case studies in automobile industry. For
example, in order to optimize a response by many factors, the first step may be
screening active factors from many candidate factors. Design of experiments, such as
supersaturated design, etc help at this screening problem. After fining some active
factors, the next step may be approximation of the response by an appropriate function.
Composite design, Uniform design is helpful to fit second order model as an
approximation.
Computer Experimental Designs to Achieve Multiple Objectives
Leslie M. Moore
Los Alamos National Laboratory
Purpose: Issues and strategies for designing computer experiments are reviewed for
conducting sensitivity analysis and construction of an emulator.
Abstract: Simulator codes are a basis for inference in many complex problems including
weapons performance, materials aging, infrastructure modeling, nuclear reactor
production, and manufacturing process improvement. Goals of computer experiments
include sensitivity analysis to gain understanding of the input space and construction of
an emulator that may form a basis for uncertainty analysis or prediction. Orthogonal
arrays, or highly fractionated factorial designs, and near-orthogonal arrays are used for
computer experiments for sensitivity analyses. Latin hypercube samples, possibly
selected by space-filling criterion, are in common use when Gaussian spatial processes
are the modeling paradigm or uncertainty analysis is the objective. Orthogonal-array
based Latin hypercube designs are used to achieve both objectives. Improvement in
terms of obtaining a space-filling design will be demonstrated for orthogonal-array based
Latin hypercube design. The impact of competing experiment objectives will be
discussed in terms of loss of efficiency in sensitivity analysis conducted with data from a
Latin hypercube design.
Common Mistakes in Statisical Applications Session
Common Mistakes When
Using SPC (and What to do About Them)
Douglas Fair
InfinityQS International, Inc.
Abstract:
Common Mistake Number 1
Incorrectly Using Related Data in a Subgroup. This discussion will center upon the
common mistake of using a subgroup of data points that have been collected from a
single part. For example, measuring a single can's height in 3 places and calling it a
subgroup. This violates one of the fundamental rules of subgrouping, that of data
independence. However, this error can be rectified by using the proper innovative control
chart, a "3D Control Chart."
Common Mistake Number 2
Combining multiple process streams in the same subgroup. This discussion involves
data collected from multiple machines, production lines or filling heads to be used on the
same control chart. In this case, if a single product weight value is collected from say, 5
production lines, and all 5 weight values are used within a subgroup, then the resulting
Xbar and Range control chart will be less than useful. In fact, alarms will be masked and
process control will be impossible to assess. However, this error can be rectified by
using the proper innovative control chart, a "Group Chart."
Each of these two topics will include discussions concerning
1. Real-life situations that have been encountered
2. Software that can be used to automatically enter data and display it on the innovative
charts discussed above.
Common Practitioner Mistakes in Data Analysis
Scott M. Kowalski
Minitab, Inc.
Purpose: To share some experiences with statistical misconceptions about data
analysis.
Abstract: In companies around the globe, the majority of data analysis is done by people
with very little training in statistics. For example, most green/black belts are given 3-5
weeks of Six Sigma training and only part of that training focuses directly on statistics.
This leaves the typical data analyst armed with a statistical software program, but
without sufficient savvy to know right from wrong. Unfortunately, many analysts make
mistakes when using everything from basic statistical tools to quality tools to design of
experiments. In this talk, I present some of the statistical misconceptions that I have
encountered over the last several years of traveling to companies that are using
statistics to analyze data.
Friday, 8:00 to 9:30 am
Journal of Quality Technology Session
A Dual-Response Approach to Robust Parameter Design for a Generalized Linear
Model
William R. Myers
William A. Brenneman
The Procter & Gamble Company
Raymond H. Myers
Virginia Tech
Absract: Robust Parameter Design (RPD) has been used extensively in industrial
experiments since its introduction by Genichi Taguchi. RPD has been studied and
applied, in most cases, assuming a linear model under standard assumptions. More
recently, RPD has been considered in a generalized linear model (GLM) setting. In this
paper, we apply a general dual-response approach when using RPD in the case of a
GLM. We motivate the need for exploring both the process mean and process variance
by discussing situations when a compromise between the two is necessary. Several
examples are provided in order to further motivate the need for applying a dual-response
approach when applying RPD in the case of a GLM.
Analysis of Performance Measures in Experimental Designs Using the Jackknife
Asokan Mulayath Variyath
Bovas Abraham
Jiahua Chen
University of Waterloo
Abstracts: Experimental designs with performance measures as responses are common
in industrial applications. The existing analysis methods often regard performance
measures as sole response variables without replicates. Consequently, no degrees of
freedom are left for error variance estimation in these methods. In reality, performance
measures are obtained from replicated primary-response variables. Precious information
is hence lost. In this paper, we suggest a jackknife-based approach on the replicated
primary responses to provide an estimate of error variance of performance measures.
The resulting tests for factor effects become easy to construct and more reliable. We
compare the proposed method with some existing methods using two real examples and
investigate the consistency of the jackknife variance estimate based on simulation
studies.
Response Surface Methodology Session
Comparison of Global Characterization Techniques in Response Surfaces
Francisco Ortiz Jr.
James R. Simpson
FAMU-FSU
Drew Landman
Old Dominion University
Purpose: A comparative study of different modeling techniques for the characterization
of systems with a
complex underlying response model.
Abstract: The optimization of multiple responses in experimental design requires
satisfying different, sometimes conflicting objectives. Individual objectives typically
correspond to optimality regions in different locations in the experimental design space.
The practitioner must then locate a region of the design space where the trade-off
between objectives is acceptable. In order to find these acceptable tradeoff regions it
may be useful to characterize the larger region of operability rather than the region
around the current operating setting. This expansion of the region of interest may involve
fitting complex, highly nonlinear surfaces. The purpose of this research is to perform
comparative study of relevant modeling techniques for the characterization of systems
with a complex underlying model. A case study from wind tunnel tests of a model X-31
aircraft is used to demonstrate the advantages and disadvantages of these techniques.
Results from this research could also prove useful for the single response situation
involving relatively large design spaces or optimization cases where it would be useful to
perform a global search in lieu of the traditional steepest ascent local search approach.
Response Surface Design Evaluation Using Mean Square Error Criteria
Christine Anderson-Cook
Los Alamos National Lab
Connie M. Borror
University of Illinois
Abstract: Evaluation of designs for response surface models has, for the most part,
been based on assessment of the scaled prediction variance (SPV). Bias and mean
square error received some attention as bases for choosing satisfactory designs in the
early 1990s, but have largely been ignored as design assessment criteria. A drawback
to the use of bias or mean square error as the evaluation criteria has been the difficulty
to visually assess these quantities over various design regions. This difficulty is a result
of the fact that the user must not only assume a particular form of the model of interest,
but must also state the form of the model to be protected against. As a result, numerous
graphical displays (such as variance dispersion graphs) would be needed to adequately
assess competing designs.
A recently developed graphical display, the fraction of design space (FDS) plot, will be
used to evaluate competing response surface designs using bias-related criteria. The
FDS plot was originally developed to display SPV over the operating region of interest
plotted against the fraction of design space. A single line summarizes the distribution of
the prediction variance by plotting SPV values versus the fraction of the design space at
or below that value. The FDS plot shows promise as an excellent way of showing
graphically the potential bias for various forms of assumed models and various
experimental designs. We will present the results of these comparisons and provide
recommendations.
Graphical Methods Session
Using a Pareto Chart to Select Effects for a Two-Level Factorial DOE
Pat Whitcomb
Stat-Ease, Inc.
Purpose: To present a new method of using a Pareto chart of t-values for properly
displaying relative effect sizes, along with t-limits that aid in the selection of effects.
Abstract : In the past, we have exclusively recommended using the half-normal plot of
effects for identification of potentially significant effects. However, if constructed
properly, the Pareto chart is a useful tool for showing the relative size of effects,
especially to non-statisticians. Problems with either technique arise if the design is not
an orthogonal balanced 2k-p design. In these situations the effects have differing
standard errors, thus the size of the effect may not indicate its statistical significance.
This problem has been addressed for the half-normal (and full normal) plot of effects by
standardizing the effects1. This presentation shows that, for Pareto charts, plotting the tvalues of the effects addresses the problem of non-orthogonal and/or imbalanced
designs. Furthermore, limits based on the significant t-value and the Bonferroni
corrected t-value are added to the Pareto chart and aid in the selection of effects.
Extreme Makeover: Data Edition
Julia C. O’Neill
Lori B. Pfahler
Merck & Co., Inc.
Purpose: To present practical guidelines for more effective presentation of data in tables
and graphics.
Abstract: Even the best educated scientists earn graduate degrees without a single
course focused on presenting data clearly. Small changes in the formatting of tables and
graphics can have a tremendous impact on the clarity of the information displayed.
During our redesign of the statistics training curriculum of a large chemical company, a
comprehensive needs analysis of over 1000 technologists uncovered this previously
unvoiced expectation: the technologists needed advice on effectively presenting data.
This launched our research into the body of knowledge on data presentation and
perception.
Although some excellent works by authorities such as Cleveland and Tufte have been
published, these are not easily accessible by most technologists. We synthesized the
principles and guidelines we found in the literature, and reduced them to a practical set
of rules for data presenters. We use many interesting and relevant examples to illustrate
the impact of each rule.
This presentation has been shared with enthusiastic audiences of scientists, quality
professionals, and teachers and parents of high-school science fair participants. Those
who apply the guidelines presented will see a dramatic improvement in the clarity of their
tables and graphics.
Friday, 10:00 to 11:30 am
Measurement Systems Session
Two-Dimensional Guidelines for Measurement System Indices
T. Kevin White
Voridian
Abstract:
Motivation or Background
When evaluating the adequacy of a measurement process, it is common to perform a
Gauge Repeatability & Reproducibility (R&R) study and look at the % Gauge R&R and
the Precision/Tolerance (P/T) Ratio. The acceptable guidelines for these measurement
system indices were originated in the parts industry and are often not realistic for many
measurement systems in the chemical and process industries. The result is that many
process improvement efforts, including Six Sigma projects, get stuck or detoured into
investigating the measurement process when there is still plenty to be gained by working
on the process. Six Sigma is partially to blame, as these guidelines from the parts
industry have been carried over into the chemical and process industries in the last
several years due to more and more companies starting Six Sigma efforts.
Description (Describe the work done)
The traditional guidelines will be challenged. It will be shown that even when these
guidelines are exceeded, only very small improvements in overall process capability can
be achieved by improving the measurement system. Work will also be shown that
explains the relationship between % Gauge R&R, P/T Ratio, and Cp. From this work,
two-dimensional guidelines for % Gauge R&R and P/T Ratio have been developed that
are more realistic for the chemical and process industries. Understanding how these
indices are inter-related is the key to making a decision regarding the adequacy of the
measurement system.
Significance (Are there improvements, applications, new abilities, new points of
view, etc.? How will the status quo be changed?)
These two-dimensional guidelines will be especially helpful to Black Belts and Green
Belts (and other quality improvement professionals) when performing a measurement
systems analysis. These new guidelines will challenge the traditional guidelines and be
a simple, yet improved way of making a determination regarding measurement system
adequacy. Ultimately, there will be fewer improvement efforts getting side-tracked by
the measurement system when there is still plenty to be gained by working on the
process itself.
On the Comparison of Two Measurement Devices
Joseph G. Voelkel
Rochester Institute of Technology
Bruce E. Siskowski
Reichert Inc
Purpose: To review the problem and difficulties associated with comparing measurement
devices and illustrate how to do it in a methodologically correct way.
Abstract: Measurement devices in the medical industry often have no reference
standards with which they may be compared. In such a case, devices are usually
compared to each other.
Frequently-used methods of comparison include regression, correlation, or the so-called
Bland-Altman plotting method. We review each of these, including any shortcomings
they may have—for example, regression is inappropriate because both the X’s and Y’s
include measurement error. We also contrast our problem to the standard Gage R&R
study that is commonly performed in industry.
A reasonable approach to take is based on so-called structural equation modeling. We
show how such modeling is appropriate for comparing two measuring devices. We show
how two devices may be compared by a sequence of likelihood-ratio tests. An example
based on two devices used to measure intra-ocular pressure of the human eye is used
to illustrate the technique, as well as possible extensions to it.
These methods are not new, but they seem to be unknown to many statisticians. We
hope this talk will help the audience learn about this important topic.
Generalizing Gage R&R summaries beyond two-way crossed models
Annie Dudley Zangi
Nicole Hill Jones
SAS Institute, Inc.
Purpose: To encourage more extended use of Gage R&R in the workplace.
Abstract: Traditional Gage R&R methods and summaries are based on a two-way
crossed model, with operator and part in the model. Often your environment doesn’t fit
into a two variable crossed model, but you still want to identify the contribution of part
versus every other factor you have available (i.e. repeatability and reproducibility). This
paper extends the definitions to include three and four way crossed models, main effects
models, and nested models, then illustrates how to analyze these using JMP® software.
Reliability Session
The Analysis and Comparison of Start-up Demonstration Tests
Michelle L. Depoy Smith
William S. Griffith
University of Kentucky
Purpose: The purpose of this paper is to review briefly the start-up demonstration test
literature and to present new results for existing tests and to propose new alternative
tests which are easier to implement and which, in a number of situations, outperform the
existing tests.
Abstract: Start-up demonstration tests were introduced by Hahn and Gage in the
Journal of Quality Technology in 1983. They can be used to determine the acceptability
of equipment. For example, a vendor may be required to demonstrate the reliability of
the equipment before a customer accepts it. In a start-up demonstration test, there are
successive start-up attempts and some procedure for deciding whether to accept or
reject the equipment. In the Hahn and Gage paper, a specified number of consecutive
successes was required to accept the equipment. In 2000, Balakrishnan and Chan
introduced a modification of the Hahn and Gage test where termination of the test and
rejection of the unit would occur if a total of f failures occurred prior to achieving k
consecutive successful start-ups (CSTF test). Smith and Griffith (2003 and to appear)
and Martin (2004) in separate papers have further investigated the CSTF test. In the
present paper, some alternative test schemes which are easier for the practitioner to
implement and which, in a number of situations, outperform this CSTF test are
introduced and studied. Markov chain techniques are used to do the probabilistic
analysis of the tests and we provide some practical advice on choosing a good start-up
test based on various criteria.
An Early Detection Test for the Compatibility of Two Software Environments
Daniel R. Jeske
Qi Zhang
University of California, Riverside
Purpose: Develop a test to determine if software failure data from a field environment
can be pooled with the failure data from its test environment for the purpose of improving
the precision of failure rate estimates that are made early in the early stages of the field
interval.
Abstract: In order to achieve higher precision in software failure rate predictions, it is
often the case that failure data from the test interval is pooled with failure data from the
field interval. The validity of pooling these two sources of data depends on the two
environments being compatible with respect to the manner in which the software is
used. In this paper, we formulate the hypothesis of compatible environments in terms of
a statistical hypothesis and develop an appropriate test procedure. We assume the
underlying failure process of the software follows a nonhomogeneous Poisson process
and develop a methodology to test whether or not the same Poisson process that is fit to
the test data continues to hold in the field environment. Our procedure includes an early
detection test for the case where there is insufficient field data for asymptotic tests to be
suitable, such as would often be the case in the early stages of field deployment. The
early detection test is conservative, but we outline a bootstrap algorithm that can be
used to approximately calibrate its size. We illustrate our test procedure by applying it to
two real software projects, and also report on a simulation study that confirms its size
properties.
Meeting Challenges in New Product Development Phases Using Accelerated Life
Testing
Sarath Jayatilleka
Maytag Appliances
O. Geoffrey Okogbaa
University of South Florida
Abstract: Some concepts of the uses of accelerated life tests (ALT) at different stages of
the new product development process are discussed here. Specific accelerated life tests
at component and subsystem levels were designed and data were analyzed to identify
early failures and achieve the improvements of product reliability. Goals were defined in
reliability design specifications for the product performance under the specified
operational conditions over the expected life. Several examples from product
development processes are provided in order to explain the above concepts. In the first
phase of a design, time-compressed accelerated life tests (ALT) and Highly Accelerated
Life Tests (HALT) were used. Based on such test results, further design iterations were
tested with HALTs and benchmarked with the previous design iterations. Test fixtures
were designed in order to simulate the normal operating levels and also to accelerate
specific failure modes. Failures in the eyes of the customer such as noise levels were
captured during the normal level operations. Weibull probability plots were successfully
used in reliability growth monitoring processes. In summary, above tests and analyses
were effectively and efficiently used to increase the degree of reliability improvements
between design iterations and shorten the design cycle time.
Process Analytical Technology
Engineering a Proactive Decision System for Pharmaceutical Quality
Ajaz S. Hussain
Food and Drug Administration
Purpose: To engineering a proactive decision system for pharmaceutical quality,
articulate key challenges, and seek solutions to these challenges in both short and long
term.
Abstract: This presentation briefly highlights important opportunities for engineering a
more proactive decision system, discusses its desired characteristics and attempts to
identify operational, tactical and strategic elements to be considered. Uncertainty in the
current decision environment needs to be reduced to facilitate effective and efficient
[risk-based] decisions. This may be achieved through enhanced quality by design (QbD)
principles which may include development of knowledge centric relationships between
material properties, controls, product quality attributes and/or intended clinical
performance. Although in the current system it is well recognized that quality can not be
tested into products - it has to be by design, yet testing to document quality is the most
prevalent approach. Utility of knowledge centric relationships for regulatory decisions
pose many challenges; for example, what are acceptable formulations of knowledge
centric relationships and how should these be evaluated? Two relatively recent
proposals, namely; "Science of Design" and "Engineering Systems", will be examined
and interpreted in a pharmaceutical context to identify and articulate key challenges and
to seek short and long term solutions to these challenges.
Multivariate Calibration for Analysis of Content and Coating Uniformity in
Pharmaceutical Tablets
John F. Kauffman
Food and Drug Administration
John A. Spencer
Food and Drug Administration
Purpose: To present a multivariate calibration for near infrared (NIR) spectra that
simultaneously quantifies tablet coating thickness and tablet compression force.
Abstract: Near infrared spectrometry (NIR) is widely used in process analysis and
control. NIR reflectance of solid pharmaceutical tablets can be used to quantify the
amount of active pharmaceutical ingredient in the tablet, but it is also well known that
NIR spectra are influenced by the compression force used to prepare the tablet.
Typically, spectra exhibit a decrease in the high energy portion of the spectrum as the
tableting force increases. Addition of a polymeric coating to the tablet surface has a
similar effect. This presents a problem when NIR spectrometry is used to monitor both
compression uniformity and coating thickness uniformity in pharmaceutical tablets. We
will present a multivariate calibration method for making simultaneous determinations of
coating thickness and tablet compression force in pharmaceutical tablets. The work is
based on analysis of NIR spectra from a set of coated tablets whose coating thickness is
known, and a separate set of tablets prepared with varying compression forces. We will
also discuss the applicability of similar calibration methods to spectroscopic image
analysis for the purpose of monitoring content and coating uniformity in coated
pharmaceutical tablets.
Application of PAT for Development of a Pharmaceutical Unit Operation
Steven M. Short
Carl A. Anderson
James K. Drennen III
Robert P. Cogdill
Zhenqi Shj
Duquesne University
Purpose: To illustrate the utility of the application of PAT to a unit operation for
secondary pharmaceutical manufacturing.
Abstract: One of the important ideas of process analytical technology is the optimization
of pharmaceutical manufacturing processes. In this presentation, the ideas espoused in
the FDA's PAT guidance are used for unit operation development. Specific tools used for
the development of a unit operation were risk analysis, design of experiments,
application of sensors, and multivariate analysis. The reliability of the unit operation
under study is thereby optimized as an input to subsequent unit operations. An important
objective of this exercise is to demonstrate how application of PAT increases the level of
process understanding thereby allowing less restrictive operating conditions.
Friday, 1:30 to 3:00 pm
Screening DOE Session
Using Fractional Factorial Split-Plots: Minimum Aberration or Optimum Blocking
James M. Lucas
J. M. Lucas and Associates
Frank Anbari
George Washington University
Purpose: To compare the Minimum Aberration and the Optimum Blocking approaches to
designing Fractional Factorial Split-Plot experiments and to suggest a combined
approach
Abstract: Industrial experiments often involve factors that are hard-to-change and
factors that are easy-to-change. A split-plot experiment is often appropriate in this
situation. When a full factorial is too expensive or time consuming to conduct, a
fractional factorial split-plot can be used. This is the situation we address.
A Minimum Aberration (MA) Split-Plot chooses the defining contrast to minimize the
number of “words” of shortest length. Before the publication of Bingham, Sitter and
Shone (2004) all MA papers considered visiting each level of the whole-plot factors only
once. Therefore, the MA literature considered only a small subset of the potential splitplot experiments. BSS removed this restriction and considered blocking patterns that
allowed multiple visits.
Optimum Blocking (OB) chooses the fractional factorial to minimize the maximum
variance of prediction; that is to maximize the global (G-) efficiency. This is achieved by
minimizing the number of model terms that are confounded with blocks. The MA
approach does not sufficiently penalize poor blocking, the criterion can allow too many
model terms to be confounded with blocks. Therefore the two approaches can differ.
We compare MA with OB and show when the two approaches differ.
The 26-1 case will be described in detail. An industrial example will be discussed. Future
work combining MA and OB will be discussed. The combined approach will be preferred
for future applications.
Sequential Supersaturated Designs for Efficient Screening
Angela M. Jugan
David Drain
University of Missouri – Rolla
Purpose: Generating supersaturated designs sequentially for efficient screening.
Abstract: Screening experiments are used to determine which factors have a
significant effect on a response with the assumption of effect sparsity. This can be
crucial in testing potential drugs or improving an industrial process. Supersaturated
designs are one answer to this problem, but most existing designs assume the entire
experiment will be done at the same time. In many situations, the experiment can be
done sequentially, with experimental runs later in the experiment chosen on the basis of
results from earlier runs. We evaluate a population of candidate supersaturated designs
with eighteen factors and eight initial runs, then choose eight additional runs
sequentially. The designs are analyzed with stepwise regression with the use of the
SWEEP operator to find three or less significant variables, and then one new run is
added to the design using the information found. After eight new runs are added, a
fitness value is assigned to the design based on how accurately it identified the true
significant parameters. The genetic algorithm takes candidates with the highest fitness
values and creates a new population of designs to evaluate. The process continues until
set criteria are met. This eighteen factor design is more efficient than any published
design and controls alpha and beta risks at least as well.
Multivariate Regression Session
Evolutionary Algorithms in Multicollinearity Situations: A Case study with
Stabilizing Transformations
Flor A Castillo
Carlos M. Villa,
The Dow Chemical Company
Purpose: In this paper the potential of using computing evolutionary algorithms (GPgenerated symbolic regression) for alleviating multicollinearity problems is presented
with a case study in an industrial setting
Abstract: In many industrial applications, observational data is collected and stored to
later become the focus of a modeling exercise usually for the purpose of process control.
The most desirable models, from the point of view of plant personnel, are often those
developed by multiple linear regression because of the relative simplicity of
interpretation. However, modeling with this type of data provides many challenges. Of
special importance is data multicollinearity or strong relationships between inputs.
Severe multicollinearity can badly affect the precision of the estimated regression
coefficients making them sensitive to the data collected and producing models with poor
precision. In these cases, techniques such as Principal Component Regression, and
Partial Least Squares are often used. However variable interpretation of the resulting
principal components is often difficult, especially for plant personnel. In order to improve
multiple linear regression modeling in the presence of severe multicollineaity, an
approach based on genetic programming (GP) symbolic regression was developed in
the Dow Chemical Company.
GP-generated symbolic regression is a computer-based algorithm that can generate
several functional mathematical expressions (equations) that fit a given set of data. The
algorithm starts by assembling a specified number of equations (a population) by
combining input variables and linking them by mathematical operators. Assembled
equations are evaluated in terms of a fitness function such sum square errors (SSE) or
correlation coefficient. Equations with low fitness are removed and the fittest solutions
are allowed to pass as members of a new generation of solutions (self-reproduction).
The algorithm also creates new solutions by combining bits and pieces of good ones
(crossover) and occasionally making a random change and introducing a complete new
solution (mutation). The new generation of equations then goes to the same procedure.
This process is repeated until a predefined number of generations of equations is
reached. This procedure produces several possible models of the response as a
function of the input variables. This set of equations or models offer a unique opportunity
since it shows possible relationships among variables that, when applied to multiple
regression modeling, have the potential to alleviate severe multicollinearity. Thus
providing a stable regression model in terms of the original variables which is easier to
implement and understand . In this paper the potential of GP-generated symbolic
regression (GP) for alleviating multicollinearity problems in multiple regression is
presented with a case study in an industrial setting. The GP-generated algorithm was
implemented as a tool box in MATLAB
Historical plant data was obtained and a first order multiple linear regression model was
fit to the data. Severe multicollinearity was confirmed by large variance inflation factors
(VIF). The GP algorithm was used and a set of resulting models or equations was
obtained. Equations were ranked in terms of R2 and the one with the largest R2 was
considered first. From this equation, the relationships between the variables were
observed and the original variables were transformed according to the functional form of
this relationship. Later, a first order polynomial was fitted to the transformed variables
and the resulting model was assessed in terms of R2. A residuals analysis was also
performed to check the IIDN(0,σ2) assumption. If the error assumptions were correct,
the VIF were evaluated to ensure that no severe multicollinearity was observed. If the
error structure was still incorrect, then other GP equation from the selected set was
chosen and the process was repeated until a model with no severe Multicollinearity and
with appropriate error structure was obtained. This approach was successfully applied in
a chemical process.. The key advantage of the approach is that produces a model that is
in terms of the original variables with no severe Multicollinearity. This simple, stable
polynomial model is easier for plant personnel to understand and implement.
Hierarchical Monitoring of Defect Rates Using Process Data
David R. Forrest
Virginia Institute of Marine Science
Christina M. Mastrangelo
University of Washington
Purpose: To present a hierarchical GLM and to demonstrate its application in a
manufacturing environment where it is ‘oriented’ toward a specific product goal.
Abstract: In semiconductor manufacturing, discovering the processes that are
attributable to defect rates is a lengthy and expensive procedure. This talk presents a
viable approach for understanding the impact of process variables on defect rates. By
using a process-based hierarchical model, we can relate sub-process manufacturing
data to layer-specific defect rates. This talk will demonstrate a hierarchical modeling
method using process data drawn from the Gate Contact layer, Metal 1 layer, and
Electrical Test data to produce estimates of defect rates. A binary logistic model is used
for the sub-process and meta-models because of the dichotomous nature of the data. A
semiconductor chip is either conformant or non-conformant, and logistic regression
provides a method of estimating the probability of non-conformance. Logistic modeling
is a form of generalized linear models, maximizing the likelihood of a linear model to
estimate the logit link function of the response variable. A benefit of the hierarchical
approach is that the parameters of the high-level model may be interpreted as the
relative contributions of the sub-models to the overall yield. In addition, the talk will
demonstrate how control charts may be used to monitor sub-process outputs where
these outputs have been oriented towards a product performance characteristic and
compares these to multivariate control charts on the sub-processes.
Measurement System Analysis Session
Bayesian Models for the Characterization of Reference Materials
Will Guthrie
National Institute of Standards and Technology
Purpose: To use Bayesian methods to characterize the physical properties of Standard
Reference Materials (SRM's) for use in calibration and measurement assurance.
Abstract: As part of the nation's metrology system, the National Institute of Standards
and Technology (NIST) characterizes the physical properties of a wide range of
materials for use as calibration and measurement assurance standards. Straight-forward
application of linear models often can be used successfully for the summarization of a
Standard Reference Material's properties. However, the measurement processes used
in the certification of some materials require more complex models. This talk will
describe the use of Bayesian models, fit using Markov-Chain Monte Carlo methods, as a
convenient and flexible way to extend the family of models typically used for SRM
certification. Examples will include the use of a three-parameter lognormal model for the
certification of sulfur in diesel fuels and a hierarchical errors-in-variables regression
model for the certification of the specific surface area of cement.
Open Source Excel Tools for Statistical Analysis of Complex Measurements
Hung-kung Liu
Will Guthrie
John Lu
Juan Soto
National Institute of Standards and Technology
Purpose: To develop statistical metrology tools for scientists and engineers that facilitate
statistical analysis by combining a familiar interface with powerful, high-quality, flexible
statistical software.
Abstract: Excel is the tool of choice for data collection for many scientists. However,
it has limited statistical capabilities. The S language is often the vehicle for research in
statistical methodology, and R provides an open source route to participation in that
activity. We utilize existing Excel plugins to harness the power of R scripts to enable
Excel users to use NIST recommended methodologies for the statistical analysis of their
data. In this talk, we will demonstrate two such examples: an automatic uncertainty
calculator, and a sensitivity coefficients generator for mass calibration.
Bayesian 3D Reconstruction of Chemical Composition from 2D Electron
Microscopy
Donald Malec
National Institute of Standards and Technology
Purpose: To develop and study new models for identifying and classifying the chemical
composition of unknown compounds.
Abstract: A 3D adaptation of 2D Markov Random Field models to electron microscopy
will be presented, including the incorporation of energy spectrum recordings to
identify chemical elements. Inherent differences, such as electron scattering, between
electron microscopy and other areas where tomography is used, such as X-ray
tomography, will be discussed.