IE 361 Module 4
Metrology Applications of Some Intermediate Statistical Methods for
Separating Components of Variation
Reading: Section 2.2 Statistical Quality Assurance for Engineers
(Section 2.3 of Revised SQAME )
Prof. Steve Vardeman and Prof. Max Morris
Iowa State University
Vardeman and Morris (Iowa State University)
IE 361 Module 4
1 / 27
A Relatively Simple First Method for Separating
Contributions of Two Sources of Variation
In Module 2 we observed that
1
repeated measurement of a single measurand with a single device
allows one to estimate σdevice , and
2
single measurements made on multiple measurands from a stable
process
qusing a linear device allow one to estimate
σy =
σ2x + σ2device
and remarked that these facts might allow one to somehow …nd a way to
estimate σx (a process standard deviation) alone. Our …rst goal in this
module is to provide one simple method of doing this.
The next …gure illustrates a data collection plan that combines the
elements 1. and 2. above.
Vardeman and Morris (Iowa State University)
IE 361 Module 4
2 / 27
One Method of Separating Process and Measurement
Variation
Figure: One Possible Data Collection Plan for Estimating a Process Standard
Deviation, σx
Vardeman and Morris (Iowa State University)
IE 361 Module 4
3 / 27
One Method of Separating Process and Measurement
Variation
Here we will use the notation y for the single measurements on n items
from the process and the notation y 0 for the m repeat measurements on a
single measurand. The sample standard deviation
of the y ’s, sy , is a
q
natural empirical approximation for σy = σ2x + σ2device and the sample
standard deviation of the y 0 ’s, s, is a natural empirical approximation for
σdevice . That suggests that one estimate the process standard deviation
with
q
c
(1)
σx = max 0, sy2 s 2
as indicated in display (2.3), page 20 of SQAME. (The maximum of 0
and sy2 s 2 under the root is there simply to ensure that one is not trying
to take the square root of a negative number in the rare case that s
exceeds sy .)
Vardeman and Morris (Iowa State University)
IE 361 Module 4
4 / 27
One Method of Separating Process and Measurement
Variation
cx is not only a sensible single number estimate of σx , but can also be used
σ
to make approximate con…dence limits for the process standard deviation.
The so-called Satterthwaite approximation suggests that one use
s
s
ν̂
ν̂
cx
cx
and σ
as limits for σx
σ
χ2upper
χ2lower
where appropriate "approximate degrees of freedom" are
ν̂ =
sy4
n
1
cx 4
σ
+
m
s4
1
(and we will always round down to an integer degrees of freedom).
Vardeman and Morris (Iowa State University)
IE 361 Module 4
5 / 27
One Method of Separating Process and Measurement
Variation
Example 4-1
In Module 2, we considered m = 5 measurements made by a single analyst
on a single physical sample of material using a particular assay machine
that produced s = .0120. Suppose that subsequently, samples from
n = 20 di¤erent batches are analyzed and sy = .0300. An estimate of real
process standard deviation (unin‡ated by measurement variation) is then
r
q
2
2
cx = max 0, sy s = max 0, (.0300)2 (.0120)2 = .0275
σ
and this value can used to make con…dence limits. The Satterthwaite
"approximate degrees of freedom" are
ν̂ =
sy4
n
1
cx 4
σ
+
Vardeman and Morris (Iowa State University)
m
=
s4
1
(.0275)4
= 11.96
(.0300)4 (.0120)4
+
19
4
IE 361 Module 4
6 / 27
One Method of Separating Process and Measurement
Variation
Example 4-1
and rounding down to ν̂ = 11, an approximate 95% con…dence interval for
the real process standard deviation, σx , is
!
r
r
11
11
, .0275
i.e. (.0195, .0467)
.0275
21.920
3.816
Vardeman and Morris (Iowa State University)
IE 361 Module 4
7 / 27
A Second Application of the First Statistical Method
The insight just applied to separating process and measurement variation
is that of taking a di¤erence between two sample variances, one estimating
variability due to two sources (process and measurement), and the other
representing variation due to only one of the two components
(measurement). This thinking can also be used to separate components
of measurement variation. That is, it provides a …rst very simple way of
separating repeatability and reproducibility variation.
Now use the notation y for single measurements on an item made by n
di¤erent operators (devices) and the notation y 0 for m repeat
measurements made on that item by a single operator (device). A
combination of things said in Modules 2 and 3 implies that the sample
standard
q deviation of the y ’s, sy , is a natural empirical approximation for
σy = σ2δ + σ2device while the sample standard deviation of the y 0 ’s, s, is a
natural empirical approximation for σdevice . This is pictured in the next
…gure.
Vardeman and Morris (Iowa State University)
IE 361 Module 4
8 / 27
A Second Application of the First Statistical Method
Figure: A Simple Data Collection Plan Allowing the Separation of Repeatability
and Reproducibility Variances (Assuming σdevice , i.e. Repeatability, is Constant)
Vardeman and Morris (Iowa State University)
IE 361 Module 4
9 / 27
A Second Application of the First Statistical Method
In this second context
σbδ =
q
max 0, sy2
s2
estimates the reproducibility standard deviation (the standard deviation of
cx
device biases). Then, the formulas on panel 5 with σbδ replacing σ
provide approximate con…dence limits (not for the process standard
deviation but) for the reproducibility standard deviation, σδ .
Vardeman and Morris (Iowa State University)
IE 361 Module 4
10 / 27
A Second Application of the First Statistical Method
Example 4-2
Suppose that in an IE 361 style measurement exercise, a single student
TM
measures the size of a Styrofoam
packing peanut m = 5 times using a
crude plastic caliper with a sample standard deviation of s = .012 in.
Suppose that subsequently, n = 6 di¤erent students measure a single
peanut once each with a resulting standard deviation of sy = .030 in. An
estimate of reproducibility standard deviation (unin‡ated by repeatability
variation) is then
r
q
2
2
σbδ = max 0, sy s = max 0, (.030)2 (.012)2 = .0275 in
and this value can used to make con…dence limits for σδ . The
Satterthwaite "approximate degrees of freedom" are
ν̂ =
sy4
n
1
σbδ 4
+
Vardeman and Morris (Iowa State University)
m
=
s4
1
(.0275)4
= 3.42
(.030)4 (.012)4
+
5
4
IE 361 Module 4
11 / 27
A Second Application of the First Statistical Method
Example 4-2
and rounding down to ν̂ = 3, an approximate 95% con…dence interval for
the reproducibility standard deviation, σδ , is
!
r
r
3
3
, .0275
i.e. (.016 in, .103 in)
.0275
9.348
.216
In contrast to this inference, notice that applying the basic con…dence
limits for a standard deviation (based on s = .012 and
ν = m 1 = 4 1 = 3), 95% limits for the the repeatability standard
deviation, σdevice , here are
!
r
r
3
3
.030
i.e. (.017 in, .112 in)
, .030
9.348
.216
Vardeman and Morris (Iowa State University)
IE 361 Module 4
12 / 27
One Way Random E¤ects Models and Associated Inference
One of the basic models of intermediate statistical methods is the
so-called "one-way random e¤ects model" for I samples of observations
y11 , y12 , . . . , y1n1
y21 , y22 , . . . , y2n2
..
.
yI 1 , yI 2 , . . . , yInI
This model says that the observations may be thought of as
yij = µi + eij
where the eij are independent normal random variables with mean 0 and
standard deviation σ, while the I values µi are independent normal random
variables with mean µ and standard deviation σµ (independent of the e’s).
(One can think of I means µi drawn at random from a normal distribution
of µi ’s, and subsequently observations y generated from I di¤erent normal
populations with those means and a common standard deviation.)
Vardeman and Morris (Iowa State University)
IE 361 Module 4
13 / 27
One Way Random E¤ects Models and Associated Inference
In this model, the 3 parameters are σ (the "within group" standard
deviation), σµ (the "between group" standard deviation), and µ (the
overall mean). The squares of the standard deviations are called "variance
components" since for any particular observation, the laws of expectation
and variance imply that
µy = µ + 0 = µ and σ2y = σ2µ + σ2
(i.e. σ2µ and σ2 are components of the variance of y ).
Two quality assurance/metrological contexts where this model can be
helpful are where
multiple measurands from a stable process are each measured
multiple times on the same device
a single measurand is measured multiple times on multiple devices
These two scenarios and the accompanying parameter values are
illustrated in the next two …gures.
Vardeman and Morris (Iowa State University)
IE 361 Module 4
14 / 27
One Way Random E¤ects Models and Associated Inference
Figure: Cartoon Illustrating Multiple Measurands from a Stable Process Each
Measured Multiple Times With the Same (Linear) Device
Vardeman and Morris (Iowa State University)
IE 361 Module 4
15 / 27
One Way Random E¤ects Models and Associated Inference
Figure: Cartoon Illustrating a Single Measurand Measured Multiple Times With
Multiple Devices
Vardeman and Morris (Iowa State University)
IE 361 Module 4
16 / 27
One Way Random E¤ects Models and Associated Inference
There are well established (but not altogether simple) methods of inference
associated with the one-way random e¤ects model, that can be applied to
make con…dence intervals for the model parameters (and inferences of
practical interest in metrological applications). Some of these are based
on so-called ANOVA methods and the one-way ANOVA identity that says
∑ (yij
y .. )2 =
i ,j
∑ ni ( y i .
i
y .. )2 + ∑ (yij
y i . )2
i ,j
or
SSTot = SSTr + SSE
For example, with n = ∑ ni , the quantity
r
p
SSE
σ̂ = MSE =
n I
Vardeman and Morris (Iowa State University)
IE 361 Module 4
17 / 27
One Way Random E¤ects Models and Associated Inference
is a square root of a (weighted) average of the I sample variances and can
be used to make con…dence limits for σ as
s
s
n I
n I
σ̂
and σ̂
χ2upper
χ2lower
where the appropriate degrees of freedom are ν = n I . And, although
we won’t illustrate them here, the Satterthwaite approximation can be
used to make approximate con…dence limits for σµ .
Operationally, the most e¢ cient way to make inferences based on the one
way random e¤ects model is to use a high quality statistical package like
JMP and rely on its implementation of the best known methods of
estimation of the parameters σ, σµ , and µ. We proceed to illustrate that
possibility in a metrological application.
Vardeman and Morris (Iowa State University)
IE 361 Module 4
18 / 27
One Way Random E¤ects Models and Associated Inference
Example 4-3
Consider the case of Problem 2.10, pages 50-51 of SQAME, and in
particular the two hardness measurements made on each of the I = 9 parts
by Operator A. This is a scenario of the type illustrated on panel 15. The
following series of …gures shows …rst a JMP data sheet for this example
(note that part is a nominal variable and hardness is a continuous
variable), then the dialogue box for a Fit Model procedure appropriate
here (the part e¤ect has been made a random e¤ect by using the menu
under the red triangle by "attributes" in the dialogue box), and …nally a
JMP report for the analysis, showing con…dence limits for σ2x (= σ2µ here)
and for σ2device (= σ2 here).
What is clear from this analysis is that this is a case where part-to-part
variation in hardness (measured by σx ) is small enough and poorly
determined enough in comparison to basic measurement noise (measured
by σdevice ) that it is impossible to really tell its size.
Vardeman and Morris (Iowa State University)
IE 361 Module 4
19 / 27
One Way Random E¤ects Models and Associated Inference
Example 4-3
Figure: JMP Data Sheet for Example 4-3 (Data From Page 51 of SQAME )
Vardeman and Morris (Iowa State University)
IE 361 Module 4
20 / 27
One Way Random E¤ects Models and Associated Inference
Example 4-3
Figure: JMP Fit Model Dialogue Box for Example 4-3
Vardeman and Morris (Iowa State University)
IE 361 Module 4
21 / 27
One Way Random E¤ects Models and Associated Inference
Example 4-3
Figure: JMP Report for Example 4-3
Vardeman and Morris (Iowa State University)
IE 361 Module 4
22 / 27
One Way Random E¤ects Models and Associated Inference
Example 4-4
Consider the case of Problem 2.12, page 52 of SQAME, and in particular
the three weight measurements made on piece 1 by each of the I = 5
operators. This is a scenario of the type illustrated in panel 16 and further
illustrates the concepts of "repeatability" (device) variation and
"reproducibility" (operator-to-operator) variation …rst discussed in Module
3.
The following series of …gures shows …rst a JMP data sheet for this
example (note that operator is a nominal variable and weight is a
continuous variable), then the dialogue box for a Fit Model procedure
appropriate here (the operator e¤ect has been made a random e¤ect by
using the menu under the red triangle by "attributes" in the dialogue box),
and …nally a JMP report for the analysis, showing con…dence limits for σ2δ
(= σ2µ here) and for σ2device (= σ2 here).
Vardeman and Morris (Iowa State University)
IE 361 Module 4
23 / 27
One Way Random E¤ects Models and Associated Inference
Example 4-4
Figure: JMP Data Sheet for Example 4-4 (Data From Page 52 of SQAME )
Vardeman and Morris (Iowa State University)
IE 361 Module 4
24 / 27
One Way Random E¤ects Models and Associated Inference
Example 4-4
Figure: JMP Fit Model Dialogue Box for Example 4-4
Vardeman and Morris (Iowa State University)
IE 361 Module 4
25 / 27
One Way Random E¤ects Models and Associated Inference
Example 4-4
Figure: JMP Report for Example 4-4
Vardeman and Morris (Iowa State University)
IE 361 Module 4
26 / 27
One Way Random E¤ects Models and Associated Inference
Example 4-4
Recognizing that although the JMP report lists a negative lower
con…dence bound for σ2δ , this quantity can never be smaller than 0, we
estimate with 95% con…dence that
p
0 < σδ < 4.5 10 5 = .0067
and that
.0057 =
p
3.2
10
5
< σdevice <
p
.0002014 = .0142
and this is a case where repeatability (device) variation is clearly larger
than reproducibility (operator-to-operator) variation in weight measuring.
If one doesn’t like the overall size of measurement variation seen in the
data of panel 24, it appears that some fundamental change in equipment
or how it is used will be required. Simple training of the operators aimed
at making how they use the equipment more uniform (and reduction of
di¤erences between their biases) has far less potential to improve
measurement precision.
Vardeman and Morris (Iowa State University)
IE 361 Module 4
27 / 27