The Analysis and Comparison of Gauge Variance Estimators
Peng-Sen Wang and Jeng-Jung Fang
Department of Management and Information Technology
Southern Taiwan University of Technology
710 Yung-Kung City, Tainan County, Taiwan
Abstract
The research improves the analysis of the measurement system. It modifies the
method for calculating gauge variance. The comparison of gauge variance estimators for
many methods based on the criterion of the biasness, variance, and mean squares error
(MSE) of the estimators by a factorial arrangement of three parameters in a simulating
measurement system shows that the modified estimator proposed is better than or equal to
the others.
Keywords: Gauge System Analysis, Repeatability, Reproducibility, Interaction
1. Introduction
Due to the rapid development of the high-tech industry and the requirement of the high
quality from the customers, the accuracy and precision on the measurement system will
affect the quality of statistical analysis. Before data collection a complete check including
the alignment of gauge itself, the training of operators, and gauge variation study is
required so that the analyst can make the correct decision. In the QS 9000, ISO/IEC
17025, and TS 16949 quality assurance systems, the analysis of repeatability and
reproducibility of the measurement variation is to apply the verified measurement system
to the measurement process. Most of the industries apply these assurance systems to judge
the acceptance of their measurement systems.
1
2
Tsai (1988) defined the gauge reproducibility as the variability due to different operators
using the same gauge and the gauge repeatability as the variability reflecting the basic
precision of the gauge itself. The gauge variability is composed of the gauge
reproducibility and the gauge repeatability. The variance components of the total
observed variability includes both the part variability and gauge variability. The random
effects model used for analyzing a measurement system is presented as
Yijk    Pi  O j  POij  Rijk
i  1, 2, ..., n;

 j  1, 2, ..., p;
k  1, 2, ..., k

(1)
The Yijk is the kth repeated measurement on the ith part by the jth operator.  is the
process average. Pi is the ith part effect. Oj is the jth operator effect. POij is the interaction
between the ith part and the jth operator. Rijk is the error term. The source of variation can
be divided into four components which are parts, operators, interaction between parts and
operators, and the random error term. Pi 、 Oj 、 POij, and, Rijk are random factors
normally distributed with mean 0 and constant variances equal to  P ,
2
 R2
2
 O2 ,  PO
, and
respectively. The effects of parts and operators, the interaction between parts and
operators, and the error terms are assumed to be additive. The estimated gauge variance is
the sum of variances of repeatability and reproducibility. That is
2
2
ˆ2
ˆ gauge
 ˆ repeatabil
ity   reproducibility
(2)
3
where
=
ˆ R2
2
ˆ2
ˆ reproducib
ility =  O
and
2.
2
ˆ repeatabil
ity
2
+ ˆ PO
(3)
(4)
Methods to Analyze Gauge Variation
There are three analysis methods widely used to estimate the components of variance in
the gauge system. They are ANOVA, Classical GR&R, and Long Form methods. Tsai
(1988) developed the ANOVA method to compute the variance components.
Montgomery and Runger (1993a, 1993b) emphasized the repeatability and reproducibility
study is equivalent to an experiment designed to estimate certain components of variance.
Negative estimate of variance component
2
 PO
is
possible. One method to solve the
problem is to set it to zero and recalculate the other variance component estimates. A
reduced model is then fitted and the estimates of repeatability and reproducibility are then
adjusted.
Barrentine (1991) introduced the control chart tabular methods to estimate the variance
components of repeatability and reproducibility. Montgomery and Runger （1993a）
called the methods “Classical Gauge Repeatability and Reproducibility Study” (Classical
GR&R). The tabular method tabulates the collected data for k equal to 2 as the form of
Table 1. The estimate of repeatability is obtained from the average of the p average
ranges.
 p

R    R  j  p
 j 1

4
where the R j is the average of the ranges of repeated measurements over all the parts
taken by the jth operator. The Classical GR&R method estimates repeatability as

R
d2
 repeatability 
(5)
The value d2 depends on the number of repeated measurements taken on the same part by
the same operator. Here d2 is equal to 1.128.
Table 1 Tabular Form for the Classical Gauge Repeatability and Reproducibility Study
Operator 1
…
Operator p
Measurements
…
Measurements
Part
Number
X
R
…
1
2
x112
X 11
R11
…
x1p1
x211
x212
X 21
R21
…
.
.
.
.
.
…
n
xn11
xn12
X 21
Rn1
X 1
R1
1
2
1
x111
2
Average
X
R
x1p2
X 1 p
R1p
x2p1
x2p2
X 2 p
R2p
.
.
.
.
xnp1
xnp2
X np
Rnp
X  p
R p
…
The estimator of reproducibility is obtained from the difference
R  max X  j   min X  j  , j = 1, 2, …, p
X
The estimator of reproducibility is
j
j
(6)
5
ˆ reproducibility 
R
X
(7)
d2
MSA (1991) reference manual introduced another tabular approach similar to control
chart tabular method which is called Long Form method. The estimators of
repeatability and reproducibility are computed as
ˆ repeatability
ˆ reproducibilty
=
= R
R
 X
 d2

(8)
d2
R 
2
 d 

2
 

nk

2
(9)
The repeatability and reproducibility of the three analysis methods are summarized
in the Table 2.
Table 2 Repeatability and Reproducibility Estimators of Classical GR&R, ANOVA,
and Long Form Methods
ˆ repeatability
Classical GR&R
ANOVA
Long Form
ˆ reproducibility
R
R d2
MS R
R d2
X
d2
MS O  n  1MS PO  nMS R  nk
 RX

 d2





2
R 
 d 
2


nk
2
6
Other related gauge analysis methods can be found in the articles such as Dolezal
(1998), and Burdick and Larsen (2002).
Table 3 and Table 4 are the results of gauge variability analysis for measurements
without and with interaction between operators and parts respectively. Comparing the
results of the two tables, one can observe that Classical GR&R and Long Form methods
cannot precisely estimate the gauge variance while there is interaction between parts and
operators.
Table 3 Repeatability, Reproducibility, and Gauge Variance Estimates for
Data without Interaction Between Parts and Operators
ˆ repeatability
ˆ reproducibility
ˆ gauge
Classical GR&R
3.783
1.418
4.039
Long Form
3.783
1.285
3.995
ANOVA
2.576
3.983
4.743
Table 4 Repeatability, Reproducibility, and Gauge Variance Estimates for
Data with Interaction Between Parts and Operators
ˆ repeatability
ˆ reproducibility
ˆ gauge
Classical GR&R
2.261
1.462
2.692
Long Form
2.261
1.418
2.668
7
ANOVA
3.506
3.863
5.216
In order to make both Classical GR&R and Long Form methods suitable to the cases of
with interaction between parts and operators, referring to the tabular form in Table 1, we
first calculate
R X ij  max X ij  min X ij , i = 1, 2, …, n
 n

R X ij     R X ij   n
 i 1

and
By replacing
(10)
j
j
(11)
R X with R X into the reproducibility variance estimator of Classical
ij 
GR&R and Long Form methods, the two methods can be applied to estimate the
gauge variance for the cases there are interactions between parts and operators. The
modified reproducibility estimator of the two analysis methods are summarized in
the Table 5.
Table 5 Modified Reproducibility Estimators of Classical GR&R and Long
Form Methods
ˆ reproducibility
Modified
R X ij  d 2
Classical GR&R
8
Modified
Long Form
 R X ij 

 d2

R 
2
  d2 

 

nk

2
Table 6 shows the values of the repeatability, reproducibility, and total gauge
variance estimate for the five methods of ANOVA, Classical GR&R, Modified
Classical GR&R, Long Form, and Modified Long Form based on the same
measurements used in Table 4. It showed that Modified Classical GR&R and
Modified Long Form methods are both much better to estimate the gauge variance
than Classical GR&R and Long Form methods while there is interaction between
parts and operators.
Table 7 shows Modified Classical GR&R and Modified Long Form methods also
generate good variance estimates for data without interaction between operators and parts
based on the same measurements used in Table 3.
In summary, the Modified Classical GR&R and Modified Long Form methods are
much better than the Classical GR&R and Long Form methods in estimating the
gauge variance on either case of with or without interaction between parts and
operators.
Table 6 Repeatability, Reproducibility, and Gauge Variance Estimates for
Data with Interaction Between Parts and Operators
9
ˆ repeatability
ˆ reproducibility
ˆ gauge
ANOVA
3.506
3.863
5.216
Classical GR&R
2.261
1.462
2.692
2.261
4.592
5.119
Long Form
2.261
1.418
2.668
Modified Long Form
2.261
4.579
5.1062
Modified Classical
GR&R
Table 7 Repeatability, Reproducibility, and Gauge Variance Estimates for Data
without Interaction Between Parts and Operators
ˆ repeatability
ˆ reproducibility
ˆ gauge
ANOVA
2.576
3.983
4.743
Classical GR&R
3.783
1.418
4.039
3.783
1.949
4.255
Long Form
3.783
1.285
3.995
Modified Long Form
3.783
1.855
4.213
Modified Classical
GR&R
3.
Comparison
Now we will compare the gauge variance estimators of the five different analysis
methods by simulating a measurement system based on the criterion of the biasness,
10
variance, and mean squares error (MSE) of the gauge variance estimate. We generated the
measurements according to the random effects model of equation (1) and calculated the
gauge variance estimates of the five methods. Then compare the gauge variance estimates
with its true value. The procedure of simulating the measurements is as follows:
1.
Choose the values of n, p, and k.
2.
Set the values of  ,
3.
Simulate one set of measurements according to the values in step (1) and (2).
4.
Compute the gauge variance estimates of the five methods (ANOVA, CRR, LF,
 P2 ,  O ,  PO , and  R2 .
2
2
MCRR, and MLF).
4.
5.
Repeat step (3) and (4) for 10000 times.
6.
Calculate the criterion values of each 10000 estimates for comparison.
Criterions for Comparison
There are three criteria for comparison. They are MSE, sample variance, and mean
ratio of the gauge variance estimates.
The MSE of gauge variance estimator is
n
 (ˆ
MSE =
i 1
2
gauge
2
  gauge
)2
n
(12)
where n is the number of simulation runs. In our example n is equal to 10000. The MSE is
a combined index of accuracy and precision of an estimator. The smaller of the MSE
value, the better the estimator is.
11
The sample variance of the gauge variance estimates is presented as Equation (13). The
smaller of it, the higher precision of the estimator and the narrower width of its
confidence interval are.
 n 2 
  ˆ gauge 
n
2
2
 i 1

ˆ
(

)

n

gauge


n
i 1


2


Var ( ˆ gauge ) =
n 1
2
(13)
The mean ratio of the gauge variance estimates to its true value is showed as Equation
(14). If the ratio is equal to 1, the estimator is then unbiased.
n
Mean ratio =
5.
2
ˆ gauge

i 1
2
gauge
n
(14)
Result of Comparison
The research first compared the five methods under various combinations of npk values
for the case with interaction between parts and operators. We set process average μ to 0,
 P2
to 0,
2
2
 O2 to 1,  PO
to 1, and  R to 0.25. Ten thousand simulation runs are
conducted for various npk values. Sample variance, MSE, and mean ratio of the gauge
variance estimates are calculated for each of the five methods.
The result of mean ratios of the gauge variance estimates for the five different methods
are presented in Table 8. It showed that ANOVA method produces unbiased gauge
12
variance estimator. The MCRR method generates overestimated estimator while MLF,
CRR, and LF methods generate underestimated estimators. They also revealed MCRR
and MLF estimators are much better than those of CRR and LF.
Table 9 showed the result of sample variances of gauge variance estimates. The MLF
estimator has the smallest variance while the LF estimator is the second smallest.
Table 8 Mean Ratios of Gauge Variance Estimates Computed by Five Methods for
Various npk Values under the Case with Interaction Between Parts and
Operators
npk
ANOVA
CRR
MCRR
LF
MLF
30
0.997
0.808
1.153
0.656
0.927
40
1.005
0.764
1.124
0.661
0.965
60
0.997
0.717
1.096
0.652
0.992
80
0.994
0.692
1.098
0.601
0.945
90
0.998
0.800
1.113
0.652
0.898
Table 9 Sample Variances of Gauge Variance Estimates Computed by Five Methods for
Various npk Values under the Case with Interaction Between Parts and
Operators
npk
ANOVA
CRR
MCRR
LF
MLF
30
1.716
2.481
2.461
1.534
1.522
13
40
2.660
6.072
5.033
2.493
2.067
60
1.302
1.985
1.832
1.227
1.132
80
2.304
5.433
4.337
2.226
1.778
90
1.219
1.899
1.710
1.174
1.057
The results of mean square errors (MSE) of the gauge variance estimates are presented in
Table10. It showed that MLF method generates the best estimator while considering the
accuracy and precision together in spite that MLF estimator is biased.
In summary under the interaction between parts and operators, the ANOVA estimator is
unbiased while MLF estimate has the smallest variance. The MLF estimator is the best
while both accuracy and precision are taken into consideration. The MCRR and MLF
methods are much better than CRR and LF methods.
Table10 Mean Square Errors (MSE) of the Gauge Variance Estimates Computed by Five
Methods for Various npk Values under the Case with Interaction Between Parts
and Operators
npk
ANOVA
CRR
MCRR
LF
MLF
30
1.686
2.667
2.522
2.133
1.514
40
1.130
1.735
1.525
1.623
1.042
60
1.389
2.442
2.063
2.076
1.259
80
0.884
1.640
1.168
1.636
0.817
90
1.179
2.263
1.711
2.026
1.073
14
The research also compared the five methods under various combinations of npk values
for the case without interaction between parts and operators. The results are shown in
Table 11, 12, and 13. Under the case without interaction between parts and operators,
ANOVA gauge variance estimator is unbiased as expectation. Both ANOVA and MLF
gauge variance estimators about have the same precision. While taking accuracy and
precision into consideration together, ANOVA and MLF gauge variance estimators are
the best and there is almost no difference between them.
Table11 Mean Ratios of Gauge Variance Estimates Computed by Five Methods for
Various npk Values under the Case without Interaction Between Parts and
Operators
npk
ANOVA
CRR
MCRR
LF
MLF
30
1.005
1.256
1.313
1.025
1.070
40
1.002
1.178
1.241
1.022
1.076
60
1.000
1.237
1.300
1.013
1.062
80
0.988
1.145
1.216
0.997
1.057
90
1.001
1.232
1.296
1.009
1.060
120
0.988
1.213
1.279
0.995
1.047
240
0.999
1.100
1.182
1.007
1.080
Table12 Sample Variances of Gauge Variance Estimates Computed by Five Methods for
Various npk Values under the Case without Interaction Between Parts and
Operators
15
npk
ANOVA
CRR
MCRR
LF
MLF
30
1.030
1.695
1.673
1.049
1.035
40
2.054
5.078
4.979
2.083
2.043
60
1.075
1.770
1.739
1.094
1.075
80
1.976
4.884
4.780
2.003
1.961
90
1.008
1.656
1.629
1.024
1.007
120
1.000
1.646
1.618
1.017
1.000
240
0.963
1.577
1.548
0.974
0.956
Table 13 Mean Square Errors (MSE) of the Gauge Variance Estimates Computed by Five
Methods for Various npk Values under the Case without Interaction Between
Parts and Operators
npk
ANOVA
CRR
MCRR
LF
MLF
30
1.072
1.855
1.887
1.085
1.080
40
0.710
1.084
1.107
0.742
0.737
60
1.033
1.796
1.821
1.056
1.045
80
0.670
0.996
1.015
0.690
0.680
90
1.004
1.725
1.752
1.014
1.004
120
0.997
1.714
1.735
1.015
1.000
240
0.396
0.557
0.572
0.437
0.430
16
6.
Conclusion
In this article, we modified both the Classical GR&R and Long Form methods to
make them both suitable to either case of with or without interaction between parts and
operators. After comparing the five estimation methods (ANOVA, Classical GR&R, Long
Form, Modified Classical GR&R, and Modified Long Form methods) by simulating a
measurement process and calculating the gauge variance estimates for each method, it
showed the Modified Long Form method is a very good choice to estimate gauge
variability no matter there is interaction between parts and operators or not.
Reference
1. AIAG Editing Group, “Measurement Systems Analysis-Reference Manual (MSA)”,
Automotive Industries Action Group, 1991.
2. Barrentine, L. B., Concepts for R&R Studies, ASQC Quality Press, Milwaukee, WI.
3. Burdick, R. K.; Larsen, G. A. and Allen, A. Elizabeth, “Comparing Variability of
Two Measurement Process Using R&R Studies,” Journal of Quality Technology,
Vol.34, No.1, pp.97-105 2002
4. Dolezal, K. K. and Burdick, R. K. and Birch, N. J., “Analysis of a Two-Factor R&R
Study with Fixed Operators,” Journal of Quality Technology, Vol.30, No.2, pp.163170 1998
5. Montgomery, D. C. and Runger, G. C., “Gauge Capability Analysis and Designed
Experiments. Part II: Experimental Design Models and Variance Component
Estimation”, Quality Engineering, Vol.6, No.2, pp.289-305 1993a
17
6. Montgomery, D. C. and Runger, G. C., “Gauge Capability Analysis and Designed
Experiments. Part I: Basic Methods”, Quality Engineering, Vol.6, No.1, pp.115-135
1993b