The Analysis and Comparison of Gauge
Variance Estimators
Peng-Sen Wang and Jeng-Jung Fang
Southern Taiwan University of Technology
Tainan, Taiwan
1
Content
Background
Objectives
Assumptions
Literatures
3 Criterions for
comparison
8 estimators for
comparison
－Definitions
－References
－Methods for Estimating
Gauge Variance
2
Background and Objectives
 The precision of measurement system will affect
the quality of statistical analysis.
 3 methods for estimating GR&R varaince：
 ANOVA
 Classical GR&R Studies
 Long Form
 Before doing GR&R research, 3 parameters must
be decided. n: number of parts, p: number of
operators, k :number of repetitions
3
Assumptions
Parts can be measured repeatedly.
Quality characteristic is quantitative.
Single quality characteristic.
Quality characteristic is normally distributed.
Independent measurements among parts.
Other factors are controllable.
4
Literature
Definition
 Repeatability ：The
variability of gauge itself.
Same operator measures
same part.
重複性

Repeatabilty
rept
 Reproducibility：The variability due to different operators using
the same gauge. Different operators measures same part.
量測人員A
量測人員B
量測人員C
再現性
Reproducibility
5
Definition
Literature
Gauge Repeatability and Reproducibility :
(GR&R）：The overall performance of gauge
capability, call it measurement variation.
6
Literature
GR&R related reference
 AIAG Editing Group (1991), “Measurement Systems
Analysis-Reference Manual（MSA）”,1nd ed., Automotive
Industries Action Group.
 Barraentine, L. B. (1991), “Concepts for R&R Studies”,
ASQC Quality Press , Milwaukee, Wisconsin.
 Montgomery, D. C. and Runger, G. C. (1993a), “Gauge
Capability Analysis and Designed Experiments. Part I : Basic
Methods”, Quality Engineering, Vol.6, No.1, pp.115-135.
 Montgomery, D. C. and Runger, G. C. (1993b), “Gauge
Capability Analysis and Designed Experiments. Part II :
Experimental Design Models and Variance Component
Estimation”, Quality Engineering, Vol.6, No.2, pp.289-305.
7
Literature
Methods for estimating gauge variance
ANOVA
 Based on the ANOVA model of Montgomery and Runger
（1993b.).
 Two-factor random effects model：One factor is part（P）
with n levels, the other is operator (O) with p level. With k
repeated measurements for each combination, the linear
model is：
X ijk    Pi  O j  POij  Rijk
i  1, 2, , n 


 j  1, 2, , p 
k  1, 2, , k 


 Xijkis the kth repeated measurement on the ith part by the jth operator. Pi is the
ith part effect. Oj is the jth operator effect. POij is the interaction. Rijk is the
error term. Random factors are normally distributed with mean 0 and
constant variances.
8
Literature
ANOVA
ANOVA of random effects model
變異來源
平方和
自由度
均方
期望均方
Source of
Sum of
Degrees of
Mean
Expected
Variability
Squares
Freedom
Squares
Mean Squares
產品
Parts
量測人員
Operators
產品×量測人員
Parts×Operators
誤差項
Error
總和
Total
SSp
N－1
MSp
SSo
P－1
MSo
SSpo
（n－1）（p－1） MSpo
SSR
np（k－1）
SST
npk－1
MSR
2
E  MS P    R2  k PO
 pk P2
2
E  MS O    R2  k PO
 nk O2
2
E  MS PO    R2  k PO
E  MS R    R2
 When the interaction exists, the unbiased estimators for gauge capability is:
2
ˆ R2  MS R
ˆ repeatabil
ity  
2
2
ˆ O2  ˆ PO
ˆ reproducib
 MS O  n  1MS PO  nMS R  nk
ility  
2
2
2
ˆ reproducib
ˆ gauge
 ˆ repeatabil
ity  
ility  MS O  n  1MS PO  n k  1MS R  nk
9
Literature
ANOVA
2
 0, usually define it 0. Assume that no interaction
 If ˆ po
exists. A reduced model is fitted as:
X ijk    Pi  O j  Rijk
i  1,2,, n 


 j  1,2, , p 
k  1,2 , κ 


 Without interaction existing, the estimators for gauge
capability are:
2
2
ˆ
ˆ repeatabil


ity
R  MS R
2
2
ˆ
ˆ reproducib


ility
O  MS O  MS R  nk
2
2
2
ˆ
ˆ gauge
 ˆ repeatabil


ity
reproducibility  MS O  nk  1MS R  nk
10
Literature
Methods for estimating gauge variance
Classical GR&R
 Montgomery and Runger （1993a）called it “Classical Gauge
Repeatability and Reproducibility Study” 。
 Estimator for repeatability：
ˆ repeatability 
R
d2
where d2 is determined by the number of repetitions k.
 Estimator for reproducibility： ˆ reproducibility 
where R  max X  j   min X  j  , X
 j
X
j
RX
d2
is the overall average of the
j
jth operator and d2 is determined by the number of operators.
11
Literature
Methods for estimating gauge variance
Long Form Method
 Introduced in the MSA manual of QS 9000 system without
interaction being considered.
 The repeatability and reproducibility estimators are：
 repeatability 
ˆ reproducibility
where
R
d2
 RX
  *
d
 2
2


  R d2

nk


2
d 2 is in appendix B（g=1，m=number of operators）
12
Literature
Repeatability and Reproducibility
Estimators
標準差
方法
ˆ repeatability
MS R
̂ reproducibility
MS O  n  1MS PO  nMS R  nk (交互作用)
變異數分析法
MS O  MS R 
傳統 GR&R
R
d2
長表格
R
d2
nk (交互作用不顯著)
R
X
d2
 RX

 d*
 2
2


  R d2

nk


2
13
Revised Classical GR&R and Long Form Methods
Classical GR&R and Long Form methods can’t be
used under the cases with interaction between
operators and parts.
Adjust the estimator of reproducibility as:
R X ij
其中， X ij
 n

  R X ij 

  i 1
n
 k

  X ijk 
i  1, , n 
   k 1
,

j

1
,

,
p
k


RX ij  max X ij  min X ij , i  1,, n
j
j
14
Revised Classical GR&R and Long Form Methods
Measurement Layout
人員
重
產 複
品
量測人員 1
量測值
x111
x 112
1
2
…
n
…
x 11k
x211
x 212
…
x 21k
…
…
xn11
x n12
…
x n1k
X 1
…
量測人員 2
平均
全距
X 11
R11
X 21
R21
…
…
X n1
R n1
R 1
量測值
x121
x 122
…
x 12k
x221
x 222
…
x 22k
…
…
xn21
x n22
…
x n2k
X  2
平均
全距
X 12
R12
X 22
R22
…
…
X n 2
R n2
R 2
量測人員 p
量測值
…
…
…
x 1p1
平均
全距
X 1 p
R1 p
X 2 p
R2 p
…
…
x 1p2
…
x 1pk
x 2p1
x 2p2
…
x 2pk
…
…
x np1
x np2
…
x npk
X  p
X
np
R np
R p
15
Revised Classical GR&R and Long Form Methods
Lin(2005) revised Classical GR&R and Long Form methods as:
R X ij 
d2
'
ˆ reproducib
ility 
"
ˆ reproducib
ility
 R X ij 
  *
 d2


2

  R d2

nk


2
Montgomery and Runger (1993a) mentioned
12
2
E(ˆ ' reproducibility )   O2   PO
  R2 n  .
Thus in the research, the estimators for GR&R are revised as the
following to make them unbiased.
ˆ ' reproducibility
"
ˆ reproducib
ility
 RX
  ij
 d2
 R X ij 
  *
 d2


2

R d2
 

n

2


2

  R d2

n


2
16
Revised Classical GR&R and Long Form Methods
Burdick and Larsen（1997）found the number of operators
have major effect on the confidence interval of repeatability
and reproducibility. Jiang（2002）proposed more operators
under the same npk vlaue. Based on the two researches, the
reproducibility estimator of Long Form method is revised as:
 RX ij
"'
ˆ reproducibility   *
 d2

2


  R d2

npk


2
17
Criterions for comparing GR&R estimators
Assume repeatability and reproducibility are
known, simulate N runs to calculate the average
values of repeatability, reproducibility, and total
gauge variance.
The criterions were used in the research:
Mean Ratio of Estimated Gauge Variance
Variance of Estimated Gauge Variance
Mean Squares Error of Estimated Gauge Variance,
(MSE）。
18
Criterions for comparing GR&R estimators
 Mean Ratio
 To evaluate accuracy of estimator to its true value
(Unbiasedness)
 The equation is：
N  ˆ 2

gauge
 量測總變異估計變量 





2



真值
 i 1   gauge 
模擬次數

N
 Decision：The closer the ratio to 1, the more accurate the
estimator is.
19
Criterions for comparing GR&R estimators
 Variance of gauge variance estimate
 After simulating N runs, N gauge variance estimates are
obtained and its variance is computed. It is used to
evaluate the precision of the gauge variance estimator.
 The equation is:


2
  ˆ gauge


 N  i 1


N




N 1
N
 ˆ
N
i 1

2
2
gauge
2
 Decision：The smaller the variance, the more precise the
estimator is, and the narrower its confidence is.
20
Criterions for comparing GR&R estimators
 Mean Square Errors（MSE）


MSE  E ˆ  


2

 
 E ˆ  E ˆ
2
 
 E E ˆ  
2
 Var ˆ  Bias 2
 MSE is composed of two parts:

Var ˆ shows the precision while bias measures the accuracy of the
estimator. MSE combines accuracy and precision into one index.
 Equation：
 ˆ
2
N
 量測總變異估計變異  真值
模擬次數
2

i 1
2
gauge
2
  gauage

N
 Decision：The smaller the MSE, the more accurate and precise the
estimator is.
21
Criterions for comparing GR&R estimators
 MSE
 Bickel and Doksum（1977）points out that MSE both
considers accuracy and precision. The estimator with
minimum MSE indicates that it is a best estimator.
 The research used MSE as a major criterion for
comparing estimators while considering mean ratio and
variance of estimated gauge variance as supplementary
rules.
22
Simulation result and comparison of estimators
模擬開始
n 為 15 , 20 和 25
p為2,3和4
k為2和3
設定產品數 n、
量測人員數 p、
量測重複次數 k
設定
2
 O2、 PO
及
產生模擬量測值
2
R
 O2 為 1　和 2
2
 po
為 0.5 和 １　 R2 為　0.25 和　0.5
模擬
10000次
估算重複性變異
再現性變異
量測總變異
量測再現性變異和總變異的平均真值比
量測再現性變異和總變異的變異數
量測再現性變異和總變異的均方誤差
模擬結束
程式模擬流程圖
23
Eight gauge variance estimators for comparison
標準差
方法
ˆ repeatability
方法
變異數分析法
（ANOVA）
MS O  n  1MS PO  nMS R  nk (交互作用)
MS R
MS O  MS R  nk
傳統 GR&R
(CRR)
長表格（LF）
R
d2
R
d2
(交互作用不顯著)
RX
R
d2
林郁智（2005）
Modified Classical
GR&R（MCRRL）
標準差
̂ reproducibility
d2
 RX

 d*
 2
2
 

  R d2

nk

R X ij 
d2
2
ˆ repeatability
林郁智（2005）
̂ reproducibility
2
 
R
d2
 R X ij 

 d 2*

Modified Classical
GR&R（MCRRN）
R
d2
 R X ij 
R d2


 d   n
 2 
Modified Long
Form（MLFN1）
R
d2
 R X ij 

 d 2*


  R d2

n

Modified Long
Form（MLFN2）
R
d2
 R X ij 

 d 2*


  R d2

npk

Modified Long
Form（MLFL）

  R d2

nk

2
2
2
2
 
2
 
2
 
2
24
Simulation result and comparison of estimators
Data from the case study of Montgomery (1993a）
GR&R 估算方法的之比較（交互作用不顯著）
ANOVA
ˆ
ˆ
CRR
LF
MCRRL
MLFL
MCRRN
MLFN1
MLFN2
2
repeatability
0.88316 1.03883 1.03883 1.03883 1.03883
1.03883 1.03883 1.03883
2
reproducibility
0.01063 0.03687 0.00298 0.33182 0.23461
0.27988 0.20864 0.25192
0.89379 1.07570 1.04182 1.37065 1.27344
1.31871 1.24747 1.29076
2
ˆ gauge
1.6
1.4
變異數估計值
1.2
ANOVA
1
CRR
0.8
LF
0.6
MCRRL
MLFL
0.4
MCRRN
0.2
MLFN1
0
MLFN2
Repeatability
Reproducibility
Gauge
25
Simulation result and comparison of estimators
Data from the case study of Montgomery (1993a）
GR&R 估算方法的之比較（交互作用顯著）
2
ˆ repeatabil
ity
ANOVA
CRR
LF
MCRRL
MLFL
MCRRN
MLFN1
MLFN2
0.81111
0.85673
0.85673
0.85673
0.85673
0.85673
0.85673
0.85673
2
1.95556 0.32617 0.22759 1.90040 1.46385 1.81473 1.40673 1.48289
ˆ reproducib
ility
2
ˆ gauge
2.76667
1.18290
1.08432
2.75713
2.32058
2.67146
2.26346
2.33962
3
變異數估計值
2.5
ANOVA
2
CRR
1.5
LF
MCRRL
1
MLFL
MCRRN
0.5
MLFN1
MLFN2
0
Repeatability
Reproducibility
Gauge
26
comparison of estimators
For the case with interaction
 Mean ratios of estimated gauge variances under
various npk values
1.600
1.400
平均真值比
ANOVA
ANOVA estimator is most
closest to 1 and is the best
one. LF estimator is the worst
one.
CRR
1.200
LF
1.000
MCRRL
MLFL
0.800
MCRRN
MLFN1
0.600
不同參數組合數之量測總變異的平均真值比之比較圖
MLFN2
60
80
90 100 120 135 150 160 180 200 225 240 300
參數組合npk
The estimators of LF and
ANOVA won’t changed with
the increase of npk values.
Other estimators will be
closer to the true value as the
npk values increase.
27
comparison of estimators
For the case with interaction
 Variance of estimated gauge variances under
various npk values
14.000
12.000
ANOVA
變異數
10.000
CRR
8.000
LF
6.000
MCRRL
4.000
MLFL
2.000
MCRRN
不同參數組合數之總變異的變異數比較圖
0.000
60 80 90 100 120 135 150 160 180 200 225 240 300
參數組合npk
MLFN1
MLFN2
MLFN1, MLFL,and MLFN2
methods have the smallest
variances. ANOVA and LF are
the second. MCRRN, MCRRL,
and CRR are the worst. All the
variances decreases as the npk
values increase.
When the npk value equals
160, all the variance estimators
decrease rapidly and then
become steady thereafter.
28
comparison of estimators
For the case with interaction
 MSE of estimated gauge variances under various
npk values
均方誤差
16.000
14.000
ANOVA
12.000
CRR
LF
10.000
MCRRL
8.000
MLFL
6.000
MCRRN
4.000
MLFN1
2.000
不同參數組合數之量測總變異的均方誤差之比較圖
MLFN2
0.000
60 80 90 100 120 135 150 160 180 200 225 240 300
參數組合npk
MLFN2, MLFL, and
MLFN1methods have the
smallest MSE values while
ANOVA and LF methods are
the second. MCRRN, MCRRL,
and CRR are the worst ones.
All the MSE values decrease
with the increase of npk vlaues.
When the npk value equals
160, all the variance estimators
decrease rapidly and then
become steady thereafter. 29
comparison of estimators
For the case with interaction
 The MSE values of estimated gauge variances while npk equals 120.
5
25
4.5
ANOVA
4
LF
3
MCRRL
2.5
MLFL
2
MCRRN
1.5
MLFN1
1
均方誤差
3.5
均方誤差
ANOVA
20
CRR
CRR
LF
15
MCRRL
MLFL
10
MCRRN
MLFN1
5
MLFN2
0.5
MLFN2
0
0
1.75
2
2.25
2.5
2.75
3
3.25
1.75
3.5
2
2.25
2.5
2.75
3
3.25
3.5
量測總變異
量測總變異
（15,4,2）量測總變異的均方誤差比較圖
（20,2,3）量測總變異的均方誤差比較圖
均方誤差
8
7
ANOVA
6
CRR
5
LF
MCRRL
4
MLFL
3
MCRRN
2
MLFN1
1
MLFN2
0
1.75
2
2.25
2.5
2.75
3
3.25
3.5
量測總變異
（20,3,2）量測總變異的均方誤差比較圖
Given npk value being fixed,
increasing the number of operators is
suggested first. The second choice is to
increase the number of parts.
Increasing the number of repetitions is
not recommended.
30
comparison of estimators
For the case without interaction
 Mean ratios of estimated gauge variances under various
npk values
ANOVA estimator is the most
closest to 1 and is the best one.
1.600
ANOVA
平均真值比
1.400
CRR
1.200
LF
1.000
MCRRL
MLFL
0.800
MCRRN
不同參數組合數之量測總變異的平均真值比之比較圖
0.600
MLFN1
60 80 90 100 120 135 150 160 180 200 225 240 300
MLFN2
參數組合npk
LF, MLFN1, MLFL, and MLFN2
methods are close to one another, and
there is only little difference among
them and ANOVA method. CRR,
MCRRN, and MCRRL methods are
the worst.
LF, ANOVA, MLFN1, MLFL, and
MLFN2 won’t change as the npk
increases while MCRRL, MCRRN,
and CRR get closer to true value.
31
comparison of estimators
For the case without interaction
 Variance of estimated gauge variances under
various npk values
變異數
14.000
12.000
ANOVA
10.000
CRR
8.000
LF
6.000
MCRRL
4.000
MLFL
2.000
MCRRN
不同參數組合數之總變異的變異數比較圖
0.000
60 80 90 100 120 135 150 160 180 200 225 240 300
參數組合npk
MLFN1
MLFN2
ANOVA, MLFN1, MLFL,
MLFN2, and LF methods are
close to one another. CRR,
MCRRN, MCRRLare the worst.
All the variances decreases as
the npk values increase.
When the npk value equals
160, all the variance estimators
decrease rapidly and then
become steady thereafter.
32
comparison of estimators
For the case without interaction
 MSE of estimated gauge variances under various
npk values
ANOVA, MLFN1, MLFL,
均方誤差
16.000
14.000
ANOVA
12.000
CRR
10.000
LF
8.000
MCRRL
6.000
MLFL
4.000
2.000
MCRRN
不同參數組合數之量測總變異的均方誤差之比較圖
0.000
MLFN1
60 80 90 100 120 135 150 160 180 200 225 240 300
參數組合npk
MLFN2
MLFN2, and LF methods are
the same good. CRR, MCRRN,
and MCRRL are the worst.
All the variances decreases
as the npk values increase.
When the npk value equals
160, all the variance
estimators decrease rapidly
and then become steady
thereafter.
33
comparison of estimators
For the case without interaction
 The MSE values of estimated gauge variances while npk equals 120.
5
25
ANOVA
CRR
3
LF
2
MCRRL
1
MLFL
ANOVA
20
均方誤差
均方誤差
4
CRR
15
LF
10
MCRRL
MLFL
5
MCRRN
0
1.25
1.5
2.25
2.5
MLFN1
MLFN2
量測總變異
ANOVA
均方誤差
6
CRR
LF
4
MCRRL
2
MLFL
MCRRN
0
2.25
1.5
2.25
MLFN1
2.5
MLFN2
（20,2,3）量測總變異的均方誤差比較圖
8
1.5
1.25
量測總變異
（15,4,2）量測總變異的均方誤差比較圖
1.25
MCRRN
0
2.5
MLFN1
MLFN2
Given npk value being fixed,
increasing the number of operators is
suggested first. The second choice is
increasing the number of parts.
Increasing the number of repetitions is
not recommended.
量測總變異
34
（20,3,2）量測總變異的均方誤差比較圖
Conclusion
 MLFN1 and MLFN2 are good estimators both in the
cases of with interaction and without interaction.
MLFN2 method is a little better than MLFN1.
 Under the case with interaction, MLFN1, MLFN2, and
MCRRN methods are better than Classical R&R and
Long Form methods. MLFN2 estimator is the same good as
ANOVA method.
 Suggest using MLFN2 method, both its accuracy and
precision are the same good as ANOVA method no matter
there is interaction or not.
35
Conclusion
 Given npk value being fixed, increasing the
number of operators is suggested first. The
second choice is increasing the number of parts.
Increasing the number of repetitions is not
recommended.
 At least three operators is suggested so that the
variance and MSE of estimated gauge variance will
be small enough.
 An npk value of 160 is suggested so that the variance
and MSE of estimated gauge variance decrease
rapidly and then become steady thereafter.
36
Thanks for your attention
37