Gauge R&R: A GUM/Metrological/
Bayesian Perspective
Dave LeBlond
MBSW-38
May 19, 2015
1
Acknowledgements
Thanks to:
• Stan Altan (J&J)
• Bill Porter (PPP LLC)
• Yan Shen (J&J)
• Jyh-Ming Shoung (J&J)
for organizing this session, inviting me, inspiring
discussions, and providing the 3 Gauge R&R examples.
2
Outline
• Measurement Uncertainty (MU) from a Bayesian perspective
• Computational considerations
•
Examples
1. Dissolution measurement
2. Particle size measurement
3. Bioassay measurement
•
Conclusions
•
Bibliography
3
The GUM perspective
ISO/GUM compliant practice
Current practice
Reportable result =
m
±U
Q: What is U?
A: “A parameter, associated with the result of a measurement, that
characterizes the dispersion of the values that could reasonably be
attributed to the measurand” (i.e., The part of the result after the ±).
4
MU requires a complete probability model
Reportable result =
m
±U
• Let m be an analytical measurement that estimates m
• Let m be the unknown measurand quantity
• fixed, but uncertain
• MU refers to the uncertainty in m (not m)
• ISO: “…values that could reasonably be attributed to the
measurand.”
• (Don’t forget to put the u in the mu)
• The posterior, p(m|m), expresses current knowledge about m
• To get p(m|m), we need prior knowledge and Bayes’ rule
• Inference about m requires a Bayesian approach
5
U has a probability interpretation
Reportable result =
m
±U
• Bayesian interpretation: posterior distribution of m, conditional on m
• Assume symmetrical, location-scale family with scale independent of m
m U
p(m|m)
 p  m | m  d m  e.g., 0.95
m U
m-U
m
m+U
m
6
MU needs 2 spaces & 2 experiments
Space
Observed
Parameter
Experiment
p(q)
Gauge R&R
p(s)
p(y|q,s)
p’(s) = p(s|y)  ∫p(y|q,s)p(q)p(s)dq
p(m)
Routine
Measurement
p’(s)
p(m|m,s)
p(m|m)  ∫p(m|m,s)p(m)p’(s)ds
likelihood
prior
posterior
y = vector of observed Gauge R&R results
q = vector of true levels
s = true analytical imprecision
m = true but uncertain measurand quantity value
m = analytical measurement that estimates m
7
MU needs 2 spaces & 2 experiments
Space
Observed
Parameter
Experiment
p(q)
Gauge R&R
p(s)
p(y|q,s)
p’(s) = p(s|y)  ∫p(y|q,s)p(q)p(s)dq
p(m)
Routine
Measurement
p’(s)
p(m|m,s)
p(m|m)  ∫p(m|m,s)p(m)p’(s)ds
likelihood
prior
posterior
Assume
• m is unbiased for m
• normal likelihoods
• diffuse priors for q and m: p(q) = p(m) = 1
Key outputs
• Gauge R&R: p’(s) and U
• Routine Measurement: p(m|m) … or at least m ± U
8
Getting ∫p(m|m,s)p(m)p’(s)ds by MC Integration
likelihood
MCMC draws
Diffuse
prior
l(m|m,s)
location-scale
m arbitrary
s*
p(m) = 1
l(m|m=0,s)
Plug in
each s*
p(m*|m=0)
l(m|m=0, s*)
Obtain 1 MC draw
for each s*
m*
Estimate
percentiles
P2.5
0
P97.5
m*
Estimate U
U = ½(P97.5 – P2.5)
9
Sometimes buried in the SAS Log…
NOTE: Convergence criteria met.
NOTE: Estimated G matrix is not positive definite.
NOTE: Asymptotic variance matrix of covariance parameter estimates has been
found to be singular and a generalized inverse was used. Covariance
parameters with zero variance do not contribute to degrees of
freedom computed by DDFM=KENWARDROGER.
Covariance Parameter Estimates
Cov Parm
Day(Site)
ExpRu*HPLC(Site*Day)
ExpRun(Day)
ExpRun(Day)
ExpRun(Day)
ExpRun(Day)
Residual
Residual
Residual
Residual
Group
Estimate
Standard
Error
Site
Site
Site
Site
Site
Site
Site
Site
0.2612
0.9645
6.3519
0.8339
10.3038
0
5.7744
3.0847
8.6020
6.7212
0.5356
0.5322
3.5080
0.9441
5.2917
.
1.1481
0.6446
1.6357
1.2652
G
I
L
T
G
I
L
T
Lower
0.03723
0.4129
2.7176
0.2028
4.6221
.
4.0502
2.1285
6.1194
4.7965
Upper
503733
4.2521
28.0595
80.8289
39.6342
.
8.8978
4.8713
12.9815
10.0968
MIXED arbitrarily sets a negative variance estimate to zero, effectively
performing model reduction and ignoring uncertainty in the estimate.
The Bayesian prior would disallow a negative variance so that an interval
estimate of the variance is available.
10
Implementing MCMC
• An MCMC chain is a correlated multivariate sample from the multivariate posterior
distribution of all parameters in the model, given the data and prior assumptions.
• Independent “noninformative” (wide uniform or normal with huge variance) used
for all parameters.
• Square root of variance components as parameters
• Run 3 MCMC chains from different (random) starting points
• Only retain every 40th iteration as a draw (to reduce autocorrelation)
• Discard first 3,000 draws from each chain (“burn-in”)
• Save 10,000 draws from each chain (30,000 total)
• Checked for convergence of the 3 chains
11
Comparison of approaches
Consideration
MIXED
BUGS
Estimation Method
REML, ML, MIVQUE0, Type1,
Type2, Type3
MCMC, method chosen by
BUGS
Denominator df
Method
BW, CON, KR, RES, SAT, DDF=list
No worries
Multiple Comparison
Adjustment Method
BON, SCHEFFE, SIDAK,
SIMULATE, T
No worries
Effect coefficients
E1, E2, E3
No worries
CONVF, CONVG, CONVH, DFBW,
Computational options EMPIRICAL, NOBOUND,
No worries
RIDGE=, SCORING=, NOPROFILE
Asymptotic normality? yes
No approximations
Iteration convergence
Warnings provided
Manual
Speed
Generally very fast
Can be slow (hours)
Syntax mimics…
Mixed model algebra
Data generation
Non-negative
bounding of variances
Arbitrarily set to zero
Handled through prior
12
1. Data
i indexes Site (S, 4 unique sites)
j indexes Day within Site (SD, 24 unique Days)
k indexes ExpRun within Site (E, 48 unique
ExpRuns)
l indexes HPLC*ExpRun combinations within Site
(HE, 96 unique combinations)
m indexes Batch (B, 3 unique batches)
obs
1
2
3
…
70
71
72
73
74
75
…
142
143
144
145
146
147
…
214
215
216
217
218
219
…
286
287
288
i
2
2
2
j k l m
y
7 20 25 2 81.00
7 20 25 3 80.62
7 20 25 1 82.46
2 12 23 48 3 83.36
2 12 23 48 2 85.23
2 12 23 48 1 85.01
1 1 8 1 2 90.39
1 1 8 1 3 95.50
1 1 8 1 1 92.40
1 6 11 24
1 6 11 24
1 6 11 24
3 13 32 49
3 13 32 49
3 13 32 49
3
2
1
2
3
1
81.01
86.82
91.73
87.12
88.72
86.20
3
3
3
4
4
4
3
2
1
2
3
1
76.74
81.62
82.42
93.51
91.59
92.07
18
18
18
19
19
19
35
35
35
44
44
44
72
72
72
73
73
73
4 24 47 96 3 85.47
4 24 47 96 2 90.00
4 24 47 96 1 89.44
13
1. Statistical model
Diss30
Site
Batch
y obs  q  Si obs   Bmobs 
“fixed”(but uncertain) effects
obs  1...288
i obs   1...4
m obs   1...3
Set to zero restrictions
Day(Site)
ExpRun(Day) HPLC*ExpRun(Site*Day)
 SD j obs   Ek obs   HEl obs    obs
measurement uncertainty effects
SD j obs  ~ N  0,sigma.SD 2  , j obs   1...24
Ek obs  ~ N  0,sigma.Ei2[ obs ]  ,k obs   1...48
HEl obs  ~ N  0,sigma.HE 2  ,l obs   1...96

 obs ~ N 0,sigmai2obs 

S4  B3  0
14
model{
1. BUGS model
# Priors
theta~dnorm(0,0.000001)
# Likelihood
for(obs in 1:n.obs){
mu[obs] <- theta + S[i[obs]] + B[m[obs]] + SD[j[obs]] + E[k[obs]]+ HE[l[obs]]
y[obs] ~ dnorm(mu[obs],tau[i[obs]])
}
for(jj in 1:24){
SD[jj] ~ dnorm(0,tau.SD)
}
for(ll in 1:96){
HE[ll] ~ dnorm(0,tau.HE)
}
for(kk in 1:12){
E[kk] ~ dnorm(0,tau.E[1])
}
for(kk in 13:24){
E[kk] ~ dnorm(0,tau.E[2])
}
for(kk in 25:36){
E[kk] ~ dnorm(0,tau.E[3])
}
for(kk in 37:48){
E[kk] ~ dnorm(0,tau.E[4])
}
for(site in 1:3){
S[site]~dnorm(0,0.000001)
}
S[4]<- 0
for(batch in 1:2){
B[batch]~dnorm(0,0.000001)
}
B[3]<- 0
for(site in 1:4){
sigma[site] ~ dunif(0,100)
tau[site] <- pow(sigma[site],-2)
sigma.E[site] ~ dunif(0,100)
tau.E[site] <- pow(sigma.E[site],-2)
}
sigma.SD~dunif(0,100)
tau.SD<-pow(sigma.SD,-2)
sigma.HE~dunif(0,100)
tau.HE<-pow(sigma.HE,-2)
}
15
1. Joint posterior marginals
theta
S[1]
S[2]
S[3]
B[1]
B[2]
sigma.SD
sigma.E[1]
sigma.E[2]
sigma.E[3]
sigma.E[4]
sigma.HE
sigma[1]
sigma[2]
sigma[3]
sigma[4]
mean
87.86865
-1.85345
-4.80681
-6.78676
1.67055
-0.57506
0.63573
2.81409
0.99404
3.65876
0.56074
0.95419
2.47125
1.82693
3.00323
2.65304
sd
0.56354
1.10904
0.75670
1.32984
0.33997
0.34345
0.42448
0.91456
0.57189
1.09118
0.44593
0.31807
0.25124
0.19887
0.29334
0.25404
2.5%
86.75000
-4.06602
-6.31602
-9.42800
1.00500
-1.24800
0.02972
1.39297
0.08034
2.02900
0.02289
0.21159
2.03500
1.48300
2.49100
2.20997
25%
87.50000
-2.57400
-5.29025
-7.63400
1.44500
-0.80692
0.30180
2.19200
0.58310
2.90500
0.22100
0.76710
2.29400
1.68600
2.79800
2.47400
50%
87.87000
-1.85200
-4.80400
-6.78500
1.66700
-0.57430
0.58095
2.68500
0.94945
3.48000
0.46070
0.97680
2.45200
1.81000
2.98200
2.63400
75%
88.23000
-1.14400
-4.32000
-5.93800
1.90100
-0.34580
0.89732
3.28100
1.32700
4.22000
0.79032
1.17100
2.62800
1.95200
3.18300
2.80900
97.5%
88.99000
0.33300
-3.29998
-4.18898
2.34400
0.10550
1.59600
4.99000
2.25900
6.24702
1.66800
1.52400
3.01800
2.26100
3.64602
3.20302
Rhat
1.00100
1.00098
1.00101
1.00102
1.00098
1.00100
1.00151
1.00095
1.00095
1.00097
1.00109
1.01856
1.00114
1.00102
1.00095
1.00106
n.eff
30000
30000
30000
29000
30000
30000
3500
30000
30000
30000
14000
1100
11000
30000
30000
19000
16
1. Marginal posterior distribution of sigma by
site
Posterior for sigma
1
L
2
3
4
5
T
15
10
Percent of Total
5
0
G
I
15
10
5
0
1
2
3
4
5
sigma
30,000 mcmc draws
17
1. Marginal posterior distribution of sigma.E by
site
Posterior for sigma.E
0
L
5
10
T
30
20
Percent of Total
10
0
G
I
30
20
10
0
0
5
10
sigma.E
30,000 mcmc draws
18
1. Marginal Posterior distributions of sigma.SD and
sigma.HE
Posterior for sigma.SD
Posterior for sigma.HE
6
Percent of Total
Percent of Total
6
4
2
0
4
2
0
0
1
2
sigma.SD
30,000 mcmc draws
3
0.0
0.5
1.0
1.5
2.0
2.5
sigma.HE
30,000 mcmc draws
19
0.2
0.5
0.6
0.7
0.
9
0.4
0.5
0.3
sigma.E4
1.0
0.1
1.5
1. Bivariate posterior kernal density
4
0.
0.0
1
0.0
0.8
0.5
1.0
1.5
sigma.SD
30,000 draws
20
1. Variance Components Estimates
(SAS, Bayesian)
Variance Components
Group
Day(Site)
DissRun*HPLC(Site*Day)
Site G
Site I
DissRun(Day)
Site L
Site T
Site G
Site I
Residual
Site L
Site T
Estimate (REML,
median)
0.26
0.34
0.96
0.95
6.35
7.21
0.83
0.90
10.30
12.11
0.00
0.21
5.77
6.01
3.08
3.28
8.60
8.89
6.72
6.94
Lower95%
Upper95%
0.04
0.0009
0.41
0.04
2.72
1.94
0.20
0.006
4.62
4.12
0.00
0.0005
4.05
4.14
2.13
2.20
6.12
6.21
4.80
4.88
5.04E+05
2.55
4.25
2.32
28.06
24.90
80.83
5.10
39.63
39.02
0.00
2.78
8.90
9.11
4.87
5.11
12.98
13.29
10.10
10.26
21
1. Error Budget
(SAS, Bayesian)
Site
G
I
L
T
Source
% of Total
Residual
Day(Site)
DissRun*HPLC(Site*Day)
DissRun(Day)
Total
Residual
Day(Site)
DissRun*HPLC(Site*Day)
DissRun(Day)
Total
Residual
Day(Site)
DissRun*HPLC(Site*Day)
DissRun(Day)
Total
Residual
Day(Site)
DissRun*HPLC(Site*Day)
DissRun(Day)
Total
43
2
7
48
100
60
5
19
16
100
43
1
5
51
100
85
3
12
0
100
Posterior quantiles of % of Total
Median(2.5th-97.5th quantile)
40(18-65)
2(0-18)
6(0-17)
49(18-78)
56(30-81)
6(0-33)
16(1-39)
16(0-53)
39(17-64)
1(0-11)
4(0-11)
54(26-80)
79(56-94)
4(0-24)
10(1-25)
2(0-25)
22
1. Posterior distribution of Error
m  m  Error
Error  SD  E  HE  
[1]
SD ~ N  0,sigma.SD 2 
E ~ N  0,sigma.Ei2 
HE ~ N  0,sigma.HE 2 
[2]
 sigmai2 
 ~ N  0,

6


To obtain a sample from the posterior distribution of Error:
1. Start with 30,000 MCMC draws of the 4 sigmas (a 4-vector for a given site i)
2. For each draw vector,
a. simulate a random sample of the 4 error contributors using [2]
b. Plug these into [1]
3. The result is 30,000 MCMC draws from the posterior distribution of Error
23
1. Expanded uncertainties (U) for each site
• 95% credible interval of posterior distribution of Error: 0 ± U
• Results should be reported as m ± U
Posterior Distribution of Error at 4 Sites
-10
Site : 3
0
Site : 4
U = 3.63
(SAS: 3.06)
Percent of Total
U = 8.38
(SAS: 7.20)
Site : 1
25
20
15
10
5
0
0
25
20
15
10
5
0
Site : 2
U = 6.66
(SAS: 5.84)
-10
10
U = 3.72
(SAS: 3.21)
10
Error
30,000 mcmc draws
24
1. Adjusting for site bias
G
84
86
T
82
I
L
80
95%CI of Site Mean
88
Site Means for Batch C
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Site
(based on 30,000 mcmc draws)
• ISO/GUM philosophy: Do everything in your power to adjust for bias
• First must decide what “truth” is
• Is one site the “reference” site?
• Take the average across sites as the “truth”?
• Determine the bias appropriate for each site.
• Adjust reported values from each site by subtracting the site-bias
25
2. Statistical model
Dv50a
Site
Batch
Day(Site)
Day*Batch(Site)
y obs  q  Si obs   B j obs 
 SDk obs   SBDl obs    obs
“fixed”(but uncertain) effects
measurement uncertainty effects
obs  1...252
i obs   1...7
j obs   1...6
Set to zero restrictions
SDk obs  ~ N  0,sigma.SD 2  ,k obs   1...21
SBDl obs  ~ N  0,sigma.SBD 2  ,l obs   1...84
 obs ~ N  0,sigma2 
S7  B6  0
26
2. Joint posterior marginals
mean
theta
3.55274
S[1]
-0.10220
S[2]
-0.00934
S[3]
0.10382
S[4]
-0.01691
S[5]
0.00924
S[6]
-0.01911
B[1]
-0.27059
B[2]
0.35566
B[3]
-0.24313
B[4]
-0.04624
B[5]
0.04943
sigma.SD
0.02372
sigma.SBD 0.04174
sigma
0.24960
sd
0.06061
0.06714
0.06736
0.06718
0.06750
0.06726
0.06704
0.05847
0.05898
0.05759
0.05878
0.05874
0.01882
0.02713
0.01202
2.5%
3.43300
-0.23440
-0.14100
-0.02814
-0.14980
-0.12190
-0.15110
-0.38580
0.24000
-0.35480
-0.16140
-0.06520
0.00089
0.00200
0.22710
25%
3.51300
-0.14660
-0.05363
0.05949
-0.06217
-0.03564
-0.06396
-0.30962
0.31590
-0.28230
-0.08572
0.00971
0.00914
0.01944
0.24130
50%
3.55300
-0.10230
-0.00958
0.10385
-0.01745
0.00929
-0.01881
-0.27090
0.35510
-0.24310
-0.04664
0.04954
0.01956
0.03869
0.24920
75%
97.5%
3.59300 3.67200
-0.05748 0.03012
0.03472 0.12530
0.14870 0.23650
0.02786 0.11630
0.05394 0.14190
0.02593 0.11090
-0.23100 -0.15500
0.39510 0.47280
-0.20450 -0.12980
-0.00697 0.06928
0.08884 0.16480
0.03373 0.07047
0.06058 0.09957
0.25750 0.27440
Rhat
1.00097
1.00096
1.00104
1.00097
1.00108
1.00110
1.00118
1.00100
1.00101
1.00095
1.00104
1.00096
1.00164
1.00308
1.00098
n.eff
30000
30000
21000
30000
15000
13000
8500
30000
30000
30000
23000
30000
2900
2600
30000
27
2. Variance Component Estimates
(SAS, Bayesian)
Variance
Components
Estimate Lower95% Upper95%
% of Total
Day(SITE_APP)
0
0.0004
?
8E-7
?
0.005
0
1(0-7)
BATCH*Day
(SITE_APP)
0.00047
0.0015
0.00006
4E-6
3.61E+109
0.01
1
2(0-14)
Residual
0.06
0.06
0.05
0.05
0.08
0.08
99
96(83-100)
Total
0.06
0.07
0.05
0.05
3.61E+109
0.08
100
100
Bayesian point estimate is the posterior median
Bayesian % of Total is posterior median(central 95% credible interval)
28
2.
Posterior for sigma.SD
3000
0
1000
Frequency
500 1000
0
0.05
0.10
0.15
sigma.SBD
30,000 mcmc draws
0.00
0.05
0.10
0.15
sigma.SD
30,000 mcmc draws
500 1000
2000
Posterior for sigma
0
0.00
Frequency
Frequency
2000
Posterior for sigma.SBD
0.20
0.22
0.24
0.26
sigma
30,000 mcmc draws
0.28
0.30
29
8
6
0.0
7
2
0.0
8
0.0
0.11
0.1
0.0
0.
06
4
0.
05
0
Day*Batch variance (% of Total)
10
2. Bivariate posterior kernal density of 2 error
budget components
0.5
1.0
0.01
0.02
4
0.03
1.5
2.0
2.5
Day to Day variance (% of Total)
30,000 draws
3.0
30
2. Posterior distribution of Error
m  m  Error
Error  SD  SBD  
[1]
SD ~ N  0,sigma.SD 2 
SBD ~ N  0,sigma.SBD 2 
[2]
 ~ N  0,sigma2 
To obtain a sample from the posterior distribution of Error:
1. Start with 30,000 MCMC draws of the 3 sigmas (a 3-vector)
2. For each draw vector,
a. simulate a random sample of the Error contributors using [2]
b. Plug these into [1]
3. The result is 30,000 MCMC draws from the posterior distribution of Error
31
2. Expanded uncertainty (U)
• 95% credible interval of posterior distribution of Error: 0 ± U
• Future results should be reported as m ± U
Posterior Distribution of Error
Percent of Total
6
U = 0.5031
(SAS: 0.5027)
4
2
0
-1.0
-0.5
0.0
0.5
1.0
Error
30,000 mcmc draws
32
3. Statistical model
Log_assay
Analyst
Conc
y obs  q  Ai obs   C j obs 
“fixed”(but uncertain) effects
Run(Analyst) Plate(Run*Analyst)
 Rk obs   Pl obs    obs
measurement uncertainty effects
obs  1...60
i obs   1...2
j obs   1...6
Set to zero restrictions
Rk obs  ~ N  0,sigma.R 2  ,k obs   1...10
Pl obs  ~ N  0,sigma.P 2  ,l obs   1...20
 obs ~ N  0,sigma2 
A2  C6  0
33
3. Joint posterior marginals
mean
sd
2.5%
25%
50%
75%
97.5%
theta
4.62917 0.03696
4.55700
4.60600
4.62900
4.65200
4.70300
A[1]
0.02415 0.04812
-0.07098
-0.00528
0.02386
0.05341
0.12050
C[1]
-0.11067 0.02275
-0.15490
-0.12580
-0.11090
-0.09574
-0.06523
C[2]
-0.03164 0.02257
-0.07554
-0.04675
-0.03179
-0.01675
0.01311
C[3]
0.02867 0.02249
-0.01549
0.01371
0.02860
0.04377
0.07298
C[4]
0.02143 0.02304
-0.02285
0.00603
0.02111
0.03644
0.06818
C[5]
0.00480 0.02271
-0.03912
-0.01040
0.00459
0.01977
0.05001
sigma.R
0.04537 0.03047
0.00243
0.02309
0.04154
0.06171
0.11680
sigma.P
0.06873 0.01772
0.03955
0.05644
0.06687
0.07890
0.10870
sigma
0.04523 0.00574
0.03568
0.04117
0.04466
0.04867
0.05811
deviance -203.08263 10.09257 -219.90000 -210.30000 -204.20000 -197.00000 -180.70000
Rhat
1.00103
1.00098
1.00100
1.00100
1.00099
1.00099
1.00101
1.00105
1.00099
1.00100
1.00100
n.eff
36000
60000
60000
60000
60000
60000
52000
25000
60000
60000
60000
34
3. Variance Components Estimates (in log scale)
(SAS, Bayesian)
Variance
Components
Run(Analyst)
Plate(Analyst*Run)
Residual
Total
Estimate
0.0015
0.0017
0.0039
0.0045
0.0019
0.0020
0.0073
0.0090
Lower95% Upper95%
0.0003
0.000006
0.0017
0.0016
0.0012
0.0013
0.0032
0.0051
1.4679
0.0136
0.0159
0.0118
0.0032
0.0034
1.4871
0.0219
% of Total
21
21(0-71)
53
53(15-83)
26
22(8-45)
100
100
Bayesian point estimate is the posterior median
Bayesian % of Total is posterior median(2.5th-97.5 percentiles)
35
3. Posterior distribution of Error
m  m  Error
Error  R  P  
 sigma.R 2 
R ~ N  0,

3


 sigma.P 2 
SBD ~ N  0,

6


[1]
[2]
 sigma2 
 ~ N  0,

6 

To obtain a sample from the posterior distribution of Error:
1. Start with 60,000 MCMC draws of the 3 sigmas (a 3-vector)
2. For each draw vector,
a. simulate a random sample of the 3 error contributors using [2]
b. Plug these into [1]
3. The result is 60,000 MCMC draws from the posterior distribution of Error
36
3. Expanded uncertainty
Percent of Total
-U = -0.0924
+U = +0.0924
10
m  m U
5
0
-0.1
0.0
0.1
Error
Translate to the Relative Potency scale
e m  e m U  e m  e U ,e U 
Percent of Total
e-U = 0.912
e+U = 1.097
10
em
em – em-U
em+U - em
80
7.1
7.7
102.5
9.0
9.9
120
10.6
11.6
SAS
5
0
0.9
1.0
exp(Error)
60,000 mcmc draws
1.1
1.2
7.8
37
Conclusions
Metrological Approach
Pros
• Scientifically sound
• Model based (forces analytical introspection)
• ISO compliant
Cons
• Learning curve for CMC, regulators
• Metrological approach still evolving (slowly)
• How to deal with site and/or instrument biases?
• How to deal with transformed scales of measurement?
Bayesian Version of Metrological Approach
Pros
• Permits direct probability statements (risk management)
• BUGS syntax mimics data generation mechanism
• GUM revision moving toward Bayesian perspective
Cons
• Steep learning curve for CMC, regulators
• Unfamiliar software tools (BUGS, R, STAN, JAGS,…)
• MCMC requires care, maybe long computing times
38
Bibliography
1. Gelman A, et al (2014) Bayesian data analysis, 3rd edn, CRC Press
2. Burdick R, et al (2005) Design and analysis of gauge R&R studies, SIAM
3. Willink R (2013) Measurement uncertainty and probability, Cambridge University Press [gives
a frequentist perspective]
4. Working Group 1 of the Joint Committee for Guides in Metrology (JCGM/WG 1), JCGM
100:2008, GUM 1995 with minor corrections, Evaluation of measurement data — Guide to
the expression of uncertainty in measurement.
5. Working Group 2 of the Joint Committee for Guides in Metrology (JCGM/WG 2), JCGM
200:2012, VIM, 3rd edition, 2008 version with minor corrections, International vocabulary of
metrology – Basic and general concepts and ssociated terms.
6. Eurachem working group (Editors: S L R Ellison , A Williams) Eurachem/CITAC Guide CG 4
(2012), Quantifying Uncertainty in Analytical Measurement, 3rd edn.
7. Hubert et al (2004, 2007) Harmonization strategies for the validation of quantitative
analytical procedures: A SFSTP proposal Part 1, J Pharm Biomed Anal 36, 579-586. Part 2, J
Pharm Biomed Anal 45: 70-81, Part 3, J Pharm Biomed Anal 45: 82-96.
8. Feinberg et al (2004) New advances in method validation and measurement uncertainty
aimed at improving the quality of chemical data, Anal Bioanal Chem 380:502-514.
9. Howson C, and Urbach P, Scientific reasoning: the Bayesian approach 3rd edn. , Open Court,
Chicago, IL [argues that scientific inference requires a Bayesian perspective]
39
“… To endure uncertainty is difficult,
but so are most of the other great
virtues”.
- Bertrand Russell, 1950
- Thank you for your endurance!!
P.S. Don’t forget to put the u in the mu
40