Evaluation of Measurement
Uncertainties Using the
Monte Carlo Method
Speaker: Chung Yin, Poon
Standards and Calibration Laboratory (SCL)
The Government of the Hong Kong Special
Administrative Region
1
Standards and Calibration Laboratory, SCL
GUM Uncertainty Framework (GUF)
“Propagation of Uncertainties”
Measurement Model: Y= f(X1, X2, XN)
Estimate xi of the input quantities Xi
Determine u(xi) associated with each estimate xi and its degrees
of freedom
Estimate y = f(xi) of Y
Calculate the sensitivity coefficient of each xi at Xi = xi
Calculate u(y)
Calculate the effective degrees of freedom veff and coverage
factor k with coverage probability p
Calculate the coverage interval: yku(y)
2
Standards and Calibration Laboratory, SCL
GUM Uncertainty Framework (GUF)
x1, u(x1)
x2, u(x2)
Problems:
Y = f (X)
y, u(y)
xN, u(xN)
 The contributory uncertainties are not of
approximately the same magnitude
 Difficult to provide the partial derivatives of the
model
 The PDF for output quantity is not a Gaussian
distribution or a scaled and shifted t-distribution
3
Standards and Calibration Laboratory, SCL
Monte Carlo Method (MCM)
“Propagation of Distributions”
Measurement Model: Y= f(X1, X2, XN)
Assign probability density function (PDF) to each X
Select M for the number of Monte Carlo trials
Generate M vectors by sampling from the PDF of each X
(x1,1, x1,2,  x1,M)  (xN,1, xN,2,  xN,M)
Calculate M model values y = (f(x1,1,  xN,1),  f(x1,M,  xN,M))
Estimate y of Y and associated standard uncertainty u(y)
Calculate the interval [ylow,yhigh] for Y with corresponding
coverage probability p
4
Standards and Calibration Laboratory, SCL
Monte Carlo Method (MCM)
gX1(x1)
gX2(x2)
Y = f (X)
gY(h)
gXN(xN)
5
Standards and Calibration Laboratory, SCL
Operation Modes For MCM
• There are three modes of operations
– Fixed-Number-of-Trials Mode
– Adaptive Mode
– Approximated Adaptive (or Histogram) Mode
6
Standards and Calibration Laboratory, SCL
Adaptive Monte Carlo Procedure
Calculate sy, su(y), syhigh, sylow
Set number of significant digit
ndig (usually = 2)
Use h  M model values
calculate u(y) and then d
Set M = max (100/(1-p), 104)
h=1
Y
Perform MCM trial
y(h)=(y1(h),y2(h),...yM(h))
Any 2sy, 2su(y), 2sylow or
2syhigh > d/5?
N
Calculate u(y(h)), ylow(h), yhigh(h)
h>1?
N
Y
Use h  M model
values calculate y,
u(y), ylow, yhigh
h=h+1
7
Standards and Calibration Laboratory, SCL
Validation of GUF
• Calculate: dlow = y – Up – ylow  and
dhigh = y + Up – yhigh 
• If both differences are not larger than d, then
the GUF is validated.
8
Standards and Calibration Laboratory, SCL
Histogram Procedure
• If the numerical tolerance d is small, the value
of M required would be larger. This may
causes efficiency problems for some
computers
• Experiences show that a very precise
measurement will require a M of up to 107
• Using histogram to approximate the PDF
9
Standards and Calibration Laboratory, SCL
Histogram Procedure
1.
2.
Build the initial histogram for y with
Bin = 100,000
Continue generate the model
 Update y and u(y) for each iteration
 Check stabilization. (Same as the adaptive
procedure, i.e. check the four s values)
 Update the histogram
 Store the outliers (i.e. those values beyond
the boundaries of the histogram)
10
Standards and Calibration Laboratory, SCL
Histogram Procedure
4.
When stabilized:
 Build complete histogram to include the
outliers
 Transform the histogram to a
distribution function
 Use this discrete approximation to
calculate the coverage interval
11
Standards and Calibration Laboratory, SCL
Determine Coverage Intervals
• By Inverse linear interpolation [Annex D.5 to D.8 of GS1]
GY(h)
p
ylow = y(r ) + (y(r+1) - y(r ) )
pr+1
a
pr+1 - pr
Similary
pr
y(r+1)
ylow
y(r)
y
GY(h)
p
ps+1
p+a
ps
y(s+1)
yhigh
y(s)
12
a - pr
p + a - ps
yhigh = y(s) + (y(s+1) - y(s) )
ps+1 - ps
y
Standards and Calibration Laboratory, SCL
Shortest Coverage Interval
• Repeat the method to determine a large
number of intervals corresponding to
(a, p+a) and find the minimum value.
E.g. a = 0 to 0.05 for 95 % coverage interval.
• The precision level is related to the
incremental step of a in the search.
• The step uses in this software is 0.0001, i.e.
total 501 steps.
13
Standards and Calibration Laboratory, SCL
MCM Software
MCM Engine
(MATLAB Codes)
MCM Code
Generator
(VB Codes)
MATLAB
Script File
- Perform MCM Procedure
-GUF Validation
-Graphical Output of
Results
14
Standards and Calibration Laboratory, SCL
GUI of the MCM Code Generator
15
Standards and Calibration Laboratory, SCL
Results for example 9.4.3.2 of GS1
• PDF for the y values in histogram
• GUF Gaussian/t-distribution
• Coverage Intervals
• MCM and GUF results for y, u(y), ylow and yhigh
• GUF validation result
• Number of MCM trials
16
Standards and Calibration Laboratory, SCL
Example
Calibration of a 10 V Zener Voltage Reference using
Josephson Array Voltage Standard
n f
 V L  V O  V m  V ran
• Measurement Model:
KJ
• PDF parameters input to the software:
y 
Input Quantity
17
PDF Parameter
Symbol
Description
PDF /
Constant

n
Quantum (Step)
Number
constant
63968
f
Frequency
N(, 2)
75.6 GHz
KJ
Josephson
Constant
constant
483597.9
GHz/V
VL
Leakage
VO

v
a
b
R(a,b)
-5 nV
5 nV
Offset
R(a,b)
-0.1 V
0.1 V
Vm
Null Voltage
R(a,b)
3.722 V
3.712 V
3.732 V
Vran
Random Noise
tv(, 2)
0V
5.13 Hz
30 nV
39
Standards and Calibration Laboratory, SCL
Parameters Input to the MCM Code Generator
18
Standards and Calibration Laboratory, SCL
Results
19
Method
y (V)
u(y) (nV)
ylow (nV)
yhigh (nV)
GUF
10
67
-131
+131
MCM1
(Fixed number)
10
66
-120
+120
MCM2
(Adaptive)
10
66
-120
+120
MCM3
(Histogram)
10
66
-120
+120
Method
dlow
(nV)
dhigh
(nV)
GUF validated? No. of Trials
Computation Time
(s)
MCM1
(Fixed number)
-11
+11
No
1,000,000
<2
MCM2
(Adaptive)
-11
+11
No
6,210,000
89
MCM3
(Histogram)
-11
+11
No
6,270,000
8
Computer Configurations:
Windows XP; MATLAB R2008b (version 7.7); CPU: Core Due T5600, 1.83 GHz, 2 GB
Ram, 80 GB Harddisk
Standards and Calibration Laboratory, SCL
Thank You
20
Standards and Calibration Laboratory, SCL