Inter-Laboratory Comparison Study
Using Modular Instrumentation and
Lessons Learned
Author:
Dimaries Nieves – National Instruments
Senior Metrology Engineer
Speaker:
Jorge Martins – National Instruments
Principal Metrology Engineer
NCSLI Conference 2013
1
Learning Objectives
•
How can we demonstrate performance and
competence of modular instrumentation as part
of metrology and accreditation process?
•
Propose a process to perform an Inter-laboratory
Comparison using a modular instruments.
•
Standard Documents for ILC evaluation.
•
Review some of the results and lessons learned.
NCSLI Conference 2013
2
•
Inter-laboratory Comparison (ILC) is a key
criterion for laboratory accreditation
•
ILC is an important asset in:
•
•
•
•
•
•
Measurement Reliability
Measurement Assurance
Calibration Process Confidence
Measurement Method
Technician Proficiency
There is no commercial proficiency testing
available for Modular Instrumentation.
For the last two years, National Instruments performed an
interlaboratory comparison using Modular Instrumentation as the
primary standard
NCSLI Conference 2013
3
Modular Instruments
•
•
•
Gradually taking an important role in test and
measurement
Enable more cost effective and flexible measurements
Used in Research, Calibration, Validation and
Production test
In metrology and laboratory
accreditation process
• Need to demonstrate performance and
competence performing regular calibration
services.
NCSLI Conference 2013
4
NI Inter-Laboratory Comparison Proposal
•
The ILC was designed based on:
•
•
NCSLI’s RP-15
ASTM E691-13
•
NIC Metrology Laboratory serves
as the pivot Laboratory
•
Unit was monitored using the
petal or flower model
•
NI Certified Calibration Centers
were selected to participate
NCSLI Conference 2013
5
ILC Equipment Bias and Stability
•
Stability of the reference unit was determined by the pivot
Laboratory before starting the ILC process.
•
No significant bias was found within the pivot
measurements.
•
The pivot laboratory monitor the performance of the unit
to ensure that any unexpected changes in the traveling
standard are promptly detected.
•
Comparison of the pivot lab data
showed that the unit remained in
control throughout the all ILC.
NCSLI Conference 2013
6
ILC Reference Value
•
Reference values were determined using the pivot lab
measurements.
•
•
•
All pivot runs were included in the analysis
Statistical Analysis of the data was performed
Uncertainties reported by the pivot laboratory include ILC
process bias and deviations.
NCSLI Conference 2013
7
First ILC Run
•
NI PXI-4072 Flex DMM and LCR
Meter (6 ½ digits DMM)
•
NI Published Calibration Procedure
•
“Verify Mode Only” procedure
using NI’s calibration software,
Calibration Executive
•
Participants use their own chassis, standards, and
cables to perform the measurements
• Report
only one measurement result per point with
an expanded uncertainty at 95% of confidence
(k=2).
NCSLI Conference 2013
8
For Evaluation
An En value was calculated for each measurement
provided for each laboratory
En =
NCSLI Conference 2013
Lab - Re f
U
2
Lab
9
+U
2
Re f
Resistance Measurement Verification in 2-wire mode
First ILC (2011)
NCSLI Conference 2013
10
Resistance Measurement Verification in 4-wire mode
First ILC (2011)
NCSLI Conference 2013
11
Second ILC Run
•
NI PXI-4072 FlexDMM and LCR Meter
(6 ½ digits DMM) , Chassis and Cables
(entire Setup).
•
NI Published Calibration Procedure
•
“Verify Mode Only” procedure using NI’s calibration
software, Calibration Executive
•
Performed three runs on the unit, doing a manual “Self
Calibration” before each run
•
Verify that the board temperature was stable at 35 ºC ± 2
ºC before performing the verification
• Report
measurement result with an expanded
uncertainty at 95% of confidence (k=2).
NCSLI Conference 2013
12
Resistance Measurement Verification in 2-wire mode
Second ILC (2012)
Pivo
t
NCSLI Conference 2013
13
Resistance Measurement Verification in four-wire mode
Second ILC (2012)
NCSLI Conference 2013
14
Additional Analysis
•
Consistency Evaluation
•
Within the laboratory
S
k=
Sr
•
•
•
p
Si2
Sr = å
1 p
S = Standard Deviation for one laboratory
Sr = Repeatability standard deviation of the equipment
p = Number of laboratories participating in the ILC
NCSLI Conference 2013
15
Additional Analysis
•
Consistency Evaluation
•
Between Laboratory
d
h=
Sx
•
•
p
d2
Sx = å
1 p -1
Where, d = x - x = Lab average minus the average of the
lab averages
Where, Sx = Standard deviation of the lab averages
NCSLI Conference 2013
16
Within-laboratory consistency (k) for DC Voltage measurement
Second ILC (2012)
NCSLI Conference 2013
17
Between-laboratory consistency (h) for DC Voltage measurement
Second ILC (2012)
NCSLI Conference 2013
18
Conclusion
•
Modular Instruments meet the published measurement
specifications, regardless of the combination of chassis,
controller and measurement I/O.
•
However, for an ILC the higher requirements for
repeatability and reproducibility make it advantageous to
provide, not just the instrument but, the entire
measurement system.
•
Including the entire system in the ILC reduces differences
between participants, improving the correlation of data.
•
Several statistical tools are needed
NCSLI Conference 2013
19
Questions?
NCSLI Conference 2013
20