Quality in testing laboratory:
basic terms and statistical tools
Department of Food Chemistry and Analysis, ICT Prague
Vladimír Kocourek
Prague, 2012
Quality of test results
Data are considered to be of high quality if suitable for intended use
in processing, decision making and/or planning („fit-for-purpose“).
(JM Juran).
Measurement
process of experimentally obtaining one or more
quantity values that can reasonably be attributed to a
quantity.
Measurand
Quantity intended to be measured. In chemistry,
”analyte“, or the name of a substance or compound, are
terms frequently used together with quantity value (mol
or g). Specification of analyte with regard to the method
principle is necessary to define measurand.
Quality of test results
Measurement result (result of measurement)
set of quantity values being attributed to a measurand
together with any other available relevant information.
A measurement result is generally expressed as a single measured
quantity value and a measurement uncertainty.
If the measurement uncertainty is considered to be negligible for
some purpose, the measurement result may be expressed as a
single measured quantity value.
No civilization can escape the need for a system of
measurement. Before people had measuring tools,
they found points of reference such as seeds,
stones and their own limbs. Thus, an inch was
accepted as the distance from the tip of the thumb
to the first joint; a yard was the distance from the
tip of a king's nose to the end of his middle finger.
Statistics, why and when?
 Evaluation of uncertainty of results
 Evaluation of inter-laboratory comparison
 Conformity assessment (complience)
 Quality assurance Quality control:
• Method performance, validation (accuracy; LOQ,..)
• Optimisation of measurement procedures
• Statistical process control (control charts)
Distribution of measured values
Histogram
Probability
Distribution Function
(Gauss)
1. There is a strong tendency for the variable to take a central value;
2. Positive and negative deviations from this central value are equally likely;
3. The frequency of deviations falls off rapidly as the deviations become larger.
Rectangular distribution
The value is between the
limits:
a  a
2a(=  a)
The expectation of
y  xa
Assumed standard
deviation:
1/2a
X
s  a/ 3
One can only assume that it is equally probable for the
value to lie anywhere within the interval
Rectangular distribution - examples
“It is likely that the value is somewhere in that range”
Rectangular distribution is usually described in terms of:
the average value and the range ( ± a )
Certificates or other specification give limits where the value
could be, without specifying a level of confidence (or degree of
freedom).
Examples:
Concentration of calibration standard is quoted as
(1000  2) mg/l
Assuming rectangular distribution the standard uncertainty is:
s  u ( x )  a / 3  2 / 3  1.16 mg / l
The purity of the cadmium is given on the certificate as
(99.99  0.01) %
Assuming rectangular distribution the standard uncertainty is:
s  u ( x)  a / 3  0.01 / 3  0.0058 %
Example: Ascorbic acid QC using candidate IRM
Day A.A. mg/L
sorted
1
100,6
99,2 MINIMUM
2
101,3
99,3
3
99,6
99,4
4
100,4
99,4
5
99,9
99,5
6
99,5
99,6
7
100,4
99,6 25% percen
8
100,5
99,6
9
101,1
99,6
10
100,3
99,7
11
100,1
99,9
12
99,6
99,9
13
99,2
99,9 MEDIAN
14
99,6
100,1
15
99,4
100,1
16
99,6
100,3
17
99,3
100,4
18
99,9
100,4
19
100,5
100,4 75% percen
20
99,5
100,5
21
100,1
100,5
22
100,4
100,6
23
100,9
100,9
24
99,9
101,1
25
99,7
101,3 MAXIMUM
100,1 AVERAGE
Concentration in time period ?
Statistical tools in quality control
Histogram asymetric (skewed):
• unsufficient data
•„systematic“ (non-random) effect
• one-side limited (e.g. close to zero or LOD)
Statistical tools in quality control
Histogram with two appex, i.e. low frequencies inside:
Data population is:
non-homogeneous
or
affected by more than
one factor
10
9
8
7
6
5
(different analysts,
instruments,
calibrations,...
4
3
2
1
0
1
2
3
4
5
6
7
8
9
Statistical tools in quality control - histograms
Box diagram: distribution of values within data population
Frequency
Normal distribution
(t-distribution)
For a set of n values xi
±1s
±2s
Average (Mean value)
1
x 
n
X
n
 x 
i 1
i
n
2
1
  xi  x 
n 1 i 1
(Est.) Standard Deviation
s( xi ) 
Variance of the average
V ( xi )  s 2 ( xi )
s ( xi )
Relative Standard Deviation RSD 
x
( absolute or %)
Variance – standard error of mean
V ( q )  s
s q
s q
k
2
( q ) 

s
2
(q )
n
distribution of
the mean

distribution of the single
values of one individual de
determination of q
mean
Skewed data: Life is log-normal
Environmental contaminants in food,
Sensitivity of the individuals in a population to a chemical compound,
Set of values just above quantification limit,
Income of employed persons
ACCURACY as TRUENESS AND PRECISION
MEASUREMENT = SHOT AT THE TARGET
true value
(real content of analyte in sample)
TARGET
measurement
ACCURACY as TRUENESS AND PRECISION
Probability distribution of random errors „normal“ – „Gauss curve“
frequency
s qk
MEAN
q
measurement
Standard deviation
of n independent measurements
Instrument
Random number output
z  5.65
s( zi )  0.74
s( zi )
s( z ) 
 0.17
n
(zi)
5.70
5.94
5.82
5.80
5.08
6.73
6.30
4.92
5.75
5.32
6.82
5.79
5.80
3.66
5.04
5.51
6.46
6.28
5.46
4.74
© Berglund, 2004
Standard deviation of n independent measurements
Random error:
Standard deviation of mean value is decreasing with
0
replicates - infinite number of replicates: sz
Vladimir.Kocourek@vscht.
Estimation of mean value of population
Arithmetic mean:
1 n
x   xi
n i 1
Median: value in the middle of data sorted in ascending sequence.
Modus: value most frequently observed.
Conffidence interval (mean):
x  t1 / 2 ( )
s
   x  t1 / 2 ( )
s
n
n
0,707 s
0,707 s
~
~
x
u
med
x
u




Conffidence interval (median): 50 1 / 2
50
1 / 2
n
n
Systematic error of result value
ACCURACY of measurement results
What is included in the result value X ?
X
=
μ
+
ε
+
Σδ
μ...true value
ε...systematic error – the same value and sign (+/-)
δ...random errors – variable value and sign (+/-)
Accuracy = Trueness (ε) + Precision (δ)
Accuracy
Closeness of agreement between a test
result of a measurement and the accepted
reference value
(ISO 3534-1)
Accuracy is not given by the spread of a
normal distribution, but by the deviation of
the arithmetic mean of a series of results
from accepted reference value
Accuracy   Deviation  (zero)
ERROR (measurement error)
Error should not be confused with a mistake !
(ε + Σ δ) = X – μ
measured quantity value (X) minus a
reference quantity value (μ).
…both systematic (ε) and random sources (δ)
!
in the Error Approach to describing measurement, a true
quantity value is considered unique and, in practice, unknowable...
Uncertainty about it ?
TRUENESS (measurement trueness)
Trueness is inversely related to systematic error,
(but is not related to random measurement error)
Systematic error (ε + Σ δ) = X – μ
closeness of agreement between the average
of an infinite number of replicate measured
values and a reference/true value
estimate of a systematic error: bias
in analytical chemistry: often related to recovery
(cx/c0)
Correction: compensation for an estimated systematic effect
TRUE value
... is a quantity value consistent with the
definition of a quantity itself.
???
True value is an idealized concept and „true value“
cannot be known exactly!
Hence the REFERENCE VALUE represents a true
value in routine practice
Reference value: Quantity value used as a basis
of comparison with values of quantity of the
same kind.
Reference value
Reference value usually provided with reference
to:
1. certified reference material,
2. reference measurement procedure,
3. comparison of measurement standards.
The other possibilities:
1.
2.
3.
4.
theoretical value (calculated from chem. comp.)
interlaboratory comparison of test results
known amount of analyte spiked into the sample*)
mean value of replicates (intralaboratory reference)
*) Surrogate:
pure compound or element added to the test material, the
chemical and physical behaviour of which is taken to be representative of
the native analyte.
QUALITY CONTROL
ASSESSMENT OF TRUENESS USING REFERENCE MATERIAL
SAMPLES
ANALYTICAL PROCESS
(CERTIFIED)
REFERENCE
MATERIAL
COMPARISON
VALUE ± UNCERTAINTY
RESULT
Errors
RANDOM error
 component of measurement error that - in replicate
measurements - varies in an unpredictable manner
 random error = error - systematic error
 correction of random error can not be made (it’s getting worse !)
Some sources of random errors:
 methods (procedure, calibration,...)
 laboratory (facility, environment)
 equipment and materials / reagents /calibrants
 personnel
 time
PRECISION (repeatability, reproducibility)
random errors of a set of replicate measurements form
a distribution that can be summarized by its
expectation (assumed to be zero) and its variance
PRECISION is calculated as a standard deviation of
replicate measurements σx
less precision is reflected by a larger standard
deviation
precision depends critically on the conditions !
Repeatability and Reproducibility conditions are
particular sets of extreme conditions.
 ...nothing to do with true or reference value !
Repeatability, Reproducibility
Repeatability: a set of conditions that includes
 the same measurement, procedure, operators, same
measuring system, operating conditions and
location, and replicate measurements on the same or
similar objects over a short period of time
Reproducibility: a set of conditions that includes
 different locations, operators, measuring systems, or
even methods on the same or similar objects.
Intermediate precision (intralaboratory reproducibility):
 the same laboratory, method, procedure but within an extended
period of time - may include new calibrations, calibrants,
operators, measuring systems, etc.
Variability of results: 2 operators
Standard
40
200
150
30
FLD1 A, Ex=248, Em=374, TT (C:\DATAHP~1\FLD154-4\DATA02\MT021016\048-3301.D)
A22.872
re
a:
21
4
23.152 - 1-MePyr
Ar 5.3
ea 2
:2
10
3.
88
50
LU
25.004 - B[a]A
60
FLD1 A, Ex=248, Em=374, TT (C:\DATAHP~1\FLD154-4\DATA02\MT021016\048-3301.D)
A22.872
re
a:
25
7
23.152 - 1-MePyr
Ar 7.7
ea 6
:2
99
7.
03
23.591
Ar - BaA
ea
:9
68
9.
11
24.530 - 1-MePyr
FLD1 A, Ex=248, Em=374, TT (C:\DATAHP~1\FLD154-4\DATA02\MJKOLONY
LU
Integration 2
Integration 1
LU
200
150
100
100
50
50
0
0
20
10
23.75
24
24.25
24.5
24.75
25
25.25
25.5
25.75 m
21.5
22
22.5
23
23.5
24
24.5
m
21.5
22
22.5
23
23.5
Difference between results
– 42 % (concentration level ppb)
24
24.5
m
2 different measuring instruments
100
5.47
6.22
6.35
6.07
Same sample, conditions,
operator,...
6.84
7.05
%
7.65
7.54
8.10
9.67
0
5.40
5.60
5.80
6.00
6.20
6.40
6.60
6.80
7.00
7.20
7.40
7.60
7.80
8.00
8.20
8.40
8.60
8.80
9.00
9.20
9.40
9.60
9.80
Two different
chromatographs of the
same type
8.00
8.50
9.00
9.50
10.00
10.50
11.00
11.50
Repeatability limit (r)
Calculated by multiplication of sr obtained under
repeatability conditions by factor reflecting number of
replicates. Implicitly for duplicates (n = 2):
r = f . √2 . σr
r = 2,8 . sr
Application:
Hypothesis H0: differences between duplicates can be
accounted for random errors (which are inherent and
acceptable) only if:
|x1 - x2| ‹ r
Trueness of such results can not be estimated by R…
Calculation of Repeatability (sr)
Procedure 1:
 replicate measurements (n = 8 až 15) of the same test
sample
 calculation of standard deviation sr
Procedure 2:
 similar (routine) samples are analysed in duplicates,
differences Di are recorded,
 calculation of standard deviation sr
sr2 = Σ Di2/(2n)
Reproducibility (R)
Two components:
σ2r – intra-laboratory variance (typical)
σ2L – between laboratories variance
σ2R = σ2r + σ2L
Assessment of R is based on:
1. interlaboratory study data
2. Horwitz (Thompson) empirical model of precision
„William Horwitz“ …
Reproducibility (R) - Horwitz
50
H o r w it z o v a k ř iv k a
40
Precision (reproducibility) CV (%)
30
20
10
0
1
10 0
10 0 0 0
ppb
1 000 000
Concentration
100 000 000
Reproducibility (R) - Horwitz
Relative standard deviation – variation coefficient:
 lower concentration of analyte → increasing CV
 nature of analyte, matrix, analytical method etc.: less
important – even can be ignored !
CVR = 2(1- 0.5*logX)
X is an analyte concentration expressed as a mass
ratio
Thompson:
If X < 1.2*10-7 (i.e. ≤ 0,12 mg.kg-1) → CVR ≤ 22 %
If X > 0.14 (i.e. >14 %) → CVR = 0,01*X0,5
Reproducibility (R)
Reproducibility (R) – Horwitz model
Reproducibility (R) – Horwitz model
Reproducibility (R) – Horwitz model
Acceptable HORRAT values for inter-laboratory studies:
HORRAT(R): The original data developed from interlaboratory (amonglaboratory) studies assigned a HORRAT value of 1.0 with limits of
acceptability of 0.5 to 2.0.
The corresponding within-laboratory relative standard deviations were
found to be typically one half to two thirds the among-laboratory relative
standard deviations.
Acceptable HORRAT values for intra-laboratory studies: repeatability
Within-laboratory acceptable predicted target values for repeatability
are given usualy at 1/2 of PRSDR, which represents the best case.
Reproducibility (R) – Horwitz model
Official requirements on precision – trace analysis:
Concentration
(ppb)
CVr (%)
Codex Alimentarius
CVR (%)
Horwitz
<1
35
> 45
1 - 10
30
32 -45
10 - 100
20
22 – 32
> 100
15
< 22
Requirements on repeatability and reproducibility
Depend only on random error distribution at certain
concentration level.
Example: Maximum acceptable differences in test results: (EN
12393-1:1998)
Concentration level
[mg/kg]
Difference between
duplicates
[mg/kg]
Difference between
2 laboratories
[mg/kg]
0.01
0.1
1
0.005 (50 %)
0.025 (25 %)
0.125 (12 %)
0.01 (100 %)
0.05 (50 %)
0.25 (25 %)
Intralaboratory precision
Commission Decision 2002/657/EC: e.g. mycotoxins
Requirements on maximum values of variation coefficient
as regards intralaboratory precision:
Concentration
(mg/kg)
Variation coefficient
CV (%)
≤ 10 - 100
20
100 - 1000
15
 1000
10
Intralaboratory precision
Example of repeatability, reproducibility and
interlaboratory precision obtained in the validation
interlaboratory study.
(ISO 15753: Determination of PAHs in vegetable fats and oils)
REGULATION No 401/2006/EC
methods of analysis for the official control of mycotoxins in foodstuffs
REGULATION 401/2006/EC
methods of analysis for the official control of mycotoxins in foodstuffs
REGULATION No 401/2006/EC
methods of analysis for the official control of mycotoxins in foodstuffs
Relationships between type of error, characteristics and
their quantitative expression
Accred Qual Assur (2007) 12: 45–47
Errors vs. uncertainty in chemical measurements
each measurement result comprises errors*)
mean value shall be accompanied by the uncertainty
statement [e.g. X ± x g/l]
to state the compliance with the specification, the
uncertainty shall be considered.
Example:
*) Error
•
analysis (method) results in mean value X ± 1,5%
•
at least 97% of the analyte is required by specification
•
to acchieve a reasonable probabability (> 95%) that the
concentration of analyte is >97%, mean value of result
shall be > 98,5 %
is an idealized concept and errors cannot be known exactly
Performance characteristics to be validated
Thank you for your
attention…
▐ vladimir.kocourek@vscht.cz ▌