Measurement of uncertainty
Validation and verification
OCA Course Code:
OCA-QMAugust-2021
Schedule
Tutor and Participants
Ahmed Abd-Elmoneim, PhD
Food microbiology and ISO 17025:2017
Background
Expectations
Terms and definitions
Measurand:
 Particular quantity subject to measurement
 The concentration of an analyte
 quantity intended to be measured
Precision
Closeness of agreement between independent test results obtained under
Stipulated (specified or fixed) conditions.
Less precision is reflected by a larger standard deviation and vice versa.
8/22/2024
Terms and definitions
True value
value which characterizes a quantity or quantitative characteristic
perfectly defined in the conditions which exist when that quantity or
quantitative characteristic is considered.
The true value of a quantity or quantitative characteristic is a theoretical
concept and, in general, cannot be known exactly.
Error (of measurement)
Measured quantity value minus a reference quantity value:
 Systemic error
 Random error
Bias Estimation of the systemic measurement error
8/22/2024
Terms and definitions
Terms and definitions
Terms and definitions
 Measurement Uncertainty (MU)
 Parameter, associated with the result of a measurement, which characterizes the dispersion
of the values which could reasonably be attributed to the measurand.
 Is the doubt that exists about the result of any measurement.
 NOTE:
Uncertainty of measurement does not imply doubt about the validity of a
measurement; on the contrary, knowledge of the uncertainty implies increased confidence in
the validity of a measurement result.
Terms and definitions
Standard uncertainty (u):
When expressed as a standard deviation, an uncertainty component is
known as a standard uncertainty.
Combined standard uncertainty{u c(y)}
An estimated standard deviation equal to the positive square root of the
total variance obtained by combining all the uncertainty components.
* measurement result (y)
8/22/2024
 Expanded uncertainty (U):
Quantity defining an interval about the result of a measurement that may be expected
to encompass a large fraction of the distribution of values that could reasonably be
attributed to the measurand.
U = k × uc
 Coverage factor (k)
Numerical factor used as a multiplier of the combined standard uncertainty in order to
obtain an expanded uncertainty .
A coverage factor is typically in the range 2 to 3.
--
 Arithmetic mean (Average)(X )
Arithmetic mean value of a sample of n results.
--
X
=Sum Xi /n
 Sample Standard Deviation (s)
An estimate of the population standard deviation (σ )from a sample of (n) results.
S=SQRT{SUM(Xi -X-- )2 /(n-1)}
 Relative Standard Deviation (RSD %)
An estimate of the standard deviation of a population from a (statistical) sample of n results
divided by the mean of that sample.
Often known as coefficient of variation (CV %).
--
RSD=S/ X
Types of measurement uncertainty
 Type A evaluations
Uncertainty estimates using statistics (usually from repeated readings).
Standard deviation of the mean (standard uncertainty ) When a set of several repeated
readings has been taken
u=S/ 𝑛
Type A evaluations
n
Xi
10
Sum
n-1
SD or (σ) or u
Mean (µ)
(Xi-µ) 2
(Xi-µ)
15.33
15.321
0.009
8.1E-05
15.21
15.321
-0.111
0.012321
15.31
15.321
-0.011
0.000121
15.25
15.321
-0.071
0.005041
15.35
15.321
0.029
0.000841
15.37
15.321
0.049
0.002401
15.27
15.321
-0.051
0.002601
15.28
15.321
-0.041
0.001681
15.41
15.321
0.089
0.007921
15.43
15.321
0.109
0.011881
153.21
0.04489
9
0.070624201
0.070624201
Kindly estimate the following:
1- Mean of each analyst
2- Uncertainty of each analyst
3- Total Average
4- Expanded uncertainty
5-RSD % (CV %)
Analyst 1 Analyst 2
15.33
15.45
15.21
15.2
15.31
15.33
15.25
15.22
15.35
15.4
15.37
15.34
15.27
15.25
15.28
15.23
15.41
15.39
15.43
15.43
 Type B evaluations
Uncertainty estimates from any other information. This could be information from past
experience of the measurements, from calibration certificates, manufacturer’s specifications,
from calculations, from published information, and from common sense.
Typical sources of uncertainty are :
 Sampling
 Storage Conditions
 Instrument effects(As analytical balance & a temperature controller )
 Reagent purity
 Measurement conditions
 Sample effects (Matrix composition)
 Operator effects (Competence of analyst)
 Random effects
Type B evaluations
Fish Bone
Sampling
Storing conditions
Environmental
condition
Combined Uncertainty
𝒂𝟐 + 𝒃𝟐 + ⋯
Manual
Homogeneity
Bias
Calibration
Instrumental effect
Composition
Matrix
Competence
of analyst
Microbiology measurement uncertainty
Microbiology measurement uncertainty
 Technical uncertainty
Uncertainty resulting from operation variability associated with technical steps of the analytical
procedure (Include the variability of taking, mixing and dilution of test portion taken from
laboratory sample to prepare initial suspension and subsequent dilutions and also the effect of
variability in incubation and media).
 Matrix uncertainty
Uncertainty resulting from the extent to witch the test portion is not truly representative of lab
sample.
 Distributional uncertainty (Poisson)
Uncertainty resulting from intrinsic variability associated with the distribution of the
microorganisms in the sample, the initial suspension and subsequent dilution.
Verification
“Are you testing it right?”
Validation
“Are you testing the right thing?”
Terms and definitions
 Verification
In the preferable case of standard and validated methods being used, the laboratory is still
required to prove that it can implement them in a reliable way.
 For quantitative in most cases (Optional), to
 Determine repeatability,
 Measurement uncertainty,
 and limit of quantitation or reproducibility.
 For qualitative methods the limit of detection.
Terms and definitions
 Validation:
The confirmation by examination and the provision of objective evidence that the
particular requirements for a specific intended use are fulfilled.
 Verification:
Provision of objective evidence that a given item fulfils specified requirements.
Terms and definitions
 Particular requirements:
1. Selectivity
2. Specificity
3. Accuracy
4. Precision
5. Reproducibility
6. Limit of Detection
7. Limit of Quantification
8. Calibration Curve
9. Linearity
10. Robustness and Ruggedness
Terms and definitions
 Limit of detection (LOD):
Applied to qualitative microbiological tests: The lowest number of micro-organisms or quantity
that can be detected, but in numbers that cannot be estimated accurately.
YLOD =Y blank +3SDblank
where Y blank and SD blank are the average value of the blank signal and its corresponding
standard deviation, respectively, obtained by measuring at least a minimum of 10 independent
sample blanks. Alternatively, when sample blank cannot produce any response (i.e.
voltammetry), 10 independent sample blanks fortified at the lowest acceptable concentration
of the analyte are measured and then, YLOD = 3 SD, SD being the standard deviation of the set
of measurements.
 Limit of quantification (LOQ):
Applied to quantitative microbiological tests: The lowest number of micro-organisms or quantity
within a defined variability that may be determined under the experimental conditions of the
method under evaluation.
 Is the lowest concentration of analyte that can be determined quantitatively with an
acceptable level of precision.
 The procedure for evaluating LOQs is equivalent to that of LODs, by measuring at least 10
independent sample blanks and using the factor 10 instead of 3 for calculations:
YLOQ =Y blank +10SD blank
IUPAC assumes relative precision 10%
a series of blanks are spiked at several analyte concentrations and measured in triplicate. the
amount that corresponds to a previously defined precision RSD is interpolated and taken as the
ZLOQ
Terms and definitions
 Measurement precision:
Closeness of agreement between indications or measured quantity values obtained by
replicate measurements on the same or similar objects under specified conditions.
CV=(SD*100)/ Mean
 Relative trueness
The degree of correspondence of the results of the method under evaluation to those obtained
using a recognized reference method.
Terms and definitions
 Negative deviation
Occurs when the alternative method gives a negative result without confirmation when the
reference method gives a positive result. This deviation becomes a false negative result when
the true result can be proved as being positive.
 Positive deviation
Occurs when the alternative method gives a positive result without confirmation when
the reference method gives a negative result. This deviation becomes a false positive
result when the true result can be proved as being negative.
 False Negatives - A negative outcome of a binary classification test when the true outcome is positive.
False Positives - A positive outcome of a binary classification test when the true outcome is negative.
 True Negative - A negative outcome of a binary classification test when the true outcome is negative.
True Positive - A positive outcome of a binary classification test when the true outcome is positive.
Terms and definitions
 Repeatability
Measurement precision under a set of repeatability conditions of measurement.
For quantitative methods (considered in some cases as SD)
Condition of measurement, out of a set of conditions that includes the same
measurement procedure, same operators, same measuring system, same operating
conditions and same location, and replicate measurements on the same or similar
objects over a short period of time.
 Reproducibility
(i) For quantitative methods
Condition of measurement, out of a set of conditions that includes different locations,
operators, measuring systems, and replicate measurements on the same or similar
Objects. (Considered in some cases as RSD).
Terms and definitions
 Sensitivity
The fraction of the total number of positive cultures or colonies correctly assigned in the
presumptive inspection.
Sensitivity = True positive/( True positive+ false negative) x100
 Specificity :
The fraction of the total number of negative cultures or colonies correctly assigned in the
presumptive inspection.
Specificity= true negative/(true negative+ false positive) x100
Terms and definitions
 Accuracy
Accuracy is the nearness of a measurement or the mean of a set of measurements to the true
value. Accuracy is assessed in terms of percent recovery for quality control check samples and
matrix spikes.
%accuracy = (TP+TN)/(TP+FP+FN+TN) x100=
 Ruggedness
The ruggedness (a measure of robustness) of a method is the degree to which results are
unaffected by minor changes from the experimental conditions described in the method, for
example, small changes in temperature, pH, reagent concentration, flow rates, extraction times.
 Recovery
The recovery of an analyte in an assay is the detector response obtained from an
amount of the analyte added to and extracted from the matrix, compared to the
detector response for the true concentration of the pure authentic standard (seized
materials).
R=(C1-C2)/C3
Where: c1 = measured concentration in fortified (spiked)sample
c2 = measured concentration in unfortified sample
c3 = concentration of fortification
Recovery is expressed as a percentage by multiplying the result by 100.
 Linearity
Linearity is the ability of the method to elicit results that are directly proportional to analyte
concentration within a given range.
 Range
The range of an analytical procedure is the interval between the upper and lower
Concentration (amounts) of analyte in the sample (including these concentrations) for which it
has been demonstrated that the analytical procedure has a suitable level of precision,
accuracy and linearity.
(Prepare a set of standard solutions with known concentrations (should extend
to at least 40% beyond the specification concentration and include the target
concentration), Perform the determination, and generate a standard curve. )