A MESSAGE FROM JAMES O. WESTGARD
TO BE UNCERTAIN OR IN ERROR?
THAT IS THE QUESTION
Evolving concepts of analytical quality
 Precision and accuracy
 Total analytical error
 Operating specifications
ISO concepts of trueness and uncertainty
Difficulties in implementing the ISO
recommendation





Need for statistical skills
Trend towards lower skilled personnel
No practical benefit over error concepts &
terminology.
Quality requirements already defined as allowable
errors.
Changes will take a long time.
And the answer is….
Efforts to provide worldwide standards
have been led by the International
Organization of Standards, or ISO as it is
commonly known. The ISO standards were
initially aimed at industry but are now
being adapted to healthcare laboratories.
One of the areas currently under
discussion is the standardization of
concepts, definitions, and terminology
related
to
analytical
quality
management, as discussed by Dr.
Xavier Fuentes-Arderiu in an essay on
"Trueness and Uncertainty" and an
associated
"Glossary
of
ISO
Metrological and Related Terms and
Definitions
Relevant
to
Clinical
Laboratory Sciences."
I applaud efforts to standardize and clarify concepts, definitions, and terminology, but I
believe there are serious issues that need to be resolved before the proposed ISO
definitions and terminology are adopted.
Evolving concepts of analytical quality
Precision and accuracy were the terms being used to describe the performance of analytical
measurements in the 1960s when healthcare laboratories began to be automated and to grow
into high volume production operations. In contrast to many industrial applications where
replicate test measurements are made routinely, single test measurements are the norm in
healthcare testing. This distinction is important because the effects of imprecision can be
minimized by performing multiple measurements, hence industrial applications could easily
control the amount of random analytical error by increasing the number of replicate
measurements. Therefore, the most important measure of industrial quality was the
inaccuracy or systematic error.
Total analytic error was recommended in the mid 70s [1] as a better way to evaluate the
performance of clinical measurements where only a single measurement is generally made in
determining a test result. All individual test results may be in error due to both the imprecision
and inaccuracy of the method, therefore the combination of the two errors, or the total
analytical error, determines the quality of the test result. In developing this new concept, the
emphasis was on "errors", their random and systematic components, and the net or total
effect of those components. It took over a decade for this concept to become established in
healthcare laboratories. Its relevance is especially clear in the US today because the CLIA
laboratory regulations specify criteria for acceptable performance in the form of allowable total
errors that are used in grading analytical performance in proficiency testing surveys [2].
Operating specifications were developed in the early 90s as an outgrowth of total quality
management and the interest in making quality control a quantitative technique for managing
routine production [3]. The operating specifications for a method define the imprecision and
inaccuracy that are allowable and the QC that is necessary (control rules, number of control
measurements) to assure that a stated quality requirement will be achieved in routine
production. The important point here is that operating specifications consider both the stable
performance of the method (imprecision and inaccuracy) and the capability of the QC
procedure to detect changes (unstable performance).
ISO concepts of trueness and uncertainty
Trueness is used by ISO to describe the "closeness of agreement between the mean
obtained from a large series of results of measurement and a true value." The emphasis on
the "mean obtained from a large series of results of measurement" limits this concept to the
systematic error or inaccuracy (bias) of a method.
Uncertainty is used by ISO to describe a "parameter, associated
with the result of a measurement that characterizes the
dispersion of the values that could reasonably be attributed to
the measurand." This term could be quantitatively described by
calculating a standard deviation or some multiple (confidence
interval).
In short, the new ISO terminology recommends trueness instead
of accuracy, inaccuracy, and systematic error and uncertainty
instead of random error, imprecision, and precision. Total error
wouldn't exist! Furthermore, estimates of uncertainty would
combine different sources and estimates of random error
through
propagation
of
errors
These
competent
calculations.
calculations would require a
statistician, a clinical chemist with some mathematical
or statistical aptitude, or special computer programs that
could be used by laboratory analysts.
The objective is to describe the uncertainty of a measurement or test result. For example, a
test result of 100 means a value in the range from say 96 to 104. The test result is good to
within 4 units. Of course, that also means it may be off by up to 4 units, but the result isn't in
error - just uncertain!
Difficulties in implementing the ISO recommendation
Any change is difficult - even one that is entirely beneficial. A change becomes impossible
when the difficulties outweigh the benefits. One way to illustrate the benefits versus difficulties
is to use a "force field diagram" to describe the forces that are driving the change vs those
that are restraining the change. The accompanying diagram identifies the driving forces as
being the benefit (in principle) of uniform concepts and terminology for communication in an
international marketplace, benefit of uniformity with other industries, the ISO momentum in
the European marketplace, and the benefit from a pure theory of measurements. The
restraining forces represent the following difficulties:
Need for statistical skills. Few healthcare practitioners, whether laboratorians, nurses, or
physicians, are skilled in statistics. Their training is quite different from the strong
mathematical orientation of the engineers who are in key positions in other industries.
Healthcare personnel often find statistics to be confusing. They certainly don't have a
quantitative understanding of uncertainty, however, they do have a quantitative understanding
of errors. They know that errors are bad, should be eliminated when possible, and always
kept small relative to the medical use and interpretation of test results. The concept of error
has real meaning whereas the concept of uncertainty has only statistical meaning.
Trend towards lower skilled personnel. The application of more complicated concepts usually
requires more sophisticated practitioners. Today, the education, training, and skills of
professionals in healthcare laboratories are actually decreasing, as part of the efforts to
reduce the costs of healthcare. There will likely be fewer PhD clinical chemists available to
deal with the complicated calculations that are necessary to properly estimate the uncertainty
of test results. The availability of computer programs may facilitate the calculations, but may
not improve the understanding and interpretation of the calculated results.
No practical benefits over the error concept and terminology. There are no practical benefits
to practitioners to change from the understandable concepts and terminology of errors to
more difficult statistical concepts and terminology. The error concepts do not in any way limit
current practices in quality management. The real problem is that laboratories need to define
the quality required for the tests they provide.
Quality requirements already defined as allowable errors. Given that quality requirements
must be defined if analytical quality management is to be improved, it is important to
recognize that most recommendations for analytical quality are given in the form of allowable
errors - allowable SD, allowable bias, allowable total error. What amount of uncertainty is
allowable? How do you develop recommendations for the allowable uncertainty?
Changes will take a long time. In the late 1970s, IFCC recommended that the terms
imprecision and inaccuracy replace precision and accuracy to properly emphasize that the
deviations (or errors) from the true or correct values. Several years went by before the new
terms of imprecision and inaccuracy were being used in the scientific literature. Likewise, the
concept of total error took about ten years from introduction to general acceptance by the
laboratory community. Operating specifications, which were first introduced in 1990, are not
yet universally accepted. It can be expected that the new ISO concepts of trueness and
uncertainty will take a decade or more to become established and useful.
And the answer is "roles for both"
The practical management of analytical quality in a service laboratory is best served by
concepts and terminology that relate to analytical errors, rather than by the newly
recommended concepts of trueness and uncertainty. Current quality requirements and quality
management tools are based on error concepts and theory.
The role for the new concepts is in the applications of reference methods, not field methods.
Reference methods are used to establish true values or to assign values to calibration
materials. In such applications, it is important to make replicate measurements and minimize
random errors, to correct for known systematic errors, and then to be able to express the
uncertainty in the measured or assigned values.
References:
Westgard JO, Carey RN, Wold S. Criteria for judging precision and accuracy in method
development and evaluation. Clin Chem 1974;20:825-33.
Westgard JO, Wiebe DA. Cholesterol operational process specifications for assuring the
quality required by CLIA proficiency testing. Clin Chem 1991;37:1938-44.
Westgard JO. Charts of operational process specifications ("OPSpecs charts") for assessing
the precision, accuracy, and quality control needed to satisfy proficiency testing criteria. Clin
Chem 1992;38:1226-33.
--------------------------------------------------------------------------------