MASTER COPY 10/18/10
1
American Society of Veterinary Clinical Pathology (ASVCP)
2
Quality Assurance and Laboratory Standards Committee (QALS)
3
Guidelines for the Determination of Reference Intervals
4
in Veterinary Species and other related topics
5
Friedrichs K, Harr, KE, Freeman K, Szladovits B, Walton R, Barnhart K, Blanco J
6
7
Grasbeck and Saris introduced the concept of population-based reference values in 1969
8
to describe the variation in blood analyte concentrations in well characterized groups of
9
healthy individuals (Grasbeck, 1969). Reference values are typically reported as
10
reference intervals (RI) comprising 95% of the healthy population. Since their
11
introduction, population-based RI have become one of the most commonly used
12
laboratory tools employed in the clinical decision-making process (Horn and Pesce, 2005,
13
Chapter 1, page 1-2). Although use of population-based RI is universally accepted, the
14
optimal method for their derivation is frequently debated and is a recurring topic in the
15
clinical laboratory literature.
16
17
The standard for production of human population-based RI was commissioned by the
18
International Federation of Clinical Chemistry (IFCC) in 1970. The Expert Panel on the
19
Theory of Reference Values (EPTRV) authored a 6-part series on the production of
20
reference values, which was adopted by several professional organizations including the
21
Clinical and Laboratory Standards Institute (CLSI).(Solberg 1987, PetitClerc 1987,
22
Solberg 1988, Solberg 1991, Solberg 1987, Dybkǽr 1987) Subsequent to these
23
publications, alternative statistical methods for identifying outliers, analyzing reference
1
MASTER COPY 10/18/10
24
values, and determining the need for partitioning have been proposed. In addition,
25
concerns were expressed regarding the complexity and expense of compliance with the
26
original IFCC-CLSI standard and the lack of recommendations for handling small
27
reference sample sizes. This led to a revision, completed in 2008, that includes
28
recommendations for transference and validation of RI from other sources and promotes
29
robust methods for determining RI from small sample sizes. (Horowitz, 2008).
30
31
The ASVCP has recommended adherence to the CLSI-IFCC guidelines for the
32
determination of population-based RI. However, access to these guidelines is not readily
33
available in the veterinary community. In addition, fulfillment of human laboratory
34
standards is often deemed too expensive in veterinary medicine where the recommended
35
number of reference samples is seldom achievable. The need for veterinary-specific
36
guidelines that address concerns unique to animal species is recognized within the
37
veterinary community.
38
39
In response, the QALS Committee of the ASVCP formed a subcommittee to generate
40
guidelines for the determination of reference intervals in veterinary species and to address
41
additional topics of interest. The goal of this subcommittee was to develop balanced and
42
practical recommendations that are statistically and clinically valid. Guidelines that are
43
excessively complex or inaccessible due to high cost or large reference sample sizes will
44
fail to create the desired continuity within the veterinary community. In addition to
45
providing guidelines for the determination of de novo population-based RI for new tests
46
or methods or for new populations of animals, this document provides recommendations
2
MASTER COPY 10/18/10
47
on related topics, including transference and validation of RI from other sources, subject-
48
based and common RI, use of published RI, and establishing decision thresholds (or
49
decision limits). A consistent approach to the development and reporting of population-
50
based RI, and other clinical decision-making values, will benefit all veterinary
51
professionals.
52
53
These guidelines are modeled on the revised guidelines of the CLSI-IFCC for
54
establishing population-based RI, which were summarized in Veterinary Clinical
55
Pathology. (Horowitz 2008; Geffre 2009) Recommendations on other topics are based
56
on current literature and on the experience of individuals working in veterinary laboratory
57
medicine. These guidelines were independently reviewed by experts in the field of
58
clinical laboratory medicine and medical statistics and passed a formal review and
59
approval by the membership of the ASVCP. Examples of statistical procedures are
60
available in a forthcoming supplement to this document. These examples were generated
61
using the RI freeware, Reference Value Advisor.
62
(http://www.biostat.envt.fr/spip/spip.php?article63, accessed August 26, 2010)
63
64
As a guideline, these procedures may be applied as written or modified by the user for
65
specific purposes. The intended users of these guidelines include individuals working in
66
veterinary reference diagnostic laboratories, animal research clinical laboratories,
67
manufacturers of veterinary diagnostic equipment and assays, and authors of articles on
68
RI in veterinary species. In addition, veterinary clinicians who use RI should be familiar
69
with these procedures so that they can evaluate RI studies to ensure appropriate
3
MASTER COPY 10/18/10
70
application to their patient population. A list of definitions for terms used RI studies is
71
available in the Geffre et al review paper. (Geffre, 2009)
72
73
DETERMINATION OF DE NOVO REFERENCE INTERVALS FOR NEW
74
ANALYTES, NEW METHODS, OR NEW POPULATIONS
75
76
Preliminary investigation
77
Investigation of sources of biological variability and interference affecting measurement
78
of the analyte(s) in question is recommended in order to determine specifications for
79
collection and handling of samples and for selection and preparation of reference
80
individuals. This information also may be used to establish inclusion and exclusion
81
criteria and determine the need for separate RI based on animal factors (e.g., age, sex,
82
breed) or preanalytical techniques (e.g., serum vs. plasma). Analytical interference from
83
bilirubin, lipemia, and hemolysis may be considered in this investigation; however,
84
reference samples with these alterations typically are rejected as evidence of illness, non-
85
fasting, or poor sample handling.
86
87
Selection of the reference population
88
1. Define the reference population of interest, as well as the criteria used to confirm
89
health in individuals selected from this population (selection, inclusion, exclusion,
90
and partitioning criteria). The demographics of the reference population should be
91
representative of the patient population for which the RI will be used in making
92
clinical decisions.
4
MASTER COPY 10/18/10
93
1.1 Selection criteria must be defined. These are used to characterize the population
94
and verify the health of individuals. They may include but are not limited to the
95
following: (Walton 2001)
96

Biological (age, sex, breed, stage of reproductive cycle, production type)
97

Clinical (history and physical examination to establish health and
98
99
husbandry practices)

100
Geographical (location) and seasonal (effects of temperature and day
length) characteristics.
101
NOTE: Additional testing may be required to establish health. The type and extent of
102
additional testing depends on the intended use of the proposed RI and may include, but is
103
not limited to, a complete minimum database (CBC, biochemical profile, urinalysis),
104
imaging, functional tests, fecal examination for parasites, lymphocyte phenotyping, and
105
historical or clinical follow-up.
106
107
108
1.2 Exclusion criteria are used to eliminate individuals that should not be included in
the reference population. They may include but are not limited to the following:

109
Biological (e.g., fasted or non-fasted state, intense exercise, level of stress
or excitement)
110

Physiological factors (illness, lactation, pregnancy, other). (Poole 1997)
111

Administration of pharmacologically active agents. Individuals receiving
112
pharmacologically active agents for treatment of specific disorders are
113
usually excluded. However, routine administration of preventative
114
dosages of anthelmintic medications typically is acceptable in most RI
115
studies. (Poole 1997)
5
MASTER COPY 10/18/10
116
NOTE: Some of these inclusion and exclusion criteria may be used to partition
117
reference populations into subgroups, for example, by age, sex, or reproductive
118
status. (Walton 2001)
119
2. Develop a questionnaire that will establish whether a reference individual conforms to
120
selection criteria, belongs to a partitioned subgroup, or should be excluded. The
121
questionnaire is filled out by the owner and by the individual(s) examining the subject
122
and collecting the sample. Owner consent for participation in the RI study is often
123
included in this questionnaire and is mandatory in many institutions.
124
3. Consider the number of healthy reference subjects available to provide reference
125
samples. Ideally a minimum of 120 reference individuals is available for determining
126
nonparametric RI and 90% confidence intervals (CI) with enough extra individuals to
127
allow for some rejection. Reference intervals determined from smaller sample sizes
128
are commonplace and often necessary in veterinary medicine. However, thorough
129
consideration for the effect of small sample size on the accuracy of population-based
130
RI should be addressed early in the study. The smaller the sample size, the higher the
131
degree of uncertainty in the estimation of the RI.
132
4. Reference individuals may be selected by either direct or indirect methods. Direct
133
methods involve selection of known healthy individuals from a general population
134
using specific criteria. Indirect sampling methods use medical databases containing
135
results from both healthy and non-healthy individuals. Statistical and nonstatistical
136
methods are employed to exclude samples from obviously unhealthy individuals, and
137
RI are generated from the remaining values. Because RI established using indirect
138
sampling methods inadvertently may include unhealthy individuals, RI derived in this
6
MASTER COPY 10/18/10
139
manner may not accurately reflect the distribution of analyte quantities in a healthy
140
population. In addition, information about preanalytical and analytical factors may
141
not be available. Consequently, direct sampling methods are strongly recommended,
142
and indirect sampling methods only should be used when other options for
143
establishing RI are unavailable. Two types of direct sampling are possible:
144
4.1 a priori in which inclusion and exclusion criteria are established prior to
145
selection of healthy reference individuals. This method is preferred when
146
information about how biological and preanalytical factors affect the analyte(s)
147
are well documented.
148
4.2 a posteriori in which inclusion and exclusion criteria are established after
149
selection and testing of healthy reference individuals. This method typically is
150
used when there is limited information about a new analyte or laboratory test, and
151
it is not known how biological or preanalytical factors affect analyte quantities.
152
153
Preanalytical procedures – Patient preparation, sample collection, and analytical
154
quality assessment
155
5. Preparation of the reference individuals, sample collection, sample handling, and
156
sample processing should be performed in a standardized manner that is consistent
157
with the methods used for patient testing. In addition, consideration should be given
158
to potential adverse effects of preanalytical factors in order to reduce variation that is
159
not due to inter- or intra-individual variability.
160
5.1 Patient preparation and handling (e.g. fasting or non-fasting, method of capture
161
and restraint, use of sedatives or anesthesia) should be standardized.
7
MASTER COPY 10/18/10
162
5.2 Sample collection (site, preparation of the site, vial type, collection system) and
163
handling of the specimen (transportation, temperature, and centrifugation) should
164
be standardized based on prior knowledge of the analyte.
165
5.3 Sample collection may need to be standardized for time of day or for season,
166
depending on the analyte being measured and the intended use of the RI. This is
167
especially important for certain hormones. Alternatively, samples may need to be
168
collected across seasons or throughout the day for a more general representation
169
of analyte concentrations.
170
171
5.4 Special sample handling requirements for certain analyte(s) must be known and
strictly followed (e.g. on ice, anaerobic).
172
5.5 Analyte stability should be determined prior to the RI study. This information is
173
necessary to determine if certain analytes require specific storage conditions or
174
analysis within a defined interval and if sample storage and batch analysis can be
175
performed.
176
6. Estimates of analytical error (CV and bias) should be recorded for all methods. These
177
may be determined during the RI study or during initial method validation (MV)
178
studies. This information is necessary if transference of RI is utilized in the future.
179
These estimates of analytical error should fall within the quality requirement goals for
180
imprecision (CV), inaccuracy (bias), and total allowable error (TEa) established by
181
the laboratory for existing methods or during method validation studies for new
182
methods. Quality goals may be based on biologic variation, clinical interpretation of
183
test results, consensus documents, or all of these.
184
8
MASTER COPY 10/18/10
185
Analytical procedures
186
7. Analyze samples using methods that are monitored with strict quality control
187
procedures. (ASVCP QC Guidelines VCP publication or
188
http://www.asvcp.org/pubs/qas/index.cfm Conditions for analysis should be well
189
defined in a manner consistent with analysis of patient samples in order to reduce
190
variation that is not due to inter- or intra-individual variability. However, variation
191
that is part of everyday operation, such as changes in reagent lots and technical staff,
192
should be integrated into RI studies whenever possible to approximate normal
193
working conditions.
194
7.1 Establish a laboratory submission policy for RI study samples.
195
7.2 Establish rejection criteria for samples of inadequate quality.
196
7.3 Monitor results in real time so that errors can be detected when re-measurement
197
is still possible. This will prevent excessive rejection of reference values by
198
reducing the number of potential outliers.
199
NOTE: Certain analytes in avian and reptiles may be unusual, and yet physiologic, during
200
stages of the reproductive cycle, e.g., ionized calcium. Analytical interference with other
201
analytes caused by these unusual concentrations must be known and accounted for during
202
RI studies.
203
NOTE: Method(s) employed in establishing RI should be documented in detail, including
204
the specific make and model of analyzer and the source of the reagents and quality
205
control materials.
206
Statistical analysis of reference values
9
MASTER COPY 10/18/10
207
208
8. Prepare and examine histograms of the reference values for initial assessment of
distribution and identification of potential outliers.
209
9. Identify outliers. Outliers are values that do not truly belong to the underlying
210
distribution of reference values. Inclusion of outliers will significantly affect
211
reference limits; however, apparent outliers or unexpected values should not be
212
eliminated indiscriminately.
213
9.1 Examine values that appear to be outliers in the histogram for errors. Errors may
214
include, but are not limited to the following:
215

Transcription errors
216

Preanalytical and analytical errors, including those resulting from
217
inclusion of inappropriate or poor quality samples (improper sample,
218
hemolysis, lipemia)
219

Inclusion of inappropriate reference individuals (animals subsequently
220
proven to be unhealthy, animals that should have been excluded due to
221
age, breed, physiology, etc.)
222
NOTE: Values resulting from these types of errors should be eliminated whether or not
223
they are located within the extremities of the distribution.
224
9.2 Use an appropriate statistical method to further examine the reference data for
225
outliers. The following are 2 of the most commonly used tests to detect outliers
226
in reference data, although other methods are available (Grubb 1950, Jain
227
2010). Optimal performance of these tests requires that the data approximate a
228
Gaussian distribution. (Horn 2001) Therefore, application of these tests
10
MASTER COPY 10/18/10
229
typically occurs after data has undergone transformation (if not Gaussian) and is
230
tested for normality.
231

Dixon’s outlier range statistic typically identifies the single most extreme
232
value at the upper or lower limits as an outlier. (Dixon 1983) The simplest
233
criterion of rejection is D/R > 0.3, where D is the absolute difference
234
between the most extreme value and the next nearest value divided by the
235
range of all values (R) including the extreme values. (Reed 1971) This
236
criterion is fairly conservative and favors retention of reference values. If
237
several values at one extremity appear to be outliers, the least most extreme
238
value can be treated as the most extreme for calculating the ratio (block
239
procedure). If the least most extreme value is determined to be an outlier,
240
all the more extreme values can be eliminated. (Barnett 1978) The ratio (r
241
criteria) also can be compared to published tables of critical values for more
242
stringent evaluation of outlier status. (Rorabacher 1991) Critical values vary
243
depending on the number of anticipated outliers at one or both extremities
244
(one- or two-tailed), the number of reference values, and the desired level of
245
confidence in detecting outliers (e.g.,  = 0.1 or 0.05).
246

Horn’s algorithm using Tukey’s interquartile fences identifies multiple
247
outliers located at the upper and lower extremities. (Horn and Pesce 2003,
248
Horn and Pesce 2005) The criterion for rejection is values exceeding
249
interquartile (IQ) fences set at Q1 - 1.5*IQR and Q3 + 1.5*IQR (IQR =
250
interquartile range; IQR = IQ3 – IQ1 where IQ1 and IQ3 are the 25th and 75th
11
MASTER COPY 10/18/10
251
percentiles, respectively). This test is more stringent than Dixon’s range
252
statistic.
253
9.3 Correct known errors in outliers if possible.
254
9.4 Eliminate outliers proven not to belong to the reference population by statistical
255
or other means.
256
9.5 Once outliers are eliminated, retest the remaining data for additional outliers.
257
9.6 When data do not approximate Gaussian distribution or cannot be transformed to
258
Gaussian distribution, these procedures do not perform optimally and may
259
erroneously identify values located in the tail of skewed distributions as outliers.
260
(Horn 2001) When data are non-Gaussian, nonparametric methods should be
261
used to establish RI (see below). Because nonparametric methods establish
262
reference limits by trimming the most extreme values, outliers have less of an
263
effect on the RI than with parametric and robust methods. (Horowitz 2008)
264
9.7
Boxplots are a nonparametric, graphical representation of quartile analysis with
265
the lower and upper edges of the box representing the 25th and 75th percentiles,
266
respectively. If reference data are displayed using boxplots, the whiskers should
267
be set at Q1 - 1.5*IQR and Q3 + 1.5*IQR, which are the fences delineating
268
outliers according to Horn’s algorithm. Values outside the whiskers are
269
considered outliers. Realize, however, that if the data are non-Gaussian, these
270
fences may not perform optimally.
271
272
Not all outliers and inappropriate values will be detected with these statistical methods.
273
(Solberg & Lahti 2005) Multiple outliers located at one or both extremities may have the
12
MASTER COPY 10/18/10
274
effect of masking the presence of outliers and rendering these methods unsuitable for
275
outlier detection. (Horn 2001) The best means by which to avoid inclusion of
276
inappropriate values within the reference data is to ensure that all reference individuals
277
are healthy and belong to the desired demographic and to avoid unintended preanalytical
278
and analytical variation. The challenge of correctly identifying outliers is magnified in
279
wildlife RI studies where health is difficult to substantiate and where manual or field
280
methods may introduce a higher level of imprecision and inaccuracy.
281
282
When health is well-defined and can reasonably be substantiated, conservative outlier
283
tests, which favor retention of values, are acceptable (e.g., Dixon’s range statistic,
284
confidence levels of  = 0.05). This is analogous to low Type I error. However, when
285
“convenience samples” are used or when health is difficult to substantiate, as is often the
286
case with wildlife RI studies, more stringent outlier tests should be applied (Horn’s
287
algorithm using Tukey’s interquartile fences or comparing Dixon’s r criteria to tables of
288
critical values with confidence levels of  = 0.1). These more stringent criteria are more
289
likely to exclude potentially erroneous values (low Type II error).
290
291
In conclusion, attempts to identify and eliminate outliers should be made during the
292
analysis of reference values. Outliers should not be eliminated indiscriminately because
293
these values may represent the true distribution of values in a healthy population.
294
However, inclusion of true outliers can significantly affect the determination of reference
295
limits rendering the RI less representative of the desired demographic. Clinical
13
MASTER COPY 10/18/10
296
experience and common sense also must be employed in determining when to retain or
297
eliminate outlying values.
298
299
10. If parametric or robust methods will be used to determine RI, use a goodness-of-fit
300
test to determine if the distribution of the reference data is Gaussian. One of the
301
following methods may be selected:
302

Anderson-Darling
303

Kolomogorov-Smirnov
304

Shapiro-Wilk
305

D’Agostino and Pearson omnibus
306
If distribution is not Gaussian, transform the data using an appropriate function (e.g.,
307
log or Box-Cox transformation) and reassess the distribution. If Gaussianity cannot
308
be established after several transformation trials, parametric methods cannot be used
309
to establish RI. Although best applied to data with a symmetrical distribution, the
310
robust method may be used when reference values do not exhibit Gaussian
311
distribution. (Horowitz, 2008; Horn & Pesce, 2005, Chapter 6, page 47-57)
312
11. Select statistical method for analysis based on the number and distribution of
313
reference values.
314
11.1 Nonparametric methods do not require assumption of a particular distribution of
315
the data and are recommended when ≥120 reference samples are available.
316
Nonparametric methods typically encompass the central 95th percentile of
317
reference values and use the 2.5th and 97.5th fractile as the lower and upper
318
reference limit, respectively. (Horn and Pesce 2003) Nonparametric determination
14
MASTER COPY 10/18/10
319
of 90% confidence intervals (CI) of the lower and upper reference limits are
320
possible with ≥120 reference samples. Although robust methods are preferred
321
when there are <120 reference values (see below), nonparametric methods can be
322
used when the distribution of reference data is not Gaussian; however, 90% CI
323
cannot be calculated nonparametrically. (Horowitz 2008) Forty is the minimum
324
number of values required to determine central 95% RI nonparametrically, but in
325
this situation, the most extreme values serve as the lower and upper reference
326
limits. Because these extreme values may include outliers, it is important to have
327
a sufficient number of samples to allow elimination of outliers (Horn and Pesce
328
2003); for example, n = 40 after elimination of outliers or n = 44, which should
329
allow trimming at both extremities.
330
11.2 Robust methods are recommended when 20 ≤ x ≤ 120 reference samples are
331
available. The robust method utilizes an iterative process to estimate location and
332
spread of the data. (Horn & Pesce 2005, 1999, 1998) Although the robust method
333
performs best when reference data has a symmetrical distribution, it can be used
334
in the absence of Gaussianity. Ninety percent CI should be determined using
335
robust methods. The robust method is included in several clinical laboratory
336
statistical (or software) programs (CBstat
337
http://direct.aacc.org/ProductCatalog/Product.aspx?ID=2475, Reference Value
338
Advisor freeware available at http://www.biostat.envt.fr/spip/spip.php?article63 ),
339
and MedCalc at http://www.medcalc.be/.
340
341
11.3 Parametric methods may be used when 20 ≤ x ≤ 120 reference samples are
available and the data has, or can be transformed to, Gaussian distribution.
15
MASTER COPY 10/18/10
342
Parametric methods encompass slightly more than the central 95% of the data and
343
establish the lower and upper reference limits at mean minus 2SD and mean plus
344
2SD, respectively (SD = standard deviation). Ninety percent CI should be
345
determined using parametric methods.
346
11.4 When RI are calculated from 10 ≤ x ≤ 20 reference values, the following
347
information should be included:
348

A histogram of the data
349

A list of all reference values
350

Mean (if Gaussian) or median (if not Gaussian) (minimum and maximum
351
352
353
values should be reported if a list of all reference values are not included)

RI should be calculated by the robust (distribution independent) or parametric
(if Gaussianity can be established) method
354
Although not ideal, it is acknowledged that situations occur in veterinary
355
medicine in which RI must be calculated from 10 ≤ x ≤ 20 reference values.
356
Emphasis should be placed on collecting samples that are free from unintended
357
variability by paying strict attention to selection of suitable reference subjects and
358
adherence to standardized collection techniques and well controlled methods of
359
analysis. Evaluation for the presence of outliers is particularly important with
360
small sample sizes, because the presence of a single outlier has a significant effect
361
on estimated reference limits. If outliers are eliminated, every attempt should be
362
made to collect replacement samples.
363
NOTE: Reference intervals calculated by several of the methods listed above can be
364
compared. If the RI are similar, any of the statistical methods is acceptable. (Horn 1998)
16
MASTER COPY 10/18/10
365
When RI differ, clinical judgment may be required to select the most appropriate
366
reference limits with some experts recommending selection of the narrower RI in order to
367
limit the number of false negative results. (Horn 1998)
368
11.5 This document does not contain recommendations for determining RI from <10
369
reference samples because reliable estimates of population-based reference limits
370
cannot be made from such small sample sizes.
371
NOTE: Confidence intervals around the upper and lower reference limits should be
372
calculated whenever sample size permits. Confidence intervals provide an estimate of
373
the uncertainty of the reference limits and are generally narrower for larger sample sizes
374
than for smaller sample sizes. Boyd and Harris recommend that CI should not exceed 0.2
375
times the width of the RI. (Harris 1995) When CI exceed this limit, an effort to collect
376
additional reference samples should be made.
377
12. Determine the need for partitioning into subclasses based on physiological
378
differences that are expected to result in important differences in RI. Partitioning
379
favors homogeneous sub-populations, decreasing variability between individuals and
380
narrowing the RI. However, partitioning only should be considered if there is a
381
minimum of 20 individuals within each subclass. Partitioning criteria should consider
382
not only the mean of the analyte, but also the subgroup standard deviations (SD).
383
More recent partitioning criteria examine the proportion of each subgroup that would
384
fall outside the upper and lower limits of a combined RI (Lahti 2004, Ceriotti 2009).
385
This proposal is based on the fact that common RI contain 95% of reference values
386
and exclude 2.5% of values at the upper and lower extremities. Consideration also
387
must be given to unequal sizes of the subgroups within the general population as well
17
MASTER COPY 10/18/10
388
as within the reference sample group. (Lahti 2002 – partitioning) The following are
389
some simple recommendations:
390
12.1 Partitioning is recommended if the difference between subgroup means exceeds
391
25% of the reference range (range = upper limit – lower limit) of the combined
392
central 95% RI. (Sinton, 1986) This method is conservative in recommending
393
partitioning and requires Gaussian distribution of data for optimal performance.
394
(Lahti 2004 existing methods)
395
12.2 Partitioning is recommended if the ratio of subgroup SD (larger SD/small SD)
396
exceeds 1.5, regardless of the subgroup means. (Harris and Boyd 1990) If this
397
criterion is exceeded, Harris and Boyd recommend that subgroup means be
398
compared by the standard normal deviate test. If there are 120 samples in each
399
subgroup, partitioning is recommended if z > 3. The z value is calculated as
400
follows:
z = mean1 – mean2 / [(SD12/n1) + (SD22/n2)]1/2
401
402
If there is a different number of samples in the subgroups (n), a correction factor
403
is applied to the “critical z-statistic” (critical z-statistic = 3 x (naverage/120)1/2).
404
This procedure works optimally when the data has a Gaussian distribution and the
405
subclasses are of similar size and SD. (Lahti 2004 – existing)
406
12.3
Partitioning recommendations by Lahti et al for Gaussian distributions begin
407
with the same subgroup SD ratio criterion of >1.5. However, if the SD ratio is ≤
408
1.5, then the differences between the upper (DU) and lower limits (DL) of the 2
409
subgroups should be examined in relation to the smaller of the subgroup SD
410
(SDsmaller). (Lahti 2002 – Gaussian)
18
MASTER COPY 10/18/10
411
DU = URL1 – URL2 (use absolute values)
412
DL = LRL1 – LRL2
413
Partitioning is not recommended when both DL and DU < 0.25 SDsmaller
414
Partitioning is recommended when either DL, DU or both ≥ 0.75 SDsmaller
415
When either DU or DL or both fall in between these criteria, the decision on
416
whether to partition is made using non-statistical criteria.
417
12.4
The partitioning recommendations above, while reasonably simply to
418
calculate, contain several weaknesses. (Lahti 2004 – existing methods) To correct
419
for these weaknesses, Lahti et al made the following general recommendations for
420
partitioning; however, application of these criteria is more challenging. (Lahti
421
2004) Partitioning is recommended if >4.1% or <0.9% of a subgroups falls
422
outside the upper or lower limits of a combined RI. If 1.8% < x < 3.2 % of a
423
subgroup falls outside the combine RI, partitioning is not recommended. When
424
the proportions of a subgroup outside a combined RI are between these criteria,
425
the decision to partition the RI is made using non-statistical criteria.
426
12.5 Non-statistical criteria that support partitioning include:

427
428
If descriptors used to assign a patient to a partitioned subgroup are easily
obtainable and maintained in the patient record.
429

If reference limits serve as critical clinical decision limits.
430

If the literature documents differences between subgroups
431
13. Document all previous steps and procedures so that RI are clearly defined. A
432
complete and detailed RI study summary document should be available to users upon
433
request.
19
MASTER COPY 10/18/10
434
14. Reference intervals determined de novo within a laboratory should be reviewed
435
every 3 to 5 years and re-validated if needed. Re-validation is necessary anytime there
436
has been a change in assay methodology or a significant change in patient
437
populations. (See the section on transference and validation.)
438
Postanalytical procedures – Presentation of reference intervals
439
15. Information included in a laboratory report should aid the clinician in the medical
440
decision making process and be presented in a clear and concise manner.
441
15.1 Reference intervals typically are printed on the patient report; however, this
442
443
should occur only when the RI is applicable to that patient.
15.2 Reference intervals that deviate from customary percentiles and limits or are
444
specific to a certain subclass (age, sex) should clearly be identified on the
445
report.
446
447
448
15.3 It is useful to indicate which patient values are increased or decreased in
comparison to the RI.
15.4 Information that may be important for clinical decision-making, but that cannot
449
be contained within the patient’s laboratory report, should be available to the
450
clinician in a written report (RI study summary document). This information
451
may include, but is not limited to: (Plebani) (see Table 3)
452

453
Reference population demographics and number of reference subjects
sampled
454

Subject preparation and time or season of collection, if relevant
455

Sample type and handling
20
MASTER COPY 10/18/10
456

457
458
reagents employed

459
460
461
Quality specifications of the analytical method and specific analyzer and
Statistical methods used to evaluate the reference values and to determine
the reference limits

Confidence intervals around the reference limits
15.5 Changes in RI owing to introduction of new methods or analyzers or
462
adjustments to RI resulting from changes in population demographics should
463
be communicated to all users.
464
465
TRANSFERENCE AND VALIDATION OF REFERENCE INTERVALS
466
In order to forego the expense and difficulty of establishing intra-laboratory RI, many
467
laboratories adopt RI from other laboratories or from instrument manufacturers. The
468
following procedures are recommended for the transference and validation of RI adopted
469
from other sources.
470
1. Several issues should be scrutinized when external RI are considered for transference.
471
1.1 The appropriateness of the reference population with respect to age, sex,
472
breed, geography, physiology, etc.
473
1.2 Differences in pre-analytical techniques, such as patient preparation and
474
collection method.
475
1.3 Differences in test methodology
476
1.4 Differences in instrument accuracy and imprecision (analytical quality).
477
1.5 Differences in analytical quality requirements established by the laboratory
478
donating the RI and the laboratory adopting them. (K Freeman)
21
MASTER COPY 10/18/10
479
If significant differences are detected in the above areas, transference may not be
480
appropriate.
481
information necessary to determine the appropriateness of transference.
Many RI studies are not well documented and often lack the detailed
482
2. Estimates of analytical error (CV and bias) are necessary to determine transferability.
483
RI can be transferred between laboratories using different analytical methods as long as
484
the methods possess similar analytical quality (see section 3.5). In addition, a transferred
485
RI only remains valid as long as the laboratory maintains the original precision (CV) and
486
accuracy (bias) that were used to establish transference validity. (Ceriotti 2009)
487
3. A comparison of methods study may be used to determine if analytical methods are
488
similar. (Jensen 2006 and ASVCP QC Guidelines website)
489
3.1 Analytical methods are comparable if the slope of the regression line
490
approximates 1.0 and the y-intercept is small relative to the data range and RI. If
491
comparable, RI can be transferred directly.
492
3.2 If systematic difference (bias) exists between analytical methods, regression
493
statistics are used to adjust the upper and lower reference limits. This commonly
494
is used when a laboratory changes instruments or methods and transfers their
495
existing RI to the new instrument or method. Transference using regression
496
statistics only should be employed for one change of instrumentation or
497
methodology. Depending on the distribution of data, alternate statistics may be
498
required, e.g., Passing-Bablock or Deming regression, difference of means for
499
analytes with narrow distributions such as electrolytes. An example of adjusting a
500
RI for bias using regression statistics can be found in the forthcoming supplement
501
to this document.
22
MASTER COPY 10/18/10
502
4. Once it is determined that a RI is suitable for transference, validation of the transferred
503
RI may be accomplished by one of the following procedures:
504
4.1 Subjective assessment requires complete and detailed documentation of the
505
original RI study including all the factors listed in section 3.1 as well as
506
information concerning statistical methods used for the analysis of the reference
507
values. Because such detailed summaries are infrequently available, this
508
validation procedure is seldom sufficient.
509
4.2 Evaluating 20 samples representative of the laboratory’s own patient
510
population against the candidate RI is relatively quick and straightforward. These
511
values first should be examined for outliers. If outliers are detected, they should
512
be eliminated and additional samples collected. If ≤ 2 of the 20 values fall outside
513
the candidate RI, it is considered transferable. If 3 or 4 values fall outside the RI,
514
another 20 patients can be tested and interpreted in the same manner as the
515
original 20 samples. If > 4 of the original 20 values fall outside the candidate RI,
516
transference is rejected for that analyte. This procedure is basically a binomial
517
test and will not determine whether the transferred RI is too wide for the receiving
518
laboratory. If all 20 samples fall within the candidate RI, it may be
519
inappropriately wide for the adopting laboratory. When this occurs, another 20
520
samples should be evaluated. The probability that all 20 results will be within the
521
candidate RI is about 0.36 and with 40 samples, the probability falls to about 0.13.
522
If all 40 samples fall within the candidate RI, then it is likely too wide and de
523
novo RI should be determined. (Horn and Pesce 2005, Chapter 10, page 77-80.)
23
MASTER COPY 10/18/10
524
4.3 A more thorough validation uses results from 40-60 healthy subjects from the
525
laboratory’s patient population. If individual results are available from the
526
original RI study, the reference values from both groups are compared using more
527
sophisticated statistical equations (Mann-Whitney U test, median test, Siegel-
528
Tukey test, Kolmogorov Smirnov test). (Horowitz) With the introduction of the
529
robust method, acceptable RI can be generated from 40-60 reference samples,
530
obviating the need for transference and validation using this method.
531
NOTE: The clinical use of the test should be considered when selecting the method for
532
validating a transferred RI, e.g., for more critical tests, procedure 4.2 or 4.3 should be
533
used.
534
5. It is the responsibility of the end user to validate RI provided by manufacturers of
535
veterinary diagnostic tests and analyzers. Manufacturers may generate RI using a single
536
analyzer or multiple analyzers at one or more locations (multicenter RI – see below).
537
Manufacturers should provide sufficient information regarding the RI so that their clients
538
can assess the quality and applicability of the RI study to their patients. Information
539
provided by the manufacture should include, but is not limited to the following:
540

Selection method of reference subjects (direct or indirect)
541

Demographics of the reference sample group
542

Preanalytical and analytical factors
543

Number of reference values used to establish RI
544

Statistical methods used to identify outliers and generate RI
545
546
6. Indications for re-validation of RI include but are not limited to:

Excessive false positive and false negative results
24
MASTER COPY 10/18/10
547

548
549
Significant changes in patient populations, preanalytical techniques, or analytical
quality

Periodic reassessment of RI every 3-5 years
550
551
COMMON (or multicenter) REFERENCE INTERVALS
552
Another alternative to determining RI de novo within each laboratory is for several
553
laboratories serving a similar patient population to contribute to the generation of
554
common (or multicenter) RI. The following summarizes the advantages and important
555
considerations of this approach. (Petersen 2004, Horowitz 2008, Jones 2004)
556
1. Advantages of common RI include:
557
1.1 Standardization of results, reporting, and interpretation in a mobile population
558
in which veterinary patients may change locations and laboratories during their
559
lifetimes.
560
1.2 Ability to recruit large numbers of reference samples from multiple
561
participating laboratories.
562
1.3 Large numbers of reference values allow partitioning according to desired
563
criteria (age, sex, breed, use, activity, other).
564
1.4 Distribute cost of establishing RI among multiple laboratories.
565
2. Laboratories participating in a common RI study should adhere to the following
566
procedures:
567
2.1 Laboratories contributing results to a common RI do not have to have the
568
same analyzer (manufacturer and model number). However, all analyzers must
569
be calibrated to produce comparable results (Jensen 2006) and must meet the
25
MASTER COPY 10/18/10
570
same quality requirements. Common calibration and performance quality are
571
essential if the resulting RI are to be used by all participating laboratories. If
572
common RI are adopted by a laboratory that did not participate in the study, the
573
common RI must be validated, even if the laboratory uses the same analyzer as
574
was used in the study.
575
2.2 Determine if similar populations are present among the participating
576
laboratories – a prerequisite for the use of common RI.
577
2.3 Establish uniform selection and exclusion criteria for defining reference
578
individuals and establishing health, as detailed in section 1.2 of these guidelines.
579
2.4 Standardize pre-analytical and analytical factors, as detailed in sections 1.6
580
and 1.7 of these guidelines.
581
2.5 Establish quality requirement goals for imprecision (CV) and inaccuracy
582
(bias) for all analytes. These may be based on biologic variation, clinical
583
interpretation of test results, or both.
584
2.6 Bias, in particular, must be controlled and minimized by each laboratory. If
585
bias varies significantly among participating laboratories, the use of common RI
586
may lead to clinical misclassifications. A ‘maximum allowable bias’ should be
587
established based on calculated TE using the maximum CV obtained by
588
participating laboratories. If a laboratory exceeds this predetermined bias limit,
589
measures should be initiated (re-calibration, etc.) to return bias to limits that allow
590
continued use of the common RI.
26
MASTER COPY 10/18/10
591
2.7 Use calibrators traceable to an international standard to determine the
592
trueness and comparability of measurements across all participating laboratories.
593
Alternatively, a single, pooled specimen may be used as a common calibrator.
594
2.8 Once common RI are established, variability caused by changes in calibrator
595
lots must be minimized. Instead of using the mean assigned to the calibrator by
596
the manufacturer, a mean value for a common calibrator should be determined
597
from results submitted by all participating laboratories. Each laboratory
598
establishes a mean calibrator value by repeat analysis (n = 10 to 20 repetitions)
599
and then submits this mean for determination of a common mean.
600
2.9 Validate quality control procedures designed for high probability of error
601
detection and low probability of false rejection in order to monitor and maintain
602
stable performance.
603
2.10 If possible, use the same quality control materials and reagents to facilitate
604
standardization and comparison of results. For analyzers with unique QC
605
materials (hematology analyzers), this may not be possible.
606
NOTE: Manufacturers of diagnostic equipment may provide users with ‘common RI’
607
determined from one or more analyzers; however, RI validation is recommended to
608
ensure that local patient populations are represented by the common RI and that
609
performance of the on-site analyzer meets calibration and quality performance
610
requirements established in the common RI study.
611
612
USE OF PUBLISHED REFERENCE INTERVALS
27
MASTER COPY 10/18/10
613
Interpreting clinical data using inappropriate RI may lead to misclassification of a patient,
614
which can result in misdiagnoses, improper treatments or both. Unless a RI is
615
representative of the patient’s demographics and is determined using similar preanalytical
616
procedures, and comparable analytical methods, it is not appropriate as a diagnostic
617
reference for clinical decision-making. Reference intervals published in textbooks,
618
journal articles, and web-based databases or provided by instrument manufacturers may
619
or may not contain sufficient information to determine whether the RI is appropriate for a
620
particular patient. In addition, the quality of published RI is quite variable. In addition to
621
information regarding preanalytical and analytical procedures, the quality of a RI depends
622
on the reference sample size, the use of procedures to detect and eliminate outliers, and
623
correct use of statistical procedures to estimate the reference limits. Published RI should
624
be used with caution, and only when sufficient information is available to determine their
625
quality and applicability to the patient and analytical method. If published RI are adopted
626
for extended use, appropriate validation procedures should be performed as described in
627
this document.
628
629
BIOLOGICAL VARIATION, INDIVIDUALITY, AND SUBJECT-BASED
630
REFERENCE INTERVALS
631
Population-based reference intervals serve as a comparison for patient test results when
632
an alternative frame of reference is not available. However, due to relatively high inter-
633
individual variability, population-based reference intervals sometimes lack necessary
634
sensitivity to detect changes in the health of an individual. The following
635
recommendations provide guidance for the appropriate use of subject-based RI.
28
MASTER COPY 10/18/10
636
1. Subject-based RI are more sensitive than population-based RI for disease detection
637
when intra-individual biologic variation, or coefficient of variation of an individual
638
(CVI), is less than inter-individual variation, or CV of a group (CVG). (Fraser 2004)
639
1.1 The index of individuality provides an objective criterion to determine the
640
relative utility of subject-based versus population-based RI. A quantitative
641
measure of individuality, the index of individuality is represented by the equation,
642
(CVI2 + CVA2)1/2 / CVG, where CVA is analytical variation (random error or
643
imprecision). Because CVA < CVI for many automated assays, the index of
644
individuality is often simplified as CVI/CVG. (Fraser 2004)
645
1.2 When the index of individuality is < 0.6, a subject-based RI is preferable to a
646
population-based RI, whereas when it is >1.4, individual RI yield no more
647
information than population-based RI. (Fraser and Harris 1989) For patient
648
monitoring, using an index < 0.48 rather than <0.6 increases the probability of the
649
subject-based RI detecting change in a monitoring situation. (Iglesias Canadell et
650
al. 2005)
651
1.3 With subject-based RI, a reference change value (RCV) serves to determine
652
whether a difference between consecutive measurements in an individual is
653
significant (p ≤ 0.05). The RCV is based upon CVI values in health and the
654
dispersion of these variations across a population: RCV = z x [2 x CVI2 + CVA2]1/2
655
where z represents the z-statistic; since CVA < CVI for many automated assays,
656
RCV is simplified as z x 21/2 x CVI. RCV is best applied when CVA < 0.5 x CVI.
657
1.4 The z-statistic conventionally used for RCV calculation is z = 1.96 [5%
658
probability of type I error]. When larger z-factors are used, the probability of a
29
MASTER COPY 10/18/10
659
significant change being detected increases; with z = 3.34 the probability of
660
detecting change increases from 50% (z = 1.96) to 90%. (Iglesias Canadell et al.
661
2004)
662
2. To use subject-based RI, it is necessary to know the CVI for each analyte, as well as
663
the imprecision of the analytical method for that analyte.
664
2.1 Published CVI are available for many analytes in the dog (Jensen and
665
Kjelgaard-Hansen, Wiinberg) and for some exotic species (Bertelsen).
666
2.2 Biologic variation can be measured with only a small number of healthy
667
reference individuals when care is taken to minimize pre-analytical variation;
668
thus, these data may readily be measured by a single laboratory when published
669
biologic variation data are not available. (Fraser and Harris)
670
2.3 Use of CVI and RVC derived from healthy individuals to determine
671
significant changes in analyte quantities in patients with stable chronic diseases
672
may not be appropriate. Consequently, estimates of CVI from patients with
673
chronic stable disease are being developed in human medicine to better monitor
674
chronic disease conditions. (Ricos 2007) This information currently is not
675
available in veterinary medicine.
676
677
EVALUATING TEST ACCURACY AND ESTABLISHING DECISION
678
THRESHOLDS (or decision limits)
679
The clinical utility of a test depends partly on test accuracy – that is, the ability of the test
680
to discriminate between patients with disease and without disease. Test accuracy, in turn,
681
depends on the decision limit used for determining a positive or negative result. Unlike
30
MASTER COPY 10/18/10
682
RI, which are “defined by statistical methods and are descriptive of specific populations,
683
decision thresholds are defined by consensus and distinguish among different
684
populations”. (Horowitz 2008) The following procedure outlines the steps and basic
685
principles required for designing prospective studies to evaluate the diagnostic accuracy
686
of a clinical laboratory test and to establish diagnostic threshold. (Jorgensen 2004,
687
Peblani 2004, Zwieg 1995) These same principles can be used to critically evaluate data
688
retrospectively.
689
1. Define the clinical question with regards to the following factors:
690
1.1 Characterize the target population as to the frequency and duration of disease,
691
as well as age, sex, and other relevant information that aids in interpretation of
692
test results.
693
1.2 State the management decision to be made concerning the disease in
694
question.
695
1.3 Identify the role of the test in the clinical decision-making process with
696
regards to the disease.
697
2. Prospectively select individuals (study sample) that are representative of the relevant
698
target population. The study sample should consist of subjects that are anticipated to test
699
both positive and negative for the test under investigation. Consult a statistician to
700
establish the size of the study sample required for statistically relevant results.
701
2.1 Diversity within the target population should accurately represent what is
702
expected in clinical practice.
703
2.2 If the clinical question concerns the presence or absence of a disease, a
704
statistically relevant number of individuals should have the disease in question.
31
MASTER COPY 10/18/10
705
2.3 If the clinical question concerns determining the severity or prognosis of a
706
disease, the entire sample may consist of diseased animals representing the entire
707
spectrum of disease severity or outcome.
708
2.4 To prevent bias, selection of study subjects should be independent of test
709
results for the test or analyte being evaluated.
710
3. Whenever possible, study subjects should be tested with the test under investigation
711
without knowledge of their disease classification to prevent prejudiced decision-making.
712
3.1 When comparing performance of multiple tests, tests should be executed on
713
all subjects at the same time or at the same stage of disease, and all tests should be
714
performed on the same sample.
715
3.2 If sample stability permits, assaying samples in a single batch minimizes
716
between-run analytical variance.
717
3.3 Subjects with unexpected test result should not be excluded from the study to
718
avoid bias favoring test performance.
719
4. Comprehensive examination and alternative testing are used to establish true disease
720
positive and true disease negative status. For tests evaluating disease severity or
721
prognosis, standardized staging, grading, or scoring schemes (Hayes, 2010) and common
722
outcome assessments should be used.
723
5. Receiver-operating characteristic (ROC) curve are created by plotting the sensitivity
724
and specificity of the test under investigation at a variety of decision thresholds. (Gardner
725
2006, Stephan 2003) ROC curves subsequently can be used to compare performance of
726
multiple tests and to select of decision thresholds that optimally answer the clinical
727
question or satisfy the diagnostic requirement.
32
MASTER COPY 10/18/10
728
5.1 The area under the curve (AUC) is an estimate of test accuracy. Under most
729
circumstances when comparing multiple tests, the more accurate test has the
730
higher AUC.
731
5.2 Optimal decision limits are selected by location on the ROC plot (most upper
732
left point) or by determining the decision limit with the highest proportion of
733
correct interpretations (most true positive and true negative results).
734
5.3 Confidence intervals around points on an ROC curve can be derived
735
parametrically and non-parametrically.
736
6. Further confirmation of the true clinical state of each subject should be done during or
737
after the completion of the study without utilizing the results of the test under
738
investigation.
739
6.1 This provides quality control crosscheck to insure that previous criteria for
740
disease classification are accurate.
741
6.2 Confirmation procedures may include histopathology (surgical biopsy or
742
necropsy) or preferably long-term follow-up regarding the course of disease.
743
744
Summary and closing
745
A uniform and consistent approach to establishing RI will benefit the entire veterinary
746
medical community. The acceptance of statistical methods for establishing RI from small
747
samples sizes, the approval of transference and validation as a valid method for obtaining
748
RI, and a growing interest in common RI will expand the ability of veterinary diagnostic
749
laboratories to provide RI for a variety of species and distinct subgroups. Adherence to
750
these guidelines by all those establishing and publishing RI in animals should facilitate
33
MASTER COPY 10/18/10
751
communication within the broader veterinary community. By providing detailed
752
information in RI study summaries, judicious use of published RI may be possible (see
753
Table 3). In the absence of appropriate, population-based RI, subject-based RI may be a
754
viable alternative for interpreting clinical data in certain circumstances.
755
756
757
758
References
759
ASVCP Quality Control Guidelines
760
http://www.asvcp.org/pubs/pdf/ASVCPQualityControlGuidelines.pdf accessed
761
10/13/2010
762
Barnett V, Lewis T. Outliers in Statistical Data. Chichester, England: John Wiley and
763
Sons, Ltd.; 1978;381-386.
764
Bertelsen MF, Kjelgaard-Hansen M, Howell JR, Crawshaw GJ. Short-term
765
biological variation of clinical chemical values in Dumeril’s monitors (Varanus
766
dumerili). J Zoo Wildl Med 2007;38(2):217-21.
767
Ceriotti F, Hinzmann R, Panteghini M. Reference intervals: the way forward. Ann Clin
768
Biochem 2009;46(Pt 1): 8-17.
769
Dixon WJ. Processing data for outliers. Biometrics 1983;9:74-89. (corrections to this
770
article were made by Rorabacher)
771
Dybkǽr R. Approved recommendation (1987) on the theory of reference values. Part 6.
772
Presentation of observed valued related to reference values. Clin Chim Acta
773
1987;170:S33-S42.
34
MASTER COPY 10/18/10
774
Fraser CG. Inherent biological variation and reference values. Clin Chem Lab Med
775
2004;42:758-764.
776
Fraser CG, Harris EK. Generation and application of data on biological variation in
777
clinical chemistry. Crit Rev Clin Lab Sci 1989;27:409-437.
778
Gardner IA, Greiner M. Receiver-operating characteristic curves and likelihood ratios:
779
improvements over traditional methods for the evaluation and application of veterinary
780
clinical pathology tests. Vet Clin Pathol 2006;35:8-17.
781
Geffré A, Friedrichs K, Harr K, Concordet D, Trumel C, Braun JP. Reference
782
values: a review. Vet Clin Pathol 2009;38(3):288-98.
783
Grasbeck R, Saris NE. Establishment and use of normal values. Scand J Clin Lab
784
Invest. 1969;26(Suppl 110):62-63.
785
Grubb FE. Sample criteria for testing outlying observations. Annal Math Stat
786
1950;21:27-58.
787
Harris EK, Boyd JC. On dividing reference data into subgroups to produce separate
788
reference ranged. Clin Chem 1990;36(2):265-270.
789
Harris EK, Boyd JC. Statistical bases of reference values in laboratory medicine. New
790
York: Marcel Dekker, 1995:77-91.
791
Hayes G, Mathews K, Kruth S, Doig G, Dewey C. Illness severity scores in veterinary
792
medicine: what can we learn. J Vet Intern Med 2010;24:457-466.
793
Horn PS, Feng L, Li Y, Pesce AJ. Effect of outliers and nonhealthy individuals on
794
reference interval estimation. Clin Cehm 2001;47(12):2137-2145.
795
Horn PS, Pesce AJ. Reference intervals: an update. Clin Chim Acta 2003;334:5-23.
35
MASTER COPY 10/18/10
796
Horn PS, Pesce AJ. Reference intervals: a user’s guide, AACC Press, Washington DC,
797
2005.
798
Horn PS, Pesce AJ, Copeland BE. A robust approach to reference interval estimation
799
and evaluation. Clin Chem 1998;44(3):622-631.
800
Horowitz GL, Altaie S, Boyd J, Ceriotti F, Gard U, Horn P, Pesce A, Sine H,
801
Zakowski J. Clinical and Laboratory Standards Institute (CLSI). Defining, establishing,
802
and verifying reference intervals in the clinical laboratory; approved guidelines, 3rd ed,
803
CLSI document C28-A3, Vol. 28, No. 3, 2008.
804
Iglesias Canadell N, Hyltoft Petersen P, Jensen E, Ricos C, Jorgensen PE. Reference
805
change values and power functions. Clin Chem Lab Med 2004;42:415-422.
806
Iglesias Canadell N, Hyltoft Petersen P, Ricos C. Power function of the reference
807
change value in relation to cut-off points, reference intervals and index of individuality.
808
Clin Chem Lab Med 2005;43:441-448.
809
Jain RB. A recursive version of Grubb’s test for detecting multiple outliers in
810
environmental and chemical data. Clin Biochem 2010;xx:xx-xxx.
811
Jensen AL, Kjelgaard-Hansen M. Method comparison in the clinical laboratory. Vet
812
Clin Pathol 2006;35:276-286.
813
Jones GRD, Barker A, Tate J, Lim C, Robertson K. The case for common reference
814
intervals. Clin Biochem Rev 2004;25:99-104.
815
Jorgensen LG, Brandslund I, Hyltoft Petersen P Should we maintain the 95 percent
816
reference intervals in the era of wellness testing? A concept paper. Clin Chem Lab Med
817
2004;42:747-51.
36
MASTER COPY 10/18/10
818
Lahti A. Partitioning biochemical reference data into subgroups: comparions of existing
819
methods. Clin Chem Lab Med 2004;42:725-733.
820
Lahti A, Hyltoft Petersen P, Boyd JC. Impact of subgroup prevalences on partitioning
821
Gaussina-distributed reference values. Clin Chem 2002;48:1987-99.
822
Lahti A, Hyltoft Petersen P, Boyd JC, Fraser C, and Jørgensen N. Objective criteria
823
for partitioning Gaussian distributed reference values into subgroups. Clin Chem
824
2002;48(2):338-352.
825
Lahti A, Hyltoft Petersen P, Boyd JC, Rustad P, Laake P, Solberg HE. Partitioning of
826
nongaussian-distributed biochemical reference data into subgroups. Clin Chem
827
2004;50(5):891-900.
828
Petersen H, Rustad P. Prerequisites for establishing common reference intervals. Scand
829
J Clin Lab Invest; 2004; 6; 64(4):285-92.
830
PetitClerc C, Solberg HE. Approved recommendation (1987) on the theory of reference
831
values. Part 2. Selection of individuals for the production of reference values. Clinica
832
Chimica Acta; 1987; 170(2-3):S1-S11.
833
Plebani M. What information on quality specifications should be communicated to
834
clinicians, and how? Clin Chim Acta 2004;346(1):25-35.
835
Poole T. Happy animals make good science. Lab Anim; 1997; 31(2):116-24.
836
Reed AH, Henry RJ, Mason WB. Influence of statistical method used on the resulting
837
estimate or normal range. Clin Chem 1971;17:275-284.
838
Ricos C, Iglesias N, Garicia-Lario J, et al. Within-subject biological variation in
839
disease : collated data and clinical consequences. Ann Clin Biochem 2007;44:343-352.
37
MASTER COPY 10/18/10
840
Rorabacher DB. Statistical treatment for rejection of deviant values: critical values of
841
Dixon’s Q parameter and related subrange ratios at the 95% confidence level. Anal Chem
842
1991;63(2):139-146.
843
Sinton TJ, Cowley D, Bryant SJ. Reference intervals for calcium, phosphate, and
844
alkaline phosphatase as derived on the basis of multichannel-analyzer profiles. Clin
845
Chem 1986;32:76-79.
846
Solberg HE. Approved recommendation (1986) on the theory of reference values. Part 1.
847
The concept of reference values. Clin Chim Acta 1987;165:111-118.
848
Solberg HE. Approved recommendation (1987) on the theory of reference values. Part 5.
849
Statistical treatment of collected reference values. Determination of reference limits. Clin
850
Chim Acta 1987;170:S13-S32.
851
Solberg HE. Approved recommendation (1988) on the theory of reference values. Part 3.
852
Preparation of indivdiduals and collection of specimens for the production of reference
853
values. Clin Chim Acta 1988;177:S1-S12.
854
Solberg HE, Lahti A Detection of outliers in reference distributions: performance of
855
Horn's algorithm. Clin Chem 2005;51(12):2326-32.
856
Solberg HE, Stamm D. IFCC recommednation: The theory of reference values. Part 4.
857
Control of analytical variation in the production, transfer and application of reference
858
values. J Automat Chem 1991;13(5):231-234.
859
Stephan C, Wesseling S, Schink T, Jung K. Comparison of eight computer programs
860
for receiver-operating characteristic analysis. Clin Chem 2003;49(3):433-439.
861
Walton RN. Establishing reference intervals: Health as a relative concept. Seminars in
862
Avian and Exotic Pet Medicine; 2001; 10(2):66-71.
38
MASTER COPY 10/18/10
863
Wiinberg B, Jensen AL, Kjelgaard-Hansen M, et al. Study on biological variation of
864
haemostatic parameters in clinical healthy dogs. Vet J 2007;174:62-68.
865
Zweig, MH. Assessment of the clinical accuracy of laboratory tests using receiver
866
operating characterisitic (ROC) plots; Approved Guideline, Jan. 1995. CLSI/NCCLS
867
Document GP10-a. NCCLS, 940 W Valley Road, Suite 1400, Wayne, Pennsylvania
868
19087, USA
869
870
39
MASTER COPY 10/18/10
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
Figure 1. Procedural flow chart for de novo determination of RI for new methods or new
populations.
LITERATURE SEARCH
↓
DEFINE REFERENCE POPULATION
and ESTABLISH SELECTION, INCLUSION, AND EXCLUSION CRITERIA
↓
DEVELOP QUESTIONNAIRE
↓
DETERMINE NUMBER OF REFERENCE INDIVIDUALS AVAILABLE or
REQUIRED for REFERENCE LIMITS with SUFFICIENTLY NARROW
CONFIDENCE INTERVALS
↓
SELECT REFERENCE INDIVIDUALS by DIRECT METHODS
↓
COLLECT and HANDLE SAMPLES IN STANDARDIZED MANNER
↓
ANALYZE SAMPLES USING WELL-CONTROLLED METHODS
↓
PREPARE HISTOGRAM
↓
IDENTIFY OUTLIERS
↓
DETERMINE DISTRIBUTION OF REFERENCE SAMPLES (Gaussian or nonGausian). IF USING PARAMETRIC METHODS, TRANSFORM DATA and RETEST
DISTRIBUTION.
↓
CALCULATE REFERENCE LIMITS USING an APPROPRIATE STATISTICAL
METHOD BASED ON DISTRIBUTION of DATA and NUMBER OF SAMPLES
↓
CALCULATE CONFIDENCE INTERVALS FOR THE UPPER and LOWER
REFERENCE LIMITS
↓
DETERMINE THE NEED FOR PARTITIONING IF THERE ARE SUFFICIENT
SAMPLE NUMBERS
↓
DOCUMENT ALL PREVIOUS STEPS for a COMPREHENSIVE REFERENCE
INTERVAL REPORT
40
MASTER COPY 10/18/10
911
Table 1. Criteria for the selection or partitioning of reference individuals.
Selection
criteria
Biological
Clinical
Geographical
Examples
Exclusion
criteria
Biological
Age
e.g., neonate, juvenile,
adult
Sex
e.g., female, male,
altered
Breed
History of health or
Physiologic
illness
Preventative drug history
e.g., vaccines, routine
anthelmintics
Physical examination
Further diagnostic
evaluation
e.g., parasitic testing
Husbandry
e.g., farmed, free-living,
diet
Coastal
Medications
Temperate
Mountain
Specific state or region
Ambient temperature
912
913
41
Examples
Metabolic
e.g., fasted or non-fasted,
intense exercise, high stress
Cell damage
e.g., traumatic venipuncture,
physical or chemical restraint
Illness
Medications
Lactation
Pregnancy
Hormones or growth
promoters
Enzyme induction
e.g., corticosteroids,
antiseizure drugs
MASTER COPY 10/18/10
914
915
916
Table 2. Recommended procedures for establishing RI based on reference sample size
and distribution
Sample size Data distribution
(innate or transformed)
Not applicable
≥ 120
Gaussian
20 ≤ x <120
Non-Gaussian
Gaussian
10 ≤ x < 20
917
918
919
920
921
922
Statistical method
Nonparametric with 90% CI
Robust with 90% CI
Parametric with 90% CI
Robust with 90% CI (preferred)
Nonparametrica
Robust with 90% CIb
Parametric with 90% CIb
Robust with 90% CI
Not recommended
Non-Gaussian
Not applicable
< 10
Confidence interval (CI)
a
Cannot determine 90% CI with <120 reference sample
b
Include the following information: histogram, mean or median, minimum and maximum
Table 3. Information to include in documenting or publishing RI studies.
Item
Explanation
Demographics of reference
Geographic location
population
Source of reference individuals/samples
Species and breed(s)
Number of individuals
Age and gender distribution
Husbandry (housing, diet, vaccines, parasite control, etc.)
Determinants of health status
Other details if pertinent
Preanalytical methods
Patient preparation
Sample collection method
Sample handling and processing
Time/season of collection if pertinent
Analytical methods
Analyzer
Methodology and reagents
Quality specifications
Quality control reagents and procedures
Method of data analysis
Histogram
Outlier identification
Evaluation of distribution
Definition of interval (e.g. central 95%, 2.5th and 97.5th
fractile limits)
Interval determination (e.g., parametric, nonparametric,
robust)
Confidence interval
42
MASTER COPY 10/18/10
923
43