Full title: An update on FIV and FeLV test performance using a Bayesian statistical
approach
Short title: FIV FeLV Bayesian
M.D.G. Pinches, G. Diesel, C. R. Helps, S. Tasker, K. Egan, T.J. Gruffydd-Jones
Mark D. G. Pinches BVSc MSc MRCVS
Gillian Diesel * BVSc MSc MRCVS
Christopher R. Helps BSc PhD
Séverine Tasker BVSc BSc PhD DSAM DipECVIM-CA MRCVS
Kathy Egan ART (Can)
Tim J Gruffydd-Jones BvetMed PhD DipECVIM-CA MRCVS
University of Bristol, Department of Clinical Veterinary Science, Langford House,
Langford, BS40 5DU, UK
* The Royal Veterinary College, University of London, Hawkshead Lane, Hertfordshire,
AL9 7TA
Keywords: FIV, FeLV, Bayesian analysis
1
Abstract
Background:
Screening tests for feline retroviruses are thought to have high sensitivity and specificity
although previous studies that have evaluated these tests are slightly limited. Novel
statistical approaches have been developed that allow estimation of sensitivity and
specificity in situations where the true state of disease in individual animals cannot be
assured.
Objective:
The purpose of this study was to evaluate the sensitivity and specificity of a variety of
retrovirus tests, including some screening tests, in a population of cats potentially
infected with either FeLV and/or FIV using a Bayesian statistical approach.
Methods
Four hundred and ninety blood samples from cats being evaluated for FIV infection were
tested by two rapid immunomigration tests (Witness Single (WS), Witness Combi (WC);
Synbiotics) and a plate based ELISA (Petcheck, IDEXX) for FIV antibody and by a
newly designed real time PCR assay for FIV provirus. Four hundred and eighty four
blood samples from cats being evaluated for FeLV infection were tested by two rapid
immunomigration tests (Witness Single (WS), Witness Combi (WC); Synbiotics) and a
plate based ELISA (Petcheck, IDEXX) for FeLV antigen and by a FeLV virus isolation
technique. Results were then analysed using a Bayesian statistical method.
Results
For FIV tests, median sensitivity estimates were 0.98 for WS, 0.97 for WC, 0.98 for
ELISA, and 0.92 for PCR. Median specificity estimates were for 0.96 for WS, 0.96 for
2
WC, 0.93 for ELISA, and 0.99 for PCR. For FeLV tests median sensitivity estimates
were 0.97 for WS, 0.97 for WC, 0.98 for ELISA, and 0.91 for VI. Median specificity
estimates were 0.96 for WS, 0.96 for WC, 0.98 for ELISA, and 0.99 for VI.
Conclusions
The use of Bayesian statistical methods overcomes a variety of methodological problems
associated with diagnostic test evaluations including lack of definitive reference test. The
sensitivity and specificity of all six evaluated screening tests is high, however specificity
estimates were slightly lower.
3
Introduction
Feline leukaemia virus (FeLV) and feline immunodeficiency virus (FIV) are important
retroviruses of cats. Accurate diagnosis of these diseases is important both for the
diagnosis of retrovirus related disease and for the identification of infected cats in the
control of both infections. A variety of different types of diagnostic tests are available
including: rapid screening tests for antigen or antibody in plasma; virus isolation (VI) to
detect infectious virus in the plasma; and a number of polymerase chain reaction (PCR)
assays which detect proviral DNA in the blood. Immunoassays of differing types are
most widely used in practice and these tests offer the advantage of speed and
convenience. However, the information available about their performance is limited and
somewhat conflicting 1,2,3. The previous studies that have evaluated FeLV and FIV
immunoassay performance are limited to a degree, as all sample naturally infected
populations, where true disease status is unknown. Further these studies also reference
test only samples with positive results. Such an approach introduces selection bias and
introduces some error into test performance calculations in particular estimates of test
specificity 4. Also, the choice of reference test in retrovirus diagnosis is not without its
difficulties, as there are questions as to which tests can be considered suitable as a
reference test for infections in which virus integrates into the host genome. Virus
isolation for example, the usual reference test for FeLV, has drawbacks as although this
test carries a high specificity its sensitivity is potentially reduced in some situations
5,6,7,8,9,10
. Often therefore, this makes it impossible to unambiguously state the disease
status of tested animals 11. Thus misclassification of samples as true positive or true
negative by the reference tests inevitably leads to errors in the sensitivity and specificity
4
estimations of the evaluated tests 12,13 .
However, there exist a variety of alternate statistical approaches which have been
developed that allow estimation of test sensitivity and specificity when no definitive
reference test is available 12,13,14 . One such approach is based upon Bayes’s theorem,
where error probabilities (prior distributions) based upon previous knowledge of the
evaluated tests, reference test and prevalence are introduced into the analysis. Using new
data (likelihood) estimates of test sensitivity, test specificity and prevalence are updated
(posterior distributions) using a Markov-chain Monte Carlo simulation. In essence this
allows estimates of tests sensitivity and specificity in situations where analysis by
traditional methods would have led to error 4,13,14 .
In this study, Bayesian analysis, a “non gold standard” based statistical approach to the
test evaluation, is applied to the evaluation of a variety of tests for FeLV and FIV.
5
Materials and methods
The samples used in this study were consecutive diagnostic samples submitted to
Langford Veterinary Diagnostics Laboratories by veterinarians for FIV and/or FeLV
testing between September 2002 and June 2004. These were whole blood EDTA and/or
heparin samples for which there was sufficient sample volume to perform all tests. Only
samples from animals over 20 weeks of age were included in the study, to avoid
problems related to the persistence of maternally derived antibody. All samples were split
into two aliquots at submission. Plasma was used in the immunoassays whilst whole
blood was used for FeLV Virus isolation (VI) and/or in the provirus FIV polymerase
chain reaction assay (FIV PCR).
FIV
The FIV immunoassays that were evaluated were two rapid immunomigration (RI) tests,
Witness single (WS) FIV (Witness; Synbiotics, distributed by Woodley
Equipment, Horwich, UK), Witness combi (WC) FIV/FeLV (Witness; Synbiotics) and a
plate based FIV ELISA test (PetChek FIV, IDEXX Laboratories, Wőrrstadt, Germany).
A newly developed real-time provirus FIV polymerase chain reaction (FIV PCR) test was
also used.
Four hundred and ninety samples were tested at submission by both RI tests and the
ELISA according to the manufacturers’ instructions. Positive results were recorded when
6
the intensity of colour change by visual assessment equalled or exceeded that of the
positive control. Samples which gave colour change that was not as intense as the
positive control were read by plate reader and results were recorded as positive if colour
intensity was >50% and equivocal if colour intensity was <50% of that of the positive
control. Negative results were recorded where there was no visible colour change. The RI
tests were read by visual assessment. Occasional equivocal results were also generated.
These included very faint colour change, only ½ an indicator line developing or where
colour change occurred after 10mins.
Samples for FIV PCR were frozen at –20C and tested once a week as a batch. DNA was
extracted from 100l of EDTA blood using the DNeasy Blood Kit (Qiagen, Crawley,
UK) in accordance with the manufacturer's instructions. Real-time PCR for FIV was
performed using an iCycler IQ (Bio-Rad Laboratories Ltd, Hemel Hempstead, United
Kingdom) with primers and probe designed to a consensus region of the GAG gene from
Clade A viral isolates. Previously designed primers and probes to the feline 28S rDNA
gene were also included in the PCR to act as an internal control 15. The PCR reaction
consisted of 12.5l of Hotstartaq Master mix (Qiagen); 200nM FIV forward and reverse
primers (Invitrogen, Paisley, Scotland); 100nM FIV Taqman probe (Cruachem Ltd,
Glasgow, Scotland.)(see Table 1 for primer and probe sequences); plus 200nM 28S
rDNA forward and reverse primers and 200nM 28S rDNA Taqman probe; a final MgCl2
concentration of 4.5mM; 5l genomic DNA and water to 25l. After an initial incubation
at 95C for 15 min, 50 cycles of 95C for 10 sec and 60C for 30 sec were carried out.
Fluorescence was detected at 530nm and 620nm at each annealing step (60C). Threshold
7
values were then calculated using the iCycler software ver3.0. Samples were deemed FIV
positive by the real-time PCR if their relative fluorescence exceeded that of the threshold
value (set at 100 relative fluorescence units).
8
Table 1
FIV 995 forward
5’- TTAAGCCAGAAAGTACCCTAGAAG-3’
primer
FIV 1133 reverse
5’- AAACACACTGGTCCTGATCC-3’
primer
FIV 1064 Taqman
5’-FAM-
probe
TGCAACTCTTGGCAGAAGCTCTTACA-BHQ13’
9
The reaction efficiency and sensitivity of the FIV PCR assay was determined using a
plasmid construct containing the FIV PCR amplicon. The plasmid concentration was
determined spectrophotometrically and used to calculate the copy number. Serial ten-fold
dilutions (from 4x106 to 4 copies per 5l) of the plasmid were run in triplicate in the FIV
PCR assay. By plotting the threshold cycle value against Log of the starting copy number
a straight line is generated (R2=0.999) and the slope of this line was used to determine the
reaction efficiency. Using this standard curve the FIV PCR assay was shown to have an
efficiency of 100% and could detect all three replicates at 4 copies per PCR reaction.
DNA samples from known FIV positive, experimentally infected cats and non-infected
SPF-derived cats, as well as water samples, were subjected to PCR as positive and
negative controls.
Additional tests used to investigate discrepant samples
Ten samples that were positive by ELISA alone were further tested for FIV antibodies by
a previously described immunofluorescent technique (IFA) 16. Here Crandell-Reese
feline kidney (CRFK) cells chronically infected with FIVGL8 were mixed with 3-fold
more uninfected CRFK cells, seeded into flasks and incubated at 37oC for 3 days, until
syncytia appeared obvious. Then the cells were then harvested, mixed with an equal
number of uninfected CRFK cells and plated on to each well of Teflon-coated
microscope slides. The plates were incubated at 37oC for 10 hours, at which time the
cells were fixed, and the slides subsequently stored, in methanol at -20oC. Before use the
cells were washed in water and dried. Ten-fold dilutions of cat serum from 10-1 to 10-4
10
were made in PBS and 25μl of each dilution was applied to appropriate wells, which
were incubated in a moisture chamber at 37oC for 1.5 hours. The cells were then washed
twice in PBS and once in water, and dried in air. A volume of 25μl of FITC-conjugated
anti-cat IgG was plated onto each well and the cells were incubated as before for a further
hour. The plates were then washed and dried as above and examined in a UV microscope
with a X10 objective. The antibody titre was taken as the reciprocal of the dilution of
serum that showed obvious fluorescence within the cytoplasm of syncytia.
Nine of 15 samples that were positive by all tests but negative by the real-time FIV PCR
were further examined. These nine samples were those in which sufficient volume
remained to perform the analysis. Degenerate primers were designed to flank the realtime PCR product and were used to generate amplicons for automated DNA sequencing.
PCR was performed using a MJ Research PTC-200 DNA engine (GRI, UK). The PCR
reaction consisted of 25l of Hotstartaq Master mix (Qiagen); 400nM FIV forward
(ACYCARGAACARCAAGCAGA) and reverse (TGCTGCAYTTGRTTYACTGG)
flanking primers; a final MgCl2 concentration of 4.5mM; 2l genomic DNA and water to
50l. After an initial incubation at 95C for 15 min, 40 cycles of 95C for 15 sec and
62C for 30 sec and 72C for 60 sec were carried out. After separation on a 1% agarose
gel stained with 0.1g/ml ethidium bromide the 578bp amplicons were purified using
QIAquick gel extraction kit (Qiagen) and submitted for automated fluorescent DNA
sequencing (University of Dundee Sequencing Service, Dundee, Scotland).
11
FeLV
The FeLV immunoassays evaluated were the WS FeLV, WC FIV/FeLV and a plate
based ELISA test (Petcheck; Idexx). FeLV VI was also performed. 484 samples were
tested at submission using both RI and ELISA according to the manufacturers’
instructions. Some results were considered equivocal for reasons as described for the FIV
tests. Samples for VI were frozen at –20C before batch testing weekly using the
methodology described by Jarrett and Ganière (1996)17. Briefly, samples (200 μl EDTA
plasma) were inoculated into one well of a 12-well cluster plate seeded 24 h previously
with 5 x 104 QN10 cells in 1 ml of Dulbecco’s medium with 10 μg/ml of Polybrene, 1%
pyruvate solution and 10% FCS. The cells were incubated at 37°C, and the medium was
replaced after 2 hours. Plates were examined for foci of transformation/degeneration on
day 5 after inoculation. Negative samples were subcultured and incubated for a further
week before being reported as negative.
Additional tests used to investigate discrepant samples
Twenty four samples that were positive by ELISA and/or WS/WC but negative by VI,
were further examined for FeLV provirus by a real-time PCR technique previously
described by Pinches et al (In print) 18. Briefly primers and a probe designed to the U3
region of the long terminal repeat sequences of FeLV subgroups A B and C were used
and primers and a probe to the feline 28S rDNA gene included to act as an internal
control. DNA was extracted from 100μl of EDTA blood using the DNeasy blood Kit
12
(Qiagen, Crawley, UK) in accordance with the manufacturer's instructions. Real-time
PCR was performed using an iCycler IQ (Bio-Rad Laboratories, Hemel Hempstead, UK).
The PCR reaction consisted of 12.5 μl of Hotstartaq Master mix (Qiagen), 100nM FeLV
forward and reverse primers, 100nM FeLV Taqman probe, 200nM 28S rDNA forward
and reverse primers, 200nM 28S rDNA Taqman probe, a final MgCl2 concentration of
6mM, 5μl genomic DNA (approx. 100ng) and water to 25 μl. After an initial incubation
at 95°C for 15 min, 45 cycles of 95°C for 10 sec and 60°C for 30 sec were carried out.
Fluorescence was detected at 620nm and 680nm at each annealing step (60°C).
Statistical analysis
Results were tabulated in Excel (Microsoft, US) and contingency tables were constructed.
WS and WC were evaluated independently to each other. Results from the RI, ELISA
and either the FIV PCR or the FeLV VI were used to calculate estimates of test
performance. All equivocal results and the results of the discrepant sample analysis were
excluded from the estimation of test sensitivities and specificities.
Test sensitivity (Sn) and specificity (Sp) were estimated using a Bayesian model. The
freeware program WinBUGS 1.4 (http://www.mrc-bsu.cam.ac.uk/bugs/welcome.shtml)
was used as the platform for all modelling. As the ELISA and RI tests are based on the
same biological mechanism they were considered to be dependent tests. FIV PCR and
FeLV VI were considered independent of these. Thus the model used was a three test in
one population model, with two tests being conditionally dependent, as described by
13
Branscum et al. (2005) 14. In order to account for the dependence between the ELISA
and RI tests, a covariance parameter is included in the model, this was defined by a
uniform distribution. WinBUGS code for this model was downloaded from
http://www.epi.ucdavis.edu/diagnostictests/AB3tests1popn.htm.
Prior information about the test sensitivities, specificities and disease prevalence were
specified using beta distributions. These were calculated using BetaBuster 1.0 software
using estimates of test performance and disease prevalence provided by Hosie et al
(1989), Griessmayer et al. (2002), and Crawford et al (2005)19,1,20 (Tables 3 and 5).
Markov-chain Monte Carlo simulation was used to estimate the median and 95%
credibility intervals from the posterior distributions. For each analysis, an initial burn in
of 10000 iterations was discarded, and node estimates were based on a further 100000
iterations. Convergence for each model was assessed by simultaneously running five
chains with widely differing starting values.
A sensitivity analysis was run for each model by changing the prior beta distributions for
all test parameters to a uniform (0.5, 1) distribution to ensure that the results were
repeatable. All the node estimates of test sensitivity and specificity fell within + 3% of
their original estimates except for the VI sensitivity which increased by 4%.
Fisher’s exact test was also used to compare the proportion of equivocal results
generated. The null hypothesis being that there was no difference between tests and the
significance level was set at 0.05.
14
Results:
FIV tests
The test results obtained for the 490 samples are summarised in Table 2. For WS, 112
samples tested positive by all tests, for WC, 111 samples tested positive by all tests. For
WS, 338 samples tested negative by all tests, for WC, 336 samples tested negative by all
tests. A number of discrepancies between tests were found, these included 15 samples
that were negative by FIV PCR but positive by all other tests and 10 samples that were
positive by ELISA but negative by all other tests.
The ELISA generated more equivocal results than WS or WC (n=11 ELISA, n=3 WS
and n=5 WC) although this failed to reach significance (p=0.055, p=0.074) when
examined by Fisher’s exact test
Median sensitivity estimates were for WS 0.98, WC 0.97, ELISA 0.98, and PCR 0.92.
Median specificity estimates were for WS 0.96, WC 0.96, ELISA 0.93, and PCR 0.99.
An estimated prevalence within the sampled population was 25%. The medians and 95%
posterior credible intervals calculated from the posterior distributions for sensitivity and
specificity estimates based on these data are shown in Table 3.
Of the ten samples found to be positive by ELISA and negative by all other methods, 9 of
these samples were further examined by IFA, all tested negative. The remaining sample
15
could not be evaluated further as insufficient serum remained.
Further sequencing of the amplified PCR product of 9 of 15 samples that were negative
by real-time PCR but positive by all other methods, showed sequence alterations or major
deletions in the GAG gene that would be expected to prevent primer annealing (data not
shown).
16
Table 2. Test results obtained for FIV from the WS, WC, ELISA and PCR (excluding
equivocal results)
PCR +ve
PCR -ve
WS+ WS- WS WC+ WC- WC WS+ WS- WS WC+ WC- WC
Eq
Eq
Eq
Eq
(n=
(n=
(n=
(n=
3)
5)
3)
5)
ELISA +
112
0
3
111
1
3
15
10
0
15
10
0
ELISA -
0
0
0
0
0
0
1
338
0
1
336
2
ELISA Eq
3
0
0
3
0
0
1
7
0
1
7
0
(n=11)
17
Table 3. Prior estimates, and Median and 95% posterior credible intervals for sensitivity
and specificity of WS, WC, ELISA and PCR FIV tests.
Test
Prior
mode
95% prior Reference
credible
interval
ELISA Sn
0.95
0.770
0.988
– 1
Median
of
posterior
credible
interval
0.98
95%
posterior
credible
interval
0.952
–
0.997
ELISA Sp
0.99
0.881
0.998
– 1
0.93
0.907
–
0.961
WS Sn
0.95
0.770
0.988
– 1
0.98
0.952
–
0.997
WS Sp
0.99
0.881
0.998
– 1
0.96
0.936
–
0.983
WC Sn
WC Sp
PCR Sn
0.95
0.99
0.8
0.770 –
0.988
0.881
0.998
0.563
0.923
Estimates
0.97
extrapolated
from
Griessmeyer
et al. 2002
– Estimates
0.96
extrapolated
from
Griessmeyer
et al. 2002
– 20
0.92
0.939
-
0.994
0.9360.983
0.850
–
0.970
PCR Sp
0.98
0.944
0.993
– 20
0.99
0.988
–
0.999
18
Prevalence
0.15
0.062
0.329
– 19
0.25
0.21-0.29
19
FeLV Tests
The test results obtained from the 484 samples are shown in Table 4. For WS, 64 samples
tested positive by all tests, for WC, 61 samples tested positive by all tests. For WS, 393
samples tested negative by all tests, for WC, 396 samples tested negative by all tests. A
number of discrepancies between tests were found, most of these were discrepancies
between either WS, WC or ELISA and VI. These included 14 ELISA positive samples
that were VI negative, 19 WS positive samples that were VI negative and 17 WC positive
samples that were VI negative.
The ELISA generated significantly less equivocal results than both WS and WC (ELISA
n=5, WS n=16, WC n=18) (p=0.007 & p=0.002 respectively).
Median sensitivity estimates were for WS 0.97, WC 0.97, ELISA 0.98, and VI 0.91.
Mean specificity estimates were for WS 0.96, WC 0.96, ELISA 0.98, and VI 0.99. An
estimated prevalence within the sampled population was 15%. Calculated posterior
distributions for sensitivity and specificity estimates based on these data are shown in
Table 5.
Twenty four samples that were discordant between VI and WS, WC and or ELISA were
further tested by FeLV PCR. Of the 14 ELISA positive samples that were discordant to
VI (VI negative), 12 were positive by PCR. Of the 19 WS positive samples that were
discordant to VI, 7 were positive by PCR. Of the 17 WC positive samples that were
discordant to VI, 10 were positive by PCR.
20
Table 4. Test results obtained for FeLV from WS, WC, ELISA and VI. (excluding
equivocal results)
VI +ve
WS+ WS-
ELISA
WS
VI -ve
WC+ WC-
WC
WS+ WS-
WS
WC+ WC-
WC
Eq
Eq
Eq
Eq
(n=16)
(n=18)
(n=16)
(n=18)
64
0
0
61
0
3
10
0
4
9
0
5
0
2
0
0
2
0
8
393
9
7
396
7
0
0
0
0
0
0
1
1
3
1
1
3
+
ELISA
ELISA
Eq
(n=5)
21
Table 5. Prior estimates, and Median and 95% posterior credible intervals for sensitivity
and specificity of WS, WC, ELISA and PCR FeLV tests.
Test
Prior
95% prior
credible
interval
Reference
1
Median of
posterior
credible
interval
0.98
95%
posterior
credible
interval
0.938 -0.998
ELISA Sn
0.99
0.881 –
0.998
ELISA Sp
0.92
0.776 –
0.973
1
0.98
0.960 –
0.993
WS Sn
0.97
0.885 –
0.992
1
0.97
0.938 -0.994
WS Sp
0.92
0.776 –
0.973
1
0.96
0.935 -0.976
WC Sn
0.97
0.885 –
0.992
0.97
0.937 –
WC Sp
0.92
0.776 –
0.973
VI Sn
0.9
0.780 –
0.957
Estimates
extrapolated
from
Griessmeyer
et al. 2002
Estimates
extrapolated
from
Griessmeyer
et al. 2002
Expert
opinion
0.975 –
1.000
0.048 –
0.276
VI Sp
1.00
Prevalence
0.12
0.994
0.96
0.939 -0.979
0.92
0.845 –
0.966
Expert
opinion
0.999
0.988 –
0.999
19
0.25
0.21-0.29
22
Discussion
This is the first study to apply Bayesian modelling to the evaluation of FIV and FeLV
screening tests. This technique is a powerful statistical approach that is implemented
using an iterative Markov-chain Monte Carlo simulation method and Gibbs sampling run
on WinBUGS software. While this technique is well established in veterinary
epidemiology, it has only been applied to a limited number of veterinary diagnostic test
evaluations, although this number is increasing 21. The model employed by the current
study was designed by Branscum et al. (2005) 14 and allows evaluation of two dependent
tests (the ELISA and WS/WC) and one test that is independent of these (PCR or VI)
using sampled data from one population. However a variety of different models exist for
different situations.
A key feature of Bayesian modelling is the ability to introduce a level of uncertainty into
the performance of all the tests and the disease prevalence of the tested population. Such
models are then able to estimate test sensitivity and specificity in situations where the
true disease status of individual animals is unknown 13, 14. Such situations arise from the
use of imperfect reference tests or in diseases where the stage of disease influences test
sensitivity 4. In the current study, the nature of retrovirus infection and the limitations of
the available diagnostic tests can both confound the definitive classification of disease
status in some individuals. These uncertainties would bias classic ‘gold standard’
statistical approaches to the test evaluation; however the Bayesian approach is capable of
performing test evaluations on all the tests involved, including the reference tests in these
conditions.
23
The results of the sensitivity analyses showed that most results fell within 3% of the
original model results despite having substituted the partially informative uniform (0.5, 1)
distribution for the prior estimates provided by Hosie, Griessmeyer et al., and Crawford
et al.19,1,20. This suggests that these chosen prior distributions were appropriate. However,
when the distribution was used as the prior for the sensitivity of the VI in the FeLV
model it was found that the median of the posterior distribution for this parameter
increased by 4%. This indicates that the prior for this parameter used in the model is
having a strong effect on the node estimate. This may have been caused by using an over
cautious estimate of test sensitivity for the prior distribution and therefore the actual
estimate of test sensitivity for FeLV VI may be somewhat higher than that reported here.
The results from the Bayesian analysis demonstrates that the current performance of the
six named FIV and FeLV screening tests remains very high. All screening tests achieved
sensitivity estimates of greater than 0.97. Such performance is similar to previous studies
1,2,3,22,23
. However, specificity estimates were slightly lower for some tests than
previously reported in the above studies which may be due to differences in study design
(reference testing only positive samples) in the previous studies. Such an approach misses
false negative results and positively biases evaluated test specificity.
All tests generated some equivocal results. For FIV tests WS (n=3) and WC (n=5) gave
fewer equivocal results than ELISA (n=11). For FeLV tests, ELISA (n=5) generated
significantly fewer equivocal results than WS (n=16) and WC (n=18); this finding may
24
be related to the confirmatory procedure for positive samples that is incorporated in the
FeLV ELISA method. This procedure for clarifying the status of positive samples, may
allow some equivocal results (which initially gave weak colour change) to be clearly
defined as positive or negative. In contrast to the FeLV ELISA, FeLV WS and WC tests
have no facility for positive test confirmation and results are evaluated only by the
subjective assessment for the presence or absence of a pink line on the test kit. Weak
colour change in the tests is therefore more likely to result in an interpretation of
equivocal. Exclusion of the equivocal results from the Bayesian models could result in
biased estimates of the sensitivity and specificity, however due to the small proportion of
these results the impact is likely to be small.
The FIV ELISA was found to have a lower median specificity estimate (0.93) than both
FIV WS (0.96) and FIV WC (0.96), although there is some overlap between the posterior
distributions that suggest that this finding may not be significant. However, the estimate
of FIV ELISA specificity is lower than that given in the most recently published study by
Griessmayer et al. (2002) 1 which estimated FIV ELISA specificity to be 0.99. The lower
test specificity found in this study appears to be associated with a number of ELISA
positive results that were found to be negative by other test methods. These 10 discrepant
positive results represent 7.1% of all positive results given by the ELISA. They are
considered false positive on the basis that other forms of testing, including FIV PCR and
FIV IFA, showed no evidence for FIV infection. There are a variety of possible
explanations for these discrepancies. The most frequently suggested is that the sample
contained antibodies that cross-react with the ELISA test 23,24. Although the nature of
these antibodies has not yet been determined, it has been suggested that such antibodies
25
could be induced following feline foamy virus (FFV) infection. However, serological
studies of cats with discrepant FIV results have shown that a number of cats have no
detectable antibodies against FFV (A. German, personal communication). Furthermore,
following experimental infection of SPF cats with FFV none developed antibodies that
cross-reacted with the FIV ELISA (A. German, personal communication). Another
suggested explanation is a rise in non-specific reactivity to ELISA components following
vaccination, in particularly with killed virus vaccines 23.
The FIV PCR described here, had an estimated sensitivity of 0.92 and specificity of 0.99
by Bayesian analysis. The estimate of sensitivity is lower than that of the screening tests
also evaluated in this study. These results are however similar to those reported by
Crawford et al. (2005) 20 who evaluated a similar real time PCR assay. There are a variety
of issues which may reduce PCR sensitivity. One important factor in FIV PCR is the
wide heterogeneity in FIV sequences that exists between different isolates. This
heterogeneity interferes with primer/probe annealing thus affecting the PCR reaction
20,25,26,27
. In the current study 15 of 490 samples were identified that were positive by both
ELISA and Witness yet negative by PCR. Nine of these 15 samples had sequencing
performed on a PCR product which flanked the real-time PCR amplicon; all showed
sequence alterations or major deletions that were sufficient to prevent the real-time PCR
primers from annealing. These findings emphasise the potential lack of sensitivity within
FIV PCR assays due to FIV sequence heterogeneity.
The FeLV VI technique used in the current study was found to have a sensitivity of 0.91
26
and specificity of 0.99 by the Bayesian model. The low sensitivity is due to well
recognised differences between ELISA which detects circulating antigen and VI which
detects circulating whole virus. Such differences between results have been termed
discordant 5, 6, 9, 10. The present study found 25% (n=14) of ELISA results, 30% (n=19) of
WS and 30% (n=17) of WC results to be discordant compared to VI. There are a variety
of possible explanations, both biological and methodological, that have been proposed for
such discordance. Possible biological explanations include; expression of cross reacting
antigens from endogenous FeLV genes 6; localised infection with selective release of
antigen but not virus, as has been shown for the mammary gland 8; or recovery from
viraemia but failure to clear antigenaemia in cats with transient infections after the
immune response has been activated. The latter explanation is unlikely to explain the
discrepancies in most cases, as given the sampling methods used in this study, it is
improbable that cats would be tested during this short period. Methodological concerns
include cross reactivity in the ELISA with some other antigen 28, or a lack of sensitivity
of the VI technique due to limited viability of virus during transport; although Jarrett et al
(1982)6 suggest that virus deterioration is not a concern for FeLV VI.
Studies that have followed the subsequent outcome in these discordant cases have found
that the majority (75%) of cats become ELISA negative given time 29. However, a small
proportion of cats have been shown to eventually develop clear evidence of infection, and
it is postulated that this is due to a reduction in immunological viral restraint 8. The
current study further examined the nature of the discordant results by evaluation of each
discordant sample (positive for antigen but negative by VI) with a real time PCR for
27
FeLV provirus, a method that has been shown to be highly sensitive 18. For ELISA most
(86%) were also positive by PCR. However for WS and WC only 36% and 59%
respectively were also positive by PCR. These findings suggest that whilst discordant
ELISA results may reflect true FeLV infection and low VI sensitivity, around 50% of
discordant Witness results may be false positive.
We conclude that the performance of all six evaluated screening tests is good. However
the slightly reduced specificities reported here for FIV ELISA and FeLV Witness
reemphasises the advice by the advisory panels on feline retrovirus testing 30 that
positive results be confirmed by other forms of testing, especially in asymptomatic cats,
or when testing a low prevalence population.
Furthermore the use of a statistical method that overcomes a variety of methodological
problems associated with diagnostic test evaluations in real world situations is
demonstrated here. It is hoped that such techniques become more widely adopted in
future studies.
Acknowledgements
M Pinches is sponsored by Axiom Veterinary Laboratories. This research was funded in
part by Synbiotics.
28
References
1. Griessmayr P, Greene CE, Egberink H, et al. Comparison of different new tests for
feline immunodeficiency virus and feline leukaemia virus infection [abstract]. In:
International Feline Retrovirus Research Symposium. 2002: Dec. 2-5
2. Hartmann K, Werner RM, Egberink H, et al. Comparison of six in-house tests for the
rapid diagnosis of feline immunodeficiency and feline leukaemia virus infections. Vet
Rec. 2001;149(11):317-20
3. Robinson A, DeCann K, Aitken E, et al. Comparison of a rapid immunomigration test
and ELISA for FIV antibody and FeLV antigen testing in cats. Vet Rec.1998;
142(18):491-2
4. Greiner M, Gardner IA. Epidemiologic issues in the validation of veterinary diagnostic
tests. Prev Vet Med. 2000;45(1-2):3-22
5. Jarrett O, Golder MC, Weijer K. Comparison of three methods of feline leukaemia
virus diagnosis. Vet Rec.1982;110:325-328
6. Jarrett O, Golder MC, Stewart MF. Detection of transient and persistent feline
leukaemia virus infections. Vet Rec.1982;110(10):225-8
7. Lutz H, Pedersen NC, Theilen GH. Course of feline leukemia virus infection and its
29
detection by enzyme-linked immunosorbent assay and monoclonal antibodies. Am J Vet
Res.1983;44(11): 2054-2059
8. Pacitti AM, Jarrett O, Hay D. Transmission of feline leukaemia virus in the milk of a
non viraemic cat. Vet Rec.1986;118:381
9. Weijer K, van Herwijnen R. Detection of FeLV antigen. Vet Rec.1995;137(5):127
10. Jarrett O, Pacitti AM, Hosie MJ, et al Comparison of diagnostic methods for feline
leukemia virus and feline immunodeficiency virus. J Am Vet Med
Assoc.1991;199(10):1362-4
11. Tyler JW, Cullor JS. Titers, tests, and truisms: rational interpretation of diagnostic
serologic testing. J Am Vet Med Assoc.1989;194(11):1550-8.
12. Pouillot R, Gerbier G, Gardner IA. Tags' a program for the evaluation of test accuracy
in the absence of a gold standard. Prev Vet Med. 2002;53(1-2):67-81
13. Enøe C, Georgiadis MP, Johnson WO. Estimation of sensitivity and specificity of
diagnostic tests and disease prevalence when the true disease state is unknown. Prev Vet
Med.2000;45(1-2):61-81
30
14. Branscum AJ, Gardner IA, Johnson WO.Estimation of diagnostic-test sensitivity and
specificity through Bayesian modelling. Prev Vet Med.2005;68(2-4):145-63.
15. Dean R, Harley R, Helps C, et al. Use of Quantitative Real-Time PCR to Monitor the
Response of Chlamydophila felis Infection to Doxycycline Treatment
J Clin Microbiol.2005; 43(4):1858–1864
16.Hosie MJ, Jarrett O. Serological responses of cats to feline immunodeficiency virus.
AIDS 1990;4:215-20
17. Jarrett O, Ganière JP. Comparative studies of the efficacy of a recombinant feline
leukaemia virus vaccine. Vet Rec.1996;138(1):7-11
18. Pinches MDG, Helps CR, Gruffydd-Jones T, et al. (In print) Diagnosis of feline
leukaemia virus infection by quantitative real-time polymerase chain reaction. J Feline
Med Surg.
19 Hosie MJ, Robertson C, Jarrett O. Prevalence of feline leukaemia virus and
antibodies to feline immunodeficiency virus in cats in the United Kingdom.
Vet Rec. 1989 Sep 9;125(11):293-7.
20 Crawford PC, Slater MR, Levy JK. Accuracy of polymerase chain reaction assays for
diagnosis of feline immunodeficiency virus infection in cats. J Am Vet Med Assoc
31
2005;226(9):1503-7
21 Gardner IA. The utility of Bayes’ theorem and Bayesian inference in veterinary
clinical practice and research. Aust. Vet. J.2002;80:758–761.
22 Hawks DM, Legendre AL, Rohrbach BW. Comparison of four test kits for feline
leukaemia virus antigen. J Am Vet Med Assoc.1991;199:1373-1377
23 Barr MC, Pough MB, Jacobson RH, et al. Comparison and interpretation of diagnostic
tests for feline immunodeficiency virus infection. J Am Vet Med
Assoc.1991;199(10):1377-81
24 Pedersen NC, Barlough JE. Clinical overview of Feline Immunodeficiency virus J Am
Vet Med Assoc.1991;199(10):1298-1305
25 Klein D, Janda P, Steinborn R, et al. Proviral load determination of different feline
immunodeficiency virus isolates using real-time polymerase chain reaction: influence of
mismatches on quantification. Electrophoresis.1999;20(2):291-9
26 Leutenegger CM, Klein D, Hofmann-Lehmann R, et al. Rapid feline
immunodeficiency virus provirus quantitation by polymerase chain reaction using the
TaqMan fluorogenic real-time detection system. J Virol Methods.1999;78(1-2):105-116
32
27 Bienzle D, Reggeti F, Wen X, et al. The variability of serological and molecular
diagnostics for feline immunodeficiency virus infection. Can Vet J.2004;9:753-7
28 Lopez NA, Jacobson RH, Scarlett JM, et al. Sensitivity and specificity of blood test
kits for feline leukemia virus antigen. J Am Vet Med Assoc.1989;195(6):747-51
29 Hardy WD Jr, Zuckermann EE. Ten year study comparing enzyme-linked
immunosorbent assays with the immunofluorescent antibody test for detection of feline
leukaemia virus infection in cats. J Am Vet Med Assoc.1991;199:1365-1373
30 Richards J.. 2001 report of the American association of feline practitioners and
academy of feline medicine advisory panel on feline retrovirus testing and management.
J Feline Med Surg.2003;5: 3-10
33