Quality Assurance For Pneumococcal Assays in Europe
Background and Aims
The European Primary Immunodeficiencies (PIDs) Consensus Conference in Langen (2006) outlined the need
for quality assurance for assays to measure specific antibodies to common pathogens and immunization
antigens. In order to make a diagnosis of primary antibody failure, it is important to show that a patient is
unable to make antibodies to pathogens; this may be demonstrated using either common exposure antigens
or those used for immunization. Streptococcus pneumoniae, a common gram positive bacterium, is one such
pathogen that is estimated by the WHO to be responsible for over one million acute deaths worldwide. This
includes patients with primary immune deficiencies, who are particularly susceptible to infections caused by
encapsulated bacteria.
Assays for antibody responses to S. pneumoniae are of two types: those that measure total pneumococcal
IgG antibody to Pneumovax® by enzyme linked immunosorbent assays (ELISA), and those that measure
serotype specific antibodies also by ELISA. The current European Society for Immunodeficiencies (ESID)
guidelines (de Vries 2005) recommend the use of the unconjugated pneumococcal vaccine (Pneumovax®) for
diagnosing PIDs by monitoring IgG responses to booster immunisations.
In recent studies various pneumococcal serotype specific ELISA assays have been used for the same
purposes. Akikusa and Kemp found that a low response to serotype 3, post Pneumovax ® vaccination, in
combination with a low titre to serotype 4/6B, was suggestive of a severe immune deficiency state in children
being investigated following recurrent upper or lower respiratory tract infections. They found a significant
correlation between a low IgG response to serotype 3 with pneumonia.
Boyle et al. used specific IgG response of children with recurrent respiratory infections to 12 serotypes (1, 3,
4, 5, 6B, 7, 9V, 14, 15, 18C, 19F, 23F) following Pneumovax® vaccination in order to diagnose specific
antibody deficiency (SAD). They deemed an adequate response to any one serotype was an IgG value of 1.3
mg/l, or a fourfold increase in the IgG concentration. An unsatisfactory response to more than 50% of the
serotypes tested was used to define a SAD. They also found that failure to respond to serotypes 4, 9V, 15 or
23F significantly increased the risk of SAD, suggesting that these serotypes might be sufficient for assessing
specific antibody defects.
In order to assess the variability of these antibody measurements in Europe, prior to re-examining diagnostic
criteria, a pilot study was undertaken. The aims were to develop quality assurance for pneumococcal assays,
particularly for new laboratories in Europe, and to lay the foundation for a comparison of available assays for
test immunization in the diagnosis of PIDS, particularly between the Pneumovax® total ELISA assay and the
serotype specific assays. Binding Site UK kindly provided total pneumococcal IgG ELISA kits to all
participating laboratories, facilitating comparisons of performance between assays.
METHODS
Serotype ELISA assays
In-house ELISA assays for the detection of human immunoglobulin G (IgG) to 7 common pneumococcal
serotypes (4, 6B, 9V, 14, 18C, 19F and 23F) were developed. Optimum antigen coating concentration for
each serotype (antigens provided by LGC Promochem) was determined, following the Standard Operating
Procedure provided by Nahm and Goldblatt (2002). This was also used to establish the optimum working
dilution for the enzyme labeled secondary antibody (anti IgG phosphatase BioSource Ltd.) in each serotype
assay.
1
Validation
This was undertaken using the 12 pneumococcal WHO calibration sera samples. These are supplied with
values for the expected specific pneumococcal serotype IgG concentrations and are obtained from the
National Institute for Biological Standards (Colindale, UK). They were tested for specific pneumococcal IgG
using the Standard Operating Procedure (Nahm, Goldblatt 2002) and used to validate the developed serotype
assays for routine and the QA purposes. The concentrations of serotype specific IgG were determined in 16
control sera samples along with 16 known positive sera samples.
The in-house Pneumovax ELISA assay and the ELISA kit provided by The Binding Site UK were already
validated.
Samples
Random serum samples from 16 individuals were tested for total anti-pneumococcal IgG using the in-house
Pneumovax® ELISA assay & ELISA kit provided by The Binding Site UK in addition to the serotype specific
assays.
Three serum samples were selected for QA distribution on the basis of having high, medium and low titres of
anti-pneumococcal IgG. These were distributed to 16 collaborating centres (table 1) in Europe and were
tested using both their established pneumococcal assay (whether for total pneumococcal or serotype specific
antibodies) and a Binding Site kit.
Contact
Location
Assay
1
Mr
Daniel
Harrison,
Dr
Helen Chapel
Oxford - UK
2
Dr Vojtech Thon
3
Dr Rasa Duobiene
4
Dr
Michael
Schlesier
Brno - Czech
Rep
Vilnius
Lithuania
Freiburg
Germany
4, 6B, 9V, 14, 18C, 19F,
23F
serotypes.
Pneumovax total IgG
ELISA and Binding Site
kit.
Binding Site Kit
5
Dr
Ulrich
Baumann
Pr Arnaldo Caruso
Dr Andy Gennery
6
7
8
9
10
11
Mrs Lynn Day
Dr
Françoise
Mascart,
Julie
Smet
Dr Anna Sediva,
Dr Ales Janda
Dr Ray Borrow
Hannover
Germany
Brescia - Italy
Newcastle
UK
Slough - UK
Brussels
Belgium
Prague - Czech
Rep
Manchester UK
12
Mrs.Rajee
Goswami,
Dr.Tony Williams
SouthamptonUK
13
Dr Paul Herbrink
14
Robert K. Hallam,
Delft
Netherlands
Papworth - UK
B
Site
received
Yes
Yes
No
4,5,6B,7F,9V,14,18C,19F,
23F
serotypes
and
Pneumovax total IgG
ELISA
Pneumovax total IgG
ELISA
Yes
Yes
No
No
Binding Site Kit
1,7F,14,19,23F serotypes
Yes
Binding Site Kit
Yes
1, 3, 4, 5, 6B, 7F, 9V, 14,
18C, 19A, 19F, 23F
serotypes
(mulitplex
assay)
4,6B,9V,14,18C,19F,23F
serotypes,
Pneumovax
total IgG ELISA and
Binding Site Kit
1,3,4,5,9N,23F
serotypes.
No
Yes
No
No
2
15
16
17
Mrs
Kate
Campbell
Fiona Alcock
Xavier
Bossuyt,
Margaretha
Charlier
Peter Ciznar, MD
Birmingham
UK
Leuven
Belgium
-
Binding Site Kit
Yes
-
3,4,9N,18F,19F
serotypes
No
Bratislava
- Binding Site Kit
No
Slovakia
Table 1. Table of participants in the pilot study. Those contacts who did not receive the
Binding Site assay failed to confirm the delivery address.
Protective Level
Although normal ranges for age have been determined for the total pneumococcal assay (Windebank et al
1987), protection levels of these antibodies have not been established in large groups of patients, only in
those with lymphoid malignancies from whom a value of 30 U/ml was derived (Griffiths et al 1992). For
serotype specific assays, the value of 0.2 mg/l was used as the putative protection level, based on data from
Black et al. and Henckaerts et al. Black et al. demonstrated that more than 95% of their pneumococcal
conjugate recipients developed >= 0.15 mg/l of serotype specific IgG after a third dose of vaccine compared
with the non-vaccinated state (Figure 1).
The 22F serotype was used as an absorbent, to block non-specific responses and cross-reactive antibodies.
This slightly decreases the antibody concentration observed and will alter the protection level as
demonstrated in Henckaerts et al. They showed that levels of serotype specific IgG post vaccination were
greater than 0.35 mg/l in 78.9% of individuals. This percentage corresponded to a threshold of 0.20 mg/l
compared with the non-22F absorbed sera, seen in Fig 2. Therefore, in the presence of absorption with 22F
in a routine assay, 0.2 mg/l is the correct protective level. Since the WHO recommended assay includes
absorption with 22F material, this level was used.
Fig 1. Cumulative distribution curves of post-Dose 3 antibody
concentrations. (from Black et al)
Fig.2.
Shows
cumulative
distribution
of the 30 post-vaccination samples for the
non-22F ELISA and the 22F inhibition ELISA.
Curves were calculated from antibodies to
serotypes 4, 6B, 9V, 14, 18C, 19F, and 23F.
curves
Statistical Analysis
Serotype specific IgG results were compared to results obtained from the existing total Pneumovax ® ELISA
assay using Kappa coefficients (Dr Martin Lee, University of California at Los Angeles).
3
RESULTS
Validation of serotype specific ELISA Assays for comparison with other laboratories
The results shown in Figure 3 suggest that the assays in our laboratory are detecting similar concentrations
of specific IgG with the calibration sera. However some assays appear to be performing better than others;
for example, the assay using 9V material gives less satisfactory validation. However, in many instances the
sensitivity of the assays appears better for the observed values than for the expected.
4
6B
16.00
35.00
14.00
30.00
12.00
25.00
Exp
8.00
Obs
[IgG] mg/l
[IgG] mg/l
10.00
20.00
Exp
Obs
15.00
6.00
10.00
4.00
5.00
2.00
0.00
0.00
730
734
738
742
744
748
752
754
760
764
768
770
730
734
738
742
744
Serum number
748
752
754
760
764
768
770
Serum number
9V
14
18.00
350
16.00
300
14.00
250
10.00
Exp
Obs
8.00
[IgG] mg/l
[IgG] mg/l
12.00
200
Exp
Obs
150
6.00
100
4.00
50
2.00
0.00
0
730
734
738
742
744
748
752
754
760
764
768
770
730
734
738
742
744
Serum number
748
752
754
760
764
768
770
Serum number
19F
18C
70.00
20.00
18.00
60.00
16.00
50.00
[IgG] mg/l
12.00
Exp
10.00
Obs
[IgG] mg/l
14.00
40.00
Exp
Obs
30.00
8.00
6.00
20.00
4.00
10.00
2.00
0.00
0.00
730
734
738
742
744
748
752
754
760
764
768
730
770
734
738
742
744
748
752
754
760
764
768
770
Serum number
Serum number
23F
30.00
25.00
[IgG] mg/l
20.00
Exp
15.00
Obs
10.00
5.00
0.00
730
734
738
742
744
748
752
754
760
764
768
770
Serum number
Fig 3. Graphical comparison between the expected WHO IgG concentrations (in red) for 8 calibration sera with the observed IgG
concentrations (in blue) using the developed serotype ELISA assays.
QA of European Serotype ELISA assays
The three selected QA samples gave similar serotype results between laboratories, except for the sample
with the higher titers (Figures 4, 5 and 6). In particular Figure 6 shows good reproducibility between
laboratories for the lower titer sample 3 (except for serotype 14). Figure 4 shows that reproducibility for
sample 1 (highest levels of antibodies) was less good in quantitative terms, though all laboratories gave
results above the protective level. Figure 5 shows that there was good reproducibility for sample 2, but that
for serotypes 14, 19F and 23F was particularly poor. However, despite there being poor reproducibility for
the higher titre sera, there was good agreement in terms of protection throughout.
4
The results for individual laboratory performance for the 7 serotypes are shown alongside. It is evident from
these graphs that although there was agreement in terms of protection, the sensitivity of these serotype
assays varied hugely, explaining the large range of results especially for the higher titre sample.
Table 2 demonstrates that serotype results from most laboratories correspond well in terms of protection
since there was almost always a majority outcome of a positive or negative response. The level of agreement
was not as consistent for the lower titre sample due to the number of serotype concentrations that were
borderline (close to 0.20 mg/l).
Sample 1
Distribution of Immunity Across 7 Pneumococcal Serotypes For Sample 1
6
Specific IgG Concentration in μg/ml
IgG ug/ml
5
4
3
2
1
0
4
6B
9V
14
18C
19F
23F
Lab A
B
C
D
5
4
E
protection
F
G
3
2
1
0
4
Pneumococcal serotype
6B
9V
14
18
19F
23F
Pneumococcal Serotype
Fig 4. and Fig 4a. Range of sample IgG for sample 1 as calculate d by European
Labs. Fig 4a. Distinguishes results between labs.
Sample 2
Distribution of Immunity Across 7 Pneumococcal Serotypes For Sample 2
4
Lab A
B
C
4
Specific IgG Concentration in μg/ml
IgG ug/ml
3
2
1
D
3.5
E
3
protection
F
2.5
G
2
1.5
1
0.5
0
0
4
4
6B
9V
14
18C
19F
6B
9V
23F
14
18
19F
23F
Pneumococcal Serotype
Pneumococcal serotype
Fig 5. and Fig 5a. Range of sample IgG for sample 2 as calculate d by European
Labs. Fig 4a. Distinguishes results between labs.
Sample 3
Distribution of Immunity Across 7 Pneumococcal Serotypes For Sample 3
3
Specific IgG Concentration in μg/ml
IgG ug/ml
3
2
1
0
Lab A
B
C
D
E
protection
F
G
2.5
2
1.5
1
0.5
0
4
6B
9V
14
18C
19F
23F
4
6B
9V
14
18
19F
23F
Pneumococcal Serotype
Pneumococcal serotype
Fig 6. and Fig 6a. Range of sample IgG for sample 3 as calculate d by European
Labs. Fig 4a. Distinguishes results between labs.
5
Serotype
4
6B
9V
14
18C
19F
23F
Sample 1
+ve
-ve
66.7
33.3
100
0
100
0
80
20
100
0
100
0
100
0
Sample 2
+ve
-ve
66.7
33.3
75
25
100
0
100
0
80
20
100
0
100
0
Sample 3
+ve
-ve
0
100
25
75
0
100
100
0
80
20
83.3
16.7
50
50
Overall agreement for
for serotype
77.8
83.3
100
93.3
86.7
94.4
83.3
Table 2. Table shows the level of agreement, as a percentage, between European labs for the comparison samples, in terms of a
positive or negative response to the particular serotypes. The overall agreement highlights the serotype assays which are most likely
to give a unanimous verdict, whether positive or negative. The putative protective value of 0.2mg/l was used to judge a positive or
negative outcome.
Total Pneumococcal ELISA assays
The laboratories also received a Binding Site kit to test alongside their serotype specific assay or their inhouse total pneumococcal assay. The results from the laboratories using this kit are shown in Figure 7.
Binding Site assay
Binding Site assay
200
Total Pneumovax
ELISA
350
300
IgGIgG
response
U/ml
mg/l
IgG mg/l
150
100
50
250
200
150
100
50
0
0
1
2
Sample tested
3
Fig 7. Range of total pneumococcal IgG for the 3 QA samples
calculated by labs using the Binding Site assay.
1
2
3
Sample tested
Fig 8. Range of total pneumococcal IgG for the 3 QA samples
calculated by labs using their own in house Pneumovax® assay.
This shows that overall results from the Binding Site kit correlated well. As with the serotype assays, the
higher titre comparison serum (sample 1) tended to give more varied results whereas the lower titre sera
were closely matched between laboratories. Only 3 laboratories gave results for their in-house
Pneumovax® ELISA method (Figure 8) but since there is no international reference preparation yet for this
type of assay, the results were given in local units and were not convertible to standard unitage for
comparison.
6
Serotype assays
1
0
1
4
2
1
0
Binding
Site ELISA
Fig 8. Diagram showing the frequency of assay usage
for the European labs.
In House
Pneumovax
ELISA
Serotype
4
6B
9V
14
18C
19F
23F
Pneumovax
ELISA
0.24
0.35
0.26
N/A
0.16
N/A
0.60
Table 3. Kappa coefficient correlation between
serotype specific and total pneumococcal ELISA assays.
Comparison of total pneumococcal assay and serotype specific assays
Direct comparison between serotype specific assays and total pneumococcal assays, using all samples found
that a good IgG response to most serotypes generally indicated a high total pneumococcal result. Statistical
analysis of the 16 random samples was carried out using Kappa coefficient correlations, to look for
associations between individual serotype results and the total pneumococcal result. This was achieved by
allocating each serotype result and each total Pneumovax result to a positive or negative status, based on
the putative protection concentration of 0.2 mg/l (Black et al. and Henckaerts et al.) and a putative
protection level of >20 U/ml for our total Pneumovax in house assay. A correlation was found between the
Pneumovax assay and the 23F serotype assay only [Table 3]. For Kappa coefficients, a value of 0.6 is
indicative of a significant result, but more samples are required for confirmation.
Exact comparison of pre and post immunization samples between the total Pneumovax assay and the
serotype assays was not possible as too few samples were available.
Recommendations and Future
The most widely used assay currently is the Binding Site total pneumococcal IgG kit, though some
laboratories are using pneumococcal specific antibodies. This study demonstrates that once assays are
validated, a reference preparation is essential to cover all the various serotypes tested in order to convert
the results in the three different types of methodology. Such a reference preparation should be available
throughout Europe.
Antibody responses to the Prevenar vaccine, in order to look for a ‘protective’ response, form the basis of
diagnostic assays in children. In regard to test immunization, the definition of an “adequate” response
as an IgG value of 1.3 mg/l or a fourfold increase in the IgG concentration for a particular serotype is
arbitrary (Boyle et al.). The question of how many such positive responses are required in the serotype
specific assays to exclude a diagnosis of immune deficiency remains uncertain, though the criteria quoted
above provide the best evidence so far. One approach is to select suitable serotypes, as in other studies, that
give the best indication of response and in terms of PID diagnosis (Kemp et al.). From the present study, the
9V and 23F serotypes look to be good candidates, based on the high level of agreement observed between
laboratories (Table 2) and the high statistical correlation with the total pneumococcal ELISA assay (Table 3).
7
Standardisation of both types of assays used is also important. Using 22F as an absorbent to block
unspecific responses and cross-reactive antibodies, as recommended by the WHO, is appropriate.
Acknowledgements:
We are grateful to Dr Martin Lee, University of California at Los Angeles for statistical analysis and to The
Binding Site Ltd for their generous offer of kits for all participants. This study was only possible due to
funding from the EU 6th Framework policy EURO-POLICY-PID [SP23-CT-2005-006411]. We are indebted to Dr
William Egner at UKNEQAS for agreeing to take on this scheme once the pilot phase was completed.
References
de Vries, E. 2005. Patient-centred screening for primary immunodeficiency: a multi-stage diagnostic protocol
designed for non-immunologists. Clinical and Experimental Immunology. 145: 204-214.
Nahm, M. H., and D. Goldblatt. 2002. Training manual for Enzyme linked immunosorbent assay for the
quantitation of Streptococcus pneumoniae serotype specific IgG (Pn PS ELISA). www.vaccine.uab.edu
Black, S., H. Shinefield, B. Fireman, E. Lewis, P. Ray, J. Hansen, L. Elvin, K. M. Ensor, J. Hackell,
G. Siber, F. Malinoski, D. Madore, I. Chang, R. Kohberger, W. Watson, R. Austrian and K.
Edwards. 2000. Efficacy, safety and immunogenicity of heptavalent pneumococcal conjugate vaccine in
children. Pediatr. Infect. Dis. J. 19:187-95
Henckaerts, I., D. Goldblatt, L. Ashton, and J. Poolman. 2006. Critical differences between
pneumococcal polysaccharide enzyme-linked immunosorbent assays with and without 22F inhibition at low
antibody concentrations in pediatric sera. Clinical and Vaccine Immunology. Mar:356-360.
Akikusa, J. D., and A. S. Kemp. 2001. Clinical correlates of response to pneumococcal immunization. J.
Paediatr. Child Health. 37:382-387
Boyle, R. J., C. Le, A. Balloch, and M. L-K. Tang. 2006. The clinical syndrome of specific antibody
deficiency in children. Clinical and Experimental Immunology. 146:486-492
Windebank, KP., Faux, J., Chapel H 1987 ELISA determination of IgG antibodies to pneumococcal
capsular polysaccharides in a group of children. J Immunolog Methods 104: 143-148
Hargreaves, RM., Lea, JR., Griffiths, H, Faux, JA., Holt, JM., Reid,C., Bunch, C., Lee, M., Chapel, H.
1995 Immunological factors and risk of infection in plateau phase myeloma. J. Clin. Path. 48: 260-266
Griffiths, H. Lea, J. Bunch, C. Lee, M. Chapel H. 1992
lymphocytic leukaemia (CLL) Clin Exp Immunol. 89, 374-377
Predictors of infection in chronic
Daniel Harrison
Helen Chapel
16/03/2008
8