Reproducibility for C4d
immunohistochemistry in renal allografts –
results from the Banff Trial
Banff Initiative for Quality Assurance in Transplantation (BIFQUIT)
Samantha Chan, Jessie Climenhaga, Parmjeet Randhawa, Heinz
Regele, Yaël Kushner, Robert Colvin,
and Michael Mengel
University of Alberta, Edmonton, Canada
University of Pittsburgh, Pittsburgh, USA
Massachusetts General Hospital, Boston, USA
Detection of C4d is crucial for diagnosing antibody
mediated rejection: reproducibility studies are limited
C4d in paraffin
sections: Results
can directly
influence patient
care
Design of the C4d BIFQUIT trial
Collect cases (n=19) for
analytical spectrum
(i.e. Negative, mildly to
strongly positive)
C4d
Sent slides requested
by participating
institutions
Tissue microarrays
(TMA)
Centers contributing samples:
Volker Nickeleit, Chapel Hill, USA
Verena Bröcker, Hannover, Germany
Parmjeet Randhawa, Pittsburgh, USA
Bernard Collins, Boston, USA
Heinz Regele, Vienna, Austria
Michael Mengel, Edmonton, Canada
Cinthia Beskow-Drachenberg, Maryland, USA
Surya Seshan, New York, USA
Combine with retrieved
online survey data
Data Analysis
ALL TMA slides for
score by review panel:
the ‘best stain returned
was identified per
consensus as the
‘reference case’
Participants/
observers stain slides
and evaluate
Complete online or paper
survey on staining methods
& C4d scoring
Enter survey data
into data base
Mail survey + stained
slides back to
organizing centre
C4d
Collect and organize
slides (e.g. by
institution/
participant)
Reproducibility of immunohistochemical stains
• Inter-institutional variability =
 staining/laboratory procedure + subjectivity/interpretation by observer
 weighted kappa values were calculated by comparing the C4d scores
provided by each participant from their locally stained slides to the
corresponding C4d scores provided by the local participant from the
reference case
• Inter-laboratory variability =
 staining/laboratory procedure
 weighted kappa values were calculated by comparing the panel consensus
read for each tissue core on each slide to the panel read of the
corresponding tissue core on the reference slide
• Inter-observer variability =
 subjectivity/interpretation by observer
 weighted kappa values were calculated by comparing the reads of the local
participants to the consensus reads of the panel recorded on the same TMA
slide
Summary of mean kappa-values for the different components
influencing the reproducibility of a C4d stain
mean Kappa-value
comparing Banff
C4d scores to
reference case
mean Kappa-value
comparing Banff C4d
scores to majority calls
by all participants
mean Kappa-value
comparing
positive/negative C4d
calls to reference case
0.17 ± 0.3
0.26 ± 0.2
0.65 ± 0.3
reproducibility
(-0.68 – 0.65)
(-0.5 – 0.64)
(-0.46 – 1.0)
Inter-laboratory
0.46 ± 0.6
0.60 ± 0.4
0.78 ± 0.4
reproducibility
(-0.77 – 1.0)
(-0.36 – 1.0)
(-0.66 – 1.0)
Inter-observer
0.45 ± 0.2
NA
0.60 ± 0.4
reproducibility
(0.11 – 0.9)
Inter-institutional
(-1.0 – 1.0)
Significance of kappa values: <0 indicating no agreement (or less agreement than expected by chance), 0–0.20
slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, and 0.81–1 almost perfect agreement.
NA = Not Applicable, because for the inter-observer comparison per definition the comparator has to be the panel
read and cannot be a majority call
kappa values for inter-observer reproducibility
Scatter plots showing the inter-laboratory and inter-observer
reproducibility for each individual participant / participating laboratory
using the Banff C4d schema (A) or positive vs. negative calls (B)
A: using the Banff C4d grading schema
1.00
18%
B: using positive negative calls
1.00
0.80
0.80
0.60
0.60
0.40
0.40
0.20
0.20
0.00
0.00
0.20
0.40
0.60
0.80
0.00
1.00
0.00
59%
0.20
0.40
0.60
0.80
kappa values for inter-laboratory reproducibility
kappa >0.6 = at least moderate reproducibility
kappa >0.4 = at least fair reproducibility
Variance in scoring between observers:
• was greater with C4d1 and C4d2 cases compared to C4d0 and C4d3 cases
• the panel consistently under-scored the local observers, i.e. Pathologists adjust their
scoring to the sensitivity of their local laboratory
1.00
Influence of analytical variables on C4d staining:
comparing the top 15 and bottom 15 slides ranked by kappa
values for inter-laboratory reproducibility
45
40
Top 15
45
40
Bottom 15
Buffer pH Top 15
35
Number of Responses (%)
30
25
20
15
10
30
25
Top 15
Buffer pH Bottom 15
Bottom 15
20
Highest Temperature
Top 15
15
Highest Temperature
Bottom 15
50
40
Antibody
Dilution
Bottom 15
35
30
Antibody
Incubation
Top 15
25
Antibody
Incubation
Bottom 15
20
15
10
5
120+ min
60 min
40-49 min
30-39 min
no
response
prediluted
1:2000
1:400
1:200
1:100
1:80
1:50
1:40
1:30
1:25
1:15
1:10
0
40+ min
35 to 39
min
30 to 34
min
20 to 24
min
15 to 19
min
10 to 14
min
Antibody
Dilution Top
15
45
Number of Responses (%)
4 to 9 min
< 4 min
no response
unspecified
10+
9-9.9
8-8.9
6-6.9
No Response
Tris-EDTA
EDTA
Tris
0
Dako
0
Citrate
5
Bondmax
5
7-7.9
10
Ventana - CCI
No Response
Number of Responses (%)
35
Influence of Fixation for C4d
Inter-Laboratory reproducibility
1
2
3
0.64
0.45
1.00
1
Protocol
Standard
2
Delay
3
Frozen
4
Overfix
5
Underfix
6
Etha
4
5
6
0.82 -0.23 -0.32
Description
Immediate onset of fixation for 24h in buffered, standard
Formalin (4%)
Delayed onset of fixation for 12h (storing the tissue at
4˚C) and then fixation for 24h in buffered, standard
Formalin (4%)
Snap freezing of tissue (like simulating a frozen section
procedure) and then immediate fixation for 24h in
buffered, standard Formalin (4%)
Immediate onset of fixation for 5 days (simulation of
shipment over long weekend) in buffered, standard
Formalin (4%)
Immediate onset of fixation for only 1h (simulation of
very rush processing) in buffered, standard Formalin
(4%)
Immediate onset of fixation for 24h in Ethanol (100%)
Mean kappa values
Below Chance
<0.00
Slight
0.00 – 0.20
Fair
0.21 – 0.40
Moderate
0.41 – 0.60
Substantial
0.61 – 0.80
Almost Perfect
0.81 – 1.00
Summary and recommendations
• The BIFQUIT trial results indicated that C4d staining on paraffin sections
is of limited reproducibility.
• Refinement of the current Banff C4d scoring schema and
standardization of tissue processing and staining protocols is necessary
to achieve acceptable reproducibility for C4d staining on paraffinembedded material
• Recommendations from this first BIFQUIT trial for C4d
immunohistochemistry on paraffin sections:
–
–
–
–
Avoid under (<1 hour) and/or ethanol fixation of tissue specimen
Use heat induced epitope retrieval with citrate buffer at pH6-7
Incubate polyclonal anti-C4d antibody concentrated at <1:80 for >40 minutes
Better results were observed with harsher pre-treatment (e.g. autoclave
epitope retrieval), higher polyclonal anti-C4d antibody concentrations (1:10),
and longer incubation times for the polyclonal anti-C4d antibody (over night /
12-16 hours)
– Simplifying the 2007 Banff C4d grading schema will reduce inter-observer
variability
Acknowledgements
Many thanks to participants in the Banff Working
Group Trials!
The following Institutions contributed
cases to the BIFQUIT trial:
Cinthia Beskow-Drachenberg, Maryland, USA
Volker Nickeleit, Chapel Hill, USA
Verena Bröcker, Hannover, Germany
Bernard Collins, Boston, USA
Heinz Regele, Vienna, Austria
Surya Seshan, New York, USA
Students helped with logistics and analysis:
Samantha Chan
Jessie Climenhaga
Victoria Sheldon
Akshatha Raghuveer
Astellas Pharma Canada provided an
unrestricted research grant to the Banff
Working Groups
The panel