Dataset of Observed Features on Endoscopic Colorectal Biopsies from
Normal Subjects and Patients With Chronic Inflammatory Bowel
Disease (Crohn’s Disease and Ulcerative Colitis)
Dr Simon S Cross
Senior Lecturer
Department of Pathology
University of Sheffield Medical School
Beech Hill Road
Sheffield S10 2RX, Great Britain
E-mail: s.s.cross@sheffield.ac.uk
Introduction:
Ulcerative colitis and Crohn’s disease are disorders of the digestive tract which extends
from the mouth, through the stomach and intestines to the back passage (anus)
(Anonymous 1996). Both disorders are grouped under the more generic term ‘chronic
idiopathic inflammatory bowel disease’ (CIIBD). CIIBD describes diseases of the bowel
which are characterised by acute and chronic inflammation and which have no identified
aetiological agent (such as an infective agent). Ulcerative colitis (UC) is restricted to the
large intestine which may be inflamed over a variable proportion of its length; if the
rectum only is inflamed the condition is described as proctitis. The lining of the intestine
becomes red and ulcerated; the inflamed area bleeds readily. For those with extensive
colitis there is an increased risk of bowel cancer which can develop in a young person.
Crohn’s disease most commonly involves the lower part of the small intestine (ileum) at
its junction with the large intestine (‘regional ileitis’) or the small and large intestine, or
the colon alone; swelling of the lips and ulceration of the mouth can occur, ulceration and
infection around the anus are common. In Crohn’s disease, not only is the lining of the
gut swollen and ulcerated, but there is also thickening of the wall of the intestine. The
inflammation may spread through the wall to involve neighbouring structures. Local
perforation of the wall can lead to widespread or localised infection, or an opening
(fistula) on the skin through which intestinal contents emerge. There is an increased risk
of cancer, particularly when the large intestine is extensively inflamed.
Histopathology is usually recognised as the 'gold standard' for the diagnosis of CIIBD.
The two major diagnostic decisions to be made in this area are: 1. Does the subject have
CIIBD or not? 2. If the subject does have CIIBD is it Crohn's disease or ulcerative
colitis? These decisions have important implications for patient management. The
division between CIIBD and not CIIBD determines whether a patient requires long term
follow-up or not. The follow-up for CIIBD could include examination of the colon by
colonoscopy (a flexible tube passed round the bowel from the anus) on a yearly basis, a
procedure that has a small associated mortality, relatively high patient discomfort and a
high cost. However lack of follow-up in a patient who has CIIBD could result in a missed
opportunity to detect colorectal carcinoma at an earlier, and thus more treatable, stage.
The distinction between Crohn's disease and UC becomes important if a patient requires
surgical removal of their colon for disease that does not respond to medical therapy. In
1
UC a total colectomy will be curative (since the disease only affects the colon) and the
surgical procedure can include formation of an ileoanal pouch providing continence for
the patient rather than an ileostomy (a stoma on the abdominal wall with an attached
bag). In Crohn's disease such reconstructive surgery is not advised because of the risk of
fistula formation between loops of small bowel.
Samples for the histopathological diagnosis of CIIBD are taken from the colon at
endoscopic examination. These small biopsies (2 mm in diameter) are embedded in
paraffin wax and thin sections are stained with haematoxylin and eosin to be examined by
light microscopy. The histopathological diagnosis is made subjectively by trained
histopathologists. Histopathologists acquire their knowledge and decision-making
processes from textbooks and from teaching by more experienced histopathologists, often
using a double-headed microscope so teacher and pupil are viewing the same image. In
Britain a trainee histopathologist (who will be medically-qualified) is required to have 5
years postgraduate training in recognised laboratories before she/he can take the final
examinations of the Royal College of Pathologists and be eligible to become a consultant
histopathologist. The diagnostic process in histopathology is poorly-understood but is
believed to be a combination of pattern recognition and some form of heuristic logic
(Underwood, 1987). The performance of histopathologists in the diagnosis of CIIBD has
been investigated by a few published studies but most of these have been carried out in
specialist centres for identified studies and so are likely to represent better performance
than the overall standard. However these studies produce a sensitivity for the diagnosis of
Crohn's disease or ulcerative colitis in the range of 40% to 82% and a specificity for these
diagnoses in the range of 73% to 98% (Frei and Morson, 1981; Thompson et al. 1985;
Jenkins, 1988; Seldenrijk et al. 1991; Surawicz et al. 1994). There is thus scope for a
decision-support system in the histopathological diagnosis of CIIBD to improve the
sensitivity and PPV of fully-trained histopathologists and for use in the long training
period required for histopathology novices.
The Dataset
Study population
The study population was drawn from large bowel endoscopic biopsies reported in the
Department of Histopathology, Royal Hallamshire Hospital, Sheffield between 1990 and
1995 (inclusive) (Dube et al. 1998). Biopsies originating in diverted bowel, rectal
stumps or pouches were excluded, as were those with a diagnosis of neoplasm. The
diagnosis was confirmed by the finding of typical endoscopy appearances seen on video
photographs in the clinical notes, subsequent bowel resection, pattern of disease on
radiological investigation or microbiological culture results. In cases without
confirmation by subsequent resection specimens this final diagnostic outcome was made
with review of the patient's case notes. The biopsies were a mixed population of single
distal biopsies and colonoscopic series from initial presentation and follow-up of disease.
2
The observed features
The biopsies were examined (blind to all clinical details) by a single experienced
observer (SSC) using a computer interface which implements the BSG Guidelines for the
Initial Biopsy Diagnosis of Suspected Chronic Idiopathic Inflammatory Bowel Disease
(Jenkins et al.
1997) with digitised images representing examples of each
histopathological feature (Cross et al. 1997). Some of the features are dichotomous
variables, e.g. the presence or absence of mucosal granulomas, whilst others are ordinal
categories, e.g. mucin depletion classified into none, mild, moderate or severe. The
observed features and their coding are given in table 1. Observation was spread over a
period of 9 months with no more than 30 biopsies observed in a single day.
Table 1. The observed features and other descriptors in the dataset.
Feature
Year (=year in which the biopsy was
taken)
Lab No (=unique laboratory
accession number for specified year)
Age
Sex
Active inflammation (subset
classifier, not observed feature)
Mucosal surface
Type
Range 0
1
Real integer
Binary
Binary
14-84
0,1
0,1
Male
No
Female
Yes
Flat
Irregular
Crypt architecture
Crypt profiles
Increased lamina propria cellularity
Mild & superficial increase in
lamina propria cellularity
Increased lymphoid aggregates in
lamina propria?
Patchy lamina propria cellularity?
Marked & transmucosal increase in
lamina propria cellularity
Cryptitis extent
Cryptitis polymorphs
Crypt abscesses extent
Crypt abscesses polymorphs
Lamina propria polymorphs
Epithelial changes
Mucin depletion
Intraepithelial lymphocytes
Subepithelial collagen
Lamina propria granulomas
Submucosal granulomas
Basal histiocytic cells
Confirmed diagnosis
Method of confirmation
Initial pathologists diagnosis
Observing pathologists diagnosis
Ordinal categorical
Real integer
Binary
Binary
0,1,2,3 Normal
2-7
0,1
No
0,1
No
Mild
Binary
0,1
No
Yes
Binary
Binary
0,1
0,1
No
No
Yes
Yes
Ordinal categorical
Ordinal categorical
Ordinal categorical
Ordinal categorical
Ordinal categorical
Ordinal categorical
Ordinal categorical
Binary
Binary
Binary
Binary
Binary
0,1,2,3
0,1,2,3
0,1,2,3
0,1,2,3
0,1,2
0,1,2,3
0,1,2,3
0,1
0,1
0,1
0,1
0,1
None
None
None
None
Absent
Normal
Normal
Normal
Normal
Absent
Absent
Absent
Little
Few
Little
Few
Focal
Flattening
Mild
Increased
Increased
Present
Present
Present
Ordinal categorical 0,1,2
3
2
3
Villous
projections
Moderate
Severe
Yes
Yes
Moderate
Several
Moderate
Several
Diffuse
Degeneration
Moderate
Marked
Many
Marked
Many
Erosion
Severe
Partitioning of the dataset
The Excel file contains 4 worksheets of data that include one with all cases and three
subsets for analysis.
All cases
This worksheet contains all the cases in the study in the order in which they were
observed. It is included only as a reference because it contains cases that are not suitable
for analysis. These are cases of other diseases, such as mucosal prolapse and melanosis
coli, which have small numbers of cases (and so are unlikely to be adequately represented
in training and test sets) and which the BSG defined observations do not cover the
specific diagnostic features of these diseases (e.g. smooth muscle passing up between
crypts in mucosal prolapse, pigment-containing macrophages in the lamina propria in
melanosis coli).
All IBD&normal
This worksheet contains 809 cases of which 165 are normal, 473 UC and 171 Crohn’s
disease. These cases are all verified cases of these outcomes with active or inactive
inflammation. The cases are in a randomised order and this order should be retained
when training and testing. The first 270 cases should be used as a training set, the next
270 cases as an optimisation/verification set and the final 269 cases as a test set. If the
methodology does not require a separate optimisation/verification set then the training set
can be the first 540 cases. The outcome can be taken as normal v. CIIBD (i.e. UC and
Crohn’s combined into a single CIIBD category) or as a tripartite normal v. Crohn’s v.
UC outcome. A sequential process could be applied to this dataset with an initial division
into normal or CIIBD and then a subsequent division of the CIIBD set into UC or
Crohn’s (or inactive and active inflammation and a third division of actively-inflamed
cases into UC or Crohn’s).
All IBD
This worksheet contains 644 cases of which 473 are UC and 171 Crohn’s disease. These
cases are all verified cases of these outcomes with active or inactive inflammation. The
cases are in a randomised order and this order should be retained when training and
testing. The first 215 cases should be used as a training set, the next 215 cases as an
optimisation/verification set and the final 214 cases as a test set. If the methodology does
not require a separate optimisation/verification set then the training set can be the first
430 cases. The outcome will be Crohn’s v. UC.
Active IBD
This worksheet contains 370 cases of which 283 are UC and 87 Crohn’s disease. These
cases are all verified cases of these outcomes with active or inactive inflammation. The
cases are in a randomised order and this order should be retained when training and
testing. The first 124 cases should be used as a training set, the next 123 cases as an
optimisation/verification set and the final 123 cases as a test set. If the methodology does
not require a separate optimisation/verification set then the training set can be the first
247 cases. The outcome will be Crohn’s v. UC.
4
Working practices for analysis:
1. Use the randomised order of data as given in the worksheets
2. Use the partitioning into training, optimisation/verification and test sets as specified
above for each set
3. For any process with a variable threshold express the results on a receiver operating
characteristic (ROC) curve and use McNemar’s test to compare the area under the
curve with other similar processes. Store the coordinates of the ROC curves in clearly
labelled Excel worksheets so that these can be imported into graphing programmes
4. At an optimal threshold calculate the sensitivity, specificity, predictive value of a
positive result (PV +ve), predictive value of a negative result (PV -ve) and the kappa
statistic – all with 95% confidence intervals. Discuss what these optimal thresholds
might be with SSC e.g. is sensitivity or specificity most important in separating UC
from Crohn’s in biopsies with active inflammation
Human performance:
All IBD&normal initial pathologists' diagnosis
Confirmed outcome
Normal
157
Crohn's disease
24
Ulcerative colitis
22
Totals
203
Crohn's disease highly suggestive
Crohn's disease suggestive
Ulcerative colitis highly suggestive
0
62
0
62
0
14
5
19
0
0
219
219
Ulcerative colitis suggestive
CIIBD
indeterminate
Infective type
colitis
0
2
54
56
3
41
146
190
0
0
3
3
Lymphocytic
colitis
0
0
0
0
Melanosis coli
Inflammation unclassified
2
3
1
27
0
24
3
54
165
171
473
809
Normal
Initial
pathologists'
diagnosis
Totals
5
If the CIIBD categories are combined and all other categories combined as not CIIBD
this table results:
Confirmed outcome
Initial
Not CIIBD
pathologists'
diagnosis
CIIBD
Not CIIBD
162
CIIBD
101
Totals
263
3
165
543
644
546
809
Totals
Sensitivity
84% (82-87%)
Specificity
98% (96-99%)
PV +ve result 99% (98-99%)
PV -ve result 62% (56-67%)
Kappa
0.68 (0.62-0.73)
If the highly suggestive and suggestive of Crohn's disease categories are combined and
all other categories combined as not Crohn's disease this table results:
Confirmed outcome
Initial
Not Crohn's disease
pathologists'
diagnosis
Crohn's disease
Not Crohn's disease
633
Crohn's disease
95
Totals
728
5
638
76
171
81
809
Totals
Sensitivity
44% (37-52%)
Specificity
99% (98-99%)
PV +ve result 94% (89-99%)
PV -ve result 87% (85-89%)
Kappa
0.54 (0.46-0.63)
If the highly suggestive and suggestive of UC categories are combined and all other
categories combined as not UC this table results:
Confirmed outcome
Not UC
UC
Initial
pathologists'
Not UC
UC
334
2
200
273
Totals
534
275
diagnosis
Totals
336
473
809
Sensitivity
Specificity
PV +ve result
PV -ve result
58%
99%
99%
63%
6
(53-62%)
(98-99%)
(98-99%)
(58-67%)
Kappa
0.53
(0.47-0.58)
All IBD&normal observing pathologist's diagnosis
Confirmed outcome
Normal
151
Crohn's disease
64
Ulcerative colitis
90
Totals
305
Crohn's disease highly suggestive
0
15
2
17
Crohn's disease suggestive
Ulcerative colitis highly suggestive
0
11
12
23
1
14
138
153
Ulcerative colitis suggestive
CIIBD
indeterminate
Infective type
colitis
Lymphocytic
colitis
0
9
64
73
3
30
108
141
0
3
2
5
0
0
1
1
Melanosis coli
Inflammation unclassified
1
9
1
24
1
55
3
88
165
171
473
809
Normal
Observing
pathologist's
diagnosis
Totals
If the CIIBD categories are combined and all other categories combined as not CIIBD
this table results:
Confirmed outcome
Observing
Not CIIBD
pathologist's
diagnosis
CIIBD
Not CIIBD
315
CIIBD
87
Totals
402
4
319
403
490
407
809
Totals
Sensitivity
82% (79-86%)
Specificity
99% (98-99%)
PV +ve result 99% (98-99%)
PV -ve result 78% (74-82%)
Kappa
0.77 (0.73-0.82)
7
If the highly suggestive and suggestive of Crohn's disease categories are combined and
all other categories combined as not Crohn's disease this table results:
Confirmed outcome
Observing
Not Crohn's disease
pathologist's
diagnosis
Crohn's disease
Not Crohn's disease
624
Crohn's disease
145
Totals
769
14
638
26
171
40
809
Totals
Sensitivity
15% (10-21%)
Specificity
98% (97-99%)
PV +ve result 65% (50-80%)
PV -ve result 81% (78-84%)
Kappa
0.18 (0.07-0.29)
If the highly suggestive and suggestive of UC categories are combined and all other
categories combined as not UC this table results:
Confirmed outcome
Observing
Not UC
pathologist's
diagnosis
UC
Totals
Not UC
312
UC
271
Totals
583
24
336
202
473
226
809
Sensitivity
43% (38-47%)
Specificity
93% (90-96%)
PV +ve result 89% (85-93%)
PV -ve result 54% (49-58%)
Kappa
0.32 (0.26-0.38)
8
All IBD initial pathologists' diagnosis
Confirmed outcome
Crohn's disease
25
Ulcerative colitis
22
Totals
47
Crohn's disease highly suggestive
62
0
62
Crohn's disease suggestive
Ulcerative colitis highly suggestive
14
5
19
0
219
219
Ulcerative colitis suggestive
CIIBD
indeterminate
Infective type
colitis
Melanosis coli
2
54
56
41
146
187
0
3
3
1
0
1
26
24
50
171
473
644
Normal
Initial
pathologists'
diagnosis
Inflammation unclassified
Totals
If the highly suggestive and suggestive of Crohn's disease categories are combined and
all other categories combined as not Crohn's disease this table results:
Confirmed outcome
Initial
Not Crohn's disease
pathologists'
diagnosis
Crohn's disease
Not Crohn's disease
468
Crohn's disease
95
Totals
563
5
473
76
171
81
644
Totals
Sensitivity
44% (37-52%)
Specificity
99% (98-99%)
PV +ve result 94% (89-99%)
PV -ve result 83% (80-86%)
Kappa
0.52 (0.44-0.61)
9
If the highly suggestive and suggestive of UC categories are combined and all other
categories combined as not UC this table results:
Confirmed outcome
Initial
Not UC
pathologists'
diagnosis
UC
Totals
Not UC
169
UC
200
Totals
369
2
171
273
473
275
644
Sensitivity
58% (53-62%)
Specificity
99% (97-99%)
PV +ve result 99% (98-99%)
PV -ve result 46% (41-51%)
Kappa
0.41 (0.35-0.48)
All IBD observing pathologist's diagnosis
Confirmed outcome
Crohn's disease
64
Ulcerative colitis
90
Totals
154
Crohn's disease highly suggestive
Crohn's disease suggestive
Ulcerative colitis highly suggestive
15
2
17
11
12
23
14
138
152
Ulcerative colitis suggestive
CIIBD
indeterminate
Infective type
colitis
9
64
73
30
108
138
3
2
5
0
1
1
1
24
1
55
2
79
171
473
644
Normal
Observing
pathologist's
diagnosis
Lymphocytic
colitis
Melanosis coli
Inflammation unclassified
Totals
10
If the highly suggestive and suggestive of Crohn's disease categories are combined and
all other categories combined as not Crohn's disease this table results:
Confirmed outcome
Observing
Not Crohn's disease
pathologist's
diagnosis
Crohn's disease
Not Crohn's disease
459
Crohn's disease
145
Totals
604
14
473
26
171
40
644
Totals
Sensitivity
15% (10-21%)
Specificity
97% (96-99%)
PV +ve result 65% (50-80%)
PV -ve result 76% (73-79%)
Kappa
0.16 (0.05-0.28)
If the highly suggestive and suggestive of UC categories are combined and all other
categories combined as not UC this table results:
Confirmed outcome
Observing
Not UC
pathologist's
diagnosis
UC
Totals
Not UC
148
UC
271
Totals
419
23
171
202
473
225
644
Sensitivity
43% (38-47%)
Specificity
87% (81-92%)
PV +ve result 90% (86-94%)
PV -ve result 35% (31-40%)
Kappa
0.20 (0.13-0.27)
11
Active IBD initial pathologists' diagnosis
Confirmed outcome
Crohn's disease
0
Ulcerative colitis
0
Totals
0
Crohn's disease highly suggestive
48
0
48
Crohn's disease suggestive
Ulcerative colitis highly suggestive
13
4
17
0
143
143
Ulcerative colitis suggestive
CIIBD
indeterminate
Infective type
colitis
Inflammation unclassified
2
40
42
20
85
105
0
3
3
4
8
12
87
283
370
Normal
Initial
pathologists'
diagnosis
Totals
If the highly suggestive and suggestive of Crohn's disease categories are combined and
all other categories combined as not Crohn's disease this table results:
Confirmed outcome
Not Crohn's disease
Crohn's disease
Initial
pathologists'
Not Crohn's disease
Crohn's disease
279
4
26
61
Totals
305
65
diagnosis
Totals
283
87
370
Sensitivity
70% (60-80%)
Specificity
99% (97-99%)
PV +ve result 94% (88-99%)
PV -ve result 91% (88-95%)
Kappa
0.75 (0.67-0.84)
12
If the highly suggestive and suggestive of UC categories are combined and all other
categories combined as not UC this table results:
Confirmed outcome
Initial
Not UC
pathologists'
diagnosis
UC
Totals
Not UC
85
UC
100
Totals
185
2
87
183
283
185
370
Sensitivity
65% (59-70%)
Specificity
98% (95-99%)
PV +ve result 99% (97-99%)
PV -ve result 46% (39-53%)
Kappa
0.45 (0.36-0.54)
Active IBD observing pathologist's diagnosis
Confirmed outcome
Crohn's disease
0
Ulcerative colitis
1
Totals
1
Crohn's disease highly suggestive
Crohn's disease suggestive
Ulcerative colitis highly suggestive
13
2
15
11
10
21
13
130
143
Ulcerative colitis suggestive
CIIBD
indeterminate
Infective type
colitis
8
43
51
25
67
92
3
2
5
Inflammation unclassified
14
28
42
Totals
87
283
370
Normal
Observing
pathologist's
diagnosis
13
If the highly suggestive and suggestive of Crohn's disease categories are combined and
all other categories combined as not Crohn's disease this table results:
Confirmed outcome
Observing
Not Crohn's disease
pathologist's
diagnosis
Crohn's disease
Not Crohn's disease
262
Crohn's disease
61
Totals
323
21
283
26
87
47
370
Totals
Sensitivity
30% (20-40%)
Specificity
93% (90-96%)
PV +ve result 55% (41-70%)
PV -ve result 81% (77-85%)
Kappa
0.27 (0.13-0.41)
If the highly suggestive and suggestive of UC categories are combined and all other
categories combined as not UC this table results:
Confirmed outcome
Observing
Not UC
pathologist's
diagnosis
UC
Totals
Not UC
66
UC
110
Totals
176
21
87
173
283
194
370
Sensitivity
61% (55-67%)
Specificity
76% (67-85%)
PV +ve result 89% (85-94%)
PV -ve result 38% (30-45%)
Kappa
0.27 (0.17-0.37)
Discussion of human performance
The performance of the initial reporting pathologists is better than that of the observing
pathologist despite the fact that the observing pathologist is an experienced specialist
gastrointestinal pathologist whereas the initial reporting pathologists were a
heterogeneous group of trainee and consultant pathologists without specialism in
gastrointestinal pathology. This is likely to be explained by the different environments in
which the diagnoses were made. The observing pathologist made his diagnosis with no
clinical details about the case whereas the initial reporting pathologists would have taken
into account the clinical details written on the histopathology request form. These details
may have been very specific, e.g. 'long-standing UC, surveillance colonoscopy', and thus
given the clinician's diagnosis to the pathologist before the pathologist had viewed the
slides. For the purposes of comparison with statistical methods the performance of the
observing pathologists should be taken as the most appropriate benchmark.
14
References:
Anonymous (1996) Inflammatory Bowel Disease. London: British Society for
Gastroenterology.
Cross, S.S., Dube, A.K., Underwood, J.C.E., Pialle, H., Sharkey, A. and Jenkins, D.
(1997) A visual image-based data entry system for reporting non-neoplastic
endoscopic colorectal biopsies. J.Pathol. 181, 17A(Abstract)
Dube, A.K., Cross, S.S. and Lobo, A.J. (1998) Audit of the histopathological diagnosis
of non-neoplastic colorectal biopsies: achievable standards for the diagnosis of
inflammatory bowel disease. J.Clin.Pathol 51, 378-381.
Frei, J.V. and Morson, B.C. (1981) Medical audit of rectal biopsy diagnosis of
inflammatory bowel disease. J.Clin.Pathol. 35, 341-344.
Jenkins, D. (1988) Computing and histopathology of intestinal inflammation. In: Vicery,
F.R., (Ed.) Computers in Gastroenterology, pp. 193-204. London: SpringerVerlag]
Jenkins, D., Balsitis, M., Gallivan, S., Dixon, M.F., Gilmour, H.M., Shepherd, N.A.,
Theodossi, A. and Williams, G.T. (1997) Guidelines for the initial biopsy
diagnosis of suspected chronic idiopathic inflammatory bowel disease. The
British Society of Gastroenterology Initiative. J.Clin.Pathol. 50, 93-105.
Seldenrijk, C.A., Morson, B.C., Meuwissen, S.G., Schipper, N.W., Lindeman, J. and
Meijer, C.J. (1991) Histopathological evaluation of colonic mucosal biopsy
specimens in chronic inflammatory bowel disease: diagnostic implications. Gut
32, 1514-1520.
Surawicz, C.M., Haggitt, R.C., Husseman, M. and McFarland, L.V. (1994) Mucosal
biopsy diagnosis of colitis: acute self-limited colitis and idiopathic inflammatory
bowel disease. Gastroenterology 107, 755-763.
Thompson, E.M., Price, A.B., Altman, D.G., Sowter, C. and Slavin, G. (1985)
Quantitation in inflammatory bowel disease using computerised interactive image
analysis. J.Clin.Pathol. 38, 631-638.
Underwood, J.C.E. (1987) Introduction to Biopsy Interpretation and Surgical Pathology,
2 edn. London: Springer-Verlag.
15