INTRA-OBSERVER
REPRODUCIBILITY AND
ACCURACY OF 1D, 2D, AND
3D LUNG TUMOUR
MEASUREMENTS
Laura Close
Medical Biophysics
The University of Western Ontario
March 22, 2011
Acknowledgements
• Dr. Grace Parraga
• Mr. Amir Owrangi
• Ms. Lauren Villemaire
• Mr. Andrew Wheatley
Introduction
• Lung cancer is the leading cause cancerrelated deaths in Canada1
• Many variables contributing to
effectiveness of patient treatment
• Choice/length of treatment
• Monitoring of treatment
• tracking tumour size
• Need effective measuring technique
1The
future of cancer control in Canada. (2011). Canadian Partnership Against Cancer.
Theory
• 1979: World Health Organization (WHO) 1
• 2D measurement
• (longest diameter) x (longest perpendicular bisector)
• 2000: Response Evaluation Criteria in Solid
Tumours (RECIST) 1
• 1D measurement
• Longest diameter
• 1D & 2D Limitations:
• Ex: Out-of-plane dimensions, sphericity assumed1
• 3D Quantify, created at Robarts Research Institute
(London, ON)
• Volume measurement
• Contours & 3D triangular meshes
• Irregular shapes, asymmetrical growth rates1
1Wilson,
L.C.R. (2010). Development of multi-dimensional x-ray computed
tomography measurements of lung tumours (Master’s thesis).
Objectives
• Determine intra-observer reproducibility of
1D, 2D, and 3D measurement techniques
• Determine accuracy of 1D, 2D, and 3D
measurements in relation to ground truth
measurements
• Use these factors to gain insight into
whether 3D measurements are
appropriate for clinical settings
Approach
• X-ray CT images:
• 2 patient tumours at 9 time points
• 3 phantom tumours at 4 slice thicknesses
• Software programs:
• ClearCanvas
• 3D Quantify
• Measurements:
• 1D: RECIST (ClearCanvas)
• 2D: WHO (ClearCanvas)
• 3D: Volume (3D Quantify)
Methods
• Ensuring Non-Bias:
• 2 patient tumours x 9 time points x 5 rounds &
• 3 phantoms x 4 slice thicknesses x 5 rounds
• All randomized
• Blind to ground truth measurements for
phantoms
• Did not exceed 1 round of measurements per
day
1D Measurements
(RECIST Diameter)
• Using ClearCanvas
• Longest diameter
Patient Tumour (Large),
Time Point 1
Phantom Tumour (Large),
0.5mm Slice Thickness
Phantom Tumour (Medium),
0.5mm Slice Thickness
Patient Tumour (Small),
Time Point 1
Phantom Tumour (Small),
0.5mm Slice Thickness
2D Measurements
(WHO)
• Using ClearCanvas
• Longest diameter x
longest
perpendicular
bisector
Phantom Tumour (Large),
0.5mm Slice Thickness
Patient Tumour (Large),
Time Point 1
Phantom Tumour (Medium),
0.5mm Slice Thickness
Patient Tumour (Small),
Time Point 1
Phantom Tumour (Small),
0.5mm Slice Thickness
3D Measurements
(Volume)
• Using 3D Quantify
• Volume of triangular
mesh
Patient Tumour (Large),
Time Point 1
Phantom Tumour (Large),
0.5mm Slice Thickness
Phantom Tumour (Medium),
0.5mm Slice Thickness
Patient Tumour (Small),
Time Point 1
Phantom Tumour (Small),
0.5mm Slice Thickness
Ex: Generation of Medium Phantom 3D Mesh
1D Measurements
Results
Mean RECIST Diameter (cm)
6
5
Patient Tumour Size Over Time (Large)
4
3
2
1
0
0.00
0.50
1.00
1.50
2.00
Time (years)
3D Measurements
20
160
18
140
Mean Volume (cm^3)
Mean WHO (cm^2)
2D Measurements
16
14
12
10
8
6
4
120
100
80
60
40
2
20
0
0
0.00
0.50
1.00
Time (years)
1.50
2.00
0.00
0.50
1.00
Time (years)
1.50
2.00
1D Measurements
Patient Tumour Size
Over Time (Small)
Mean RECIST Diameter (cm)
3
2.5
2
1.5
1
0.5
0
0.00
0.50
1.00
1.50
2.00
Time (years)
3D Measurements
6
30
5
25
Mean Volume (cm^3)
Mean WHO (cm^2)
2D Measurements
4
3
2
1
0
20
15
10
5
0
0.00
0.50
1.00
Time (years)
1.50
2.00
0.00
0.50
1.00
Time (years)
1.50
2.00
Measured Phantom Size
Vs. Slice Thickness
0.5mm Slice
Thickness
5.0mm Slice
Thickness
No obvious trends in
relation to slice thickness
1D Measurements
2D Measurements
Small
14
4
Large
12
Large
3.5
10
3
2.5
Mean WHO (cm^2)
Mean RECIST Diameter (cm)
Medium
Large
4.5
Medium
2
Small
1.5
8
6
4
Medium
2
Small
1
0.5
0
0
0
2
4
Slice Thickness (mm)
6
0
2
4
Slice Thickness (mm)
6
Increased slice thickness
relates to an increase in
volume measurement
Measured Phantom Size
Vs. Slice Thickness
5.0mm Slice
Thickness
3D Measurements
25
20
Mean Volume (cm^3)
Medium
Large
0.5mm Slice
Thickness
Large
y = 1.0784x + 14.919
R² = 0.9939
15
10
Medium
y = 0.3217x + 2.9817
R² = 1
5
Small
Small
y = 0.2044x + 1.5324
R² = 0.6955
0
0
1
2
3
4
Slice Thickness (mm)
5
6
Intra-Observer Reproducibility: Intraclass Correlation
Coefficients
ICC Values
≥0.9
≥0.8
≥0.7
For clinical measurements, ICC values ≥0.9 recommended1
(Indicated in green)
•
Phantom Tumours
Slice Thickness
1D
2D
3D
ICC(A)
ICC (C)
ICC(A)
ICC (C)
ICC(A)
ICC (C)
0.5mm
0.996
0.997
0.995
0.993
0.997
0.997
1.0mm
0.997
0.998
0.995
0.998
0.997
0.997
2.0mm
1
1
0.991
0.995
0.996
0.997
5.0mm
0.995
0.995
0.993
0.995
0.984
0.997
Patient Tumours
Time Point
1D
1 Portney
2D
3D
ICC(A)
ICC (C)
ICC(A)
ICC (C)
ICC(A)
ICC (C)
1
0.983
0.983
0.982
0.977
0.98
0.974
2
0.966
0.96
0.999
0.999
0.839
0.888
3
0.971
0.966
0.977
0.996
0.993
0.991
4
0.995
0.996
0.986
0.989
0.992
0.993
5
0.975
0.979
0.991
0.993
0.847
0.849
6
0.975
0.972
0.948
0.958
0.891
0.897
7
0.982
0.982
0.973
0.973
0.718
0.712
8
0.982
0.99
0.972
0.978
0.923
0.925
9
0.987
0.99
0.957
0.957
0.858
0.843
LG, Watkins MP. Foundations of clinical research: applications to
practice. Norwalk, CT: Appleton & Lange 1993; 505-528.
•
1D, 2D, and 3D phantom
tumour measurements are
reliable
•
1D, 2D, and some 3D
patient tumour
measurements are reliable
•
Certain time points more
obscured
Small and large tumours, time point 9
Accuracy of Measurements in Relation to
Known Ground Truth Measurements
• T-tests compare image measurements to ground
truth measurements (for each tumour at each slice
thickness)
p<0.001
0.001<p<0.01
0.01<p<0.05
p>0.05
Slice Thickness
0.5mm
1.0mm
2.0mm
5.0mm
• 2-sample, 2-tailed t-tests, assuming unequal variance
(all F-test p-values were <0.05)
Tumour 1D
2D
Large
0.000
Medium
0.000
Small
0.019
Large
0.000
Medium
0.000
Small
0.001
Large
0.000
Medium
0.000
Small
0.000
Large
0.000
Medium
0.000
Small
0.003
• H0: Means are equal
3D
0.000
0.000
0.000
0.000
0.000
0.001
0.000
0.000
0.001
0.000
0.000
0.000
0.008
0.038
0.540
0.291
0.002
0.001
0.957
0.008
0.010
0.020
0.001
0.001
• Using
α =0.05, H0 is not
rejected for values in
green
• 3D measurements show
strongest potential for
reproducing ground truths
Discussion
• Results account for: imaging, software,
measurement technique, individual
• Measurement technique of great
importance
• 3D volume measurement favourable
• 3D measurements are reproducible
• 3D recreates ground truth measurements
• Ability to measure irregular phantoms promising
as real tumours usually demonstrate simpler
geometry
Discussion (cont.)
• Tumour measurements impact
research/development of treatments
• Roadblocks for 3D
measurements:
• Imaging resolution limitations
impacting 3D accuracy
overcome
• Time-consuming
• Could stress time is worthwhile
for truer measurements
• Could increase efficiency of 3D
method
Vs.
Conclusion
• 1D, 2D, and 3D measurements are
reproducible
• 3D measurements display greatest
potential in accurately recreating ground
truth measurements
• Definite advantages in clinical settings
once drawback of time-consumption is
overcome