Disclosure
Image registration,
deformation, and enhanced
contouring for radiotherapy
with MIM MaestroTM
• JP is a developer, employee, and has
ownership interest in MIM Software Inc.,
Cleveland, OH
Debra H. Brinkmann, Ph.D., Mayo Clinic Rochester, MN
Jon Piper, MIM Software Inc., Cleveland, OH
Outline
• Introduction/Overview of MIM MaestroTM JP
• Clinical Examples DHB / Technical features JP
– Image Registration
• Rigid
• Deformable
Introduction / Overview
MIM Maestro 5.1
– Enhanced Contouring
• Adaptive Contouring
• PET segmentation tools
• Atlas-based Segmentation
• Future Directions of MIM MaestroTM JP
1
MIM Maesro Introduction
• Evolution: Diagnostic tools redesigned for Radiation
Oncology
• Deformable tools initially developed in 2007
• Contouring, registration, fusion, dose review
Image Registration: Rigid
Image registration: Rigid
Image Registration: Rigid
Workflow
Clinical Examples – MR/CTplan
spyglass
• Select image series
• Select workflow &
follow instructions
• Registration options:
blending
– Assisted Alignment
– Box-based Alignment
– Contour-based
Alignment
– Manual
– View/edit translations
/rotations
• Review match
• Save as reformatted
images
2
Image Registration: Rigid
Image Registration: Rigid
Clinical Examples – MR/CTplan
Clinical Examples – MR/CTplan
Image Registration: Rigid
Image Registration: Rigid
Clinical Examples – MR/CTplan
Clinical Examples – initial alignment off
assisted
alignment
box-based
alignment
reset translations
define ROI for box-based alignment
manually
close
resulting move
registration
3
Image Registration: Rigid
Image Registration: Rigid
Tips from clinical experience
Tips from clinical experience
• Use workflows
– Customizable instructions
– Streamlines process
– Consistent output
• Use box-based/contour-based alignment
– Focuses/Improves registration over region of interest
• If initial match is strange
– Reset shift
– Get close with manual tools
– Then re-run alignment method
Rigid registration
• For multiple MR to CT fusion, assign best
anatomical MR series as first series to match
– Applies match to all series from same exam
• opportunity to adjust if needed
– Pay attention – assignment order currently flips around
each time you enter workflow
• Orientation issues resolved in 5.1.2 beta
– Trouble with other systems interpreting orientation of
reformatted FFS matched to HFS
Rigid registration
• Assisted alignment: uses field of view displayed
• Evaluate/adjust registration in all planes
• Contour/Box-based alignment: uses only data inside
• Be careful when using rigid alignment to solve a
• All algorithms employ nMI
• Point-based alignment: for fiducials, but beware
deformable problem
• Select rigid surrogates carefully: bones, small structures
• Multiple locally rigid registrations approximate deformation
• Remember MIM uses displayed contrast for rigid
alignments
4
Image Registration: Deformable
Workflow
Image registration: Deformable
• Initial rigid alignment
to get close
• Deformable CT-CT
alignment
– Uses entire overlapping
volume
• Apply deformation to
other series
(PET, SPECT, RTSS, RTDose)
• Evaluation
(voxel-to-voxel)
• Save as reformatted
images
Image Registration: Deformable
Image Registration: Deformable
Clinical Examples – PET/CTnm/CTplan
Clinical Examples – PET/CTnm/CTplan
5
Image Registration: Deformable
Tips from clinical experience
• We tend to try “regional” rigid registration first
(in case that is sufficient)
• Results can be strange - less so in 5.1
• During initial rigid alignment – focus on ROI
(only input the user has control over)
• Be aware of potential offset between
PET/SPECT and “inherently registered” CT
• Wishlist: evaluation map (where was deformation
significant)
Evaluation of an Intensity-Based Free-form
Deformable Registration Algorithm
JW Piper1,2
1MIM
Software Inc, 2Wake Forest University
Methods
• Constrained Intensity-based freeform
(DOF: millions)
• Validation
•Correlation
•Recover known deformations
•Consistency (forward and reverse)
Results
• Correlation: on the order of 1.4mm
error
• Phantom: 1.1mm mean error
• Consistency: 3.1mm mean
concatenated error
PET/CT Deformable Registration
PET/CT Deformable Registration
PET/CT Arms Down - SIM CT Arms Up
PET/CT Arms Down - SIM CT Arms Up
6
Utility of Deformable PET Fusion in Elucidating
GTV in Head & Neck Malignancies
Deformable PET/CT
SE Fogh1, GJ Kubicek1, R Axelrod1, WM Keane1, JW Piper2,3, Y Xiao1, M Machtay 1,
1Thomas
Jefferson University Hospital, 2MIM Software Inc, 3Wake Forest University
Methods
• 15 patients in different positions
(4 BOT, 5 Tonsil, 6 Larynx)
• GTV definition
•Manual correlation (gold standard)
•Rigid registration
•Deformable registration
• Non-correspondences
• Point-correlation is important to evaluation
• PET/CT deformation in the thorax and abdomen
•PET/CT is 3D CT but 4D PET
•Best will be 4D PET/CT or use 4DCT for planning with careful
selection of phase for deformation
Results
• Rigid errors: 7.81 (5.36) mm
• Deformable errors: 2.63 (2.76) mm
Enhanced Contouring: Adaptive Re-contouring
Workflow
Enhanced Contouring:
Adaptive Contouring
• Input: original CT,
original RTSS, new CT
• Initial rigid alignment
to get close
• Deformable CT-CT
alignment
• Apply deformation
to original structures
• Save deformed
structures
7
Enhanced Contouring: Adaptive Re-contouring
Enhanced Contouring: Adaptive Re-contouring
Clinical Examples – 5.1 vs. 4.1
Clinical Examples – CBCT changes
CBCT
Planning
CT
week1
Version
5.1.2beta
Extra
smoothing?
CBCT
week1
week5
Version
4.1
Enhanced Contouring: Adaptive Re-contouring
Enhanced Contouring: Adaptive Re-contouring
Clinical Examples – Replan
Tips from clinical experience
Bolus masked out
Rigid
Defining ROI for box-based
alignment to update initial
rigid registration over area of
interest (tongue)
Deformable
• During initial rigid alignment – focus on ROI
• Have user recreate PTV from deformed
GTV/CTV
• Make sure users save structures to NEW CT
• Works well for head and neck, CNS…
(Can use mask function as a workaround e.g. if bolus in only
one scan)
• Works well for lung CBCT
(using first CBCT as reference)
8
Enhanced Contouring: Adaptive Re-contouring
Tips from clinical experience
• Does not work well for abdomen if contrast
present in only one scan
• Do not use deformable dose currently
– Have included relevant isodose lines as contours
– RTDose output (new in MIM 5) is 16bit vs 32bit needed by our
TPS – should be available in MIM 5.2
• We do not use clinically yet, but functionality
available for propagating contours on 4DCT
phases to generate ITV, play movie loops…
Deformable Adaptive Re-contouring
• Automatically deforms structure sets to match anatomy
in replanning CT
• Contours should be edited as necessary
• Data from Tsuji, Hwang, and Weinberg* indicate
•No significant impact on dose between manual and adaptive OAR
•CTVs are significantly different due to changed treatment strategy
*Tsuji SY, Hwang A, Weinberg V, et al. Dosimetric Evaluation of Automatic Segmentation for Adaptive IMRT for Head-and-Neck
Cancer. IJROBP 2010;77(3):707-714.
Evaluation of a Deformable Re-Contouring
Method for Adaptive Therapy
Deformable Dose Accumulation
RC Fragoso1, JW Piper2,3, AS Nelson2, AS Harrison1, M Machtay2, Y Xiao1,
1Thomas
Jefferson University Hospital, 2MIM Software Inc, 4Wake Forest University
Methods
• 2 CTs obtained for 7 H&N patients
• Contouring methods
•Manually generated contours
•Automatic adaptive re-contouring
• Modification of automatic contours
Results
• 68-86% reduction in contouring time
• Difference Automatic to Modified less than
Old CT
Original
dose
New CT
Deformed
dose
Manual to Modified (p < 0.01)
•
Automatic contours of iso-volumic regions
(cord, brainstem, etc.) more consistent with
original contours than manual (p < 0.05)
9
Deformable Dose Accumulation
Deformable Dose Accumulation
• Dose accumulation commonly effects treatment plan for
recurrence
Added
dose
• Accurate deformable registration is needed because of
changes in anatomy
New
dose
Deformed
old dose
CBCT for Dose Tracking in HNC
K Hu1, A Surapeneni1, J Dolan1, JW Piper2,3, A Neff1, LB Harrison1
1Beth
Deformation for Tumor Response
Israel Medical Center, 2MIM Software Inc, 3Wake Forest University
Methods
• Acquire CT (CTr) & CBCT within 1 day
• Deform original CT to CBCT (CTd)
• Compare dose from CTd and CBCT to CTr
Results
• Reduction in error using CTd compared
•
•
•
•
•
•
•
with CBCT
PTV D95:
PTV Dmean:
Cord Dmax:
Brainstem Dmax:
R Parotid D50:
L Parotid D50:
Mandible Dmax:
0.60%
0.85%
0.45%
6.41%
11.40%
26.20%
0.90%
0.40%
0.47%
1.29%
0.03%
1.06%
2.57%
1.84%
10
4DCT
4DCT Deformable ITV Generation
• Deformably propagate contours from one phase to all
phases for more accurate ITV generation
• Dose, contours, and fusions on 4D cine
• 4D DVH
• The deformable algorithm has a broad capture range
Where to be Careful
• Adaptive re-contouring: some physicians like to keep
targets large even if the anatomy is shrinking.
• Use rigid transfer for targets and deformable for nodes and OARs
• Contrast differences in CT are challenging for intensitybased algorithms
• CBCT to CBCT is usually fine. CBCT to simCT is more
Enhanced Contouring:
PET segmentation tools
variable
• Generally fine for correcting HU for dose calculation
• Recent reconstruction or hardware produces more consistent results
• Calibrate the HU on your OBI - or use our correction tools
• Use first fraction CBCT as your reference
11
Enhanced Contouring: PET segmentation tools
Enhanced Contouring: PET segmentation tools
Workflow
Examples – Segmentation Tools
• Select desired tool
– % threshold
– SUV
– PET Edge
Different
%
PET
threshold
Edge
thresholds
• Set desired value
(%, SUV)
• Click and drag to define
ROI to search for %,
SUV or edge
Identifying most
active region
absolute
PET
Edge
threshold
defined
in each
plane gives
results within
1-2cc
Enhanced Contouring: PET segmentation tools
Enhanced Contouring: PET segmentation tools
Clinical Examples – SPECT/CT
Tips from clinical experience
• PET Edge can be used with other modalities
Plan CT
SPECT/CT
12
PET Tumor Segmentation
PET Edge
Challenge:
Accurate segmentation of PET GTV
• Manual contouring
• Subject to contrast/window/level settings
• Threshold-based contouring
•
•
•
•
Small tumors
Low activity tumors
Variable activity tumors
Variable background activity
Solution:
Gradient-based PET tumor segmentation
• Image processing algorithm segments using maximum spatial
gradients
• Accurate, robust, and reproducible
PET Tumor Segmentation
PET Tumor Segmentation: Validation of a
Gradient-Based Method Using a NSCLC Phantom
AD Nelson1, KD Brockway1, AS Nelson1, JW Piper1,2
Monte Carlo Lung Phantom
Lung tumors were simulated in a Monte Carlo Zubal phantom
at the University of Chicago,Med Phys 2008:35:33313342
1MIM
Software Inc, 2Wake Forest University
Methods
•
•
31 Monte Carlo simulated tumors
Contouring methods
•Gradient-based method
•15-50% of max in 5% increments
Results
• Mean absolute % error in volume
•Gradient:
11.0% (SD: 12%)
•25% thresh:
17.5% (SD: 29%)
• Slope of the best fit line
• Gradient:
1.03
• 15% thresh: 0.93
• 25% thresh: 0.77
• 35% thresh: 0.64
• 45% thresh: 0.44
13
PET Tumor Segmentation: Multi-Observer
Validation Using a NSCLC PET Phantom
AS
Nelson1,
1MIM
M
Werner-Wasik2,
W
Choi3,
Y
Arai4,
P
Faulhaber5,,
P
Kang3,
F
Almeida6,
N
Ohri2,
JW
Piper1,
AD
Nelson1
Pathologic Correlation of PET-CT Based Auto
Contouring for Radiation Planning in Lung Cancer
SE Fogh1, J Kannarkatt1, A Farach1, C Intenzo4, R Axelrod2, P McCue3, AS Nelson5, M Warner-Wasik1
Software Inc., 2Thomas Jefferson University Hospital, 3Beth Israel Medical Center, 4University of Pittsburgh Medical Center,
5University Hospitals Case Medical Center, 6University of Arizona Health Systems
Methods
• 31 MC simulated tumors
• 7 observers (3 rad, 4 RO)
• Contouring methods
•Gradient-based method
•25-50% of max
•Manual contouring
Results
• Mean absolute % error in volume
•Gradient
11.0% (SD: 12%)
•25% thresh: 17.5% (SD: 29%)
• Mean absolute % error in volume
•p < 0.01 gradient vs MC or 25% threshold
Departments of 1Radiation Oncology, 2Medical Oncology, 3Pathology, 4Radiology, Thomas Jefferson University, 5MIM Software Inc
Methods
• 18 PET or PET/CT and lobectomy
• Max resected tumor diameter vs Max
•
•
dimension of contour
PET Edge
34% of Max Threshold
Results
• Pearson’s Correlation Coefficient
•PET Edge:
0.72
•34% thresh:
0.08
PET Edge
• Check all three planes to verify you're starting close to
the center if the target
• Targets with funny shapes may require multiple passes
with PET Edge
• Contrast, contrast, contrast
• Clinical decision rests with the physician
Enhanced Contouring:
Atlas-based segmentation
14
Enhanced Contouring: Atlas-based segmentation
Enhanced Contouring: Atlas-based segmentation
Workflow
Example – with atlas built from cases
• Input: original CT, original
RTSS, new CT
• Select range (sup-inf)
• Select filter parameters for
atlas subjects
• Select desired contours
• After best-match is found,
select any additional
contours to transfer
• Save deformed structures
under the new CT
Enhanced Contouring: Atlas-based segmentation
Tips from clinical experience
• Prior to building an atlas with input from
multiple users, take time up front to
standardize contours
• Anticipate co-pilot contouring tool will
facilitate manageable editing for clinical use
Multi-institutional Experience with
Atlas-Based Segmentation in Head and Neck IMRT
K Hu1, A Lin2, A Young2, G Kubicek1, JW Piper3,4, AS Nelson3, J Dolan1, R Masino1, M Machtay2
1Beth
Israel Medical Center, 2Thomas Jefferson University Hospital, 3MIM Software Inc, 4Wake Forest University
Methods
• OPX (6), NPX (3), LAX (3)
• Comparison
•(A) Resident editing atlas contours
•(B) Atlas instead of resident
• Attending edits resident/atlas contours
Results
• 68% reduction in contouring time (A)
• 87% reduction in contouring time (B)
• Atlas contours saved as much time as resident
contours (B)
15
Atlas-Based Segmentation in Prostate IMRT:
Time-savings in the Clinical Workflow
MIM Auto-contouring Tools
A Lin1, G Kubicek1, JW Piper2,3, AS Nelson2, AP Dicker1, RK Valicenti1
1Thomas
Jefferson University Hospital, 2MIM Software Inc, 3Wake Forest University
Methods
• 98 patient atlas
• Comparison between
• Resident Attending
• Atlas Resident Attending
Results
• 46% reduction in contouring time
• 47% for resident
• 36% for attending
•
•
54% for femurs
46% for prostate
•
•
45% for bladder
35% for rectum
Auto-contouring
• It isn’t auto... yet. Verify and edit the contours
• Slice-to-slice contour deformation
• Re-shapes contour based on underlying image
• Works on any (anatomical) modality and in any plane
• Works on any contour
• Edit atlas contours or contour from scratch
Future enhancements
• Rigid alignment first guess improvements for small FOV
• Major performance enhancement for deformable
registration
• Continued improvements to the deformable algorithms
• Encouraged to QA versions of MIM with known data
• Contour CoPilot is still in its infancy
• 4D Dose accumulation
• Ability to limit and influence the registration
• MR/CT deformable registration... not yet.
• Session Save
16