Multi-tracer analysis for patient’s following using
multi-observation statistical image fusion :
a feasibility study
S. David1, M. Hatt1, P. Fernandez2, M. Allard2,
O. Barrett2, D. Visvikis1
1. LaTIM, INSERM U650, Brest, France
2. Department of nuclear medicine
Hospital Pellegrin – CHU Bordeaux
Outline
Context of oncology
• Positron Emission Tomography (PET)
• External radiotherapy
Use of PET in clinical application
• Multi-tracer analysis for dose-painting
• Patient monitoring in PET
Multi-observation fusion
• Multiband segmentation of a spectroscopic line data cube
• Developed method
• Preliminary results
Further work
Context of oncology
Cancer
• In 2002 : 11 millions new cases and 7 millions deaths
• Foresee in 2030 : 11 millions deaths
Diagnosis
• Computed tomography (CT)
• Magnetic resonance imaging (MRI)
• Emission imaging (PET, SPECT)
Since 2000 : PET / CT combined
Gold standard for the diagnosis
Treatment
• Surgery
• Chemotherapy
• Radiotherapy
Usually combined
Focus on the PET imaging and its application :
• Radiotherapy planning
• Therapy response assessment
Context and motivations
Positron Emission Tomography (PET) :
• Functional imaging : visualization of physiological processes
• Mainly used for cancer diagnosis and staging
Principle :
• Injection of a radiotracer
o Biological tracer targets the tumor
o Radionuclide : β- emitter
• Detection of the 2 γ rays
• Image reconstruction by tomography
Drawbacks of the PET imaging :
• Blur (spatial resolution)
• High noise (acquisition variability)
• Low resolution ( >5 mm)
Context and motivations
PET/CT imaging :
• Since 2000, acquisition of PET and CT in the same bed
• Gold standard for the diagnosis
CT image
PET image
PET / CT fusion
+ Combination of anatomical and functional information
+ Allow to anatomically locate the tracer uptake in the PET image
- Registration of the CT and PET scans
- Difference in the image resolution ( CT <1mm, PET >5 mm)
 PET / CT used in the radiotherapy planning
Context and motivations
Principle of radiotherapy :
• Use the ionizing radiation to kill the malignant cells
External radiotherapy (most widely used) :
• Photon or electron beams produced by a linear accelerator
• Shape of the beam defined by the collimator
Definition of the target volumes :
• Gross tumor volume (GTV)
o Defined with conventional imaging modalities
• Clinical target volume (CTV) > GTV
o Volume computed considering the anatomical information
• Planning target volume (PTV) > CTV
o Take into account the physiological variations
Use of PET in clinical application
Biological image-guided dose escalation
• Development of Intensity-modulated radiation therapy (IMRT)
• Administration of a non-uniform dose
o Adapt the treatment to the patient
o Reduce the irradiation of organ at risk (OAR) and surrounding healthy tissues
PET / CT : Target volume definition
IMRT planning
 Improvement of the tumor volume definition with the multi-tracer analysis
Multi-tracer analysis
• 18F-FDG : Measure the glucose consumption (tumor highly glucose consumer)
o Radiotracer the most widely use in PET
o Not tumor specific (other physiological processes need glucoses)
o Uptake in inflammatory tissues
Use of PET in clinical application
Development of specific radio-marker measuring ≠ tumor features :
• FMISO : measure of the tumor hypoxia (lack of oxygen)
o Hypoxia induces resilience to the radiotherapy
 Hypoxic tumors require a dose boosting
• 18F-FLT : measure of the tumor proliferation
oTumor specific radio-marker : ≠ FDG
o Avoid inflammatory tissues
o Lower uptake in the tumor than FDG
FDG coronal PET scan
• Merging all features measured by the tracers
• Each radiotracer is measuring a specific biologic process
o Scans are quantitatively not comparable
Y. Yamamoto et al, European Journal of Nuclear Medicine and Molecular Imaging , 2008
FLT coronal PET scan
Use of PET in clinical application
Patient follow-up with PET :
• Patients underwent chemo / radio-therapy
o Early assessment of the response to the treatment
• PET acquisition during the course of the therapy
o Estimation of prognosis
Tumors
PET1 pre-treatment
PET2 post-treatment
o Adapt the therapy to the response
o Avoid toxic and costly cares for non-responding patients
Goal of our work
Multi-sensor observation of an objet
• Many clinical data available with the multi-modality imaging
Goal :
• Fusing all the scans obtained :
o with the ≠ radiotracers
 Should improve the tumor volume definition
o at different time of the treatment
 A more accurate assessment of therapy response
Approach :
• With the statistical segmentation framework
• Model the information of multi-tracer and/or follow-up scans
Analogy with the astronomical framework
Multiband segmentation of a spectroscopic line data cube
Segmentation process based on Bayesian inference
• Observation with radio interferometers
o 3D data cubes (astronomical coordinates and frequency third axis)
Reduction of the dimension (42 channels) of the cube
• EM algorithm on Gaussian mixture model
o fit the spectrum : assessment of mean and variance
• Choice of the 6 most pertinent Gaussian
o K-means
• Computation of weight associated to each class
o Levenberg-Marquardt algorithm
Maps of the weights of the 6 Gaussian
with the NGC 4254 Cube
F. Flitti et al, Multiband segmentation of a spectroscopic line data cube : application to the H1 data cube of the spiral galaxy NGC 4254
Multiband segmentation of a spectroscopic line data cube
Bayesian classifier
• Hierarchical Markovian Model
o Models the spatial dependencies between neighbors
Steps of the segmentation process
• Initialization : K-mean algorithm
• Parameter estimation step
o Unsupervised with the Iterative Conditional Estimation (ICE)
• Segmentation step
o Criteria of maximum a posteriori (MAP)
Results on the NGC 4254 Cube
• Creation of a label map
F. Flitti et al, Multiband segmentation of a spectroscopic line data cube : application to the H1 data cube of the spiral galaxy NGC 4254
Multi-observation method
Statistical image segmentation :
• Estimation of X  ( xt )tT (hidden) with Y  ( yt )tT (measurements)
• No determinist link between X and Y
o Probabilistic approach (Bayesian inference)
p ( x, y )  p ( y | x ) p ( x )
Global
prior model
(spatial or contextual)
Local
Observation model
(noise)
Iterative estimation of
the parameters
Segmentation
Markovian model
(field, quadtree…)
Blind, contextual, adaptive…
Gaussian, generalized
gaussian
deterministic (EM)
stochastic (SEM)
hybrid (ICE)
MAP, MPM criteria
Multi-observation method
• X : labels field
• Y : measurements field
Measures Y
time
PET FDG
…
PET MISO
...
…
…
…
…
PET Tracer N
…
…
…
…
...
...
Label X
Multi-tracer analysis
Estimation of the (X,Y) distribution
• Mixture defined by θ=(α,β)
o αi : priors of X
o βi=(mi,Гi) : distribution of Y conditional to X
Label X
Patient follow-up
…
…
…
…
Multi-observation method : preliminary work
Fusion process :
Initialization :
• K-means
• Fuzzy K-means
Estimation of the (X,Y) distribution
• Maximization of the log-likelihood
• Implementation of EM and SEM algorithm
o Blind and adaptive (AEM,ASEM) version
Decision step
• MAP criteria
o Creation of a segmented map
Test on data set :
• Synthetic images
• Simulated tumors
Preliminary results
Synthetic images :
Map of labels X
• Map of labels X
o 2 or 3 labels per image
• Measurements Y
o Mean discrimination (MD)
o Variance discrimination (VD)
• Fusion of measurement with the same map of labels
o Number of labels in segmentation map = Number of labels per image
Segmentation results :
Measurement Y
Segmentation map
• N = 2 spectral bands
• K = 2 labels
• Random initialization
MD
VD
MD
VD
AEM segmentation
ASEM segmentation
• N = 2 spectral bands
• K = 3 labels
• Random initialization
AEM segmentation
ASEM segmentation
Preliminary results
Synthetic images :
• 2 or 3 labels per image
• Fusion of measurement with the different map of labels
o Additional labels in the segmented map
Segmentation results :
Measurement Y
Segmentation map
• N = 2 spectral bands
• K = 2 and 3 labels
• 4 classes in segmentation map
• Fuzzy K-means initialization
AEM segmentation
ASEM segmentation
AEM segmentation
ASEM segmentation
• N = 2 spectral bands
• K = 3 labels per scan
• 5 classes in segmentation map
• Fuzzy K-means initialization
Preliminary results
Simulated tumors :
 The ground truth of each tumor scan will be different
 The segmentation process should identify the new labels
• Each band : a tracer scan
• Label in the scan : uptake of the tracer
3 cases :
• N = 2 tracer scans
• K = 2 levels of tracer uptake per scan
• Fuzzy-K mean initialization
• N = 3 tracer scans
• K = 2 levels of tracer uptake per scan
• Fuzzy-K mean initialization
• N = 2 tracer scans
• K = 3 levels of tracer uptake per scan
• Fuzzy-K mean initialization
Preliminary results
Segmentation results :
• N images = 2 , K tracer uptake =2
AEM segmentation
ASEM segmentation
• N images = 3 , K tracer uptake =2
AEM segmentation
ASEM segmentation
AEM segmentation
ASEM segmentation
• N images = 2 , K tracer uptake =3
Preliminary results
Fusion of the synthetic images
• In the different situations (N Bands, K classes)
o Supervised, semi-supervised and unsupervised segmentation
o Satisfactory classification
Fusion of simulated tumors
• Segmentation error depends on :
o The fuzzy K-means initialization
o Noise level in the scans
o The number of classes in the label map
Limitations :
• Segmentation not totally unsupervised :
o Number of labels has to be defined by the user
• Fusion of few spectral bands
Further work
• Fusion process with more data
o Other tracers images and / or follow-up scans
• Segmentation totally unsupervised
o Estimation of the classes number in the label map
• Test the method on simulated data with GATE
o More realistic simulated tumors
o Computation of classification error
• Application of our method in a radiotherapy planning station
Thank you for your attention