Motion Correction in fMRI time series
J.-F. Mangin, L. Freire, A. Roche, C Poupon
Service Hospitalier Frédéric Joliot, CEA, Orsay, France
Instituto de Biofisica e Engenharia Biomedica, FCUL, Lisboa, Portugal
Instituto de Medicina Nuclear, FML, Lisboa, Portugal
Projet Epidaure, INRIA, Sophia-Antipolis, France
Medical Vision Laboratory, Oxford, UK
IPAM 2004
SHFJ
DSV
Motion correction, why?
Taskcorrelated
motion
f MRI time series
IPAM 2004
SHFJ
voxel time series
DSV
Which solutions?
• Some constraints in the scanner to minimize motion
• Tracking (anatomical or artificial landmarks)
• On line corrections (MR Echo Navigator)
• Postprocessing: image registration
IPAM 2004
SHFJ
DSV
Image registration
Definition: given two images, find the geometrical
transformation that « best » aligns homologous voxels
First 3D image
IPAM 2004
Second 3D image
SHFJ
DSV
Classification of registration problems
• Search space (rigid, non-rigid)
• Monomodal / multimodal
• Intra-subject / inter-subjects
IPAM 2004
SHFJ
DSV
General formulation of image registration
Given two images I et J,
Optimization scheme
Tˆ  arg max S ( I , J , T )
Similarity measure
TΤ
Search space (rigid, affine, spline, etc.)
IPAM 2004
SHFJ
DSV
Building a similarity measure
Geometric Approach
• Detect features (points, lines, surfaces,… graphs)
• Measure the distance between these features
Iconic Approach
Direct comparison of intensities
IPAM 2004
SHFJ
DSV
Intuitive example
How to register these images ?
IPAM 2004
SHFJ
DSV
Geometric/Iconic Approach
Feature detection (here, points with high curvature)
2
Measure: for instance, S (T )   T (x k )  y k
k
IPAM 2004
SHFJ
DSV
Geometric/Iconic Approach
Feature detection (here, points with high curvature)
2
Measure: for instance, S (T )   T (x k )  y k
k
IPAM 2004
SHFJ
DSV
Geometric/Iconic Approach
No segmentation!
S (T )   (ik  jk ) 2
Measure: e.g.,
Interpolation: jk  J (T (x k ))
k
T
jk
T (x k )
IPAM 2004
ik
xk
SHFJ
DSV
Geometric/Iconic Approach
The simple case of binary images…
No segmentation!
S (T )   (ik  jk ) 2
Measure: e.g.,
k
T1 =Id
IPAM 2004
SHFJ
DSV
Geometric/Iconic Approach
No segmentation!
S (T )   (ik  jk ) 2
Measure: e.g.,
Partial overlap
k
T2
IPAM 2004
SHFJ
DSV
Iconic approach for fMRI!
A lot of
widely used softwares:
SPM, AIR, AFNI, etc.
Minimize the
sum of squared differences
A typical fMRI image
of the nineties
IPAM 2004
Search through rigid motions
(sometimes affine)
SHFJ
DSV
Main differences between softwares?
•Preprocessing (smoothing)
•Interpolation
nearest neighbor
spline
linear
•Optimization scheme (iterative)
•Powell
•Levenberg Marquardt
•Modified Gauss-Newton
IPAM 2004
Fighting with local optima
Improving accuracy
SHFJ
DSV
Iterative optimization of smooth measure
Small motion = good initialization
IPAM 2004
SHFJ
DSV
Is everything fine?
Block design, 3T magnet
Activations?
18 frames (2 sec/frame)
versus
180 frames
t
Motion correction
IPAM 2004
SHFJ
DSV
The consequences of motions
Motion-related artifacts:
– intrascan motion;
– the spin history effect;
– interaction between motion and
susceptibility;
– Non perfect estimation and interpolation.
Task-correlated motion =
Confounds in cognitive analysis
Use motion estimations as regressors?
IPAM 2004
SHFJ
DSV
BUT!
Could activated areas be responsible
For the task-correlated motion estimation?
Is this monomodal situation so secure?
IPAM 2004
SHFJ
DSV
Assumed relationship:
Conservation of intensity
Intensities of image I
Classification of standard similarity measures
Intensies of image J
Adapted measures :
Sum of squared differences
Sum of absolute differences
IPAM 2004
SHFJ
DSV
Classification of standard similarity measures
Intensities of image I
Assumed relationship:
Affine
Intensies of image J
Adapted measures
Correlation coefficient
 IJ (T ) 
1
n I  J

(
i

I
)(
j
 k
k  J )
k
1
Ratio of image uniformity (Woods)
R
N
i
 Rm 
2
i
Rm
IPAM 2004
SHFJ
DSV
Classification of standard similarity measures
Functional
Intensities of image I
Assumed relationship:
Intensies of image J
Adapted measures
Woods criterion (1993)
Woods variants (Ardekani, 95; Alpert, 96; Nikou, 97)
Correlation Ratio (Roche, 98)
IPAM 2004
SHFJ
DSV
Classification of standard similarity measures
Statistical
Intensities of image I
Assumed relationship:
Intensies of image J
Adapted measures
Joint entropy (Hill, 95; Collignon, 95)
Mutual Information (Collignon, 95; Viola, 95)
Normalized Mutual Information (Studholme, 98)
IPAM 2004
SHFJ
DSV
Do activated areas bias Least Square approaches?
II. Methods
Similarity Measures
•
•
•
•
•
•
IPMI’01
Let us compare various methods…
LS-SPM (Friston);
LS-AIR (Woods);
GM - custom (INRIAlign);
RIU-AIR (Woods);
CR – custom (Roche)
MI - custom (Wells, Maes, Viola).
Confounds related to interpolation
method or search method ?
The “sum of squared
differences” measure
assumes Gaussian
noise for the difference
between both images…
In some experiments
• LS - Custom;
• RIU - Custom;
IPAM 2004
SHFJ
DSV
Entropy for Image Registration
Define a joint probability distribution:
– Generate a 2-D histogram where each axis is the
number of possible greyscale values in each image
– Each histogram cell is incremented each time a pair
(I_1(x,y), I_2(x,y)) occurs in the pair of images
• If the images are perfectly aligned then the histogram is highly
focused. As the images mis-align the dispersion grows
• recall Entropy is a measure of histogram dispersion
IPAM 2004
SHFJ
DSV
Using joint entropy for Image Registration
– Define joint entropy to be:
H ( A, B)   p(i, j )  log[ p (i, j )]
i, j
– Images are registered when one is transformed relative
to the other to minimize the joint entropy
– The dispersion in the joint histogram is thus minimized
IPAM 2004
SHFJ
DSV
Definitions of Mutual Information
Commonly used definitions:
I(A,B) = H(A) + H(B) - H(A,B)
Maximizing the mutual info is equivalent to
minimizing the joint entropy (last term)
 p ( a, b) 

I ( A, B)   p(a, b)  log 
a ,b
 p(a) p(b) 
This definition is related to the Kullback-Leibler
distance between two distributions
IPAM 2004
SHFJ
DSV
This is not a robustness or accuracy study !
IPMI’01
Aim of the comparison of methods:
– assess the potential bias in motion parameter
estimation due to activation presence, whatever
the actual accuracy of each method.
Accuracy study has been done in several works
(Jiang, Frouin, West, Holden, etc.), and requires
the study and optimization of each parameter
influence, which is far beyond the scope of this
work.
IPAM 2004
SHFJ
DSV
II. Methods
Image Acquisition
A few words about the datasets
IPMI’01
– Brucker scanner operating at 3T using a 30 contiguous
slice 2D EPI sequence (slice array of 64x64 voxels).
– Pixel size of 3.75mm and slice thickness of 4mm.
– Various simulated time series.
– Actual time series using a block design.
IPAM 2004
SHFJ
DSV
Simulated Activation Patterns
IPMI’01
A first pattern inferred from a simple visual experiment
Two smaller patterns stemming from erosion
of the initial largest pattern
Pattern
Size (%)
Mean(%)
Max(%)
IPAM 2004
A1
12.4
1.26
2.04
A2
6.2
1.19
2.03
A3
3.2
1.18
2.03
SHFJ
DSV
Design of Simulated Time Series
IPMI’01
3x3x3
median
filtering
Tsim applied. 62x62x28 geometry.
Gaussian/Rician noise ( = 2.5%)
Activation
pattern
IPAM 2004
Reference
image
1
SHFJ
Test images
2
n
DSV
I
Simulated Activations Without Motion
+
0
x
1
2
Test,1
Test,2
39
40 3-D
Frames
Test,39
Run the six registration methods and evaluate the
transformation parameters (tx, ty, tz, rx, ry, rz) for each package.
Compute cross-correlation between each parameter and A1
time course and infer activated areas.
IPAM 2004
SHFJ
DSV
A couple of estimated motion parameters
IPAM 2004
SHFJ
I
DSV
Detection of “activated areas” using General Linear Model
I
Spatial Gaussian
smoothing - 5mm
Low-pass filtering
with 2-frame width
Voxels activated if
p-value > 0.001
Spurious
Clustered Voxels
(false positives)
LS-SPM – 227
LS-AIR –16
RIU-AIR and MI - 0
IPAM 2004
SHFJ
DSV
Simulated activations with simulated motion
IPMI’01
II
1
2
20
1
2
20
1
2
20
t=0.1mm
Tsim
t=5.0mm
r=0.1deg
r=2.0deg
t = 0.1, 0.2, 0.5, 1.0, 2.0 and 5.0 mm
r = 0.1, 0.2, 0.5, 1.0, and 2.0 deg
IPAM 2004
SHFJ
DSV
Simulated activations with simulated motion
Nº
1
2
3
4
5
6
7
8
II
8 Different Reference Images
Pattern
Mean Signal Increase (%)
NA
NA
A1
0.63
A1
1.26
A1
2.52
A1
5.04
A1
10.08
A2
2.52
A3
2.52
IPAM 2004
SHFJ
DSV
Simulated activations with simulated motion IIA
IPMI’01
Influence of motion amplitude
- 2 Reference images:
1
2
3
4
5
6
7
8
Pattern
NA
A1
A1
A1
A1
A1
A2
A3
Mean Signal Increase (%)
NA
0.63
1.26
2.52
5.04
10.08
2.52
2.52
- Test images:
t = 0.1, 0.2, 0.5, 1.0, 2.0, 5.0 mm.
r = 0.1, 0.2, 0.5, 1.0, 2.0 deg.
IPAM 2004
SHFJ
DSV
Simulated activations with simulated motion
IPAM 2004
SHFJ
IIA
DSV
Simulated activations with simulated motion
IPAM 2004
SHFJ
IIA
DSV
Simulated activations with simulated motion IIB
IPMI’01
Influence of activation amplitude
- 6 Reference images:
Pattern
NA
A1
A1
A1
A1
A1
A2
A3
1
2
3
4
5
6
7
8
Mean Signal Increase (%)
NA
0.63
1.26
2.52
5.04
10.08
2.52
2.52
- Test images:
t = 0.2 mm.
r = 0.2 deg.
IPAM 2004
SHFJ
DSV
Simulated activations with simulated motion
IPAM 2004
SHFJ
IIB
DSV
Simulated activations with simulated motion
IPAM 2004
SHFJ
IIB
DSV
Simulated activations with simulated motion IIC
IPMI’01
Influence of activation size
- 4 Reference images:
Pattern
NA
A1
A1
A1
A1
A1
A2
A3
1
2
3
4
5
6
7
8
Mean Signal Increase (%)
NA
0.63
1.26
2.52
5.04
10.08
2.52
2.52
- Test images:
t = 0.2 mm.
r = 0.2 deg.
IPAM 2004
SHFJ
DSV
Simulated activations with simulated motion
IPAM 2004
SHFJ
IIC
DSV
Simulated activations with simulated motion
IPAM 2004
SHFJ
IIC
DSV
Evaluation of spatial filtering pre-processing effect III
IPMI’01
– We have also performed an additional experiment on
the influence of the initial spatial smoothing applied by
SPM and AIR
– RIU-AIR problems are overcome by a large smoothing,
but motion estimates turn out to be biased.
– For SPM, low smoothing implies smaller bias but
motion estimates are less accurate.
IPAM 2004
SHFJ
DSV
III
IPAM 2004
SHFJ
DSV
IV
Experiment With Actual Time Series
IPMI’01
18 frames (2 sec/frame)
180 frames
IPAM 2004
t
SHFJ
DSV
IPAM 2004
SHFJ
DSV
Experiment With Actual Time Series
IV
Statistical Inference:
Gaussian smoothing (FWHM 5mm);
High-pass temporal filtering (period 120s);
Low-pass temporal filtering by a Gaussian function (4s).
Two explanatory variables:
Periodic stimulus convolved with a standard hemodynamic
response;
Time derivative of hemodynamic response.
Voxels reported activated if the p-value > 0.05.
IPAM 2004
SHFJ
DSV
Experiment With Actual Time Series: “activations”
IPAM 2004
SHFJ
DSV
Activations can bias “sum of squares” measure
IV. Discussion
“Robust” similarity measures have been classicaly used in
multimodality registration studies.
In monomodality studies, when the residuals are
endowed with a Gaussian distribution, LS-based
methods are optimal estimators.
The experiments presented in this work prove that the use
of LS-based methods for functional studies may be
questioned, due to the presence of activated areas.
IPAM 2004
SHFJ
DSV
Measure comparison, first simulation
– In first simulation, the use of the LS-custom method
shows that the bias in LS-based similarity measures is
related to the nature of the measure and not to the
intrinsic computational implementation of each method.
– The bias may induce spurious activations along highcontrast brain edges, in the absence of subject motion.
– RIU-AIR and MI are more robust to activation
presence, but presented qualitatively different results.
Accuracy vs. Robustness.
IPAM 2004
SHFJ
DSV
Measure comparison, other simulations
– The second set of simulations indicates that RIU-AIR
and MI accuracy does not depend on the presence of
activations.
– Also that the bias magnitude is highly related to the
signal change amplitude. This may explain why our
3T magnet led to more difficulties than more usual 1.5T
scanners. Indeed, it can be seen that activation level has
a more dramatic role on the accuracy decline than
activation size.
IPAM 2004
SHFJ
DSV
Actual time series
– The experiment with actual time series seems to be
consistent with our interpretation of the simulation
studies.
– LS-SPM and LS-AIR give different results, particularly
in pitch. The other two methods do not detect this
putative motion.
– Yaw estimations obtained by the four methods do not
agree. This may be related to distortions that cannot be
corrected with rigid-body transformations.
IPAM 2004
SHFJ
DSV
Spurious activations
– LS-based methods create spurious clusters of activated
voxels, whose localization depends, in our opinion, on
the brain edge orientation relatively to actual
activation localization.
– Spurious activations may appear at the same place
across individuals and survive to group analysis
– While we hope that this alarming prediction is too
pessimistic, it calls for trying to minimize the risk.
IPAM 2004
SHFJ
DSV
V. Conclusion
So what?
IPMI’01
– Our work has shown that “robust” similarity measures
could improve the situation with activation-based
outliers
– MI was used for historical reasons, but it may not be
the best choice. MI is prone to local maxima problems.
– Could we build a dedicated robust similarity
measure?
IPAM 2004
SHFJ
DSV
Mutual Information weaknesses (Pluim, Tsao)
Estimation of the joint histogram (Parzen windows…)
Number
of bins
Interpolation
method
IPAM 2004
SHFJ
DSV
Which similarity measures?
Taxonomy
S. Measure
Expression
Intensity
Conservation
Least-squares (LS)
Least-squares with a GM
estimator (GM)
Affine
Dependence
Ratio of Image Uniformity
(RIU)
Functional
Dependence
Correlation Ratio (CR)
Statistical
Mutual Information
Dependence
(MI)
IPAM 2004
SHFJ
 pab a  b 
2
a
b
2

a  b
pab

1  a  b2 / c 2 
a
b
1
N
 R
i
 Rm 
2
i
Rm
1
1
 A2
p
b
2
B
b
pab
pab log

pa pb
a b
DSV
The influence of the cut-off parameter in GM
Back to first simulation…
In the GM expression:
2

a  b
pab

2


a

b
a
b
1
c2
one shall tune the value of c.
However, the robustness
against “outliers” may lead
to local optima problems.
IPAM 2004
SHFJ
DSV
The influence of spatial smoothing
GM with no smoothing leads
to many local minima
problems, thus smoothing is
required.
In return, the CR and CRsym
methods turn out to be more
biased as the smoothing level
increases.
IPAM 2004
SHFJ
DSV
Simulated motionless time series.
IPAM 2004
SHFJ
DSV
The actual time series
IPAM 2004
SHFJ
DSV
Discussion
- MI and GM methods seem to be the most robust methods
relatively to activation presence.
- The GM has a disadvantage, which is related to the tuning
of the c parameter. A dynamic strategy to perform this
tuning could eventually lead to better results.
- However, some correlations with the activation task
persist.
- Build new dedicated measure after activation detection?
IPAM 2004
SHFJ
DSV
Let us try this simple idea with a couple of measures
•
•
•
•
•
LS – Custom (Friston et al., 1996);
GM – Custom (Nikou et al., 1998);
RIU – Custom (Woods et al., 1992);
CR – Custom (Roche et al., 1998);
MI – Custom (Collignon et al. 1995, Wells et al.,
1996).
Similarity measures implemented according to the
conventional (1) and the proposed approach (2), which
discards the activation signal.
IPAM 2004
SHFJ
DSV
Discarding the Activated areas
reference
Mask
IPAM 2004
test images
LS(1) RLS(1) RIU(1)
CR(1)
MI(1)
LS(2) RLS(2) RIU(2)
CR(2)
MI(2)
SHFJ
DSV
Results (first simulation)
tx
ty
tz
rx
ry
rz
LS 1
0.09
0.84
0.84
0.79
0.04
0.25
IPAM 2004
LS 2
0.01
0.38
0.16
0.17
0.17
0.13
GM 1
0.06
0.45
0.49
0.03
0.28
0.13
Simulation without motion
GM 2
RIU 1
RIU 2
0.07
0.07
0.14
0.12
0.82
0.32
0.12
0.79
0.04
0.01
0.79
0.24
0.21
0.19
0.05
0.20
0.22
0.29
SHFJ
CR 1
0.20
0.34
0.18
0.07
0.05
0.10
CR 2
0.05
0.03
0.40
0.03
0.13
0.06
MI 1
0.04
0.40
0.25
0.23
0.03
0.07
MI 2
0.33
0.05
0.09
0.28
0.16
0.01
DSV
Results (actual time series)
IPAM 2004
SHFJ
DSV
Results (3 actual time series)
tx
ty
tz
rx
ry
rz
LS 1
0.27
0.65
0.46
0.72
0.02
0.01
LS 2
0.27
0.17
0.14
0.10
0.05
0.13
GM 1
0.14
0.32
0.16
0.35
0.14
0.18
GM 2
0.13
0.12
0.34
0.03
0.09
0.15
tx
ty
tz
rx
ry
rz
LS 1
0.17
0.57
0.63
0.72
0.05
0.20
LS 2
0.24
0.27
0.29
0.37
0.16
0.03
GM 1
0.03
0.45
0.27
0.53
0.12
0.03
GM 2
0.02
0.18
0.02
0.26
0.04
0.02
tx
ty
tz
rx
ry
rz
LS 1
0.36
0.67
0.64
0.69
0.01
0.38
LS 2
0.40
0.29
0.11
0.05
0.04
0.17
GM 1
0.03
0.51
0.33
0.44
0.06
0.13
GM 2
0.15
0.17
0.03
0.01
0.03
0.09
IPAM 2004
SET 1
RIU 1
0.29
0.64
0.17
0.74
0.07
0.13
SET 2
RIU 1
0.20
0.58
0.45
0.73
0.21
0.03
SET 3
RIU 1
0.36
0.62
0.46
0.68
0.10
0.29
RIU 2
0.23
0.12
0.25
0.13
0.07
0.18
CR 1
0.04
0.27
0.12
0.36
0.06
0.07
CR 2
0.05
0.01
0.27
0.01
0.03
0.13
MI 1
0.02
0.41
0.19
0.58
0.14
0.18
MI 2
0.14
0.06
0.42
0.06
0.07
0.01
RIU 2
0.30
0.18
0.19
0.33
0.20
0.07
CR 1
0.13
0.25
0.11
0.41
0.01
0.01
CR 2
0.01
0.09
0.15
0.02
0.02
0.18
MI 1
0.01
0.34
0.17
0.57
0.10
0.08
MI 2
0.02
0.04
0.05
0.15
0.20
0.06
RIU 2
0.41
0.22
0.04
0.14
0.02
0.22
CR 1
0.03
0.17
0.33
0.23
0.04
0.12
CR 2
0.09
0.08
0.06
0.08
0.15
0.19
MI 1
0.29
0.32
0.23
0.40
0.07
0.08
MI 2
0.21
0.07
0.10
0.13
0.01
0.06
SHFJ
DSV
Results (2 actual time series, activations)
IPAM 2004
SHFJ
DSV
Discussion
- This last work rules out the hypothesis of a true taskrelated motion. Indeed, discarding about 20% of the voxels
almost removes the correlation with the task.
- The dilation of the mask is fundamental in order to avoid
contamination of neighbor voxels by activated voxels
- The proposed strategy is easy to implement and suitable
for most conventional studies. Improvements may be
required when registering complex studies.
IPAM 2004
SHFJ
DSV
General Conclusions & Further Work
Be careful!
- We have shown that the problem of registration of fMRI
time series should be revised in order to take into account
the influence of activation.
- We will have to study situations including true taskcorrelated motion.
- Detecting motion and activation simultaneously ?(Orchard
et al., 2003).
- Interactions between motion and distortions!
IPAM 2004
SHFJ
DSV
Eddy-Current Distortion Correction
for MR Diffusion Imaging
+
IPAM 2004
SHFJ
DSV
Diffusion tensor imaging
Diffusion-weighted signal attenuation:
Tensor estimation:
Several gradient
directions ( at least
6)
One gradient
chronogram
=
One b matrix
D
<x2>1/2
17
50
Chronogram
IPAM 2004
50
SHFJ
Td
Several gradient
strengths and
durations
DSV
Eddy current related EPI distortions
Spatial resolution :1,875 mm x 1,875 mm x 2,8 mm
Diffusion-sensitizing gradient
related distortion
Gradient 0 mT/m
No gradient
Scaling
Translation
Shearing
Gradient 22 mT/m
8 mm
IPAM 2004
SHFJ
DSV
Slice by slice estimation of affine transformations
Mutual information landscape
Powell
algorithm
Maximum
T1

Parzen window = truncated gaussian
Linear resampling
grey value coded with 6 bits
T0
S
T1
Mutual information maximization
IPAM 2004
SHFJ
DSV
Results for
one slice
Higher variability
with CR
than with MI
128 x 128
2mm x 2mm
Reference:
T2-weighted
4 repetitions
X gradient:
shearing
(frequency)
Y gradient:
shrinking
(phase)
Diffusion
gradients:
6 directions
5 amplitudes
4 repetitions
Z gradient:
translation (T0)
(slice)
IPAM 2004
SHFJ
DSV
Improvements achieved by the new estimation scheme
T2-weighted
Fractional anisotropy Fractional anisotropy
without corrections with the corrections
Fractional
anisotropy
=
variability
of
the tensor
eigenvalues
IPAM 2004
SHFJ
DSV
That’s all
A SPM plug-in performing fMRI motion
correction using the Geman McClure robust estimator
is freely distributed by the Epidaure group (INRIA)
INRIAlign…
IPAM 2004
SHFJ
DSV