International Journal for Research and Development in Engineering (IJRDE)
www.ijrde.com
ISSN: 2279-0500
Special Issue: pp- 388-393
NSCT Based Multimodal Medical Image Fusion
S. Niveditha1, P. Mahalakshmi2, Florence Valencia Henry3
Electronics and Communication Engineering, Jeppiaar Engineering College, Chennai, India
ABSTRACT
Fusion imaging is one of the most accurate and
useful diagnostic techniques in medical imaging.
Although image fusion can have different purposes,
the main aim of fusion is spatial resolution
enhancement or image sharpening. This paper
proposes a new image fusion framework for
multimodal medical images, which relies on the
non-subsampled contour let transform (NSCT)
domain. The source medical images are
transformed using NSCT into low and high
frequency components. Phase congruency and
directive contrast are used for combining low and
high-frequency coefficients respectively. The fused
image is constructed by the inverse NSCT with all
composite coefficients. Experimental results and
comparative study show that the proposed fusion
framework provides an effective way to enable
more accurate analysis of multimodality images.
Keywords – Image fusion, NSCT, Phase
Congruency, Sum-Modified-Laplacian, Directive
Contrast, Inverse NSCT.
I.
INTRODUCTION
Fusion imaging is one of the most modern, accurate
and useful diagnostic techniques in medical imaging
today. Although image fusion can have different
purposes, the main aim of fusion is spatial resolution
enhancement or image sharpening. Also known as
integrated imaging, the benefits are even more
profound in combining anatomical imaging modalities
with functional ones. Different modalities of medical
imaging reflect different information of human organs
and tissues, and have their respective application
ranges [1-4]. A single modality of medical image
cannot provide comprehensive and accurate
information. As a result, combining anatomical and
functional medical images to provide much more
useful information through image fusion has become
the focus of imaging research and processing.
Multimodal medical image fusion not only helps in
diagnosing diseases, but it also reduces the storage cost
by reducing storage to a single fused image instead of
multiple-source images. So far, extensive work has
been made on image fusion technique with various
techniques dedicated to multimodal medical image
fusion. These techniques have been categorized into
three categories according to merging stage. These
include pixel level, feature level and decision level
fusion where medical image fusion usually employs the
pixel level fusion due to the advantage of containing
the original measured quantities, easy implementation
and computationally efficiency [5-8].
The well-known pixel level fusion is based on principal
component analysis (PCA), independent component
analysis (ICA), contrast pyramid (CP), gradient
pyramid (GP) filtering, etc. These are not the highly
suitable for medical image fusion. Wavelet transform
has been identified ideal method for image fusion.
However, it is argued that wavelet decomposition is
good at isolated discontinuities, but not good at edges
and textured region [9]. However, these methods often
produce undesirable side effects like block artifacts,
reduced contrast etc. due to the drawbacks of wavelet
transform such as lack of symmetry, critical sampling,
less vanishing moments etc.
This project is an attempt to rectify the drawbacks of
wavelet transform in image fusion. For this purpose,
non-subsampled contour let transform (NSCT) is used
in the proposed framework. NSCT has properties such
as multistate, localization, multidirectional, and shift
invariance, but only limits the signal analysis to the
time frequency domain. Two different fusion rules are
proposed for combining low and high-frequency
coefficients. For fusing the low-frequency coefficients,
the phase congruency based model is used. A new
definition of directive contrast in NSCT domain is
proposed and used to combine high-frequency
coefficients. Finally, the fused image is constructed by
the inverse NSCT with all composite coefficients.
Experimental results and comparative study show that
the proposed fusion framework provides an effective
way to enable more accurate analysis of multimodality
images. The salient contributions of the proposed
Methods Enriching Power and Energy Development (MEPED) 2014
388 | P a g e
International Journal for Research and Development in Engineering (IJRDE)
www.ijrde.com
ISSN: 2279-0500
Special Issue: pp- 388-393
framework over existing methods can be summarized
as follows.
•
This paper proposes a new image fusion
framework for multimodal medical images, which
relies on the NSCT domain.
•
Two different fusion rules are proposed for
combining low and high-frequency coefficients.
•
For fusing the low-frequency coefficients, the
phase congruency based model is used. The main
benefit of phase congruency is that it selects and
combines
contrastand
brightness-invariant
representation contained in the low frequency
coefficients.
•
On the contrary, a new definition of directive
contrast in NSCT domain is proposed and used to
combine high-frequency coefficients. Using directive
contrast, the most prominent texture and edge
information are selected from high-frequency
coefficients and combined in the fused ones.
•
The definition of directive contrast is
consolidated by incorporating a visual constant to the
SML based definition of directive contrast which
provides a richer representation of the contrast.
Fig 1. Three-stage non-subsampled pyramid decomposition
Fig 2. Four-channel non-subsampled directional filter bank
II.
PRELIMINARIES
This section provides the description of concepts on
which the proposed framework is based. These
concepts include NSCT and phase congruency and
directive contrast are described as follows.
1.
NON – SUBSAMPLED CONTOURLET
TRANSFORM (NSCT)
NSCT, based on the theory of CT, is a kind of multiscale and multi-direction computation framework of
the discrete images. It can be divided into two stages
including non-subsampled pyramid (NSP) and nonsubsampled directional filter bank (NSDFB). The
former stage ensures the multiscale property by using
two channel non-subsampled filter bank, and one lowfrequency image and one high-frequency image can be
produced at each NSP decomposition level. The
subsequent NSP decomposition stages are carried out
to decompose the low-frequency component available
iteratively to capture the singularities in the image. As
a result, NSP can result in k+1 sub-images, which
consists of one low- and high-frequency images having
the same size as the source image where denotes the
number of decomposition levels.
Fig 1. Gives the NSP decomposition with levels. The
NSDFB is two channel non subsampled filter banks
which are constructed by combining the directional fan
filter banks. NSDFB allows the direction
decomposition with stages in high-frequency images
from NSP at each scale and produces directional subimages with the same size as the source image.
Therefore, the NSDFB offers the NSCT with the multidirection property and provides us with more precise
directional details information. A four channel NSDFB
constructed with two-channel fan filter banks is
illustrated in Fig. 2.
2.
PHASE CONGRUENCY:
Phase congruency is a measure of feature perception in
the images which provides an illumination and contrast
invariant feature extraction method. This approach is
based on the Local Energy Model, which postulates
that significant features can be found at points in an
image where the Fourier components are maximally in
phase. First, logarithmic Gabor filter banks at different
discrete orientations are applied to the image and the
local amplitude and phase at a point (x,y) are obtained.
Methods Enriching Power and Energy Development (MEPED) 2014
389 | P a g e
International Journal for Research and Development in Engineering (IJRDE)
www.ijrde.com
ISSN: 2279-0500
Special Issue: pp- 388-393
The phase congruency, Pox,y is then calculated for each
orientation as Pox,y shown
where Woz,y is the weight factor based on the frequency
spread, Ao,nx,y and ɸo,nx,y are the respective amplitude
and phase for the scale n, ɸox,y is the weighted mean
phase, is a noise threshold constant and is a small
constant to avoid divisions by zero. The symbol +
denotes that the enclosed quantity is equal to itself
when the value is positive, and zero otherwise. Only
energy values that exceed T the estimated noise
influence and are counted in the result. The appropriate
noise threshold T is readily determined from the
statistics of the filter responses to the image. The main
properties, which acted as the motivation to use phase
congruency for multimodal fusion, are as follows.
The phase congruency is invariant to different
pixel intensity mappings. The images captured with
different modalities have significantly different pixel
mappings, even if the object is same. Therefore, a
feature that is free from pixel must be preferred.
The phase congruency feature is invariant to
illumination and contrast changes. The capturing
environment of different modalities varies and results
in the change of illumination and contrast.
Phase congruency provides the improved
localization of the image features, which lead to
efficient fusion.
whether the pixels are from clear parts or not.
Therefore, the directive contrast is integrated with the
sum-modified- Laplacian to get more accurate salient
features.
In general, the larger absolute values of high-frequency
coefficients correspond to the sharper brightness in the
image and lead to the salient features such as edges,
lines, region boundaries, and so on. However, these are
very sensitive to the noise and therefore, the noise will
be taken as the useful information and misinterpret the
actual information in the fused images. Hence, a proper
way to select high-frequency coefficients is necessary
to ensure better information interpretation. Hence, the
sum-modified-Laplacian is integrated with the directive
contrast in NSCT domain to produce accurate salient
features.
Mathematically, the directive contrast in NSCT domain
is given by:
Where SMLl,θ is the sum-modified-Laplacian of the
NSCT frequency bands at scale l and orientation θ. On
the other hand, IL (i, j) is the low-frequency sub-band
at the coarsest level (l). The sum-modified-Laplacian is
defined by following equation
Where
2.1. DIRECTIVE
CONTRAST
IN
NSCT
DOMAIN:
The contrast feature measures the difference
of the intensity value at some pixel from the
neighboring pixels. Local contrast is developed and is
defined as
In order to accommodate for possible
variations in the size of texture elements, a variable
spacing (step) between the pixels is used to compute
partial derivatives to obtain SML and is always equal
to 1. Hence, the directive contrast in NSCT domain is
given as.
Where L is the local luminance and LB is the luminance
of the local background. Generally, LB is regarded as
local low-frequency and hence, L - LB = LH is treated as
local high-frequency. This definition is further
extended as directive contrast for multimodal image
fusion. These contrast extensions take high-frequency
as the pixel value in multiresolution domain. However,
considering single pixel is insufficient to determine
Where α as a visual constant representing the slope of
the best-fitted lines through high-contrast data, which is
determined by physiological vision experiments, and it
ranges from 0.6 to 0.7. The proposed definition of
directive contrast, not only extract more useful features
from high-frequency coefficients but also effectively
Methods Enriching Power and Energy Development (MEPED) 2014
390 | P a g e
International Journal for Research and Development in Engineering (IJRDE)
www.ijrde.com
ISSN: 2279-0500
Special Issue: pp- 388-393
deflect noise to be transferred from high-frequency
coefficients to fused coefficients.
III.
PROPOSED IMAGE FUSION FRAME
WORK
Considering, two perfectly registered source images A
and B, the proposed image fusion approach consists of
the following steps:
1.
Perform l-level NSCT on the source images to
obtain one low-frequency and a series of highfrequency sub-images at each level and direction θ, i.e.,
A :{ClA , CAl,θ } and B:{ClB, CBl,θ}
*
Where Cl are the low-frequency sub-images and C*l,θ
represents the high frequency sub images at level lϵ
[1,l] in the orientation θ.
2.
Fusion of Low-frequency Sub-images: The
coefficients in the low-frequency sub-images represent
the approximation component of the source images. A
new criterion is proposed here based on the phase
congruency. The complete process is described as
follows.
a.
First, the features are extracted from lowfrequency sub-images using the phase congruency
extractor, denoted by PClA and PClB respectively.
b.
Fuse the low frequency sub-images as
3.
Fusion of High-frequency Sub-images: The
coefficients in the high-frequency sub-images usually
include details component of the source image. Noise
is also related to high-frequencies and may cause
miscalculation of sharpness value and therefore affect
the fusion performance. Therefore, a new criterion is
proposed here based on directive contrast. The whole
process is described as follows.
a.
First, the directive contrast for NSCT highfrequency sub-images at each scale is obtained and
4.
Perform l-level inverse NSCT on the fused
low-frequency (CFl) and high frequency (CFl,θ)
subimages, to get the fused image (F).
IV.
RESULTS AND DISCUSSION
Some general requirements for fusion algorithm are:
It should be able to extract complimentary
features from input images,
It must not introduce artifacts or
inconsistencies according to Human Visual System
It should be robust and reliable. Generally,
these can be evaluated subjectively or objectively.
IMAGE DATA SET 1:
Fig 3. Image data set – 1
RMSE = 0.0516
The psnr performance w.r.t. MRI is 33.88 dB
RMSE = 0.0211
The psnr performance w.r.t. CT is 41.66 dB
denoted by
and
at each level l ∈ [1, l] in
the direction θ.
b.
Fuse the high-frequency sub-images as
Fig 4. MR-CT Data Set
Methods Enriching Power and Energy Development (MEPED) 2014
391 | P a g e
International Journal for Research and Development in Engineering (IJRDE)
www.ijrde.com
ISSN: 2279-0500
Special Issue: pp- 388-393
coefficients which extract all prominent information
from the images and provide more natural output with
increased visual quality. Therefore, it can be concluded
that both the visual and statistical evaluation proves the
superiority of the proposed method over existing
methods.
Fig 5. Other transformation methods
Fig 6. NSCT fused image
From figure and table, it is clear that the proposed
algorithm not only preserves spectral information but
also improve the spatial detail information than the
existing algorithms, which can also be justified by the
obtained maximum values of evaluation indices. The
PCA algorithm gives baseline results. PCA based
methods give poor results relative to other algorithms.
This was expected because this method has no scale
selectivity therefore it cannot capture prominent
information localized in different scales. This
limitation is rectified in pyramid and multiresolution
based algorithms but on the cost of quality i.e., the
contrast of the fuse image is reduced which is greater in
pyramid based algorithms and comparatively less in
multiresolution based algorithms.
Among multiresolution based algorithms, the
algorithms based on NSCT performs better. This is due
to the fact that NSCT is an multi-scale geometric
analysis tool which utilizes the geometric regularity in
the image and provide asymptotic optimal
representation in the terms of better localization,multidirection and shift invariance. This is also justified by
the fact that shift-invariant decomposition overcomes
pseudo-Gibbs phenomena successfully and improves
the quality of the fused image around edges.
The main reason behind the better performance is the
proposed fusion rules for low- and high-frequency
CONCLUSION
V.
In the project, a novel image fusion framework is
proposed for multi-modal medical images, which is
based on non-subsampled contourlet transform and
directive contrast. For fusion, two different rules are
used by which more information can be preserved in
the fused image with improved quality. The low
frequency bands are fused by considering phase
congruency whereas directive contrast is adopted as the
fusion method for high-frequency bands. Three groups
of CT/MRI images are fused the proposed framework.
The visual and statistical comparisons demonstrate that
the proposed algorithm can enhance the details of the
fused image, and can improve the visual effect with
much less information distortion than its competitors.
These statistical assessment findings agree with the
visual assessment.
REFERENCES
[1]
F. Maes, D. Vandermeulen, and P. Suetens,
“Medical image registration using mutual information,” Proc.
IEEE, vol. 91, no. 10, pp. 1699–1721, Oct. 2003.
[2]
G. Bhatnagar, Q. M. J. Wu, and B. Raman, “Real
time human visual system based framework for image
fusion,” in Proc. Int. Conf. Signal and Image Processing,
Trois-Rivieres, Quebec, Canada, 2010, pp. 71–78.
[3]
Cardinali and G. P. Nason, “A statistical multiscale
approach to image segmentation and fusion,” in Proc. Int.
Conf. Information Fusion, Philadelphia, PA, USA, 2005, pp.
475–482.
[4]
P. S. Chavez and A. Y. Kwarteng, “Extracting
spectral contrast in Landsat thematic mapper image data
using
selective
principal
component
analysis,”
Photogrammetric Eng. Remote Sens., vol. 55, pp. 339–348,
1989.
[5]
Toet, L. V. Ruyven, and J. Velaton, “Merging
thermal and visual images by a contrast pyramid,” Opt. Eng.,
vol. 28, no. 7, pp. 789–792, 1989.
[6]
http://www.mathworks.in/products/matlab/descripti
on1.html
[7]
H. Li, B. S. Manjunath, and S. K. Mitra,
“Multisensor image fusion using the wavelet transform,”
Methods Enriching Power and Energy Development (MEPED) 2014
392 | P a g e
International Journal for Research and Development in Engineering (IJRDE)
www.ijrde.com
ISSN: 2279-0500
Special Issue: pp- 388-393
Graph Models Image Process., vol. 57, no. 3, pp. 235–245,
1995.
[8]
A. Toet, “Hierarchical image fusion,” Mach. Vision
Appl., vol. 3, no.1, pp. 1–11, 1990.
[9]
X. Qu, J. Yan, H. Xiao, and Z. Zhu, “Image fusion
algorithm based on spatial frequency-motivated pulse
coupled neural networks in nonsubsampled contourlet
transform domain,” Acta Automatica Sinica, vol. 34, no. 12,
pp. 1508–1514, 2008.
Methods Enriching Power and Energy Development (MEPED) 2014
393 | P a g e