Introduction to Image Fusion
There have been geographic studies that combine the principle component transformation with
image fusion for geographic purposes. In these studies, the images were broken into different
color bands that served as the principle components. [1-6] The chemical image map produced by
ToF-SIMS uses the intensity of brightness as a means to represent the quantitative information.
Using some examples from geography, there are two or three methods that combine the use of
the principle component transform with that of image fusion. Deciding which method is
appropriate depends on the data. When the objective is to create a hybrid image that would
quantify information, there are certain assumptions that must be made. There has to be a degree
of correlation present between the multi-spectral band and the brightness on the intensity scale.
The method results in the formation of super-pixels that have the same field of view as the
original multi-spectral pixels. Each super-pixel is comprised of sub-pixels that are re-sampled at
the same field of view. The equation used to calculate the hybrid, high-resolution pixels’
brightness is:
𝑁
′
𝐷𝐶𝑖𝑎𝑣𝑔
𝐷𝐶𝑝𝑗
𝐷𝐶𝑖
=(
)∑
= 𝐷𝐶𝑖
𝐷𝐶𝑝𝑎𝑣𝑔
𝑁
(1)
𝑗=1
′
𝐷𝐶𝑖′ is the resolution value given to the hybrid sub-pixels of any spectral band. 𝐷𝐶𝑖𝑎𝑣𝑔
is the sum
of all the values given to all the hybrid pixels, divided by the number of sub-pixels in the superpixel. 𝐷𝐶𝑖 is the resolution value given to the super-pixel. 𝐷𝐶𝑝𝑎𝑣𝑔 is the mean digital count in the
peak of interest for the high-resolution pixels of interest. [1]
Low
Resolution,
High
Information
Image
Fusion
High
Resolution,
Low
Information
Hybrid Image
Supplemental Figure 1: Schematic of Image Fusion
FLIR Image
Radar Image
Fused
Supplemental Figure 2: Image fusion satellite and radar image example. See reference
for more details and basic advantages for this fusion technique. [8]
(a)
(b)
(c)
Supplemental Figure 3: A low resolution multi-spectral image (RIS-1C LISS-III) with a
resolution of 23.5 meters (a), a high resolution panchromatic image (IRS-1C PAN) with a
resolution of 5.8 meters (b), and a hybrid image that is the result of PCA applied to the
image fusion of (a) and (b). [9]
Supplemental figure 3 is an example of image fusion done in combination with a common
multivariate statistic technique, PCA. This example shows a common application in satellite
imaging while also showing some of the adverse effects of the pre-treatment and normalization
necessary to perform PCA. (For more specific details see reference [9].)
1.7 Introduction to Fast Fourier Transform (FFT)
Fast Fourier Transform (FFT) is an algorithm developed by Cooley and Tukey. [3, 4] This
algorithm increased the speed of the typical Discrete Fourier Transform (DFT), which was
extremely useful in transforming data into frequency domains. However, DFTs produce matrices
with large amounts of redundancy, due to the symmetry and periodicity in the FT code matrix.
[3] The equation for the DFT is as follows: [4]
𝑁−1
𝐹(𝑣𝑚 ) = ∑ 𝑓(𝑡𝑛 )
(2)
𝑁=0
Here, 𝐹(𝑣𝑚 ) is the frequency-domain (complex) spectral data point, from N (real or complex)
time-domain data points [f(tn), n=0, 1, 2, … n-1], and where 𝐹𝑛𝑚 = 𝑒𝑥𝑝 (
−𝑖2𝜋𝑛𝑚
𝑁
), with Cooley
and Tukey’s adaptation, the N-point time-domain, is divided in two halves, and the discrete
Fourier transform is performed on each half. The results are then recombined by multiplication,
such that each subset is reduced to two data points. With this alteration, Equation 2 becomes the
following equation: [4]
𝑁−1
𝐹(𝑚) = ∑ 𝑁𝐹
𝑛𝑚𝑡(𝑛)
;
n = 0,1,2, … , N − 1
(3)
𝑛=0
The index, N, has been introduced into the Fourier “code” element NFnm, in order to specify that
discrete FT is applied to an N-point data set. [4]
4
1.0
0.8
0.6
0.4
0.2
185
190
195
200
Mass (u)
(a)
TaO
1e3
1e2
1e-1
^180Ta^18O
1e0
196
197
198
Ta^18O
1e1
^180TaO
Intensity (counts)
x10
199
(b)
Supplemental Figure 4: (a) Negative ion spectrum produced by the ToF-SIMS from the Cameca
grid sample. (b) ratio produced from the characteristic Ta oxide components.
4
2.0
1.5
1.0
0.5
170
175
180
185
190
Mass (u)
(a)
Ta
1e3
1e2
1e1
1e0
^180Ta
Intensity (counts)
x10
1e-1
180.50
180.75
181.00
181.25
181.50
(b)
Supplemental Figure 5: (a) The positive ion spectra produced by the ToF-SIMS from the Cameca
grid sample. (b) The ratio from the representative isotopes.
(a)
(b)
Supplemental Figure 6: FFTs of SIMS and
SEM Images. (a) The FFT of the SIMS image.
(b) The FFT of the SEM image.
(a)
(b)
(c)
(d)
(e)
(f)
Supplemental Figure 7: Segments of the FFT corresponding to
characteristic portions of the A-grid ToF-SIMS image (a, c, e), and
their respective inverse FFT segments (b, d, f).
(a)
(b)
(c)
(d)
(e)
(f)
Supplemental Figure 8. Segments of the FFT corresponding to characteristic
portions of the A-grid SEM image (a, c, e), and their respective inverse FFT
segments (b, d, f).
a
b
c
d
Supplemental Figure 9: Inverse FFT Analysis of Background and Edge Effects.
(a) The SIMS FFT of the background / edge effect. (b) The SIMS Inverse FFT of
the background / edge effect. (c) The SEM FFT of the background / edge effect.
(d) The SEM Inverse FFT of the background / edge effect.
In supplemental figure 7 and 8, an inverse FFT analysis was performed to validate the FFT
results in supplemental figure 6. The inverse FFT analysis allowed specific frequency domains
from the FFT (see supplemental figure 7(a,c,e) and 8(a,c,e)) to be selected and analyzed. Thus,
this technique provided validation by demonstrating that the specific domains were characteristic
of the same portion of the letter "A" in both the SEM and ToF-SIMS images. For both the SEM
and ToF-SIMS images a distinct frequency domain could be attributed to edge and noise and is
imaged in figure supplemental figure 9. As can been seen in Supplemental figure 7 and 8 each
distinct portion of the letter "A" is identified. For example, Figure 7e shows the bright vertical
band (frequency domain) found in the FFT of the ToF-SIMS image. Figure 7f shows the
corresponding inverse FFT, which clearly shows the crossbar of the letter "A" in the ToF-SIMS
image. By comparing this portion of both the SEM and ToF-SIMS FFTs, which clearly display
the crossbar feature of the A-grid, the width discrepancy discussed above was again apparent.
There was one additional feature found in both FFT images (Supplemental figure 9,
Supplemental Information). When the inverse of this feature was examined, it could not be
attributed to any portion of the letter “A”, however it is related to edge effects and background
present in both images. With the results from FFT and inverse FFT analysis the expected spatial
correlation that should be present when using two types of analyses to obtain images from the
same copper TEM A-Grid was evident.
References
1) Schott, J. R., Remote Sensing: The Image Chain Approach. Oxford University Press: New
York, 1997.
2) Sodhi, R. N. S., Time-of-flight secondary ion mass spectrometry (ToF-SIMS):- versatility in
chemical and imaging surface analysis. Analyst, 2004, 129, 483-487.
3) Bracewell, R.N. The Fourier Transform and Its Applications. 2nd Ed. McGraw-Hill, Inc.;
1986.
4) Marshall, A.G. and Verdun, F.R. Fourier Transforms in NMR, Optical, and Mass
Spectrometry: A User’s Handbook. Elsevier, 1990.
5) Sanjeevi, S. , Vani, K., Lakshmi, K., Comparison of Conventional and Wavelet Transform
Techniques for Fusion of IRS-IC LISS-III and PAN Images; Remote Sensing Conference,
2001.
6) Schetslelaar, Earnst M. On preserving spectral balance in image fusion and its advantages
for geological image interpretation. Photogramm Eng Rem S, 2001, 67, 925-934.
7) Webster, R.; Oliver, M., Geostatistics for Environmental Scientist. John Wiley & Sons: New
York, 2001.
8) http://vision.arc.nasa.gov/personnel/al/hsr/fusion/fr.gif
9) Sanjeevi, S. , Vani, K., Lakshmi, K., Comparison of Conventional and Wavelet Transform
Techniques for Fusion of IRS-IC LISS-III and PAN Images; Remote Sensing Conference,
2001.
10) http://www.answers.com/topic/correlation-coefficient
11) http://www.geovariances.com/software/exploratory-data-analysis-ar0065.html
12) Goovaerts, P., Geostatistics in soil science: state-of-the-art and perspectives. Geoderma
1999, 89, 1-45.
13) Goovaerts, P. Geostatistics for Natural Resources Evaluation; Applied Geostatistics Series;
Oxford University Press: New York: NY, 1997; pp 1-483.