International Journal of Engineering Trends and Technology (IJETT) – Volume 10 Number 5- April 2014
Interactively Reconstructing Low Resolution
Capsule Endoscopic Images
Shashank Singh#1, Tarun Gulati*2, Maninder Pal#3
#1
M.Tech Scholar ,*2 Associate Professor, #1 *2Department of Electronics & Communication Engineering,
Maharishi Markandeshwar University, Mullana
Ambala, Haryana, India
#3
Research Associate, NIBEC, University of Ulster, Jordanstown, Northern Ireland, UK
Abstract—This paper focuses on enhancement of low resolution
images taken by the camera of wireless capsule endoscopy. The
resolution of this camera is usually 256 by 256 pixels, which is
usually low and therefore needs enhancement. Image processing
for such low resolution digital images is very challenging. It is
because of the errors due to quantization and sampling. Over the
last several years, significant improvements have been made in this
area; however, it is still very challenging. Therefore, this paper
focuses on investigating the effect of interpolation functions on
increasing the resolution of endoscopic images. A DWT based
algorithm with error feedback is proposed and evaluated. The
proposed resolution enhancement technique uses DWT to
decompose the input image into different subbands. Then, the
high-frequency subband images and the input low-resolution
image have been interpolated, followed by combining all these
images to generate a new resolution-enhanced image by using
inverse DWT. The proposed technique has been tested on
endoscopic images. The calculate values of PSNR, MSE and
maximum error, and visual results show the superiority of the
proposed technique over the traditional image super-resolution
techniques.
(a)
Keywords - Super resolution; Up-sampling; Wavelet; Zooming;
Resizing; Interpolation and Magnification.
I. INTRODUCTION
This paper focuses on enhancement of low resolution
images taken by the camera of wireless capsule endoscopy.
Wireless capsule endoscopy of the digestive tract was firstly
started by Gabi Iddan and Paul Swain [1-2]. However, instead
of competing they decided to join defense services. In 1997,
Paul Swain swallowed the first wireless capsule endoscope in
Israel. The transmitted images were of poor quality but the
possibility of wireless transmission from the digestive tract to
an outside receiver (recorder) was proven to be possible [3-4].
This opened the way for a controlled study. The capsule
endoscope has a camera on one end and the radio transmitter
unit on the other end, as shown in Figure 1. The capsule can
enter the small bowel either with the camera or with the radio
transmitter leading. Since the small bowel is narrow, the
length of the capsule (27 mm) prevents it from turning around.
The capsule thus remains oriented in the same direction as it
enters the small bowel and transmits images that cover the
entire length of the small bowel. In endoscopy, the capsule has
to travel through the stomach and small bowel to reach the
colon.
ISSN: 2231-5381
(b)
Fig 1: Example of PillCam (size: 31 mm × 11 mm) used in Endoscopy [3].
While the capsule travels through the intestinal tract to reach
the colon, it transmits images. The colon usually has a much
wider diameter. This allows the capsule to flip around its own
axis. Therefore, the camera can change directions: at times the
front of the capsule with the camera may be leading and at
times the camera may be oriented in the opposite direction. So
with a standard capsule there are areas which would be
screened twice (when the capsule flips around its axis) and
areas that were not to be screened at all. The engineers solved
this problem by adding another camera, so that both ends of
the capsule transmit images. This promises that the entire
surface of the colon is screened no matter how many times the
capsule rotates around its own axis in the colon.
http://www.ijettjournal.org
Page 217
International Journal of Engineering Trends and Technology (IJETT) – Volume 10 Number 5- April 2014
During the journey, the surface of the colon can be covered
by debris and fecal material and if the colon is not perfectly
clean, the mucosa of the colon will not be visualized by the
capsule. The colon thus has to be vigorously cleansed before
the capsule is deployed. This bowel cleansing has to be
superior to the cleansing process applied for conventional
colonoscopy since no suction of liquid remnants is possible
during capsule endoscopy. Therefore, the quality of images
taken depends upon the cleanliness of the colon (Figure 2).
This mostly depends upon the individual patient and a high
level of cleanliness is often reported difficult to achieve [3-5].
In addition, because of low light, the images taken by
capsule’s camera are not of high quality; thus, the processing
on these images is required. The focal length of the endoscopy
camera is usually small, so it gives low resolution images and
their enhancement is necessary. The camera of wireless
capsule in endoscopy normally takes images with a resolution
of 256 by 256 pixels, as shown in Figure 3. For diagnosis
purposes, these images are usually considered as of low
resolution. For example, a small area of the obtained image in
Figure 3 is extracted for investigating the fine details;
however, when zoomed this extracted area then the results
(Figure 3) obtained are of poor quality because of very low
resolution. Therefore, it is preferred to apply interpolation
techniques to increase resolution. This paper is focused on
processing (in particularly zooming) the digital images
obtained in the endoscopy process.
Fig 2: Examples of digital images taken in endoscopy.
Fig 3: An example of image taken in wireless capsule in endoscopy [4].
The small part of the endoscopic image is zoomed but the quality is very poor.
II. THEORY OF INTERPOLATION
The interpolation based image processing algorithms can be
grouped into two categories: adaptive and non-adaptive.
Adaptive methods change depending on what they are
interpolating; whereas, non-adaptive methods treat all pixels
equally. Non adaptive interpolation algorithms interpolates by
fixed pattern for all pixels. These are usually easy to perform
and have low calculation cost. Adaptive interpolation
algorithm estimates lost pixel values using features of
surrounding pixels and can, therefore, get better images than
non adaptive algorithm. However, these algorithms usually
involve lots of calculations. Most of these algorithms apply on
a pixel-by-pixel basis. The key traditional interpolation
algorithms are: ideal filter or sinc function, nearest-neighbor,
ISSN: 2231-5381
Bilinear interpolation, and Bicubic interpolation. These are
discussed below.
1. Ideal filter or sinc function: The ideal interpolating
function would have constant one value in the pass band and
zero value within the stop band in frequency domain [5-8].
The ideal interpolating function, (sinc function), provides the
best performance; however it has infinite length in space
domain. Its synthesis function is as follows:
h(x)=
( )
(1)
It behaves like positive from 0 to 1, negative from 1 to 2,
positive from 2 to 3, and so on. This feature usually makes the
interpolation not practical because it is difficult to convolve
signal with such infinite function.
http://www.ijettjournal.org
Page 218
International Journal of Engineering Trends and Technology (IJETT) – Volume 10 Number 5- April 2014
2. Nearest-neighbor: The simplest interpolation method is the
nearest-neighbor method which has a rectangular shape in
space domain. It can be expressed as:
| |
(2)
ℎ( ) = 1 < 0.5
0
ℎ
The nearest-neighbor method is one of the most efficient
methods from the computation point of view but at the cost of
poor quality as can be observed from its frequency domain
[6-9]. It is because the Fourier transformation of a square
pulse is equivalent to a sinc function whose gain in the pass
band quickly falls off.
generated output pixel. Each weighing value is proportional to
distance from each existing pixel. This method has advantage
of simple calculation and output image using bilinear
interpolation is usually better than nearest neighbor replication
approach.
4. Bicubic interpolation: Bicubic interpolation is an
extension of spline interpolation for interpolating data points
on a two dimensional regular grid. This technique uses the
information from an original pixel and sixteen of the
surrounding pixels to determine the color of the new pixels
that are created from the original pixel. From literature, it is
formed that Bicubic interpolation offers significant
improvements over the previous two interpolation methods for
two reasons: (1) Bicubic interpolation uses data from a larger
number of pixels, and (2) Bicubic interpolation uses a more
sophisticated calculation system than the previous
interpolation methods. For two dimensional image processing,
the Bicubic interpolation is obtained by the product of two
cubic spline equations in directions x and y as mentioned in
equation 4. In equation 4,
is the bicubic spline function is
for each rectangular element and has 16 parameters obtained
by satisfying the 16 continuity equations on the corners of
each element. These equations are same as for 1-dimensional
cubic B-spline function.
3. Bilinear interpolation: The mathematical representation of
linear function is given by:
| | | |
(3)
ℎ( ) = 1 − < 1
0
ℎ
The linear interpolation has a triangle shape in space domain.
Analytically, bilinear interpolation is an extension of linear
interpolation for interpolating functions of two variables (e.g.,
x and y) on a regular grid. The bilinear interpolated function
uses the term xy, which is the bilinear form of x and y.
Bilinear interpolation considers the closest 2×2 neighborhood
of known pixel values surrounding the unknown pixel. It then
takes a weighted average of these 4 pixels to arrive at its final
interpolated value. This results in much smoother looking
images than nearest neighbor [10-11]. Weight is determined
by assigning weighted value of 4-nearest neighbor pixel to
( , )=
+
( −
( −
( − )+ ( − )+
)
−
+ ℎ ( − )
)
−
+ ( − )
+
( −
)
+
+
−
+
( −
( −
−
5. Lancoz interpolation: Lanczos interpolation has the best
properties in terms of detail preservation and minimal
generation of aliasing artifacts for geometric transformations
not
involving
strong
downsampling.
The Lanczos
interpolation function of order n in one dimension is given by
(5)
( ).
| | ≤
( ; > 0) =
0Otherwise
( , )=
1
−
) +
( − ) −
+
−
+ ( − )
−
+ ( − )
( − )
+
−
(4)
) +
−
−
Where, the normalized sinc function is:
1
=0
( ) = sin( )
Otherwise
(6)
Interpolation of a two-dimensional image f with a Lanczos
filter of order n is performed with the following algorithm:
(⌊ ⌋ + , ⌊ ⌋ + ). − + ⌊ ⌋; ). ( −
+ ⌊ ⌋; )
(7)
Where, (x, y) are the coordinates of the interpolation point and
⌊·⌋ is, as before, the floor operator (the largest integer less than
or equal to the argument). The filter weight w is applied by
division to preserve flux, as given by equation 8. Lanczos
=
interpolation uses a neighborhood of the 2n×2n nearest
mapped pixels. A two-dimensional Lanczos filter is
nonseparable, so the complexity of Lanczos interpolation is
O(N×4n2).
(8)
− + ⌊ ⌋; ). ( − + ⌊ ⌋; ) For image processing, the one-dimensional interpolating
functions need to be transformed into two-dimensional
functions. The general approach is to define a separable
interpolation function as the product of two one-dimensional
functions. In two dimensional cases, the interpolation step
must reconstruct a two dimensional (2-D) continuous signal
ISSN: 2231-5381
s(x, y) from its discrete sample s(k ,l ) with s, x, y є R and k, l є
N. Thus, the amplitude at any position (x, y) needs to be
estimated first from its discrete neighbors. This can be
described formally as the convolution of the discrete image
samples with the continuous 2-D impulse response h(x, y) of a
2-D reconstruction filter.
http://www.ijettjournal.org
Page 219
International Journal of Engineering Trends and Technology (IJETT) – Volume 10 Number 5- April 2014
( , )=
( , )ℎ( − , − )
(9)
Usually, symmetrical and separable interpolation kernels are
used to reduce the computational complexity.
III.
SYSTEM MODEL
High-frequency components (i.e. the edges) are main loss of
an image after being super-resolved by applying interpolation.
This loss occurs due to the smoothing caused by interpolation.
To increase quality of the super-resolved image, preserving
the edges is essential. Therefore, Gholamreza Anbarjafari and
Hasan Demirel proposed a super-resolution technique based
on interpolation of the high-frequency sub-band images
obtained by discrete wavelet transform (DWT) of the input
image, as shown in Figure 4. This technique uses DWT to
decompose an image into different subband images; namely,
low-low (LL), low-high (LH), high-low (HL), and high-high
(HH). These subband images contain the high-frequency
components of the input image. In this technique, the
interpolation is applied to high-frequency subband images as
well as the input image. Finally, the IDWT of the interpolated
subband images and the input image produce the final highresolution output image. In this technique, the employed
interpolation method is same for all subband and input
functions are the two important factors in determining quality
of the super-resolved images. The benefit in this scheme is
that this super-resolution process considers the higher and
lower frequency components into considerations; which is not
taken into account in direct interpolation. Also it is known
that in the wavelet domain, lowpass filtering of the high
resolution image produce the low resolution image. In other
words, low frequency subband is the low resolution of the
original image. Therefore, instead of using low frequency
subband, which contains less information than the original
high resolution image, Hasan Demirel and Gholamreza
Anbarjafari used the input image for the interpolation of low
frequency subband image. The quality of the super resolved
image increases using input image instead of low frequency
subband. However, even this approach is reported to give
jaggies and blurriness in the super resolved images. Therefore,
the research proposed in this paper focuses to use an error
routine to reduce the artifacts produced in images when
zoomed using wavelet based techniques. This is discussed
below.
Fig 4: Block diagram of DWT-SR method.
In medical images, the accuracy and sharpness of edges is
very essential. For example, in endoscopic images, it is highly
desirable to gain information about the disorders/diseases in
track through which the wireless endoscopic capsule passes.
These disorders/diseases usually have similar color as that of
the normal track; however, have sharp edges. Therefore, it is
desirable to have a zooming operation; which maintains the
sharpness of the edges. Thus in the proposed methodology, a
feedback error routine is developed as shown in Figure 5. In
the developed methodology, the image is super-resolved over
an interval of zoom factor of 2. At each stage, the error is
computed with respect to the original Bicubic interpolated
image and the obtained error is added back to the DWT super
resolved image at each stage. The idea of error routine is that
ISSN: 2231-5381
when the interpolation operation is performed on high
frequency components obtained using the DWT transform,
then the interpolation will smooth out these components.
Although, the results obtained using the DWT based superresolution techniques is better than the traditional techniques;
yet, it can be further enhanced. For this reason, the
interpolation is performed in steps of 2; and at each stage the
error is calculated with respect to the original super-resolved
image using interpolation (Bicubic Interpolation in this paper).
This error is then added to the input image of the next stage.
In this way, the error can be reduced and a better quality
image can be obtained. However, this method has a drawback
of time consuming and the zooming/super-resolution factor
should be a multiple of order of 2.
http://www.ijettjournal.org
Page 220
International Journal of Engineering Trends and Technology (IJETT) – Volume 10 Number 5- April 2014
Fig 5: Block diagram of proposed DWT SR method with error routine.
IV. RESULTS & DISCUSSION
In order to evaluate the performance of the proposed
algorithm with the nearest neighbor, Bilinear, Bicubic and
Lancoz interpolation techniques, the same are implemented in
Matlab. As required for zooming endoscopic images, the
mentioned techniques are implemented in Matlab and are
applied on endoscopic low resolution images. The results
obtained are shown in Figures 6 to 9. In these cases, the input
image is first downsampled by a factor of 2 and then zoomed
back to original image using the nearest neighbor, Bilinear,
Bicubic, Lancoz and the proposed algorithm. Three
quantitative parameters (PSNR, MSE and Maximum Error) is
used to evaluate the performance. The quality of the images
produced is also evaluated visually. The key conclusions
made from the results obtained are:
1. PSNR: The PSNR of the zoomed images is computed with
respect to the original image and is tabulated in Tables 1 & 2.
From the results, it is found that the proposed algorithm offers
the best performance, which confirms the theoretical results as
well. For all the images used for evaluating the performance
of the mentioned algorithms, a minimum of 10% increase in
PSNR from the Bicubic and Lancoz functions; which are
found to have similar performances. So, it is concluded that
ISSN: 2231-5381
for zooming low resolution images it is advisable to use the
proposed algorithm.
2. MSE: The MSE of the zoomed images is also evaluated
with respect to the original images. A low value of MSE is
always preferred and ideally it should be zero. The proposed
algorithms gives a reduction of minimum of 20% of MSE
compared to the highest value obtained for nearest neighbor.
3. Maximum error: The term maximum error refers to the
maximum value of the difference between the values of same
pixel in two images. The value of maximum error will vary in
between 0 to 255. Ideally zero, but practically a closet value to
zero is always desirable. In the results obtained, the proposed
algorithm gives the minimum value of maximum error. Thus,
the proposed algorithm gives the best performance.
4. Visual quality appearance: To investigate the visual
quality appearance, a small portion of the input images is
selected and is zoomed with a factor of 4 using the nearest
neighbor, Bilinear, Bicubic, Lancoz and the proposed
algorithm. The results obtained are shown in Figures 8 to 10.
It can be seen that the proposed algorithm gives a sharp and
clear edges as desired. Thus, the proposed algorithm gives the
best performance compared to nearest neighbor, Bilinear,
Bicubic and Lancoz interpolation functions.
http://www.ijettjournal.org
Page 221
International Journal of Engineering Trends and Technology (IJETT) – Volume 10 Number 5- April 2014
Table 1: PSNR, MSE & Max. error corresponding to results in Figure 6.
PSNR
MSE
Max. Error
Nearest
Neighbour
29.668
70.1903
153
Bilinear
Bicubic
Lancoz
31.2114
49.1973
116
31.1411
50.0001
129
31.1339
50.0829
128
Original Image
Proposed
Algorithm
40.8199
5.3838
83
Reduced Image
Nearest Neighbor
Bicubic
Lanczos
Bilinear
50
PSNR (dB)
40
30
20
10
0
Nearest Bilinear
Neighbour
Proposed
Bicubic
Lancoz
Proposed
Algorithm
PSNR
Fig 6: An endoscopic image of Crohn disease in bowel is reduced to 50% of its original size and then zoomed back to its original size using
nearest neighbor, Bilinear, Bicubic, Lancoz and proposed algorithm.
ISSN: 2231-5381
http://www.ijettjournal.org
Page 222
International Journal of Engineering Trends and Technology (IJETT) – Volume 10 Number 5- April 2014
Table 2: PSNR, MSE & Max. error corresponding to results in Figure 7.
PSNR
MSE
Max. Error
Nearest
Neighbour
22.4117
373.1754
193
Bilinear
Bicubic
Lancoz
23.9536
261.6505
154
23.6708
279.2575
174
23.646
280.8522
176
Proposed
Algorithm
34.9939
20.5918
132
Original Image
Reduced Image
Nearest Neighbor
Bilinear
Bicubic
Lanczos
PSNR (dB)
40
30
20
10
0
Nearest Bilinear
Neighbour
Proposed
Bicubic
Lancoz Proposed
Algorithm
PSNR
Fig 7: An endoscopic image showing colonic metastasis is reduced to 50% of its original size and then zoomed back to its original size using nearest neighbor,
Bilinear, Bicubic, Lancoz and proposed algorithm.
Original Image
Nearest Neighbor
Bilinear
Fig 8: Zooming 4 times a small portion of the endoscopic image of oesophageal cancer using nearest neighbour & Bilinear algorithm.
ISSN: 2231-5381
http://www.ijettjournal.org
Page 223
International Journal of Engineering Trends and Technology (IJETT) – Volume 10 Number 5- April 2014
Bicubic
Lanczos
Proposed
Fig 9: Zooming 4 times a small portion of the endoscopic image of oesophageal cancer using Bicubic, Lancoz and proposed algorithm.
Original Image
Nearest Neighbor
Bilinear
Bicubic
Lanczos
Proposed
Fig 10: Zooming 4 times a small portion of the endoscopic image of Crohn disease in bowel using nearest neighbor, Bilinear, Bicubic, Lancoz
and proposed algorithm.
IV.
CONCLUSION
REFERENCES
This paper presented the results obtained on applying the
DWT and error routine based proposed algorithm on low
resolution endoscopic images. In addition to the proposed
function, the nearest neighbor, Bilinear, Bicubic and Lancoz
interpolation function is also applied. These functions are
evaluated on various endoscopic images for a zoom factor
of 2, 4, etc. The results obtained are evaluated both visually
and analytically in terms of PSNR, MSE and maximum
error. It is observed that the visual results of proposed
algorithm are better as compared to nearest neighbor,
bilinear, bicubic and Lancoz interpolation functions. A
minimum 10% increase in PSNR is obtained for the images
zoomed using the proposed algorithm as compared to
nearest neighbor, Bilinear, Bicubic and Lancoz interpolation
functions. Whereas, a minimum 10% decrease in the value
of MSE and a minimum 20% decrease in maximum error is
noticed for the proposed function. However, the proposed
function is found computationally expensive. In nutshell,
the proposed algorithm preserves the sharpness of edges in
zoomed images and is more suitable for zooming
endoscopic images compared to traditional nearest neighbor,
Bilinear, Bicubic and Lancoz interpolation functions.
ISSN: 2231-5381
[1]
Iddan G., Meron G., Glukhovsky A., Swain P. (2000), “Wireless
capsule endoscopy”, Nature, vol. 405: pp. 417–417.
[2]
Iddan G.J., Swain C.P. (2004), “History and development of
capsule endoscopy”, Gastrointest Endosc Clin N Am, vol. 14,
pp. 1–9.
[3]
Lewis B.S., Swain P. (2002), “Capsule endoscopy in the
evaluation of patients with suspected small intestinal bleeding:
Results of a pilot study”, Gastrointest Endosc, vol. 56: 349–353.
[4]
Liu Jing, Gan Zongliang and Zhu Xiuchang (2013), “Directional
Bicubic Interpolation - A New Method of Image SuperResolution”, Published by Atlantis Press, pp. 470-477.
[5]
Wonseok Kang, Jaehwan Jeon, Eunsung Lee, Changhun Cho,
Junghoon Jung, Taechan Kim, Aggelos K. Katsaggelosy and
Joonki Paik (2013), “Real-time super-resolution for digital
zooming using finite Kernel-based edge orientation estimation
and truncated image restoration”, IEEE International Conference
on Consumer Electronics, pp. 1311-1315.
[6]
Wonseok Kang, Jaehwan Jeon, Eunjung Chae, Minkyu Park, and
Joonki Paik (2013), “Real-Time Digital Zooming for Mobile
Consumer Cameras Using Directionally Adaptive Image
Interpolation and Restoration”, IEEE International Conference on
Consumer Electronics, pp. 294-295.
http://www.ijettjournal.org
Page 224
International Journal of Engineering Trends and Technology (IJETT) – Volume 10 Number 5- April 2014
[7]
Yu-Chen Hu, Bing-Hwang Su, Wu-Lin Chen and Wan-Yu Lu
(2012), “Image Zooming for Indexed Color Images Based on
Bilinear Interpolation”, International Journal of Multimedia and
Ubiquitous Engineering, Vol. 7, No. 2.
[8]
J. W. Han, J. H. Kim, S. H. Cheon & S. J. Ko (2010), “A Novel
Image Interpolation Method Using the Bilateral Filter,” ICASSP89, pp. 811-816.
[9]
M. Hajizadeh, M. S. Helfroush & A. Tashk (2009), “Improvement
of Image Zooming Using Least Directional Differences based on
Linear and Cubic Interpolation,” ICASSP-91, pp. 365-369.
[10]
M. Zamani (2009), “An Investigation of B-spline and Bezier
Methods for Interpolation of Data,” Contemporary Engineering
Sciences, Vol. 2, No. 8, pp. 361-371.
[11]
X. Zhang & X. Wu (2008), “Image Interpolation by Adaptive 2-D
Autoregressive Modeling and Soft-Decision Estimation,” IEEE
Transactions on Image Processing, Vol. 17, No. 6, pp. 887-896.
[12]
L. Zhiwei, Z. Min & W. Jiechao (2006), “An Image Zooming
Technique Based on the Relative Color Difference of Pixels,”
IEEE Transactions on Image Processing, Vol. 15, No.2, 2006.
ISSN: 2231-5381
http://www.ijettjournal.org
Page 225