2022 International Conference on Big Data, Information and Computer Network (BDICN) | 978-1-6654-8476-3/22/$31.00 ©2022 IEEE | DOI: 10.1109/BDICN55575.2022.00137
2022 International Conference on Big Data, Information and Computer Network (BDICN)
A Design of Optimized Colour Image Interpolation
Algorithm Based on Edge Gradient
Runchan Ge
Harbin Institute of Technology
Harbin, Heilongjiang, China
1170300808@hit.edu.cn
and Kriging are more comprehensive in terms of detail
processing [8]. In addition, the merit of algorithms depends on
different evaluation metrics. Not only considering quality,
Parsania P S et al. compared seven algorithms and used the
computation time of the algorithm as a tool to measure the
merit of interpolation algorithms [9]. Pan et al introduced
CPSNR (colour peak signal-to-noise ratio) and SSIM
(Structure Similarity) to evaluate the merit of interpolation
algorithms [10].
Abstract—Most of the popular digital cameras are equipped
with image sensors of Complementary Metal Oxide
Semiconductor(CMOS), in which the Bayer filter is used to store
the information of the image. The original image obtained from
Bayer filter is then restored to colour by interpolation algorithms.
However, there has been low quality with recovered image. In
order to solve this problem, an optimization based on the edge
gradient interpolation algorithm is proposed in this paper. Firstly,
more accurate edges for interpolation are obtained by reoptimizing the calculation of the horizontal and vertical gradients.
Then, the missing channels are recovered by the method of intraneighbourhood averaging, mainly 3x3 neighbourhood. Several
sets of comparison experiments based on MATLAB were
conducted and the colour peak signal-to-noise ratio was
introduced to evaluate the image quality. It has proved that the
optimized algorithm proposed in this paper recovers colour
images with higher quality compared to the traditional edge
gradient interpolation algorithm
Keywords-Chromatic
interpolation
aberration,
colour
image,
In summary, the traditional edge gradient interpolation
algorithms are simple in determining the direction of the edge,
which makes the image edge judgment not very accurate.
Meanwhile, the quality of the recovered image by adaptive
weighted interpolation and other methods has certain
improvements, but the complexity of the algorithm is high,
which may have limitations in practical applications. In view of
the above problems, this paper improves and optimizes the
edge gradient interpolation algorithm to recover colour images
with higher quality while the computational complexity is close
to that of the traditional edge gradient interpolation algorithm.
gradient
I. INTRODUCTION
II.GRADIENT INTERPOLATION OPTIMIZATION ALGORITHM
DESIGN
Interpolation algorithms are a class of algorithms used to
recover a colourful image from the original image obtained
from the Bayer filter, including bilinear interpolation algorithm,
COK algorithm and edge gradient interpolation algorithm (also
known as Hibbard algorithm) [1], but all the algorithms above
have problems such as edge blurring and moire fringes. To
solve the problem, Sajid Khan et al proposed gradient-based
edge direction estimation, which found edges from images and
distinguishes them into strong and weak edges by hysteresis
thresholding and nonmaximal suppression methods, and used
different interpolation methods for both to obtain better colour
image edges [2]. Hwang J W et al applied the inverse gradient
to the bilinear and Bicubic algorithms, resulting in better image
recovery at the edges and irrespective of the magnification
factor [3]. Monno Y and J.Wu et al effectively improved the
image sharpness by image demosaicing [4-6].
A.Edge Gradient Interpolation Algorithm
In order to recover the colour image, there are two main
steps in the edge gradient interpolation algorithm: First the
green channel on the red and blue pixel points are recovered, so
that all pixel points have a green channel, which is then used to
recover the missing red or blue channel of all pixel points. The
channels obtained from the Bayer filter are shown in Figure 1.
Various interpolation algorithms were experimented and
compared for their advantages and disadvantages. Lu compared
the gradient interpolation with the traditional bilinear
interpolation algorithm and the COK algorithm [7]. By
comparing the false colour, noise and colour moire fringes of
colour images obtained by the three types of algorithms, it was
proved that the edge gradient interpolation has the least moire
fringes and noise. Fadnavis compared seven algorithms and
proved that the Bicubic algorithm has better results and DWT
978-1-6654-8476-3/22/$31.00 ©2022 IEEE
DOI 10.1109/BDICN55575.2022.00137
Figure 1. Bayer filter.
The edge gradient interpolation algorithm introduces a
horizontal gradient α and a vertical gradient β. Taking a red
pixel point as an example, the Hibbard algorithm [1] calculates
the gradient by using the green channel within the 3x3
708
Authorized licensed use limited to: UNIV OF ENGINEERING AND TECHNOLOGY TAXILA. Downloaded on March 17,2023 at 05:23:46 UTC from IEEE Xplore. Restrictions apply.
neighborhood, while the Laroche algorithm [1] does that by
using the red channel within the 5x5 neighborhood. The
gradients of the two algorithms have been calculated as follows.
(1)
(2)
(3)
(4)
The larger the value of the gradient, the greater the
possibility that a boundary exists. Therefore, when α > β, the
greater the possibility of the existence of a boundary in the
horizontal direction, then the interpolation should be carried out
along the vertical direction as follows.
(10)
So far, the colour image has been successfully recovered.
However, the conventional interpolation algorithm is not very
accurate in judging the gradient, so the quality of the recovered
colour image has rooms to improve.
(5)
And when α < β, the greater the probability that a boundary
exists in the vertical direction, then the interpolation should be
carried out along the horizontal direction as follows.
B.Algorithm Optimization Design
To address the shortcomings of the traditional interpolation
algorithm, this paper attempts to improve and optimize the
gradient calculation method in the process of the green channel
recovery. The algorithms of Hibbard and Laroche have been
combined to consider both methods of gradient computation.
Take a red pixel point as an example:
(6)
The gradient is calculated for all red and blue pixel points
in the image except those at the boundary, then the
interpolation calculation is carried out to get their
corresponding green channels. For the red and blue pixel points
at the boundary, the mean value of the green channels in the
3x3 neighborhood can be used as the value of theirs.
The conventional interpolation algorithm utilizes the idea of
constant chromatic aberration to recover the red and blue
channels. The constant chromatic aberration is generally
defined as the chromatic aberration between red and green(R-G)
and between blue and green(B-G), which remain constant
within a small smooth area of the image.
(7)
(11)
(12)
In equations above, the former represents the gradient
difference of adjacent green pixel points, while the latter
represents the gradient difference of adjacent red pixel points.
The reason for combining the two types of gradient calculation
is that different images are sensitive to different channels. For
example, an image with a predominantly blue colour may have
very little red channel, which is naturally not very accurate
when used to determine the gradient. The calculation of blue
pixel points is the same as the red pixel point calculation
example above.
(8)
Where denotes the size of the neighborhood, which is a
square
with side length
q
. For instance, in terms of
, the red channel of a green pixel point is:
(9)
Since the red and blue pixels are not adjacent to each other
in the Bayer filter, the red and blue channels of the green pixels
are recovered first. After recovering all the channels of the
green pixel points, the blue channel of the red pixel points and
the red channel of the blue pixel points can be recovered.
Taking the former as an example, using the idea of constant
chromatic aberration:
And when it comes to recovering the missing channels of
red and blue pixel points, All the 3x3 neighborhoods of the
pixel points have been used for the recovery of the missing
channels. Take the recovery of the blue channels of red pixel
points as an example:
709
Authorized licensed use limited to: UNIV OF ENGINEERING AND TECHNOLOGY TAXILA. Downloaded on March 17,2023 at 05:23:46 UTC from IEEE Xplore. Restrictions apply.
(13)
Similarly, the red channels of the blue pixel points have
been recovered in the same way.
The flow chart of the optimization algorithm is shown in
Figure3.
The horizontal and vertical gradients have been first
calculated, and then all the missing green channels have been
calculated by interpolating in the direction of the smaller
gradient. Next, the traditional idea of constant chromatic
aberration has been utilized to recover the red and blue
channels of the green pixel points. Eventually, when recovering
the missing red and blue channels of the remaining pixel points,
the traditional colour interpolation algorithm considers that the
blue components of the green pixel points around the red ones
are interpolated and calculated, so that reuse for the recovery of
the blue channels of the red pixel points will reduce the
accuracy. However, the recovery of green pixel points is
considered more comprehensively than the traditional
interpolation algorithm, so the method of mean value within
3x3 neighborhood has been finally used for recovery.
Figure 2. Algorithm Procedure.
III.ALGORITHM IMPLEMENTATION
The design of the above optimized algorithm has been
implemented based on MATLAB programming, and the
program structure is shown in figure 3.
Figure 3. Diagram of Optimized Colour Image Implementation Algorithm.
In order to simulate the operation of the Bayer filter, the
colour image has been firstly loaded in. For different locations,
the Bayer filter template has been used for filtering to simulate
the original image obtained by the camera. Then, for all red and
blue pixels, the current gradient value has been calculated to
determine the interpolation direction, and the mean value has
been used to recover the green channel. After that, for the green
pixels, the mean value of the chromatic aberration of the red
pixels in its 3x3 neighborhood has been used to recover the
missing red channel, and the same for the blue channel. Finally,
the mean value of the chromatic aberration in the 3x3
neighborhood has been utilized again to recover the missing
blue channel of the red pixels and the missing red channel of
the blue pixels.
and lines, which is a typical image for recovery and can reflect
the overall accuracy.
The results obtained from the first set of comparison
experiments are shown in Figure 4 and Figure 5. ‘Hibbard1’
indicates the colour image recovered after applying the Hibbard
algorithm, while ‘Modified1’ indicates the colour image
recovered after applying the improved algorithm.
In order to show the improvement more clearly, the
zoomed-in local images are shown in Figure 6 and Figure 7. It
can be seen that the improved algorithm has a significant
decrease in the number of noise points at the edges compared
to the traditional Hibbard algorithm.
The results obtained from the second set of comparison
experiments are shown in Figure 8 and Figure 9. ‘Hibbard2’
indicates the colour image recovered after applying the Hibbard
algorithm, while ‘Modified2’ indicates the colour image
recovered after applying the improved algorithm.
IV.RESULTS AND DISCUSSIONS
In this paper, two images have been selected for the
comparison. The first image has extreme sharp edges, which is
eligible to test whether the algorithm processes edges better.
The second image contains numbers, letters, various colours
Similarly, the zoomed-in local images are shown in Figure
10 and Figure 11 It can be seen that the optimized algorithm
710
Authorized licensed use limited to: UNIV OF ENGINEERING AND TECHNOLOGY TAXILA. Downloaded on March 17,2023 at 05:23:46 UTC from IEEE Xplore. Restrictions apply.
gets the noise reduction at the edges of the black lines, same as
above.
Figure 4. Hibbard1.
Figure 7. Modified1-zoomed.
Figure 5. Modified1.
Figure 8. Hibbard2.
Figure 6. Hibbard1-zoomed.
Figure 9. Modified2.
711
Authorized licensed use limited to: UNIV OF ENGINEERING AND TECHNOLOGY TAXILA. Downloaded on March 17,2023 at 05:23:46 UTC from IEEE Xplore. Restrictions apply.
recovered image. P represents the current peak, e.g., a value of
255 for an 8-bit image per channel.
The running time of the MATLAB program has also been
introduced for comparison, which is estimated by the inner
function of MATLAB.
TABLE I. CPSNR AND RUNNING TIME
Images
CPSNR
Running Time/s
Figure 4
0.70668
Figure 4
0.76166
Figure 8
0.22064
Figure 9
0.22189
Experiments have shown that the CPSNR value of the
improved algorithm has a certain increase compared with the
traditional interpolation algorithm, which indicates the higher
quality of the recovered images. Meanwhile, the stability of the
algorithm complexity has been proved by the nearly constant
running time of the MATLAB program.
Figure 10. Hibbard2-zoomed.
REFERENCES
[1]Battiato, S., Bruna, A. R., Messina, G., & Puglisi, G. Image processing for
embedded devices[M]. Bentham Science Publishers, 2010.
[2]Khan, S., Lee, D. H., Khan, M. A., et al. Image Interpolation via Gradient
Correlation-Based
Edge
Direction
Estimation[J].
Scientific
Programming,2020,2020.
[3]Hwang, J. W., Lee, H. S. Adaptive image interpolation based on local
gradient features[J]. IEEE signal processing letters, 2004, 11(3): 359-362.
[4]Kiku D, Monno Y, Tanaka M, et al. Beyond Colour Difference: Residual
Interpolation for Colour Image Demosaicking, in IEEE Transactions on
Image Processing, vol. 25, no. 3, pp. 1288-1300, March 2016, doi:
10.1109/TIP.2016.2518082.
[5]Wu J, Timofte R, Van Gool L. Demosaicing Based on Directional
Difference Regression and Efficient Regression Priors, in IEEE
Transactions on Image Processing, vol. 25, no. 8, pp. 3862-3874, Aug.
2016, doi: 10.1109/TIP.2016.2574984.
[6]Monno Y, Kiku D, Tanaka M, et al. Adaptive residual interpolation for
colour and multispectral image demosaicking[J]. Sensors, 2017, 17(12):
2787.
[7]Lu J. Analysis and Comparison of Three Classical Colour Image
Interpolation Algorithms[J]. Journal of Physics: Conference
Series,2021,1802(3):
[8]Fadnavis S. Image interpolation techniques in digital image processing: an
overview[J]. International Journal of Engineering Research and
Applications, 2014, 4(10): 70-73.
[9]Parsania P S, Virparia P V. A comparative analysis of image interpolation
algorithms[J]. International Journal of Advanced Research in Computer
and Communication Engineering, 2016, 5(1): 29-34.
[10]Pan X Y, Li C C, Hao W, et al. FPGA Acceleration of Colour
Interpolation Algorithm Based on Gradient and Colour Difference[J].
Sensing and Imaging, 2020, 21(1): 1-16.
[11]Iriyama T, Sato M, Aomori H, et al. Channel-Wise Predictive Filter Flow
for Demosaicking[J]. Journal of Signal Processing, 2020, 24(4): 187-190.
Figure 11. Modified2-zoomed.
In order to compare the experimental results more
objectively and visually, the Colour Peak Signal-to-Noise Ratio
(CPSNR) has been introduced [11]. It is a common image
evaluation metric that focuses on the difference between the
generated image and the original image, and is widely used in
image recovery.
(14)
Where W and H is the width and height of the image,
is the original image, and
is the
respectively.
712
Authorized licensed use limited to: UNIV OF ENGINEERING AND TECHNOLOGY TAXILA. Downloaded on March 17,2023 at 05:23:46 UTC from IEEE Xplore. Restrictions apply.