1029
Journal of Intelligent & Fuzzy Systems 44 (2023) 1029–1041
DOI:10.3233/JIFS-221654
IOS Press
CO
PY
Tunicate swarm-based grey wolf algorithm
for fetal heart chamber segmentation and
classification: a heuristic-based optimal
feature selection concept
C. Shobana Nageswaria,∗ , M.N. Vimal Kumarb , N. Vini Antony Gracea and J. Thiyagarajanb
a R.M.D.
b Sona
Engineering College, Chennai, Tamilnadu, India
College of Technology, Salem, Tamilnadu, India
AU
TH
OR
Abstract. Ultrasound image quality management and assessment are an important stage in clinical diagnosis. This operation
is often carried out manually, which has several issues, including reliance on the operator’s experience, lengthy labor, and
considerable intra-observer variance. As a result, automatic quality evaluation of Ultrasound images is particularly desirable
in medical applications. This research work plans to perform the fetal heart chamber segmentation and classification using
the novel intelligent technology named as hybrid optimization algorithm Tunicate Swarm-based Grey Wolf Algorithm (TSGWA). Initially, the US fetal images data is collected and data undergoes the preprocessing using the total variation technique.
From the preprocessed images, the optimal features are extracted using the TF-IDF approach. Then, Segmentation is processed
on optimally selected features using Spatially Regularized Discriminative Correlation Filters (SRDCF) method. In the final
step, the classification of fetal images is done using the Modified Long Short-Term Memory (MLSTM) Network. The fitness
function behind the optimal feature selection as well as the hidden neuron optimization of MLSTM is the maximization of
PSNR and minimization of MSE. The PSNR value is improved from 3.1 to 9.8 in the proposed method and accuracy of the
proposed classification algorithm is improved from 1.9 to 12.13 compared to other existing techniques. The generalization
ability and the adaptability of proposed TS-GWA method are described by conducting the various performance analysis.
Extensive performance result shows that proposed intelligent techniques performs better than the existing segmentation
methods.
Keywords: Fetal heart chamber segmentation, optimal feature selection, modified long short term memory tunicate swarmbased grey wolf algorithm, fetal heart chamber classification
1. Introduction
Fetal heart defect (FHD) is a abnormalities with a
baby’s heart that arise while the infant is still growing.
It is a major frequent and dangerous birth abnormalities in the present situation [1, 2]. Asia has the
greatest rate of FHD birth defects, with 9.3%. (95 per∗ Corresponding author. Associate Professor, R.M.D. Engineering College, Chennai, Tamilnadu, India. E-mail: drshobananages
wari@gmail.com.
cent CI: 8.9-9.7). FHD has the highest incidence and
fatality in China for the previous 12 years, and it is
the leading cause of children death [3]. Despite substantial breakthroughs in diagnosis and care, FHD
is still the leading cause of mortality in newborns
during their first year. To reduce birth malformations
and death, efficient prevention method and control
techniques are required [4]. An efficient tool for
identifying foetal cardiac abnormality is still foetal
echocardiography. Standard foetal heart US slices are
ISSN 1064-1246/$35.00 © 2023 – IOS Press. All rights reserved.
C. Shobana Nageswari et al. / Fetal heart chamber segmentation and classification
foetal FCH images is more challenging than training
a stable method from adult FHD images [13], there
has been little study on detecting FHD utilizing deep
learning. The images of a foetus and an adult have
two major variances. The initial is that various foetal
placements inside the mother’s body make it harder to
register. The other is that depending on the gestational
week, the form of the FCH images varies dramatically. As a result, reliably training a network to
identify FHD is tough. According to recent research,
the identification rate of FHD ranges from 65 percent
to 81 percent. Inadequate data of foetal FHD images,
on the other hand, may diminish the model’s robustness and lead to overfitting; indeed, this training is
difficult for most of the large networks suggested
in the computer vision area [14]. Overfitting may
be addressed and accuracy increased by using transfer learning, a training strategy that integrates and
transfers information between various tasks. Deep
learning has already been shown to be beneficial for
transfer learning tasks since the features learnt by
Deep Neural Networks (DNN)s capture the majority
of the important information needed for categorization [15]. Yet, owing to large distribution mismatches
among the target (medical images) and source (natural images) domains, traditional transfer learning can
decrease classification results in the medical image
domain. Furthermore, video slices are used to boost
the robustness of transfer learning, although accuracy
ratings are rarely published.
The main contribution of this paper are.
AU
TH
OR
still challenging to obtain since the foetal position
inside the mother fluctuates [5]. An echocardiogram
is a comprehensive US test of the baby’s heart performed before the delivery. As a consequence of
the pathophysiological variations in the foetal heart
at distinct stages of pregnancy, the two-dimensional
structure and hemodynamics of the foetal heart are
constantly varying, necessitating practitioners with
substantial knowledge in development and disease
cognition. Hence, The evaluation is performed by
hand, which is labor-intensive and time-consuming,
and so may be clinically ineffective [6]. It is challenging for radiologists, particularly inexperienced
sonographers, to perform an adequate US sweep for
complicated structures like the embryonic heart in
clinical practice.
Furthermore, competent radiologists are still in
low supply in poor areas. As a result, there exists
a great demand for a completely automated quality
control system for foetal US images [7]. We should
first develop a quantitative quality control methodology for the target planes in great detail. One or
multiple networks are used to attain quantitative evaluation of the protocol’s elements on the basis of the
protocol [8]. Generally, all US planes’ procedures
(e.g., cranial, cardiac, and abdominal views) should
include the following three portions. To begin, we
must establish whether a scanned image is the target plane (for example, foetal Cardiac Four-chamber
Plane (CFP). If that’s the case, we’ll go on to the target
plane’s next step [9]. Few significant characteristics
of US imaging, including as gain and zoom, as well
as numerous key anatomical features on the target
plane, should be considered further on the basis of
the preceding phase. Gain and zoom have typically
been disregarded in prior investigations [10].
The Convolutional Neural Networks (CNN) has
been frequently utilized to evaluate medical images
since the invention of deep learning. Generative
adversarial network (GAN) describes a new unsupervised form of learning network that was recently
suggested [11]. It’s a form of neural network method
where two networks are trained at the same time, one
for image production and the other for discriminating. It enables the learning of deep representations
without the need for significant labeled training material. Owing to their tolerance to overfitting as well as
capability to capture data distributions, GANs have
attracted interest in both business and academics,
making image generating jobs easier [12].
FHD images during the end-systolic stage are complex to collect, and training a stable method from
CO
PY
1030
• Fetal heart chamber segmentation and classification process using the novel intelligent
technology.
• To accomplish the optimal feature selection,
where the features are tuned with an objective
concept.
• To perform the classification by MLSTM, network, where the hidden neurons of LSTM are
tuned with the consideration of PSNR maximization and MSE minimization.
• To propose a novel form of optimization algorithm called TS-GWA for enhancing the optimal
feature selection and the classification phases of
the introduced fetal heart chamber segmentation
and classification and to compare the developed
method with existing algorithms
The rest of the article describes as follows, Section 2 describes the fatal image segmentation related
work and its feature. The proposed model and preprocessing for the fetal heart chamber segmentation
C. Shobana Nageswari et al. / Fetal heart chamber segmentation and classification
2. Related works
AU
TH
OR
In this section, we present related work of Fetal
Heart Chamber Segmentation and Classification.
Also this section describes the various research methods adopted for US fetal segmentation.
Chen et al. [16] suggested a novel method called
DGACNN that performed best in identifying FHD,
with an accuracy rate of 85 percent. The goal of
this network to address the issue of inadequate training datasets for building a strong model. There were
numerous unlabeled video slices, but annotating them
was difficult and time-consuming. As a result, understanding how to employ these un-annotated video
slices to increase the DGACNN capabilities for
detecting FHD with respect to both robustness and
recognition accuracy was crucial for FHD screening.
The DGACNN surpassed various existing networks
by 1 percent to 20% in identifying FHD, according to
the testing. A comparative experiment demonstrated
that this network already surpasses professional cardiologists in detecting FHD, with an accuracy rate
of 84 percent in one test. As a result, the suggested
design has a strong chance of assisting cardiologists
in completing early FHD tests.
Yagel et al. [17] have developed a general deep
learning architecture for foetal US CFP quality control. The suggested framework was composed of three
networks: (1) a basic CNN (B-CNN), which roughly
classified four-chamber views from raw data; (2) a
Deeper CNN (D-CNN), which determined the gain
as well as zoom of the target images using multi-task
learning; and (3) the ARVBNet, which detected the
key anatomical structures on a plane.
Del Bianco et al. [18] have suggested a general
framework on the basis of instance segmentation for
precisely and concurrently segmenting the four heart
chambers. Experimental findings demonstrated that
the technique could obtain higher segmentation performance vs conventional techniques in the gathered
dataset, which comprised echocardiogram images
having four-chamber views of 319 foetuses. The
model obtained Dice coefficients of 0.7956, 0.7619,
0.8199, and 0.7470 for the four cardiac chambers,
having an average precision of 45.64 percent, utilizing fivefold cross-validation.
Rahmatullah et al. [19] have included an element
for extracting ROI on the basis of target detection,
and determined the four-chamber view in order to
improve classification performance. However, the
merged neighbor frame difference into image channels not lose the time dependency. To increase
diagnostic accuracy, researchers have devised a simple yet efficient RLDS for identifying embryonic
CHD that used CNNs to extract discriminative
aspects of the foetal cardiac anatomical structures
[20]. Extensive testing has shown that the suggested
RLDS was quite successful in detecting foetal CHD.
Moreover, in the test set, the suggested RLDS attained
an accuracy of 93% and a recall of 93%, considerably
improving the prenatal detection rate for foetal CHD.
The separation of the four heart chambers in
foetal echocardiograms using a unique group subspace approach is presented in [21]. The method was
able to leverage the intrinsic structure of echocardiograms by merging the group reconstruction error,
sparsity, and distinguishing term into a cohesive
framework, resulting in the production of a discriminative group dictionary. To create a small dictionary
having high atom usage and minimal complexity, a
unique adaptive group dictionary learning technique
was devised. The reconstruction residue was used
to distinguish the starting position of four chambers
using the learnt dictionary. The final outlines were
refined regionally using the local appearances. Extensive testing was done to assess the performance of
the developed AGDL and its use in foetal echocardiography segmentation. The findings showed that
the technique has superior representation as well as
discriminative power than other competitive traditional sparse representation approaches and general
intensity methods. The method could learn a more
concise and discriminative group vocabulary, resulting in more reliable and appropriate four-chamber
segmentation outcomes.
Verdurmen et al. [22] have developed a system for
keeping track of the main factors that characterize the
content of every frame of freehand 2D US scanning
movies of a healthy foetal heart. This was a critical
initial step in developing tools to aid in the identification of CHD in unusual situations. The visibility,
position, orientation of the foetal heart in the image,
and the observing plane label from every frame, were
predicted using regression and classification forests.
Author also created a new regression forest adap-
CO
PY
and is presented in Section 3. Section 4 describes
the feature extraction and segmentation of fetal heart
chamber. Optimization process and MLSTM network
for the fetal heart chamber classification is elaborated in Section 5. The detailed performance analysis
is presented in Section 6. Section 7 concludes this
research work.
1031
C. Shobana Nageswari et al. / Fetal heart chamber segmentation and classification
tion for circular variables to cope with cardiac phase
forecasting. Chen et al. reported findings from a difficult dataset collected in a real-world clinical context
and compared them to expert annotations, reaching
equal levels of accuracy to inter- as well as intraobserver variance. The Aplio i800 (CANON Medical
Systems Corporation, Tochigi, Japan) and a convex
probe (4 MHz) for foetuses were used for the entire
recordings [23].
An end-to-end DW-Net presented in [24] for
precise segmentation of seven major anatomical components. The ssuggested DW-Net might help further
extract valuable clinical indicators in early FE and
enhanced the prenatal diagnosis effectiveness and
accuracy of CHDs by accurately and automatically
segmenting the A4 C image. Traditional segmentation and classification procedures for the entire
foetus, as well as the foetal brain, brain, lungs, heart,
liver, and placenta in MRI and three-dimensional US
have been addressed [25]. Potential clinical uses of
the aforesaid technologies were also investigated.
CO
PY
1032
3.1. Proposed model
TH
OR
3. Proposed model for the fetal heart
chamber segmentation and classification
Fig. 1. Proposed architectural model of fetal heart chamber segmentation and classification.
The proposed fetal heart chamber segmentation
and classification frameworks contain the following
phases
data collection,
pre-processing,
feature extraction,
optimal feature selection,
segmentation,
classification.
3.2. Pre processing
AU
•
•
•
•
•
•
the developed fetal heart chamber segmentation and
classification framework is shown in Fig. 1.
In the data collection phase, the data is collected
from the US fetal images. This undergoes the preprocessing phase using the total variation technique.
Next, the features are extracted in the feature extraction phase using the TF-IDF approach. Since the
length of the extracted features seems to be long,
it is necessary to select the optimal features using
proposed TS-GWA. Further, the segmentation is
accomplished using the SRDCF technique. Finally,
the classification is done by LSTM, where the hidden
neurons are tuned by same TS-GWA with the intention of PSNR maximization and MSE minimization
thus referred as MLSTM. This MLSTM classifies the
final fetal heart chamber output. The architecture of
Sample pre-processed images of fetal heart chamber segmentation is presented in Fig. 2. The
pre-processing is done to remove the noises present
in the gathered data. Here, the pre-processing is
performed by the total variation technique. Total variation [26] regularisation is a term used in signal
processing to describe total variation denoising. This
method is most commonly utilized in digital image
processing to remove noise. It is based on the idea that
signals having a lot of unnecessary and possibly erroneous features have a lot of overall variance because
the integral of the signal’s absolute gradient is large.
Minimizing the signal’s overall variance achieves
a tight match to the original signal and eliminates
extra detail while maintaining important characteristics like edges. This noise reduction approach offers
advantages over basic approaches like median filtering, which decrease noise as well as smooth edges to
1033
CO
PY
C. Shobana Nageswari et al. / Fetal heart chamber segmentation and classification
Fig. 2. Sample pre-processed images of fetal heart chamber segmentation and classification.
a lesser or greater extent at the similar time. Therefore, even at low SNRs, total variation denoising is
extremely effective at retaining edges while smoothing out noise in flat regions. Thus,
z, y, o ∈ o
(1)
TH
OR
z = y + o,
The noise model is represented by the Equation
(1), in which z is the actual image and y is the
observed image having noise o. Finding the signal
y while reducing the objective function defined in
the Equation (2)
K (y) =
z − y22
+ λBy1
the cosine similarity measures. Since, we are using
fatal images for feature extraction process, so that the
value of TF-IDF is closely monitored. The value of
TF-IDF value increases in certain region, that region
is closely observed and that region converted into
image set. From this feature extraction techniques,
fine details related the fetal images are observed for
further process.
(2)
AU
The objective function is minimized by control
the regularisation parameter λ which also limits the
amount of smoothing that may be done in the preprocessing stage.
4. Feature extraction and segmentation for
the fetal heart chamber
4.1. Feature extraction
The feature extraction supports in minimizing
the redundant data count from the provided preprocessed dataset. It enhances the learning speed as
well as the generalization steps. Here, the feature
extraction is accomplished using the Term Frequency
– Inverse Document Frequency (TF-IDF) technique
[27].
In the TF-IDF feature extraction techniques, Fetal
pre-processed image dataset is converted into raw
vectors. Then each raw vectors are assessed through
4.2. Optimal feature selection
The fine features of fatal images are obtained from
the previous step. This optimal feature selection process cuts the number of wrong features in half while
keeping real positive rates the same. This implies it is
more practical in selecting the appropriate variables,
leading in a system that is simpler, more understandable, and more realistic. Since the extracted features
seems to be lengthy, it is necessary to select the significant features in order to reduce the complexity.
This is done by the introduced TS-GWA, where the
features are being tuned. When creating forecasting
models, it represents the method of minimising the
count of input variables. The count of input variables
should be reduced to lower the computational cost of
modelling and, under certain situations, to increase
the effectiveness of the algorithm. Removing extraneous data increases learning accuracy, decreases
computation time, and makes the adaptive learning
or data easier to grasp.
4.3. Segmentation
The segmentation signal divides a signal into many
epochs having similar statistical features, like ampli-
C. Shobana Nageswari et al. / Fetal heart chamber segmentation and classification
multiplication. The convolution function equals to the
Tg (yl ) which is given by Equation (4)
Tg (yl ) =
AU
min
g
u
l=1
e
2 ω gm 2
αl Tg (yl ) − zl +
(3)
m=1
Here, αl ≥ 0 represents the weight of each training sample yl , spatial regularisation is provided by
ω, which shows a Gaussian shaped function having
smaller values in the centre region and larger values
in the marginal area, and symbolises element wise
e
ylm ∗ gm
(4)
m=1
Tg (a) = F
−1
e
â ĝ
m
m
(5)
m=1
The SRDCF uses the same detecting technology as
ordinary DCF-oriented trackers. The object tracking
represented in the Equation (5).The SRDCF uses a
scaling pool to manage target scale variations during
the detection stage, and the Fast Sub-grid approach
to improve the detection findings.
5. Proposed optimization techniques for fetal
heart chamber classification
5.1. Proposed TS-GWA
The proposed TS-GWA is used for enhancing the
optimal feature selection and the classification phases
of the fetal heart chamber. It optimizes the features
in optimal feature selection phase as well as hidden
neurons of LSTM with the intention of PSNR maximization and MSE minimization. The Grey Wolf
Optimization (GWO) [35] is modelled after the natural leadership structure and hunting operation of
grey wolves. For emulating the leadership structure,
four sorts of grey wolves are used: beta, alpha, delta,
and omega. Furthermore, the three basic processes
of hunting are installed: seeking for prey, encircling
prey, and attacking prey.
The GWO has several advantages such as better
convergence, local optima avoidance, exploitation,
and exploration. It also has superior performance in
challenging, unknown search spaces. But, it cannot
solve the binary as well as the multi objective versions. Thus, to overcome this drawback, Tunicate
Swarm Algorithm (TSA) [36] is integrated into it and
the resulting algorithm is called as TS-GWA. This
TS-GWA can handle the binary as well as the multi
objective optimization problems.
In the navigation as well as the foraging phase, the
suggested algorithm mimics tunicate jet propulsion
and swarm behaviours. Tunicate includes the capacity
to locate a food source in the water. Unfortunately, in
the supplied search space, there exists no information
regarding the food source. Two tunicate behaviours
TH
OR
tude as well as frequency. Here, the segmentation
is done using Spatially Regularized Discriminative
Correlation Filters (SRDCF) approach. Because of
the online aspect of the tracking issue, DCF-oriented
trackers are becoming increasingly familiar in the
tracking community owing to their high effectiveness, simplicity, and reliability. Color-Name [29],
HOG [30], and deep features [31, 32] are examples of
novel features that have been extensively employed;
feature incorporation has also been employed [33].
Part-oriented trackers [34] are extensively used to
alleviate occlusion. On the basis of the periodic
presumption of the training instances, the entire
correlation filter-oriented trackers employ FFT to
considerably minimise the training as well as detection processing work. The periodic assumption, on
the other hand, resulted in undesirable boundary consequences.
The SRDCF [28] formulized in the spatial domain
is initially translated to the Fourier domain to obtain a
complex equation, which is then converted to a realvalued equation and solved using the Gauss-Seidel
technique. The results of the Gauss-Seidel technique
must be altered again in order to obtain the correlation filters we want. When they obtain the complex
equation, they divide it into real as well as imaginary
parts, rebuild the issue as a real-valued equation, and
then devise a simplified inverse approach to obtain
a closed-form response; nevertheless, the simplified
inverse operation has a high computational complexity. Transitions from spatial to Fourier or from
complex to real-valued equations are seamless, and
correlation filters are calculated immediately.
A collection of training instances {(yl , zl )}ul=1 is
used to train convolution filters g. Each training sample yl ∈ Re×N×O is made up of a e-channel feature
map taken from a training image patch having a spatial size of N × O. The mth feature layer of yl is
represented by ylm . The best convolution output for
the training sample yl is zl . From the convex issue,
the Spatially Regularized Discriminative Correlation
Filters (SRDCF) used to minimize the following
Equation (3)
CO
PY
1034
C. Shobana Nageswari et al. / Fetal heart chamber segmentation and classification
are used to select the best food source, i.e., optimal.
Jet propulsion and swarm intelligence are two of these
traits. A tunicate must satisfy three standards in order
to statistically describe jet propulsion behaviour: prevent conflicts among search agents, move in the path
of the best search agent’s location, and stay near to
the optimal search agent. The swarm activity, on the
other hand, will keep remaining search agents up to
date on the best optimum answer.
The TS-GWA works on the basis of fitness. If fit ≤
0.5, then the update takes place using GWO which is
represented in the Equations (6)–(10).
(9)
Z = 2z · ra1 − z
(10)
AU
In these Equations (6)–(10), the components of z
are minimized from 2 to 0 and the random vectors are
given by ra1 and ra2 respectively.
Otherwise, if fit > 0.5, then the update takes place
by TSA as in Equation (11) – Equation (12).
A (p) + A (p + 1)
2 + b1
F R + Z · P C,
ifrand ≥ 0.5
F R − Z · P C,
ifrand < 0.5
(11)
(12)
Here, Z is used for the novel search agent position
computation. The pseudo code of TS-GWA is given
below.
i.
ii.
iii.
iv.
A (p + 1) =
iii.
v.
vi.
vii.
A (p) + A (p + 1)
2 + b1
Repeat till maximum iterations
Compute best fitness
Return best solution
Stop
The proposed MLSTM is used for enhancing the
classification phase of the developed fetal heart chamber classification model, by optimizing the hidden
neurons of LSTM by TS-GWA with the consideration of PSNR maximization and MSE minimization.
Hochreiter and Schmidhuber [38] created LSTM
[37], a development of RNN, to overcome the RNN
shortcomings by including more interactions per
module (or cell). LSTMs are a type of RNN that, by
default, can learn long-term dependencies and recall
information for lengthy periods of time. The LSTM is
structured in a chain format, according to Olah [39].
The recurring module, on the other hand, contains a
distinct structure. It features four interacting layers
having a unique form of communication, rather than
a single NN like a normal RNN.
The initial stage in building an LSTM network is to
recognize information that not needed and will be left
out of the cell. The sigmoid function, which considers
the output of the final LSTM unit (iu−1 ) at time u −
1 and the present input (Yu ) at time u, determines
the procedure of detecting and excluding data. The
sigmoid function also decides whether parts of the
old output should be removed. The forget gate gu
defined in Equation (13), describes a gate in which
gu shows a vector having values ranging from 0 to
1 that corresponds to every count in the cell state,
Du−1 .
TH
OR
B = 2 · r a2
A (p) =
iii. else
CO
PY
(6)
A1 = Aα − Z1 · Cα , A2 = Aβ − Z2 · Cβ ,
A3 = Aδ − Z3 · Cδ
(7)
Cα = B1 · Aα − A , Cβ = B2 · Aβ − A ,
(8)
Cδ = B3 · Aδ − A
A 1 + A2 + A3
3
5.2. Modified long short term memory network
A1 + A 2 + A 3
A (p + 1) =
3
A (p + 1) =
A (p + 1) =
1035
Start
Initialization of population
Initialization of parameter
Calculation of fitness
If fit ≤ 0.5
gu = σ Xg iu−1 , Yu + cg
(13)
Here, σ shows the sigmoid function, and Xg and cg
describes the forget gate’s weight matrices and bias,
accordingly. The decision and storage of information
from the new input (Yu ) in the cell state, as well as
updating the cell state, are the next steps. The sigmoid layer and the tanh layer are the two sections
of this stage. The sigmoid layer determines whether
new information must be adjusted or discarded (0 or
1036
C. Shobana Nageswari et al. / Fetal heart chamber segmentation and classification
the intention of PSNR maximization and MSE minimization. The objective function is modelled as in
Equation (19).
1
fit = arg max
PSNR +
(19)
MSE
{Ft,HNLSTM }
Du = Du−1 gu + Ou ju
(14)
ju = σ Xj iu−1 , Yu + cj
(15)
Ou = tanh Xo iu−1 , Yu + co
(16)
In the above Equation (19), the features to be optimized are shown by Ft, hidden neurons of LSTM are
shown by HNLSTM , and the fitness is shown by fit
respectively. PSNR and MSE is expressed in Equations (20) and (21) Respectively.
2552
PSNR = 10 × lg
(20)
MSE
Where,
iu = Pu tanh (Du )
Pu = σ X0 iu−1 , Yu + c0
(17)
(18)
The structure of MLSTM for fetal heart chamber
classification is shown in Fig. 3.
AU
5.3. Objective model
1 Ig (i, j) − Ig (i, j)
C×D
D
MSE =
C
The major objective of introduced TS-GWA-based
fetal heart chamber segmentation and classification
is to optimize the features of optimal feature selection phase as well as hidden neurons of LSTM with
Fig. 3. MLSTM for fetal heart chamber classification.
2
i=1 j=1
(21)
6. Results and discussion
TH
OR
Du−1 and Du describes the cell states at time u − 1
and u, correspondingly, whereas X and c shows the
cell state’s weight matrices and bias, accordingly. The
output values (iu ) defined in Equation (17) in the last
step are filtered versions of the output cell state (Pu )
expressed in Equation (18). A sigmoid layer is used
to determine whether aspects of the cell state find it
to the output. The sigmoid gate output (Pu ) is then
multiplied using the new values produced by the tanh
layer from the cell state (Du ), having a value ranging
from -1 to 1.
CO
PY
1), and the tanh function assigns weight to the values
that pass through, determining their relevance (1 to
1). To update the new cell state, the two values are
multiplied. The old memory Du−1 is then joined to
the new memory, producing in Du which is defined
in Equation (14).
6.1. Fetal heart US images
The images of the second trimester foetal cardiac
US utilised in this study came from Mediscan systems, a facility for US, prenatal care, and genetics in
Chennai, India, as well as internet data. The foetal cardiac images are more visible in the second trimester
than in the first trimester. since the growth of the
cardiac system of the foetus occurs in the second
trimester. Hence, only second trimester foetal cardiac
images are considered in this research work. There are
50 normal foetal cardiac US images in the database,
18 abnormal images with AVSD, 18 abnormal images
with VSD, and 14 abnormal Ebsteins images. The
images were taken with a GE Healthcare Voluson E8
US machine having a 51 Hz linear transducer. The
images attained were 800 x 600 pixels in size. Before
filtering, all of the images were shrunk to 512 x 512
pixels. This study also used 25 normal foetal heart
images from the internet. Figure 4. Shows the normal foetal heart images considered for this work. The
following Performance and Error Measures given
in Equations (22) to (24) are used in this work
to evaluate the effectiveness of proposed intelligent
technique.
n
(yi − yi )2
RMSE = (22)
n
i=1
1037
AU
TH
OR
CO
PY
C. Shobana Nageswari et al. / Fetal heart chamber segmentation and classification
Fig. 4. Sample US Fetal Images associated with various datasets.
1
|yi − y|
n
(23)
TP + TN
TP + TN + FP + FN
(24)
n
MAE =
6.2. MSE analysis
i=1
Accuracy =
The MSE analysis of various heuristic-based algorithms for the fetal heart chamber segmentation and
classification is shown in Fig. 5. It is clearly understood that the MSE error measure of TS-GWA is
lower than the other methods, thereby revealing its
C. Shobana Nageswari et al. / Fetal heart chamber segmentation and classification
Fig. 5. MSE Analysis of different heuristic-based methods for the
fetal heart chamber segmentation and classification.
6.3. PSNR analysis
Fig. 6. PSNR Analysis of different heuristic-based methods for
the fetal heart chamber segmentation and classification.
TH
OR
superiority. The MSE of TS-GWA is 0.81%, 1.43%,
2.31%, and 0.96% higher than PSO, GOA, TSA, and
GWO.
CO
PY
1038
AU
The PSNR analysis of several heuristic-based algorithms for the segmentation and classification of
foetal heart chambers is shown in Fig. 6. The PSNR
error measure of TS-GWA is clearly lower than that
of the other approaches, demonstrating its superiority. PSO, GOA, TSA, and GWO have PSNRs of 3.21
percent, 2.31 percent, 1.89 percent, and 0.80 percent, respectively. As a result, it can be deduced that
the presented TS-GWA has a higher PSNR value,
demonstrating its superiority in foetal heart chamber
segmentation and classification.
6.4. RMSE analysis
Figure 7 shows the RMSE analysis of multiple
heuristic-based techniques for segmenting and classifying foetal cardiac chambers. TS-GWA’s RMSE
error metric is significantly lower than the other
techniques, proving its superiority. RMSE s of 1.82
percent, 2.76 percent, 1.52 percent, and 2.15 percent
are found in PSO, GOA, TSA, and GWO, respectively. As a consequence, the provided TS-GWA has
a better RMSE value, confirming its superiority in
the segmentation and categorization of foetal heart
chambers.
Fig. 7. RSME Analysis of different heuristic-based methods for
the fetal heart chamber segmentation and classification.
6.5. MAE analysis
The MAE analysis of different heuristic-oriented
approaches for segmenting and classifying foetal cardiac chambers is depicted in the Fig. 8. The MAE
error statistic of TS-GWA is substantially lower than
the others, demonstrating its superiority. PSO, GOA,
TSA, and GWO had RMSEs of 2.53 percent, 3.01
percent, 1.15 percent, and 2.03 percent, correspondingly. As a result, the offered TS-GWA has a lower
MAE, indicating that it is superior at segmenting and
classifying foetal heart chambers.
6.6. Accuracy analysis
Figure 9 shows the accuracy analysis of several heuristic-oriented techniques for segmenting and
classifying foetal cardiac chambers. TS-GWA has a
C. Shobana Nageswari et al. / Fetal heart chamber segmentation and classification
1039
Table 1
Time efficiency analysis
Methods
PSO [40]
GOA [41]
TSA [36]
GWO [35]
TS-GWA
5
10
2.5
1.2
7.3
2.3
0.2
1.7
1.3
7.7
2.1
0.2
Iteration (t in Sec)
15
20
2.1
1.7
6.5
2.3
0.1
5.1
3.4
1.4
2.2
0.4
25
7.2
4.1
2.7
3.2
0.1
Fig. 8. MAE Analysis of different heuristic-based methods for the
fetal heart chamber segmentation and classification.
CO
PY
cent, 3.02 percent, 1.13 percent, and 2.01 percent.
The supplied TS-GWA is therefore better at segmenting and categorising the foetal heart chambers since
it saves time than the existing methods.
7. Conclusion
TH
OR
In this work, we proposed revolutionary intelligence technologies to segment and classify the
fetal heart chambers. The data was initially acquired
through prenatal imaging in the US system. The
total variation approach was used to pre-process the
acquired data. The TF-IDF technique was used to
extract features from pre-processed images. Because
the length of the extracted features was so long, the
best features were extracted. The SRDCF technique
was used to segment images based on the ideally
selected characteristics. The LSTM was used to classify prenatal images in the last stage, where the hidden
neurons were tweaked, resulting in MLSTM. The
unique hybrid optimization technique combined TSA
and GWO, resulting in TS-GWA for feature selection and hidden neuron optimization in MLSTM.
From the extensive experiments results, higher PSNR
and Accuracy has been achieved to segment the fatal
images. Also, the proposed TS-GWA based optimization technique achieve the better prediction of fatal
image content over the existing algorithms.
AU
Fig. 9. Accuracy Analysis of different heuristic-based methods for
the fetal heart chamber segmentation and classification.
significantly higher accuracy than the others, proving
its superiority. The accuracies for PSO, GOA, TSA,
and GWO, respectively, were 1.99 percent, 3.14 percent, 5.61 percent, and 4.98 percent. As a result, the
TS-GWA that is being supplied has a higher accuracy, indicating that it is better at segmenting and
classifying foetal heart chambers.
6.7. Time efficiency analysis
Time efficiency analysis of various heuristicoriented methods for segmenting and categorising
foetal cardiac chambers are presented in Table 1.
The advantage of TS-GWA is shown by the fact that
its time efficiency statistic is significantly lower than
those of the competitors. The time efficiency for PSO,
GOA, TSA, and GWO were respectively 2.51 per-
References
[1]
[2]
[3]
Y. Gong et al., Fetal Congenital Heart Disease Echocardiogram Screening Based on DGACNN: Adversarial
One-Class Classification Combined with Video Transfer
Learning, in IEEE Transactions on Medical Imaging 39(4)
(2020), 1206–1222.
J. Dong et al., A Generic Quality Control Framework for
Fetal Ultrasound Cardiac Four-Chamber Planes, in IEEE
Journal of Biomedical and Health Informatics 24(4) (2020),
931–942.
Shan An, Haogang Zhu and Yihua He, A category attention
instance segmentation network for four cardiac cham-
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
A. Del Bianco, S. Russo, N. Lacerenza, M. Rinaldi, G.
Rinaldi, L. Nappi, et al., Four chamber view plus threevessel and trachea view for a complete evaluation of the
fetal heart during the second trimester, Journal of Perinatal
Medicine 34 (2006), 309–312.
B. Rahmatullah, I. Sarris, A. Papageorghiou and J.A. Noble,
Quality control of fetal ultrasound images: Detection of
abdomen anatomical landmarks using AdaBoost, in 2011
IEEE International Symposium on Biomedical Imaging:
From Nano to Macro 2011 (2011), 6–9.
L.J. Salomon, J.P. Bernard, M. Duyme, B. Doris, N. Mas
and Y. Ville, Feasibility and reproducibility of an imagescoring method for quality control of fetal biometry in the
second trimester, Ultrasound in Obstetrics and Gynecology
27 (2006), 34–40.
M.A. Maraci, C.P. Bridge, R. Napolitano, A. Papageorghiou
and J.A. Noble, A framework for analysis of linear ultrasound videos to detect fetal presentation and heartbeat,
Medical Image Analysis 37 (2017), 22–36.
K.M. Verdurmen, N.B. Eijsvoogel, et al. A systematic
review of prenatal screening for congenital heart disease by fetal electrocardiography, International Journal of
Gynaecology and Obstetrics: The Official Organ of the
International Federation of Gynaecology and Obstetrics
(2016).
Jayaraman, T., Chinnusamy, G.S. Investigation of filtering
of rain streaks affected video sequences under various quantisation parameter in HEVC encoder using an enhanced
V-BM4D algorithmna, IET Image Processing 14(2) (2021),
337–347.
M. Ithuralde and R. Neirotti, Screening of fetal congenital
heart disease: the challenge continues, [J]. Rev Bras Cir
Cardiovasc 28(3) (2013), V–VII.
T. Jayaraman and G.S. Chinnusamy, Analysis of deep
rain streaks removal convolutional neural network-based
post-processing techniques in HEVC encoder, Journal of
Circuits, Systems and Computers 32(2) (2021).
Abdul Wahid and Hyo Jong Lee, Image denoising method
based on directional total variation filtering, IEEE Access,
ICTC (2017).
Juan Ramos, Using TF-IDF to Determine Word Relevance
in Document Queries (2015).
Xiaoxiang Hu and Yujiu Yang, Faster Spatially Regularized
Correlation Filters for Visual Tracking (2014).
J. van de Weijer, C. Schmid, J.J. Verbeek and D. Larlus,
Learning color names for real-world applications, In TIP
(2009).
N. Dalal and B. Triggs, Histograms of oriented gradients for
human detection, In CVPR (2005).
C. Ma, J.-B. Huang, X. Yang and M.-H. Yang, Hierarchical
convolutional features for visual tracking, In ICCV (2015).
H. Nam and B. Han, Learning multi-domain convolutional
neural networks for visual tracking, In CVPR (2016).
Y. Li and J. Zhu, A scale adaptive kernel correlation filter tracker with feature integration, In ECCV Workshop
(2014).
T. Liu, G. Wang and Q. Yang, Real-time part-based visual
tracking via adaptive correlation filters, In CVPR (2015).
Seyedali Mirjalili, Seyed Mohammad Mirjalili and Andrew
Lewis, Grey wolf optimizer, Advances in Engineering Software 69 (2014), 46–61.
Satnam Kaur, Lalit K. Awasthi, A.L. Sangal and Gaurav
Dhiman, Tunicate Swarm Algorithm: A new bio-inspired
based metaheuristic paradigm for global optimization, Engineering Applications of Artificial Intelligence 90 (2020).
CO
PY
[5]
bers segmentation in fetal echocardiography, Computerized
Medical Imaging and Graphics 93 (2021).
Bin Pu, Ningbo Zhu, Kenli Li and Shengli Li, Fetal cardiac
cycle detection in multi-resource echocardiograms using
hybrid classification framework, Future Generation Computer Systems 115 (2021), 825–836.
Sibo Qiao, Shanchen Pang and Zhihan Lv, RLDS: An
explainable residual learning diagnosis system for fetal congenital heart disease, Future Generation Computer Systems
128 (2022), 205–218.
Yi Guo, Li Yu, Yuanyuan Wang, Jinhua Yu, Guohui Zhou
and Ping Chen, Adaptive group sparse representation in
fetal echocardiogram segmentation, Neurocomputing 240
(2017), 59–69.
Christopher P. Bridge, Christos Ioannou and J. Alison
Noble, Automated annotation and quantitative description
of ultrasound videos of the fetal heart, Medical Image Analysis 36 (2017), 147–161.
Akiko Ohira, Kei Hayata, Sakurako Mishima, Kazumasa
Tani, Jota Maki, Takashi Mitsui, Eriko Eto and Hisashi
Masuyama, The assessment of the fetal heart function using
two-dimensional speckle tracking with a high frame rate,
Early Human Development 151 (2020).
Lu Xu, Mingyuan Liu and Yihua He, DW-Net: A cascaded convolutional neural network for apical four-chamber
view segmentation in fetal echocardiography, Computerized
Medical Imaging and Graphics 80 (2020).
Jordina Torrents-Barrena, Gemma Piella, Narcıs Masoller,
Eduard Gratacos, Elisenda Eixarch, Mario Ceresa, Miguel
Angel and González Ballester, Segmentation and classification in MRI and US fetal imaging: Recent trends
and future prospects, Medical Image Analysis 51 (2019),
61–88.
P. Jeanty, R. Chaoui, I. Tihonenko and F. Grochal, A Review
of Findings in Fetal Cardiac Section Drawings: Part 1: The
4 – Chamber View, Journal of Ultrasound in Medicine 26
(2007), 1601–1610.
T. Jayaraman and G.S. Chinnusamy, Performance analysis
of a dual stage deep rain streak removal convolution neural
network module with a modified deep residual dense network, International Journal of Applied Mathematics and
Computer Science 32(1) (2022), 111–123.
L. Wu, J.Z. Cheng, S. Li, B. Lei, T. Wang and D. Ni, FUIQA:
Fetal ultrasound image quality assessment with deep convolutional networks, IEEE Transactions on Cybernetics 47
(2017), 1336–1349.
C.F. Baumgartner, K. Kamnitsas, J. Matthew, T.P. Fletcher,
S. Smith, L.M. Koch, et al., SonoNet: real-time detection
and localisation of fetal standard scan planes in freehand
ultrasound, IEEE Transactions on Medical Imaging 36
(2017), 2204–2215.
L.J. Salomon, N. Winer, J.P. Bernard and Y. Ville, A scorebased method for quality control of fetal images at routine
second-trimester ultrasound examination, Prenatal Diagnosis 28 (2008), 822–827.
H. Chen, Q. Dou, D. Ni, J.-Z. Cheng, J. Qin, S.
Li, et al., Automatic Fetal Ultrasound Standard Plane
Detection Using Knowledge Transferred Recurrent Neural
Networks, in Medical Image Computing and ComputerAssisted Intervention – MICCAI 2015, Cham 2015 (2015),
507–514.
S. Yagel, S.M. Cohen and R. Achiron, Examination of the
fetal heart by five short-axis views: a proposed screening
method for comprehensive cardiac evaluation, Ultrasound
in Obstetrics & Gynecology 17 (2001), 367–369.
[21]
[22]
[23]
TH
OR
[4]
C. Shobana Nageswari et al. / Fetal heart chamber segmentation and classification
AU
1040
[24]
[25]
[26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34]
[35]
[36]
C. Shobana Nageswari et al. / Fetal heart chamber segmentation and classification
[41]
J. Kennedy and R. Eberhart, Particle swarm optimization,
Proceedings of ICNN’95 – International Conference on
Neural Networks 4 (1995), 1942–1948.
Y. Meraihi, A.B. Gabis, S. Mirjalili and A. Ramdane-Cherif,
Grasshopper Optimization Algorithm: Theory, Variants,
and Applications, in IEEE Access 9 (2021), 50001–50024.
CO
PY
[39]
[40]
TH
OR
[38]
Xuan-Hien Le, Hung Viet Ho, Giha Lee and Sungho Jung,
Application of Long Short-Term Memory (LSTM) Neural
Network for Flood Forecasting, Water 11 (2019).
S. Hochreiter and J. Schmidhuber, Long short-term memory,
Neural Comput 9 (1997), 1735–1780.
C. Olah, Understanding LSTM networks, Available Online
(2018).
AU
[37]
1041