Accuracy improvement of emotional state detection for exist methods
Bayan Ali Hussein Ali1, Abdeldime Mohamed S2, 3,
Eltaf Abdalsalam M3
1
23
23
Karary University, Sudan.
Department of Electrical Engineering, Karary
University, Sudan,
Department of Electrical Engineering, Blue Nile
University, Sudan,
Abstract - A brain computer interface (BCI) has
opened up a new world of creative possibilities
for academics and designers, providing a variety
of routes of communication to people with
movement limitations. In intelligent medical
treatment and intelligent transportation,
emotion recognition is extremely important.
Human-computer interface technology has
advanced, allowing for the recognition of
emotions via electroencephalograms (EEG). An
EEG emotion recognition framework is put forth
in this study. First, intrinsic mode functions
(IMFs) at various frequencies are obtained by
decomposing the nonlinear and non-stationary
EEG signals using variational mode
decomposition (Fun-VMD). Variational mode
decomposition (Fun-VMD), a time-frequency
feature extraction technique, was individually
integrated with various classification techniques,
including K closest neighbors (KNN), artificial
neural networks (ANN), and support vector
machines (SVM). Eleven subjects'
experimentally recorded data were gathered in
order to validate this approach. The Fun-VMD
feature extraction-based method with SVM
classification achieved an average accuracy of
95.3% in the trial results using the public
dataset. The proposed framework based on SVM
is more precise resulting in the improvement of
accuracy, compared to other classification
methods
Index Terms - Brain-computer interface (BCI),
EEG, Motor imagery, Variational mode
decomposition (VMD), Artificial Neural Networks
(ANN), Support vector machine (SVM) , K nearest
neighbors (KNN) and fast Fourier transform
reference (FFT) .
I. INTRODUCTION
A new era for brain applications has been introduced
by the brain-computer interface (BCI). A unique type
of communication has been made available by BCI
for controlling wheelchair, robot, security,and alarm
systems. Electroencephalography (EEG) is used to
identify the signals, which are then inputted to the
alarm activist. There are several ways to measure
brain signals. Invasive, semi-invasive, and noninvasive techniques are classified among them. In an
invasive procedure, the signals are obtained from
electrodes that have been put into the cortex, whereas
in a semi-invasive procedure, the signals are obtained
from the dura or the arachnoid. The scalp is used to
collect the signals for non-invasive devices. EEGbased emotional sensing has attracted increased
interest, particularly for investigations on impaired
security. Human emotions are fundamental to
decision-making, social interaction, the diagnosis of
mental illnesses like depression, and other processes
[1]. Traditionally, facial expressions like audio
signals, body posture, and gesture have been used by
humans to determine emotions. In comparison, a
machine cannot comprehend another person's
emotions. Affective computing in this context strives
to enhance communication between people and
machines by sensing human emotions, hence
increasing this interaction's accessibility, usability,
and effectiveness. Physiological alterations in the
body are correlated with emotional experience [2].
As a result, understanding each emotion's
physiological response is crucial for conducting an
emotion analysis. Therefore, studies have been
carried out to identify emotions by physiological
markers. Emotions are an important part of human
psychological structure, and as the human-computer
interaction technology develops, emotional
perception computing has also improved. One of the
important prerequisites for conducting emotion
research is to elicit objective, stable and reliable
emotions [3]. Researchers use a variety of emotion
stimuli, such as images, sounds and videos, to induce
emotions. Video materials of different emotions are
widely used by researchers through visual and
auditory stimuli. The subjects may feel personally on
the scene [4]. Existing publicly available EEG
databases based on emotion video stimuli includes
DEAP, MAHNOB-HCI and SEED [5]. This study
aims to assess the brain signals of people with
disabilities who are in ricks state. The P300 model,
which focuses on the visual cortex, serves as the
foundation for signal detection. Hardware and
software could be used to develop the system in an
experimental setting. The alarm system could help
with additional configuration and interface of the
EEG signals.
Activity
Feature
extraction
method
Emotion
Sensing
[6 ]
DFT,
FFT,CUD
A
Emotional
State Evoked
by Virtual
Environment
[7 ]
PCA
Clas
sific
atio
n
met
hod
CN
N
XG
Boo
st ,
SV
C,
LR
Number of
classes
In each task
imagination
of emotional
states were
encouraged
and using a
set of prerecorded
verbal
suggestion
emotions
subject.
Collected and
analyzed
EEG signals
in virtual
environments
to identify
emotional
states based
on neural
variations. To
compare
emotional
influences in
Nu
mbe
r of
part
icip
ants
34
15
virtual
content.
Emotion
Detection
[ 8]
statistical
modelling
method
LCT
Emotion
Classification
Using EEG [9
]
Fear learning
in humans [10
]
Emotion
Recognition
[11 ]
Cross-Day
EEG-Based
Emotion
Recognition[1
2]
The brain
wave of
human lies[13
]
manual
feature
extraction
FFT
The
Librosa
tool in the
python
package
22
CN
N,R
NN
KN
N,D
NN
As
Ma
p+C
NN
Neu
rom
eric
EE
G.
SV
M,
Ran
do
m
For
est,
Dec
isio
n
Tre
e
TCA
SV
M
Wave
voltage
powers .
Stati
stica
l
sign
ifica
nt
valu
es
PCD
RFC
Classification
accuracy
72.03%.
The best
classification
Multi-Class
Emotion
Recognition
[14 ]
Emotion
Recognition
Based on EEG
Signal Using
Deep
Learning
[15 ]
EMD and
VMD
SV
M,
KN
N
emotion:
anger, calm,
happy and
sad,
75%
Three basic
human
emotions
named
positive,
negative, and
neutral.
Fear
15
30
95.56%
Happiness,
Sadness ,
Angry ,
Fear
2
90%
12
83.03%,
5
Theta, Alpha
and Gamma that
related to the
human lie
activity
23
99.79%
positivenegative, joysadness, joyanger, and
joy-fear
Selected the
pictures of
cat, lion,
tiger,
elephant,
octopus,
crap, fish,
starfish, turtle
and pig for
this research.
We simulated
situation like
human lie by
answering
about the nottrue type of
animal they
see.
emotions
within the
Valence­Aro
usal­Domina
nce model
ExcitedDelightedDelightedCalm
ContentTired–
DepressedFrustrated
Angry Tense- Bored
-Relaxed
97.10%
70.89%
28
In this paper, the following questions are addressed:
(1) Is it possible to achieve discriminations between
the 4 classes of emotional movements (laugh- TenseRelaxed- sad)?
(2) Do feelings of laughter, relaxation and tension
produce a difference in performance despite sharing
the same electrode?
(3)Which method is more suited for the classification
of three emotions for BCI?
II. MATERIAL AND METHODS
A. Participant and Experimental Setup
Ten healthy participants between the ages of 20
and 30 (4 men and 6 women) took part in the
study. The 4.1 standard deviation. The test setup
included a Muse headset, a portable,
rechargeable headband. It has four EEG
electrodes: two near the ears, one just above the
eyes, and two on the forehead. It also features a
gyroscope and an accelerometer, allowing you to
determine the head orientation. 2016 Headband
EEG apparatus Positions of the EEG sensors
were TP9, AF7, AF8, and TP10. The ground
electrode and signal reference are both located in
the right ear lobe. According to Fig. 1, the
sampling frequency was 500Hz. [5]
This study has designed EEG experiments [16 ] for
studying emotions , which allows us to collect
sufficient emotional samples for deep neural
network studies, and investigate the properties of
EEG signals . We selected emotional video
materials for the experiment because they may
provide the subjects with both visual and auditory
stimulation, giving them the impression that they
are in a real-life situation. Videos of the four
emotion types guffaw. - Sad, tense, and relaxed
have been chosen.
Affective Video It includes comedies, crime, war,
documentary, and horror films, etc. As shows in
Fig. 2 the experimental session perform mental task
thinking.
Throughout the session, a video is displayed on the
participant's screen (represents a specific emotional)
Each stimulus is displayed continuously for three
minutes, but once that time is over, the screen will
remain blank for one minute before the next
stimulus appears. As a result, there is a break
between each run of one minute during which the
subject may blink or stretch.
Fig. 2 Experimental session protocol
Fig. 1 EEG Muse device
B. Experimental Session
C. Pre-processing
Pre-processing is an important step for enhancing the
quality of EEG signals. The input signals underwent
a number of processing processes to lower the signal-
to-noise ratio and data dimensionality, including the
elimination of data artefacts, EMG, eye movement
and eye blink potentials .using steps were applied to
the pre-processing stage involved the following
processes: the inclusion of a temporal filter which is
a 0.5 Hz to 30 Hz band-pass filter using a third order
Butterworth; the usage of a fast Fourier transform
reference (FFT) filter [17] as shows in Fig3.
VMD transforms the signal decomposition process
into a variational framework and realizes the
adaptive decomposition of the signal by searching
for the optimal solution of the constrained variational
model. During the iterative solution of the variational
model, the frequency center and the bandwidth of
each component of the IMF are continuously Sensors
2022, 22, 6698 4 of 19 updated. Finally, the adaptive
division of the signal frequency band is completed
according to the frequency characteristic of the
signal and a plurality of narrow-band IMF
components are obtained.[19] Assuming that the
original signal is decomposed into K IMF
components through VMD, the expression of the
constrained variational model is
Fig. 3 Block Diagram Pre-processing
D. Feature Extraction
The next subsections detail the feature extraction
approaches that were utilized to depict EEG signals.
D.1 Variational mode decomposition
VMD is an approach for adaptive and quasi
orthogonal signal decomposition. Solution of a
minimization problem is found in a sequence of
iterative sub - optimizations. Multi - component
signals are decomposed into several band limited
intrinsic mode functions IMF as the bandwidthconstrained AM-FM signal. Method aims to
decompose a composite EEG signal, x (t) into m
number of user-defined modes, μ (t) called sub
signals [18].
VMD algorithm decomposes adaptively a signal.
Where
and
is the amplitude envelope of
is the instantaneous phase of
, and
where
decomposed
by
is the K IMF components
the
VMD
method,
is the frequency center of
each IMF component
is the partial derivative
of the function time, δ (t) is the unit impulse
function, j is the imaginary unit, ∗ means
convolution, and f represents the original signal
[20].The quadratic penalty function term α and the
Lagrangian multiplication operator λ are introduced
to solve the optimal solution of the above constrained
variational problem, and the expression is:
In the formula, α is the quadratic penalty factor. The
function is to reduce the interference of Gaussian
signals. In order to ensure the accuracy of signal
reconstruction, α is generally set to be a large enough
positive number, and λ is the Lagrange multiplication
operator. The optimal solution of the constrained
variational model of Equation is to use the alternating
direction multiplier algorithm to obtain the saddle
point of the Lagrangian function, thereby obtaining
the narrow-band IMF component [21]. The specific
process is as follows:
(1) Initialization parameter
(2) n = n + 1;
(3) k = k + 1, traversing k = 1 − K, update uˆ n+1
k and ωˆ n+1 k with the following formulas,
respectively:
(4) Update the Lagrange multiplier λ
In the formula, γ is the noise tolerance, which
meets the fidelity requirements of signal
decomposition.
Corresponds to the Fourier transform of
respectively[22].
(5) Repeat steps 2–4 until the convergence
condition of the following equation is satisfied
Fig. 1 Flowchart of empirical mode Variational
decomposition algorithm
For a given judgment accuracy ε > 0, end the loop.
D.2 Feature selection
In this study three different statistical features were
selected for EEG classification using approach, i.e.,
VMD, aiming at decreasing the dimensionality of
EEG data. The rationale to use signal statistics, or to
extract statistical features is to capture important
information while keeping the data dimensions low.
These statistical features are:
. Energy
. Entropy
. Absolute power values
solve Classification and Regression problems. SVM
selects the extreme vectors and points that aid in the
creation of the hyper plane. The approach is referred
described as a "support vector machine" because of
these extreme circumstances [24].
E.3 K-Nearest Neighbors
K-Nearest Neighbors (KNN) is a straightforward
supervised machine learning (ML) technique that is
often used in missing value imputation and can be
utilized for classification or regression applications.
It is predicated on the notion that the observations
most "similar" to a given data point are those that are
closest to it in the data set, allowing us to categorize
unanticipated points based on the values of the
existing points that are closest to them. The user can
choose K to specify how many neighboring
observations will be used in the algorithm [25].
Fig. 2 the proposed for Feature Extraction Code
using the VMD method and Power, Energy Entropy
E. Classification
E.1 Artificial Neural Networks
The most popular classifiers used in BCIs are
artificial neural networks (ANN), which might offer
a useful foundation for pattern-recognition issues.
Similar to human interaction, ANN can analyses,
identify, and model nonlinear interactions between
data [23 ].
Fig. 3 Flowchart of the proposed classifier using the
VMD method
III. RESULTS AND ANALYSIS
E.2 Support Vector Machines
One of the most well-liked methods for Supervised
Learning, Support Vector Machine (SVM) is used to
A.1 Data analysis
A-1-1 Variational mode decomposition
For the purpose of results presentation, the EEG of a
single subject is illustrated. Figure 4 shows. The EEG
signal of TP9 channel of muse device, for video
watched by a single subject. Figure 5 shows.
Fig. 4 Mind Monitor –Absolute Brain Waves
Figure 6. IMFs after EEG signal decomposition of a
single subject.
All IMF1–IMF5 features are extracted using a
sliding window. There are 16,800 dimensional
characteristics. Obtained. The CRRAEN is utilized
for variable selection in order to decrease the time
cost, and the 168- After variable selection,
dimensions features are obtained and entered into the
WCF for classification.
Figure5. A single subject in the TP9 channel EEG
signal.
A series of IMFs with various center frequencies are
obtained after VMD decomposition, as shown in
Figure 6. As can be seen from the figure
A-1-2 Classification outcome
From the experimental, the data was classified
offline, the majority of the subject obtained an
accuracy of 53-93.5%.
However, accuracy values lower than 88.1% were
observed in five subjects. The mean accuracy
average over the subjects was
Approximately 95.3 using SVM. Table 1 shows the
classification accuracy obtained from three
classifiers.
Table 1 Results obtained the classification of
participants
subject
KNN
SVM
ANN
Accuracy
Accuracy
Accuracy
)%(
)%(
)%(
S1
89.9%
85.2%
87.5%
S2
S3
S4
S5
S6
S7
S8
S9
S10
S11
Mean
90.3%
88.3%
90.5%
89.5%
89.3%
91.9%
93.2%
85.5%
87.3%
90.1%
89.6
85.9%
89.7%
92.3%
91.2%
87.5%
88.3%
83.7%
81.9%
85.9%
88.4%
95.3
86.5%
91.4%
88.7%
90.5%
83.8%
93.3%
87.7%
83.8%
89.5%
86.7%
88.1
The effects of different classifiers and subjects
performance on the accuracy are presented in
Fig.7.The accuracies obtained were 89.9% with
KNN 85.2% with SVM and 87.5% with ANN for S1,
for S2 were 90.3% with KNN, 85.9% with SVM and
86.5% with ANN, for S3 were 88.3% with KNN
89.7% with SVM and 91.4% with ANN and for S4
were 90.5% with KNN, 92.3% with SVM and 88.7%
with ANN.
Class motor imagery using SVM classifier is
presented in Fig8
Fig. 5 The Roc curve for hello and bad using SVM
IV. CONCLUSION
A system based on Muse EEG and Fun-VMD feature
extraction, was successfully built. In addition this
system was developed and obtained average
accuracy by 95% by using SVM. We think that by
learning more about some of the key elements
involved in the experiments described, a higher
performance can be attained. Some of these elements
include setting up the framework with various EEG
acquisition tools, trying to adapt, choosing the
optimal electrode placements near the visual brain,
and conducting testing. The framework's
performance under various circumstances and track
variations.
Fig.7 the Comparisons among KNN, SVM and
ANN
ACKNOWLEDGMENT
The receiver operating characteristic (ROC) curves
of the 4
We would like to thank the writers for their
contributions and acknowledge the support of the
participants.
REFERENCES
1- Ali, B. A. H., Mohamed, E. A., & Idriss, A. B. A
Security System for Door Opened by Using Brain
Signal. In 2020 International Conference on
Computer, Control, Electrical, and Electronics
Engineering (ICCCEEE) (pp. 1-5). IEEE
2- Merkel, R., Boer, G., Fegert, J., Galert, T.,
Hartmann, D., Nuttin, B., & Rosahl, S. (2007).
Central neural prostheses. Intervening in the brain:
Changing psyche and society, 117-160.
3- Shivganesh, S. (2022). EEG Based Emotion
Sensing using convolutional neural networks. arXiv
preprint arXiv:2201.03432.
4- Kimmatkar, N. V., & Babu, B. V. (2021). Novel
Approach for Emotion Detection and Stabilizing
Mental State by Using Machine Learning
Techniques. Computers, 10(3), 37.
5- He, Z., Zhuang, N., Bao, G., Zeng, Y., & Yan, B.
(2022). Cross-Day EEG-Based Emotion Recognition
Using Transfer Component Analysis. Electronics,
11(4), 651.
6- Shivganesh, S. (2022). EEG Based Emotion
Sensing using convolutional neural networks. arXiv
preprint arXiv:2201.03432.
7- Jung, D., Choi, J., Kim, J., Cho, S., & Han, S.
(2022). EEG-Based Identification of Emotional
Neural State Evoked by Virtual Environment
Interaction. International Journal of Environmental
Research and Public Health, 19(4), 2158.
8-Su, Q., Cheng, G., & Huang, J. (2023). A review
of research on eligibility criteria for clinical trials.
Clinical and Experimental Medicine, 1-13.
9-Ahmed, M. Z. I., Sinha, N., Phadikar, S., &
Ghaderpour, E. (2022). Automated Feature
Extraction on AsMap for Emotion Classification
Using EEG. Sensors, 22(6), 2346.
10- Santos-Mayo, A., de Echegaray, J., & Moratti, S.
(2022). Conditioned up and down modulations of
short latency gamma band oscillations in visual
cortex during fear learning in humans. Scientific
reports, 12(1), 1-10.
11-Hulliyah, K. (2021). Analysis of Emotion
Recognition Model Using Electroencephalogram
(EEG) Signals Based on Stimuli Text. Turkish
Journal of Computer and Mathematics Education
(TURCOMAT), 12(3), 1384-1393.
12-He, Z., Zhuang, N., Bao, G., Zeng, Y., & Yan, B.
(2022). Cross-Day EEG-Based Emotion Recognition
Using Transfer Component Analysis. Electronics,
11(4), 651.
13-Lawpradit, W., Petharn, S., & Fongngen, W.
(2022). The EEG Brain Signals Related to Humans
Lie Activity by Lightweight Device. วารสาร
วิชาการ พระจอมเกล้า พระนครเหนือ, 32(1), 134142.
14-Gaertner, M., Sauter, D., Baumgartl, H., Rieg, T.,
& Buettner, R. (2021). Multi-Class Emotion
Recognition within the Valence-Arousal-Dominance
Space Using EEG. In AMCIS.
15-Abdulrahman, A., Baykara, M., & Alakus, T. B.
(2022). A Novel Approach for Emotion Recognition
Based on EEG Signal Using Deep Learning. Applied
Sciences, 12(19), 10028.
16- Mohamed, E. A., Yusoff, M. Z., Malik, A. S.,
Bahloul, M. R., Adam, D. M., & Adam, I. K. (2018).
Comparison of EEG signal decomposition methods
in classification of motor-imagery BCI. Multimedia
Tools and Applications, 77(16), 21305-21327.
17- Maitin, A. M., Romero Muñoz, J. P., & GarcíaTejedor, Á. J. (2022). Survey of machine learning
techniques in the analysis of EEG signals for
Parkinson’s disease: A systematic review. Applied
Sciences, 12(14), 6967.
18-Abdulrahman, A., Baykara, M., & Alakus, T. B.
(2022). A Novel Approach for Emotion Recognition
Based on EEG Signal Using Deep Learning. Applied
Sciences, 12(19), 10028.
19-Xu, D., Qin, X., Dong, X., & Cui, X. (2023).
Emotion recognition of EEG signals based on
variational mode decomposition and weighted
cascade forest. Mathematical Biosciences and
Engineering, 20(2), 2566-2587.
20-Deng, X., Lv, X., Yang, P., Liu, K., & Sun, K.
(2022, August). Emotion Recognition Method Based
on EEG in Few Channels. In 2022 IEEE 11th Data
Driven Control and Learning Systems Conference
(DDCLS) (pp. 1291-1296). IEEE.
21- Sakkas, K., Tsogka, A., Gkimitzoudis, A.,
Giannakeas, N., Tzimourta, K. D., Tsipouras, M., ...
& Tzallas, A. T. (2022, September). Analysis of
Emotions through the Use of Physiological Signals.
In 2022 7th South-East Europe Design Automation,
Computer Engineering, Computer Networks and
Social Media Conference (SEEDA-CECNSM) (pp.
1-7). IEEE.
22- Vesala, G. T., Srinivasarao, G., Ghali, V. S.,
Sastry, D. V. A., & Naik, R. B. (2022). NonStationary Thermal Wave Mode Decomposition: A
Comparative Study of EMD, HVD, and VMD for
Defect Detection. Russian Journal of Nondestructive
Testing, 58(6), 521-535.
23- Woldseth, R. V., Aage, N., Bærentzen, J. A., &
Sigmund, O. (2022). On the use of artificial neural
networks in topology optimisation. Structural and
Multidisciplinary Optimization, 65(10), 1-36.