CLASSIFCATION OF DIFFERENT MENTAL TASKS USING NEURAL
NETWORKS
KOUHYAR TAVAKOLIAN
Computer Science, UNBC
Prince George, Canada
SIAMAK REZAEI
Computer Science, UNBC
Prince George, Canada
A.M. NASRABADI
Tehran Polytechnics
Tehran,Iran
Abstract: - In this research, some important aspects of mental task classification are investigated. The effect of neural
network topology on the accuracy of classifications is evaluated. The implemented approach towards this problem is leaveone-out-method or LOOM. Different combinations of mental tasks are classified and are compared with each other.
According to the results for three of our subjects, better mental tasks are determined. This can be developed into a method for
increasing the accuracy of Brain-Computer Interfaces.
Key-words: -Electroencephalographic Signal, Leave-One-Out-Method, Brain Computer Interface, Mental Tasks.
1 Introduction
Classification
of
mental
tasks
by
recognizing
Electroencephalographic patterns is an important and
challenging biomedical signal processing problem. This
would enable a patient to communicate by a computer without
an overt physical action and just by processing his/her brain
waves [1]. Developments of faster digital computers and
better EEG devices have motivated many researchers to work
on brain computer interfaces.
Linear and Nonlinear features have been extracted from the
EEG signal and have been processed in recognition and
classification of mental tasks. Anderson et al [2], [3] had
multivariate AR model coefficients as their feature vector by
considering cross channel EEG features. Nasrabadi et al [4],
[5] extracted fractal dimension features from the signal
considering the nonlinear aspects of EEG feature and using
Higuchi and Petrosian methods.
So far in many previous works, neural network has been the
classifier [3]. We have also used a FeedForward neural
network trained by the error back propagation algorithm. As
part of our current research we have investigated the effects of
Neural Network topology (i.e. the number of hidden layers
and neurons) on classification of mental tasks.
It is shown that for each subject, a set of mental tasks can be
better discriminated compared to other tasks. In other words
each subject can produce more distinct EEG signal during
specific mental tasks. In the context of brain computer
interface this may play an important role to increase the
accuracy. So that we can let the subject to choose those tasks
which are more sited for him/her to drive the interface. In this
research, to some extent we have addressed this problem.
2 Methods
2.1 EEG dataset and mental tasks
In this research, we have applied our algorithms to the EEG
dataset gathered by Z. Keirn [1]. This dataset has been a
reference of several different works [1], [2], [3] and had been
taken from 7 subjects during performance of 5 different
mental tasks. The subjects were seated in a sound controlled
booth with dim lighting and noiseless fans for ventilation. An
Electro-Cap elastic electrode cap was used to record from
positions C3, C4, P3, P4, O1, and O2, defined by the 10-20
system of electrode placement. The electrodes were connected
through a bank of Grass 7P511 amplifiers and bandpass
filtered from 0.1--100 Hz. Data was recorded at a sampling
rate of 250 Hz with a Lab Master 12 bit A/D converter
mounted in an IBM-AT computer. Eye blinks were detected
by means of a separate channel of data recorded from two
electrodes placed above and below the subject's left eye.
The subjects were asked to perform five mental tasks: a
baseline task, for which the subjects were asked to relax as
much as possible; the letter task, for which the subjects were
instructed to mentally compose a letter to a friend or relative
without vocalizing; the math task, for which the subjects were
given nontrivial multiplication problems, such as 49 times 78,
and were asked to solve them without vocalizing or making
any other physical movements; the visual counting task, for
which the subjects were asked to imagine a blackboard and to
visualize numbers being written on the board sequentially; and
the geometric figure rotation, for which the subjects were
asked to visualize a particular three-dimensional block figure
being rotated about an axis. Data was recorded for 10 seconds
during each task and each task was repeated five times per
session.
2-2 Neural Network Topology:
To investigate the efficiency of different network topologies
in the classification of mental tasks we have two parallel
approaches. In both approaches we considered classification
of five mental tasks by extracting scalar autoregressive
coefficients (6th orders) from half second windows with
overlap of quarter second and from the EEG signal of just one
trial. So there were 38 feature vectors for each mental task [5].
The method for testing classification accuracy in this approach
is leave-one-out-method (LOOM). The LOOM trains on all
but one of the vectors and tests on the one that is left out [1].
This is then repeated leaving a different vector out each time
until all of the data has been tested. The classification
accuracy is then determined by dividing the number of
correctly classified vectors by the total number of vectors.
This method is more conservative and also time consuming
than the others. By considering the above number of windows
and five mental tasks the neural network was trained and
tasted for 190 times. The scalar AR (6) coefficients were
extracted from the subject five during her first session. The
classification was performed by several neural network
topologies including neural network having one, two, three
layers of hidden neurons. In the case of one hidden layer, the
neural network having 20 neurons seemed to have a better
classification accuracy compared to others. So for other parts
of our research we considered one hidden layer of 20 neurons
as the working structure.
2-3 Combination of Different trials
Considering the non-stationary nature of the EEG signal, in
this part we merged the signal of different trials and sessions
together and classification was done on this signal. We
expected the classification to be lower than the single trial
case seen in the previous section. In this phase of our research
according to the previous results we have used a neural
network having a single layer of 20 neurons, as the classifier.
We have implemented a 5-fold cross validation [6] to select
training and testing data for the classifier. In each
classification we considered averaging the classification
results on 44 different combinations of the training set.
Considering the 5-fold cross validation this means all the
possible combinations of the data segments. To comply with
[2] autoregressive model coefficients of order 6 were
extracted from the signal. In this part we considered 1 second
windows that had 0.5 second overlap (except for section 3-1)
It should be considered that in our dataset, subjects 2 and 7
have attended in just one session and subject 5 has attended in
three sessions and other subjects have 2 sessions of EEG
signal. So we are expecting subject 5 to have more consistent
results compared to others.
3 Results
3-1 Classification of five mental tasks
In this section we considered classification of five mental
tasks. As before we considered five neurons in the output,
corresponding to each mental task. The neural network was
trained for 44 times and in each time it was trained for 6000
epochs and the final results were averaged on 44 repetitions.
For windows length of 0.5s and overlap of 0.25s the average
of classification on subject’s number 1, 3, 5 and 6 was 63.85%
but by averaging the results on the output for every 6
consecutive windows this classification was increased to
85.34%.
3-2 Classification of 2 mental tasks
In this section two mental task were classified for different
subjects. These tasks were chosen to be Baseline and
Multiplication. In this phase a single neuron was considered at
the output with threshold of 0.5 for classifying two tasks. The
experiment settings were set to be similar to [2] to compare
the final results but classification were done 44 times,
compared to 30 times of [2], which would be expected to yield
more consistent results.
Both experiments were done on the same dataset. From [2] it
is clear that the experiment has been done on subjects having
two EEG session (1, 3 and 6) and one of single session
subjects (2 and 7) which is not stated explicitly which one.
The two tasks were Baseline and Multiplication. Anderson
reported the result of 90.6±2 as the average classification
accuracy of four subjects.
We considered the same experiment but with two different
possibilities of the above subjects meaning subjects: 1, 3, 6
and 2 which resulted in 92.75±3.19 and subjects: 1, 3, 6 and 7
that resulted in 92.5±3.33 result. In both cases our results were
better. This can be related to the way in which the neural
networks weights are updated. In our method by the help of
Matlab we used an adaptive method to update learning rates
but in [2] they have used constant learning rate for each layer
which remains constant during training.
3-3 Binary, ternary and 4-nary Classification
In our work the classification accuracy of different
combinations of mental tasks were compared and notable
results were achieved. In this part as before the same neural
network architecture was considered but we considered
windows length of 1 second with overlap of 0.5s. As before 5
fold cross validation was implemented and averaged over 44
combinations of training and dataset.
In case of subject 5 aside from total of three sessions we
considered classification of different two combinations of
sessions and the results were compared to each other. The
following results could be observed from the attained
classification tables. In the following section we have
considered the first letter of each mental task name as an
abbreviation for that task (i.e. B for Baseline, M for
Multiplication, C for Counting, R for Rotation and L for
Letter writing).
Determination of the optimal mental tasks for each subject
will help us to increase the accuracy and reliability of an
important aspect of brain-computer interfaces, based on
classification of mental tasks.
For subject five it was noted that for binary mental task
classifications, L/R and M/R combinations have always been
superior to other eight combinations (Figure 1). By focusing
on ternary mental task classification table, it becomes clear
that the B/M/C combination has the least classification
accuracy compared to other nine ternary combinations (Figure
2). This is the combination that excludes the L and R mental
tasks. The quaternary mental task classification shows that
B/M/L/C combination which excludes R task is lower in
classification compared to other four combinations (Figure 3).
According to the above conclusions it can be said that L/R
combination is the best binary combination of mental tasks
and R (i.e. Rotation) is the best task for this particular subject.
For subject 5 the above observations are summarized in three
charts (see the appendix).
References:
For subject six it is observed, for the quaternary (4-nary)
mental tasks, combination B/L/R/C which excludes the M task
is lower in accuracy compared to others. Binary mental task
classification for this subject shows that the two combinations
C/M and B/M are better than others. Ternary combinations
show that the combination excluding C and M task is worse
than others. So it can be inferred that for this subject the C/M
combination is better than others and the M (i.e.
Multiplication task) is the best one.
For subject three, it is observed that the combination B/M/R/C
has the lowest rate between the quaternary tasks which
excludes L task. It is also observed that the two combinations
C/L and B/L have the highest classification accuracy so it can
be said that the task L is the best task for this subject. For
subjects two and four, as there was just one session of EEG
signal available, we had no possibility of comparison. For
subject one, we could not acquire meaningful results.
4 Conclusion
The efficiency of the feedforward neural network with single
hidden layer of 20 neurons in the classification of different
mental tasks was investigated. For different subjects some
mental tasks are determined to be better than others. We
should note that these results are obtained from a small set of
EEG signal sessions and for more consistent results we need
more EEG signal sessions.
In our future work we aim to acquire EEG signal from
subjects during performance of different mental tasks (more
than 15 tasks). We will consider multi session signal
acquisition to be able to generalize our results during the time.
[1] Zachary A.Keirn Jorge I. Aunon, ‘A new mode of
Communication between Man and His surroundings’ IEEE
Trans. On BME, Vol. 37, No. 12, December 1990.
[2] C. W. Anderson, E. A. Stolz, S. Shamsunder “Multivariate
Autoregressive Models for Classification of Spontaneous
Electroencephalographic Signals During Mental Tasks”, IEEE
Trans. On BME, Vol. 45, No. 3, March 1998.
[3] Janne Lehtonen ‘EEG-based Brain Computer Interfaces’
Master’s thesis. Helsinki University of Technology, May
2002.
[4]Kouhyar Tavakolian and M.Nasrabadi, ‘Comparison
between Linear and Nonlinear EEG Signal Processing during
Different Mental Activities’, SCI2003, July 27-30, 2003
Orlando, Florida, USA
[5]Kouhyar Tavakolian, A.M.NasrAbadi, S.K Setarehdan,
M.A Khalilzadeh, R. Miri, M.Falaknaz, ‘EFFECTS OF
DIFFERENT FEATURE VECTORS AND NEURAL NETWORK
TOPOLOGY ON EEG MENTAL TASKS CLASSIFICATION’,
Proceedings of the International IASTED Conference,
Biomedical Engineering, pages 39-43. , Salzburg, Austria, 2527 June 2003
[6] Jan Larsen and Cyril Goutte, ‘On Optimal Data Split for
Generalization Estimation and Model Selection’, Department
of athematical Modelling, Technical University of Denmark.
Submitted for IEEE Neural Networks for Signal Processing,
1999.
5. Appendix
88.77
100
89.97
84.39
78.87
86.39
77.67
73.69
80
93.98
89.17
88.2
60
40
20
0
B/M
B/C
B/L
B/R M/C M/L M/R C/R
C/L
L/R
Figure 1 Binary Classification of Mental tasks for subject 5
90
80
70
60
50
40
30
20
10
0
81.6399
77.6934
78.2531
76.139
74.2424
67.6025
67.5579
60.2941 60.2941
59.3583
L/R/C L/M/R C/M/R C/M/L B/L/R B/R/C B/L/C B/L/M B/M/R B/M/C
Figure 2 Ternary Classification of Mental tasks for subject 5
62.2159
70
60
56.484
58.3723 61.8817
48.5127
50
40
30
20
10
0
B/M/R/C B/M/R/L B/M/L/C B/L/R/C L/M/R/C
Figure 3 Quaternary (4-nary) Classification of Mental tasks for subject 5