CONTROL OF ONE-DIMENSIONAL CURSOR MOVEMENT BY NONINVASIVE
BRAIN-COMPUTER INTERFACE IN HUMANS
SITI ZURAIMI SALLEH
A thesis submitted in fulfilment of the requirements for the award of the degree of
Master of Engineering (Electrical)
Faculty of Electrical Engineering
Universiti Teknologi Malaysia
APRIL 2011
To my mother and father, husband, and sensei, for their precious moments iii
iv
ACKNOWLEDGEMENT
In the name of Allah, the Almighty and the Merciful. Alhamdulillah, praise to
Allah S.W.T for the guidance, strength and bless which He gave upon me to complete this research work.
First and foremost, I would like to express my sincere gratitude and appreciation to my beloved supervisor, Dr. Norlaili bt Mat Safri for her advice, knowledge, supports, kindness and patience throughout this research work. I also would like to express my thanks to Dr. Fauzan, Mr. Afzan, Mrs. Husnaini and Ms. Ashikin for their useful discussion, assistance and supports.
I am also indebted to Ms. Wan and Ms. Fiza from Medical Electronic Lab,
Universiti Teknologi Malaysia for spending their time of my overtime works. Finally, special appreciation is dedicated to my husband, family and friends for their constant support and prayers during my research work.
v
ABSTRACT
Noninvasive brain-computer interface (BCI), using electroencephalogram
(EEG) that records from scalp, could provide a new non-muscular channel for sending messages and command to the external world. The objectives of this study are to determine parameters that will drive cursor in BCI using noninvasive EEG signal and to control one dimensional cursor movement using the extracted parameters. This experimental-based study involved six normal subjects aged from
20 to 26 years old. Subjects were asked to perform tasks in two condition i.e. control condition and task condition. In the control condition, subjects were required to relax
(resting) and fix their eyes on the centre of the screen without image displayed on the screen. In the task condition subjects were tasked to imagine a movement to move the cursor on the screen towards the target. These control and task conditions were repeated four times and each condition lasted for 10 seconds. Using Fast Fourier
Transform, data in frequency domain for control and task have been obtained and analyzed in two of time interval of 1024 ms and 512 ms. Frequency is divided into six groups, i.e. delta band (0-4 Hz), theta band (4-7 Hz), alpha band (8-13 Hz), beta band (13-30 Hz), gamma band (31-50 Hz) and high gamma band (>51 Hz). Each frequency band in all frequencies of the task condition has been compared to the control condition. The present study finds optimum difference in delta frequency band between resting and active imagination at the parietal region. Furthermore, the parietal region is associated with sensory interaction and could be one of the input regions to control cursor movement. However, it is found that the delta frequency band is only applicable to a specific one-dimensional cursor movement as any imagination may produce the same results. Nevertheless, this study provides a platform for a more advance two-dimensional cursor movement study.
vi
ABSTRAK
Pengantaramuka otak-komputer (BCI) tak invasif menggunakan elektroensefalogram (EEG) yang direkod dari kulit kepala boleh menyediakan laluan baru tanpa otot untuk menghantar isyarat dan arahan ke persekitaran. Objektif kajian ini adalah untuk mengenalpasti beberapa parameter yang akan menggerakkan kursor dalam BCI dengan menggunakan isyarat EEG tak invasif dan mengawal pergerakan kursor dalam satu dimensi menggunakan parameter yang telah diekstrak. Kajian yang berunsurkan ekperimen ini melibatkan enam orang subjek normal berusia di antara 20 hingga 26 tahun. Kesemua subjek perlu melaksanakan tugas dalam dua keadaan iaitu keadaan tugasan dan keadaan kawalan. Dalam keadaan kawalan, subjek hanya perlu berehat dengan memandang ke arah tengah skrin tanpa paparan imej. Manakala dalam keadaan tugasan, subjek diminta membayangkan satu pergerakan untuk menggerakkan kursor ke arah target pada skrin. Kedua-dua keadaan ini diulang sebanyak empat kali dan setiap satunya berlangsung selama 10 saat. Menggunakan jelmaan Fourier cepat, data dalam bentuk frekuensi diperolehi dan dianalisis dalam dua sela masa iaitu 1024 ms dan 512 ms. Julat frekuensi pula dibahagikan kepada enam kumpulan iaitu delta (0-4 Hz), teta (4-7 Hz), alfa (8-13
Hz), beta (13-30 Hz), gama (31-50 Hz) dan gama tinggi (> 51 Hz). Setiap frekuensi dalam keadaan tugasan dan kawalan dibandingkan. Kajian ini mendapati bahawa perbezaan frekuensi dalam keadaan rehat dan imaginasi adalah optimum dalam julat delta di bahagian parietal. Dalam kajian ini, bahagian parietal berkaitan dengan interaksi deria dan menjadi salah satu input untuk mengawal gerakan kursor. Walau bagaimanapun, didapati bahawa jalur frekuensi delta hanya boleh diaplikasi ke atas gerakan kursor satu dimensi sahaja di mana sebarang imaginasi boleh memberikan hasil keputusan yang sama. Namun demikian, kajian ini juga memberi ruang yang menjurus kepada kajian gerakan kursor dua dimensi.
vii
TABLE OF CONTENTS
CHAPTER TITLE PAGE
DECLARATION ii
DEDICATION iii
ACKNOWLEDGEMENT iv
ABSTRACT v
ABSTRAK vi
CONTENTS vii
TABLES xi
LIST OF FIGURES xii xvi
APPENDICES xvii
1 INTRODUCTION 1
1.3 Objectives 3
viii
REVIEW 6
International Electrode
BCI 11
Published on
Published on
Published on
Published on
2.6 Discussion on Published Work in BCI 20
METHODOLOGY 22
Board Adapter 24
3.3 Data Acquisition and Recording Process 26 ix
Flow of
4 FREQUENCY ANALYSIS AND FINDINGS
4.2 EEG Signals
35
35
4.3.2 Time
5 DISCUSSION 48
CONCLUSION 51 x
REFERENCES 53
xi
TABLE NO.
LIST OF TABLES
TITLE PAGE
4.3 and time 46
Comparison of hit rates and movement precision 46 of and
FIGURE NO. xii
LIST OF FIGURES
TITLLE PAGE
2.1
2.1 (a)
2.1 (b)
2.1 (c)
EEG waveforms
Delta wave
Theta wave
Alpha wave
2.1 (d)
2.1 (e)
Beta wave
Gamma wave
2.3
The placement
Basic design and operation of a BCI system
8
8
8
8
8
8
10
xiii used 23
3.1 (a)
3.1 (c)
Electro-cap 23
Electro adapters 23
Body harness 23 needle/syringe 23
Ear electrodes 23 3.1 (f)
Electrode adapter
3.3
3.4
3.5
Division of data for 1024 ms and 512 ms time interval
Flow chart of parameter determination process
Flow chart of finalized parameter determination process
27
29
30
3.6(b) Workspace 31
3.7(b) Baseline 32
3.8(b) Cursor 33
3.9(b)
4.1
4.2
4.3
4.4 (a)
4.4 (b)
4.4 (c)
4.4 (d)
4.5 (a)
4.5 (b)
4.5 (c)
Target
Example of 1 second of raw EEG signal from subject 1
34 at P
Z
channel
Example of power spectrum of EEG signal from subject 1
36 at P
Z
channel 36
Example of 1 second data of EEG signal from subject 1 37
The occurrences of maximum DP for trial 1 for 1024 ms time interval 38
The occurrences of maximum DP for trial 2 for 1024 ms time interval 39
The occurrences of maximum DP for trial 3 for 1024 ms time interval
The occurrences of maximum DP for trial 4 for 1024 ms
39 time interval
The occurrences of maximum DP for trial 1 for 512 ms
40 time interval
The occurrences of maximum DP for trial 2 for 1024 ms
41 time interval
The occurrences of maximum DP for trial 3 for 1024 ms
41 time interval 42 xiv
4.5 (d)
4.6
4.7
The occurrences of maximum DP for trial 4 for 1024 ms time interval
Average of percentage of the maximum DP for 1024 ms
42 time interval
Average of percentage of the maximum DP for 1024 ms
43 time interval 44 xv
4.9 Location of cursor and target in workspace 47
LIST OF ABBREVIATIONS
- Amyotrophic sclerosis xvi
xvii
APPENDIX
LIST OF APPENDICES
TITLE PAGE of 57
1
CHAPTER 1
INTRODUCTION
1.1
Introduction
For normal people, communication is a need to undergo their daily activities.
Communication is a process to transmit or transfer information, thought or by feeling by or to or between people or groups. It is a connection allowing access between persons by either verbal contact or action.
However, there are people who suffer from “locked-in syndrome” which means that they are completely unable to control any muscle that prevent them from communicating with their caregivers or environment [1]. For such users, a braincomputer interface (BCI) is the only hope for even communicating with loved ones, controlling even simple devices like televisions or lamps or otherwise expressing oneself. BCI is a novel augmentative communication system that translates human intentions into a control signal for an output device such as a computer application
[2] or a mobile robot [3], in which users send information using brain activity alone without conventional peripheral nerves and muscles [3].
2
BCI can be divided into two general categories, i.e., invasive and noninvasive
[3]. Most of noninvasive BCI systems use electroencephalogram (EEG) signals; i.e., the electrical brain activity recorded from electrodes placed on the scalp. The main source of the EEG is the synchronous activity of thousand of cortical neurons.
Measuring the EEG is a simple noninvasive way to monitor electrical brain activity, but it does not provide detailed information on the activity of single neurons (a few
μVolts) and noisy environment (especially if recording outside shield rooms) [3]. In invasive BCI systems, the activity of single neurons (their spiking rate) is recorded from microelectrodes implanted in the brain. Such systems are being studied mainly in nonhuman primates, for example see [4]. These invasive BCIs face substantial technical difficulties and entail significant clinical risks as they require that recording electrodes be implanted in the cortex and function well for long periods, and they risk infection and other damage to the brain [5]. For human, therefore, noninvasive
BCI systems are applied due to clinical risks and ethics [3].
1.2
Problem Statement
Among of the most important evaluation in one or two dimensional cursor movement-BCI is hit rate. It is an onset element to design a practical application of
BCI that depends in speed and accuracy. If the speed and accuracy can be substantially improved, the range of applications and the number of potential users would greatly increase [1].
In one dimensional cursor movement-electrocorticogram (ECoG) study, they achieved success rates of 74-100% [6] whereas in two-dimensional cursor movement-EEG study, hit rate was accomplished until 92% [5].
3
1.3
Objectives
This study was conducted with the objective of:
•
To determine parameters that will drive or control cursor in brain-computer interface (BCI) using noninvasive EEG signal.
•
To determine parameters that can produce faster hit rate of brain-computer interface.
1.4
Scope of Project
Figure 1.1 depicts the block diagram for this study’s scope of work. First,
EEG is recorded from the scalp and digitalized using acquisition system. Then the digitalized signals are subjected to feature extraction procedures, such as spectral analysis or spatial filtering. Afterwards, translation algorithm converts the EEG feature into command of the cursor movement. At the monitor screen, the cursor moves and hit target.
Acquisition
System
Subject’s head
Feature
Extraction
Translation
Algorithm
Cursor
Movement
Figure 1.1
Scope of project
4
1.5
Project Contribution
BCI uses human brain signal to control other devices. This technology has advances significantly in the world but not many researches can be found in
Malaysia. Advanced research has lead to potential use in medical field as well as in computer games industry. This research has few purposes; firstly to develop human capital in BCI for Malaysia. Secondly, is to provide platform for a more advance application such as a two-dimensional cursor movement. A study by other researchers has found that slow frequency gave highest correct response rate when presented with visual presentation of target and non-target. This research also has found slow frequency as optimum parameter to control cursor movement. However, it is noted that the methodology for the experimental set-up was weak, leading to limited application to only specific one-dimensional cursor movement as any imagination may bring about the same results. However, with proper methodology and feature extraction, there is possibility of using slow frequency for BCI application that will increase the versatility of the field.
The contribution of this research would be in the developed in-house software of BCI. The developed software is integrated with an existing hardware. The software main contribution is that it is an open source, has adjustable cursor and target size, and is panel window size-independent software. With the robust software, a more advance application can be developed and together with correct methodology, the objectives are obtainable.
1.6
Thesis Outline
This report is divided into six chapters. This initial chapter presents introduction to the project which includes the project background, problem
5 statement, objectives of the project, the main objectives, scope of the project and the project contribution. This chapter is referred to conclude the findings of the project in conclusion part.
Chapter 2 discussed about reviewed literatures and published works on BCI in view of applied methodologies over the years. This chapter also discussed on several types of electrophysiology sources and experimental designs including general overview of BCI study.
The project methodology is covered in Chapter 3 that include discussion on the processes and steps involve in managing the raw data from EEG, the formula that being used and the information on software and hardware utilize in this study.
Chapter 4 contains the result of feature extraction and feature translation in offline analysis. In feature extraction, EEG signals were analyzed using Fast Fourier
Transform (FFT) in two time interval i.e. 1024 ms and 512 ms. Afterward, two parameters from the transformed data in frequency domain were extracted, i.e., scalp location and frequency band. Applying a value coordinated by those two parameters, translation algorithm was performed. The performance in mean time, hit rate and movement precision were observed and compared with previous studies.
Chapter 5 summarizes the results and findings from the previous chapter. The values of maximum difference in power which are pointed by the two parameters might be use to control and one-dimensional cursor movement.
The overall project is rationalized and concluded in the final chapter, Chapter
6. Some suggestion and recommendation are also discussed.
6
CHAPTER 2
LITERATURE REVIEW
2.1 Introduction
Brain-computer interface (BCI) is an important medical device for patients who have lost control of their motor faculties, and no longer outwardly express their needs and thoughts. A BCI enables these “locked in” patients to control a computer with their brain activity for communication, mobility and other purposes. This chapter introduces brain waves or electroencephalogram (EEG) as basic disciplines in BCI as well as basic design of BCI and also the type of BCI. Discussions about literatures and published works on BCI over the years are also summarized.
2.2 Electroencephalography
The discovery by Hans Berger that the brain develops a low-level subaudiofrequency electrical activity has led to establishment of a neurophysiological specialty known as electroencephalography [7]. Electroencephalography (EEG) is a
7 measuring and recording method of electrical activity recorded from the scalp and the tracing result is known as electroencephalogram. The electroencephalogram or
EEG is widely used by physicians and scientists to study brain function and to diagnose neurological disorders.
An EEG waveform is a wave of electricity generated from the brain nerves and an information signal to indicate the cerebrum activity. Therefore, an EEG waveform offers important data to judge whether the brain activity is normal or not.
However since an EEG waveform is taken through the skull, the absolute voltage is very small and it is difficult to indicate the activities of each brain part in detail owing to this natural insulation and also to the depth of each part.
Usually, EEG signals are divided into a few bands according to their frequencies. They are summarized in Table 2.1 and the waveforms are shown in
Figures 2.1 (a)-(e).
Table 2.1
Frequency band for EEG signals
Frequency band
Delta
Theta
Alpha
Beta
Frequency range (Hz)
0 – 4 Hz
4 – 7 Hz
8 – 13 Hz
13 – 30 Hz
8
Delta and theta waves are known collectively as slow waves. Delta waves normally are seen in deep sleep in adults as well as in infants and children while theta waves normally are seen in sleep at any age. Delta waves have the largest amplitude of all waves. Alpha waves generally are seen in all groups but are most common in adults. They tend to be present posteriorly more than anteriorly and are especially prominent with closed eyes and with relaxation. Alpha activity disappears normally with attention, for example mental arithmetic, stress and opening eyes. Beta waves are observed in all age groups. They tend to be small in amplitude and usually are symmetric and more evident anteriorly. Drugs, such as barbiturates and benzodiazepines [8], can augment beta waves.
(a) Delta wave
(b) Theta wave
(c) Alpha wave
(d) Beta wave
(e) Gamma wave
Figure 2.1 EEG waveforms
9
2.3 The International 10-20 of Electrode Placement System
For the purposes of EEG signals recording, there is a system known as the
International 10-20 of Electrode Placement System [9]. This system is the most widely used method to describe the placement of electrodes at specific intervals along the head. As shown in Figure 2.2, each electrode site has a letter to identify the lobe, along with a number or another letter to identify the hemispheric location. The letters F, T, C, P and O represent Frontal, Temporal, Central, Parietal and Occipital, respectively. Even numbers 2, 4, 6, and 8 indicate the electrodes at right hemisphere while electrodes at the left are indicated by odd numbers 1, 3, 5 and 7. Other than that, small letter ‘z’ refers to an electrode placed in the midline. The ‘10’ and ‘20’ are referred to the 10% and 20% inter-electrode distance [9].
Figure 2.2 The 10-20 of electrode placement system
10
A BCI is a system that allows a user to communicate with the environment only through cerebral activity, without using muscular output channels. To establish a direct link between the brain and a computer, the cerebral activity is measured and then analyzed with the help of signal processing and machine algorithms. Once a certain mental activity has been detected by the computer, a response can be displayed on a screen or a command can be sent to a device – for example a robot or a remote control. The main application area for BCI is assistive technology for ablebodied as well as handicapped subjects. The basic design of BCI is shown in Figure
2.3.
BCI SYSTEM
SIGNAL
ACQUISITION
DIGITIZED
SIGNAL
SIGNAL PROCESSING
Feature
Extraction
Translation
Algorithm
DEVICE
COMMANDS
Figure 2.3: Basic design and operation of any BCI system [1]
In the BCI system, signals from brain are acquired by electrodes on the scalp and processed to extract specific signal features that reflect the user’s intent. Then, these features are translated into commands that will operate a device [1].
11
2.4.1 Invasive and Non-invasive BCI
BCI can be divided into two major methodologies which are invasive or noninvasive method. Invasive BCIs, that use activity recorded by brain implanted micro- or macroelectrodes, have higher spatial resolution, and might provide control signals with many degrees of freedom; whereas non-invasive BCIs, that use brain signals recorded with sensors outside the body boundaries [3], are convenient, safe and inexpensive, but have low spatial resolution. Another method called electrocorticography (ECoG) is an intermediate BCI methodology [10]. This moderately invasive method is using ECoG activity that recorded from the cortical surface. ECoG has higher spatial resolution than EEG, i.e., tenths of millimeter 0-100 versus centimeter. It has broader bandwidth and higher amplitude, i.e., 0-200 Hz and
50-100 μV rather than 0-40 Hz and 10-20 μV in EEG, respectively [6].
Experience of BCI research in human beings has so far primarily involved non-invasive EEG-based investigation, for example [5] and most intracortical
(invasive) BCI involved animal studies [4].
Although there was a belief that only invasive BCI could provide users with real-time multidimensional control, previous study by Wolpaw et al. [5] presented that multidimensional control also could be provided using non-invasive BCI. They demonstrated that multidimensional control of a cursor movement on a computer screen can be learned in a few session of training with non-invasive BCI in humans.
The subjects were able to move a cursor within ten seconds into one of eight goals appearing randomly at one of the four corners of the screen. The flexibility, speed and learning performance in generally equal to that seen when invasive multielectrode BCI systems were tested in animals [5].
At present, it seems probable that different recording methods will be useful for different applications, different users, or both [10].
12
2.4.2 Imagination vs. actual movement
The goal of BCI research is to provide new communication option to people with severe neuromuscular disorders such as Amyotrophic Lateral Sclerosis (ALS), brainstem stroke, cerebral palsy and spinal cord injury [11]. Although people suffered from ALS or other severe motor disabilities, but 67.5% of them [12] appear that the cognitive functions are not impaired.
The imagination of a movement generates the same mental processes and even physical processes as the performance of the movement [13]. Furthermore, brain signals produced by imagination movement are similar as actual movement. It is supported by a previous study on a tetraplegic patient, 31 years old male with a complete traumatic spinal cord injury (SCI) and five healthy volunteers with agematched acted as control group [14]. The patient underwent somatosensory evoked potential (SEP) and motor evoked potential (MEP) recordings to confirm the extent and location of cerebrospinal disconnection. They found that contrasts of imagery of repetitive hand and foot movement versus rest in the patient demonstrated significant activation of sensorimotor networks similar to the patterns defined by contrasts of active movement versus rest in the healthy control group. Another study [6] showed the spatial and spectral foci of task-related electrocorticogram (ECoG) activity, as revealed in the r
2
analysis, were usually similar for action and for imagination of the same action (e.g. saying the word ‘move’ and imagining saying the word ‘move’).
In BCI systems, electrophysiological sources refer to the neurological mechanisms or processes employed by a BCI user to generate control signals. Noninvasive BCI use the electrical activity of the brain recorded with single or multiple
13 electrodes from the scalp surface as input signal for BCI control. Usually participants are presented with stimuli or are required to perform specific mental tasks while the electrical activity of their brain is being recorded. Specific features of the EEG are either regulated by the BCI user such as slow cortical potential or sensorimotor rhythm, or elicited by sensory stimulation such as P300.
A review from Bashahati et al.
[15] reported that current BCIs fall into seven main categories, based on neuromechanics and recording technology they use. Five major groups categorized by Wolpaw [1] are sensorimotor activity, P300, visual evoked potential (VEP), slow cortical potentials (SCP) and activity of neural cell
(ANC). The other two, added by Bashahati et al . [15] are ‘response to mental tasks’ and ‘multiple neuromechanics’. All of them are summarized in Table 2.2 with simple descriptions.
Table 2.2
: Electrophysiological sources used in BCI [15]
Electrophysiological Sources Description
Sensorimotor activities
P300
Visual evoked potential (VEP)
Slow cortical potential (SCP)
Changes in brain rhythms (mu, beta, gamma)
Typically evoked in the EEG over the parietal cortex a positive peak at about 300 ms after the stimulus is received.
Small changes in the ongoing brain signal.
They are generated in response to a visual stimulus.
Slow, non-movement potential changes generated by subjects.
Activity of neural cell (ANC)
“Response to mental tasks”
“Multi-neuromechanism”
When movements are executed in the preferred direction of neurons, the firing rates of neuron in the motor cortex are increased.
Assume that different mental tasks lead to distinct, task-specific distributions of EEG frequency patterns over the scalp.
Use a combination of two or more of the above-mentioned neuromechanisms.
14
Figures 2.3 (a)-(d) show the present-human BCI system types which are sensorimotor rhythm, slow cortical potential, P300 evoked potential and cortical neuronal activity, respectively.
(a) Sensorimotor rhythm (b) Slow cortical potentials
(c) P300 evoked potential (d) Cortical neuronal activity
Figure 2.4
Present-day human BCI system types [1]
Sensorimotor rhythm (SMR) is a majority preferable method of BCI because it is related to the motor cortex with distributions of somatosensory areas. It includes a frequency of 10 Hz (range 8-11 Hz) often mixed with a beta, around 20 Hz and a gamma component, around 40 Hz recorded over somatosensory cortices. It is most preferable over C
3
and C
4.
SMR desynchronizes with movement, movement imagery and movement preparation (event-related desynchronization, ERD), and increases or
15 synchronizes (event-related synchronization, ERS) in the post-movement period or during relaxation [16].
Slow cortical potential (SCP) is among the lowest frequency features of the scalp-recorded EEG generated in cortex. Negative SCPs are typically associated with movement and other functions involving cortical activation, while positive SCPs are usually associated with reduces cortical activation [1]. SCP is used in [17] for thought translation device as one of the BCI application.
The P300 component of event-related potential (ERP) is a positive deflection in the EEG time-locked to stimuli presentation. It is typically seen when participants are required to attend a stream of rare target stimuli [16]. P300 is prominent only in the responses elicited by the desires choice, and BCI uses this effect to determine the user’s intent. A P300-based BCI has an apparent advantage in that it requires no initial user training. It is typically response to a desired choice [1].
BCI technology is used to record and analyse brain signals to determine the output that is desired by the user such as which letter to select for spelling a word, which direction to move a cursor and so on. Therefore, signal processing is one of the important components in BCI research. This signal processing has two stages
[10]. The first stage is feature extraction which is the measurement of the characteristics of the signals that encode the output. These features can be simple measures such as amplitudes of particular evoked potentials or of particular rhythm.
The second stage is the translation of these signal features into device commands using a translation algorithm. Brain signal characteristics such as rhythm amplitudes are translated into command that specify outputs, such as letter selection or cursor
16 movement. Translation algorithm can be simple such as linear equation [11] or complex such as neural network [18] or support vector machines [19].
In this study, signal processing is used to extract features and the method used was frequency analysis. Bashashati et al. [15] reviewed that the magnitude of the Fourier Transform (FT) of the signal may be considered as a representation with perfect spectral resolution but with no temporal information. Frequency-based features have been widely used in signal processing because of their ease of application, computational speed and direct interpretation of the result.
Recent and current studies seek to improve the speed and accuracy of this control by improving the selection of signal features and their translation into device commands, by incorporating additional signal features and by optimizing the adaptive interaction between the user and the system [11].
2.5 Published Work on BCI
A bibliometric study in [20] reviewed the growing interest with associated number of publications over the last two decades. Earliest paper describing a BCI system was published in 1973 by J. J. Vidal [21]. Two decades later, the number of papers grew substantially until over a thousand papers. [20] found that the most productive authors in BCI research are G. Purtscheller from Graz University of
Technology (Austria), N. Birbaumer from University of Tubingen (Germany) and J.
R. Wolpaw from Wadsworth Center (New York, USA).
For instance, Wolpaw and his group have been published papers to move cursor in one dimensional and two dimensional. They focused on mu and beta
17 rhythm [5]. Their published work on two-dimensional (2D) cursor movement motivated current study for developing local brain-computer interface system. Using mu and beta rhythm component, their research tested on four disables subject produced the result that falls within invasive study using monkey. They emphasized that non-invasive of BCI also could provide multidimensional movement control without any implanted in the brain. In view of movement time, precision and accuracy, the results were comparable to these with invasive BCIs.
BCI research areas include evaluation of invasive and non-invasive technologies to measure brain activity, development of new BCI applications, evaluation of control-signal (i.e. patterns of brain activity that can be used for communication), development of algorithm for translation of brain signals into computer commands, and the development and evaluation of BCI system specifically for disabled subjects.
2.5.1 Published Work on Neurofeedback
Neurofeedback is a type of biofeedback that uses real time displays of electroencephalography to show brain activity, often with a goal of controlling central nervous system. It is a form of training or exercise for the brain which is assisted with a very sophisticated technology and guided or directed by knowledge gained through the advances of neuroscience. Neurofeedback in advanced is used to improve attention, anxiety, mood and behaviour.
Regardless of clinical and psychological application, the currently most advanced BCI make use of the fact that humans can acquire control over EEG parameters by means of neurofeedback, i.e., real-time visual or auditory feedback of a specific EEG parameter.
18
For example, previous study by Hwang et al.
[22] has developed a kind of neurofeedback systems to train motor imagery by presenting participants with timevarying activation maps of their brain, using a real-time cortical rhythmic activity monitoring system. By employing ten healthy subjects, five subjects in training group and another five subjects in control group, they conducted experiments consisted of two sessions which were motor imagery training session and EEG recording session. In the first session, participants were trying to increase their mu rhythm activations around the sensorimotor cortex while they were watching their cortical activation map through the real-time. In the second session, two EEG recordings were performed each before and after motor imagery training session to demonstrate the effect of their neurofeedback-based motor imagery training system.
They found that there were meaningful changes of brain activity in training group around sensorimotor rhythm (10 and 20 Hz) compared to control group. With this, they confirmed the effectiveness of their neurofeedback-based motor imagery training in used of BCI application.
2.5.2 Publish Work on Cursor Control
Cursor control movement is one of the application and assessment in BCI.
Basically the study starts with one-dimensional cursor movement, horizontal or vertical and continues with advanced 2D or 3D cursor movement. As mentioned earlier, Wolpaw and colleagues have been published SMR-based BCI in cursor movement.
Other than that, a study by Wilson et al.
[23] published in Journal of
Visualized Experiments (JoVE) measured the amplitude of the EEG signals in selected frequency bins into a device command, i.e., the horizontal and vertical velocity of computer cursor. They were utilizing BCI2000 application as analysis tool. Using the power in mu (8-12 Hz) and beta (18-28 Hz), the user in the study was
19 asked to make several different imagined movements with their hands and feet to determine the unique EEG features that change with the imagined movements. They determined which EEG features are strongly correlated with each movement by finding the large r-squared values in the plots produced from the analysis tool and found on channels 9, 10, 17, 18 and 19 according to 10-20 international system electrode placement. The subject’s goal was to move the cursor to the correct target using imagined movements corresponding to the desired direction of movement (e.g. right hand to move right or feet to move down). They found that a trained subject should be able to move cursor to the target within 1 or 2 seconds.
2.5.3 Published Work on Other Control
The concept of BCI stems from a need for alternative, augmentative communication and control options for individuals with severe disabilities, though its potential uses extend to rehabilitation of neurological disorders, brain-state monitoring as well as gaming [16].
Previous study by Lalor et al . [24] addressed an application of steady-state visual evoked potential (SSVEP) – based BCI design within a real-time gaming framework. The video game involved the movement of an animated character within a virtual environment
Finke et al.
[25] also presented a BCI game named the MindGame, based on the P300 event related potential. In the MindGame interface, P300 events were translated into movements of a character on a three-dimensional game board. While game and multimedia control with a BCI is inherently a challenging field, it potentially also opens the door for interesting applications in a clinical context [25].
20
Another study with different application was published by Bell et al.
[26].
Their study was about to control a humanoid robot using non-invasive BCI. They also used P300 evoked potential as their subject’s command to the robot. The P300 was used to discern which object the robot should pick up and which location the robot should bring the object to. The robot which was including a pan-tilt camera head for visual feedback to the user and two hands for grasping objects, transmitted the image of objects discovered by its cameras back to user for selection. The user was instructed to attend to the image of the object of their choice while the border around each image was flashed in a random order. Machine learning techniques were used to classify the subject’s response to a particular image flash as containing a
P300 or not, thereby the user’s choice of the object could be inferred. A similar procedure was used to select a destination location. From the study, their results showed that a command for the robot can be selected from four possible choices in
5s with 95% accuracy [26].
In other study, a group of researcher from Singapore [27] was developing a new BCI application which could also provide novel ways of entertainment. They developed software and Graphical User Interface (GUI) on music composition using
BCI. They investigated the potential of BCI in music composition encompassing the topic of gaming, entertainment and possibly rehabilitation with the objectives to investigate other communication channel than spelling words and to stimulate brainstorming on future BCI research avenues while reviewing the limitations of the current state-of-the-art [27].
2.6 Discussion on Published Work in BCI
Sensorimotor rhythms (SMRs) are the most often used of EEG component in non-invasive BCI system. Produced in sensorimotor cortex and associated areas, they are mostly directly related to movement. In addition, previous studies also suggested
21 that people could learn to control their SMRs amplitude. However, patients or users need to be trained several time to control their rhythms. For example in [5].
Moreover, not all users are able to obtain control of their brain waves after intensive training.
P300 signal is one of the typical patterns captured from EEG at the central cortex and also mostly choose as a method in BCI system. Despite of its robustness, the such system may require permanent attention to external stimuli which may be fatiguing for some users.
Considered as imagined-movement mental-task BCI system, the method in this study does not require a priori selection of subject-specific frequency and not required in-between interface like the screen of alphanumeric characters used in the
VEP-based BCI. Our method assumed that different mental tasks, i.e., task and control condition, lead to distinct, task-specific distributions of EEG frequency patterns over the scalp.
22
CHAPTER 3
METHODOLOGY
3.1 Introduction
This chapter discusses the methodology used in this study. It concerns two main issues of how the data were collected and generated as well as how they were analyzed towards the purpose of this study which is to examine and determine the parameter to drive cursor movement. This chapter includes experimental set up, the equipment, procedures and the flow chart of the parameter selection.
3.2 Experimental Set Up
The experiments were conducted at Medical Electronic Lab, Faculty of
Electrical Engineering, UTM. EEG signals were recorded using Neurofax EEG-9000
V.05-90 data acquisition from Nihon Kohden, Japan. The experiments roughly took about one and half hour of setting up that includes preparing the subject, recording the data and cleaning the subject and equipments.
23
3.2.1 Equipment and Supplies
There were a few equipments and supplies that need to be used together with
EEG machine to complete the experiment. They were ECI
TM
electro-caps, electrode board adapter, body harness, disposable sponge disks, ECI Electro-Gel
TM
, blunted needle/syringe kit, ear electrodes, color-coded head measuring tape and electrode board adapter connector as shown in Figure 3.1(a)-(g).
(a) Electro-cap (b) Electrode board adapter (c) Body harness
(d) Disposable sponge disks (e) Blunted needle/syringe kit (f) Ear electrodes
(g) Electrode board adapter connector
Figure 3.1 Equipments used in the experiment
24
3.2.2 Preparation of Subject
To operate EEG experiments, the subject must be relaxed with no stress and fatigue to avoid the EEG machine recorded unwanted signals. Therefore, to prepare the subject, steps taken were as follows: a.
Body harness was put on the subject chest to support the cap. b.
Before applying the suitable cap, the circumference of subject’s head was measured. c.
Ear electrodes were attached at the ears and gel was added. d.
Disposable sponge disks were placed around F
P1
and F
P2
channels to absorb perspiration and prevent the spread of electrode gel onto the forehead. e.
Then, the proper size of cap was slipped onto the subject’s head and the cap straps were attached to the body harness. f.
Cap connector was connected to the electrode board adapter. g.
Each electrode cavity was filled with electrode-gel by moving the blunted needle/syringe rapidly back and forth. h.
The electrode impedance was checked. i.
Finally, the subject was moved to the chair for recording.
3.2.2.1 Installation of Electrodes onto Electrode Board Adapter
The Electrode Board Adapter is used to connect the Electro-Cap to EEG instruments. Each wire/connector pin (socket) on the Electrode Board Adapter is color-coded for easy identification. Each wire contains of two tips which are white and red tip. Each of them represents the electrode placement as in Tables 3.1.
25
White Tip
Table 3.1
: Electrodes position
Wire Color Red Tip
F
P1
Brown F
P2
F
3
Red F
4
C
3
Orange C
4
P
3
Yellow P
4
0
1
Green 0
2
F
7
Blue F
8
T
3
Violet T
4
T
5
Gray T
6
GND White C
Z
F
Z
Black P
Z
3.2.3 Clean Up
When EEG recording finished, the cap, ear electrodes and body harness were removed from subject. ECI
TM
Electro-Caps must be cleaned frequently for sanitary reasons. In addition, if all the gel is not washed from a cap, the material will lose its elasticity and shortened the life of the cap.
26
3.3 Data Acquisition and Recording Process
3.3.1 Participant
Six normal subjects aged from 20 to 26 years old, gave informed consent and participated in the study.
3.3.2 Conditions
Subjects sat on a chair facing a monitor screen, placed one meter in front of the subjects. In Control condition, subjects were asked to relax (resting). In Task condition, subjects were asked to imagine voluntary movement, e.g. imagine moving a cursor on a monitor screen.
EEG signals were obtained from 19 scalp electrodes, placed on the scalp based on 10-20 electrode placement system as discussed in section 2.3
with reference electrodes attached to left and right ears. Signals were recorded with pass bands of
0.5 – 120 Hz and stored in a personal computer with a sampling frequency of 1 kHz.
A single trial lasted for 10 second and four trials were performed for each condition with intervening rest period to avoid fatigue as shown in Figure 3.2.
27
CC TC CC TC CC TC CC TC
10s 10s 10s 10s 10s 10s
Trial 1 Trial 2 Trial 3
Notes:
CC : Control Condition
TC : Task Condition
Figure 3.2
Sequence of data acquisition
10s 10s
Trial 4
A 10 seconds data of Control (CC) and Task (TC) conditions (Figure 3.2) were divided into ten frames so that each frame consists of one second data. To obtain optimum speed and accuracy, the EEG data were divided at various intervals, i.e., 1024 ms and 512 ms as illustrated in Figure 3.3.
10 second of data
1 st
frame 2 nd
frame
…………. 10 th
frame
1024 data
1024 data
1024 ms Time Interval
1 st
frame 2 nd
frame …………. 10 th
frame
512 data
512 ms Time Interval
512 data
Figure 3.3 Division of data for 1024 ms and 512 ms time interval
28
Using Fast Fourier Transform (FFT), for each interval, data in frequency domain of each frame were obtained. For Control condition, five frames in the middle were averaged and assigned as baseline. Frequency was divided into six groups, i.e., delta band (0-4 Hz), theta band (4-7 Hz), alpha band (8-13 Hz), beta band (13 – 30 Hz), gamma band (31-50 Hz) and high gamma band (>51 Hz). Each frequency band in all frequencies of the Task condition was compared to the Control condition. Differences between these two condition were obtained and observed using Equation 1. difference in power (DP) (f) = power in Task (f) - power in Control (f) (1)
29
3.4 Flow Chart of Data Analysis
Figure 3.4 shows the flow to analyze EEG signals using C programming in
Linux. The process was repeated four times to analyze four trials of one subject.
Control condition
(10 s)
Calculate power spectrum using FFT
Average Control power spectrum
START
Record EEG signals
Compute difference power between
Task and Control power spectrum
Task condition
(10 s)
Calculate power spectrum using FFT
Find maximum difference power occurrence at which scalp location and frequency band
END
Figure 3.4
Flow chart of parameter determination process
30
After that, all maximum DP values were gathered into one folder to choose a single scalp location and frequency band. The process was summarized as in Figure
3.5. The maximum DP values from identified parameters were used to move cursor in further step.
Get maximum
DP from
Trial 1
Get
Trial 2
START maximum
DP from identical?
NO
DP for all trials
Determine parameters
END
Get maximum
DP from
Trial 3
Scalp location and frequency bands were
Average maximum
Get maximum
DP from
Trial 4
YES
Figure 3.5
Flow chart of finalized parameter determination process
31
The software development has been carried out using the X Window System
(commonly referred to as X or X11) on LINUX platform. This system is a networktransparent graphically windowing system based on a client/server model. Full coding commands can be referred in Appendix A ( Movecursor.c
) and B ( Features.c
) .
Both of them were developed using C language in LINUX. However in this section, the explanation focuses on the main graphics that associated to the purpose of the study.
Figures 3.6(a), 3.7(a), 3.8(a) and 3.9(a) show some writing codes in X. Figure
3.6(a) shows the command to develop a workspace. The size of workspace was optimized according to the dimension of screen computer used, as shown in Figure
3.6(b).
Figure 3.6(a) Syntax of workspace
Figure 3.6(b) Workspace
32
In addition, the baseline inside the workspace as shown in Figure 3.7(b) was drawn using the coding command as in Figure 3.7(a). The coding drew one horizontal line and one vertical line which met at the centre.
!" #"
$%
%
Figure 3.7(a) Syntax of baseline
Figure 3.7(b) Baseline
Furthermore, Figure 3.8(a) shows the syntax to build a block of rectangular cursor in blue colour. This variable cursor size was initiated at the centre of the screen. For example, the cursor size in this study was 10 x 10ixels as shown in
Figure 3.8(b).
33
!
!" #"
!" #" !" #"
!" #"
Figure 3.8(a) Syntax of cursor
Figure 3.8(b) Cursor
Figure 3.9(a) shows the language structure to display target. The target was a graphic located at the right side of the workspace as shown in Figure 3.9(b). It was inconstant value and designed in black colour. The size of the target was 30 x 30 pixels used in this study.
34
"
" "
"
Figure 3.9(a) Syntax of target
Figure 3.9(b) Target
In the situation of cursor movement, it should jump from the initial state according to a value indicated by extracted parameters. If the cursor succeed hit the target within ten seconds, a reward in form of string text would appear.
35
CHAPTER 4
FREQUENCY ANALYSIS AND FINDINGS
This chapter presents the result and analysis of the project. The results are arranged into two parts which are feature extraction results and offline feature translation results.
Figure 4.1 shows an example of raw EEG signals of one channel in one second, taken from subject 1 at trial one located at P
Z
channel. Meanwhile the example of power spectrum of EEG signal is shown in Figure 4.2. From 19 channels of EEG signals recorded, however, there were only nine channels analyzed i.e. F
3
,
F
Z
, F
4
, C
3
, C
Z
, C
4
, P
3
, P
4
, P
Z
and P
3
(Figure 4.3) that represents the location commonly used in BCI research.
36
Notes:
Task
Control
Figure 4.1
Example of 1 second of raw EEG signal from subject 1 at P
Z channel
Notes:
Task
Control
Figure 4.2
Example of power spectrum of EEG signal from subject 1 at P
Z channel
37
F3
C3
P3
FZ
CZ
PZ
F4
C4
P4
1s
Notes:
Task
Control
Figure 4.3 Example of 1 second data of EEG signal from subject 1
4.3 Feature Extraction Analysis
To extract mean values from EEG signals, two time intervals were applied, i.e., 1024 ms time interval and 512 ms time interval (Figure 3.3). The findings from both time intervals were compared.
38
4.3.1 1024 ms Time Interval
Figures 4.4(a)-(d) show the results of percentage of maximum DP using 1024 ms time interval for all trial (6 subjects x 10 frames). For trial 1 (Figure 4.4 (a)), maximum DP was found at P
4
site with 56.7% occurrence. For trial 2, (Figure 4.4
(b)), maximum DP was observed at another site i.e. P
3
with 51.7% occurrence. For trials 3 and 4 (Figures 4.4 (c) and (d)), both of maximum DP was found at P
Z
site with 61.7% and 60.0% occurrence, respectively. Generally, the maximum DP occurred at delta band, which ranged from 0 to 4 Hz, for all trials.
(Delta, P
4
, 56.7%)
(a) Trial 1
(Delta, P
3
, 51.7%)
(b) Trial 2
(Delta, P
Z
, 61.7%)
(c) Trial 3
39
40
(Delta, P
Z
, 61.7%)
(d) Trial 4
Figure 4.4
The occurrences of maximum DP for 1024 ms time interval
4.3.2 512 ms Time Interval
The maximum DP result for all trials (6 subjects x 10 frames) using 512 ms time interval is shown in Figures 4.5 (a)-(d). In section 4.3.1, the frequency range was constant for all trials i.e. in delta band. However, there were two different frequency band observed in time interval 512 ms. For trials 1 and 2 (Figures 4.5 (a) and (b)), both maximum DP occurred at F
3
site with 38.3% and 40% occurrence respectively, but in different frequency band. For trial 1, maximum DP occurred within delta band whereas for trial 2, within beta band. For trials 3 and 4 (Figures 4.5
(c) and (d)), the frequency range was seen constant but not the scalp location. The maximum DP occurred in delta band for both trials 3 and 4. For trial 3, the maximum
DP was found at P
3
site with 40% meanwhile there were three different sites
41 assembled the maximum DP for trial 4 with similar percentage 38.3% i.e. F
3
, P
3
and
P
4
.
(Delta, F
3
, 38.3%)
(a) Trial 1
(Beta, F
3
, 40%)
(b) Trial 2
(Delta, P
3
, 40%)
(c) Trial 3
(Delta, P
4,
P
3,
F
3
, 38.3%)
(d) Trial 4
Figure 4.5 The occurrences of maximum DP for 512 ms time interval
42
43
4.3.3 Mean for All Trials
Initially, it is expected that every trial in both time intervals will produce similar result to determine the parameters. Unfortunately, for 1024 ms time interval, although the frequency band remains, but the maximum DP occurred at various scalp location for all trials. Meanwhile for 512 ms time interval, the extracted parameter occurred at different frequency band and at various scalp location, too. Therefore, to overcome these distinctions, averaging process was done for all trials and the results are shown in Figures 4.6 and 4.7 for 1024 ms time interval and 512 ms time interval, respectively.
Figure 4.6
Average of percentage of the maximum DP for 1024 ms time interval
Figure 4.6 shows the scalp location with highest percentage of averaged maximum DP for 1024 ms time interval. Delta band was determined first because of its consistency for all trials. Therefore, this averaging process only involved the
44 maximum DP values in various scalp location within delta frequency range. It shows that the maximum DP was highly occurred at P
Z
with 56.6% followed by P
3
(54.33%) and P
4
(53.3%).
(Delta, P
3
, 35.4%)
Figure 4.7 Average of percentage of the maximum DP for 512 ms time interval
For 512 ms time interval, the occurrences of maximum DP were vary in both parameters i.e. scalp location and frequency range, for all trials. Thus, averaging process was for the maximum DP values from various scalp location and frequencies range. Figure 4.7 shows that the highest percentage of averaged maximum DP occurred at P
3
site within delta frequency range with 35.4%. The other two locations were P
4
and F
3
, with 34.6% and 33.3% respectively, in the similar ranged of frequency.
From the result, delta band and parietal region were selected as the parameters to drive the cursor movement. The DP values from P
Z
site in 1024 ms time interval were used instead of P
3
site from 512 ms interval due to its highest percentage of occurrence.
45
By applying linear equation in Equation 2, cursor movement was controlled using the maximum DP values from P
Z
site. For each trial, there were ten values of maximum DP originated from ten frames of EEG data in frequency domain. These values will drive cursor to hit the target. The formula use to translate brain signal at
P
Z
site to pixel value is
C
P n
= C
P (n-1)
+ k DP n
(2) where C
P n
is current cursor position, C
P (n-1) is previous cursor position, k is gain (in this study k = 64 is selected) and DP n value is determined from feature extraction.
The equation is expanded as in Appendix A as shown in Figure 4.8.
&'
" () (**+
&' &'*&'
Figure 4.8
Translation equations in C structure
The performance of cursor movement was also tested using the combination of two or three parietal scalp locations in addition of P
Z
site, i.e. P
Z
and P
3
; P
Z
and P
4
; and P
Z
, P
3
and P
4
. The combinations of the channels were summarized in Table 4.1.
Table 4.1
: Channels combination
Combination Channel(s)
- P
Z
P
Z and P
3
A
CM
B
CM
C
CM
P
Z and P4
P
Z
, P
3
and P
4
46
One dimensional cursor movement using DP values from P
Z
produced 4.41s mean time with hit rate of 95.83%. The combination of two scalp location; P
Z
and P
3 and P
Z
and P
4 yielded mean time of 3.25s and 3.58s with 95.83% and 100% hit rate, respectively. Meanwhile, the summing of DP values from all those parietal sites gave
2.91s mean time with 100% hit rate (Table 4.2).
Table 4.2
Hit rate and mean time
(%) (s)
-
A
CM
B
CM
C
CM
P
Z
P
Z and P3
P
Z and P4
P
Z
, P3 and P4
95.83
100
100
3.25
3.58
2.91
Notes: n = (6 subject x 4 trials)
= 24
Table 4.3
Comparison of hit rates and movement precision
Current study
Wolpaw [5]
Leuthardt [12]
Hit rate (%)
95.83 – 100 %
92 %
74 – 100%
Movement precision
0.114
4.9
-
Movement precision was calculated as target size as percentage of workspace, with the smaller size the better. Table 4.3 shows the movement precision result for this study and others as well. From the table, it is obvious that current study produced better precision compared to the other. In this research, the size of target and cursor were chosen as shown in Table 4.4. The location of the cursor and target are shown as in Figure 4.9.
47
Table 4.4: Size of target, cursor and workspace
Item Size
Target 30 x 30
Cursor
Workspace
10 x 10
1024 x 768
Figure 4.9
Location of cursor and target in workspace
48
CHAPTER 5
DISCUSSION
This chapter discusses further on the analysis from feature extraction and translation. The maximum DP is found in two parameters which are in delta frequency and parietal region.
In this study, the maximum difference in power between resting and active imagination was observed at which scalp location and frequency band it occurred.
These two features (scalp location and frequency band) are similarly identified by
Leuthardt et al. [6] in their BCI study using electrocorticography (ECoG). However, they focused on r
2
value instead of maximum difference in power.
49
From the findings, the maximum different in power (DP) between the two conditions occurs in delta band at posterior area, i.e. P
Z
for 1024 ms time interval and
P
3
for 512 ms time interval. Comparing the two time intervals, delta band at central posterior area (P
Z
) provides higher percentage of difference in power, hence can be use to control one dimensional cursor movement [10]. By selecting the power difference of delta frequency band (< 4 Hz) in central posterior area, it is expected that the cursor can move further and faster in one dimensional direction towards targeted location. It is also expected that no training is required to obtain optimum results. Many researchers have shown that that BCI training must be conducted many times to achieve best performance for each subject. For example, Wolpaw et al.
[5] had conducted over twenty sessions per subject, at a rate of two to four per week. In this study, the result was based on a single session of an experiment; hence it is believed that the extracted feature can be used to control a one dimensional cursor movement without prior training. However further study is needed to delineate this speculation.
Although it is always been reported that BCI researches used mu and beta rhythm which is associated with actual movement or imagination of movement, for this research, however, the values in the delta band are used instead to convert EEG signals to cursor movement. Wolpaw et al.
[1] reported that movement or preparation for movement is typically accompanied by a decrease in mu (8-12 Hz) and beta (13-30Hz) rhythm amplitudes, therefore, minimizing their different amplitude values in power. In this study, we found maximum difference in power occur in delta frequency band even though the slow rhythm is always being associated with sleep wave in adult. A few other studies have shown that slow frequency rhythm can be use as indicator for communication aid device [28, 29].
Neshige et al.
[28] presented a study designed to develop a communication aid device utilizing event-related potentials (ERPs) for patient with severe motor disabilities. The eight subjects were given illustrated stimuli consisted of four pictorial symbols (eat, toilet, drink and cross). By analyzing P300 at P
Z
, they found correct response rate after averaging 5 or 10 epochs of ERPs was in frequency range of 2-5 Hz. Another study by Harmony et al.
[29] investigated EEG delta activity by performing two experiments. One of them was performing a difficult mental
50 calculation task and control stimuli. They found the increase of power from 1.56 to
5.46 Hz was observed only during the performance of the task and not during the control condition. By any chance, the different power between task and control condition increased in that frequency.
Fernandez et al.
[31] discussed that one possible explanation for the increase of delta power during the performance of mental tasks is the activation of the corticofugal pathway that inhibits the thalamocortical cells, producing a disconnection of the cortex from environmental stimuli, a state which we call internal concentration.
However, more research is needed to elucidate the intrinsic mechanisms for the generation of these slow waves during mental activity. The two studies from Neshige et al.
and Harmony et al.
have shown that information could also be found in the low frequencies of EEG signal. This study also has found slow frequency as optimum parameter to control cursor movement. However, it is noted that the methodology for the experimental set-up in this research was weak, leading to limited application to only specific one-dimensional cursor movement as any imagination may bring about the same results. However, with proper methodology and feature extraction, there is possibility of using slow frequency for BCI application that will increase the versatility of the field.
As for the optimum scalp location, the maximum power difference was found at posterior area that contains primary and associations cortices for somatosensation
[31]. The region can be divided into two functional regions, one that involves sensation and perception and the other that concern with integrating sensory input, primarily with the visual system [32]. Since the subject was provided with the visual of targeted location and the cursor location, the posterior area probably constructs a spatial coordinate system to represent the two locations. Further study is needed in this aspect.
51
CHAPTER 6
CONCLUSION
6.1 Conclusion
BCI allows users to directly communicate their intention without any involvement of the motor periphery. The development of non-visual BCI renders this technology feasible for patients who lost control of eye movement due to disease progression or injury.
In this study, we have presented an imagined-movement mental task to drive cursor in BCI application. We have extracted different power between task and control condition in delta frequency at center posterior region. By using the different power value, cursor movement was performed. Rate of cursor hits target is ranged from 95.83-100%. However, it is found that the delta frequency band is only applicable to a specific one-dimensional cursor movement as any imagination may produce the same results. Nevertheless, this study provides a platform for a more advance two-dimensional cursor movement study.
52
6.2 Recommendation for Future work
In this experiment, the participants were healthy subject. The ultimate goal of
BCI is to provide an option for paralyzed and motor-disabled persons. To observe the reliability of BCI, the author suggests to involved paralyzed or motor-disabled persons as subjects.
Besides, BCI researchers also may used dry electrode instead of gelelectrode. This new device has the advantages of no need for skin-preparation or application. Hence, the experiments become less time consuming and decreased the uncomfortable of the subjects [33].
Despite recent advancement, BCI is a field still in its infancy and several issues need to be addressed to improve the speed and performance of BCI. It is suggested to explore more on local components of brain activity with fast dynamics that subject can consciously control [3]. For this, the increasing knowledge of the brain (where and how cognitive and motor decisions are made) is needed as well as the application of more powerful digital signal processing methods that those commonly used to date.
53
REFERENCES
1.
Wolpaw, J.R., Birbaumer, N., McFarland, D.J., G. Pfurtscheller, Vaughan.
T.M. (2002). Brain-Computer Interfaces for Communication and Control.
Clinical Neurophysiology , 113: 767-791
2.
McFarland, D.J. and Wolpaw, J.R. (2005). Sensorimotor Rhythm-Based
Brain-Computer Interface (BCI): Feature Selection by Regression Improves
Performance. IEEE Transaction on neural Systems and Rehab., 13(3): 372-
379
3.
Millan, J.R., Renkens, F., Mourino J. and Gerstner, W. (2004). Brain-
Actuated Interaction. Artificial Intelligence. 159: 241-259
4.
Carmena, J.M., Lebedev, M.A., Crist, R.E., O’Doherty, J.E., Santucci, D.M.,
Dimitrov, D.F., Patil, P.G., Henriquez, C. S. and Nicolelis, M. A. L. (2003).
Learning to Control a Brain-Machine Interface for Reaching and Grasping by
Primates. PLOS Biology . 1(2): 193-208
5.
Wolpaw, J.R., McFarland, D.J., Vaughan, T.M. and Schalk, G. (2004).
Control of A Two Dimensional Movement Signal by A Noninvasive Brain-
Computer Interface in Humans. PNAS. 101(51):17849-17854
6.
Leuthardt, E.C., Schalk, G. and Wolpaw, J.R. (2004). A Brain-Computer
Interface Using Electrocorticographic Signals in Human.
J. Neural
Engineering.
1: 63-71
7.
Shipton, H. W. (1975). EEG Analysis: A History and A Prospectus.
AnnualReview of Biophysics and Bioengineering. 4, 1–13
54
8.
Sucholeiki, R. (2008). Normal EEG Waveforms. E-Medicine . 1-7
9.
Biomedical Signals Amplifier , ElettronicaVeneta, pp. 27 (2006)
10.
Daly, J. J. and Wolpaw, J. R. (2008). Brain-Computer Interface in
Neurological Rehabilitation. Journal of Lancet Neurology.
7, 1032-1043a
11.
Wolpaw, J.R., McFarland, D.J. and Vaughan, T.M. (2003). Brain-computer interface research at the Wadsworth Center. IEEE Transaction on Neural
Systems and Rehab. 8(2), 222-226
12.
Cognitive Impairment Appears To Be Common in ALS Patients. Medical
News Today
13.
Ianez, E., Azorin, J. M., Ubeda, A., Ferrandez, J. M. and Fernandez, E.
(2010). Mental Tasks-Based Brain-Robot Interface. Robotics and
Autonomous Systems. 58, 1238-1245
14.
Enzinger, C., Ropele, S., Fazekas, F., Loitfelder, M., Gorani, F., Seifret, T.,
Reiter, G., Neuper, C., Pfurtfscheller, G. and Müller-Putz, G. (2008). Brain
Motor System Function in A Patient with Complete Spinal Cord Injury
Following Extensive Brain-Computer Interface Training. Exp Brain Res.
190,
215-223
15.
Bashashati, A., Fatourechi, M., Ward, R. K. and Birch, G. E. (2007). A
Survey of Signal Processing Algorithms in Brain-Computer Interfaces Based on Electrical Brain Signals. J. Neural Engineering.
4, 32-57
16.
Laureys, S., Tomoni, G., and Kubler, A. (2009). The Neurology of
Consciousness: Brain-Computer Interfaces for Communication in Paralysed
Patients and Implications for Disorders of Conciousness.
Elsevier. Ltd.
17.
Kubler, A., Kotchoubey, B., Hinterberger, T., Ghanayim, Nimr., Perelmouter,
J., Schauer, M., Fritsch, C., Taub, E. and Birbaumer, N. (1999). The Thought
55
Translation Device: A Neurophysiological Approach to Communication in
Total Motor Paralysis. Exp Brain Res.
124, 223–232
18.
Nakayama, K., Horita, H. and Hirano, Akihiro. (2008). Neural Network
Based BCI by Using Orthogonal Components of Multi-channel Brain Waves and Generalization. LNCS.
5164, 879-888
19.
Qin, J., Li, Y. and Sun, W. (2007). A Semisupervised Support Vector
Machines Algorithm for BCI Systems. Computational Intelligence and
Neuroscience.
2007, 1-9
20.
Hamadicharef, B. (2010). Brain-Computer Interface (BCI) Literature – A
Bibliometric Study. 10 th
International Conference on Information Science,
Signal Processing and their Applications.
May 10 – 13, 2010. Kuala Lumpur,
Malaysia: ISSPA. 2010. 626-629.
21.
Vidal, J. J. (1973). Toward Direct Brain–Computer Communication.
AnnualReview of Biophysics and Bioengineering.
2, 157–180
22.
Hwang, H. J., Kwon, K. and Im, C. H. (2009). Neurofeedback-Based Motor
Imagery Training for Brain-Computer Interface (BCI). Journal of
Neuroscience Methods.
179, 150-156
23.
Wilson, J. A., Schalk, G., Walton, L. M. and Williams, J. C. (2009). Using an
EEG-Based Brain-Computer Interface for Virtual Cursor Movement with
BCI2000. Journal of Visualized Experiments.
29, 1-4
24.
Lalor, E. C., Kelly, S. P., Finucane, C., Burke, R., Smith, R. Beilly, R. B. and
McDarby, G. (2005). Steady-state VEP-based Brain-Computer Interface
Control in An Immersive 3D Gaming Environment. EURASIP Journal on
Applied Signal Processing . 19, 3156-3164
25.
Finke, A., Lenhardt, A. and Ritter, H. (2009). The MindGame: A P300-based
Brain-Computer Interface Game. Neural Networks.
22, 1329-1333
56
26.
Bell, C. J., Shenoy, P. and Chalodhorn R. (2008). Control of a Humanoid
Robot by A Noninvasive Brain-Computer Interface in Humans. J. Neural
Engineering. 5, 214-220
27.
Hamadicharef, B., Xu, M. and Aditya, S. (2010). Brain-Computer Interface
(BCI) Based Musical Composition. Proceedings of the 2010 International
Conference on CYBERWORLDS (CW2010). 20 – 22 October. Singapore,
282-286
28.
Neshige, R., Murayama N., Igasaki, T, Tanoue, K., Kurokawa, H. and
Asayama, S. (2007). Communication Aid Device Utilizing Event-Related
Potentials for Patients with Severe Motor Impairment. Brain Research . 1141,
218-227
29.
Harmony, T., Fernandez, T., Silva, J., Bernal, J., Diaz-Comas, L., Reyes, A.,
Marosi, E., Rodriguez, M. and Rodriguez, M. (1996). EEG Delta Activity:
An Indicator of Attention to Internal Processing During Performance of
Mental Tasks. International Journal of Psychophysiology . 24, 161-171
30.
Fernandez, T., Harmony, T., Rodriguez, M., Bernal, J., Silva, J., Reyes, A. and Marosi, E. (1995). EEG Activation Patterns During the Performance of
Tasks Involving Different Components of Mental Calculation.
Electroencephalography and Clinical Neurophysiology . 94, 175-182
31.
Martin, G. N. (1998). Human Neuropsychology.
Prentice Hall, pp. 90
32.
Kandel, J., Schwartz, J. and Jessel, T. (1991). Principles of Neural Science.
Elsevier
33.
Gargiulo, G., Calvo, R. A., Bifulco, P., Cesarelli, M., Jin, C., Mohamed, A. and Schalk, A. V. (2010). A New Recording System for Passive Dry
Elecrodes. Clinical Neurophysiology.
121:686-693
APPENDIX A
CODING OF CURSOR MOVEMENT
!""# $$
% #!" !"
& '(
& )*
& ))
& # )+
, !"-.
,/0!"'
/!".1
2!" )3
&!".*
4%!"+-
51!"6 #77 8
5!"6 " 48
#
57
58
#9 /:
; %#<
;#<
;%#<
;#<
68
68
; #<
; #<
%
%
6
"8
=
> ?
>% ># ?+@
@
" # ##
6"8@
?
6" 6"
>8"###>% >#
6" >8
! 6" >88@
!#" #
" 6"8@
# A#
#!
6"8@
@
B
$
% 6
=
" >8
59
$ @ #" C0
% !?)@##$$
$& @
%
#!# #C0
#C0
>#?+@# #C0
>"?D/@"
>"?0E@ "#$F
F>"?GE@F
>?
6"8@
?
$ 6"!H8@
6;)8=
6 I $ :I8@
B
%! #C0
6 >8=
#% 6" ! 6"
>88@
B
% 6" 6"
>88@
=
#% 6" 6"
>88@
B
% 6" ! 6"
>88@
60
#"#% #C0
'() 6">#>"
>"F>"8@
#" #C0%$$
# 6"5/8@
@
B
6 !
JK8
=
>?)@
>? )@
9> @
9> @
!
J1)KJ1)K@
#*'+ 5LM@
!
J+11K@
) >@
>?6
) 8
6>??
-.'' 8=
6>N , 6 )
6I0 " >I8@
6 8@
88@
B
?6
) 8
6??
-.' D8=
6>N , 6 ) 88@
61
B
6I0 " I8@
6 8@
"F@
+/
"@
>@
,O,M4@
2"
% #
@ #"
% ">#">##@ ##
##"
% ###@### #
!
">?6I2L/PD&QI8@ #
"
$ @ C06 #8
> @
% %" #%!@
#
@ %
6 % R?*8=
6I;<;9> <
;9> <I8@
62
6 8@B
6ISI J K8@
9> ?
6 %/ J+K8@
9> ?6
%/ J-K8@
##
"?
0 6">8@
6"??
-.'' 8=
6 IS $S$:I
%/ J)K">8@
6 8@
B
>?
6"8@
">#?
!
6" >8@
">##?
1%!
6" >8@
#?6">#8@
##?6">##8@
6I#$S$@##$S$:I###8@
?
6"###))8@
* 6"T"P !8@
C06 #8 #
?
% 6")8@
6"58@
# $
> ?
6"88@
6"
63
# #
?
(- 6"
H 8@
I IH
6 ??)8=
6 I (-
$ $ :I8@
6 8@
B
?
(- 6" > I% I
H% H% 8@
6 ??)8=
6 I (-
$% $ :I8@
6 8@
B
?
(- 6" > I%IH%
H%8@
6 ??)8=
6 I (-
$%$ :I8@
6 8@
B
?
(- 6" > I"I
H"H"8@
64
6 ??)8=
6 I (-
$"$ :I8@
6 8@
B
?
(- 6" > I I
H H 8@
6 ??)8=
6 I (-
$ $ :I8@
6 8@
B
?
(- 6" > I#I
H#H#8@
6 ??)8=
6 I (-
$#$ :I8@
6 8@
B
?
(- 6" > I%!I
H%!H%!8@
6 ??)8=
6 I&M0
$%!$ :I8@
6 8@
B
6F?)@F;-@FNN8=
#% 6"%!8@
65
' 6" )##+
#69> +8##+8@
' 6"#+6##+8-
#+##-8@
#% 6"%!8@
#% 6"# )9>
6##9> 8+9>
9> 8@
#!
6"8@
#% 6"%8@
#% 6"6#9> 8+
6##9> 8+9>
9> 8@
#!
6"8@
6,O,M4!"!"??-.8=
6)8@
B
6 8@
6F??+8=
6I/4&U4I8@
#% 6"%8@
% 6"#+##'
688@BB
#% 6"#8@
#% 6"6#9> 8+
6##9> 8+9>
9> 8@
6"3)1*-1-)-)8@
#!
6"8@
=
66
6ISI8@
665LM?
6I I88??
68@
-.'' 8=
6 8@
B
6?)@;>@NN8=
65LMISIH>JK8@
B
65LM8@
B
J K?#+@
6? >@;>@NN8=
JK?>JKNJ K@
#% 6"#8@
#% 6"))##-
##-8@
6I!S?S IN JK8@
#% 6"% 8@
% 6"##'##'+68
6??)8
"?#N )@
688@
"?J K@
#% 6"#8@
#% 6""
67
6##9> 8+9> 9> 8@
#% 6"%!8@
#% 6"# )9>
6##9> 8+9> 9> 8@
#% 6"%!8@
' 6" )##+# )
##+8@
' 6"#+##+-
#+##-8@
#% 6"%8@
#% 6"JK6##
9> 8+9> 9> 8@
#!
6"8@
6 8@
=
6JK<?6#9> 8HH;>8=
6I/V00,,2RRI8@
#% 6"%!8@
% 6"6##+-8N ))
6##+-8N )) 688@
#% 6"#8@
#% 6""6##9> 8+
9> 9> 8@
#% 6"%!8@
#% 6"# )9>
6##9> 8+9>
9> 8@
#% 6"%!8@
' 6" )##+# )
##+8@
68
' 6"#+6##+8-
#+##-8@
#% 6"%8@
#% 6"JK
6##9> 8+9> 9> 8@
#!
6"8@
6 8@
B
6 8@
6<?> 8=
6I/7UUQ6I88@
#% 6"%!8@
% 6"
6# 688+6#+8N9>
6I
688@
?SI6# 688+8@
#% 6"#8@
#% 6""6##
9> 8+9> 9> 8@
#% 6"%!8@
#% 6"# )9>
6##9> 8+9>
9> 8@
#% 6"%!8@
' 6" )##+# )
##+8@
' 6"#+6##+8-
#+##-8@
#% 6"%8@
#% 6"JK6##
9> 8+9> 9> 8@
#!
6"8@
B
B
6*8@
B
6"8@
B
69
APPENDIX B
CODING TO DETERMINE PARAMETERS
5
+ A"
%
#"9#.??<# *
;#<
;%#<
;##<
#*'+ 57V4>P @
!
> J1)K@
F>>> >>+&4@
!%>%F@
) &@
) +>+>%@
>#?.>#? *@
#@
LM4,UO&D? )+*@
&4UL? '@
,,C>/4&U4? @
,,C>,M2? '@
/&PDLMC>5U,? )))@
) A%@
70
71
6 !
JK8
=
6 % R?+8=
6I ; <:I8@
6 8@
B
?
6 J K8@
%?6
) 8
6%??
-.'' 8=
6LM4,UO&D ,
6I%$I8@
6 8@
B
6 ) 88@
6&4?)@&4;LM4,UO&D@&4NN8=
6%N&48?6
) 8 6&4ULN , 6 ) 88@
66%N&48??
-.'' 8=
6I%$I8@
6 8@
B
B
A?6
) 8
6 A??
-.'' 8=
6LM4,UO&D ,
6I A$I8@
6 8@
6 ) 88@
B
6&4?)@&4;LM4,UO&D@&4NN8=
6 AN&48?6
) 8 6&4ULN , 6 ) 88@
66 AN&48??
-.'' 8=
6I A$I8@
6 8@
B
B
72
6> I ASI 8@
6657V4>P ?
6> I I88??
-.'' 8=
6IL$S "RR:I> 8@
6 8@
B
!
6 657V4>P IS:S:S:S:S:S:S:S:
S:S:S:S:S:S:S:S:S:S:S:S
:IH%J>KJ)KH AJ>KJ KH AJ>KJ+KH AJ>
KJ-KH AJ>KJ*KH AJ>KJ1KH AJ>KJ.KH AJ>
KJ3KH AJ>KJ(KH AJ>KJ'KH AJ>KJ )KH AJ
>KJ KH AJ>KJ +KH AJ>KJ -KH AJ>KJ *KH
AJ>KJ 1KH AJ>KJ .KH AJ>KJ 3KH AJ>KJ (K
H AJ>KJ 'K8R?
+0# 8=
>NN@
>> ?>@
B
657V4>P 8@
6IS:I>> 8@
6IS:I> 8@
6IS:I AJ KJ K8@
73
#
?&?)@
6F? >#@F;>#N @FNN8=
6?)@;>> @NN8=
+>? AJKJFK@
+>%?%JKJ)K@
6;+>8=
?+>@
#?F@
&?+>%@
B
B
B
6I S?S0#S%?S:I
#&8@
6I#.?5-:#3?0-:#(?P-:#'?59:# )?09:# ?P9:# +
?5*:# -?0*:# *?P*:I8@
6I%??< +#-#*%
1.##:I8@
B
74
APPENDIX C
LIST OF PUBLICATIONS
INTERNATIONAL JOURNAL
1.
Siti Zuraimi Salleh, Norlaili Mat Safri and Siti Hajar Aminah Ali, “Moving
One Dimensional Cursor Using Extracted Parameter from Brain Signals” .
Signal Processing: An International Journal (SPIJ), Vol. 3, No. 5, pp. 110-
119. September/October 2009
INTERNATIONAL CONFERENCE
1.
Siti Zuraimi Salleh, Norlaili Mat Safri and Siti Hajar Aminah Ali, “Feature
Extraction and Translation of Electroencephalogram Signals in Humans for
Noninvasive Brain-Computer Interface” . International Conferece on Control,
Instrumentation and Mechatronic Engineering (CIM2009), 2-3 June 2009,
Melaka, Malaysia. June 2009
2.
Norlaili Mat Safri, Siti Hajar Aminah Ali, Siti Zuraimi Salleh, Nobuki
Murayama, “Modeling Information Pathway of Motor Control Using
Coherence Analysis” . Second Asia International Conference on Modeling
Symposium (AMS2008), pp. 917-922, 13-15 May 2008, Kuala Lumpur,
Malaysia. May 2008
NATIONAL CONFERENCE
Siti Zuraimi Salleh, Norlaili Mat Safri and Siti Hajar Aminah Ali,
“Parameter Extraction for Control of a One Dimensional Movement by
Brain Signal”.
Proceedings of 2008 Student Conference on Research and
Development (SCOReD 2008), 26-27 Nov. 2008, Johor, Malaysia. November
2008