CONTROL OF ONE-DIMENSIONAL CURSOR MOVEMENT BY NONINVASIVE BRAIN-COMPUTER INTERFACE IN HUMANS

advertisement

CONTROL OF ONE-DIMENSIONAL CURSOR MOVEMENT BY NONINVASIVE

BRAIN-COMPUTER INTERFACE IN HUMANS

SITI ZURAIMI SALLEH

A thesis submitted in fulfilment of the requirements for the award of the degree of

Master of Engineering (Electrical)

Faculty of Electrical Engineering

Universiti Teknologi Malaysia

APRIL 2011

To my mother and father, husband, and sensei, for their precious moments iii

iv

ACKNOWLEDGEMENT

In the name of Allah, the Almighty and the Merciful. Alhamdulillah, praise to

Allah S.W.T for the guidance, strength and bless which He gave upon me to complete this research work.

First and foremost, I would like to express my sincere gratitude and appreciation to my beloved supervisor, Dr. Norlaili bt Mat Safri for her advice, knowledge, supports, kindness and patience throughout this research work. I also would like to express my thanks to Dr. Fauzan, Mr. Afzan, Mrs. Husnaini and Ms. Ashikin for their useful discussion, assistance and supports.

I am also indebted to Ms. Wan and Ms. Fiza from Medical Electronic Lab,

Universiti Teknologi Malaysia for spending their time of my overtime works. Finally, special appreciation is dedicated to my husband, family and friends for their constant support and prayers during my research work.

v

ABSTRACT

Noninvasive brain-computer interface (BCI), using electroencephalogram

(EEG) that records from scalp, could provide a new non-muscular channel for sending messages and command to the external world. The objectives of this study are to determine parameters that will drive cursor in BCI using noninvasive EEG signal and to control one dimensional cursor movement using the extracted parameters. This experimental-based study involved six normal subjects aged from

20 to 26 years old. Subjects were asked to perform tasks in two condition i.e. control condition and task condition. In the control condition, subjects were required to relax

(resting) and fix their eyes on the centre of the screen without image displayed on the screen. In the task condition subjects were tasked to imagine a movement to move the cursor on the screen towards the target. These control and task conditions were repeated four times and each condition lasted for 10 seconds. Using Fast Fourier

Transform, data in frequency domain for control and task have been obtained and analyzed in two of time interval of 1024 ms and 512 ms. Frequency is divided into six groups, i.e. delta band (0-4 Hz), theta band (4-7 Hz), alpha band (8-13 Hz), beta band (13-30 Hz), gamma band (31-50 Hz) and high gamma band (>51 Hz). Each frequency band in all frequencies of the task condition has been compared to the control condition. The present study finds optimum difference in delta frequency band between resting and active imagination at the parietal region. Furthermore, the parietal region is associated with sensory interaction and could be one of the input regions to control cursor movement. However, it is found that the delta frequency band is only applicable to a specific one-dimensional cursor movement as any imagination may produce the same results. Nevertheless, this study provides a platform for a more advance two-dimensional cursor movement study.

vi

ABSTRAK

Pengantaramuka otak-komputer (BCI) tak invasif menggunakan elektroensefalogram (EEG) yang direkod dari kulit kepala boleh menyediakan laluan baru tanpa otot untuk menghantar isyarat dan arahan ke persekitaran. Objektif kajian ini adalah untuk mengenalpasti beberapa parameter yang akan menggerakkan kursor dalam BCI dengan menggunakan isyarat EEG tak invasif dan mengawal pergerakan kursor dalam satu dimensi menggunakan parameter yang telah diekstrak. Kajian yang berunsurkan ekperimen ini melibatkan enam orang subjek normal berusia di antara 20 hingga 26 tahun. Kesemua subjek perlu melaksanakan tugas dalam dua keadaan iaitu keadaan tugasan dan keadaan kawalan. Dalam keadaan kawalan, subjek hanya perlu berehat dengan memandang ke arah tengah skrin tanpa paparan imej. Manakala dalam keadaan tugasan, subjek diminta membayangkan satu pergerakan untuk menggerakkan kursor ke arah target pada skrin. Kedua-dua keadaan ini diulang sebanyak empat kali dan setiap satunya berlangsung selama 10 saat. Menggunakan jelmaan Fourier cepat, data dalam bentuk frekuensi diperolehi dan dianalisis dalam dua sela masa iaitu 1024 ms dan 512 ms. Julat frekuensi pula dibahagikan kepada enam kumpulan iaitu delta (0-4 Hz), teta (4-7 Hz), alfa (8-13

Hz), beta (13-30 Hz), gama (31-50 Hz) dan gama tinggi (> 51 Hz). Setiap frekuensi dalam keadaan tugasan dan kawalan dibandingkan. Kajian ini mendapati bahawa perbezaan frekuensi dalam keadaan rehat dan imaginasi adalah optimum dalam julat delta di bahagian parietal. Dalam kajian ini, bahagian parietal berkaitan dengan interaksi deria dan menjadi salah satu input untuk mengawal gerakan kursor. Walau bagaimanapun, didapati bahawa jalur frekuensi delta hanya boleh diaplikasi ke atas gerakan kursor satu dimensi sahaja di mana sebarang imaginasi boleh memberikan hasil keputusan yang sama. Namun demikian, kajian ini juga memberi ruang yang menjurus kepada kajian gerakan kursor dua dimensi.

vii

TABLE OF CONTENTS

CHAPTER TITLE PAGE

DECLARATION ii

DEDICATION iii

ACKNOWLEDGEMENT iv

ABSTRACT v

ABSTRAK vi

CONTENTS vii

TABLES xi

LIST OF FIGURES xii xvi

APPENDICES xvii

1 INTRODUCTION 1

1.3 Objectives 3

viii

REVIEW 6

International Electrode

BCI 11

Published on

Published on

Published on

Published on

2.6 Discussion on Published Work in BCI 20

METHODOLOGY 22

Board Adapter 24

3.3 Data Acquisition and Recording Process 26 ix

Flow of

4 FREQUENCY ANALYSIS AND FINDINGS

4.2 EEG Signals

35

35

4.3.2 Time

5 DISCUSSION 48

CONCLUSION 51 x

REFERENCES 53

xi

TABLE NO.

LIST OF TABLES

TITLE PAGE

4.3 and time 46

Comparison of hit rates and movement precision 46 of and

FIGURE NO. xii

LIST OF FIGURES

TITLLE PAGE

2.1

2.1 (a)

2.1 (b)

2.1 (c)

EEG waveforms

Delta wave

Theta wave

Alpha wave

2.1 (d)

2.1 (e)

Beta wave

Gamma wave

2.3

The placement

Basic design and operation of a BCI system

8

8

8

8

8

8

10

xiii used 23

3.1 (a)

3.1 (c)

Electro-cap 23

Electro adapters 23

Body harness 23 needle/syringe 23

Ear electrodes 23 3.1 (f)

Electrode adapter

3.3

3.4

3.5

Division of data for 1024 ms and 512 ms time interval

Flow chart of parameter determination process

Flow chart of finalized parameter determination process

27

29

30

3.6(b) Workspace 31

3.7(b) Baseline 32

3.8(b) Cursor 33

3.9(b)

4.1

4.2

4.3

4.4 (a)

4.4 (b)

4.4 (c)

4.4 (d)

4.5 (a)

4.5 (b)

4.5 (c)

Target

Example of 1 second of raw EEG signal from subject 1

34 at P

Z

channel

Example of power spectrum of EEG signal from subject 1

36 at P

Z

channel 36

Example of 1 second data of EEG signal from subject 1 37

The occurrences of maximum DP for trial 1 for 1024 ms time interval 38

The occurrences of maximum DP for trial 2 for 1024 ms time interval 39

The occurrences of maximum DP for trial 3 for 1024 ms time interval

The occurrences of maximum DP for trial 4 for 1024 ms

39 time interval

The occurrences of maximum DP for trial 1 for 512 ms

40 time interval

The occurrences of maximum DP for trial 2 for 1024 ms

41 time interval

The occurrences of maximum DP for trial 3 for 1024 ms

41 time interval 42 xiv

4.5 (d)

4.6

4.7

The occurrences of maximum DP for trial 4 for 1024 ms time interval

Average of percentage of the maximum DP for 1024 ms

42 time interval

Average of percentage of the maximum DP for 1024 ms

43 time interval 44 xv

4.9 Location of cursor and target in workspace 47

LIST OF ABBREVIATIONS

- Amyotrophic sclerosis xvi

xvii

APPENDIX

LIST OF APPENDICES

TITLE PAGE of 57

1

CHAPTER 1

INTRODUCTION

1.1

Introduction

For normal people, communication is a need to undergo their daily activities.

Communication is a process to transmit or transfer information, thought or by feeling by or to or between people or groups. It is a connection allowing access between persons by either verbal contact or action.

However, there are people who suffer from “locked-in syndrome” which means that they are completely unable to control any muscle that prevent them from communicating with their caregivers or environment [1]. For such users, a braincomputer interface (BCI) is the only hope for even communicating with loved ones, controlling even simple devices like televisions or lamps or otherwise expressing oneself. BCI is a novel augmentative communication system that translates human intentions into a control signal for an output device such as a computer application

[2] or a mobile robot [3], in which users send information using brain activity alone without conventional peripheral nerves and muscles [3].

2

BCI can be divided into two general categories, i.e., invasive and noninvasive

[3]. Most of noninvasive BCI systems use electroencephalogram (EEG) signals; i.e., the electrical brain activity recorded from electrodes placed on the scalp. The main source of the EEG is the synchronous activity of thousand of cortical neurons.

Measuring the EEG is a simple noninvasive way to monitor electrical brain activity, but it does not provide detailed information on the activity of single neurons (a few

μVolts) and noisy environment (especially if recording outside shield rooms) [3]. In invasive BCI systems, the activity of single neurons (their spiking rate) is recorded from microelectrodes implanted in the brain. Such systems are being studied mainly in nonhuman primates, for example see [4]. These invasive BCIs face substantial technical difficulties and entail significant clinical risks as they require that recording electrodes be implanted in the cortex and function well for long periods, and they risk infection and other damage to the brain [5]. For human, therefore, noninvasive

BCI systems are applied due to clinical risks and ethics [3].

1.2

Problem Statement

Among of the most important evaluation in one or two dimensional cursor movement-BCI is hit rate. It is an onset element to design a practical application of

BCI that depends in speed and accuracy. If the speed and accuracy can be substantially improved, the range of applications and the number of potential users would greatly increase [1].

In one dimensional cursor movement-electrocorticogram (ECoG) study, they achieved success rates of 74-100% [6] whereas in two-dimensional cursor movement-EEG study, hit rate was accomplished until 92% [5].

3

1.3

Objectives

This study was conducted with the objective of:

To determine parameters that will drive or control cursor in brain-computer interface (BCI) using noninvasive EEG signal.

To determine parameters that can produce faster hit rate of brain-computer interface.

1.4

Scope of Project

Figure 1.1 depicts the block diagram for this study’s scope of work. First,

EEG is recorded from the scalp and digitalized using acquisition system. Then the digitalized signals are subjected to feature extraction procedures, such as spectral analysis or spatial filtering. Afterwards, translation algorithm converts the EEG feature into command of the cursor movement. At the monitor screen, the cursor moves and hit target.

Acquisition

System

Subject’s head

Feature

Extraction

Translation

Algorithm

Cursor

Movement

Figure 1.1

Scope of project

4

1.5

Project Contribution

BCI uses human brain signal to control other devices. This technology has advances significantly in the world but not many researches can be found in

Malaysia. Advanced research has lead to potential use in medical field as well as in computer games industry. This research has few purposes; firstly to develop human capital in BCI for Malaysia. Secondly, is to provide platform for a more advance application such as a two-dimensional cursor movement. A study by other researchers has found that slow frequency gave highest correct response rate when presented with visual presentation of target and non-target. This research also has found slow frequency as optimum parameter to control cursor movement. However, it is noted that the methodology for the experimental set-up was weak, leading to limited application to only specific one-dimensional cursor movement as any imagination may bring about the same results. However, with proper methodology and feature extraction, there is possibility of using slow frequency for BCI application that will increase the versatility of the field.

The contribution of this research would be in the developed in-house software of BCI. The developed software is integrated with an existing hardware. The software main contribution is that it is an open source, has adjustable cursor and target size, and is panel window size-independent software. With the robust software, a more advance application can be developed and together with correct methodology, the objectives are obtainable.

1.6

Thesis Outline

This report is divided into six chapters. This initial chapter presents introduction to the project which includes the project background, problem

5 statement, objectives of the project, the main objectives, scope of the project and the project contribution. This chapter is referred to conclude the findings of the project in conclusion part.

Chapter 2 discussed about reviewed literatures and published works on BCI in view of applied methodologies over the years. This chapter also discussed on several types of electrophysiology sources and experimental designs including general overview of BCI study.

The project methodology is covered in Chapter 3 that include discussion on the processes and steps involve in managing the raw data from EEG, the formula that being used and the information on software and hardware utilize in this study.

Chapter 4 contains the result of feature extraction and feature translation in offline analysis. In feature extraction, EEG signals were analyzed using Fast Fourier

Transform (FFT) in two time interval i.e. 1024 ms and 512 ms. Afterward, two parameters from the transformed data in frequency domain were extracted, i.e., scalp location and frequency band. Applying a value coordinated by those two parameters, translation algorithm was performed. The performance in mean time, hit rate and movement precision were observed and compared with previous studies.

Chapter 5 summarizes the results and findings from the previous chapter. The values of maximum difference in power which are pointed by the two parameters might be use to control and one-dimensional cursor movement.

The overall project is rationalized and concluded in the final chapter, Chapter

6. Some suggestion and recommendation are also discussed.

6

CHAPTER 2

LITERATURE REVIEW

2.1 Introduction

Brain-computer interface (BCI) is an important medical device for patients who have lost control of their motor faculties, and no longer outwardly express their needs and thoughts. A BCI enables these “locked in” patients to control a computer with their brain activity for communication, mobility and other purposes. This chapter introduces brain waves or electroencephalogram (EEG) as basic disciplines in BCI as well as basic design of BCI and also the type of BCI. Discussions about literatures and published works on BCI over the years are also summarized.

2.2 Electroencephalography

The discovery by Hans Berger that the brain develops a low-level subaudiofrequency electrical activity has led to establishment of a neurophysiological specialty known as electroencephalography [7]. Electroencephalography (EEG) is a

7 measuring and recording method of electrical activity recorded from the scalp and the tracing result is known as electroencephalogram. The electroencephalogram or

EEG is widely used by physicians and scientists to study brain function and to diagnose neurological disorders.

An EEG waveform is a wave of electricity generated from the brain nerves and an information signal to indicate the cerebrum activity. Therefore, an EEG waveform offers important data to judge whether the brain activity is normal or not.

However since an EEG waveform is taken through the skull, the absolute voltage is very small and it is difficult to indicate the activities of each brain part in detail owing to this natural insulation and also to the depth of each part.

Usually, EEG signals are divided into a few bands according to their frequencies. They are summarized in Table 2.1 and the waveforms are shown in

Figures 2.1 (a)-(e).

Table 2.1

Frequency band for EEG signals

Frequency band

Delta

Theta

Alpha

Beta

Frequency range (Hz)

0 – 4 Hz

4 – 7 Hz

8 – 13 Hz

13 – 30 Hz

8

Delta and theta waves are known collectively as slow waves. Delta waves normally are seen in deep sleep in adults as well as in infants and children while theta waves normally are seen in sleep at any age. Delta waves have the largest amplitude of all waves. Alpha waves generally are seen in all groups but are most common in adults. They tend to be present posteriorly more than anteriorly and are especially prominent with closed eyes and with relaxation. Alpha activity disappears normally with attention, for example mental arithmetic, stress and opening eyes. Beta waves are observed in all age groups. They tend to be small in amplitude and usually are symmetric and more evident anteriorly. Drugs, such as barbiturates and benzodiazepines [8], can augment beta waves.

(a) Delta wave

(b) Theta wave

(c) Alpha wave

(d) Beta wave

(e) Gamma wave

Figure 2.1 EEG waveforms

9

2.3 The International 10-20 of Electrode Placement System

For the purposes of EEG signals recording, there is a system known as the

International 10-20 of Electrode Placement System [9]. This system is the most widely used method to describe the placement of electrodes at specific intervals along the head. As shown in Figure 2.2, each electrode site has a letter to identify the lobe, along with a number or another letter to identify the hemispheric location. The letters F, T, C, P and O represent Frontal, Temporal, Central, Parietal and Occipital, respectively. Even numbers 2, 4, 6, and 8 indicate the electrodes at right hemisphere while electrodes at the left are indicated by odd numbers 1, 3, 5 and 7. Other than that, small letter ‘z’ refers to an electrode placed in the midline. The ‘10’ and ‘20’ are referred to the 10% and 20% inter-electrode distance [9].

Figure 2.2 The 10-20 of electrode placement system

10

A BCI is a system that allows a user to communicate with the environment only through cerebral activity, without using muscular output channels. To establish a direct link between the brain and a computer, the cerebral activity is measured and then analyzed with the help of signal processing and machine algorithms. Once a certain mental activity has been detected by the computer, a response can be displayed on a screen or a command can be sent to a device – for example a robot or a remote control. The main application area for BCI is assistive technology for ablebodied as well as handicapped subjects. The basic design of BCI is shown in Figure

2.3.

BCI SYSTEM

SIGNAL

ACQUISITION

DIGITIZED

SIGNAL

SIGNAL PROCESSING

Feature

Extraction

Translation

Algorithm

DEVICE

COMMANDS

Figure 2.3: Basic design and operation of any BCI system [1]

In the BCI system, signals from brain are acquired by electrodes on the scalp and processed to extract specific signal features that reflect the user’s intent. Then, these features are translated into commands that will operate a device [1].

11

2.4.1 Invasive and Non-invasive BCI

BCI can be divided into two major methodologies which are invasive or noninvasive method. Invasive BCIs, that use activity recorded by brain implanted micro- or macroelectrodes, have higher spatial resolution, and might provide control signals with many degrees of freedom; whereas non-invasive BCIs, that use brain signals recorded with sensors outside the body boundaries [3], are convenient, safe and inexpensive, but have low spatial resolution. Another method called electrocorticography (ECoG) is an intermediate BCI methodology [10]. This moderately invasive method is using ECoG activity that recorded from the cortical surface. ECoG has higher spatial resolution than EEG, i.e., tenths of millimeter 0-100 versus centimeter. It has broader bandwidth and higher amplitude, i.e., 0-200 Hz and

50-100 μV rather than 0-40 Hz and 10-20 μV in EEG, respectively [6].

Experience of BCI research in human beings has so far primarily involved non-invasive EEG-based investigation, for example [5] and most intracortical

(invasive) BCI involved animal studies [4].

Although there was a belief that only invasive BCI could provide users with real-time multidimensional control, previous study by Wolpaw et al. [5] presented that multidimensional control also could be provided using non-invasive BCI. They demonstrated that multidimensional control of a cursor movement on a computer screen can be learned in a few session of training with non-invasive BCI in humans.

The subjects were able to move a cursor within ten seconds into one of eight goals appearing randomly at one of the four corners of the screen. The flexibility, speed and learning performance in generally equal to that seen when invasive multielectrode BCI systems were tested in animals [5].

At present, it seems probable that different recording methods will be useful for different applications, different users, or both [10].

12

2.4.2 Imagination vs. actual movement

The goal of BCI research is to provide new communication option to people with severe neuromuscular disorders such as Amyotrophic Lateral Sclerosis (ALS), brainstem stroke, cerebral palsy and spinal cord injury [11]. Although people suffered from ALS or other severe motor disabilities, but 67.5% of them [12] appear that the cognitive functions are not impaired.

The imagination of a movement generates the same mental processes and even physical processes as the performance of the movement [13]. Furthermore, brain signals produced by imagination movement are similar as actual movement. It is supported by a previous study on a tetraplegic patient, 31 years old male with a complete traumatic spinal cord injury (SCI) and five healthy volunteers with agematched acted as control group [14]. The patient underwent somatosensory evoked potential (SEP) and motor evoked potential (MEP) recordings to confirm the extent and location of cerebrospinal disconnection. They found that contrasts of imagery of repetitive hand and foot movement versus rest in the patient demonstrated significant activation of sensorimotor networks similar to the patterns defined by contrasts of active movement versus rest in the healthy control group. Another study [6] showed the spatial and spectral foci of task-related electrocorticogram (ECoG) activity, as revealed in the r

2

analysis, were usually similar for action and for imagination of the same action (e.g. saying the word ‘move’ and imagining saying the word ‘move’).

In BCI systems, electrophysiological sources refer to the neurological mechanisms or processes employed by a BCI user to generate control signals. Noninvasive BCI use the electrical activity of the brain recorded with single or multiple

13 electrodes from the scalp surface as input signal for BCI control. Usually participants are presented with stimuli or are required to perform specific mental tasks while the electrical activity of their brain is being recorded. Specific features of the EEG are either regulated by the BCI user such as slow cortical potential or sensorimotor rhythm, or elicited by sensory stimulation such as P300.

A review from Bashahati et al.

[15] reported that current BCIs fall into seven main categories, based on neuromechanics and recording technology they use. Five major groups categorized by Wolpaw [1] are sensorimotor activity, P300, visual evoked potential (VEP), slow cortical potentials (SCP) and activity of neural cell

(ANC). The other two, added by Bashahati et al . [15] are ‘response to mental tasks’ and ‘multiple neuromechanics’. All of them are summarized in Table 2.2 with simple descriptions.

Table 2.2

: Electrophysiological sources used in BCI [15]

Electrophysiological Sources Description

Sensorimotor activities

P300

Visual evoked potential (VEP)

Slow cortical potential (SCP)

Changes in brain rhythms (mu, beta, gamma)

Typically evoked in the EEG over the parietal cortex a positive peak at about 300 ms after the stimulus is received.

Small changes in the ongoing brain signal.

They are generated in response to a visual stimulus.

Slow, non-movement potential changes generated by subjects.

Activity of neural cell (ANC)

“Response to mental tasks”

“Multi-neuromechanism”

When movements are executed in the preferred direction of neurons, the firing rates of neuron in the motor cortex are increased.

Assume that different mental tasks lead to distinct, task-specific distributions of EEG frequency patterns over the scalp.

Use a combination of two or more of the above-mentioned neuromechanisms.

14

Figures 2.3 (a)-(d) show the present-human BCI system types which are sensorimotor rhythm, slow cortical potential, P300 evoked potential and cortical neuronal activity, respectively.

(a) Sensorimotor rhythm (b) Slow cortical potentials

(c) P300 evoked potential (d) Cortical neuronal activity

Figure 2.4

Present-day human BCI system types [1]

Sensorimotor rhythm (SMR) is a majority preferable method of BCI because it is related to the motor cortex with distributions of somatosensory areas. It includes a frequency of 10 Hz (range 8-11 Hz) often mixed with a beta, around 20 Hz and a gamma component, around 40 Hz recorded over somatosensory cortices. It is most preferable over C

3

and C

4.

SMR desynchronizes with movement, movement imagery and movement preparation (event-related desynchronization, ERD), and increases or

15 synchronizes (event-related synchronization, ERS) in the post-movement period or during relaxation [16].

Slow cortical potential (SCP) is among the lowest frequency features of the scalp-recorded EEG generated in cortex. Negative SCPs are typically associated with movement and other functions involving cortical activation, while positive SCPs are usually associated with reduces cortical activation [1]. SCP is used in [17] for thought translation device as one of the BCI application.

The P300 component of event-related potential (ERP) is a positive deflection in the EEG time-locked to stimuli presentation. It is typically seen when participants are required to attend a stream of rare target stimuli [16]. P300 is prominent only in the responses elicited by the desires choice, and BCI uses this effect to determine the user’s intent. A P300-based BCI has an apparent advantage in that it requires no initial user training. It is typically response to a desired choice [1].

BCI technology is used to record and analyse brain signals to determine the output that is desired by the user such as which letter to select for spelling a word, which direction to move a cursor and so on. Therefore, signal processing is one of the important components in BCI research. This signal processing has two stages

[10]. The first stage is feature extraction which is the measurement of the characteristics of the signals that encode the output. These features can be simple measures such as amplitudes of particular evoked potentials or of particular rhythm.

The second stage is the translation of these signal features into device commands using a translation algorithm. Brain signal characteristics such as rhythm amplitudes are translated into command that specify outputs, such as letter selection or cursor

16 movement. Translation algorithm can be simple such as linear equation [11] or complex such as neural network [18] or support vector machines [19].

In this study, signal processing is used to extract features and the method used was frequency analysis. Bashashati et al. [15] reviewed that the magnitude of the Fourier Transform (FT) of the signal may be considered as a representation with perfect spectral resolution but with no temporal information. Frequency-based features have been widely used in signal processing because of their ease of application, computational speed and direct interpretation of the result.

Recent and current studies seek to improve the speed and accuracy of this control by improving the selection of signal features and their translation into device commands, by incorporating additional signal features and by optimizing the adaptive interaction between the user and the system [11].

2.5 Published Work on BCI

A bibliometric study in [20] reviewed the growing interest with associated number of publications over the last two decades. Earliest paper describing a BCI system was published in 1973 by J. J. Vidal [21]. Two decades later, the number of papers grew substantially until over a thousand papers. [20] found that the most productive authors in BCI research are G. Purtscheller from Graz University of

Technology (Austria), N. Birbaumer from University of Tubingen (Germany) and J.

R. Wolpaw from Wadsworth Center (New York, USA).

For instance, Wolpaw and his group have been published papers to move cursor in one dimensional and two dimensional. They focused on mu and beta

17 rhythm [5]. Their published work on two-dimensional (2D) cursor movement motivated current study for developing local brain-computer interface system. Using mu and beta rhythm component, their research tested on four disables subject produced the result that falls within invasive study using monkey. They emphasized that non-invasive of BCI also could provide multidimensional movement control without any implanted in the brain. In view of movement time, precision and accuracy, the results were comparable to these with invasive BCIs.

BCI research areas include evaluation of invasive and non-invasive technologies to measure brain activity, development of new BCI applications, evaluation of control-signal (i.e. patterns of brain activity that can be used for communication), development of algorithm for translation of brain signals into computer commands, and the development and evaluation of BCI system specifically for disabled subjects.

2.5.1 Published Work on Neurofeedback

Neurofeedback is a type of biofeedback that uses real time displays of electroencephalography to show brain activity, often with a goal of controlling central nervous system. It is a form of training or exercise for the brain which is assisted with a very sophisticated technology and guided or directed by knowledge gained through the advances of neuroscience. Neurofeedback in advanced is used to improve attention, anxiety, mood and behaviour.

Regardless of clinical and psychological application, the currently most advanced BCI make use of the fact that humans can acquire control over EEG parameters by means of neurofeedback, i.e., real-time visual or auditory feedback of a specific EEG parameter.

18

For example, previous study by Hwang et al.

[22] has developed a kind of neurofeedback systems to train motor imagery by presenting participants with timevarying activation maps of their brain, using a real-time cortical rhythmic activity monitoring system. By employing ten healthy subjects, five subjects in training group and another five subjects in control group, they conducted experiments consisted of two sessions which were motor imagery training session and EEG recording session. In the first session, participants were trying to increase their mu rhythm activations around the sensorimotor cortex while they were watching their cortical activation map through the real-time. In the second session, two EEG recordings were performed each before and after motor imagery training session to demonstrate the effect of their neurofeedback-based motor imagery training system.

They found that there were meaningful changes of brain activity in training group around sensorimotor rhythm (10 and 20 Hz) compared to control group. With this, they confirmed the effectiveness of their neurofeedback-based motor imagery training in used of BCI application.

2.5.2 Publish Work on Cursor Control

Cursor control movement is one of the application and assessment in BCI.

Basically the study starts with one-dimensional cursor movement, horizontal or vertical and continues with advanced 2D or 3D cursor movement. As mentioned earlier, Wolpaw and colleagues have been published SMR-based BCI in cursor movement.

Other than that, a study by Wilson et al.

[23] published in Journal of

Visualized Experiments (JoVE) measured the amplitude of the EEG signals in selected frequency bins into a device command, i.e., the horizontal and vertical velocity of computer cursor. They were utilizing BCI2000 application as analysis tool. Using the power in mu (8-12 Hz) and beta (18-28 Hz), the user in the study was

19 asked to make several different imagined movements with their hands and feet to determine the unique EEG features that change with the imagined movements. They determined which EEG features are strongly correlated with each movement by finding the large r-squared values in the plots produced from the analysis tool and found on channels 9, 10, 17, 18 and 19 according to 10-20 international system electrode placement. The subject’s goal was to move the cursor to the correct target using imagined movements corresponding to the desired direction of movement (e.g. right hand to move right or feet to move down). They found that a trained subject should be able to move cursor to the target within 1 or 2 seconds.

2.5.3 Published Work on Other Control

The concept of BCI stems from a need for alternative, augmentative communication and control options for individuals with severe disabilities, though its potential uses extend to rehabilitation of neurological disorders, brain-state monitoring as well as gaming [16].

Previous study by Lalor et al . [24] addressed an application of steady-state visual evoked potential (SSVEP) – based BCI design within a real-time gaming framework. The video game involved the movement of an animated character within a virtual environment

Finke et al.

[25] also presented a BCI game named the MindGame, based on the P300 event related potential. In the MindGame interface, P300 events were translated into movements of a character on a three-dimensional game board. While game and multimedia control with a BCI is inherently a challenging field, it potentially also opens the door for interesting applications in a clinical context [25].

20

Another study with different application was published by Bell et al.

[26].

Their study was about to control a humanoid robot using non-invasive BCI. They also used P300 evoked potential as their subject’s command to the robot. The P300 was used to discern which object the robot should pick up and which location the robot should bring the object to. The robot which was including a pan-tilt camera head for visual feedback to the user and two hands for grasping objects, transmitted the image of objects discovered by its cameras back to user for selection. The user was instructed to attend to the image of the object of their choice while the border around each image was flashed in a random order. Machine learning techniques were used to classify the subject’s response to a particular image flash as containing a

P300 or not, thereby the user’s choice of the object could be inferred. A similar procedure was used to select a destination location. From the study, their results showed that a command for the robot can be selected from four possible choices in

5s with 95% accuracy [26].

In other study, a group of researcher from Singapore [27] was developing a new BCI application which could also provide novel ways of entertainment. They developed software and Graphical User Interface (GUI) on music composition using

BCI. They investigated the potential of BCI in music composition encompassing the topic of gaming, entertainment and possibly rehabilitation with the objectives to investigate other communication channel than spelling words and to stimulate brainstorming on future BCI research avenues while reviewing the limitations of the current state-of-the-art [27].

2.6 Discussion on Published Work in BCI

Sensorimotor rhythms (SMRs) are the most often used of EEG component in non-invasive BCI system. Produced in sensorimotor cortex and associated areas, they are mostly directly related to movement. In addition, previous studies also suggested

21 that people could learn to control their SMRs amplitude. However, patients or users need to be trained several time to control their rhythms. For example in [5].

Moreover, not all users are able to obtain control of their brain waves after intensive training.

P300 signal is one of the typical patterns captured from EEG at the central cortex and also mostly choose as a method in BCI system. Despite of its robustness, the such system may require permanent attention to external stimuli which may be fatiguing for some users.

Considered as imagined-movement mental-task BCI system, the method in this study does not require a priori selection of subject-specific frequency and not required in-between interface like the screen of alphanumeric characters used in the

VEP-based BCI. Our method assumed that different mental tasks, i.e., task and control condition, lead to distinct, task-specific distributions of EEG frequency patterns over the scalp.

22

CHAPTER 3

METHODOLOGY

3.1 Introduction

This chapter discusses the methodology used in this study. It concerns two main issues of how the data were collected and generated as well as how they were analyzed towards the purpose of this study which is to examine and determine the parameter to drive cursor movement. This chapter includes experimental set up, the equipment, procedures and the flow chart of the parameter selection.

3.2 Experimental Set Up

The experiments were conducted at Medical Electronic Lab, Faculty of

Electrical Engineering, UTM. EEG signals were recorded using Neurofax EEG-9000

V.05-90 data acquisition from Nihon Kohden, Japan. The experiments roughly took about one and half hour of setting up that includes preparing the subject, recording the data and cleaning the subject and equipments.

23

3.2.1 Equipment and Supplies

There were a few equipments and supplies that need to be used together with

EEG machine to complete the experiment. They were ECI

TM

electro-caps, electrode board adapter, body harness, disposable sponge disks, ECI Electro-Gel

TM

, blunted needle/syringe kit, ear electrodes, color-coded head measuring tape and electrode board adapter connector as shown in Figure 3.1(a)-(g).

(a) Electro-cap (b) Electrode board adapter (c) Body harness

(d) Disposable sponge disks (e) Blunted needle/syringe kit (f) Ear electrodes

(g) Electrode board adapter connector

Figure 3.1 Equipments used in the experiment

24

3.2.2 Preparation of Subject

To operate EEG experiments, the subject must be relaxed with no stress and fatigue to avoid the EEG machine recorded unwanted signals. Therefore, to prepare the subject, steps taken were as follows: a.

Body harness was put on the subject chest to support the cap. b.

Before applying the suitable cap, the circumference of subject’s head was measured. c.

Ear electrodes were attached at the ears and gel was added. d.

Disposable sponge disks were placed around F

P1

and F

P2

channels to absorb perspiration and prevent the spread of electrode gel onto the forehead. e.

Then, the proper size of cap was slipped onto the subject’s head and the cap straps were attached to the body harness. f.

Cap connector was connected to the electrode board adapter. g.

Each electrode cavity was filled with electrode-gel by moving the blunted needle/syringe rapidly back and forth. h.

The electrode impedance was checked. i.

Finally, the subject was moved to the chair for recording.

3.2.2.1 Installation of Electrodes onto Electrode Board Adapter

The Electrode Board Adapter is used to connect the Electro-Cap to EEG instruments. Each wire/connector pin (socket) on the Electrode Board Adapter is color-coded for easy identification. Each wire contains of two tips which are white and red tip. Each of them represents the electrode placement as in Tables 3.1.

25

White Tip

Table 3.1

: Electrodes position

Wire Color Red Tip

F

P1

Brown F

P2

F

3

Red F

4

C

3

Orange C

4

P

3

Yellow P

4

0

1

Green 0

2

F

7

Blue F

8

T

3

Violet T

4

T

5

Gray T

6

GND White C

Z

F

Z

Black P

Z

3.2.3 Clean Up

When EEG recording finished, the cap, ear electrodes and body harness were removed from subject. ECI

TM

Electro-Caps must be cleaned frequently for sanitary reasons. In addition, if all the gel is not washed from a cap, the material will lose its elasticity and shortened the life of the cap.

26

3.3 Data Acquisition and Recording Process

3.3.1 Participant

Six normal subjects aged from 20 to 26 years old, gave informed consent and participated in the study.

3.3.2 Conditions

Subjects sat on a chair facing a monitor screen, placed one meter in front of the subjects. In Control condition, subjects were asked to relax (resting). In Task condition, subjects were asked to imagine voluntary movement, e.g. imagine moving a cursor on a monitor screen.

EEG signals were obtained from 19 scalp electrodes, placed on the scalp based on 10-20 electrode placement system as discussed in section 2.3

with reference electrodes attached to left and right ears. Signals were recorded with pass bands of

0.5 – 120 Hz and stored in a personal computer with a sampling frequency of 1 kHz.

A single trial lasted for 10 second and four trials were performed for each condition with intervening rest period to avoid fatigue as shown in Figure 3.2.

27

CC TC CC TC CC TC CC TC

10s 10s 10s 10s 10s 10s

Trial 1 Trial 2 Trial 3

Notes:

CC : Control Condition

TC : Task Condition

Figure 3.2

Sequence of data acquisition

10s 10s

Trial 4

A 10 seconds data of Control (CC) and Task (TC) conditions (Figure 3.2) were divided into ten frames so that each frame consists of one second data. To obtain optimum speed and accuracy, the EEG data were divided at various intervals, i.e., 1024 ms and 512 ms as illustrated in Figure 3.3.

10 second of data

1 st

frame 2 nd

frame

…………. 10 th

frame

1024 data

1024 data

1024 ms Time Interval

1 st

frame 2 nd

frame …………. 10 th

frame

512 data

512 ms Time Interval

512 data

Figure 3.3 Division of data for 1024 ms and 512 ms time interval

28

Using Fast Fourier Transform (FFT), for each interval, data in frequency domain of each frame were obtained. For Control condition, five frames in the middle were averaged and assigned as baseline. Frequency was divided into six groups, i.e., delta band (0-4 Hz), theta band (4-7 Hz), alpha band (8-13 Hz), beta band (13 – 30 Hz), gamma band (31-50 Hz) and high gamma band (>51 Hz). Each frequency band in all frequencies of the Task condition was compared to the Control condition. Differences between these two condition were obtained and observed using Equation 1. difference in power (DP) (f) = power in Task (f) - power in Control (f) (1)

29

3.4 Flow Chart of Data Analysis

Figure 3.4 shows the flow to analyze EEG signals using C programming in

Linux. The process was repeated four times to analyze four trials of one subject.

Control condition

(10 s)

Calculate power spectrum using FFT

Average Control power spectrum

START

Record EEG signals

Compute difference power between

Task and Control power spectrum

Task condition

(10 s)

Calculate power spectrum using FFT

Find maximum difference power occurrence at which scalp location and frequency band

END

Figure 3.4

Flow chart of parameter determination process

30

After that, all maximum DP values were gathered into one folder to choose a single scalp location and frequency band. The process was summarized as in Figure

3.5. The maximum DP values from identified parameters were used to move cursor in further step.

Get maximum

DP from

Trial 1

Get

Trial 2

START maximum

DP from identical?

NO

DP for all trials

Determine parameters

END

Get maximum

DP from

Trial 3

Scalp location and frequency bands were

Average maximum

Get maximum

DP from

Trial 4

YES

Figure 3.5

Flow chart of finalized parameter determination process

31

The software development has been carried out using the X Window System

(commonly referred to as X or X11) on LINUX platform. This system is a networktransparent graphically windowing system based on a client/server model. Full coding commands can be referred in Appendix A ( Movecursor.c

) and B ( Features.c

) .

Both of them were developed using C language in LINUX. However in this section, the explanation focuses on the main graphics that associated to the purpose of the study.

Figures 3.6(a), 3.7(a), 3.8(a) and 3.9(a) show some writing codes in X. Figure

3.6(a) shows the command to develop a workspace. The size of workspace was optimized according to the dimension of screen computer used, as shown in Figure

3.6(b).

Figure 3.6(a) Syntax of workspace

Figure 3.6(b) Workspace

32

In addition, the baseline inside the workspace as shown in Figure 3.7(b) was drawn using the coding command as in Figure 3.7(a). The coding drew one horizontal line and one vertical line which met at the centre.

!" #"

$%

%

Figure 3.7(a) Syntax of baseline

Figure 3.7(b) Baseline

Furthermore, Figure 3.8(a) shows the syntax to build a block of rectangular cursor in blue colour. This variable cursor size was initiated at the centre of the screen. For example, the cursor size in this study was 10 x 10ixels as shown in

Figure 3.8(b).

33

!

!" #"

!" #" !" #"

!" #"

Figure 3.8(a) Syntax of cursor

Figure 3.8(b) Cursor

Figure 3.9(a) shows the language structure to display target. The target was a graphic located at the right side of the workspace as shown in Figure 3.9(b). It was inconstant value and designed in black colour. The size of the target was 30 x 30 pixels used in this study.

34

"

" "

"

Figure 3.9(a) Syntax of target

Figure 3.9(b) Target

In the situation of cursor movement, it should jump from the initial state according to a value indicated by extracted parameters. If the cursor succeed hit the target within ten seconds, a reward in form of string text would appear.

35

CHAPTER 4

FREQUENCY ANALYSIS AND FINDINGS

This chapter presents the result and analysis of the project. The results are arranged into two parts which are feature extraction results and offline feature translation results.

Figure 4.1 shows an example of raw EEG signals of one channel in one second, taken from subject 1 at trial one located at P

Z

channel. Meanwhile the example of power spectrum of EEG signal is shown in Figure 4.2. From 19 channels of EEG signals recorded, however, there were only nine channels analyzed i.e. F

3

,

F

Z

, F

4

, C

3

, C

Z

, C

4

, P

3

, P

4

, P

Z

and P

3

(Figure 4.3) that represents the location commonly used in BCI research.

36

Notes:

Task

Control

Figure 4.1

Example of 1 second of raw EEG signal from subject 1 at P

Z channel

Notes:

Task

Control

Figure 4.2

Example of power spectrum of EEG signal from subject 1 at P

Z channel

37

F3

C3

P3

FZ

CZ

PZ

F4

C4

P4

1s

Notes:

Task

Control

Figure 4.3 Example of 1 second data of EEG signal from subject 1

4.3 Feature Extraction Analysis

To extract mean values from EEG signals, two time intervals were applied, i.e., 1024 ms time interval and 512 ms time interval (Figure 3.3). The findings from both time intervals were compared.

38

4.3.1 1024 ms Time Interval

Figures 4.4(a)-(d) show the results of percentage of maximum DP using 1024 ms time interval for all trial (6 subjects x 10 frames). For trial 1 (Figure 4.4 (a)), maximum DP was found at P

4

site with 56.7% occurrence. For trial 2, (Figure 4.4

(b)), maximum DP was observed at another site i.e. P

3

with 51.7% occurrence. For trials 3 and 4 (Figures 4.4 (c) and (d)), both of maximum DP was found at P

Z

site with 61.7% and 60.0% occurrence, respectively. Generally, the maximum DP occurred at delta band, which ranged from 0 to 4 Hz, for all trials.

(Delta, P

4

, 56.7%)

(a) Trial 1

(Delta, P

3

, 51.7%)

(b) Trial 2

(Delta, P

Z

, 61.7%)

(c) Trial 3

39

40

(Delta, P

Z

, 61.7%)

(d) Trial 4

Figure 4.4

The occurrences of maximum DP for 1024 ms time interval

4.3.2 512 ms Time Interval

The maximum DP result for all trials (6 subjects x 10 frames) using 512 ms time interval is shown in Figures 4.5 (a)-(d). In section 4.3.1, the frequency range was constant for all trials i.e. in delta band. However, there were two different frequency band observed in time interval 512 ms. For trials 1 and 2 (Figures 4.5 (a) and (b)), both maximum DP occurred at F

3

site with 38.3% and 40% occurrence respectively, but in different frequency band. For trial 1, maximum DP occurred within delta band whereas for trial 2, within beta band. For trials 3 and 4 (Figures 4.5

(c) and (d)), the frequency range was seen constant but not the scalp location. The maximum DP occurred in delta band for both trials 3 and 4. For trial 3, the maximum

DP was found at P

3

site with 40% meanwhile there were three different sites

41 assembled the maximum DP for trial 4 with similar percentage 38.3% i.e. F

3

, P

3

and

P

4

.

(Delta, F

3

, 38.3%)

(a) Trial 1

(Beta, F

3

, 40%)

(b) Trial 2

(Delta, P

3

, 40%)

(c) Trial 3

(Delta, P

4,

P

3,

F

3

, 38.3%)

(d) Trial 4

Figure 4.5 The occurrences of maximum DP for 512 ms time interval

42

43

4.3.3 Mean for All Trials

Initially, it is expected that every trial in both time intervals will produce similar result to determine the parameters. Unfortunately, for 1024 ms time interval, although the frequency band remains, but the maximum DP occurred at various scalp location for all trials. Meanwhile for 512 ms time interval, the extracted parameter occurred at different frequency band and at various scalp location, too. Therefore, to overcome these distinctions, averaging process was done for all trials and the results are shown in Figures 4.6 and 4.7 for 1024 ms time interval and 512 ms time interval, respectively.

Figure 4.6

Average of percentage of the maximum DP for 1024 ms time interval

Figure 4.6 shows the scalp location with highest percentage of averaged maximum DP for 1024 ms time interval. Delta band was determined first because of its consistency for all trials. Therefore, this averaging process only involved the

44 maximum DP values in various scalp location within delta frequency range. It shows that the maximum DP was highly occurred at P

Z

with 56.6% followed by P

3

(54.33%) and P

4

(53.3%).

(Delta, P

3

, 35.4%)

Figure 4.7 Average of percentage of the maximum DP for 512 ms time interval

For 512 ms time interval, the occurrences of maximum DP were vary in both parameters i.e. scalp location and frequency range, for all trials. Thus, averaging process was for the maximum DP values from various scalp location and frequencies range. Figure 4.7 shows that the highest percentage of averaged maximum DP occurred at P

3

site within delta frequency range with 35.4%. The other two locations were P

4

and F

3

, with 34.6% and 33.3% respectively, in the similar ranged of frequency.

From the result, delta band and parietal region were selected as the parameters to drive the cursor movement. The DP values from P

Z

site in 1024 ms time interval were used instead of P

3

site from 512 ms interval due to its highest percentage of occurrence.

45

By applying linear equation in Equation 2, cursor movement was controlled using the maximum DP values from P

Z

site. For each trial, there were ten values of maximum DP originated from ten frames of EEG data in frequency domain. These values will drive cursor to hit the target. The formula use to translate brain signal at

P

Z

site to pixel value is

C

P n

= C

P (n-1)

+ k DP n

(2) where C

P n

is current cursor position, C

P (n-1) is previous cursor position, k is gain (in this study k = 64 is selected) and DP n value is determined from feature extraction.

The equation is expanded as in Appendix A as shown in Figure 4.8.

&'

" () (**+

&' &'*&'

Figure 4.8

Translation equations in C structure

The performance of cursor movement was also tested using the combination of two or three parietal scalp locations in addition of P

Z

site, i.e. P

Z

and P

3

; P

Z

and P

4

; and P

Z

, P

3

and P

4

. The combinations of the channels were summarized in Table 4.1.

Table 4.1

: Channels combination

Combination Channel(s)

- P

Z

P

Z and P

3

A

CM

B

CM

C

CM

P

Z and P4

P

Z

, P

3

and P

4

46

One dimensional cursor movement using DP values from P

Z

produced 4.41s mean time with hit rate of 95.83%. The combination of two scalp location; P

Z

and P

3 and P

Z

and P

4 yielded mean time of 3.25s and 3.58s with 95.83% and 100% hit rate, respectively. Meanwhile, the summing of DP values from all those parietal sites gave

2.91s mean time with 100% hit rate (Table 4.2).

Table 4.2

Hit rate and mean time

(%) (s)

-

A

CM

B

CM

C

CM

P

Z

P

Z and P3

P

Z and P4

P

Z

, P3 and P4

95.83

100

100

3.25

3.58

2.91

Notes: n = (6 subject x 4 trials)

= 24

Table 4.3

Comparison of hit rates and movement precision

Current study

Wolpaw [5]

Leuthardt [12]

Hit rate (%)

95.83 – 100 %

92 %

74 – 100%

Movement precision

0.114

4.9

-

Movement precision was calculated as target size as percentage of workspace, with the smaller size the better. Table 4.3 shows the movement precision result for this study and others as well. From the table, it is obvious that current study produced better precision compared to the other. In this research, the size of target and cursor were chosen as shown in Table 4.4. The location of the cursor and target are shown as in Figure 4.9.

47

Table 4.4: Size of target, cursor and workspace

Item Size

Target 30 x 30

Cursor

Workspace

10 x 10

1024 x 768

Figure 4.9

Location of cursor and target in workspace

48

CHAPTER 5

DISCUSSION

This chapter discusses further on the analysis from feature extraction and translation. The maximum DP is found in two parameters which are in delta frequency and parietal region.

In this study, the maximum difference in power between resting and active imagination was observed at which scalp location and frequency band it occurred.

These two features (scalp location and frequency band) are similarly identified by

Leuthardt et al. [6] in their BCI study using electrocorticography (ECoG). However, they focused on r

2

value instead of maximum difference in power.

49

From the findings, the maximum different in power (DP) between the two conditions occurs in delta band at posterior area, i.e. P

Z

for 1024 ms time interval and

P

3

for 512 ms time interval. Comparing the two time intervals, delta band at central posterior area (P

Z

) provides higher percentage of difference in power, hence can be use to control one dimensional cursor movement [10]. By selecting the power difference of delta frequency band (< 4 Hz) in central posterior area, it is expected that the cursor can move further and faster in one dimensional direction towards targeted location. It is also expected that no training is required to obtain optimum results. Many researchers have shown that that BCI training must be conducted many times to achieve best performance for each subject. For example, Wolpaw et al.

[5] had conducted over twenty sessions per subject, at a rate of two to four per week. In this study, the result was based on a single session of an experiment; hence it is believed that the extracted feature can be used to control a one dimensional cursor movement without prior training. However further study is needed to delineate this speculation.

Although it is always been reported that BCI researches used mu and beta rhythm which is associated with actual movement or imagination of movement, for this research, however, the values in the delta band are used instead to convert EEG signals to cursor movement. Wolpaw et al.

[1] reported that movement or preparation for movement is typically accompanied by a decrease in mu (8-12 Hz) and beta (13-30Hz) rhythm amplitudes, therefore, minimizing their different amplitude values in power. In this study, we found maximum difference in power occur in delta frequency band even though the slow rhythm is always being associated with sleep wave in adult. A few other studies have shown that slow frequency rhythm can be use as indicator for communication aid device [28, 29].

Neshige et al.

[28] presented a study designed to develop a communication aid device utilizing event-related potentials (ERPs) for patient with severe motor disabilities. The eight subjects were given illustrated stimuli consisted of four pictorial symbols (eat, toilet, drink and cross). By analyzing P300 at P

Z

, they found correct response rate after averaging 5 or 10 epochs of ERPs was in frequency range of 2-5 Hz. Another study by Harmony et al.

[29] investigated EEG delta activity by performing two experiments. One of them was performing a difficult mental

50 calculation task and control stimuli. They found the increase of power from 1.56 to

5.46 Hz was observed only during the performance of the task and not during the control condition. By any chance, the different power between task and control condition increased in that frequency.

Fernandez et al.

[31] discussed that one possible explanation for the increase of delta power during the performance of mental tasks is the activation of the corticofugal pathway that inhibits the thalamocortical cells, producing a disconnection of the cortex from environmental stimuli, a state which we call internal concentration.

However, more research is needed to elucidate the intrinsic mechanisms for the generation of these slow waves during mental activity. The two studies from Neshige et al.

and Harmony et al.

have shown that information could also be found in the low frequencies of EEG signal. This study also has found slow frequency as optimum parameter to control cursor movement. However, it is noted that the methodology for the experimental set-up in this research was weak, leading to limited application to only specific one-dimensional cursor movement as any imagination may bring about the same results. However, with proper methodology and feature extraction, there is possibility of using slow frequency for BCI application that will increase the versatility of the field.

As for the optimum scalp location, the maximum power difference was found at posterior area that contains primary and associations cortices for somatosensation

[31]. The region can be divided into two functional regions, one that involves sensation and perception and the other that concern with integrating sensory input, primarily with the visual system [32]. Since the subject was provided with the visual of targeted location and the cursor location, the posterior area probably constructs a spatial coordinate system to represent the two locations. Further study is needed in this aspect.

51

CHAPTER 6

CONCLUSION

6.1 Conclusion

BCI allows users to directly communicate their intention without any involvement of the motor periphery. The development of non-visual BCI renders this technology feasible for patients who lost control of eye movement due to disease progression or injury.

In this study, we have presented an imagined-movement mental task to drive cursor in BCI application. We have extracted different power between task and control condition in delta frequency at center posterior region. By using the different power value, cursor movement was performed. Rate of cursor hits target is ranged from 95.83-100%. However, it is found that the delta frequency band is only applicable to a specific one-dimensional cursor movement as any imagination may produce the same results. Nevertheless, this study provides a platform for a more advance two-dimensional cursor movement study.

52

6.2 Recommendation for Future work

In this experiment, the participants were healthy subject. The ultimate goal of

BCI is to provide an option for paralyzed and motor-disabled persons. To observe the reliability of BCI, the author suggests to involved paralyzed or motor-disabled persons as subjects.

Besides, BCI researchers also may used dry electrode instead of gelelectrode. This new device has the advantages of no need for skin-preparation or application. Hence, the experiments become less time consuming and decreased the uncomfortable of the subjects [33].

Despite recent advancement, BCI is a field still in its infancy and several issues need to be addressed to improve the speed and performance of BCI. It is suggested to explore more on local components of brain activity with fast dynamics that subject can consciously control [3]. For this, the increasing knowledge of the brain (where and how cognitive and motor decisions are made) is needed as well as the application of more powerful digital signal processing methods that those commonly used to date.

53

REFERENCES

1.

Wolpaw, J.R., Birbaumer, N., McFarland, D.J., G. Pfurtscheller, Vaughan.

T.M. (2002). Brain-Computer Interfaces for Communication and Control.

Clinical Neurophysiology , 113: 767-791

2.

McFarland, D.J. and Wolpaw, J.R. (2005). Sensorimotor Rhythm-Based

Brain-Computer Interface (BCI): Feature Selection by Regression Improves

Performance. IEEE Transaction on neural Systems and Rehab., 13(3): 372-

379

3.

Millan, J.R., Renkens, F., Mourino J. and Gerstner, W. (2004). Brain-

Actuated Interaction. Artificial Intelligence. 159: 241-259

4.

Carmena, J.M., Lebedev, M.A., Crist, R.E., O’Doherty, J.E., Santucci, D.M.,

Dimitrov, D.F., Patil, P.G., Henriquez, C. S. and Nicolelis, M. A. L. (2003).

Learning to Control a Brain-Machine Interface for Reaching and Grasping by

Primates. PLOS Biology . 1(2): 193-208

5.

Wolpaw, J.R., McFarland, D.J., Vaughan, T.M. and Schalk, G. (2004).

Control of A Two Dimensional Movement Signal by A Noninvasive Brain-

Computer Interface in Humans. PNAS. 101(51):17849-17854

6.

Leuthardt, E.C., Schalk, G. and Wolpaw, J.R. (2004). A Brain-Computer

Interface Using Electrocorticographic Signals in Human.

J. Neural

Engineering.

1: 63-71

7.

Shipton, H. W. (1975). EEG Analysis: A History and A Prospectus.

AnnualReview of Biophysics and Bioengineering. 4, 1–13

54

8.

Sucholeiki, R. (2008). Normal EEG Waveforms. E-Medicine . 1-7

9.

Biomedical Signals Amplifier , ElettronicaVeneta, pp. 27 (2006)

10.

Daly, J. J. and Wolpaw, J. R. (2008). Brain-Computer Interface in

Neurological Rehabilitation. Journal of Lancet Neurology.

7, 1032-1043a

11.

Wolpaw, J.R., McFarland, D.J. and Vaughan, T.M. (2003). Brain-computer interface research at the Wadsworth Center. IEEE Transaction on Neural

Systems and Rehab. 8(2), 222-226

12.

Cognitive Impairment Appears To Be Common in ALS Patients. Medical

News Today

13.

Ianez, E., Azorin, J. M., Ubeda, A., Ferrandez, J. M. and Fernandez, E.

(2010). Mental Tasks-Based Brain-Robot Interface. Robotics and

Autonomous Systems. 58, 1238-1245

14.

Enzinger, C., Ropele, S., Fazekas, F., Loitfelder, M., Gorani, F., Seifret, T.,

Reiter, G., Neuper, C., Pfurtfscheller, G. and Müller-Putz, G. (2008). Brain

Motor System Function in A Patient with Complete Spinal Cord Injury

Following Extensive Brain-Computer Interface Training. Exp Brain Res.

190,

215-223

15.

Bashashati, A., Fatourechi, M., Ward, R. K. and Birch, G. E. (2007). A

Survey of Signal Processing Algorithms in Brain-Computer Interfaces Based on Electrical Brain Signals. J. Neural Engineering.

4, 32-57

16.

Laureys, S., Tomoni, G., and Kubler, A. (2009). The Neurology of

Consciousness: Brain-Computer Interfaces for Communication in Paralysed

Patients and Implications for Disorders of Conciousness.

Elsevier. Ltd.

17.

Kubler, A., Kotchoubey, B., Hinterberger, T., Ghanayim, Nimr., Perelmouter,

J., Schauer, M., Fritsch, C., Taub, E. and Birbaumer, N. (1999). The Thought

55

Translation Device: A Neurophysiological Approach to Communication in

Total Motor Paralysis. Exp Brain Res.

124, 223–232

18.

Nakayama, K., Horita, H. and Hirano, Akihiro. (2008). Neural Network

Based BCI by Using Orthogonal Components of Multi-channel Brain Waves and Generalization. LNCS.

5164, 879-888

19.

Qin, J., Li, Y. and Sun, W. (2007). A Semisupervised Support Vector

Machines Algorithm for BCI Systems. Computational Intelligence and

Neuroscience.

2007, 1-9

20.

Hamadicharef, B. (2010). Brain-Computer Interface (BCI) Literature – A

Bibliometric Study. 10 th

International Conference on Information Science,

Signal Processing and their Applications.

May 10 – 13, 2010. Kuala Lumpur,

Malaysia: ISSPA. 2010. 626-629.

21.

Vidal, J. J. (1973). Toward Direct Brain–Computer Communication.

AnnualReview of Biophysics and Bioengineering.

2, 157–180

22.

Hwang, H. J., Kwon, K. and Im, C. H. (2009). Neurofeedback-Based Motor

Imagery Training for Brain-Computer Interface (BCI). Journal of

Neuroscience Methods.

179, 150-156

23.

Wilson, J. A., Schalk, G., Walton, L. M. and Williams, J. C. (2009). Using an

EEG-Based Brain-Computer Interface for Virtual Cursor Movement with

BCI2000. Journal of Visualized Experiments.

29, 1-4

24.

Lalor, E. C., Kelly, S. P., Finucane, C., Burke, R., Smith, R. Beilly, R. B. and

McDarby, G. (2005). Steady-state VEP-based Brain-Computer Interface

Control in An Immersive 3D Gaming Environment. EURASIP Journal on

Applied Signal Processing . 19, 3156-3164

25.

Finke, A., Lenhardt, A. and Ritter, H. (2009). The MindGame: A P300-based

Brain-Computer Interface Game. Neural Networks.

22, 1329-1333

56

26.

Bell, C. J., Shenoy, P. and Chalodhorn R. (2008). Control of a Humanoid

Robot by A Noninvasive Brain-Computer Interface in Humans. J. Neural

Engineering. 5, 214-220

27.

Hamadicharef, B., Xu, M. and Aditya, S. (2010). Brain-Computer Interface

(BCI) Based Musical Composition. Proceedings of the 2010 International

Conference on CYBERWORLDS (CW2010). 20 – 22 October. Singapore,

282-286

28.

Neshige, R., Murayama N., Igasaki, T, Tanoue, K., Kurokawa, H. and

Asayama, S. (2007). Communication Aid Device Utilizing Event-Related

Potentials for Patients with Severe Motor Impairment. Brain Research . 1141,

218-227

29.

Harmony, T., Fernandez, T., Silva, J., Bernal, J., Diaz-Comas, L., Reyes, A.,

Marosi, E., Rodriguez, M. and Rodriguez, M. (1996). EEG Delta Activity:

An Indicator of Attention to Internal Processing During Performance of

Mental Tasks. International Journal of Psychophysiology . 24, 161-171

30.

Fernandez, T., Harmony, T., Rodriguez, M., Bernal, J., Silva, J., Reyes, A. and Marosi, E. (1995). EEG Activation Patterns During the Performance of

Tasks Involving Different Components of Mental Calculation.

Electroencephalography and Clinical Neurophysiology . 94, 175-182

31.

Martin, G. N. (1998). Human Neuropsychology.

Prentice Hall, pp. 90

32.

Kandel, J., Schwartz, J. and Jessel, T. (1991). Principles of Neural Science.

Elsevier

33.

Gargiulo, G., Calvo, R. A., Bifulco, P., Cesarelli, M., Jin, C., Mohamed, A. and Schalk, A. V. (2010). A New Recording System for Passive Dry

Elecrodes. Clinical Neurophysiology.

121:686-693

APPENDIX A

CODING OF CURSOR MOVEMENT

!""# $$

% #!" !"

& '(

& )*

& ))

& # )+

, !"-.

,/0!"'

/!".1

2!" )3

&!".*

4%!"+-

51!"6 #77 8

5!"6 " 48

#

57

58

#9 /:

; %#<

;#<

;%#<

;#<

68

68

; #<

; #<

%

%

6

"8

=

> ?

>% ># ?+@

@

" # ##

6"8@

?

6" 6"

>8"###>% >#

6" >8

! 6" >88@

!#" #

" 6"8@

# A#

#!

6"8@

@

B

$

% 6

=

" >8

59

$ @ #" C0

% !?)@##$$

$& @

%

#!# #C0

#C0

>#?+@# #C0

>"?D/@"

>"?0E@ "#$F

F>"?GE@F

>?

6"8@

?

$ 6"!H8@

6;)8=

6 I $ :I8@

B

%! #C0

6 >8=

#% 6" ! 6"

>88@

B

% 6" 6"

>88@

=

#% 6" 6"

>88@

B

% 6" ! 6"

>88@

60

#"#% #C0

'() 6">#>"

>"F>"8@

#" #C0%$$

# 6"5/8@

@

B

6 !

JK8

=

>?)@

>? )@

9> @

9> @

!

J1)KJ1)K@

#*'+ 5LM@

!

J+11K@

) >@

>?6

) 8

6>??

-.'' 8=

6>N , 6 )

6I0 " >I8@

6 8@

88@

B

?6

) 8

6??

-.' D8=

6>N , 6 ) 88@

61

B

6I0 " I8@

6 8@

"F@

+/

"@

>@

,O,M4@

2"

% #

@ #"

% ">#">##@ ##

##"

% ###@### #

!

">?6I2L/PD&QI8@ #

"

$ @ C06 #8

> @

% %" #%!@

#

@ %

6 % R?*8=

6I;<;9> <

;9> <I8@

62

6 8@B

6ISI J K8@

9> ?

6 %/ J+K8@

9> ?6

%/ J-K8@

##

"?

0 6">8@

6"??

-.'' 8=

6 IS $S$:I

%/ J)K">8@

6 8@

B

>?

6"8@

">#?

!

6" >8@

">##?

1%!

6" >8@

#?6">#8@

##?6">##8@

6I#$S$@##$S$:I###8@

?

6"###))8@

* 6"T"P !8@

C06 #8 #

?

% 6")8@

6"58@

# $

> ?

6"88@

6"

63

# #

?

(- 6"

H 8@

I IH

6 ??)8=

6 I (-

$ $ :I8@

6 8@

B

?

(- 6" > I% I

H% H% 8@

6 ??)8=

6 I (-

$% $ :I8@

6 8@

B

?

(- 6" > I%IH%

H%8@

6 ??)8=

6 I (-

$%$ :I8@

6 8@

B

?

(- 6" > I"I

H"H"8@

64

6 ??)8=

6 I (-

$"$ :I8@

6 8@

B

?

(- 6" > I I

H H 8@

6 ??)8=

6 I (-

$ $ :I8@

6 8@

B

?

(- 6" > I#I

H#H#8@

6 ??)8=

6 I (-

$#$ :I8@

6 8@

B

?

(- 6" > I%!I

H%!H%!8@

6 ??)8=

6 I&M0

$%!$ :I8@

6 8@

B

6F?)@F;-@FNN8=

#% 6"%!8@

65

' 6" )##+

#69> +8##+8@

' 6"#+6##+8-

#+##-8@

#% 6"%!8@

#% 6"# )9>

6##9> 8+9>

9> 8@

#!

6"8@

#% 6"%8@

#% 6"6#9> 8+

6##9> 8+9>

9> 8@

#!

6"8@

6,O,M4!"!"??-.8=

6)8@

B

6 8@

6F??+8=

6I/4&U4I8@

#% 6"%8@

% 6"#+##'

688@BB

#% 6"#8@

#% 6"6#9> 8+

6##9> 8+9>

9> 8@

6"3)1*-1-)-)8@

#!

6"8@

=

66

6ISI8@

665LM?

6I I88??

68@

-.'' 8=

6 8@

B

6?)@;>@NN8=

65LMISIH>JK8@

B

65LM8@

B

J K?#+@

6? >@;>@NN8=

JK?>JKNJ K@

#% 6"#8@

#% 6"))##-

##-8@

6I!S?S IN JK8@

#% 6"% 8@

% 6"##'##'+68

6??)8

"?#N )@

688@

"?J K@

#% 6"#8@

#% 6""

67

6##9> 8+9> 9> 8@

#% 6"%!8@

#% 6"# )9>

6##9> 8+9> 9> 8@

#% 6"%!8@

' 6" )##+# )

##+8@

' 6"#+##+-

#+##-8@

#% 6"%8@

#% 6"JK6##

9> 8+9> 9> 8@

#!

6"8@

6 8@

=

6JK<?6#9> 8HH;>8=

6I/V00,,2RRI8@

#% 6"%!8@

% 6"6##+-8N ))

6##+-8N )) 688@

#% 6"#8@

#% 6""6##9> 8+

9> 9> 8@

#% 6"%!8@

#% 6"# )9>

6##9> 8+9>

9> 8@

#% 6"%!8@

' 6" )##+# )

##+8@

68

' 6"#+6##+8-

#+##-8@

#% 6"%8@

#% 6"JK

6##9> 8+9> 9> 8@

#!

6"8@

6 8@

B

6 8@

6<?> 8=

6I/7UUQ6I88@

#% 6"%!8@

% 6"

6# 688+6#+8N9>

6I

688@

?SI6# 688+8@

#% 6"#8@

#% 6""6##

9> 8+9> 9> 8@

#% 6"%!8@

#% 6"# )9>

6##9> 8+9>

9> 8@

#% 6"%!8@

' 6" )##+# )

##+8@

' 6"#+6##+8-

#+##-8@

#% 6"%8@

#% 6"JK6##

9> 8+9> 9> 8@

#!

6"8@

B

B

6*8@

B

6"8@

B

69

APPENDIX B

CODING TO DETERMINE PARAMETERS

5

+ A"

%

#"9#.??<# *

;#<

;%#<

;##<

#*'+ 57V4>P @

!

> J1)K@

F>>> >>+&4@

!%>%F@

) &@

) +>+>%@

>#?.>#? *@

#@

LM4,UO&D? )+*@

&4UL? '@

,,C>/4&U4? @

,,C>,M2? '@

/&PDLMC>5U,? )))@

) A%@

70

71

6 !

JK8

=

6 % R?+8=

6I ; <:I8@

6 8@

B

?

6 J K8@

%?6

) 8

6%??

-.'' 8=

6LM4,UO&D ,

6I%$I8@

6 8@

B

6 ) 88@

6&4?)@&4;LM4,UO&D@&4NN8=

6%N&48?6

) 8 6&4ULN , 6 ) 88@

66%N&48??

-.'' 8=

6I%$I8@

6 8@

B

B

A?6

) 8

6 A??

-.'' 8=

6LM4,UO&D ,

6I A$I8@

6 8@

6 ) 88@

B

6&4?)@&4;LM4,UO&D@&4NN8=

6 AN&48?6

) 8 6&4ULN , 6 ) 88@

66 AN&48??

-.'' 8=

6I A$I8@

6 8@

B

B

72

6> I ASI 8@

6657V4>P ?

6> I I88??

-.'' 8=

6IL$S "RR:I> 8@

6 8@

B

!

6 657V4>P IS:S:S:S:S:S:S:S:

S:S:S:S:S:S:S:S:S:S:S:S

:IH%J>KJ)KH AJ>KJ KH AJ>KJ+KH AJ>

KJ-KH AJ>KJ*KH AJ>KJ1KH AJ>KJ.KH AJ>

KJ3KH AJ>KJ(KH AJ>KJ'KH AJ>KJ )KH AJ

>KJ KH AJ>KJ +KH AJ>KJ -KH AJ>KJ *KH

AJ>KJ 1KH AJ>KJ .KH AJ>KJ 3KH AJ>KJ (K

H AJ>KJ 'K8R?

+0# 8=

>NN@

>> ?>@

B

657V4>P 8@

6IS:I>> 8@

6IS:I> 8@

6IS:I AJ KJ K8@

73

#

?&?)@

6F? >#@F;>#N @FNN8=

6?)@;>> @NN8=

+>? AJKJFK@

+>%?%JKJ)K@

6;+>8=

?+>@

#?F@

&?+>%@

B

B

B

6I S?S0#S%?S:I

#&8@

6I#.?5-:#3?0-:#(?P-:#'?59:# )?09:# ?P9:# +

?5*:# -?0*:# *?P*:I8@

6I%??< +#-#*%

1.##:I8@

B

74

APPENDIX C

LIST OF PUBLICATIONS

INTERNATIONAL JOURNAL

1.

Siti Zuraimi Salleh, Norlaili Mat Safri and Siti Hajar Aminah Ali, “Moving

One Dimensional Cursor Using Extracted Parameter from Brain Signals” .

Signal Processing: An International Journal (SPIJ), Vol. 3, No. 5, pp. 110-

119. September/October 2009

INTERNATIONAL CONFERENCE

1.

Siti Zuraimi Salleh, Norlaili Mat Safri and Siti Hajar Aminah Ali, “Feature

Extraction and Translation of Electroencephalogram Signals in Humans for

Noninvasive Brain-Computer Interface” . International Conferece on Control,

Instrumentation and Mechatronic Engineering (CIM2009), 2-3 June 2009,

Melaka, Malaysia. June 2009

2.

Norlaili Mat Safri, Siti Hajar Aminah Ali, Siti Zuraimi Salleh, Nobuki

Murayama, “Modeling Information Pathway of Motor Control Using

Coherence Analysis” . Second Asia International Conference on Modeling

Symposium (AMS2008), pp. 917-922, 13-15 May 2008, Kuala Lumpur,

Malaysia. May 2008

NATIONAL CONFERENCE

Siti Zuraimi Salleh, Norlaili Mat Safri and Siti Hajar Aminah Ali,

“Parameter Extraction for Control of a One Dimensional Movement by

Brain Signal”.

Proceedings of 2008 Student Conference on Research and

Development (SCOReD 2008), 26-27 Nov. 2008, Johor, Malaysia. November

2008

Download