Toward Brain-Computer Interfacing
2010.10.5.
이정수
Contents
 I. BCI Systems and Approaches

Introduction (p.27~30)

4.Graz-Brain-Computer Interface: State of Research (p.65~84)

6. The IDIAP Brain-Computer Interface: An Asynchronous
Multiclass Approach (p.103~110)
2
Introduction
 Two paradigms of BCI study


Active paradigm (chap. 2,3,7)
 Active & voluntary strategy for generating a specific
regulation of and EEG parameter
 Motor-related μ-rhythm
 self-regulation of slow cortical potentials(SCP)
Passive paradigm (chap. 2,4)
 Participants only have to view an item for selection
 Evoked responses such as P300
 Steady-state evoked potentials(SSVEP)
 “Let the machines learn”, Berlin group (chap. 4,5,6)



Learning is done by the computer not human
The subject will inevitably learn once feedback has started
Both aspect : subject & machine training
3
Introduction(cont.)
 Albany BCI(chap.2)



A user is trained to manipulate this μ and β rhythms
To control a cursor in 1- or 2D
BCI control based on the P300
 Tϋbingen BCI(chap.3)



Train subjects to adapt to the system using SCP
Mean for communication of ALS patients with the world
P300, μ-rhythm-based BCIs, auditory stimulation, ECoG
 Graz BCI(chap.4)

Whole BCI field : sensors, feedback strategies, cognitive
aspects, novel signal processing methods, with excellent
results
4
Introduction(cont.)
 Berlin BCI(chap.5)



Only 30minutes for training subjects rather than weeks or
months
Advanced machine learning and signal processing technology
Online feedback studies
 Martigny BCI(chap.6)



Similar to the Berlin approach
Machine learning rather than subjects training
Online adaptation to realize a BCI
 Vancouver BCI(chap.7)


Asynchronous BCI for patients
Machine learning techniques
5
4. Graz-Brain-Computer Interface
 Abstract



BCI
 Signals from the human brain -> commands of control
devices or application
 Basic communication capabilities for severe neuromuscular
disorders
Graz-BCI system
 Oscillations of β or μ rhythms, SSVEP, SSSEP
 The use of complex band power features
 The selection of important features
 Phase-coupling & adaptive autoregressive parameter
estimation to improve single-trial classification
Control of neuroprosthses, spelling system, asynchronous BCI
6
Components of Graz-BCI
7
Components of Graz-BCI
 Graz-BCI


Noninvasive EEG from the scalp
ECoG recorded during self-paced movements



To detect the motor action in single trials
Dynamic oscillations of β or μ rhythms, SSVEP, SSSEP
Three mental strategies
 Operant conditioning


Birbaumer’s lab in Tϋbingen
Predefined mental task


Moto imagery(left or right hand, both feet, tongue)
Similar cortical areas activation & temporal characteristics
Attention to an externally paced stimulus
Feedback
 Delayed(discrete) FB : correct or incorrect at the end
 Continuous FB : indicates immediately


8
Graz-BCI Control with Motor Imagery
 EEG signals from the sensorimotor cortex





Desynchronization of β or μ rhythms at the time of movement
onset
Reappearance when the movement is complete
Quantification of temporal-spatial ERD and ERS, motor
imagery can induce different types of activation patterns
 Desynchroniztion(ERD) of sensorimotor rhythms
 Synchronization (ERS) of u rhythms
 Short-lasting synchronization of central B oscillation after
termination of motor imagery
Imagery related brain activity necessary to BCI, but subject
with no changes in EEG
Diversity of time-frequency pattern reported
9
Graz-BCI Control with Motor Imagery
10
Adaptive Autoregressive(AAR) Parameters
 Spectral properties of EEG are useful feature for BCI
 Due to use of FFT, feature extraction was block-based and
no feedback in continuous time
 Other methods for spectral estimation

Autoregressive model, stationary estimators, adaptive
estimation algorithms(LMS, RLS, Kalman filtering)
 Adaptively estimated AAR parameters



Obtained with a time-resolution as high as the sampling rate
Possible to provide continuous feedback in real time
Neural network -> linear discrimant analysis(LDA)
 Simple and fast training procedure
 Provide a continuous discrimination function
 Became the standard classifier for AAR
11
Complex Band Power Features
 Bandpower features : important for classification of brain
pattern
 Squaring the values of the samples and then smoothing the
result in the time domain
 FFT in frequency domain, yield phase information
CBP(complex bandpower) feature
 To test importance of phase




CBP vs CSP(common spatial patterns)
CBP is superior to CSP
Require fewer electrode
Less training data than CSP
 Phase information is an important and useful feature in BCI
12
Phase Synchronization Features
 Almost all BCIs ignore the relationships btw EEG signals
measured at different electrode recording sites
 Logarithmic bandpower feature or adaptive autoregressive
parameter
 Quantifying the relationships among the signals of single
electrodes
 PLV (phase locking value)


PVL value of 1 : two channels are highly synchronized
PVL value of 0 : no phase synchronization
13
Adaptive Classifier
 To automatically adapt changes in the EEG patterns of the
subject
 To deal with their long-term variation
 ADIM

Estimate online the Information Matrix to compute an
adaptive version of the QDA
 ALDA

Adaptive linear discriminant analysis based Kalman filtering
 Experiments



AAR features and ADIM
BP estimation
Concatenation of AAR & BP combined with ALDA
 More details in chapter 18
14
Importance of Feature Selection
 Many feature extraction methods for BCI

Bandpower :extracts features for specific frequency ranges
 Filter methods

E.g., Fisher distance,r2
 Wrapper methods


Flexible & generally applicable but computationally demanding
E.g., genetic algorithms(to find suitable wavelet features in
ECoG data), heuristic search strategies
 So-called embedded algorithms


E.g., linear programming,
DSLVQ(Distinction sensitive learning vector quantization)
 electrode position and frequency components selection
15
Steady-State Evoked Potentials
 SSVEP,SSSEP



Both sensory respond “resonance-like” frequency regions
Visual system
 Near 10Hz : greatest SSVEP amplitude
 1618Hz : medium amplitude
 Near 40-50Hz : smallest amplitude
Somatosensory
 around 27Hz in EEG β range
 SSVEP

32LED bars (4X8), 6,7,8,13Hz modulation
 SSSEP

Stimulation frequency 25-31Hz and 20-26Hz
16
Graz-BCI Applications
 Control of Neuroprostheses



BCI for paralyzed limbs to restore their grasp
Functional electrical stimuliation(FES)
Two male with high spinal cord injury(SCI)
 One patient (30 years)




Left hand grasp function restored with FES
After 4 month training, learned to induce 17Hz oscillations
A drinking glass
The other patient (42 years)


Freehand system implanted in his right hand and arm
Only Three days training, power decrease of EEG amplitude
during left hand movement imagination
17
Graz-BCI Applications(Cont.)
 Control of a Spelling Application





A 60-year-old male patient with ALS
To enable the patient operate the cue-based two-class “virtual
keyboard”
Artificial ventilated, totally paralyzed
Two bipolar EEG channel from four gold electrodes placed
over the left and right sensorimotor area(C3, C4)
Cue-based motor imagery trials
18
Graz-BCI Applications(Cont.)
 Uncued Navigation in a Virtual Environment

Three bipolar EEG channels
19
6. IDIAP Brain-Computer Interface
 Portable BCI system based on the online analysis of
spontaneous EEG signals with scalp electrodes
 Relies on an asynchronous protocol
 Variation of EEG over several cortical areas related to
imagination of movements, arithmetic operations, or
language
 To discover task-specific spatiofrequency pattern embedded
in the continuous EEG signal
 Able to recognize three mental tasks with a statistical
Gaussian classifier
 Virtual keyboard, video game, mobile robot
20
Operant Conditioning & Machine Learning
 Subjects training



learn to control their brain activity
appropriate, but lengthy training
to generate fixed EEG patterns that the BCI transforms into
external actions
 Machine learning approach


Based on a mutual learning process where the user and the
brain interface are coupled together and adapt to each other
accelerate the training time (few hours of training)
 Rejection criteria to avoid making risky decisions


A low classification error is a critical performance criterion
Bayesian techniques for rejection purposes
21
Synchronous & Asynchronous BCI
 Synchronous BCI




EEG is time-locked to externally paced cues repeated every 410s
limited by a low channel capacity (below 0.5bits/s)
Facilitate EEG analysis
 the starting time of mental states are precisely known
 differences with respect to background EEG activity can be
amplified
Normally recognize only two mental states
 Asynchronous BCI




Subjects make self-paced decisions
Response time is below 1s, channel capacity(1 ~ 1.5 bits/s)
To steer a wheelchair, BCI delivers rapid & accurate command
“idle” states : no particular mental task
 Giving “no” response without explicit training of classifier
22
Spatial Filtering
 EEG signal : poor SNR & spatial resolution
 Surface laplacian(SL) derivation




Normally 64-128 electrodes
Better cortical activity below electrodes immediately
Global method
 Raw EEG potential interpolated using spherical splines
 Second spatial derivative is taken
Local method
 The average activity of neighboring electrodes(normally 4)
is subtracted from the electrode of interest
 Common average reference(CAR)


Raw EEG potentials is transformed to CAR
Remove the average activity of all the electrodes
 Other spatial filtering algorithms(chap 13)
23
Experimental Protocol
 Users select three mental tasks

“relax”, “imagination of left and right hand movements”,
“cube rotation”, “subtraction”, “word association”
 In training session





Subjects is seated & performed selected task for 10-15s
Consecutive training trials lasting about 5min and breaks
5~10m (repeated normally 4)
Time-shift btw starts performing & label for the next task
Acquired EEG data is not time-locked to the events
Feedback through three buttons on the screen
24
Experimental Protocol(Cont.)
 Signal acquisition & signal processing




EEG with 8~64 electrode
Raw EEG potential is transformed using SL
Extract relevant features from a few channels(8~15) and
corresponding vector is used as input to the classifier
the Welch periodogram algorithm to estimate the power
spectrum of each selected SL-transformed channel




averaging three 0.5-s segments with 50 percent overlap(2 Hz)
Normalization of the frequency band 8–30 Hz
The resulting EEG sample is analyzed by the statistical
classifier.
No artifact rejection algorithm is applied
 Statistical Gaussian Classifier

More details in chapter 16
25
Brain-Actuated Prototypes
 BCIs is used for brain-actuated applications
 A virtual keyboard on a computer screen






Whole keyboard(26 English letter & space key), 3 X 9
Divided in three blocks, each associated to one mental task
Same colors as the training phase
Three decision steps to write a single letter
3.5s waits to undo in case of wrong selection
Takes 22.0s for trained subject to select a letter
 Brain game


Classical Pac-man game
Two mental task, turn left or turn right
 Computer cursor in 2D

Trained subjects to control two independent EEG rhythms
26
Virtual Keyboard



RED : “cube rotation”
Yellow : “subtraction”
Green : “word association”
27
Brain-actuated Prototypes
 Mobile robot - a motorized wheelchair



Millan et al(2004), first asynchronous analysis of EEG signal
Fast and frequent switches are required
A key




subject can issue high-level commands at any moment


shared control between two intelligent agents :the human user
and the robot
user gives high-level mental commands that the robot performs
autonomously, “turn right at the next occasion”
robot executes these commands autonomously using the readings
of its on-board sensors
Asynchronous , not requiring waiting for external cues
The robot relies on a behavior-based controller


to implement the high-level commands to guarantee obstacle
avoidance and smooth turns
on-board sensors are read constantly and determine the next
action
28
Discussion
 Real-time control of robots and neuroprostheses is most
challenging application of BCI
 1. More powerful adaptive shared autonomy framework for
the cooperation of the human user & the robot
 2. Better electrical activity all across the brain with high
spatial accuracy using noninvasive scalp EEG



Local field potentials(LFP): electrical activity of small groups
of neurons
Estimated LFP has the potential to unravel scalp EEG
Grave de P Menedez(2005), more details in chapter 16
 3. To improve the robustness of a BCI

More details in chapter 18
 4. analysis of neural correlates of high-level cognitive
states


Errors, alarms, attention, frustration, confusion
More details in chapter 17
29