Brain-Computer Interfacing for
Rehabilitative Applications
Amit Konar
Professor, Dept. of Electronics and Tele-Communication Engg.
School of Cognitive Science and
M.Tech. Course in Intelligent Automation and Robotics
Jadavpur University
NCBC-2013
What is Brain-Computer Interfacing?
Decoding of
Brain States
Brain
Generating Stimulus to
excite the Brain
Stimulus type:
Audio/Video/Tactile/Smell
Decoding of:
sensations/thoughts/actions
Using BCI for Rehabilitative Applications
Decoding of
Brain States
Brain
Motor
Intensions
Senses
Positional
Error
Generates control
command to
execute motor
actions. Here,
position control of
the Prosthetic
Device (Artificial
Limb)
How to Measure Brain Excitations
EEG
Intra-cortical
electrodes
f-MRI
ECoG
MEG
f-NIRS
What is Electroencephalogram (EEG)?
The neuronal firing inside the brain generates electrical signals.
These electrical signals picked up from the scalp by metallic
electrodes are called EEG signals.
Note: If the electrical signals are acquired from electrodes placed
inside the cortex, they are called Electrocorticographs (ECoG).
Merits and Demerits of Using EEG in BCI
Merits of EEC Based BCI
Demerits of EEG Based BCI
1. Non-invasive
1. Poor Spatial Resolution
2. Superior temporal
resolution
2. High probability of noise pickup from neighborhood
channels/Eye Blinking/Power
supply frequency
3. Cost effective
4. Portable
3. Source Localization is not easy
Frequency Bands of an EEG Signal
Waveform
Freq(Hz)
Alpha rhythm
8-12
Beta rhythm
13-30
Delta
1-4
Theta
4-7
Sleep spindle
12-14
Lambda
Transient
Mu Rhythm
8-12
Basic Steps in Brain-Computer Interfacing
EEG Machine
EEG
EEG
Amplifier
+ADC
Signal Preprocessing
Stimulus/
Cognitive
Input
Classes
Feature
Extraction
Classification
of Brain States
or Intensions
Computer
Applications: Emotion/Cognitive Load
Classification by EEG Analysis
Existing Scheme of Positin Control of an Artificial Limb
Using EEG
Artificial
limb
error
EEG
C3, C4
Motor Intension
error
Up/down
Stepper Motor Driven open
loop position control
Close/open finger
Decoding
Motor
Lift up forearm
intensions
Pattern Classifiers
Feature
Extraction
Amplifier
Where is the Novelty in our Research?
Current position Target position
a) Initial situation
Current position Target position
Target position
Current position
c) Error-Prone situation
Error EEG
time
Channel Cz
b) Error-Prone situation
d) The error EEG (ErRP)
for (b) and (c)
Decoding Rotational Error in EEG
Current Angle
Target Angle
a) Initial Configuration
Target Angle
Current Angle
c) Error-Prone Situation
Error EEG
Current Angle
Target Angle
b) Error-Prone Situation
Channel Cz
d) The error EEG (ErRP)
for (b) and (c)
What to do with the Decoded Error Signal?
Reference Position

+
Controller
Error
Controlled
Position
Motor driven
Robot Arm
Feedback position
Error
Sensing
by Eye
Position sensor
Error
Decoding
from EEG
Robotic System
Y
Error?
N
Turn Clockwise
Turn Counter-Clockwise
Controlling
Error by
Motor
Intension
Intension
Classification
Techniques Used and Results
1. List of Extracted Features: Hzorth Parameters and
Power Spectral Density (at integer frequencies in (8-12)
and (16-24) Hz Bands
2. Classifier Used: Support Vector Machine
3. Steady-state Positional Error: 0.2%
4. Settling Time: 18 Seconds
5. Testing performed on: Dell W/S with 16GB RAM, Octa
core Intel Xeon processor
Where are the Difficulties?
We cannot measure the magnitude/sign of
error from the EEG, but can only detect
the existence of error.
The Proposed 2DF Robot Arm Manipulator
Motor M2
Motor M1
Aligning the 2D Arm with the Goal
Goal
Motor
Positioning of the Gripper towards the Goal
Motor M2
Goal
The Complete Limb Position Control Scheme
Clockwise/Move Forward
C4
Signal
Conditioning
Feature
Extraction
Cz
C3
Motor Intension
Classifier
Motor
Driver Logic
C-clockwise/ No Movement
Error
Classifier
First Rotational Error
Later Positional Error
Continue displacement until error
Motor M2
Motor M1
Continue turning until error.
A Pseudo Code for Position Control
While motor intension is turning
If no (rotational) Zero-Error occurs
Continue turning
Else stop turning;
End While; Adjust rotational offset;
While motor intension is displacement
If no (positional) Zero-Error occurs
Continue displacement
Else stop moving;
End-while; Adjust translation offset;
Copying Motor Planning by Artificial Robotic Arm
1
55
2
5
4
4
Grip
6
Grip
6
2
3
1
3
Jaco Robot Arm
The subject plans movement of his hand, the robot executes the plan
on its arm synonymously. Thus the robot copies the plan of action
of the subject.
Motor Translation by Copying Plans
1
EEG
2
C3,
C4,
Cz
5
5
5
4
4
Grip
6
RA
3
FA
Body C/CC/NM
Grip
6
2
Encoder
RA U/Dn/NA
Encoder
FA U/Dn/NA
Encoder
FA T/NT
Classifier
Encoder
1
3
Results of Classifier Performance
Planer Robot
Classifier
BP
SVM
Training
Time
22 minutes
8 minutes
Average Classification
Accuracy
78.6%
97.2%
Jaco Robot Arm
Classifier
BP
SVM
Training
Average Classification
Time
Accuracy
30 minutes
65.2%
12 minutes
92.4%
Relative Performance on Features
For the Planer Robot
Features
Classifier Training Average
Time
Classification
Accuracy
Hzorth
Parameters and
PSD
BP
22 minutes
78.6%
Wavelet Coeffs.
and PSD
BP
31 minutes
76.6%
Hzorth
Parameters and
PSD
SVM
8 minutes
97.2%
Wavelet Coeffs.
and PSD
SVM
15 minutes
82.7%
Cognitive Failure Detection for Fatigued
Car-Driver
Motor Cortex
Electrodes
Driver
Video
Stimulus
Change
Motor Imagination
failure Detection
Alertness
Detection
EMG
Occipital
Electrodes
Motor Execution
Failure Detection
Automatic
Alarming
Attention Testing While Driving
Noise
Filtering
Classification by
Thresholding
Visual stimulus about
traffic condition
Occipital
Electrodes
SSVEP
Visual
Stimulus
Time
Attentive
NonAttentive
Signal sweeps
over 30% of
Average value
(i.e., threshold)
Motor Intension Classification in Driving
C3 C4
Noise
Filtering
Feature
Extraction
Motor Intension
Classifier
Motor
Cortex
Steering
Classifier
C/CC/NA
•Steering
Acc.
Classif.
Brake
Classif
Acc./No Brake/No
Acc.
Brake
•Accelerator
+ Clutch
•Clutch +
Brake
•NA
Parallel two level classifiers to recognize motor intensions
Results of Motor Intention Classification
Modality
Classification
Technique
Average
Classification
Accuracy
Motor Intension
Classification
SVM
88.7%
-DO-
Naïve Bayes
91.2%
-DO-
BP
72.4%
Modality
Classification
Technique
Average
Classification
Accuracy
Steering
SVM/NB/BP
88.5%
94% 72%
Accelerator
-DO-
84.2% 91.2% 70%
Braking
-DO-
96% 71.4% 75.1%
Conclusions and Future Directions
1. EEG has been used as a BCI equipment for its i)non-invasive
nature, ii)good temporal resolution and iii) relative
inexpensiveness.
2. An EEG driven artificial limb position control system has been
developed, which responds spontaneously (response time < 32
seconds) after task planning is performed.
3. The car driving system helps the driver by alarming in three
ways: i)attention failure, ii) motor imagination failure, iii) motor
execution failure. The alarm generation time required is <6
seconds for all the three cases.
Conclusions and Future Directions
(Contd.)
4. The proposed artificial arm would be extended to have the ability
of haptic perception like normal human beings. We already have
developed such a system, and would like to integrate it with the
artificial limb.
5. To improve the speed of execution, we started exploring the
possible parallelism in the classification and signal processing
algorithms. The parallelisms, if fully exploited, could reduce the
speed of execution significantly.
6. A (low cost) fNIRS system if integrated with EEG, could improve
the classification accuracy because of good spatial and temporal
resolution of the overall system.
Bibliography for Further Reading
Text/Monographs
1. Brain-Computer Interfaces by Bernhard Graimann,
Brendan Allison and Gert Prefurtsheller, Springer, 2010.
2. Computational Intelligence: Principles, Techniques and
Applications by Amit Konar, Springer, 2006.
3. Human-Computer Interfaces for Rehabilitative Applications
by Amit Konar et al., Springer (2013) (to appear).
Bibliography (Continuation)
Papers
1.
Bhattacharyya S., Khasnobish A., Konar A., Tibarewala D.N. & Nagar
A.K., “Performance analysis of Left/Right Hand movement classification
from EEG signal by intelligent algorithm”, IEEE Symposium Series on
Computational Intelligence, Paris, France, Apr 11-15, 2011.
2.
Khasnobish A., Bhattacharyya S., Konar A., Tibarewala D.N, Nagar A.K.,
“A Two-fold classification for composite decision about localized arm
movement from EEG by SVM and QDA techniques,” Int. Joint Conf. Neural
Net. (IJCNN 2011): pp.1344-1351.
3.
Pfurtsheller, G. et al., The Graz-BCI: State of the Art and Clinical
Applications,IEEE Trans. On Neural Systems Rehabilitation Engineering,
June 11, pp. 177-180, 2003.
Thank You