Brain Computer Interface Mohamed Sami Mai Mohamed Project Team Nada Mohamed Ahmed Mamdoh http://bci2.k-space.org Mohamed Omar Brain Computer Interface BCI Supervisors Prof.Dr Abu Bakr M. Youssef http://bci2.k-space.org Assistant Prof.Dr Yasser M.Kadah Motivation for BCI Research There are , more than 200,000 patients live with the motor sequelae of serious injury. Locked-in Syndrome Neurological diseases may lead to paralysis of the entire motor system . Unable to use their muscles and therefore cannot communicate their needs, wishes, and emotions. http://bci2.k-space.org Reason of BCI • Allow a user to communicate with a computer through his Brain • The user can think and the computer recognizes what he thought about. • This is what we call a Brain-Computer Interface (BCI) [or Brain-Machine Interface (BMI)]. http://bci2.k-space.org The Dream •People always think about controlling environments from their mind •Anyone wish if he could read the people thoughts and know what they are thinking of him •Some people want to store their dreams and record it while they sleeping http://bci2.k-space.org Dream vs. Reality • Dream BCI – Think to whatever you want – Without recognition errors – Whenever you want • Physiological problems – No thought sensor – Partial brain knowledge – Noisy signals • Solutions in the BCI community (reality) – Limited thought – Limited recognition accuracy http://bci2.k-space.org BCI community About 60 research groups About 300 researchers Increasing published papers 140 127 120 100 80 SCI paper 60 48 40 20 0 2 4 1985-1990 1991-1995 1996-2000 2001-2004 http://bci2.k-space.org Our Goals 1. Recording Brain Signal Using EEG electrodes. 2. Isolation between subject and electronic circuit 3. Designing Data Acquisition System 4. Signal Selection 5. Interfacing with Computer by Soundcard 6. Implementing real time analysis Classification data BCI Categories • Invasive and Non-Invasive BCIs • Online and Offline BCIs • Imaginary and Mental Tasks http://bci2.k-space.org General scheme 3. Online Feedback High level commands Feedback Control interface Application Mental state Electrical activity Biosensor The brain 1. Data Acquisition http://bci2.k-space.org Preprocessing On the computer 2. BCI System Feature extraction Classification Medical Introduction Nervous System Motor Brain CNS Spinal Cord Nervous System PNS Sensory http://bci2.k-space.org Cranial Nerves Spinal Nerves Human Brain http://bci2.k-space.org EEG Electroencephalography or EEG is the measurement of neural activity within the brain. EEG has been used to detect low oxygen and high carbon dioxide levels. A clinical use of EEG is in the diagnosis of epilepsy. http://bci2.k-space.org EEG Signal http://bci2.k-space.org EEG Wave Band Alpha Beta Delta Theta 8-13 Hz 13-30 Hz 0.5-4 Hz 4-8 Hz Occupation occipital parietal and frontal lobes. Condition awake person Frequency Age ______ ______ _____ Sleeping _____ ______ infants &adults children and sleeping adults EEG Lead System http://bci2.k-space.org Data Acquisition http://bci2.k-space.org Overview Electrode Isolation Pre Amplifier Electrode Isolation Pre Amplifier Electrode Isolation Pre Amplifier MUX Electrode Isolation Pre Amplifier Latch Electrode Isolation Pre Amplifier Parallel Port Electrode Isolation Pre Amplifier http://bci2.k-space.org Gain Amplifier LPF Sound Card Matlab Workspace Biopotential Sensors Electrodes are Biopotential sensor. There are different types of electrodes: 1. Gold electrode. 2. Silver electrode. http://bci2.k-space.org Isolation Medical procedures usually expose the patient to more hazard than at home or workplace. Our main goal is to break ground loop . We decide to do that by low cast and effective way by using: 1.Isolation transformer as power isolation. 2.Opto-Couplers as signal isolation. http://bci2.k-space.org Continue Isolation In our design we used the PC817 due to: Its low turn-on and off time and high. Isolation voltage between input. http://bci2.k-space.org Instrumentation Amplifier Amplifying differential input There are two stage of signal amplification: 1.Pre-Amplification 2.Gain-Amplification We used AD620 according to many better features on it: • • • • Lower cost High accuracy Low noise High Gain Ability http://bci2.k-space.org AD620 Schematic http://bci2.k-space.org Signal Selection Multiplexer: Select data from two or more data sources into a single channel. There are two types of multiplexers: •Analog Multiplexer. •Digital Multiplexer. we used Analog Multiplexer and we choose M54HC4051 IC Some features of M54HC4051: • Low power dissipation • Fast switching • High noise immunity • Wide analog input voltage range http://bci2.k-space.org M54HC4051 Schematic http://bci2.k-space.org Latch Change output state only in response to data input Transfer data from parallel port to MUX and holding it using LE (latch enable). In our design SN74LS373 As latch IC. http://bci2.k-space.org SN74LS373 Schematic http://bci2.k-space.org Signal Filtering A low-pass filter is a Filter that passes low frequensy Component well, reduces frequencies higher than the cutoff frequency. It is sometimes called a high-cut filter, or treble cut filter. We use active 2nd order low pass filter we used UA741 IC. LPF Schematic http://bci2.k-space.org Parallel Port The Parallel Port is the most commonly used port for interfacing home made projects •Hardware Properties 8 output pins accessed via the DATA Port 5 input pins (one inverted) accessed via the STATUS Port 4 output pins (three inverted) accessed via the CONTROL Port The remaining 8 pins are grounded •Why Parallel Port ? Easy Implementation and Installation Allow Full Software Control withoutneed any Counters & Clock to Switch between Channels Ability of Communication with Matlab http://bci2.k-space.org Sound Card A sound card is a Computer PCI Card that can input and output Sound under control of computer programs General characteristics 1-Sound Chip 2- multi-channel Dacs & A/D 3-ROM or Flash memory Color Function Lime Analog line level output for the main stereo green signal (front speakers or headphones). Pink Analog Microphone input. Light Analog Line level input. blue http://bci2.k-space.org Continue Sound Card Internal Block Why we choose Sound Card • Fixed and Low Cost Acquisition Card • Easy in Implementation and installation • Ability to Convert from Analog to Digital with very high accuracy and vise versa • Easy Communication with Matlab • Ability to detect low Frequencies • Sampling Data in wide rang (8000 to 44100) • Better than designing new Interfacing System and this System in Situation to not work because of hardware troubleshooting http://bci2.k-space.org Acquiring Data with a Sound Card http://bci2.k-space.org Data Acquisition Programs 1st Release Online Signal with Filtering http://bci2.k-space.org 2nd Release Each Channel Online Plotting 3rd Release Drawing With Selection 4th Release Online Classifier 1st Release http://bci2.k-space.org 2nd Release http://bci2.k-space.org 3rd Release http://bci2.k-space.org 4th Release imagination of right hand movement imagination of left hand movement http://bci2.k-space.org Analytic methods The process of EEG signal analysis and classification consists of the Following three steps: Signal preprocessing http://bci2.k-space.org Feature extraction Statistical classification Signal Preprocessing Noisy signal ? Experiment protocol Background brain activity • Eye movements • Other movements Physiologic noise Mental task Environmental noise Measured signal Eye blink Subject • Power line 50/60 Hz • Electrode contact Heart rate Power line Electrode contact Start Offline Type of work Online Work on exist dataset Record our dataset Record EEG signal Read dataset Feature extraction feature extraction Feature extraction Make hypothesis test classification hypothesis test Classification Decision F Feature available Visual O/P Test next feature T Classification Test classifier http://bci2.k-space.org Red Green Offline Dataset BCI Competition 2003 Data Set Ia: ‹self-regulation of SCPs› provided by University of Tuebingen,Germany, Dept. of Computer Engineering (Prof. Rosenstiel) Datasets were taken from a healthy subject he was asked to move a cursor up and down on a computer screen. Data 6 EEG electrodes are used referenced to the vertex electrode Cz •Channel 1: A1-Cz (A1 = left mastoid) •Channel 2: A2-Cz (right mastoid) •Channel 3: 2 cm frontal of C3 •Channel 4: 2 cm parietal of C3 •Channel 5: 2 cm frontal of C4 •Channel 6: 2 cm parietal of C4 Sampling rate of 256 Hz. Trial structure overview consisted of three phases 1-s rest phase,1.5-s cue presentation phase and 3.5-s feedback phase. http://bci2.k-space.org Continue Offline Dataset During every trial, the task was visually presented by a highlighted goal at the top or bottom of the screen to indicate negativity or positively from second 0.5 until the end of the trial. The visual feedback was presented from second 2 to second 5.5. Only this 3.5 second interval of every trial is provided for training and testing. Trails separated into training set (268 trials) which is 2-D Matrices 135x5377 and 133x5377 testing set The test set (293 trials). Every line of a matrix contains the data of one trial. The first column codes the class of the trial (0/1). •Note For our implementation we constructed the test set from the train set. That was done by selecting 100 trails from class 0 and 100 trails from class 1. http://bci2.k-space.org Continue Offline Dataset Approach We used MATLAB (release 13) for analysis. We separated the channels of each class to be 135x896 matrix for channels in class 0 and 133x896 matrix for class 1 channels. For each EEG channel, we plotted the time-domain and frequency-domain averages across trials for each class. Note In our online BCI approach, we constructed our own dataset which consist of training set & testing set. The training set was used to tune the parameters of the classification algorithm. We also applied all the pre-processing techniques as in the offline work. http://bci2.k-space.org Feature Extraction Steps of feature extraction Choosing feature Features Vector Form http://bci2.k-space.org Choosing Features Time Domain Features mean Variance http://bci2.k-space.org Continue Frequency domain features Short-Time Fourier Transform –First we transform all signals to frequency domain by (FFT). –Then we get mean & variance in frequency domain . –calculate the amplitudes at 20 Hz. Welch method Estimate the power spectral density (PSD) of a signal using Welch is done using Pwelch Matlab function Form features vector Channel 1 Class 0 Class 1 http://bci2.k-space.org Signal 1 Signal2 Signal3 Signal4 : : : : : : : feature vector Class 0 Class 1 Std1 Std2 Std3 Std4 : : : : : : : : : Continue Form Features Vector Ch1 Feature Vector Class 0 Class 1 http://bci2.k-space.org mean1 Var1 mean2 Var2 mean3 Var3 mean4 Var4 . . . . Multi dimension feature vector Channel 1 Channel 6 Channel 2 Channel 5 http://bci2.k-space.org Hypothesis Test Perform Hypothesis testing for the difference in means of two samples. [H, P, Ci]=ttest2(X,Y) H=0 no significance H=1 significance http://bci2.k-space.org Signal Classification Techniques Classifier Minimum Distance http://bci2.k-space.org Bayes K-NN Classifier input Train feature vector Class 0 Class 1 Test feature vector Class 0 Class 1 Minimum Distance Classifier Algorithm 1. Group the design set into (n) class 2. Estimate the sample mean for each class. 3. A test sample is classified by assigning it to the class which has the nearest mean vector. 4. Error rate is estimated by the percentage of misclassified samples http://bci2.k-space.org Bayes Classifier Algorithm Compute Gaussian distribution of each class (p.d.f) Compute probabilities of sample (a) F( a Є f0) & F( a Є f1) http://bci2.k-space.org K-Nearest Neighbor (KNN) Algorithm 1. Obtain distances between test sample and all samples in the design set 2. Sort obtained distance values in ascending ordered array. 3. Assigns the test sample to the majority class in the subset. 4. Error rate is estimated by the percentage of misclassified samples Results Dataset Ia results Our dataset results http://bci2.k-space.org Dataset Ia Best results FFT feature (amplitude of 20 HZ) KNN k=3 Accuracy Error channel 3 80% 20% KNN k=5 Accuracy Error channel 4 78% 22% Pwelch feature http://bci2.k-space.org Our Dataset Best Results FFT feature (amplitude of 20 HZ) KNN k=3 Accuracy Error channel 3 54% 46% KNN k=5 Accuracy Error channel 4 58% 42% Pwelch feature http://bci2.k-space.org BCI challenge Information transfer rate. High error rate. Autonomy. Cognitive load. http://bci2.k-space.org Conclusion & Future • In our project we built a simple BCI ,which separated between left and right hand movement • System worked on online & offline data set • Online data pass through different stages: Filtration Amplification Interfacing with computer using soundcard Analysis and classify http://bci2.k-space.org Conclusion & Future Completely paralyzed patients can use a BCI to realize a spelling system (virtual keyboard) to install a new non muscular communication channel. •In the future: It will be used by total normal people to perform simple activities Spread commercially in the field of video games In military http://bci2.k-space.org Online Demo Thank You