Minimum Mean Distance and
k-Nearest Neighbor Classifiers
for Signal Processing
Kun Yi Li, Young Scholar Student, Quincy High School
Eric Lehman, Young Scholar Student, Belmont High School
Graduate research mentors: Matt Higger, Fernando Quiviria, PhD Candidate,
Northeastern University
Professor Deniz Erdogmus, Associate Professor, Northestern University College
of Computer Engineering, Cognitive Systems Laboratory
Why use brain interfaces?
Image Source:
http://i2.cdn.turner.com/cnn/dam/assets/
121016060125-orig-ideas-brainwavewheelchair-00013909-story-top.jpg
Stimulus
EEG
User
Brain Interface
Classifier
Decision
SSVEP Brain
Interface Video
•SSVEP: Stands for “Steady State Visually Evoked Potential”.
Definitions
This type of brain signal is a response to looking at repeated
intensities of light from 0 to 60 Hz.
•EEG: Stands for “electroencephalography”. EEG data is the
measurement of the brain’s electrical activity voltages on the
surface of the scalp over a certain period of time.
•Iris Dataset: A dataset that contains 3 different types for
flowers, 50 samples each, and 4 different features (sepal
length in cm, sepal width in cm, petal length in cm, petal
width in cm).
•Classifier: An algorithm that divides data into different
group based on their similarities.
• An algorithm that classifies multiple types of data.
• When given a test point, the program:
1. calculates the distance from the new data point to the
average of training data points.
Minimum
Mean Distance
Classifier
2. selects the training data point with the shortest distance
3. identifies the new data point in the same group as the
closest training point.
6
Classification of Iris Flower Dataset Using
Minimum Mean Distance Classifier
Ground Truth Class
Minimum Mean
Distance
Classifier
I. setosa
Estimated
Class I. versicolor
I. virginica
I. setosa
I.
versicolor
I.
virginica
1
0
0
0
0.92
0.14
0
0.08
0.86
7
Classification of EEG Data Using
Minimum Mean Distance Classifier
Ground Truth Class
Minimum Mean
Distance Classifier
Estimated
Class
20 Hz
15 Hz
12 Hz
20 Hz
1
0
0
15 Hz
0
1
0
12 Hz
0
0
1
8
• An algorithm that classifies and divides multiple types of data.
• When given a new test data point, the KNN classifier:
• 1. Calculates the distance from the test data to all training
data points
k-Nearest
Neighbor
Classifier
• 2. Selects the k number of training data points that are the
closest to the test data point
• 3. Identifies the test data point as the same as the most
common class among the k nearest training data points
9
Classification of Iris Flower Dataset Using
Minimum Mean Distance Classifier
Ground Truth Class
k-Nearest
Neighbor
Classifier
I. setosa
Estimated
Class I. versicolor
I. virginica
I. setosa
I.
versicolor
I.
virginica
1
0
0
0
0.94
0.04
0
0.06
0.96
10
Classification of EEG Data Using
K-Nearest Neighbor Classifier
Ground Truth Class
K-Nearest
Neighbor
Classifier
Estimated
Class
20 Hz
15 Hz
12 Hz
20 Hz
1
0
0
15 Hz
0
1
0
12 Hz
0
0
1
11
• Separates the training set from the test set by
segmenting the data into k number of sections
• The classifier will test on one section and train the
remaining sections
K Fold Cross
Validation
• Prevents overfitting
Image Source: http://classes.engr.oregonstate.edu/eecs/winter2011/cs434/notes/knn-4.pdf
12
• RSVP Typing system
• Uses P300 brain signal
to determine which letter
is the acquired target
Applications
• SSVEP brain interface
• Control robot motions
through looking at a
screen
Image Source: http://www3.ece.neu.edu/~purwar/research/photo_gallery.htm, http://www3.ece.neu.edu/~orhan/
Can classify
not just EEG
data, but
many other
types of data!
Iris Flower Dataset
Applications
Image source: http://en.wikipedia.org/wiki/Iris_flower_data_set
14
• Perform K-Fold Cross Validation
• Classify unprocessed EEG data using more advanced
concepts to determine the most likely decision
Future Work
• Classify EEG data obtained from other types of stimuli
such as tactile sensors
• Help individuals with Locked-in Syndrome to
communicate and with others through brain interfaces
•Graduate Research Mentors: Matt Higger, Fernando Quivira,
PhD Candidates, Northeastern University
•Professor Deniz Erdogmus, Department of Electrical and
Acknowledgements
Computer Engineering, Cognitive Systems Lab,
Northeastern University
•Orkan Sezer, Summer intern, Northeastern University
•Center for STEM Education
•Young Scholars Program & Team
•Claire Duggan - Director
•Kassi Stein, Jake Holstein, Chi Tse - YSP Coordinators
Questions?