Some Mathematical Ideas for Attacking the
Brain Computer Interface Problem
Michael Kirby
Department of Mathematics
Department of Computer Science
Colorado State University
Department of Mathematics
Overview





The Brain Computer Interface (BCI) Challenge
Signal fraction analysis
Takens’ theorem and classification on manifolds
Nonlinear signal fraction analysis
Conclusions and future work
Department of Mathematics
NSF BCI Group





Chuck Anderson (PI), Computer Science, Colorado State
Michael Kirby (Co-PI), Mathematics, Colorado State
James Knight, Ph.D. Student, Colorado State
Tim O’Connor, Ph.D. Student, Colorado State
Ellen Curran, Medical Ethics and Jurisprudence, Dept. of Law, Keele
University, Staffordshire, UK
 Doug Hundley, Consultant, Department of Mathematics, Whitman
 Pattie Davies, Occupational Therapy Department, Colorado State
 Bill Gavin, Dept. of Speech, Language and Hearing Sciences,
University of Colorado
“Geometric Pattern Analysis and Mental Task Design
for a Brain-Computer Interface”
Department of Mathematics
SourceForge
https://sourceforge.net/projects/csueeg/
 Development Status: 1 - Planning
 Environment: Other Environment
 Intended Audience: Science/Research
 License: GNU General Public License (GPL)
 Natural Language: English
 Operating System: Linux, SunOS/Solaris
 Topic: Artificial Intelligence, Human Machine Interfaces, Information Analysis,
Mathematics, Medical Science Apps.
Department of Mathematics
Chuck Anderson
Department of Mathematics
Pattie Davies
Department of Mathematics
BCI Headlines in the News
 Computers obey brain waves of paralyzed, Associated Press,
appearing in MSNBC News, April 6, 2005
 Brainwaves Control Video Games, BBC March 2004
 Brainwave cap controls computer, BBC December 2004
 Brain Could Guide Artificial Limbs
 Patients put on thinking caps, Wired News, January 2005
 Monkey thoughts control computer, March 2002
Department of Mathematics
Lou Gehrig’s Disease (ALS)
 Amyotrophic Lateral Sclerosis (ALS) , or “Locked-In
Syndrome”, is an extreme neurological disorder and many
patients opt against life support.
 Most commonly, the disease strikes people between the
ages of 40 and 70, and as many as 30,000 Americans have
the disease at any given time. (ALS Association website).
 Genetic factors appear to only account for 10 percent of all
ALS cases. ALS can strike anyone, anytime.
 There are no effective treatments and no cure.
 Brain activity appears to remain vigorous while muscle
control atrophies degeneritively and completely.
Department of Mathematics
Gulf War Veterans and ALS
The following information is from a news release sent out by the
Department of Veteran Affairs on December 10, 2001.
(ALS Association Web posting.)
“According to a news release on December 10, 2001 from the Department
of Veteran Affairs, researchers conducting a large epidemiological
study supported by both the Department of Veterans Affairs and the
Department of Defense have found preliminary evidence that veterans
who served in Desert Shield-Desert Storm are nearly twice as likely as
their non-deploying counterparts to develop amyotrophic lateral
sclerosis.”
Department of Mathematics
The Brain Computer Interface (BCI)
A means for communication between person and machine
via measurements associated with cerebral activity, e.g.,
EEG, fMRI, MEG.
We assume that no muscle motion is employed such as eye
twitching or finger movement.
Department of Mathematics
Low-Cost EEG
Department of Mathematics
History of EEG




Duboi-Reymond (1848) reported the presence of electrical signals
Caton (1875) measured “feeble” currents on the scalp
Berger (1929) measured electrical signals with EEG
1930-50s EEG used in psychiatric and neurological sciences relying on
visual inspection of EEG patterns
 1960s-70s witness emergence of Quantitative EEG and confirmation
of hemispheric specialization, e.g., left brain verbal and right brain
spatial.
 1980s+ observation of biofeedback
Department of Mathematics
Characteristics of Brainwaves
Delta waves [0,4] Hz associated with sleep. Also
empathy.
Theta waves [4, 7.5] associated with reverie,
daydreaming, meditation, creative ideas
Alpha waves [7.5,12] prevalent when alert and
eyes closed. Associated with relaxed positive
feelings.
Beta waves 12Hz+ associated with active state,
eyes open.
Department of Mathematics
Reasons Why EEG Should Not Work for BCI
Electrical activity generated by complex system of
billions of neurons
Brain is a “gelatinous mass” suspended in a
conducting fluid
Difficult to “register” electrode location
Artifacts from motion, eyeblinks, swallows,
heartbeat, sweating…
Food, age, time of day, fatigue, motivation of
subject
Department of Mathematics
Why EEG Can Work for BCI
Many EEG studies have reported reproducible
changes in brain dynamics that are task
dependent!
People are able to control their brainwaves via
biofeedback!
Department of Mathematics
Biofeedback
Patients may “correct” their waveforms to achieve a normal state.




Kamiya demonstrated the controllability of alpha waves in 1962.
Communication in morse code by turning alpha waves on and off.
Stress management and sleep therapy.
Move a pac-man by stimulating alpha and beta waves.
Note that artifacts are a serious problem for real-time biofeedback
applications.
Department of Mathematics
Motivation for Our Work
 Current biofeedback training requires 10 weeks to move a cursor.
 Typing requires 5 minutes/letter with 90% accuracy.
 Although there has been some mathematical work the field has been
dominated by experiment and heuristics.
 Suggestions by clinical EEG experts that understanding EEG problem
will have a strong mathematical component.
 Tremendous potential for results.
Department of Mathematics
EEG Data Set: Mental Tasks





Resting task
Imagined letter writing
Mental multiplication
Visualized counting
Geometric object rotation
Keirn and Aunon, “A new mode of communication between man and his
surroundings”, IEEE Transactions on Biomedical Engineering,
37(12):1209-1214, December 1990
Department of Mathematics
Lobes of the Brain
Frontal Lobes
Personality, emotions, problem solving.
Parietal lobes
Cognition, spatial relationships and
mathematical abilities, nonverbal
memory.
Occipital lobes
Vision, color, shape and movement.
Temporal lobes
Speech and auditory processing, language
comprehension, long-term memory.
Department of Mathematics
Electrode Placement
and Sample Data
50
C3 0
50
50
0
500
1000
1500
2000
2500
0
500
1000
1500
2000
2500
0
500
1000
1500
2000
2500
0
500
1000
1500
2000
2500
0
500
1000
1500
2000
2500
0
500
2000
2500
C4 0
50
50
P3 0
50
20
P4 0
20
20
O1 0
20
20
O2 0
20
Department of Mathematics
1000
1500
Sample (250 per second)
Geometric Filtering of Noisy Time-Series
Given a data set
The Q fraction of a basis vector
is defined as where
Department of Mathematics
Signal Fraction Optimization
Determine  such that D() is a maximum.
Solution via the GSVD equation
Department of Mathematics
Department of Mathematics
Original Signal
SVD filter
Signal fraction filter
Department of Mathematics
SVD basis
GSVD basis
0.1
0.04
0
0.02
0. 1
0.1
0
0
0.05
0
0
0. 1
0
0
100
100
200
200
300
300
400
400
500
500
600
600
700
700
800
800
900
900
1000
1000
0.05
0
0.05
0.05
0
0
0.05
0
100
200
300
400
500
600
700
800
900
1000
0.05
0
0.05
0.2
0
0
0.05
0
100
200
300
400
500
600
700
800
900
1000
0. 2
0
0.2
0.2
0
0
0. 2
0
100
200
300
400
500
600
700
800
900
1000
0. 2
0
100
200
300
400
500
600
700
800
900
1000
100
200
300
400
500
600
700
800
900
1000
100
200
300
400
500
600
700
800
900
1000
100
200
300
400
500
600
700
800
900
1000
100
200
300
400
500
600
700
800
900
1000
Department of Mathematics
SVD reconstruction
GSVD reconstruction
100
100
50
50
0
0
100
200
300
400
500
600
700
800
900
1000
0
200
200
100
100
0
0
100
200
300
400
500
600
700
800
900
1000
0
20
20
10
10
0
0
100
200
300
400
500
600
700
800
900
1000
0
30
30
20
20
10
10
0
0
100
200
300
400
500
600
700
800
900
1000
0
25
25
24.8
24.8
24.6
0
100
200
300
400
500
600
700
800
900
1000
24.6
0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
400
500
600
700
800
900
1000
0
100
200
300
400
500
600
700
800
900
1000
Department of Mathematics
Blind Signal Separation
Unknown (tall) m £ n signal matrix S
Unknown mixing n £ n matrix A
Observed m £ n data matrix X
Task: recover A and S from X alone.
In general it is not possible to solve this problem.
Department of Mathematics
Signal Fraction Analysis Separation
Theorem: The solution to the signal fraction analysis optimization problem solves
the signal separation problem X = SA given
1)
is observed
2)
3)
In particular,
Where is the  solution to the GSVD problem for signal fraction analysis.
Department of Mathematics
Original signals (unknown)
2
5
0
0
2
0
50
100
150
200
250
300
350
400
450
500
5
5
5
0
0
5
0
50
100
150
200
250
300
350
400
450
500
5
2
5
0
0
2
5
Mixed signals (observed)
0
50
100
150
200
250
300
350
400
450
500
5
5
5
0
0
0
50
100
150
200
250
300
350
400
450
500
5
0
50
100
150
200
250
300
350
400
450
500
0
50
100
150
200
250
300
350
400
450
500
0
50
100
150
200
250
300
350
400
450
500
0
50
100
150
200
250
300
350
400
450
500
Department of Mathematics
FastICA separation
5
2
0
0
5
0
50
100
150
200
250
300
350
400
450
500
2
2
2
0
0
2
0
50
100
150
200
250
300
350
400
450
500
2
2
5
0
0
2
5
Signal fraction separation
0
50
100
150
200
250
300
350
400
450
500
5
5
5
0
0
0
50
100
150
200
250
300
350
400
450
500
5
0
50
100
150
200
250
300
350
400
450
500
0
50
100
150
200
250
300
350
400
450
500
0
50
100
150
200
250
300
350
400
450
500
0
50
100
150
200
250
300
350
400
450
500
Department of Mathematics
math training data
letter training data
1
1
2
2
3
3
4
4
5
5
6
6
500
1000
1500
2000
2500
500
math test data
1
2
2
3
3
4
4
5
5
6
6
1000
1500
1500
2000
2500
2000
2500
letter test data
1
500
1000
2000
2500
500
Department of Mathematics
1000
1500
Artifact Removal
Given the separated signals  = X  we may filter the ith column of  by
setting
Where Id’ is the identity matrix with the ith row set to zero. The filtered
version of the data is now
Where recall the original data is
Department of Mathematics
Signal Fraction Filters
Department of Mathematics
Constructing Signal Fraction Filter
Department of Mathematics
Department of Mathematics
Benefits of Signal Fraction Analysis
 Can identity sources of noise such as respirators, eyeblinks, cranial
heartbeat, line noise etc…
 Filtering works over short periods of the signal, i.e., can remove
artifacts from a time series of length 500.
 Can use generalizations of the signal to noise ratio to separate
quantities of interest.
 Simple and fast to compute.
Department of Mathematics
Classification on Manifolds
 Insert slide from Istec meeting
manifold: H(x) = 0
dist(A,B) large but H(A)=H(B)=0
Department of Mathematics
Dynamical Systems Perspective
Assume a system is described by the dynamical equations
and that the solutions reside on an attracting set, e.g., a manifold. What
can be said about the full system if it is only possible to observe part of
the system? In the extreme, imagine we can only observe a scalar
value
Department of Mathematics
Time Delay Embedding
We may embed the scalar observable into a higher dimensional state space
via the construction
So now it is clear that
Department of Mathematics
Taken’s Theorem (simplified)
Given a continuous time dynamical system with solution on a compact
invariant smooth manifold M of dimension d, a continuous
measurement function h(x(t)) can be time-delay embedded in to
dimension 2d+1 such that there is a diffeomorphism between the
embedded attractor and the actual (unobserved) solution set.
Department of Mathematics
The Lorenz Attractor
Given a data point (x,y,z)
we know which lobe by
the sgn of x. But what if
we only observe the z
value? The lobe can be
classified using Taken’s
theorem and Time delay
embedding.
Department of Mathematics
Do EEG data lie on an attractor?
Department of Mathematics
Elephants in the Clouds?
Classification rate
Random data
Department of Mathematics
Super Resolution Skull Caps
How many electrodes are needed? 6, 16, 32, 128, 256, 512? We should
be able to answer this question by means of evaluating an objective
function.
Through attractor reconstruction, time delay embedding techniques may
practically enhance the resolution of skull caps leading to significant
savings in time and equipment.
Colleagues working on EEG studies in children are very enthusiastic
about this!
Department of Mathematics
Manifolds and Nonlinear Methods
(work with Fatemeh Emdad)
Veronese embeddings of the data:
Degree 1: (x,y)
Degree 2: (x2, xy, y2)
Degree 3: (x3, x2y, xy2, y3)
Degree 1: (x,y,z)
Degree 2: (x^2, xy, xz, y^2, yz, z^2)
Degree 3: (x^3, x^2y, x^2z, xy^2, xz^2, xyz, y^3, y^2z, yz^2, z^3)
Such embeddings are behind one variant of kernel SVD.
Department of Mathematics
Kernel SVD versus Kernel SFA
Numerical Experiments:
KSVD (KPCA) degree = 1, 2, 3, 4
KSFA degree = 1, 2, 3, 4
Objective: compare mode classification rates using knn for k = 1,…, 10.
Department of Mathematics
KSFA, KPCA degree 1
Department of Mathematics
KSFA, KPCA degree 2
Department of Mathematics
KSFA, KPCA degree 3
Department of Mathematics
KSFA, KPCA degree 4
Department of Mathematics
Relative Performance
Department of Mathematics
Conclusions and Future Work
 Present a geometric subspace approach for signal separation, artifact
removal and classification.
 Provided evidence that brain dynamics might reside on an attractor and
that time-delay embedding enhances classification rates.
 Illustrated a nonlinear extension to signal fraction analysis and
compared with similar extension to svd.
 These ideas are presented in the context of EEG signals but are quite
general and can be applied to images.
Department of Mathematics