French community for functional NIRS
Signal processing techniques for fNIRS and
application to Brain Computer Interfaces
Gautier Durantin, ISAE/CERCO
ISAE, Toulouse, 15.04.2014
1
Introduction
 EEG and fNIRS today encompass the most active areas of Brain Computer
Interfaces research (Min 2010)
 fNIRS is currently mainly used as a complement of EEG (Takeuchi 2009, Fazli
2011)
 Noise reduction techniques and signal improvement techniques are the
next step to improve BCI performance (Mitsukura 2013, Izzetoglu 2010)
(Bashashati 2007)
2
Filtering for fNIRS – What do we need to filter ?
 Low frequency components : linear trends, measurement bias (Jang 2008)
 High frequency components : physiological noise (cardiac frequency),
measurement noise (Huppert 2006)
Raw signal (x)
Filtered signal (y)
FILTERING
MODULE
 Use of filters to remove these components (e.g. linear filters)
!
Delay, stability,
performance
3
Moving Average Convergence Divergence Filter
 We propose a specific linear filter used by economists (Utsugi 2007, Cui
2010)
 The MACD (moving average convergence divergence), based on Exponential
moving average (EMA) filters.
Raw signal (x)
Exponential
moving average
(EMA)
𝑦𝑛 = 𝑚𝑒𝑎𝑛 𝑥𝑛 , 𝑥𝑛−1 , … , 𝑥𝑛−𝑘+1
Filtered signal (y)
= 𝑚𝑒𝑎𝑛 (𝑘 𝑙𝑎𝑠𝑡 𝑣𝑎𝑙𝑢𝑒𝑠 𝑜𝑓 𝑥)
 MACD is obtained from two EMA : one short (k small), and one long (k big)
Raw signal (x)
Shortterm EMA
Long-term
EMA
MACD
+
-
Filtered signal (y)
4
Moving Average Convergence Divergence Filter
MACD (short = 6s ; long = 13s)
 Economists use a signal line, obtained from a short EMA (5s) of MACD data,
to predict stable increases on the curve. (Appel 1999)
 A MACD crossover of the signal line predicts a stable increase in the signal
measured A MACD crossover of the signal line predicts a stable increase in
the signal measured
5
Real-time hemodynamic response onset detection
 Controlled experiment of digit sequence memorization task
9 subjects
24 trials
8
9
4
51
x-x-x
X
STIMULATION
REST (6-9sec)
ANSWER
(8 sec)
Stimulus
Increase in
the signal
MACD crossover
6
Towards a real-time BCI using fNIRS
 4 control knobs :
Speed, Heading,
Altitude, Vertical speed
 Low load (speed 200, heading 200,
Alt. 2000…)
 High load (speed 245, heading 315,
altitude 8600…)
7
Towards a real-time BCI using fNIRS
Pilot in ISAE flight
simulator
ATC msg mi Resp. windows
SOA
Signal filtering
and synchronization
(MACD)
fNIRS output
REST
Real-time
information
on pilot’s
mental state
(Rest VS Task)
Air Traffic Control
(simulated)
Overall accuracy : 98 %
(std. Dev : 2,6%)
or
Classification
Process
TASK
Load detection
Data knowledge
from phase L
8
Towards a real-time BCI using fNIRS
Pilot in ISAE flight
simulator
ATC msg mi Resp. windows
SOA
Signal filtering
and synchronization
(MACD)
fNIRS output
REST
Real-time
information
on perceived
workload
Air Traffic Control
(simulated)
or
Classification
Process
TASK
Load detection
Data knowledge
from phase L
9
Classification process
fNIRS output of
20 training trials
Signal processing
(MACD) and feature
extraction
 Use of different
features (Tai & Chau
2009)
 [HbO2], [Hhb], peak
response, kurtosis,
skewness on different
time windows
Classifier design
(LDA, SVM)
TRAINING
TESTING
fNIRS output
CLASSIFIER
Real-time
information on
workload
10
Towards a real-time BCI using fNIRS : results
28 sessions
20 training
trials
20 testing
trials
 Overall accuracy obtained during testing phase : 79 %
(std. dev : 12,8%)
 19 subjects out of 28 have more than 75% accuracy
11
Further improvements in signal processing
To improve signal processing and BCI accuracy, a solution would be to
add a priori information in processing models
 Use of hemodynamic
response models for temporal
dynamic estimation of fNIRS.
(Boynton 1996, Buxton 1997)
 Use of Kalman filtering to
include estimation of temporal
dynamics in signal processing
(Abdelnour 2009, Gagnon
2011)
12
Kalman filtering
Participant
Stimulus
NIRS
Physiological
processing
model (HRF)
Measurement
model
NIRS
signal
Dynamical model of hemodynamic
response and fNIRS measurement
fNIRS raw data
KALMAN FILTER
Confidence in
the measures
fNIRS filtered
signal
Confidence in
the model
Kalman
filtering
13
Kalman filtering : results
 Tested offline on digit span memorization task data (9 subjects) with three
levels of difficulty
LINEAR FILTERING
KALMAN FILTERING
(effect size eta²=0,2)
(effect size eta²=0,34)
Kalman filtering is a promising tool to improve signal useability
Challenges remain concerning Kalman tuning and real-time
implementation
14
Conclusion
 Signal processing is a key step towards efficient Brain Computer Interface
using fNIRS.
 Linear filtering brings good results, but improvements can be made to
improve the accuracy of BCI designed with this type of filters.
 Kalman filtering or adaptive filtering are the best opportunities to improve
signal useability.
15
Thank you for your attention
16
Digit span task
 6 levels of
difficulty
 4 trials for each
level of difficulty
17
Kalman modeling
Physiological
response model
Measurement
model
Kalman filter
18
Possible improvement of Kalman modeling
Physiological
response model
Measurement
model
Kalman filter
MACD filter for onset
prediction
19