fMRI Brain-Computer
Interfaces
[A tutorial on methods and applications]
B
rain-computer interfaces (BCIs) utilize neurophysiological signals originating in the
brain to activate or deactivate external devices or computers [1]. Different neuroelectric signals have been used to control external devices, including EEG oscillations,
electrocorticograms (ECoGs) from implanted electrodes, event-related potentials
(ERPs) such as the P300 and slow cortical potential (SCP), short latency subcortical
potentials and visually evoked potentials, and action potential spike trains from implanted multielectrodes. In comparison, the development of BCIs based on metabolic activity of the brain using
two different imaging methods, functional magnetic resonance imaging (fMRI) [2] and functional
near infrared spectroscopy [3], has been more recent.
fMRI is a noninvasive technique that measures the task-induced blood oxygen level-dependent
(BOLD) changes correlating with neuronal activity in the brain [4]. Further progress has been
made in real-time fMRI since the first description of the method by Cox et al., [5]. In contrast to
conventional fMRI, which allows analysis of images only after the scan is finished, real-time fMRI
Digital Object Identifier 10.1109/MSP.2007.910456
1053-5888/08/$25.00©2008IEEE
IEEE SIGNAL PROCESSING MAGAZINE [95] JANUARY 2008
© PHOTO CREDIT
© STOCKBYTE & DIGITAL VISION
Sitaram, Nikolaus Weiskopf, Andrea Caria,
[Ranganatha
Ralf Veit, Michael Erb, and Niels Birbaumer]
Signal Acquisition
• Echo Planar Imaging (EPI)
• Field Strength
• Spatial Resolution
• Temporal Resolution
• Relaxation Time
• Magnetic Inhomogenities
Feedback Approaches
Thermometer
Virtual Reality
Time Series
Signal Analysis
Video Feedback
Participant
BC
• Head Motion Artifact Correction
• Breathing Artifact Correction
• Correlation Maps
• Anatomical Coregistration
• ROI Selection and Analysis
I-G
• Background
• Healthy Volunteer/Patient
• Naive/Experienced
• Mood
• Motivation for Learning
• Prior Instructions
• Operant Learning
UI
Signal Feedback
• Physiological Measure
• Pattern Classification
• Feedback Modality
• Temporal Delay
[FIG1] An fMRI-BCI system is a closed-loop system with the following subsystems: signal acquisition, signal preprocessing, signal
analysis, and signal feedback. Whole brain images from healthy subjects or patients are acquired employing a conventional echo planar
imaging (EPI) sequence or any of its variants. The measured hemodynamic response due to the BOLD effect is preprocessed for
correction of different artifacts, including for head motion. The signal analysis subsystem performs statistical analysis and generates
functional maps. Feedback can be presented to the subject by different modalities, including acoustic and visual; and with a variety of
visualization methods such as functional maps, continuously updated curves of the mean activity in one or more selected region-ofinterest (ROI), varying activity levels in one or more ROIs using a graphical thermometer, and even augmented interfaces such as virtual
reality (VR).
allows simultaneous acquisition, analysis, and visualization of
whole brain images. With progress in real-time fMRI due to
higher-field MRI scanners, fast data acquisition sequences,
improved real-time preprocessing and statistical analysis algorithms, and improved methods of visualization of brain activation and feedback to the subject, implementation of fMRI-BCI
and neurofeedback became feasible. Several studies [6]–[13] have
demonstrated that human subjects using real-time fMRI feedback can learn voluntary self-regulation of localized brain
regions. Experimenters have trained subjects to volitionally control specific cortical and subcortical areas, such as supplementary motor area (SMA), posterior part of the superior temporal
gyrus, parahippocampal place area (PPA), the anterior cingulate
cortex (ACC), insula, Broca’s area, and amygdala (for a complete
review see [2] and [14]). Results from these studies demonstrate
that fMRI-BCI provides a novel approach in neuroscience for
studying brain plasticity and functional reorganization following
sustained training of volitional control of circumscribed brain
regions. Furthermore, the approach may open new opportunities
for clinical rehabilitation, for example, of movement disabilities
and emotional disorders by training patients to control abnormal
activity in selected brain regions. Despite diverse applications
that fMRI-BCI promises, only a handful of institutions in the
world have exploited this technology, as technical challenges and
implementation hurdles have hindered widespread usage. The
intention of this tutorial is to outline the important subsystems,
alternate methods, and diverse applications of fMRI-BCI in order
to make this technique more accessible.
ARCHITECTURE OF fMRI-BCI
An fMRI-BCI system is a closed-loop system (Figure 1) with the
following major subsystems: signal acquisition, signal preprocessing, signal analysis, and signal feedback. The subsystems are
usually installed and executed in separate computers for
IEEE SIGNAL PROCESSING MAGAZINE [96] JANUARY 2008
magnetization to a radio signal, and the magnetization recovers
optimizing system performance and are connected via a local
rather slowly. This had limited the possibility of implementing a
area network (LAN). Whole brain images from healthy subjects
real-time MRI. Fortunately, over the last 20 years technical
or patients are acquired slice by slice employing an echo planar
advances in imaging have enabled
imaging (EPI) sequence. The
substantial reduction in acquisigreater the number of slices the
FUNCTIONAL MAGNETIC
tion time. The most significant
brain is divided into, the longer is
RESONANCE IMAGING IS A
speed advance came with the
the time for acquisition of the
NONINVASIVE TECHNIQUE THAT
development of echo-planar imagimages. As the real-time nature of
ing (EPI). EPI is capable of imagfMRI-BCI requires rapid acquisiMEASURES THE TASK-INDUCED
ing the entire brain in 1–2 s. At
tion of whole brain images (typiBLOOD OXYGEN LEVEL-DEPENDENT
this sampling rate, fMRI can accucally in 1–2 s), a tradeoff needs to
CHANGES CORRELATING WITH
rately follow the time course of
be made between spatial and temNEURONAL ACTIVITY IN THE BRAIN.
brain activation.
poral resolution. In most of our
In a traditional fMRI experistudies, we have used 16–20 slices
ment, images are reconstructed offline only after the experiment
[13]. An fMRI voxel of size ∼ 3 × 3 × 5 mm3 contains millions
has been completed. Real-time fMRI, on which fMRI-BCI is
of neurons. When neurons in an area become active, blood rich
based, requires the simultaneous reconstruction of the images
in oxygen flows to the area. The source of the fMRI signal is the
with the acquisition of the MR signal. Cox’s group reported the
difference in the magnetic properties of oxygenated blood from
first implementation of a real-time fMRI system using a wholedeoxygenated blood. The measured hemodynamic response due
body 3T scanner (Bruker Instruments) [5]. In their implementato the BOLD effect, which is the neurovascular response to
tion, the analog signal from the signal acquisition system was
brain activity, lags behind the neuronal activity by approximatesent to a workstation for analog-to-digital conversion and image
ly 3–6 s [4]. Higher static magnetic field (B0) strengths and
reconstruction. In our laboratory, we have modified the Siemens
more sophisticated MRI pulse sequences are often used to
MR scanner’s image reconstruction software to allow online
increase the signal-to-noise ratio (SNR). The acquired images
reconstruction of whole-brain images at the end of every repetiare then preprocessed to correct for head motion, compensate
tion time (TR) and storage of these images in a prespecified foldfor signal dropouts and magnetic field distortions, and apply
er to be immediately retrieved for further processing, analysis,
spatial smoothing. The signal analysis subsystem performs staand feedback by the fMRI-BCI system. The online image recontistical analysis and generates functional maps using any of the
struction software program was written in C++ based on the
following methods: subtraction of active and rest conditions,
image reconstruction environment (ICE) provided by Siemens.
correlation analysis, multiple regression, general linear model
The RTExport system runs both on the 1.5 T (Vision) and 3 T
(GLM), and pattern classification. Feedback can be presented to
(TIM Trio) Siemens scanners.
the subject by different modalities, including acoustic and visuMany factors influencing signal acquisition have important
al, and with a variety of visualization methods, such as functionconsequences for real-time performance of fMRI-BCI: static
al maps, continuously updated curves of the mean activity in
magnetic field (B0) strength, spatial resolution, temporal resoone or more selected regions of interest (ROI), varying activity
lution, echo time, and magnetic field inhomogeneities.
levels in one or more ROIs using a graphical thermometer, and
Although high spatial resolution is desired, increasing the spaaugmented interfaces such as virtual reality (VR). Feedback is
tial resolution decreases the SNR and increases the acquisition
presented at an interval that depends on the time involved for
time, and hence a compromise needs to be made among these
image acquisition and processing, based on the computational
variables. Commonly in fMRI-BCI, 64x64 image matrices resultresources available and the efficiency of the algorithms with
ing in 3–4 mm in-plane resolution, and slice thickness of
which they are implemented, thus directly affecting the peraround 5 mm are used. For online processing after image acquiformance of the system. A short interval is critical for learning
sition, spatial filtering or averaging across an ROI helps improve
voluntary control of brain activity (for example, 1.5 s, [13]).
SNR. Reduced spatial resolution could be beneficial, compensatIn the following sections we will elaborate on each of the
ing for head motion, data complexity, and inter-subject variabilisubsystems, explaining different algorithms and methods develty [14]. A TR of 1,500 ms has been used [8], [9] in real-time fMRI
oped so far. We will consider technological and psychophysiologwith single-shot EPI. It is advisable that fMRI-BCI studies
ical factors that affect and influence the performance and
choose the echo time (TE) close to the relaxation time (T2∗ ) of
efficiency of the system. Wherever relevant we will also comment on future innovations and directions for research. Finally,
the gray matter in the brain to maximize functional sensitivity
we will describe the major applications of the fMRI-BCI.
[16]. This value is about 70 ms at 1.5 T and 45 ms at 3 T.
At the interface between tissue and air in the brain, in areas
SIGNAL ACQUISITION
such as the orbitofrontal cortex and temporal pole, a significant
Conventional MRI has been a slow imaging modality where
change in the local magnetic field is present over a short disincreases in imaging speed result in signal losses [15]. The reatance. Artifacts such as signal dropouts and geometric distorson is that the MR signal is derived from the conversion of tissue
tions (local shifts and compressions in the image) caused by
IEEE SIGNAL PROCESSING MAGAZINE [97] JANUARY 2008
rospective correction uses rigid body transformation normally
magnetic field inhomogeneities potentially affect the performconsisting of three translational and three rotational parameance of fMRI-BCI. Several methods have been developed for
ters. Realignment parameters are typically estimated by optireducing susceptibility-related signal losses in fMRI (for an
mizing a similarity measure
overview see [17]). Weiskopf et al.
based on voxel signal intensity
[17] developed a theory supported
AN fMRI-BCI SYSTEM IS A
values, quantifying the difference
by experimental evidence showing
CLOSED-LOOP SYSTEM WITH THE
between a specific image in the
that susceptibility-induced gradiFOLLOWING MAJOR SUBSYSTEMS:
time-series, and the reference
ents in the EPI readout direction
SIGNAL ACQUISITION, SIGNAL
[20]. As an adaptation for realcause severe signal losses. They
PREPROCESSING, SIGNAL ANALYSIS,
time fMRI, Mathiak et al. [19]
have proposed a model to simulate
AND SIGNAL FEEDBACK.
developed a real-time retrospecEPI dropouts to make informed
tive algorithm that performs a
choice of scan parameters dependrigid body motion correction of a
ing on the field inhomogenieties
complete multislice EPI dataset within a single TR cycle. In this
in a region. Based on this insight, they developed an optimized
method, one of the first images is chosen as the reference
EPI sequence for maximal BOLD sensitivity using a specific
image, and all subsequent images are realigned separately with
combination of an increased spatial resolution in the readout
respect to this image. The optimization criterion is to minimize
direction and a reduced echo time. We foresee the real-time
the quadratic difference between the reference image and shiftadaptation of such techniques for fMRI-BCI applications.
ed image. In the three-stage implementation of this technique,
the reference gradient maps are first calculated, motion parameSIGNAL PREPROCESSING
ters are then estimated, and finally images are corrected for the
This component of the fMRI-BCI system retrieves the reconestimated movement using re-slicing with linear interpolation.
structed images from the signal acquisition component via the
For selecting gradient maps, a set of estimation equations are
LAN and performs data preprocessing. Methods of signal preproused for the translational and rotational components for the
cessing can be head motion artifact correction, respiratory and
three axes.
cardiac artifact correction, and spatial smoothing.
Retrospective motion correction has a few drawbacks,
including blurring due to interpolation and image transformaHEAD MOTION CORRECTION
tions, potential misregistration due to local intensity changes
Head motion is one of the largest sources of artifacts in fMRI. If
from the BOLD signal, and the potential for introducing falsetwo neighboring voxels differ in intrinsic brightness by 20%,
positives or for false-negatives in the activation statistics [21].
then a motion of 10% of a voxel dimension can result in a 2%
The retrospective motion correction method has been incorposignal change—comparable to the BOLD signal change at 1.5 T
rated into a number of public-domain postprocessing software
subsequent to neural activation [18]. The motion artifacts can
packages, including SPM, AIR, and AFNI. Turbo Brain-Voyager
interfere with and reduce the detection of signal changes due to
(TBV, Brain Innovation, Maastricht, Netherlands), a commercial
neural action or even simulate signal changes. Head motion can
software for real-time fMRI which is incorporated in our fMRIbe reduced by padding and a bite bar to some extent only. RealBCI, has implemented a real-time version of retrospective
time fMRI feedback profits from robust online motion correcmotion correction. Figure 2 shows estimated values of head
tion. Because the head moves as a whole, rigid body
motion in the three translational directions and three rotational
transformations (three for translation, three for rotation) can be
directions, as computed by TBV.
estimated from a number of volume data points for motion corProspective methods correct for subject head motion before
rection. The short response latency (tens of seconds) of realimage acquisition by adjusting scanning parameters by tracktime fMRI makes it particularly sensitive to motion artifacts
ing the moving anatomy [20]. Ward et al. [21] developed such
[19]. Motion correction in real-time requires efficient algoa method by measuring rotation and translation for each of the
rithms that can be executed on fMRI data sets within a single
sagittal, axial, and coronal planes. This was achieved by incorTR. For neurofeedback applications, feedback of motion artifacts
porating a special type of sequence called orbital navigator
to the subjects and omitting rewards may discourage head
echo sequence, a separate process applied before each cycle of
movement. Two major types of head motion correction have
acquisition of multislice fMRI signals. Immediately after their
been developed: retrospective and prospective methods. Both
acquisition, the navigator signals are processed to determine
methods can be potentially applied to fMRI-BCI, when real-time
motion in three degrees of rotation and three degrees of transadaptations of the methods are made feasible and implemented.
lation. The values of the rotations and translations detected are
Retrospective motion correction for real-time fMRI involves
then used to adjust the gradient rotation matrix and the RF
image registration soon after every volume of the fMRI data is
excitation frequency prior to the excitation of the subsequent
acquired. Prospective methods correct for subject head motion
imaging sequence. Any motion relative to the baseline detereven before image acquisition.
mined from the navigator acquisitions at the start of the mulRetrospective methods realign the time-series data to a refertislice cycle is used to correct all the images of that cycle. The
ence image collected during the fMRI session. Conventional ret-
IEEE SIGNAL PROCESSING MAGAZINE [98] JANUARY 2008
File Analysis View Multi-run Help
Contrast: Left Hand > Off
Auto-Advance
Nr.
Contrast value
Auto-Start
Enable
Condition name
1
/2
Color
−1 Off
+1 Left Hand
0 Right Hand
ROI1 Time-Series
560
550
540
7
47
7
47
ROI2 Time-Series
550
Event-Related Average
500
450
2
ROI1
ROI2
Motion Parameters
1
0
1
−1
0
11
111
121
131
Current settings file: “setting−1.mtbv”.
Threshold: t(188) = 4.80.p = 0.000003.
−1
−3
0
3
6
9
12 15
Processing completed.
Ready.
[FIG2] Real-time functional maps during motor imagery of left and right hands. The functional maps are displayed using the Turbo
Brain Voyager software (TBV; Brain Innovations, Maastricht). The TBV software allows orthographic and slice-based display of
functional activations. On the left panel, statistical maps of brain activation are superimposed on the orthographic images (sagittal
image, top-left; coronal image, top-right; axial image, bottom-right) acquired by an EPI sequence. In this example, self-paced motor
imagery of the left hand and right hand resulted in activation in the right and left primary motor areas (ROI1, red rectangle, and ROI2,
green rectangle, respectively), somatosensory areas, and supplementary motor area (SMA). The upper and middle right panels show
the BOLD time courses in the ROI1 and ROI2, respectively. Blocks of left-hand imagery are colored red, right-hand imagery green, and
baseline grey. The lower right panel displays the head motion parameter (three translations and three rotations).
time from the acquisition of the first navigator echo to alteration of the EPI acquisition can be less than 160 ms. The
results of applying the technique to volunteers demonstrated
the feasibility of in vivo correction of head-motion for realtime fMRI [21].
PHYSIOLOGICAL NOISE CORRECTION
The magnetic field in the head changes during breathing
because of the bulk motion of the thorax. Breathing patterns
may change the fMRI signal more than the desirable BOLD
response. Changes in the respiratory rhythm and volume can
also change the CO2 level in the blood and cause BOLD signal
fluctuations [23]. The pulse is also known to cause artifacts.
Techniques have been developed to remove cardiorespiratory
artifacts during offline analysis [22]–[24], but they have not
been adapted to online processing for real-time fMRI. Recently
van Gelderen et al. [25] reported a real-time shimming method
to compensate for respiration induced fluctuations in the main
magnetic field (B0 field). Future implementations of fMRI-BCI
could potentially explore the use of these methods for correction
of physiological artifacts and noise. This becomes even more
important at higher static magnetic fields, because the relative
contribution of physiology to the noise increases [26].
SIGNAL ANALYSIS
While the majority of work in fMRI-BCI has used conventional
neuorimaging methods of univariate analysis, there is a growing
interest in incorporating multivariate methods of pattern analysis using machine learning techniques in the emerging field of
brain state detection. In this section, we will consider both
methods as applied to fMRI-BCI.
UNIVARIATE ANALYSIS
Univariate methods seek to find out how a particular perceptual
or cognitive state is encoded by measuring brain activity from
many thousands of locations repeatedly but then analyzing each
location separately [27]. If the responses at any brain location
differ between two states, then it is possible to use measurements of the activity at that location to determine or decode the
state. A commonly used method for detecting neuronal activity
from fMRI time series is correlation analysis. The method computes correlation coefficients between the time-series of the
IEEE SIGNAL PROCESSING MAGAZINE [99] JANUARY 2008
reference vector representing the expected hemodynamic
response and the measurement vector of each voxel. A primary
advantage of this method is that the reference vector can have
an arbitrary shape best reflecting the hemodynamic response.
Gembris et al. [28] presented a computationally efficient algorithm, implemented in the analysis software Functional
Magnetic Resonance Imaging in Real-time (FIRE), which generates correlation coefficients on a “sliding-window” of the fMRI
time series. The basic concept of this method is to restrict the
correlation computation to only the most recent data sets.
According to this method, definition of the correlation coefficient in combination with detrending is given by the equation:
ρ=
xrs
,
|xs||rs|
(1)
where x is the measurement vector of one voxel that is updated
at every time step, rs is the reference vector, and xs and r s are
detrending vectors. (For further details on implementation of
this method the reader is referred to [28].) Each new data set
replaces the data set of the previously acquired sliding-window
buffer. This method reduces the load on memory and computational time, two important factors that critically affect the performance of fMRI-BCI. The authors tested the method in an
experiment with 20 healthy participants in a paradigm comprising alternating baseline and visual stimulation blocks. The
sliding window correlation method successfully identified the
visual areas as being significantly active with voxels in this
region crossing the threshold correlation coefficient of 70%.
The method offers greater sensitivity of the correlation coefficients to changes in the signal response shape and amplitude
with passing measurements. Another advantage of the slidingwindow technique is its capability for quantifying physiological
variability when combined with a technique called reference
vector optimization [28]. This method takes into account a
realistic model of the hemodynamic response function to adapt
the reference vector to the measured data and thus increases
functional sensitivity.
The GLM provides by far the most unified framework in the
analysis of the fMRI data [29]. GLM can model multiple experimental and confounding effects simultaneously. Bagarinao et al.
[30] presented a method for real-time estimation of GLM coefficients. The observed fMRI data are considered a linear combination of L explanatory functions f i(.) and an error term:
yk, s = bk,1 f1 (ts) + · · · + bk,1 fL(ts) + k,s
(2)
where yk,s is the observation of kth voxel at time ts, s = 1..n are
scan numbers, f s(.) are basis functions that span the fMRI
responses for a given experiment, bk are coefficients that need to
be estimated and εk,s is the residual error or noise term. The
method converts the basis functions or explanatory variables of
a GLM into orthogonal functions using an algorithm called the
Gram-Schmidt orthogonalization procedure. The coefficients of
the orthogonal functions are then estimated using the orthogo-
nality condition. (For further details on implementation of this
method the reader is referred to [30].) In a conventional GLM
analysis of fMRI data, multiple trials are required to identify significantly activated voxels with sufficient consistency. However,
it is not possible to obtain many trials in an fMRI-BCI setting
with its very need for identifying significantly active voxels in
real-time. The advantage of the real-time GLM implementation
is that estimates can be updated when new image data are available, making the approach suitable for fMRI-BCIs. Furthermore,
with this approach it is not necessary to store the data as the
data are immediately used in computing the estimates, thus
reducing the memory requirements. A similar approach is taken
by the analysis software (TBV) running on our local fMRI-BCI
setup at the University of Tübingen, which uses the recursive
least squares regression algorithm [31] to incrementally update
the GLM estimates.
After identification of the significantly active voxels, either
by the method of real-time correlation or GLM analysis, their
values are passed to the signal feedback subsystem at every
time point for computation and presentation of the feedback to
the participant.
MULTIVARIATE ANALYSIS
Using univariate analysis it is often difficult to find individual
locations where the differences between conditions are large
enough to allow for efficient decoding. In contrast to the conventional analysis, recent work shows that the sensitivity of
human neuroimaging may be improved by taking into account
the spatial pattern of brain activity [32]–[35]. Pattern-based
methods use considerably more information for detecting the
current state from measurements of brain activity. In the previous studies with fMRI-BCI, brain signals from only one or two
ROIs were extracted for providing neurofeedback to the subject.
A major argument for moving away from deriving feedback signals from single ROIs is that perceptual, cognitive, or emotional
activities generally recruit a distributed network of brain regions
rather than single locations. Pattern-based methods not only
use voxel-intensities but also their spatiotemporal relationships.
Several studies have previously reported offline classification
of fMRI signals using various pattern-based methods such as
multilayer neural networks [35], Fisher Linear Discriminant
(FLD) classifier [36], and support vector machines (SVMs).
Laconte et al. [37] recently reported probably the first implementation of a real-time pattern classification system that could
be applied to neurofeedback and BCI. The aim of the study was
to first train a classification model based on early fMRI data and
thereafter to use the classifier to predict the brain state with
each acquired image and alter the stimulus based on the estimated brain state. The authors modified the Siemens scanner’s
image calculation environment (ICE) to perform SVM classification during training and testing and then transmitted the classification results to a stimulus display computer. To improve the
efficiency of classification the authors implemented a method
for segmenting brain regions from nonbrain regions with a
combination of intensity thresholded mask and an additional
IEEE SIGNAL PROCESSING MAGAZINE [100] JANUARY 2008
operate both in a feedback mode for applications involving selfvariance mask to remove signals from the eye regions. For SVM
regulation and nonfeedback mode for applications involving
classification, images from each scan were represented as a vecbrain state detection (e.g., lie detection). In this section, we elabtor whose components were intensity values for each brain voxel
orate on the techniques used to date in identifying the feedback,
at that time. The experimental condition associated with each
in computing the feedback signal, and eventually in presenting
vector was represented as a scalar class label. The SVM algothe feedback to the subject.
rithm attempts to find a decision boundary as a separating
hyperplane to discriminate between the two class labels. Once
FEEDBACK IDENTIFICATION
the SVM model was determined from the training images, indefMRI-BCI can take advantage of the high spatial resolution and
pendent testing images were classified into the specified labels.
whole brain coverage of fMRI to derive feedback from specific
Percentage classification accuracy was reported as the ratio of
anatomical locations (ROIs) [14]. Feedback from circumscribed
number of correctly classified scans to the total number of
brain regions necessitates the delineation of the target ROI by
scans. To test this approach the authors used an experimental
anatomical landmarks or by identitask consisting of rapid button
fying functional activation elicited
press blocks that alternatively
in a functional localizer experiused the left or right portion of
WHILE THE MAJORITY OF WORK IN
ment by presenting the subject a
the visual display. During the
fMRI-BCI HAS USED CONVENTIONAL
stimulus or instructing the subject
training runs an arrow in the cenNEUORIMAGING METHODS OF
to perform a mental task. Motor
ter of the display pointed toward
UNIVARIATE ANALYSIS, THERE IS A
areas can be localized by overt finthe left or right target acted as the
GROWING INTEREST IN
ger tapping, covert movement
cue. During the subsequent testINCORPORATING MULTIVARIATE
imagery, and observation of moveing run, each acquired image was
METHODS OF PATTERN ANALYSIS
ment. Primary and higher visual
classified by the SVM model, and
USING MACHINE LEARNING
areas can be localized by presentthe arrow was updated in terms of
TECHNIQUES IN THE EMERGING
ing distinct visual stimuli. For
its position and orientation based
FIELD OF BRAIN STATE DETECTION.
example, the higher visual areas
on the classifier’s left or right
like fusiform gyrus (FFA) and
decision. With additional subjects,
parahippocampal place area (PPA)
task instructions were changed to
can be localized by presenting images of faces and houses, respecfurther examine pattern classification of mood, language, and
tively. Functional localizers can also be used to identify brain
imagined motor tasks. The authors concluded that real-time
areas involved in higher cognitive and affective processing such as
pattern classification of brain states using fMRI data is possible;
the anterior cingulated cortex [8] or the insula [13]. ROI is chosen
high prediction accuracies are attainable during sustained actiby drawing a rectangular area on the functional map computed by
vation; and stimulus feedback based on pattern classification can
the signal analysis software (e.g., TBV). To improve selection of
respond to changes in brain states much earlier than the timeROIs, functional maps could be co-registered with previously
to-peak limitations of the BOLD response. The above approach
acquired anatomical scans of the subject for accurate localization
is limited to two-class classification of brain states.
of ROI, when the region is too small to be located using the EPI
We have recently implemented in our fMRI-BCI a multiclass
images alone or too hard to localize consistently by functional
pattern classification system that offers the experimenter the
localizers (e.g., amygdala). In contrast to selecting circumscribed
flexibility of selecting either an SVM or a multilayer neural netbrain regions by the ROI method employing univariate analysis,
work classification algorithm [38]. Mourao-Miranda et al. [36]
pattern-based methods [37] are able to extract brain activity from
carried out a comparison of two methods, SVM and Fisher
spatially distributed regions that dynamically interact while the
Linear Discriminant classifier (FLD), for classifying multisubject
subject learns to regulate a motor, cognitive, or affective behavior.
data from an experiment involving a face matching and location
matching task. They demonstrated that SVM outperforms FLD
FEEDBACK COMPUTATION
in classification accuracy as well as in the robustness of the spaAfter the feedback signal is identified, further processing needs to
tial maps obtained. Further work needs to be carried out to rigbe carried out to arrive at a suitable representation of brain activorously compare the performance of existing pattern
ity to be presented as feedback. By conventional univariate methclassification approaches to assess their suitability and efficacy
ods, effective signal change from an ROI is usually computed as a
for fMRI brain state classification.
difference of the average BOLD signal between the activation
block and the baseline block. Specificity of the signal can further
SIGNAL FEEDBACK
be improved by designing a protocol that includes bidirectional
Training to self-regulate a brain activity can be implemented by
control; that is, both up and down regulation of the activity in
feedback of this specific brain signal [39]. In fMRI-BCI, feedback
the ROI. Studies have used differential feedback [14] between two
provides reward and information of the BOLD signal.
ROIs to subtract global signal changes. General effects of arousal
Contingent feedback following the response with a minimum
and attention caused by the demands of the task or the state of
lag and at a high probability improves learning. FMRI-BCI can
IEEE SIGNAL PROCESSING MAGAZINE [101] JANUARY 2008
the subject are thus canceled out leaving only the effects of
increasing or decreasing the signal. A potential problem related
to differential regulation of two ROIs is that subjects may learn
to regulate only one ROI while keeping the second ROI constant.
We have recently incorporated a correlation coefficient in the
computation of feedback of two ROIs to prevent the above undesirable effect. We used the following equation to compute the
magnitude of feedback when subjects were trained to increase
BOLD in ROI1 while decreasing BOLD in ROI2 simultaneously
(i.e., negative correlation of the two time-series):
Magnitude of feedback = (BOLDROI1 –BOLDROI2 )∗ (1–R),
(3)
where R is the correlation coefficient of BOLD time-series in the
two ROIs computed from a sliding window of past n (example,
n = 10) time points. If the subject learns to maintain a high BOLD
in ROI1 compared to ROI2 and a negative correlation of the two
ROIs he receives higher reward through feedback. However, the
subject receives lesser reward if the difference between the BOLD
values in the ROIs is negative, or if there is a positive correlation
between the BOLD values in the two ROIs, or both.
In designing the experimental protocol for fMRI-BCI, the
time constants of the hemodynamic response and the time
required for task switching need to be kept in mind. Most studies reported so far have used block designs with alternating rest
periods and regulation tasks. Duration for rest periods and regulation tasks is in the range of 15–60 s. Shorter periods could be
used for overt execution of motor tasks (like finger tapping) as
they can be started and stopped quickly. Longer periods need to
be used for mental imagery and emotional regulation tasks. A
delay between brain activation and feedback of that activation in
range of seconds is inevitable due to the hemodynamic delay
and delay in image acquisition and processing. Hemodynamic
coupling introduces a delay between neuronal activation and the
BOLD signal changes [16], with the onset of signal changes
delayed by around 3 s and the peak signal change by 6 s. Due to
signal acquisition and processing an additional delay of around
1.5 s is introduced. Fortunately, the delay in acquisition allows
for temporal averaging of fMRI data to increase SNR and hence
the reliability of feedback [14].
The benefits of the high spatial resolution of the fMRI is lost
if the feedback signal is obtained from too large an ROI that
encompasses multiple disparate areas involved with the target
function. By averaging signals from individual voxels in a large
ROI, the spatial information across the circumscribed region
would be lost. To overcome this problem and to be able to compare results across subjects, we used small rectangular ROIs of a
uniform size of 6x6 voxels (36 voxels) in our studies on self-regulation of interior insula [13]. Another advantage of using
smaller ROIs is in reducing the computational bottleneck in
processing statistical information in a reduced number of voxels. The newly emerging real-time pattern-based feedback [37] is
able to circumvent the necessity of specifying anatomical ROIs,
thus providing a flexible method to account for inter-subject
variability in brain size, shape, and neural network.
FEEDBACK PRESENTATION
Although many feedback modalities (verbal, visual, auditory,
olfactory, and tactile) are possible, visual feedback has been the
most frequently used method. A variety of visual stimuli have
been employed by different researchers to indicate the required
level of activation at different time points. Scrolling time-series
graphs and curves of BOLD activation of the ROI provide immediate information to the subject [8], [9]. Yoo and Jolesz [6] used
functional maps of the brain as feedback. Sitaram et al. [40] introduced the thermometer type of feedback (see Figure 1 for an illustration of different types of feedback) that shows varying levels of
ROI activity as changing bars of a graphical thermometer. Positive
BOLD activity with respect to baseline activity is shown in one
color (red) to differentiate from negative BOLD activity (blue). We
introduced virtual reality (VR) for feedback [40], [41], which provides a playful and engaging environment to encourage the subject to continue with self-regulation training. In a recent
application of fMRI-BCI, we used video-based feedback to train
stroke patients to self-regulate ventromedial premotor cortex
[38]. Laconte et al. [37] implemented a visual feedback updated
from results of real-time pattern classification of left hand and
right finger tapping (also see section on “Multivariate Analysis”).
A target stimulus located about 10o to the left or right of the center indicated the fingers (left or right) to be tapped. The feedback
consisted of an arrow in the central visual field oriented toward
the target. The arrow position and orientation were updated after
classification of the images as left or right from the volumes collected for the previous 2 s. Based on the brain-state classification,
the arrow either continued in the current orientation or flipped
its left-right orientation. After 30 s, the target position was alternated and the arrow was recentered, pointing to the new target.
For researchers interested in getting into the field of fMRIBCI, either for further development of the methodology, for
clinical applications, or for neuroscientific research, it is necessary to acquire the following essential components of the system: 1) a real-time signal acquisition system developed
specifically for the scanner in use (similar to the RTExport software developed in our laboratory to work with Siemens MR
scanners), 2) a real-time signal preprocessing and analysis system (similar to the TBV software available from Brain
Innovations (Maastricht, Netherlands), and 3) a BCI program
that is able to compute and present feedback to the subject and
additionally has the capability for real-time pattern classification. All the above subsystems are computationally intensive,
and hence it is advisable to install and execute them on dedicated workstations connected in a local area network.
APPLICATIONS
CLINICAL REHABILITATION AND TREATMENT
The primary application of fMRI-BCI and similar real-time fMRI
systems as reported in several studies so far [6]–[13], [40] has
IEEE SIGNAL PROCESSING MAGAZINE [102] JANUARY 2008
Session 1
8.00
R
830
814
1.98
799
t(133)
1
p < 0.049834
21
41
61
81
101
121
41
61
81
101
121
Session 3
8.00
R
830
810
3.84
790
t(133)
1
p <0.000187
21
(a)
R
L
10
5
0
Sess 1
Sess 2
Sess 3
Sess 4
(b)
[FIG3] Results of emotional self-regulation of right anterior insula [13]. (a) Single subject statistical maps (left) and BOLD time-courses
(right) of the right anterior insula in the first (top) and in the last session (bottom). The selected region-of-interest (ROI) is delineated by
the green box. The time courses of BOLD response in the right anterior insula during session 1 and 3 of self-regulation training are
shown in the right top and bottom panels, respectively. (b) Random effects analysis on the experimental group showed an increased
BOLD magnitude in the right anterior insula from training session 1 to 4. (Reprinted with permission from Neuroimage).
been toward training healthy subjects and patients to volitionally
modulate a circumscribed brain region, often with the aim to
study behavioral effects. A compelling clinical application of
fMRI-BCI for chronic pain was demonstrated by de Charms et al.
[12]. The objective of the study was to investigate whether training subjects to alter activity in the rostral part of the anterior cingulate cortex (rACC), previously implicated in pain processing,
would affect perception of pain. The authors trained 16 healthy
volunteers and 12 chronic pain patients to control activity in the
rACC by real-time feedback of BOLD activity in this region. The
authors reported that if subjects deliberately induced increases or
decreases of rACC BOLD activation, there was a corresponding
change in the perception of pain caused by an applied noxious
thermal stimulus. Control experiments showed that this effect
was not observed after training without real-time fMRI feedback
or using feedback from a different region or sham feedback
derived from a different subject. Chronic pain patients were also
trained to control activation in rACC and reported decreases in
the level of ongoing pain after training.
In a recent study, we investigated [13] whether healthy subjects (n = 15) could voluntarily gain control over the insula, an
area of the brain implicated in emotion processing. All participants were able to successfully regulate the BOLD signal in the
right anterior insula within three training sessions of 4 min
each using the thermometer-type of feedback (Figure 3).
Training resulted in significantly increased activation in the
IEEE SIGNAL PROCESSING MAGAZINE [103] JANUARY 2008
DECODING BRAIN STATES
target region across sessions. Two different control conditions
Recent advances in neuroimaging and multivariate patternused to assess the effects of nonspecific feedback and mental
based techniques applied to neuroimaging data have shown
imagery demonstrated that the training effect was not due to
that it is possible to decode an individual’s conscious brain
unspecific activations or nonfeedback-related cognitive stratestates based on noninvasive measurements of brain activity
gies. Control groups undergoing these control conditions
[27]. Such methods for “brain reading” from fMRI data have
showed no enhanced activation across the sessions, which
been used to study visual perception. Haynes et al. [27] have
showed the effect of anatomical specificity of feedback and regudemonstrated that perceptual fluctuations during binocular
lation. In an extension to this study, we investigated the behavrivalry can be dynamically decoded from fMRI signals in highly
ioral effects of volitional modulation of left anterior insula [1].
specific regions of the early visual cortex. The authors trained a
The experimental condition resulted in a more negative emopattern classifier to distinguish between distributed fMRI
tional valence rating of fear-evoking pictures during insula BOLD
response patterns related to each monocular percept. From a
increase only. The effect is not only area specific but also valence
new test dataset, the classifier could predict the changing perspecific, leading to aversive pictures being rated more negative
cepts with high accuracy. The same approach can be extended
during voluntary up-regulation of insula than during the downto other types of mental states, such as covert attitudes and lie
regulation. Recently, with fMRI-BCI we assessed whether psychodetection. Recently, an fMRI paradigm was reported for studypathic subjects could be trained to self-regulate left anterior
ing volitional brain activity in noncommunicative brain injured
insula [42]. All four psychopathic subjects who have been trained
patients [44]. We anticipate the
until now in this study have
application of pattern classificalearned to regulate their left antefMRI-BCI CAN OPERATE BOTH
tion techniques in combination
rior insula after two to three days
IN A FEEDBACK MODE FOR
with such a paradigm to detect
of training, each day consisting of
APPLICATIONS INVOLVING
the presence of awareness in
four feedback runs, encouraging
SELF-REGULATION AND
these patients. Based on an
the further application of this
NONFEEDBACK MODE FOR
extension of our fMRI-BCI system
method to treatment. A potential
incorporating multiclass pattern
outcome of such investigations is
APPLICATIONS INVOLVING BRAIN
classification, we have recently
the development of methods for
STATE DETECTION.
been able to distinguish with
clinical rehabilitation, such as for
75% average accuracy four emoovercoming movement disabilities
tional states from fMRI signals elicited during the presentation
due to stroke, easing chronic pain, treating emotional disorders
of emotional pictures from the International Affective Picture
such as depression and anxiety, and other neurological problems
System (IAPS) [38]. Future studies may incorporate such detecsuch as psychopathy, social phobia, and addiction by alleviating
tion methods for neurofeedback rehabilitation after stroke and
the effect of abnormal brain activity [1].
for the treatment of emotional disorders.
NEUROSCIENTIFIC RESEARCH
SUMMARY
There are two general approaches in neuroscience for studying
Brain-computer interfaces based on fMRI enable real-time feedthe interaction between brain and behavior. The first category
back of circumscribed brain regions to learn volitional regulainvolves the manipulation of the neural substrate and the obsertion of those regions. This is an emerging field of intense
vation of behavior as a dependent variable. The effects of direct
research, with potential for multiple applications: neuroscienstimulation and lesions of brain tissue are studied with this
tific research in brain plasticity and reorganization, movement
approach. The second approach is less invasive in nature,
restoration due to stroke, clinical rehabilitation of emotional
manipulating behavior as an independent variable and neural
disorders, quality assurance of fMRI experiments, and teaching
function as a dependent variable, constituting the neuroelectric
functional imaging. We presented a general architecture of an
and neuroimaging approaches. fMRI-BCI is in a unique position
fMRI-BCI, with descriptions of each of its subsystems, and facto combine both approaches. On one hand, learned voluntarily
tors influencing their performance. We have attempted to
changed activity in a particular region of the brain can be
describe and compare a variety of approaches toward signal
regarded as an independent variable and changes in behavior
acquisition, preprocessing, analysis, and feedback.
can be observed. On the other hand, it realizes also the neuTechnological advancement in higher-field MRI scanners, data
roimaging perspective as it incorporates experimental paraacquisition sequences and image reconstruction techniques,
digms with neural response as the dependent variable.
preprocessing algorithms to correct for artifacts, more intelliFMRI-BCI can be used to study neuroplasticity [40], emotional
gent and robust analysis and interpretation methods, and faster
processing [13], pain [12] and language processing [43]. In
feedback and visualization technology are anticipated to make
fMRI-BCI, real-time processing and analysis allows functional
fMRI-BCI widely applicable. Examples of such future developmaps to be generated during the course of the experiment, thus
ments are: new MR sequences to correct for magnetic inhomoenabling rapid piloting of functional localizers of brain areas
geneity differences and to improve SNR; real-time artifact
prior to the main experiment [2].
IEEE SIGNAL PROCESSING MAGAZINE [104] JANUARY 2008
removal algorithms; connectivity analysis incorporating a
whole network of neural activity; support vector and other
machine learning and pattern classification approaches; independent component analysis for extracting BOLD response of
interest; and augmented virtual worlds for more immersive
feedback. FMRI-BCI could potentially be used for training
patients to learn self-regulation of specific brain areas for transferring them later on to a more portable EEG-BCI system [45],
[46]. FMRI-BCI has the potential of establishing itself as a tool
for neuroscientific research and experimentation and also as an
aid for psychophysiological treatment.
ACKNOWLEDGMENTS
We gratefully acknowledge support from the Deutsche
Forschungsgemeinschaft (SFB 437/F1), the Marie Curie Host
Fellowship for Early Stage Researchers Training, the Wellcome
Trust, and the National Institutes of Health (NIH) who provided
the grants for this work.
AUTHORS
Ranganatha Sitaram (sitaram.ranganatha@uni-tuebingen.de)
completed his bachelor and master degrees in engineering in
India in 1990. From 1990 to 1992 he worked as a senior
research fellow at the Bhabha Atomic Research Centre, India, in
the field of robotics. From 1992–2004 he worked as a research
scientist in the Kent Ridge Digital Labs, Singapore, in various
fields including information and communication systems, artificial intelligence, knowledge systems, and machine learning
methods for telecommunication, intelligent transport systems,
and biomedical engineering. Since 2004 he has been working as
a research scientist and also pursuing a Ph.D. in cognitive neuroscience at the Institute of Medical Psychology and Behavioral
Neurobiology, University of Tübingen, Germany. His research
activities include building fMRI-BCI and NIRS-BCI systems and
neuroimaging studies on the effects of voluntary regulation of
brain regions, emotional disorders, and movement disabilities.
He is the author of four international granted patents, two international pending patents, and several publications.
Nikolaus Weiskopf (n.weiskopf@fil.ion.ucl.ac.uk) studied
physics at the University of Miami, Florida, and the University of
Tübingen, Germany, and graduated in 2000. He received his
doctoral degree from the Graduate School of Neural and
Behavioral Sciences and the International Max Planck Research
School, University of Tübingen, in 2004. From 2004–2006 he
was a postdoctoral fellow at the Wellcome Trust Centre for
Neuroimaging (WTCN), University College London, United
Kingdom. Since 2007, he has been the principal research fellow
and the head of the Physics Group at the WTCN. His research is
focused on the development of magnetic resonance imaging
techniques, in particular MR pulse sequences, MR image reconstruction, real-time analysis of functional MRI data, and multimodal imaging.
Andrea Caria (andrea.caria@uni-tuebingen.de) graduated
with a master of science in electronic and biomedical engineering at the University of Genoa, Italy. He received the Ph.D. in
cognitive science from the University of Trento, Italy, in 2006.
From 2001–2003 he was a research assistant at the Psychology
Department of the Royal Holloway College University of
London. From 2005–2007, he had a Marie Curie fellowship for
early stage research training in multimodal biomedical imaging
for research and clinical application at the University of
Tübingen, Germany. He is now a postdoc at the Institute of
Medical Psychology and Behavioral Neurobiology of the
University of Tübingen in collaboration with the Bernstein
Center for Computational Neuroscience, Freiburg. His research
is focused on neuronal decoding and plasticity by voluntary
modulation of brain signals using MEG and fMRI-based BCI and
advanced brain imaging methods.
Ralf Veit (ralf.veit@uni-tuebingen.de) studied psychology at
the University of Tübingen, Tübingen, Germany. He received the
diploma in 1992 and the Ph.D. in 1997. From 1992–1997, he
worked on psychological influences on cardiovascular disorders.
Since 1998, he has been a member of the Institute of Medical
Psychology and Behavioral Neurobiology, University of
Tübingen. He is a lecturer in medical psychology. His research
is focused on personality disorders, emotional regulation, and
neurofeedback using fMRI/EEG and peripheral measures.
Michael Erb (michael.erb@med.uni-tuebingen.de) studied
physics at the Universities of Karlsruhe and Tübingen,
Germany, and graduated in 1985. He pursued his Ph.D. studies
on artificial neural networks at the Max Planck Institute for
Biological Cybernetic in Tübingen from 1986–1990. After a
research visit at the Institute for Brain Research, University of
Düsseldorf, he was a research fellow at the Institute for
Neurophysics, University of Marburg, Germany. Since 1995, he
has been a research fellow at the Department of
Neuroradiology, University of Tübingen, performing fMRI studies on several topics. He is involved in building MR compatible
stimulation devices and programming MP pulse sequences and
analysis software.
Niels Birbaumer (niels.birbaumer@uni-tuebingen.de)
received the Ph.D. degrees in biological psychology, art history,
and statistics from the University of Vienna, Vienna, Austria, in
1969. From 1975–1993, he was full professor of clinical and
physiological psychology, University of Tübingen, Tübingen,
Germany. In 1986–1988, he was full professor of psychology,
Pennsylvania State University, University Park. Since 1993, he
has been professor of medical psychology and behavioral neurobiology with the Faculty of Medicine of the University of
Tübingen and professor of clinical psychophysiology,
University of Padova, Padua, Italy. Since 2002, he has been
director of the Center of Cognitive Neuroscience, University of
Trento, Trento, Italy. His research topics include neuronal
basis of learning and plasticity, neurophysiology and psychophysiology of pain, and neuroprosthetics and neurorehabilitation. He has authored more than 450 publications in
peer-reviewed journals and 12 books. Among his many awards
Dr. Birbaumer has received the Leibniz-Award of the German
Research Society (DFG), the Award for Research in
Neuromuscular Diseases, the Wilhelm-Wundt-Medal of the
IEEE SIGNAL PROCESSING MAGAZINE [105] JANUARY 2008
German Society of Psychology, and the Albert Einstein World
Award of Science. He is president of the European Association
of Behavior Therapy, a Fellow of the American Psychological
Association, a Fellow of the Society of Behavioral Medicine and
the American Association of Applied Psychophysiology, and a
member of the German Academy of Science and Literature.
REFERENCES
[1] N. Birbaumer and L.G. Cohen, “Brain-computer interfaces: Communication
and restoration of movement in paralysis,” J. Physiol., vol. 579, pp. 621–636,
2007.
[2] N. Weiskopf, R. Sitaram, O. Josephs, R. Veit, F. Scharnowski, R. Goebel, N.
Birbaumer, R. Deichmann, and K. Mathiak, “Real-time functional magnetic resonance imaging: methods and applications,” Magn. Reson. Imag., vol. 25, no. 6,
pp. 989–1003, 2007.
[3] R. Sitaram, H. Zhang, C. Guan, M. Thulasidas, Y. Hoshi, A. Ishikawa, K. Shimizu,
and N. Birbaumer, “Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain-computer interface,”
Neuroimage, vol. 34, no. 4, pp. 1416–1427, 2007.
[4] N.K. Logothetis, “The underpinnings of the BOLD functional magnetic resonance
imaging signal,” J Neurosci., vol. 23, no. 10, pp. 3963–3971, 2003.
[5] R.W. Cox, A. Jesmanowicz, and J.S. Hyde, “Real-time functional magnetic resonance
imaging,” Magn. Reson. Med., vol. 33, pp. 230–236, 1995.
[6] S.S. Yoo and F.A. Jolesz, “Functional MRI for neurofeedback: Feasibility study
on a hand motor task,” Neuroreport., vol. 13, no. 11, pp. 1377–1381, 2002.
[7] S. Posse, D. Fitzgerald, K. Gao, U. Habel, D. Rosenberg, G.J. Moore, and F. Schneider,
“Real-time fMRI of temporolimbic regions detects amygdala activation during singletrial self-induced sadness,” Neuroimage, vol. 18, no. 3, pp. 760–768, 2003.
[8] N. Weiskopf, R. Veit, M. Erb, K. Mathiak, W. Grodd, R. Goebel, and N.
Birbaumer, “Physiological self-regulation of regional brain activity using real-time
functional magnetic resonance imaging (fMRI): methodology and exemplary data,”
Neuroimage, vol. 19, no. 3, pp. 577–586, 2003.
[9] R.C. deCharms, K. Christoff, G.H. Glover, J.M. Pauly, S. Whitfield, and J.D. Gabrieli,
“Learned regulation of spatially localized brain activation using real-time fMRI,”
Neuroimage, vol. 21, no. 1, pp. 436–443, 2004.
[10] N. Weiskopf, K. Mathiak, S.W. Bock, F. Scharnowski, R. Veit, W. Grodd, R.
Goebel, and N. Birbaumer, “Principles of a brain-computer interface (BCI) based
on real-time functional magnetic resonance imaging (fMRI),” IEEE Trans Biomed.
Eng., vol. 51, no. 6, pp. 966–970, 2004.
[11] S.S. Yoo, T. Fairneny, N.K. Chen, S.E. Choo, L.P. Panych, H. Park, S.Y. Lee,
and F.A. Jolesz, “Brain-computer interface using fMRI: spatial navigation by
thoughts,” Neuroreport, vol. 15, no. 10, pp. 1591–1595, 2004.
[12] R.C. deCharms, F. Maeda, G.H. Glover, D. Ludlow, J.M. Pauly, D. Soneji,
J.D. Gabrieli, and S.C. Mackey, “Control over brain activation and pain learned
by using real-time functional MRI,” Proc. Natl. Acad. Sci. USA, vol. 102, no. 51,
pp. 18626–18631, 2005.
[13] A. Caria, R. Veit, R. Sitaram, M. Lotze, N. Weiskopf, W. Grodd, and N.
Birbaumer, “Regulation of anterior insular cortex activity using real-time fMRI,”
Neuroimage, vol. 35, no. 3, pp. 1238–1246, 2007.
[14] N. Weiskopf, F. Scharnowski, R. Veit, R. Goebel, N. Birbaumer, and K. Mathiak,
“Self-regulation of local brain activity using real-time functional magnetic resonance imaging (fMRI),” J. Physiol. Paris, vol. 98, no. 4–6, pp. 357–373, 2004.
[15] M.S. Cohen, “Real-time functional magnetic resonance imaging,” Methods,
vol. 25, pp. 201–220, 2001.
[23] R.M. Birn, J.B. Diamond, M.A. Smith, and P.A. Bandettini, “Separating respiratory-variation-related fluctuations from neuronal-activity-related fluctuations in
fMRI,” Neuroimage, vol. 31, pp. 1536–1548, 2006.
[24] O. Josephs, A.M. Howseman, K. Friston, and R. Turner, “Physiological noise
modelling for multi-slice EPI fMRI using SPM,” in Proc. 5th Annu. Meeting
ISMRM, Vancouver, Canada, 1997.
[25] P. van Gelderen, J.A. de Zwart, P. Starewicz, R.S. Hinks, and J.H. Duyn, “Realtime shimming to compensate for respiration-induced B0 fluctuations,” Magn.
Reson. Med., vol. 57, no. 2, pp. 362–368, 2007.
[26] C. Triantafyllou, R.D. Hoge, G. Krueger, C.J. Wiggins, A. Potthast, G.C.
Wiggins, and L.L. Wald, “Comparison of physiological noise at 1.5 T, 3 T and 7 T
and optimization of fMRI acquisition parameters,” Neuroimage, vol. 26, no. 1,
pp. 243–250, 2005.
[27] J.D. Haynes and G. Rees, “Decoding mental states from brain activity in
humans,” Nat. Rev. Neurosci., vol. 7, no. 7, pp. 523–534, 2006.
[28] D. Gembris, J.G. Taylor, S. Schor, W. Frings, D. Suter, and S. Posse,
“Functional magnetic resonance imaging in real time (FIRE): Sliding-window correlation analysis and reference-vector optimization,” Magn. Reson. Med., vol. 43,
no. 2, pp. 259–268, 2000.
[29] K.J. Friston, A.P. Holmes, J.B. Poline, P.J. Grasby, S.C. Williams, R.S.
Frackowiak, and R. Turner, “Analysis of fMRI time-series revisited,” Neuroimage,
vol. 2, no. 1, pp. 45–53, 1995.
[30] E. Bagarinao, K. Matsuo, T. Nakai, and S. Sato, “Estimation of general linear
model coefficients for real-time application,” Neuroimage, vol. 19, no. 2, pt. 1,
pp. 422–429, 2003.
[31] D.S.G. Pollock, A Handbook of Time-Series Analysis, Signal Processing and
Dynamics. San Diego, CA: Academic, 1999.
[32] T.M. Mitchell, R. Hutchinson, M.A. Just, R.S. Niculescu, F. Pereira, and X.
Wang, “Classifying instantaneous cognitive states from FMRI data,” in Proc.
AMIA Ann. Symp., 2003, Washington, D.C., pp. 465–469.
[33] S.M. Polyn, V.S. Natu, J.D. Cohen, and K.A. Norman, “Category-specific cortical activity precedes retrieval during memory search,” Science, vol. 310, no. 5756,
pp. 1963–1966, 2005.
[34] C. Davatzikos, K. Ruparel, Y. Fan, D.G. Shen, M. Acharyya, J.W. Loughead,
R.C. Gur, and D.D. Langleben, “Classifying spatial patterns of brain activity with
machine learning methods: Application to lie detection,” Neuroimage, vol. 28,
no. 3, pp. 663–668, 2005.
[35] K.A. Norman, S.M. Polyn, G.J. Detre, and J.V. Haxby, “Beyond mindreading: multi-voxel pattern analysis of fMRI data,” Trends Cogn. Sci., vol. 10,
no. 9, pp. 424–430, 2006.
[36] J. Mourao-Miranda, A.L. Bokde, C. Born, H. Hampel, and M. Stetter,
“Classifying brain states and determining the discriminating activation patterns:
Support vector machine on functional MRI data,” Neuroimage., vol. 28, no. 4,
pp. 980–995, 2005.
[37] S.M. Laconte, S.J. Peltier, and X.P. Hu, “Real-time fMRI using brain-state classification,” Hum. Brain Mapp., vol. 28, no. 10, pp. 1033–1044, 2006.
[38] R. Sitaram, “fMRI brain-computer interfaces,” presented at 15th Annu. Conf.
Int. Society for Neurofeedback and Research, Current Perspectives in
Neuroscience: Neuroplasticity & Neurofeedback, San Diego, CA, 2007.
[39] M.S. Schwartz, Biofeedback: A Practitioner’s Guide. New York: Guilford Press,
1995.
[40] R. Sitaram, A. Caria, R. Veit, T. Gaber, A. Kuebler, and N. Birbaumer, “Realtime fMRI based brain-computer interface enhanced by interactive virtual worlds,”
presented at 45th Annu. Mtg. Society for Psychophysiological Research, Lisbon,
Portugal, 2005.
[16] C.T.W. Moonen and P.A. Bandettini, Functional MRI. Berlin: SpringerVerlag, 2000.
[41] R. Sitaram, A. Caria, V. Veit, K. Uludag, T. Gaber, A. Kübler, and N. Birbaumer,
“Functional magnetic resonance imaging based BCI for neurorehabilitation,” presented at 3rd Int. Brain-Computer Interface Workshop and Training Course, 2006.
[17] N. Weiskopf, C. Hutton, O. Josephs, R. Turner, and R. Deichmann, “Optimized
EPI for fMRI studies of the orbitofrontal cortex: Compensation of susceptibilityinduced gradients in the readout direction,” Magma, vol. 20, no. 1, pp. 39–49, 2007.
[42] N. Birbaumer and R. Sitaram, “BCI-regulation of neuronal substrates of emotions,” presented at Neuroscience 2006, 36th Meeting Society for Neuroscience,
Atlanta, GA, 2006.
[18] P.A. Bandettini, E.C. Wong, R.S. Hinks, R.S. Tikofsky, and J.S. Hyde, “Time
course EPI of human brain function during task activation,” Magn. Reson. Med.,
vol. 25, no. 2, pp. 390–397, 1992.
[43] G. Rota, R. Sitaram, R. Veit, N. Weiskopf, N. Birbaumer, and G. Dogil, “fMRIneurofeedback for operant conditioning and neural plasticity investigation: A
study on the physiological self-induced regulation of the BA 45,” in Proc.
Cognitive Neuroscience Conf., San Francisco, CA, 2006.
[19] K. Mathiak and S. Posse, “Evaluation of motion and realignment for functional magnetic resonance imaging in real time,” Magn. Reson. Med., vol. 45,
pp. 167–171, 2001.
[20] M. Tremblay, F. Tam, and S.J. Graham, “Retrospective coregistration of functional magnetic resonance imaging data using external monitoring,” Magn Reson
Med., vol. 53, pp. 141–149, 2005.
[44] M. Boly, M.R. Coleman, M.H. Davis, A. Hampshire, D. Bor, G. Moonen, P.A.
Maquet, J.D. Pickard, S. Laureys, and A.M. Owen, “When thoughts become action:
An fMRI paradigm to study volitional brain activity in non-communicative brain
injured patients,” Neuroimage, vol. 36, no. 3, pp. 979–992, 2007.
[21] H.A. Ward, S.J. Riederer, R.C. Grimm, R.L. Ehman, J.P. Felmlee, and C.R. Jack
Jr., “Prospective multiaxial motion correction for fMRI,” Magn. Reson. Med., vol.
43, no. 3, pp. 459–469, 2000.
[45] T. Hinterberger, N. Weiskopf, R. Veit, B. Wilhelm, E. Betta, and N. Birbaumer,
“An EEG-driven brain-computer interface combined with functional magnetic resonance imaging (fMRI),” IEEE Trans. Biomed. Eng., vol. 51, no. 6, pp. 971–974,
2004.
[22] G.H. Glover, T.Q. Li, and D. Ress, “Image-based method for retrospective correction of physiological motion effects in fMRI: RETROICOR,” Magn. Reson. Med.,
vol. 44, no. 1, pp. 162–167, 2000.
[46] T. Hinterberger, R. Veit, U. Strehl, T. Trevorrow, M. Erb, B. Kotchoubey, H. Flor,
and N. Birbaumer, “Brain areas activated in fMRI during self-regulation of slow cortical potentials (SCPs),” Exp Brain Res., vol. 152, no. 1, pp. 113–122, 2003.
IEEE SIGNAL PROCESSING MAGAZINE [106] JANUARY 2008
[SP]