Multi-voxel Pattern Analysis
(MVPA) and “Mind Reading”
By: James Melrose
Basic Visual Pathways
function Magnetic Resonance
Imaging (fMRI)

Based on the idea
that oxygenated
hemoglobin has
slightly different
magnetic properties
than
nonoxygenated
hemoglobin and
that areas of the
brain being used
will require this
oxygen
fMRI cont.

Hydrogen protons within the brain align
with the huge magnet in the scanner,
radio waves are then aimed towards the
head causing the rotation of the Hydrogen
to misalign, the atoms with oxygen
containing hemoglobin realign at a
different rate than atoms without oxygen,
the realignment sends out a signal that
can then be detected by the detector coil.
MVPA as alternative to
traditional fMRI



Instead of focusing on single voxels (the fMRI
version of a pixel), MVPA uses patternclassification algorithms applied to multiple
voxels to decode the patterns of activity
Due to its increased sensitivity, it makes it
easier to get statistically significant brain
activation
Also reduces the problem of noise when looking
at individual voxels
MVPA

4 Steps:




Feature selection- decide which voxels you want to
include in analysis
Pattern assembly- “involves sorting the data into
discrete brain patterns corresponding to the pattern
of activity across the selected voxels at a particular
time in the experiment”
Classifier training- feed these patterns into
multivariate pattern classification algorithm that maps
between voxel patterns and experimental conditions
Generalization testing- test to see if the trained
classifier algorithm can determine the correct
experimental condition associated with activation
pattern
How to develop MVPA paradigm
Kamitani and Tong

Perception of edge orientation
can be detected from fMRI
using MVPA and a 3x3x3 matrix
of voxels in early visual areas
(V1 & V2)
Kamitani and Tong cont.

The detection of edge orientation can be
extended when subjects are told to focus
on one of two line orientation in a grid
Haxby et al.
Had subjects look at pictures and line
drawings of faces, cats, man-made
objects, and control nonsense objects.
 The pattern of response in object
selective ventral temporal cortex correctly
identified the category being viewed in
96% of comparisons.

Haxby et al., cont.
Haxby et al., cont.

Ability of both photographs and line
drawings to elicit the same accuracy of
prediction, the representation being
detected must be of a higher level
representation of the image rather than
simple feature detection
What else has MVPA done?

Mind-reading has been accomplished in the
domains of:






Direction of dots
Picture vs. sentence
Ambiguous vs. non-ambiguous sentences
Category of viewed word
Which of 3 categories thinking of
Lying
Where can this go?
The implications of this research could
potentially lead to researchers being able
to determine what a person is looking at
or thinking about entirely based on their
pattern of brain activation
 Perhaps even one day having subjects
fall asleep in the scanner and be able to
tell what the subject is dreaming of

References



Haxby, J.V. et al. (2001) Distributed and overlapping
representations of faces and objects in ventral temporal
cortex. Science 293, 2425-2429
Kamitani, Y. and Tong, F. (2005) Decoding the visual
and subjective contents of the human brain. Nat.
Neurosci. 8, 679-685
Norman, K.A.; Polyn, S.M.; Detre, G.J.; Haxby, J.V.
(2006) Beyond mind reading: multi-voxel pattern
analysis of fMRI data. Trends in Cognitive Sciences. 10
(9), 424-430.