MEG Correlates of Visual Perception:
The M170 Response to Degraded Face and Building Images
jMASSACUSE1fiINSTITUTE
by
(y
o TEiCHNCLOGY
Valerie S. Morash
NOV 13 2008
S.B., E.E. M.I.T., 2007
Submitted to the Department of Electrical Engineering and Computer Science
in Partial Fulfillment of the Requirements for the Degree of
Master of Engineering in Electrical Engineering and Computer Science
at the Massachusetts Institute of Technology
May 2008
Copyright 2008 M.I.T. All rights reserved.
Author
Depirtment of Electrical Engineering and Computer Science
May 08, 2008
-
Certified by_______________
C Pawan
linha,Associate
*r&"
Professor
Supervisor
Accepted by
Arthur C. Smith
Professor of Electrical Engineering
Chairman, Department Committee on Graduate Theses
BARKER
1
2
Table of Contents
M otivation for face research ...............................................................................................
5
Knowing how the brain processes faces is valuable to scientists, clinicians, and industry........5
The MI 70 represents at least one step in the face processing neural mechanism..................6
Background on M EG................................................................................................................8
M EG measures the magnetic fields associated with neural activity.......................................8
There are two types of M EG sensors: magnetometers and gradiometers...............................8
Interpretation of M EG data depends on the sensor type......................................................
9
Magnetic field patterns and ECDs provide information about which brain areas are active.... 10
M EG is a fine method for fusiform gyrus research ................................................................
Background on the M 170 ....................................................................................................
11
12
The Ml 70 is reliably over occipitotemporal cortex about 170 ms after image presentation.... 12
The M l 70 is face-specific and face-generalizing...............................................................
13
The M 170 can be modulated by stimulus changes.............................................................
14
The M170 may or may not be related to the N170 and other face-specific neural signals....... 15
Pro sopagnosics do not always have or lack an M170, the elderly have a reduced M170 ........ 16
The M 170 reflects a beginning stage of face identification................................................
17
Experiment..............................................................................................................................18
Intro du ction ..........................................................................................................................
18
M eth o d s................................................................................................................................2
0
Data Co llection .................................................................................................................
20
MEG Data Noise Reduction .........................................................................................
22
Selecting SOIs: Sensors that displayedan M 1 70 ..........................................................
23
Comparingthe M 1 70 Sensor TopographiesforBuildings and Faces..............................
24
Examining the M1 70 Amplitude and Latency Across Conditions ....................................
25
R esu lts ..................................................................................................................................
27
Comparingthe M l 70 Generatorfor Buildings and Faces .............................................
27
Examining the M l 70 Amplitude andLatency Across Conditions ....................................
28
D iscussio n ............................................................................................................................
32
Contributions ..........................................................................................................................
34
References ...............................................................................................................................
35
3
4
Motivation for face research
Knowing how the brain processes faces is valuable to scientists, clinicians,and industry
How does seeing a face eventually lead to recognition: e.g. "that's Jill." Many reports (including
this one) are interested in this question, due to its intrinsic scientific appeal and application-based
benefits.
From a science perspective, there is a natural desire to know how the brain does something it is
good at: recognize faces. Humans are extremely adept at discriminating between faces, and there
is evidence for specialized face-processing brain machinery (Kanwisher 2000). Additionally,
knowing how the brain recognizes faces provides insight into how the brain generally works. It
lays down some of the operations the brain is able and likely to perform.
From a clinical perspective, knowing the steps the brain takes to recognize faces provides an
opportunity to ask, at which step do things go wrong for people with face processing disorders,
such as prosopagnosia and autism? Pinpointing a disorder's defective step illuminates the extent
of the disorder, and provides a straightforward test for diagnosis (this step is fine, but the next
step is affected). If this test can be done with non-invasive measurement on young children, it
will provide an early diagnosis leading to preemptive treatment.
From an industry perspective, computerized face identification systems are wanted for airports
and casinos to catch unwanted trespassers. Computer scientists have been working diligently to
produce such a product, but the current systems perform far worse than normal people (Willing
2003; Zhao et al. 2003). A possible solution is to create a computer system that copies the
5
human brain. Theoretically, a computer using the same steps a brain uses to recognize faces will
be just as good as a person. This provides strong motivation for discovering the "neural
algorithm" underlying face processing.
The Ml 70 represents at least one step in the face processing neural mechanism
Previous research has uncovered the basic route through the brain subserving face recognition
(for a review see Haxby et al. 2002). Major structures have been identified, and their functions
have been proposed (Fig. 1). However, further refinement of the pathway is needed. Namely,
the major structures need to be better characterized, and non-major structures need to be
identified.
Figure 1. Flow chart of the neural steps involved in face processing. Note the role and context of the
lateral fusiform gyrus, the neural area responsible for the M170. Reproduced from Haxby et al. 2002.
Intraparietal Sulcus
Spatially-directed attention
Superior Temporal Sulcus
Changeable aspects of faces
/
Auditory Cortex
Prelexical speech perception
Perception of eye gaze, expression,
lip movement
I
Inferior Occipital Gyri
Early perception of
facial features
Lateral Fusiform Gyrus
Invariant aspects of faces
Amygdala, Insula,
Limbic System
Emotion processing, emotional
response
Perception of unique identity
I
Anterior Temporal
Personal identity, name,
biographical information
I
6
I
Researchers have established that one of the important brain areas for face recognition, the
lateral fusiform gyrus (AKA the fusiform face area, FFA), produces a neural signal called the
MI 70. The MI 70 is a signal released from the brain about 170 milliseconds after viewing an
image, and is measured with magnetoencephalography (MEG).
The remainder of this report will focus on what neural step the Ml 70 may reflect. It will include
a comprehensive review of past M170 research (preceded by some important facts about MEG),
and a new experiment addressing two pivotal questions:
1)
Why does the M170 respond to buildings if it is face-selective?
2) Does the M170 reflect categorization (that's a face or not) or identification (that's Jill's
face)?
7
Background on MEG
Magnetoencephalographic principles and techniques can be both complex and obscure. This is
unfortunate, since they must be considered when comparing M170 studies, and MEG to nonMEG results. In this section, I present some of the MEG underpinnings, which are necessary to
comprehend M170 results.
MEG measures the magneticfields associatedwith neural activity
Fundamentally, MEG measures the magnetic fields surrounding weak net currents in the brain.
These currents arise from the post-synapses of thousands of pyramidal neurons firing in concert
(Himihiinen et al. 1993).
MEG is more sensitive to currents parallel/tangential to the skull, than currents orthogonal/radial
to the skull, due to external field cancellation by volume/return currents in the orthogonal case
(Hmiliinen et al. 1993). This sensitivity is opposite to that for electroencephalography (EEG),
which measures the electric fields associated with neural currents. Although MEG is blind to
purely radial currents, it is probabilistically unlikely for this to occur in the brain. Therefore, a
tangential component of most neural sources can be detected with MEG (Lueschow et al. 2004).
There are two types of MEG sensors: magnetometers and gradiometers
Both magnetometers and gradiometers make measurements of the magnetic field's radial
component. Magnetometers measure the magnetic field strength, whereas gradiometers measure
the derivative of the field (which can be used to approximate the field strength). The magnetic
8
field derivative falls off faster with distance than the field itself, making gradiometers less
sensitive to external noise and other distant magnetic generators (Clarke and Braginski 2003).
Therefore, gradiometers are often preferred for neural recordings, and magnetometers for
ambient field correction or as detectors of deep neural activity (Tarkiainen et al. 2002).
Unlike magnetometers, gradiometers come in many flavors, which differ in derivative order and
reference direction. The gradiometer order refers to the specific field derivative the sensor
detects (e.g. a second-order gradiometer is sensitive to the magnetic field's second derivative).
Radial/axial gradiometers measure the derivative with respect to the radial direction, and
planar/tangential gradiometers measure the derivative with respect to a direction orthogonal to
radial, tangential to the skull (Clarke and Braginski 2003).
Interpretationof MEG data depends on the sensor type
With a magnometer and radial gradiometer, a neural current will be measured as a magnetic sink
and source, flanking the area directly above the current. Only one sink/source may be detected if
the other falls outside the sensor range. With a planar gradiometer, a neural current will be
measured as a single sink or source directly above the current. For all types of sensors, the signal
will be strongest closest to the current, and the pattern produced is referred to as a "dipole
pattern."
Magnetic sinks are commonly denoted with negative amplitudes, and sources with positive
amplitudes. For magnetometers and radial gradiometers, but not planar gradiometers, equivalent
currents in opposite hemispheres will produce magnetic fields with opposite polarity. For
9
example, a current in the right hemisphere may produce a magnetic sink where the equivalent
current in the left hemisphere would produce a magnetic source (Sams et al. 1997; Harris et al.
2005).
Magneticfield patterns and ECDs provide information about which brain areas are active
Examining magnetic field patterns can reveal differences in field generator locations (which
brain area created the dipole pattern). Comparing magnetic iso-contours, and maximum M170
sensor locations are both adequate for this purpose (Watanabe et al. 1999; Taylor et al. 2001).
In addition to this method, researchers can "back solve" the neural current for each hemisphere,
which they refer to as an equivalent current dipole (ECD). The ECD is the current most likely to
have created the magnetic field pattern. This process collapses the data across multiple sensors,
acting like a spatial filter to reduce noise. It also provides insight into which neural areas are
responsible for the magnetic signals (Tarkiainen et al. 2002).
Sometimes more than one dipole is solved per hemisphere, but more often than not, only one
dipole is solved per dipole pattern, even though it is likely that more than one neural area's
activity created that pattern (Himiliinen et al. 1993). When activity from several neural areas is
modeled as a single dipole, the dipole represents those areas' center of gravity (Sams et al.
1997). The presence of a single dipole pattern is not evidence that only one neural site is active,
because the simultaneous activity from several neural areas, even if they are relatively
widespread, will typically produce such a pattern (Swithenby et al. 1998). Dipole localization
can be worsened by using too few trials, artifacts, or poor subject task (Defike et al. 2007).
10
MEG is a fine method for fusiform gyrus research
Using MEG to investigating the role of the fusiform gyrus has some distinct advantages. It is a
good method for detecting fusiform activity, since the fusiform gyrus currents are mostly
tangential. Additionally, MEG is sensitive to neural variation on the order of milliseconds. This
is its greatest advantage over fMRI and PET, which have sensitivities on the order of seconds
(Watanabe et al. 1999). EEG has a similar time resolution as MEG, but poorer spatial resolution.
In favorable conditions, MEG can discriminate neural sources on the order of millimeters (Lu et
al. 1991). Unfortunately, the skull and scalp tissues smear emerging electric fields, reducing
EEG's spatial resolution to the order of centimeters (Sams et al. 1997).
11
Background on the M170
Lu et al first detected the face-specific M170 response in 1991 (face-specific refers to the
signal's preferential, but not exclusive response to faces). The group was inspired by the facespecific neural potential (called the N 170), to search for a similar face-specific magnetic signal.
While recording MEG, they showed subjects many black and white images of faces, birds, and
simple drawings (a square and a circle). As is now convention in M170 studies, the MEG data
were averaged across trials with the same image type, so that average signal characteristics could
be examined. In all of their subjects, Lu et al. found a magnetic peak, about 150 ms after image
presentation, in the region of the occipitotemporal junction in both hemispheres. The peak was
larger to faces than birds, and larger to birds than simple drawings. It was not affected by
stimulus duration or luminance; and was qualitatively similar across hemispheres (Lu et al.
1991).
Since its discovery, the M170 has been examined in roughly 30 published reports. Principally,
these reports have established the M170 response characteristics, relationship to other facespecific neural signals, and clinical relevance. Finally, there is a preliminary understanding of
the M170's neural purpose.
The M1 70 is reliably over occipitotemporalcortex about 170 ms after image presentation
Overall, the M170 is extremely reliable in terms of its generation and face-specificity. In most
studies, it is reported for all subjects, even in studies with many participants (Harris et al. 2005).
However, as can happen with any neural signal, there are some subjects who lack an M170, and
others whose M170 is not face-specific (Sams et al. 1997; Watanabe et al. 1999).
12
The M170 location is widely established to be over occipitotemporal cortex (Lu et al. 1991;
Watanabe et al. 1999; Taylor et al. 2001; Tarkiainen et al. 2002; Lueschow et al. 2004). A large
amount of ECD evidence suggests that it is generated within, or near the fusiform gyrus (Halgren
et al. 1995; Sams et al. 1997; Swithenby et al. 1998; Sato et al. 1999; Kato et al. 2004;
Tanskanen et al. 2005; Barrie et al. 2006; Deffke et al. 2007; Schweinberger et al. 2007). The
neural area responsible for the MI 70 has a graded response to images, instead of an all-ornothing type response (Tanskanen et al. 2007).
Some researchers find the M170 stronger and more reliable over the right hemisphere (Halgren
et al. 1995; Marinkovic et al. 1995; Swithenby et al. 1998; Watanabe et al. 1999; Tarkiainen et
al. 2002). However, others find it to be equal over both hemispheres, or stronger over the left
hemisphere (Lu et al. 1991; Sams et al. 1997; Liu et al. 2000; Defike et al. 2007).
The Ml 70 response to faces can peak within a wide time window. Typically, it occurs between
160-180 milliseconds after image presentation; but has been reported as early as 125
milliseconds and as late as 220 milliseconds (Schweinberger et al. 2007; Tanskanen et al. 2007).
The Ml 70 is face-specific and face-generalizing
Of all the M170 characteristics, most notable is its face-specificity. The M170 is consistently
larger, and sometimes earlier to images of faces than plants, animals, face parts, body parts,
objects, houses, simple shapes, and meaningless scrambled stimuli (Lu et al. 1991; Halgren et al.
1995; Marinkovic et al. 1995; Sams et al. 1997; Swithenby et al. 1998; Watanabe et al. 1999;
13
Halgren et al. 2000; Liu et al. 2000; Taylor et al. 2001; Liu et al. 2002; Harris et al. 2005; Xu et
al. 2005; Barrie et al. 2006). This specificity is restricted to faces, and does not extend to images
of expertise objects (Xu et al. 2005).
The face M170 is smaller when it is preceded by other face or house images. This adaptation is
more severe when the preceding image was a face, insensitive to whether the face was a photo,
Mooney face, or face drawing (Harris and Nakayama 2007). There is extra adaptation when the
preceding image was of the same face as the second, however this effect dissipates if the two
faces were from different perspectives (Ewbank et al. 2008).
The M170 is not only face-specific, but also face-generalizing. Human front-view faces, human
profile faces, animal faces, cartoon faces, and face-like stimuli all elicit stronger M I70s than
non-faces. Some have found face-like and face M170s to be indistinguishable, but others have
found that human-face M170s are larger (Marinkovic et al. 1995; Halgren et al. 2000; Liu et al.
2000; Downing et al. 2001; Tarkiainen et al. 2002; Kato et al. 2004; Schweinberger et al. 2007).
The M1 70 can be modulated by stimulus changes
A large number of reports have been dedicated to what parameters do and do not affect the
M170. The M170 is not affected by low-level image characteristics, such as luminance, face
color, gender, or expression (Lu et al. 1991; Marinkovic et al. 1995). It is also not affected by
stimulus duration (when longer than 100 milliseconds), attention, or by what task the subject is
performing (Marinkovic et al. 1995; Swithenby et al. 1998; Streit et al. 1999; Lueschow et al.
2004). The M170 for faces is delayed and decreased by degradation and negation of the image
14
(Linkenkaer-Hansen et al. 1998; Tarkiainen et al. 2002; Tanskanen et al. 2005; Itier et al. 2006;
Harris and Nakayama 2007). It is delayed and amplified when the image is inverted.
(Linkenkaer-Hansen et al. 1998). The M170 amplitude can also be modulated by face gaze
direction, and changes in stimulus duration (when under 150 milliseconds) (Taylor et al. 2001;
Tanskanen et al. 2007).
M170 changes related to image degradation probably reflect efficiency changes in the processes
underlying the M170 (Liu et al. 2000). Contrastingly, other M170 differences, due to inversion,
eye gaze direction, and non-face stimuli, may reflect separate neural processes. The MI 70 for
these stimuli may be generated by a separate group of non-face-specific cells (LinkenkaerHansen et al. 1998; Swithenby et al. 1998; Watanabe et al. 1999; Liu et al. 2000; Taylor et al.
2001; Tarkiainen et al. 2002; Itier et al. 2006; Kloth et al. 2006; Schweinberger et al. 2007). For
example, images of motor-bikes, letter strings, and hands have M I70s generated by different
brain activity in/around the fusiform gyrus than what generates the face MI 70 (Swithenby et al.
1998; Watanabe et al. 1999; Tarkiainen et al. 2002).
The M1 70 may or may not be related to the N] 70 and other face-specific neuralsignals
Many researchers have proposed that the N 170, a face-specific neural potential, may be
equivalent to the M170. Indeed, both are face-selective, and seem to be generated in or around
the fusiform gyrus (Halgren et al. 1995; Zhao et al. 2003; Harris et al. 2005; Defike et al. 2007).
However, they differ in their response to experimental factors, indicating that they do not reflect
equivalent processes (Taylor et al. 2001). It's possible that the Ml 70 reflects only one of the two
15
sources underlying the N170, or the N200 (Liu et al. 2002). The fact that EEG and MEG have
dissimilar orientation sensitivities complicates the discussion (Sams et al. 1997).
Although the M170 is generated within the vicinity of the FFA, its response pattern is not
identical to that of fMRI revealed FFA activity (Liu et al. 2000). Whereas the FFA
hemodynamic response is affected by attention and expertise, the M170 is not (Lueschow et al.
2004; Xu et al. 2005; Furey et al. 2006). It is possible the same group of cells responsible for the
M170 is activated later, contributing to the fMRI signal. However, the M170 activity itself does
not contribute to the FFA hemodynamic response.
Preceding the M170 is the M100, which is the earliest face-specific signal detectable with MEG
(Liu et al. 2002). The M170 is followed by the M220, another face-specific magnetic peak (Itier
et al. 2006). Coincident with the M170 is a magnetic peak over posterior brain areas. This
signal may reflect the processing of "simple visual features" (Sams et al. 1997). It is not merely
the sink/source associated with the M170's occipitotemporal activity (which can often occur in
the same area, measured with radial gradiometers) because it is detected with planar
gradiometers. I am skeptical of the M170-coincident posterior signal reported by Itier et al,
because it looks like the posterior sink/source of the M170 detected with radial gradiometers
(Itier et al. 2006).
Prosopagnosicsdo not always have or lack an M1 70, the elderly have a reduced Ml 70
Not much research has focused on the clinical relevance of the M170. It appears that the signal
is intact in some prosopagnosic subjects, but not in others. There is no correlation between
16
Ml 70 selectivity and prosopagnosia symptoms (Harris et al. 2005). The only other clinically
relevant finding is an M170 delay in elderly subjects compared to young subjects (Nakamura et
al. 2001). This may reflect an efficiency loss for visual processing.
The M1 70 reflects a beginning stage of face identification
Based on previous evidence, the MI 70 has a role in face processing. Its amplitude correlates
with face classification and identification, modulated by image degradation and duration
(Tanskanen et al. 2005; Tanskanen et al. 2007). An earlier MEG signal (the M100) is also
correlated with face classification, implying that the MI 70 is beyond the image category decision
(Tanskanen et al. 2005). The M170 for personally familiar faces is larger than that for unknown
faces (Liu et al. 2002; Kloth et al. 2006). However, the M170 is equivalent for celebrity and
non-celebrity faces, suggesting that it does not reflect face identification per se (Kloth et al.
2006; Ewbank et al. 2008). This evidence suggests that the MI 70 reflects a beginning stage of
face identification.
17
Experiment
Introduction
This study focuses on the M170, and its association with face processing in the brain. It is
specifically directed towards the following questions:
1)
Why does the M170 respond to buildings if it is face-selective?
2) Does the M170 reflect categorization (that's a face or not) or identification (that's Jill's
face)?
This experiment tackles these questions in four parts (two for each question).
First, I searched for topographic evidence that the building and face M170s were generated in
similar/separate neural areas. The building M170 may come from non-face-specific cells in the
fusiform gyrus, similar to other non-face M I70s (Swithenby et al. 1998; Watanabe et al. 1999;
Tarkiainen et al. 2002). Alternatively, the same group of cells may generate building and face
M170s. This would explain why building images cause adaptation of the face M170.
Second, I investigated the possibility that the building M170 is a "baseline" response. If it were
not baseline, the building M170 could reflect a group of cells specific to buildings. I tested these
possibilities by measuring the M170 response to clear and degraded building images. M170
decay and delay from image degradation would imply specificity.
Third, I looked for difference in celebrity and non-celebrity MI70s, when the images were clear
and degraded. I was motivated by fact that the M170 is smaller and later for degraded faces, but
generalizes to face-like images. These phenomena seem somewhat contrary. Perhaps, the M170
18
is smaller and later to degraded faces because identification, not face-likeness, is compromised.
This could mean that the MI 70 reflects, "This face could be identified," but not face
identification by itself Deciding if an image could be identified is a prime opportunity for face
memory to chime in. If the image is degraded, but meets some template for a known face, the
image would still be a good candidate for identification. Therefore, degradation may affect
celebrity Ml 70s less than non-celebrity Ml 70s.
Fourth, I compared how much image degradation delays face identification, to the amount it
delays the Ml 70. It is likely that the face identification delay is accumulated, through many
steps in the brain. If the MI 70 delay can entirely account for the face identification delay, the
Ml 70 must be associated with a later face-processing step. Otherwise, it must represent a fairly
early step.
19
Methods
MEG was recorded while subjects viewed images of celebrity faces, non-celebrity faces, and
buildings subjected to no, low, and high blur. The M170 sensor topographies were compared to
reveal M170 generator differences. The M170 amplitudes and latencies were compared across
conditions to assess the impact of image type and blur.
Data Collection
Data were collected at the MIT/KIT MEG joint research laboratory on the MIT campus.
Informed consent was obtained from all subjects, and the MIT Committee on the Use of Humans
as Experimental Subjects (COUHES) approved the study.
MEG recordings were done using a 157 sensor whole-head system with first-order gradiometer
sensors (Kanazawa Institute of Technology, Japan). An additional 3 magnetometers recorded
local noise for reference. Subjects were supine in the MEG machine, which was inside an
actively shielded room (Vacuumschmelze, Germany). Stimuli were projected onto a screen
above the subject via a mirror using homemade Matlab scripts (Mathworks, Natick MA). MEG
recordings were done using MEG160 (Yokogawa Electric Corporation and Eagle Technology
Corporation, Japan).
Subjects performed a one-back task while viewing images. They were instructed to push a
button with their left index finger if an image was identical to the one immediately before it,
which had a 5% probability of happening. Response trials were excluded from MEG analysis.
20
The stimuli included images of 135 buildings, 135 celebrity faces, and 135 non-celebrity faces.
Each non-celebrity face was matched to a celebrity face based on race, age, hair, and general
attractiveness. All images were gathered from the Internet. The image backgrounds were
deleted, so that only the person or building was present; they were rescaled, so that the face or
major part of the structure occupied a central oval; they were desaturated and placed on a white
background; and they were reduced to a resolution of 150 pixels wide by 113 pixels tall. The
images subtended roughly 12 0 x8" visual degrees when viewed by the subjects.
The images were divided into three equal groups per category and subjected to blurring. One
group received no blurring. The other groups received Gaussian blurring, with a sigma of one
pixel (low blur) and three pixels (high blur). Therefore, there were nine conditions: no blur, low
blur, and high blur images of celebrity faces, non-celebrity faces, and buildings. The images
selected for each blur group were randomized across subjects.
In a separate study, with the same images but different subjects and image size, these blur levels
corresponded to near perfect performance on face-building categorization. They were associated
with average celebrity identification performance and latency: no blur: 95%, 700 milliseconds;
low blur: 90%, 700 milliseconds; and high blur: 60%, 850 milliseconds (Cherian et al. 2008).
Images were presented in blocks, with roughly 100 images designated to each block (100 for the
first 3 blocks, 105 for fourth). Subjects were allowed to take breaks between blocks. Each
image was presented in the context of a trial, which included a 300 ms fixation cross, the image
for 300 ms, a 1500 ms fixation cross, and a 200 ms blank screen. To keep large chunks of trials
21
from containing only certain types of images (all celebrity faces, all non-celebrity faces, or all
buildings), every 15 trials contained an equal number of each type. Besides this restraint, the
images designated to each block, and the orders of the images within the blocks were
randomized for each subject.
After the MEG portion of the experiment, subjects were asked to fill out a celebrity recognition
survey. They were given a list of names of the celebrities used in the experiment, and asked to
indicate which ones they could recognize. They were told to answer "yes" if they were sure they
could recognize the person by picture alone, and had no doubts. They were explicitly told to
answer "no" if they thought they could maybe recognize the person. Images of non-recognized
celebrities and their non-celebrity matches were excluded from MEG analysis.
Subjects were excluded if they could not recognize over 60% of the celebrities, or did not
performed above 70% on the one-back task. This retained 13 subjects, but two had noisy data
due to machine malfunction, and one lacked M170s for celebrity faces and buildings. Therefore,
I analyzed data from 9 subjects.
MEG Data Noise Reduction
All MEG data were cleaned using M160's implementation of the Continuously Adjusted LeastSquares Methods (CALM). This made use of the 3 orthongonally-oriented magnetometers to
remove external noise from the data (Adachi et al. 2001).
22
Each trial's MEG data (from 200 ms before to 600 ms after image presentation) was
automatically inspected for artifacts due to eye blinks, muscle movement, etc. A trial was
marked "bad" if its MEG data surpassed an amplitude threshold of 2 pico-Teslas. If a particular
sensor had more than 15 bad trials, it was excluded from future analysis. If a trial was found to
be bad in any of the remaining sensors, that trial was excluded from future analysis.
The remaining data were 20 Hz lowpass filtered (48 order Hamming-window based, linear-phase
filter) using zero-phase digital filtering (this made the filter essentially 96 order). The low filter
cutoff excluded more noise from the signal, but reduced the amplitude and sharpness of the
Ml 70. I chose this cutoff because it had precedence in Ml 70 research, and because I was
concerned about noise in the results due to relatively few trials per condition (45 before trial
exclusion) (Lueschow et al. 2004).
Selecting SOIs: Sensors that displayed an M170
An average MEG waveform was created for each condition, for each sensor. These were made
by averaging trial data from 0 ms to 600 ms after image presentation, with a 200 ms baseline
correction.
As previously done in M170 studies, these waveforms were used to identify sensors that
displayed an Ml 70 (sensors of interest, SOIs) (Linkenkaer-Hansen et al. 1998; Taylor et al.
2001). I identified SOIs using the waveforms belonging to no-blur celebrity face, non-celebrity
face, and building conditions. A sensor was marked as an SOI if it had a predominant peak
between 140ms - 230ms after image presentation.
23
I identified SOs by hand. I chose against identifying SOIs automatically with an amplitude
threshold, which would have been the most appropriate alternative to my method, because I
found that small deviations in the threshold resulted in large SOI changes (Harris et al. 2005).
Comparing the M170 Sensor Topographies for Buildings and Faces
I compared the SOIs for celebrities, non-celebrities, and buildings. This provided insight into the
neural generators underlying these MI70s, and the general sensor topographies that can be
expected in M170 experiments.
First, I assessed whether the face SOIs were more similar to each other than to building SOIs.
For each subject, I calculated the proportion of non-celebrity SOIs that were celebrity SOIs and
vice versa. Similarly, I calculated the agreement between non-celebrity and building SOLs; and
celebrity and building SOIs. This was done for each M170 area separately (left and right,
temporal and occipital), and the results were averaged across areas before comparison. Results
were compared using a paired t-test with Bonferroni correction.
Second, I compared the celebrity, non-celebrity, and building SOIs with the greatest M170
amplitude. These SOIs were, theoretically, closest to the M170 neural generator. I found the
sensor with the maximum amplitude for each subject, in each area (left and right, temporal and
occipital), for no blur celebrities, non-celebrities, and buildings. I compared the agreement of
these sensors (i.e. how many of the celebrity maximum SOIs were building maximum SOLs, etc.)
within each subject and area. I also compared the locations of maximal SOIs projected onto the
24
scalp, to reveal any consistent differences between the SOI locations, such as building SOIs
being more anterior than face SOIs. Results were compared within each area using a paired ttest with Bonferroni correction.
Examining the M170 Amplitude and Latency Across Conditions
The no-blur celebrity face, non-celebrity face, and building conditions were used to identify the
sensors for M170 analysis. First, M170 peak amplitudes and latencies were identified for every
SOI using the MEG waveforms. Second, these SOIs were divided, based on Ml 70 amplitude
topography, into the four areas that constitute the M170's one dipole per hemisphere pattern: left
temporal, right temporal, left occipital, and right occipital. Third, the four sensors with the
largest Ml 70 amplitudes in each area were selected for analysis. If less than four sensors were
available in an area, all of them were selected.
The selected sensors' MEG waveforms were averaged together for all conditions (for areas,
conditions, and subjects separately). I averaged across multiple sensors for the same reason I
used a 20 Hz lowpass filter: because it had precedence in M170 research, and because I was
concerned about noise in the results due to relatively few trials per condition (Barrie et al. 2006).
I chose to use only the no blur conditions for sensor selection, based on my assumption that the
Ml 70s to clear and blurry images of the same type were generated by the same neural activity.
The M170 peak amplitudes and latencies were identified for each condition (for areas and
subjects separately), using the MEG waveform averages (MEG waveforms collapsed across as
many as four sensors). Multiplying the right hemisphere temporal and left hemisphere occipital
25
SOI amplitudes by -1 rectified M170 amplitudes (these SOs produce negative M170s). Results
were compared in SPSS (SPSS Inc., Chicago), using repeated-measures ANOVA (GreenhouseGeisser corrected) with four factors: hemisphere (right, left), location (temporal, occipital),
condition (celebrity face, non-celebrity face, building), and blur (no blur, low blur, high blur).
Pairwise comparisons were done using a Bonferroni correction.
26
Results
In reporting the results, standard errors are referred to as "SE." Unless otherwise noted,
significance was at a level of p < 0.05, and non-significant effects were at a level of p > 0.1.
Comparing the M170 Generator for Buildings and Faces
Non-celebrity and celebrity SOIs agreed (mean 96%, SE 1.0%) more than both non-celebrity and
building SOIs (mean 90.4%, SE 2.4%) and celebrity and building SOIs (mean 90.1, SE 2.4%).
However, this effect was not significant (Bonferroni corrected t-test).
Maximum SOIs tended to be the same for celebrities and non-celebrities (61% of celebrity
maximum SOIs were non-celebrity maximum SOIs and vice versa). Maximum SOIs were less
congruent for celebrities and buildings (28%), and non-celebrities and buildings (33%).
Overall, the celebrity, non-celebrity, and building maximum SOI locations were not significantly
different (Bonferroni corrected t-test) (Fig. 2).
Figure 2.
A scalp plot of the maximum
M170 response sensors for each
M170 area. Ovals are centered
on the mean location, and extend
outwards to indicate the standard
deviation in the horizontal and
vertical directions. Posterior
non-celebrity sensors are
completely covered by celebrity
sensors.
Celebrity
Non-Celebrity
c//
Buildi
27
Examining the M170 Amplitude and Latency Across Conditions
The M170 peak amplitude and latency were recorded (for each subject) in the four M170 areas:
left temporal (LT), right temporal (RT), left occipital (LO), and right occipital (RO). The results
are summarized in Figure 3 and detailed in Figure 4. Effects were found using a GreenhouseGeisser corrected repeated measures ANOVA, and pairwise effects were found using a
Bonferroni correction. M170 latencies are reported in seconds, and amplitudes in 1013 Tesla.
Figure 3. A schematic summary of the results, displaying the M170 mean amplitude and latency averaged across all
M170 areas. Celebrity and non-celebrity M170s were similar (celebrity pictured is Martha Stewart). Face M170s
were earlier and larger than building M170s. Blurring reduced and delayed the M170s for all image types.
Degraded Images
Clear Images
1.5
11.51
LI
1.3
)
1.2.
1.3 .
1.2
1.1
1.1 .
~1
1.0
0.9
-
170
U
175
180
U
1.0
U
U
185
190
~
0.9
U
195
200
LI
H
E
170
175
1
U
I
I
U
180
185
190
195
200
Latency (ms)
Latency (ms)
28
Figure 4. The average M170 peak amplitude (a) and latency (b) across subjects for the four M170 areas: left
temporal (LT), right temporal (RT), left occipital (LO), and right occipital (RO). Error bars indicate the standard
error.
a
210
T
T
Celebrity
Non-Celebrity
200
Building
C
0
190
C.,
E,
T
T-rI
180
170
-J
160-
LO
RO
b
2.5
2
CelebrityII
TT
Non-Celebrityinoj
ITT
1.5
TT
BuildingoJ
IF
T
1
T
IT IT
E
0.5
0
RT
LO
29
RO
M170 latencies showed no significant effect of hemisphere: left or right. There were significant
effects of location: temporal or occipital; condition: celebrity, non-celebrity, or building; and blur
level: no, low, or high. There were no significant factor interactions.
M170 latencies (reported in seconds) were overall earlier in occipital areas (mean 189, SE 4.8)
than temporal areas (mean 177, SE 2.4). The M170 latencies for celebrity faces (meanl8l, SE
3.0) and non-celebrity faces (mean 180, SE 2.8) were not significantly different, but were both
significantly earlier than for buildings (mean 188, SE 3.6). The M170 latencies for no blur
(mean 180, SE 3.0) and low blur (mean 181, SE 3.2) were not significantly different, but were
both significantly earlier than for high blur (mean 188, SE 3.1).
M170 amplitudes showed no significant effect of hemisphere: left or right. There were
significant effects of location: temporal or occipital; condition: celebrity, non-celebrity, or
building; and blur: no, low, or high. There was a marginally significant interaction of
hemisphere and location (p < 0.7). We think this was due to artificial cancellation of the left
occipital M170 by the much larger left temporal M170. There were no other significant factor
interactions.
M170 amplitudes (reported in 10-13 Tesla) were overall smaller in occipital areas (mean 0.86, SE
0.13) than temporal areas (mean 1.56, SE 0.21). The M170 amplitudes for celebrity faces (mean
1.28, SE 0.16) and non-celebrity faces (mean 1.27, SE 0.15) were not significantly different.
The M170 amplitudes for non-celebrity faces were significantly larger than for buildings (mean
1.09, SE 0.13). The M170 amplitudes for no blur (mean 1.37, SE 0.18), low blur (mean 1.20, SE
30
0.15), and high blur (mean 1.10, SE .12) were not significantly different from each other (except
no and low were marginally significantly different, p < 0.1).
31
Discussion
In this report I addressed two questions: why there is an M170 response to buildings, which
seems contrary to the M170 face-specificity; and what is the relationship between the M170
response and face identification.
First, I discovered that different neural areas were responsible for building and face Ml 70s.
Overall, the celebrity and non-celebrity SOIs (sensors containing an M170 response) were not
more similar than the celebrity and building SOIs, nor the non-celebrity and building SOIs.
However, this may be related to the fact the face M170s were stronger than building Ml70s, and
therefore extended into more sensors.
The locations of the maximum M170 SOI (the sensor closest to the M170 neural source) were
similar for celebrities, non-celebrities, and buildings. These results indicated that the neural
areas that subserve face and building M170s were identical or very close to each other (beyond
sensor resolution). However, I found that celebrities and non-celebrities shared maximum SOIs
more often than they did with buildings (60% agreement compared to 30%). I assumed that the
same neural area generated celebrity and non-celebrity M170s. Therefore, this suggested that the
face and building M170 generators were distinct.
These findings were similar to what was found in a previous study with motor-bike and face
images. The M170 generators for these stimuli seemed to be different, but very close to one
another, and had no or little consistency in their relative positions (Swithenby et al. 1998). Hand
32
and letter string images also generated M I70s, which were subserved by different neural areas
than the face M170 (Watanabe et al. 1999; Tarkiainen et al. 2002).
Second, I found that the building MI 70 was building-specific (to a first approximation), and was
not a "baseline" response. Just like for faces, degrading building images reduced and delayed
the building Ml 70. Together with the finding that different neural areas generated building and
face M I70s, this implied that there was a cluster of cells in/around the fusiform gyrus selective
for faces, and another selective for houses (or larger category). This is congruent with the notion
of the fusiform gyrus is divided into distinct areas, each specific to a particular visual image
category (Watanabe et al. 1999). However, this possibility is somewhat troublesome. If houses
and motor-bikes had representations, there must be hundreds or thousands of objects with their
own representations in the fusiform gyrus. How many representations could this small brain area
hold? Maybe, some part of the fusiform gyrus responds to many recognizable objects, instead of
a single type. More research will be needed to address this conundrum.
Third, I found no difference between celebrity and non-celebrity MI 70s at any blur level. This
implied that the MI 70 was not influenced by recognition of the image. It reinforced the notion
that the Ml 70 is a feed-forward pre-identification step in the face processing mechanism.
Fourth, I observed that image degradation delays identification much more than the MI 70.
Celebrity recognition was delayed about 100 milliseconds (in a separate study) by degradation
that delayed the M170 about 10 milliseconds (Cherian et al. 2008). Additional delay must be
33
accumulated in neural processes after the M170, which occurs relatively early in the face
identification process.
Contributions
In summary, this report contains four major contributions. First, it contains an overview of MEG
techniques. This is written in a straightforward manner (not with equations), which is needed for
science researchers using MEG. Second, it summarizes all previous M170 findings, which is
another discussion that was previously missing from the literature. Third, with an experiment, it
established that separate neural areas generate the face and building M170s. This has interesting
implications for the organization and function of the fusiform gyrus brain area. Fourth, it
confirmed that the M170 is a relatively early step in the face identification process, and is not
impacted by face image recognizability.
34
References
Adachi Y, Shimogawara M, Higuchi M, Haruta Y, Ochaiai M. Reduction of non-periodic
environmental magnetic noise in MEGmeasurement by continuously adjusted least
squares method. Applied Superconductivity, IEEE Transactions on 2001; 11:669-672.
Barrie J, Herdman A, Tanaka J, M L (2006). A whole-cortex MEG study of face processing in
healthy adults. Society for Psychophysiological Research Annual Meeting.
Cherian T, Morash V, Sinha P. Performance and time costs associated with degraded images. In
preparation 2008;
Clarke J, Braginski Al. The SQUID handbook. Weinheim: Wiley-VCH, 2003.
Defike I, Sander T, Heidenreich J, Sommer W, Curio G, Trahms L, Lueschow A. MEG/EEG
sources of the 170-ms response to faces are co-localized in the fusiform gyrus.
Neuroimage 2007; 35:1495-1501.
Downing P, Liu J, Kanwisher N. Testing cognitive models of visual attention with fMRI and
MEG. Neuropsychologia 2001; 39:1329-1342.
Ewbank MP, Smith WA, Hancock ER, Andrews TJ. The Ml 70 reflects a viewpoint-dependent
representation for both familiar and unfamiliar faces. Cereb Cortex 2008; 18:364-370.
Furey ML, Tanskanen T, Beauchamp MS, Avikainen S, Uutela K, Hari R, Haxby JV.
Dissociation of face-selective cortical responses by attention. Proc Natl Acad Sci U S A
2006; 103:1065-1070.
Halgren E, Raij T, Marinkovic K, Jousmaki V, Hari R (1995). Magnetic fields evoked by faces
in the human brain: 1. Topography and equivalent dipole locations. Society for
Neuroscience Annual Meeting. 21.
Halgren E, Raij T, Marinkovic K, Jousmaki V, Hari R. Cognitive response profile of the human
fusiform face area as determined by MEG. Cereb Cortex 2000; 10:69-81.
Himdlinen M, Hari R, Ilmoniemi RJ, Knuutila J, Lounsamaa OV. Magneto encephalographytheory, instrumentation, and applications to noninvasive studies of the working human
brain. Reviews of Modem Physics 1993; 65:413-497.
Harris A, Nakayama K. Rapid face-selective adaptation of an early extrastriate component in
MEG. Cereb Cortex 2007; 17:63-70.
Harris AM, Duchaine BC, Nakayama K. Normal and abnormal face selectivity of the MI 70
response in developmental prosopagnosics. Neuropsychologia 2005; 43:2125-2136.
Itier RJ, Herdman AT, George N, Cheyne D, Taylor MJ. Inversion and contrast-reversal effects
on face processing assessed by MEG. Brain Res 2006; 1115:108-120.
Kanwisher N. Domain specificity in face perception. Nat Neurosci 2000; 3:759-763.
Kato Y, Muramatsu T, Kato M, Shintani M, Yoshino F, Shimono M, Ishikawa T. An earlier
component of face perception detected by seeing-as-face task. Neuroreport 2004; 15:225229.
Kloth N, Dobel C, Schweinberger SR, Zwitserlood P, Bolte J, Junghofer M. Effects of personal
familiarity on early neuromagnetic correlates of face perception. Eur J Neurosci 2006;
24:3317-3321.
Linkenkaer-Hansen K, Palva JM, Sams M, Hietanen JK, Aronen HJ, Ilmoniemi RJ. Faceselective processing in human extrastriate cortex around 120 ms after stimulus onset
revealed by magneto- and electroencephalography. Neurosci Lett 1998; 253:147-150.
Liu J, Harris A, Kanwisher N. Stages of processing in face perception: an MEG study. Nat
Neurosci 2002; 5:910-916.
35
Liu J, Higuchi M, Marantz A, Kanwisher N. The selectivity of the occipitotemporal M170 for
faces. Neuroreport 2000; 11:337-341.
Lu ST, Hamalainen MS, Hari R, Ilmoniemi RJ, Lounasmaa OV, Sams M, Vilkman V. Seeing
faces activates three separate areas outside the occipital visual cortex in man.
Neuroscience 1991; 43:287-290.
Lueschow A, Sander T, Boehm SG, Nolte G, Trahms L, Curio G. Looking for faces: Attention
modulates early occipitotemporal object processing. Psychophysiology 2004; 41:350360.
Marinkovic K, Raij T, Halgren E, Hari R (1995). Magnetic fields evoked by faces in the human
brain: 2. Cognitive profile. Society for Neuroscience Annual Meeting. 21.
Nakamura A, Yamada T, Abe Y, Nakamura K, Sato N, Horibe K, Kato T, Kachi T, Ito K. Agerelated changes in brain neuromagnetic responses to face perception in humans. Neurosci
Lett 2001; 312:13-16.
Sams M, Hietanen JK, Hari R, Ilmoniemi RJ, Lounasmaa OV. Face-specific responses from the
human inferior occipito-temporal cortex. Neuroscience 1997; 77:49-55.
Sato N, Nakamura K, Nakamura A, Sugiura M, Ito K, Fukuda H, Kawashima R. Different time
course between scene processing and face processing: a MEG study. Neuroreport 1999;
10:3633-3637.
Schweinberger SR, Kaufmann JM, Moratti S, Keil A, Burton AM. Brain responses to repetitions
of human and animal faces, inverted faces, and objects: an MEG study. Brain Res 2007;
1184:226-233.
Streit M, loannides AA, Liu L, Wolwer W, Dammers J, Gross J, Gaebel W, Muller-Gartner HW.
Neurophysiological correlates of the recognition of facial expressions of emotion as
revealed by magnetoencephalography. Brain Res Cogn Brain Res 1999; 7:481-491.
Swithenby SJ, Bailey AJ, Brautigam S, Josephs OE, Jousmaki V, Tesche CD. Neural processing
of human faces: a magnetoencephalographic study. Exp Brain Res 1998; 118:501-510.
Tanskanen T, Nasanen R, Montez T, Paallysaho J, Hari R. Face recognition and cortical
responses show similar sensitivity to noise spatial frequency. Cereb Cortex 2005; 15:526534.
Tanskanen T, Nasanen R, Ojanpaa H, Hari R. Face recognition and cortical responses: effect of
stimulus duration. Neuroimage 2007; 35:1636-1644.
Tarkiainen A, Cornelissen PL, Saimelin R. Dynamics of visual feature analysis and object-level
processing in face versus letter-string perception. Brain 2002; 125:1125-1136.
Taylor MJ, George N, Ducorps A. Magnetoencephalographic evidence of early processing of
direction of gaze in humans. Neurosci Lett 2001; 316:173-177.
Watanabe S, Kakigi R, Koyama S, Kirino E. Human face perception traced by magneto- and
electro-encephalography. Brain Res Cogn Brain Res 1999; 8:125-142.
Willing R (2003). Airport anti-terror systems flub tests. USA Today.
Xu Y, Liu J, Kanwisher N. The M170 is selective for faces, not for expertise. Neuropsychologia
2005; 43:588-597.
Zhao W, Chellappa R, Phillips PJ, Rosenfeld A. Face Recognition: A Literature Survey.
Computing Surveys 2003; 35:399-459.
36