Holistic, part-based and semantic
processes in face recognition:
Dissociations and interactions
¹ Department of Psychology, University of Toronto
² Rotman Research Institute, Baycrest Centre for Geriatric Care
³ Centre for Vision Research, York University
In collaboration with:
David Anaki
Josee Rivest
Sandra Black
Larry Leach
Yakir Kaufman
Bruce Bolster
Outline
• Historical background
• Enduring problems
• New studies
• Dissociations among three agnosic patients
• Prosopagnosia (patient DC)
• Object agnosia (patient CK)
• Pure alexia (patient CM)
• Semantic/associative prosopagnosia (patient AP)
• Interactions among face, object, and semantic
systems
Historical background
Bodamer, 1947
Definition: prosopagnosia
Acquired inability to recognize
familiar faces from vision
• Recognition triggered by alternative means:
ex.: voice, paraphernalia, gait
• Appropriate semantic information about familiar
people
• Good visual acuity
• Face recognition may be achieved by a feature-byfeature approach (?)
RECOGNITION impairment...
Must NOT be secondary to:
• sensory loss
• general intellectual loss
• memory & language problems
• or a combination of these factors
Associated Recognition Deficits
• Object agnosia
• Pure alexia
• Good visual acuity
• Good semantic memory/processing
• Appropriate orthographic units
• No agraphia, no aphasia
• Letter-by-letter readers
But again… dissociations:
• Prosopagnosia with alexia
but without object agnosia
(e.g. Buxbaum, Glosser & Coslett, 1998, De Renzi & di Pellegrino, 1998)
• Prosopagnosia with object agnosia
but without alexia
(e.g. Humphreys & Riddoch, 1989)
+
• Object agnosia
without prosopagnosia or alexia
(e.g. Humphreys & Rumiati, 1998; Rumiati, Humphreys, Riddoch & Bateman, 1994)
Neuroanatomy
From patients…
• Bilateral damage in the occipitotemporal cortices:
- the inferior temporo-occipital regions
- the lingual and fusiform gyri
(e.g. Damasio, Damasio & Van Hoesen, 1982;
Takahashi, Kawamura, Hirayama, Shiota & Isono, 1995)
• A right hemisphere lesion (in the equivalent areas)
- may be sufficient
(e.g. De Renzi, 1986a; Landis, Regard & Bliestle, 1988; Rapcsak, Polster,
Comer & Rubens, 1994; Takahashi et al., 1995)
More evidence…
“Normal” human brains:
fMRI studies
- the fusiform gyri (Face Fusiform Area: FFA)
(e.g. Kanwisher, Woods, Iacoboni & Mazziotta, 1997)
A PET study
- the medial fusiform gyri (areas 19-37)
- the right parahippocampal gyrus (area 36)
- the anterior region of the temporal poles
(Sergent & Signoret, 1992)
Enduring Debates:
Are faces special?
1.
2.
3.
4.
5.
6.
7.
Modularity vs. General Purpose
Representations vs. Processes
Face-specific vs.Individuation or Expertise
Focal/Localized vs. Distributed
Laterilzed vs. Bilateral
Holistic vs. Analytic or part-based
Different versions of holistic
CK - Object Agnosia & Alexia
• 41-year old man
• 16 years education (MA degree)
• Manager
• Closed-head injury: hit by a car when jogging
• Bilateral thinning in the occipito-temporal regions
Behrmann, Winocur & Moscovitch (1992)
Behrmann, Moscovitch & Winocur (1994)
Moscovitch, Winocur & Behrmann (1997)
CK - Object Agnosia & Alexia
IMPAIRED
• Word recognition
(reading)
• Object recognition
• Residual blindness
in upper left field
NORMAL
• Average IQ
• Acuity
• Upright face recognition
• Visual construction abilities
• Memory
• Semantic knowledge
Behrmann, Winocur & Moscovitch (1992)
Behrmann, Moscovitch & Winocur (1994)
Moscovitch, Winocur & Behrmann (1997)
Face module
• Features - primarily internal
• At least two of three needed
• Orientation specific - 0 (upright) to 60-90
degrees
• Global configurational properties
– template and/or second order relations
Face counterpart in object system
•
•
•
•
Feature - internal and external
Distinctive? Verbalizable?
Orientation sensitive
Categorical or local relations (jig-saw
effect)
• Integration of features and relations
Interaction of face and face
counterpart
• Mapping of features and relations in
counterpart to holistic face representations
stored in long-term memory (FRUs)
• Interactive activation between systems:
face, object, and semantics
FACES
Holistic
Upright faces
OBJECTS
WORDS
Part-based
Inverted faces
Double dissociation
Patients
CK
DC
CM
Faces
√
X
√
Objects
X
√
√
Recognition
Words
X
√
X
More dissociations
Patients
CK
DC
CM
Faces
√
X
√
Objects
X
√
√
Words
X
√
X
Recognition
Patients
CK
DC
CM
Faces
√
X
√
Objects
X
√
√
Words
X
√
X
Recognition
Object Agnosia
& Alexia
Pure Alexia
Prosopagnosia
DC - Prosopagnosia
• 54-year old man
• 16 years education
• Detective for the homicide division of a police
force (on disability)
• Colloid cyst hydrocephalus
• Posterior cerebral artery infarction
DC - Prosopagnosia
• Bilateral lesions
in the lingual
gyri, Brodmann
Areas 18 & 19
R
L
DC - Prosopagnosia
R
L
• Bilateral lesions in the lingual gyri: more
atrophy to the right fusiform gyrus than
the left
DC - Prosopagnosia
IMPAIRED
NORMAL
• Face recognition
• Bilateral superior
field deficits
• Superior IQ
• Acuity
• Object recognition
• Reading
• Visual construction abilities
• Space perception
• Colour vision
• Memory
• Semantic knowledge
Face recognition:
Famous faces
slow —no instant recognition
Looks like a “piecemeal” approach
Bob Hope: “I will guess that it is Bob Hope. He has a
long nose, his hair cut is him too, a side ways grin; I
am not 100% sure that it is him...”
Hollywood stars
• Jerry Seinfeld: “I do not know, an ordinary guy”
“Oh, Jerry”
—watched Seinfeld at least once a day
• Denzel Washington: “a US ball player”
—described many movies that he was acting in
• Michelle Pfeiffer: “A model: Sharon Stone? Christie Brinkley?”
His family members and myself
with hair covered:
• misidentified his son (who was in the testing room)
• misidentified me
CM - Pure Alexia
• 35-year old man
• 15 years of education
• Computer programming
• Closed-head injury / drugs
• Temporal occipital abnormalities on EEG
CM - Pure Alexia
IMPAIRED
• Word recognition
(reading):
Letter-by-letter
reader
NORMAL
• High average IQ
• Acuity
• Face recognition
• Object recognition
• Visuo-spatial abilities
Is there one part-based system?
FACES
Holistic
OBJECTS
WORDS
Part-based
Processing systems
Are there separate part-based systems?
FACES
OBJECTS
WORDS
Holistic
Part-based
Part-based
Processing systems
Does part-based face recognition
depend on one or both part-based systems?
FACES
OBJECTS
WORDS
Holistic
Part-based
Part-based
Processing systems
CK - Object Agnosia & Alexia
FACES
OBJECTS
Holistic
Part-based
WORDS
Part-based
Intact upright face recognition
Moscovitch, Winocur & Berhmann (1997)
CM - Pure Alexia
FACES
OBJECTS
WORDS
Holistic
Part-based
Part-based
Intact face and object recognition
Can part-based face recognition rely
only on systems mediating recognition
of objects and words?
FACES
OBJECTS
WORDS
Holistic
Part-based
Part-based
Or does it rely on the interaction between
part-based and holistic systems ?
FACES
OBJECTS
WORDS
Holistic
Part-based
Part-based
DC - Prosopagnosia
FACES
OBJECTS
WORDS
Holistic
Part-based
Part-based
Intact object and word recognition
Participants
Controls
Brain-damaged
Control
n=12
CK
CM
Faces
√
√
√
X
√
Objects
√
X
√
√
√
Words
√
X
X
√
√
Recognition
DC DCB
Tasks: Face recognition
• Upright faces
Results
Mean Number of People
Recognized
Mean Number of Famous People Recognized from Photos
(Max. = 70)
70
60
50
40
30
20
10
0
Controls
perfect score
CK
CM
DC
DCB
Caricatures of Famous People
Woody Allen
Ronald Regan
100
90
80
70
60
50
40
30
20
10
0
Percent correct identification
Percent mean recognition
Caricatures of Famous People Recognized
(Max. = 29)
Controls CK
DC
DCB
100
90
80
70
60
50
40
30
20
10
0
Controls CK
DC
DCB
Mothers
Child
Fathers
Percent mean recognition
Percent of Correct Responses on a Seven-Item Forced
Choice Recognition Test for Identifying Each of the
“Parent” Photos from the Photo of a “Child”
100
90
80
70
60
50
40
30
20
10
0
Controls CK
CM
DC
DCB
Tasks: Face recognition
Faces
• Inverted
• Fractured
• Disguised
Who is:
• Inverted
Jason Alexander
George
Who is:
• Fractured
Oprah
Who is:
• Disguised
Eddie Murphy
Results
Percent Recognized
Percentage of Correct Recognition of Inverted Faces
100
90
80
70
60
50
40
30
20
10
0
Controls
CK
CM
DC DCB
Results
Percent Recognized
Percentage of Correct Recognition of Fractured Faces
100
90
80
70
60
50
40
30
20
10
0
Controls
CK
CM
DC DCB
Results
Percent Recognized
Percentage of Correct Recognition of Disguised Faces
100
90
80
70
60
50
40
30
20
10
0
Controls
CK
CM
DC DCB
Summary of findings
Face
recognition
tasks
Damaged systems
Word
Object &
Word
Face
Upright
√
√
X
Disguised
√
√
X
Fractured
√
X
X
Inverted
√
X
X
There are separate part-based systems
FACES
OBJECTS
Holistic
Part-based
WORDS
Part-based
Only the part-based object system
contributes to some aspects
of face recognition
FACES
OBJECTS
WORDS
Holistic
Part-based
Part-based
It does so by interacting with
an intact upright face-system
FACES
OBJECTS
WORDS
Holistic
Part-based
Part-based
Percent mean recognition
Percent of Upright Cartoon Faces
Identified Correctly (Max. = 31)
100
90
80
70
60
50
40
30
20
10
0
Controls CK
CM
DC
DCB
Percent mean recognition
Percent of Inverted Cartoon Faces
Identified Correctly (Max. = 31)
100
90
80
70
60
50
40
30
20
10
0
Controls CK
CM
DC
DCB
Internal Features Inverted
Jacqueline Kennedy
Robin Williams
Mean Correct Recognition of Photos of Famous People
When Upright and Intact and
When Internal Features Were Inverted
Mean Correct Recognition
Internal Inverted (Max. = 11)
Intact (Max. = 11)
10
10
8
8
6
6
4
4
2
2
0
0
Controls CK
CM
DC DCB
Controls CK
CM
DC
DCB
External Features Inverted
Meg Ryan
Mean Correct Recognition of Photos of Famous People
When Upright and Intact and
When External Features Were Inverted
Intact (Max. = 12)
Mean Correct Recognition
External Inverted (Max. = 12)
12
12
10
10
8
8
6
6
4
4
2
2
0
0
Controls CK
CM
DC
DCB
Controls CK
CM
DC
DCB
Recognition
FACES
WORDS
OBJECTS
holistic
Part-based
Structural representation
Semantic system
Recognition
FACES
OBJECTS
WORDS
Part-based
holistic
Corresponding structural
representation
Semantic system
Misaligned Along the
Vertical Midline
Elizabeth Taylor
Mean Correct Recognition of Photos of Famous People That Are
Intact or Misaligned Along the Vertical Midline
(Max. = 10)
Mean Correct
Misaligned Vertical
Intact
10
10
8
8
6
6
4
4
2
2
0
0
Controls CK
CM
DC
DCB
Controls CK
CM
DC
DCB
Misaligned Along the Horizontal Midline
Madonna
Mean Correct Recognition of Photos of Famous People That Are
Intact or Misaligned Along the Horizontal Midline
(Max. = 10)
Mean Correct
Misaligned Horizontal
Intact
10
10
8
8
6
6
4
4
2
2
0
0
Controls CK
CM
DC
DCB
Controls CK
CM
DC
DCB
Dan Akroyd
Mean Correct Forced-Choice Recognition of Single Isolated Face
Parts (Eyes, Nose, or Mouth)
for Identified Faces of Famous People (Max. = 15)
Recognized Part
Mean Correct Recognition
Identified
14
14
12
10
12
10
8
6
4
8
6
4
2
2
0
0
Controls CK
CM
DC
DCB
Controls CK
CM
DC
DCB
Mean Correct Forced-Choice Recognition of Single Isolated Face
Parts (Eyes, Nose, or Mouth)
for Non-Identified Faces of Famous People (Max. = 15)
Mean Correct Recognition
Non Identified
Recognized Part
14
14
12
10
12
10
8
6
4
8
6
4
2
2
0
0
Controls CK
CM
DC
DCB
Controls CK
CM
DC
DCB
Bev Doolittle: The forest has eyes
CK - Object Agnosia & Alexia
FACES
OBJECTS
WORDS
Holistic
Part-based
Part-based
Intact upright & disguised face recognition
DC - Prosopagnosia
FACES
OBJECTS
WORDS
Holistic
Part-based
Part-based
Impairment of all aspects of face recognition
Damage to the holistic face system leads to
global face recognition deficits
FACES
OBJECTS
WORDS
Holistic
Part-based
Part-based
Face Representation
Depth segregation and “closure”
for faces and objects
Mooney Faces and Objects
Correct Identification of Mooney Faces and Objects
Mean Correct Identification
Faces (Max. = 7)
Objects (Max. = 9)
8
6
6
4
4
2
2
0
0
Controls CK
DC
DCB
Controls CK
DC
DCB
Depth Segregation for
Faces and Objects
Pierre E. Trudeau
William
Shakespeare
Michael Jackson
Eddie Murphy
Bob Hope
Percent Correct Identification and Forced Choice Recognition of
Overlapping Caricatures of Famous People (Max. = 27)
Forced Choice Recognition
Identification
100
80
80
60
60
40
40
20
20
0
0
Percent Correct
100
Controls CK
CM
DC
DCB
Controls CK
CM
DC
DCB
Arcimboldo
Identity of Composite
Faces
(i.e. made of objects)