Recognition
February 26
Troy and Al
Outline
• Theory of Cognition
• Ventral Streams
• Experiments
Theory of Recognition
• Bottom-Up Process (Marr 1977)
Observed
Image
intensity
changes
depth
Primal
Sketch
2 1/2-D
sketch
• Top-Down Process (Biederman 1987) objects are made up of
component parts (geons)
3-D
Model
How to experimentally ascertain
recognition?
•
Thought experiments require self examination of a subject. In early
experiments, the subject was usually a patient with brain damage.
In vision, this can cause agnosia: a visual disorder affecting object
recognition. Two types:
•
•
apperceptive - perception is impaired
associative - association of percept and memory is impaired
To determine associative agnosia, the patient must demonstrate:
1.
2.
3.
•
difficulty recognizing visually presented objects
must demonstrate knowledge of the object through other modalities
must be able to see the object clearly to deccribe its appearance
Today, we can use PET and fMRI on “normal” subjects as they
perform some cognitive task
An Early Experiment
Vision and Firehoses
Seeing the world around us is like drinking from a firehose - Kanwisher and
Downing (1998)
We deal with the firehose, we have attention, which allows selective processing of
the information relevant to current goals.
•
Helmholtz’s (1850) experiment to ascertain visual attention
Visual Pathways
•
Schneider (1969) first postulated anatomical seperation of
location vs. recognition
ancient retinotectal pathway
geniculostriate system
•
Underleider and Mishkin (1982)
location vs. recognition
posterior parietal
inferior temporal
WHERE
WHAT
•
Goodale and Milner (1992)
WHAT
HOW
Visual Pathways
Dorsal Pathway
Visuo-spatial disorder due to dorsal pathway damage
• Patient with parietal injury Newcombe et al (1987) Ettinger (1990)
– difficulty choosing correct route through a small 10-choice maze by using
a hand held stylus
– NO difficulty moving himself through a maze
– NO difficulty recalling complex geometrical patterns
– NO difficulty carrying out tasks involving short term spatial memory
• Some spatial processing affected, object recognition and recollection
unimpaired
Ventral Pathway
Desimone and Ungerleider (1989) and Gross et al (1993)
In monkeys, ventral pathway contains cells sensitive to shapes, color,
orientation and texture
Ventral Pathway
Types of Responses:
•
Selectivity Desimone et al (1984), Tanaka et al (1991)
Some TE cells respond only to complex stimuli or stimulus classes
•
Paradoxical Sensitivity Gross et al (1993)
Some TE cells respond to stimuli with few obvious common features
•
Li, Miller and Desimone (1993)
Other cells from TE, perirhinal area 36, and cells going into the frontal lobe
that respond to the memory of an object
(i.e. objects that are associated in time)
Ventral Pathway Damage
agnosia
• Patient had acute loss of blood pressure causing brain damage (Rubens and
Benson 1971)
– profound inability to recognize size, shape and orientation of objects
– NO difficulty describing objects
– NO difficulty recognizing the object when another
modality was used (touch or taste)
•
Patient suffered carbon monoxide poisoning which damaged areas 18 and 19,
with 17 intact (Goodale and Milner (1991))
– profound inability to recognize size, shape and orientation of objects (blocks used
in experiment)
– NO difficulty with hand/finger movements directed at objects (blocks)
Site Map
Coronal slices from a single subject, arranged posterior (top left) to anterior (bottom right)
What’s FFA and PPA?
• fusiform face area - ventral occipito-temporal cortex
• parahippocampal place area - ventromedial cortical region
These areas are studied in tandem since they have opposite response
properties
So you want to do an fMRI study?
Average cost of performing an fMRI experiment in 1998:
Average cost of performing a thought experiment:
Your Salary
CONCLUSION: Unless you are Bill Gates or Michael Jordan, a thought experiment is
much more efficient!
Be wary of fMRI results!
“I wonder whether PET research so far has taken the methods of experimental
psychology too seriously. In standard psychology we need to have the subject
do some task with an externalizable yes-or-no answer so that we have some
reaction times and error rates to analyze – those are our only data. But with
neuroimaging you’re looking at the brain directly so you literally don’t need the
button press… I wonder whether we can be more clever in figuring out how to
get subjects to think certain kinds of thoughts silently, without forcing them to do
some arbitrary classification task as well. I suspect that when you have people
do some artificial task and look at their brains, the strongest activity you’ll see is
in the parts of the brain that are responsible for doing artificial tasks.
-- Steve Pinker, interview in the Journal of Cognitive Neuroscience, 1994
Source: Nancy Kanwisher
fMRI Setup
Lateral Occipital Complex
Grill Spector et al. (2001), using fMRI, conclude: “the ventral pathway
contains one region, LOC, which exhibits little selectivity for specific
object categories” (but is activated after recognition)
Lateral Occipital Complex
• Malach et al. (1995) used fMRI to show that LOC has a higher
response when the subject views photographs of common objects than
when visual textures without obvious shapes are viewed
• Kourtz and Kanwisher (2000) used fMRI to show the cue invariance
(object format) of LOC response
Lateral Occipital Complex
• Grill Spector et al (2000) showed neuronal adaptation in LOC: after
recognition, higher LOC MR signal:
– correlation between subjects’ ability to recognize an object and LOC MR
signal.
– subjects were trained to recognize images presented for 40ms. fMRI
signal was higher after training.
• Conclusion: the LOC exhibits little selectivity for specific object
categories but is activated after recognition
Face Recognition neurons
Area of TE immediately adjacent to STP
observed by Gross et al. (1972) to have
a small population of neurons which
respond to “global aspects of the face:
the arrangement of the eyes and nose
into a facelike configuration”.
These cells respond differently depending
on
• individual monkeys and/or humans
• extent of the forehead
• presence/width of the eyes
• eye direction
unit recordings of an STP neuron
Bruce et al. (1981)
Face Recognition neurons
There are experiments that support two ideas:
• grandmother cell
• circuits of cells
Face agnosia
Prospagnosia
Caused by damage to ventral occipitotemporal and temporal cortex
• Component subprocesses in face recognition
– different deficits: inability to ascertain age, gender, emotional expressions,
and/or identity by viewing a face
– NO difficulty demonstrating knowledge of the face through other
modalities (auditory, observing clothes being worn)
– NO difficulty of perceiving face structure
• Pallis (1955) studied a patient with prospagnosia
– NO difficulty with recognizing non-face objects
– NO difficulty with localization
– NO difficulty with memory (short and long term)
Visual Attention
The effects of attention on the internal representation of a stimulus
• Wojciulik et al. (1998) found that activation of the FFA is dependent
on attention. That is, the processing of faces can be modulated by
selective attention.
Visual Attention
• O’Craven et al. (1999) used fMRI to analyze object-based attention
effects in FFA and PPA: attending to one attribute of an object should
enhance processing of the other attributes of that object compared to
attributes of irrelevant objects
– to distinguish between the effects of object based selection from
location/feature based selection, the stimulus looked like
Anterior coronal views
The task-irrelevant features received more attention when they were associated
with the attended object compared to an ignored object
==> object based selection
Mentally vs. Visually
Applied to Faces (FFA) vs Places (PPA) problem
(O’Craven and Kanwisher 2000)
Subjects were shown alternating scenes of Face-Place-Face-Place- …
Mentally vs. Visually
Faces vs Places (O’Craven and Kanwisher 2000)
Results …
1.
Response magnitudes greater for perception than for imagery
2.
Possible to decode the single cognitive from an inspection of fMRI
data from individual imagery trials
Extrastriate Body Area
Downing et al. 2001
a region of the lateral occipitotemporal cortex
EBA in three individual subjects, seen in coronal slices, arranged posterior (left) to anterior (right)
Extrastriate Body Area
Downing et al. 2001
Stimulus Space:
• Minimizes body vs non-body low level
image properties
–
–
•
•
line drawings
scrambles
Rules out EBA response to any object
with rigid subparts connected at flexible
joints
Rules out response to all animals
Results …
OP - object parts
FP - face parts
SCENE - outdoor scenes
WO - whole objects
BP - human body parts
HAND - hands
FACE - faces
SO - scrambled objects
Sources
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
Downing PE, Jiang Y, Shuman M, Kanwisher N. “A cortical area selective for
visual processing of the human body.” Science. 2001 Sep 28;293(5539):2470-3.
Cohen JD, Tong F. “The Face of Contoversy.” Science. 2001 Sep
28;293(5539):2405-7.
Kanwisher N, Downing P. “Separating the wheat from the chaff.” Science. 1998
Oct 2;282(5386):57-8.
Grill Spector K, Kourtzi Z, Kanwisher N. “The lateral occipital complex and its
role in object recognition.” Vision Res. 2001;41(10-11):1409-22. Review
Farah M, Humphreys GW, Rodman HR. “Object and Face Recognition.” The
Fundamentals of Neuroscience. Academic Press. 1999.
Goodale MA and Milner AD. “Separate visual pathways for perception and
action.”
Trends Neurosci 1992 Jan;15(1):20-5
Kanwisher N. Kanwisher Lab Homepage: http://web.mit.edu/bcs/nklab/
Downing P, Liu J, Kanwisher N. “Testing cognitive models of visual attention
with fMRI and MEG.” Neuropsychologia. 2001;39(12):1329-42. Review.
Culham JC and Kanwisher NG. “Neuroimaging of cognitive functions in
human parietal cortex.” Curr Opin Neurobiol. 2001 Apr;11(2):157-63. Review.
O’Craven KM and Kanwisher N. “Mental imagery of faces and places activates
corresponding stiimulus-specific brain regions.” J Cogn Neurosci. 2000
Nov;12(6):1013-23.
Haxby JV, Gobbini MI, Furey ML, Ishai A, Schouten JL, Pietrini P. “Distributed
and overlapping representations of faces and objects in ventral temporal cortex.”
Science. 2001 Sep 28;293(5539):2405-7.