Cognitive Architectures
Perception and Attention
Based on book Cognition, Brain and Consciousness ed. Bernard J. Baars
courses taught by Prof. Randall O'Reilly, University of Colorado, and
Prof. Włodzisław Duch, Uniwersytet Mikołaja Kopernika
and http://wikipedia.org/
http://grey.colorado.edu/CompCogNeuro/index.php/CECN_CU_Boulder_OReilly
http://grey.colorado.edu/CompCogNeuro/index.php/Main_Page
Janusz A. Starzyk
EE141
1
Recognition
Where does invariance come
from?
A 3D image based on 2D
projections, what's remembered
is just one 3D representation
(Marr 1982).
Syntactic approach: form a
whole from pieces of a model.
Variant (Hinton 1981): look for transformations (displacement, scaling,
rotation), conform to the canonical representation in the memory.
Problem: many 2D objects can form different 3D objects; it's difficult to
match the objects because the search space to connect fragments into
a whole is too large – do we really remember 3D objects?
2
EE141
Gradual transformations
In the brain, rotational invariance is
strongly limited – eg. recognizing rotated
faces.
Limited invariant object recognition can be
achieved thanks to gradual hierarchical
parallel transformations, increasing
invariance and creating increasingly
complex features of distributed
representations.
Goal: not 3D, but to retain enough details to be able to recognize objects
in an invariant manner after transformation.
•Map seeking circuits in visual cognition (D. W. Arathorn, 2002 )
3
EE141
Dorsal pathway
Recognition is a function of the ventral pathway, now let's turn to the
dorsal pathway. Functions: motion detection, localization, "where” and
how to act, but also on what to focus attention and how to shift attention
from one object to another.
Attention allows us to tie different properties of an object into one whole,
to solve the problem of cohesion of sensations in spite of distributed
processing; distributed activation => features related to each other,
referring to one object.
Mainly an attention model, an emergent process resulting from the
structure and dynamic of neural networks, mainly inhibition.
The effects of attention are universal, visible in different situations.
What to pay attention to? Is this a well posed question?
Dogs bite, but not only Spot, not only mongrels, not only black ones...
4
EE141
Lesion studies
Consequences of damage to early visual areas






Different visual deficits can result from neural damage at different
levels of the visual processing hierarchy.
Damage to the retina can result in monocular blindness
Damage to the LGN can lead to loss of vision in the contralateral
visual field
Damage to a small part of V1 can lead to a clearly defined scotoma.
Patients with damage to V1 area may still perform better than chance
forced choice discrimination of objects (blindsight), although they
claim they see nothing.
Although the pathway from retina to LGN to V1 provides most of
visual inputs to cortex, several alternative subcortical pathways project
to extrastriate areas (MT, V3, V4), bypassing V1. This may explain
forced choice results.
5
EE141
Lesion studies
Extrastriate lesions – damage outside area V1



EE141
Motion blindness caused by a lesion to area MT: the world
appears to be a series of still snapshots.
Crossing street is dangerous since the patient cannot tell how
fast the cars are approaching.
Pouring a cap of coffee becomes a challenge since she cannot
tell how fast the liquid was rising.
6
Lesion studies



Cortical color blindness may be caused by a lesion to area V4:
The world appears to be drained of color, just shades of gray.
Patients can perceive the boundaries of colors but cannot name
them.
7
EE141
Lesion studies
Damage to ventral
object areas
Visual Agnosia: Patients with
visual agnosia have difficulties with
recognizing objects because of
impairments in basic perceptual
processing or higher-level
recognition processes
Three types of agnosia:
apperceptive agnosia,
associative agnosia, and
prosopagnosia
Agnosia=to lack knowledge of
EE141
8
Lesion studies


Patients with apperceptive agnosia
can detect the appearance of visually
presented items, but they have
difficulty perceiving their shape and
cannot recognize or name them.
Associative agnosia refers to the
inability to recognize objects, despite
apparently intact perception of the
object.
 Patient can copy a picture of the object but
does not recognize it.
 A patient mistook his wife for a hat.
 Associative agnosia results from damage
to ventral temporal cortex.
9
EE141
Lesion studies

Patients with optic ataxia can perceive visual
orientation and recognize objects but cannot
perform visually guided actions.
 Optic ataxia results from damage to parietal lobe in
dorsal pathway.

Patients with prosopagnosia are still able to
recognize objects well, but have great difficulty
recognizing faces.
 All faces look the same
 Patients can recognize animals but not people

Brodman area no. 37 is responsible for
face recognition
 over 90% of cells in area 37 responds to
faces only.
10
EE141
Lesion studies



fMRI analysis of the face recognition process.
Visible is activity in right hemisphere in lower temporal area
Face recognition is important from evolutionary perspective.
11
EE141
Lesion studies

Patients with achromatopsia are unable to recognize
colors.
 This is often a result of damage to area V4 or thalamus.
12
EE141
Lesion studies

Daltonism refers to dichromacy characterized by a
lowered sensitivity to green light resulting in an
inability to distinguish green and purplish-red.
 It is an inherited defect in perception of red and green, or in
other words, red-green colorblindness.
13
EE141
Dorsal pathway lesions
Lesions in the parietal cortex strongly affect mechanisms of attention
and spatial orientation, extensive lesions in one hemisphere lead to
hemispatial neglect, the inability to focus attention to the half of the
visual space which is opposite the lesion.
For small unilateral lesions, we can see a noticeable slowing of attention
switching to the damaged side. For more severe cases, switching
attention is not possible.
Bilateral lesions lead to Balint's syndrome, difficulties with the
coordination of hand and eye movement, simultanagnosia; differences in
attention switching times in the Posner task are small.
Posner contended that this is a result of attention binding, the inability to
disengage, but he didn't give the disengagement mechanism; it follows
after focusing attention elsewhere – a better model assumes normal
competition.
14
EE141
Lesion studies
Self-portrait
EE141

Damage to the posterior parietal
lobe can lead to a unilateral
neglect, in which a patient
completely ignores or does not
respond to objects in the
contralateral hemifield.

Patients with damaged spatialtemporal recognition forget
about half the space even
though they see it

Patients with right parietal damage
may ignore the left half of the
visual field, eat half of the food
from the plate, or apply make-up to
half of the face.
15
Unilateral Neglect
Horizontal line bisection task
Copying drawings
16
EE141
Lesion studies
Bilateral lesions to parietal areas can lead to a
much more profound deficit called Balint’s
syndrome, which is primarily a disruption of
spatial attention.
 It can be characterized by three main deficits:

 Optic ataxia – inability to point into a target
 Ocular apraxia – inability to shift the gaze
 Simultanagnosia – inability to perceive more than one
object in the visual field

People with Balint’s syndrome appear blind since
they only focus on one object and cannot shift
attention to anything else.
17
EE141
Linking brain activity and visual experience
Imagine you are sitting in a dark room and looking at a jacket on a chair.
 Since you cannot see well, your perception is driven by your imagination
– you may perceive a strange animal, a person, or a statue sitting there.
 When vision is ambiguous, perception falters or alternates between
different things. This is known as multistable perception.
 There are many examples of multistable patterns or ambiguous figures
18
that
scientists
use
to
investigate
these
neural
correlates
of
consciousness.
EE141

Linking brain activity and visual experience
You can cause
binocular rivalry here
using a pair of redgreen glasses
Binocular rivalry: what you see is what you get activated


When two very different pattern are shown, one to each eye, the brain cannot
fuse them together like it would normally do.
What happens is striking: awareness of one pattern last few seconds, then the
other pattern appears
19
EE141
Linking brain activity and visual experience
What happens in the brain during binocular rivalry?
 Tong et al. tackled this problem by focusing on two category-selective
areas in the ventral temporal lobes (FFA and PPA). They used the redgreen filter glasses to present a face to one eye and house to the other
eye. Depending on which image was perceived, they observed
20
activities
either
in
FFA
(face)
or
PPA
(house).
EE141
Linking brain activity and visual experience
Strength of activation of FFA and
PPA was the same in the
rivalry experiment as in the
case of stimulus alternation.
Another approach is to train
monkey to report which of two
patterns is dominant during
binocular rivalry and measure
activity of a single neurons in
different parts of the brain.
This experiment supports
interactive model of visual
perception where feedback
projection modulates lower
levels.
21
EE141
Linking brain activity and visual experience
Another way to separate physical stimulation and perceptual
awareness is a visual detection task.





A subject has to detect a particular pattern.
The researcher makes the pattern harder and harder to see.
Sometimes there is no pattern at all in the picture.
Because this task gets difficult, people will get it wrong sometimes.
What is interesting, that when there is ‘false positive’ (people see pattern
even when it is not there), there is strong activity in areas V1, V2, and V3.
 When the faint stimulus is not detected activities in these areas are much
weaker.
 So, it does not matter what was presented, but what does matter is what is
happening in the brain.
22
EE141
Linking brain activity and visual experience
(a)
(b)
(c)
EE141
Close your left eye, look directly at the cross with your right eye
and move the page up close to your nose, then move it slowly
away from your face, while keeping your eye fixed on the
cross. At the right distance, which should be around 12 inches
(30 cm) away from the page you should notice the red dot
vanish.
Likewise, notice how the black stripes now fill-in; they become
joined and the red dot vanishes.
Brain fills-in perception of the blind spot using visual
information from around the blind spot – constructive
perception or perceptual filling-in.
23
Linking brain activity and visual experience
Adelson's motion without movement
Optical illusions are a result of
our mind filling-in patterns
based on experience
24
EE141
Linking brain activity and visual experience
Two color spirals
Zoom in on the color spiral – two colors are the same shade of green.
EE141
25
Linking brain activity and visual experience



These pictures illustrate
another type of filling-in
known as neon color
spreading (a) and visual
phantoms (b).
Neon color spreading were
found in V1 area.
In a similar way apparent
motion that we see in a
movie theater is another
type of filling-in by neural
activities in V1 area.
26
EE141
Linking brain activity and visual experience
Neural correlates of object recognition
In binocular rivalry, activity in the fusiform face area and parahippocampal
place area is closely linked to the observer’s awareness of faces and houses.
 Other studies deals with visually masked objects which can just barely be
recognized.
 Mooney face shown in figure can be recognized at right orientation, while it is
hard to recognized at different orientations.
 If the objects are recognized activity in ventral temporal region is greater, while
27
activity
in
V1
region
shows
no
difference..
EE141

Manipulations of visual awareness



To find out causal relations
between activities in various brain
regions it is useful to directly
stimulate the selected brain area
with electrical impulses.
One way is to use implants for
instance in V1 area
Another way is to use transcranial
magnetic stimulation (TMS)
TMS involves rapidly generating a
magnetic field outside of the head to
induce electrical activity on the
cortical surface.
Patients report various experiences
including ‘out of body experience’ –
seeing its own body from above.
28
EE141
Manipulations of visual awareness
Unconscious perception




We use the term unconscious perception when subjects
report not seeing a stimulus, but their behavior or brain
activity suggests that specific information about the
unperceived stimulus was indeed processed by the brain.
When two different stimuli are flashed in quick
succession, the visual system can no longer separate the
two stimuli.
Instead, what people perceive is a mix, or a fused blend
of the two images.
They may respond to individual images in various brain
areas without being aware of seeing them
29
EE141
Manipulations of visual awareness



EE141
For instance, a quick presentation of a red square followed by a green
square can be perceived as a yellow one.
Presentation of the images of the house or face in complementary colors
to different eyes has the same effect of not seeing one.
However, the brain still responds to these unseen patterns – fusiform face
area (FFA) to face and parahippocampal place area (PPA) to house. 30
Summary









Vision is our most important sensory modality.
We discussed the functional properties of neurons as visual signals
travel up from the retina to the primary visual cortex and onward to
higher areas in the dorsal and ventral visual pathways.
Progressing up the visual pathway, receptive fields gradually
become larger and respond to more complex stimuli, following the
hierarchical organization of the visual system.
V1 supports conscious vision, provides visual features like
orientation, motion and binocular disparity.
V4 is important for color perception.
MT is important for motion perception.
Damage to dorsal pathway leads to optic ataxia (neglect).
Damage to ventral temporal cortex leads to impairments in object or
face recognition.
In ventral temporal cortex some regions like LOC have general role
in object recognition, while others like FFA and PPA are more
31
specialized
EE141
Summary
Attention effects appear naturally in the model as a result of competition
between inhibition, interconnection, the necessity of compromise.
Similar effects can be seen in different cortical mechanisms.
Some psychological mechanisms (slowing attention) show themselves
to be unnecessary.
Attention effects supply specific information allowing models to be finetuned to comply with experiment results and allowing the use of these
models for other predictions; there is also a lot of neurophysiological
data concerning attention.
Limits of this model:
lack of effects connected with the thalamus (Wager, O’Reilly),
very simple representation of objects (one feature).
32
EE141
Some answers
 Why does the primary visual cortex react to oriented edges?
Because correlational learning in a natural environment leads to this
type of detector.
 Why does the visual system separate information into the dorsal
pathway and the ventral pathway?
Because signal transformations extract qualitatively different
information, strengthening some contrasts and weakening others.
 Why does damage to the parietal cortex lead to disorders of spatial
orientation and attention (neglect)?
Because attention is an emergent property of systems with
competition.
 How do we recognize objects in different locations, orientations,
distances, with different images projected on the retina?
Thanks to transformations, which create distributed representations
based on increasingly complex and spatially invariant features.
33
EE141