HOW ARE COGNITIVE MAPS
REPRESENTED IN THE BRAIN?
VISUAL VS. PROPOSITIONAL
REPRESENTATIONS AND IMPLICATIONS FOR
GPS TECHNOLOGY
Serj Mooradian
Alison Wheatley
Becky Wright
Jane Emerton
Penelope Dimitrakopoulou
INTRODUCTION


Behaviourism: from the 1920’s onward, behaviour was
explained in terms of simple stimulus-response
mechanisms. Rats learnt mazes by linking rewards
(stimulus; i.e. food) with muscle movements (response;
i.e. running in particular directions).
Tolman (1932) cast doubt on this theory – rats in
mazes: swimming versus running.
The researchers concluded that rats
construct a ‘cognitive map’ in their
brains, and use the internal
representation to solve mazes.
MORE ABOUT COGNITIVE MAPPING




A cognitive map is a cognitive representation of an
environment, that is built up whenever a person encounters
a novel environment (Lynch, 1960). It can comprise of
landmarks, nodes, and paths.
The more experience a person has with an environment, the
better their cognitive map (Jacobson et al 2001).
The type and content of how this information is
represented in the brain is still a prominent and
controversial debate.
There appears at present to be two main arguments – those
who believe cognitive maps are represented pictorially vs.
those who believe they are represented propositionally.
HOW ARE COGNITIVE MAPS
REPRESENTED IN THE BRAIN?








1. PICTORIAL vs. PROPOSITIONAL
Type and content of cognitive maps is still being
debated
Pictorial – Kosslyn
Depiction of environment
Implicates visual system in production and processing –
the ‘minds eye’
Propositional – Pylyshyn
Similar to language
Syntactical representation
PROPOSITIONAL


vs.
PICTORIAL
North, (TREE, HOUSE)
South (LAKE, HOUSE)
-Linguistic description of relationships
-Discrete symbols
-Requires a symbols to describe the
relationships
-Follows grammatical rules
-Abstract symbols
-Unambiguous
-Multiple modes of perception
-More easily accessible
-Does not require knowledge of language
-Not discrete symbols
-Does not require a symbol to describe
the relationships
-Concrete symbols
-Ambiguous
-Only visual mode of perception
EARLY COGNITIVE MAP EXPERIMENTS


Images cannot be directly observed so study brain at functional level
Kosslyn et al (1978)– Mental Image Scanning Tasks
 Method





Results


Memorise a map of 7 locations
Focus on a location
Decide if a second location was on the map
Picture a black speck moving between the two locations
Scanning time increased linearly with the distance between the two locations
Conclusion


It takes longer to scan larger distances compared to small if images
are depictions
Images are depictive rather than descriptive and have spatial
properties
EARLY EXPERIMENTS cont.
ALTERNATIVE MODEL

Pylyshyn (1981) – Critique of Kosslyn’s Experiments




Experiment




Participants asked to see a scene
Use knowledge about really world to simulate events
Does not tell us about inherent nature of imaging
Scanning phenomenon vanishes when asked to judge direction
Can switch attention between locations without simulating moving
Distance between locations had no effect on response time of direction
Conclusion


Unclear which properties are due to the inherent nature of the
mental imaging system and its mechanism
Or which properties are ‘cognitively penetrable’ by the person’s own
knowledge and beliefs during imagery.
NEW TECHNOLOGY GIVES US A
‘WINDOW INTO THE MIND’
Attempts to solve debate via behavioural methods
had led researchers down a ‘cul-de-sac’.
However…. Advances in technology offered a
new opportunity.
Opened up a “window into the mind”.
PET (POSITRON EMISSION TOMOGRAPHY)
~ ²/3 of activation shared by vision and imagery.
(Kosslyn, Thompson & Alpert, 1997)
However, PET not very precise.

rTMS (Repetitive Transcranial Magnetic
Stimulation)
Applying rTMS to the occipital cortex causes temporary impairment of
visual mental imagery (Kosslyn et al., 1999).
PYLYSHYN’S RESPONSE
Pylyshyn (2002): Possible visual system can be
involved in both vision and mental imagery
but without generating a picture-like
representation.
“It is important not only that such … areas be involved
in imagery, but also that their involvement be of the
right sort – that the way their topographical
organization is involved reflects the spatial properties
of the image…”
(Pylyshyn, 2002, p.175)
RETINOTOPICALITY
Area 17:
Located in the occipital lobe. Involved in early visual processing e.g.
shape, location.
Activity is distributed according to a ‘functional space’
i.e. the spatial properties of the visual scene are reflected in the spatial
patterns of the brain activity (Kosslyn, 1994).
Visual Spatial Spectrum:
TOP
RIGHT
SIDE
BOTTOM
FUNCTIONAL SPACE: Different spatial
areas within the visual field reflected onto the retina….
Adapted from Slotnick et al (2005)
… result in activation in different primary visual areas.
Q. DOES THE RETINOTOPICAL ACTIVITY
FOUND IN VISION ALSO OCCUR DURING
MENTAL IMAGERY?
Klein et al (2004), Slotnick et al (2005);
• fMRI study (Functional Magnetic Resolution
Imaging)
• Used simple visual stimuli:
Asked participants to both visualise and imagine the stimuli.
Vertical-axis space = red & green areas
Vision
Horizontal-axis space = blue & yellow areas
Imagery
Adapted from Slotnick et al (2005)
PYLYSHYN’S RESPONSE TO KOSSLYN


Kosslyn’s findings have not been replicated
They are inherently biased
Participants were asked to visualize an image
 By visualizing an image, they are being asked to ‘see
it’ and hence their visual cortex must be stimulated

PYLYSHYN’S RESPONSE TO KOSSLYN
cont.

Visual images are caused by a response to light
on the retina and mental imagery is a top-down
process from higher cortical processes


It is unlikely that the same response will occur to
both types of imagery
Kosslyn: literally pictures in the brain
If it takes longer to ‘scan’ greater imagined distance
(time=distance/speed)
 i.e. simulation of ‘real space’

PYLYSHYN’S RESPONSE TO KOSSLYN
cont.

Other differences in how visual and mental
images are accessed:
Signature properties
 Manipulations of mental images not accurate
 Cortical images fade more quickly
 Emmert’s Law
 Two forms of images not connected to the motor system
in the same way (reaching for an object)
 Retinal images are pre-interpretation; mental images are
the interpretation

PYLYSHYN’S RESPONSE TO KOSSLYN
cont.


Cognitive maps are developed by being in the
environment, receiving cues, interpreting route cues as
opposed to visual, pictorial cues on a map
Use propositional knowledge to navigate, as it is needed


Can be pictorial, but isn’t by nature
Blind people are able to form cognitive maps (Landau,
1986)

The experience of moving through an environment is
enough to form a cognitive map, without actually navigating
the environment (Millar, 1994)
PYLYSHYN’S RESPONSE TO KOSSLYN
cont.

When one is driving down the M4, one does not
necessarily visualize a map of the M4 going EastWest and taking exit 18 south to bath, one simply
exits at exit 18 and drives (subconscious behaviour)

One accesses one’s cognitive map of the region without
stimulating the visual cortex (however it is (hopefully)
stimulated by the road ahead – but not terms of the
cognitive map)
GLOBAL POSITIONING SYSTEMS




GPS’s are a network of satellites and computers
that can locate, track and monitor any mobile
object/individual on earth
GPS in cars provide the user with a pictorial
map and auditory directions
GPS is normally a navigation tool
However - if we regard GPS as simply an aid for
cognitive map development…
IMPLICATIONS FOR THE DESIGN OF
GLOBAL POSITIONING SYSTEMS (GPS)
… if Kosslyn is right:
 GPS technology should display visual
information….
 However, a visual display may turn out to be
more of a hindrance than a help. Kosslyn (1985)
found evidence that visual information similar to
that of a mental representation can actually
interfere with the cognitive map.
IMPLICATIONS FOR THE DESIGN OF
GLOBAL POSITIONING SYSTEMS (GPS)
…if Pylyshyn is right:
 GPS technology should provide auditory
information
GPS TECHNOLOGY cond.



London cabbies
 Accuracy/reliability of GPS??
 Knowledge of road closures, etc.
GPS instead of training or to aid training?
Navigation vs. cognitive maps
Pictorial vs Propositional debate has not been
resolved yet
Although taxi drivers may not find GPS
useful on the job, there is much agreement that
it can be useful to develop cognitive maps in
training.
Cognitive maps prove to be pictorial or
propositional in nature (or a combination of
both). This will have implications as how best
to develop future technology.
REFERENCES







Jacobson, D., Lippa, Y., Golledge, R.G., Kitchin, R., & Blades, M. (2001). Rapid development of
cognitive maps in people with visual impairments when exploring novel geographic spaces.
Bulletin of people-environment studies, 1-8.
Klein, I., Dubois, J., Mangin, J-F., Kherif, F., Flandin, G., Poline, J-B., Denis, M., Kosslyn, S. M.,
Le Bihan, D. (2004). Retinotopic organization of visual mental images as revealed by functional
magnetic resonance imaging. Cognitive Brain Research, 22, 26-31.
Kosslyn, S. M. (1983). Ghosts in the Mind’s Machine. Creating and using images in the brain. George J.
McLeod Limited, Toronto.
Kosslyn, S. M., Ball, T.M. and Reiser, B. J. (1978). Visual images preserve metric spatial
information: Evidence from studies of image scanning. Journal of Experimental Psychology: Human
Perception and Performance, 4, 47-60.
Kosslyn, S. M., Pascual-Leone, A., Felician, O., Camposano, S., Keenan, J. P., Thompson, W. L.,
Ganis, G., Sukel, K. E., and Alpert, N. M. (1999). The role of area 17 in visual imagery:
Convergent evidence from PET and rTMS. Science, 284, 167-170.
Kosslyn, S. M., Thompson, W. L., and Alpert, N. M. (1997). Neural systems shared by visual
imagery and visual perception: A positron emission tomography study. NeuroImage, 6, 320-334.
Landau, B. (1996). Early map use as an unlearned ability. Cognition, 22, 201-223.
REFERENCES cont.







Lynch, K. (1960). The image of the city. Cambridge, MA: M.I.T. Press.
Millar, S. (1994). Understanding and representing space theory and evidence from studies with blind and
sighted children. New York: Oxford University Press.
Pylyshyn, Z. (2000). Is the "imagery debate" over? If so, what was it about? In: E. Dupoux
(ed.). Cognition: a critical look. Advances, questions and controversies in honor of J. Mehler. Cambridge,
MA, MIT Press. Available online at: http://ruccs.rutgers.edu/ftp/pub/papers/pylyshynmehler.pdf
Pylyshyn, Z. (2003). Return of the mental image: are there really pictures in the brain? Trends
in Cognitive Sciences, 7(3), 113-118.
Pylyshyn, Z.W. (2002). Mental Imagery: In search of a theory. Behavioral Brain Science, 25, 157238.
Slotnick, S. D., Thompson, W. L., and Kosslyn, S. M. (2005). Visual Mental Imagery Induces
Retinotopically Organized Activation of Early Visual Areas. Cerebral Cortex, 15, 1570-1583.
Tolman, E.C. (1932). Purposive behavior in animals and men. Berkeley, CA: University of
California Press.