Clive Leddy (dvacly@cs.umu.se)
Lab
Mental Images
1 (12)
16-02-12
Clive Leddy (dvacly@cs.umu.se)
Lab
16-02-12
Contents
__________________________________________________________________________________
Introduction
Theory of Visual Images
The Nature of Visualising Images
3
4
4
Cognitive penetrability criterion
Intuition
4
4
The Function of Visual Imagery
Experiments of Visual Imagery
Mental scanning
Mental rotation
Conclusion and Experiment
Literature
6
7
7
8
10
12
2 (12)
Clive Leddy (dvacly@cs.umu.se)
Lab
16-02-12
Introduction
Consider this, if you were asked how many ten öre coins are there in a one-krona coin? You
would probably answer, without too much conscious thought or visual imagery, that there are ten
ten öre coins in one krona. But what if you are then asked to describe exactly what the two sides
of the one krona look like, without looking at a physical coin. You would probably find yourself
visualising the coin in your mind. This is what we would call seeing things in the "minds eye". In
other words you would probably use your conscious thought to visual a one-krona coin. You
might even rotate the coin to visualise both sides of the coin, even thought the coin is not present
in a physical form.
The purpose of this paper is to give the reader an account of some current theories of Mental
Images. I shall account for some of the theoretical and experimental references to mental imagery.
In my account of mental images I have started with an account of the theory of mental images
followed by an account of some of the experiments into mental imagery. I have rounded off my
paper with a discussion and an outline of an illustration of mental imagery. Throughout out the
paper I have referred to the name visual images instead of mental images because they are both
related. The word visual image is related to the physical world rather than the more abstract word
of mental images, which are images that are perceived in an individual person’s mind.
3 (12)
Clive Leddy (dvacly@cs.umu.se)
Lab
16-02-12
Theory of Visual Images
The Nature of Visualising Images
There are hypothesis based on research on visual imagery. These hypotheses are centred on that
visual imagery is a special-purpose component of the cognitive architecture. This special-purpose
component of the cognitive architecture contains representations and processes that are dedicated
to processing certain kinds of information. The information in question (i.e. visual imagery) is
distinct from the aspects of architecture that support propositional representations 1.
Cognitive penetrability criterion.
The claim that visual imagery is a distinct component of the cognitive architecture, requires that it
must be shown that the visual representations a person experience as images have to bind
characteristics that are not easily alterable as a function of beliefs. In his proposed test, Pylyshyn’s
2
cognitive penetrability criterion, which is a test for when a representation or process is
biologically built into the cognitive architecture. What this criterion means is that if imagery is
part of the architecture then it should always operate in the same way and therefor not be changed
by one’s beliefs or knowledge of the real world.
For example:
There has been experiments carried out that have produced evidence that mental images can be
scanned in away similar to the way an individual actually scans a visual image. But the scanning
operation might be cognitively penetrable. It may be that in experiments, subjects scan their
mental image just because they know that they would scan a real visual image. But under
different circumstances it might be that they can shift their attention between or answer questions
about two locations within a image without scanning. In that case it can be argued that image
scanning is neither built into the architecture or to bind component process in performing certain
tasks. Instead, it might be that subjects can choose whether or not to employ cognitively
penetration in order to mimic their perceptual experiences, but discard it if they think that
perceptual mimicry is not called for.
Intuition
Intuition is not a strong source of hypotheses for constructing a theory of imagery. If someone
gave you an abstract verbal description of an object that you can easily encode it propositionaly
and remember it. However, when that object is imagined, it seems that a number of properties
must be added to the description of the object. Yet these added properties fall short of all those
properties that would be present in an actual visual scene.
For example 3, if someone tells you to search in a box of toy blocks for a three-inch cube that has
two adjacent blue sides, you would have no problem remembering the description. Now if you
were to imagine a cube in your minds eye, there would be certainly some additional information
added to the description of the cube in your image. In other words the cube will have a particular
position relative to your point of view. The top, front and right side of the cube maybe visible.
4 (12)
Clive Leddy (dvacly@cs.umu.se)
Lab
16-02-12
Two particular sides will be blue: perhaps the top and right sides. None of this information was
included in the original description and none of it was required to form and remember a
representation of the description. But the information is required to form a mental image of the
cube. But the actual scene of the cube would contain much more information, i.e. 4 the surface of
the cube would have texture, the cube would have some form of support and not forgetting how
the cube is illuminated, resulting in casting a shadow of the cube, and so on.
So the above suggests (but does not determine) that imagery makes use of representations and
processes that are specific to the visual system. The assumption that sensory systems feed into
central systems but that the central systems do not feed back must be altered (See the diagram
2.1).
Figure 2.1
A global view of the cognitive architecture.
If the hypotheses are to be correct then we should have the following figure.
Figure 2.1
A global view of the cognitive architecture.
Stillings et al report explained then by Kosslyn 5 developed a theory that visual imagery involves
the visual system. Their view is that visual imagery involves unique representational formats and
dedicated process, which operate during visual perception. They also proposed that visual
imagery is the result of an active representation in a short-term visual buffer, which is generated
from the long-term memory and that this short-term visual buffer fades rapidly with time unless
constantly refreshed.
5 (12)
Clive Leddy (dvacly@cs.umu.se)
Lab
16-02-12
So what is the visual buffer? Stillings 6 describes the visual buffer by comparing a bit-map
memory of a computer to the visual buffer. He describes it in the following way.
Think of the visual buffer as something similar to a block of memory in a computer that is
organised to be read out directly onto the computer screens (a bit-mapped memory). Each cell in
the memory represents a point (or pixel for picture element) in the two dimensional picture
display on the monitor. Therefore the cell has two-dimensional geometric properties: a position
(x, y co-ordinates), adjacent neighbour cells, and so on. Geometric, or graphics, procedures can be
programmed to operate on the cells, i.e. to draw a line between two points, etc. A spatially
organised area of memory plus a set of graphic routines constitute a specialised subsystem in the
computers with graphic facilities.
The human visual buffer is viewed to be a similar specialised subsystem. The buffer is a shortterm memory structure with the two dimensional spatial properties. But the human visual buffer
has a number of characteristics that the computer version does not possess. For example, in vision
and imagery, the centre of visual buffer has the highest resolution, and there is a special focus of
attention that can be moved around within the buffer. This results in that we can direct our
processing on a particular location.
The Function of Visual Imagery
Stillings 6 states that the clear reason for the existence of visual imagery is to make the
computational resources of the visual system available for reasoning about the shapes and the
spatial arrangement of the objects in the absence of visual input (page 46).
What this means is that while we are problem solving with goal orientated thought we draw on
our long-term memory. Our long-term memory contains information about objects, but this
information does not include explicit spatial detail that is contained in an actual image. The built
in central processes (fig 2.1) are not designed to processes this detailed spatial information. The
result is that when spatial reasoning is required, we call on our long-term memory to construct an
image. Our long-term memory of the image contains the spatial information needed by the
specialised processes available in the visual system.
Stillings then continuous by saying, that the theory is that process of visual perception and
attention are used on the image of an object just as they would be in looking at an actual object.
With out external input to the visual buffer image perception is made difficult. The image of an
object must be continually refreshed, and details of a particular object must be generated as we
shift our attention on the object. This is quite taxing and can lead to subtle interplay between
imagery and reasoning. For example: One might solve a problem by scanning a image from our
long term memory and then use our built in central processes to facilitate reasoning about the
image. The limited capacity of visual imagination seems to be a joint product of the limited
capacities of the central processing and of visual attention.
6 (12)
Clive Leddy (dvacly@cs.umu.se)
Lab
16-02-12
Experiments of Visual Imagery
Mental scanning
Stillings 7 states that in his theory 8 outlined above is apparently reasonable but it must be
supported by laboratory evidence. A fruitful approach has been to study whether response times
during image manipulation reflects the specialised spatial characteristics that are hypothesised to
be characteristic of image representation and processing.
This means that when our attention is focused on one region of an image and then is shifted to
focus on another region, the shift involves a scan across the space represented in the image. Now
if scanning takes place at a constant rate, the shifts between regions that are further apart should
take longer.
Kosslyn, Ball, and Reiser tested the scanning assumption experimentally. But
Pylyshyn (1984) and Finke (1985) criticise the results of the experiment carried out by
Kosslyn. The results that Kosslyn found could reflect the subjects tactical knowledge of
visual scanning rather than the operation of a scanning function that is a part of the
cognitive architecture.
One approach to strengthen the evidence of built-in scanning operation is to try to set up a
situation in which subjects have to answer some questions about two spatial locations but are not
told to or encouraged to mimic visual scanning. If scanning is a primitive operation then response
time should still be affected by distance. Finke and Pinker (1992) conducted such an experiment.
Here is what they had to do. On each trial a pattern of four dots appeared on a screen, and the test
subject formed an image of the dots. The pattern was turned off and after a two-second delay an
arrow appeared on the screen in an unexpected position and orientation. The test subject's task
was to decide as quickly as possible whether the arrow was pointing at one of the previously
shown dots.
The result of the experiment should that the subjects used a scanning operation, even though they
were instructed not to scan and the demand of the task was to answer a question as fast as
possible.
7 (12)
Clive Leddy (dvacly@cs.umu.se)
Lab
16-02-12
Mental rotation
The term refers to the ability to imagine objects rotating in space. This provides a second example
of the use of response times to study the properties of mental imagery. Take a look at the fig 2.4
below.
The question is which of these fives are backward? In order to answer this question you would
probably mentally rotate, in turn, each of the fives to there correct position. Thus, the fives that
involve turning them backwards to be able to read them correctly, are them-selves backwards,
and as a result the answer to the question. It would be quite difficult to answer this question with
out doing mental rotation. Thus mental rotation is hypothesised to one of the built-in operations
that can be applied to visual images.
Stillings 10 says, the experience of mental rotation suggests the hypotheses that mental rotation is
similar to the physical rotation in the real world (page 49). This means that during mental rotation
the object represented must pass through different states that correspond to the actual path of the
physical rotation of the subject. Stillings continuous by writing, that an another hypothesis, that is
similar to the scanning case, is that mental rotation occurs at a constant rate.
Example:
Cooper 11 studied the mental rotation with the illustrated figure below.
8 (12)
Clive Leddy (dvacly@cs.umu.se)
Lab
16-02-12
Try to decide yourself whether each of the test stimulus is standard of reflected? As you make
your decision you will probably experience mental rotation. "Coopers´s resulting data from the
experiment confirmed the hypotheses that subjects make their judgements based on mental
rotation. Mental rotation does occur at a constant rate". See dig 2.6 to view the results.
Figure 2.6
Mean reaction time as a function of angular departure from the trained orientation.
The equation is for the best-fitting straight line through the four data points.
"Cooper´s experiments provided evidence that mental rotation is a precise analogue of physical
rotation". The data from Cooper´s experiments agrees "with the hypotheses that the rotation
operation is part of the cognitive architecture and that it operates on a visual buffer". But we
cannot ignore the fact that tactical knowledge may influence the mental rotation experiments.
9 (12)
Clive Leddy (dvacly@cs.umu.se)
Lab
16-02-12
Conclusion with experiment
All the literature that I have read so far has reflected the difficulty of determining if a particular
process is biologically built into the cognitive architecture. This is made clearer by thinking about
the following.
Imagine if you met an old friend that has changed their appearance? Your long-term memory of
them will be out of date so you might not recognise them straight away. You might find yourself
scanning the old image of them focusing on some detail that is common to the actual visual image
and the long term memory image.
Experiment
Test:
Draw your self-portrait (or any motive that suites your choice) blindfolded.
Goal:
The goal of this test is to illustrate how one would use and experience mental images. I am not
trying to prove any theories with this experiment, I am just giving an illustration of how a subject
might experience mental / visual images. This would be an ideal introduction to “mental / Visual
images” before any theory is presented to the test subjects.
Equipment:
All test subject(s) should posses the following pieces of equipment. A pen or pencil to write with,
a piece of paper in which to write on and a blindfold.
Instructions:
The instructor is to instruct the test subject(s) to draw their self-portrait. But the test subject(s)
must be blindfolded while they are drawing their portrait. The only time the test subject(s) is
allowed to remove their blindfold is when the test subject(s) deems that he/she has completed
their task. Upon completion of the task the test subject(s) may remove their blindfold and view
their results. The instructor may then ask the following questions or other questions that may be
reverent to the subject of mental images.
Question:
How did you (i.e. the test subject(s)) solve the problem?
Perhaps the instructor could stir the test subjects response by using their prior knowledge of
mental and visual imagery. But the basic idea is to illustrate that perhaps the test subject(s) may
have used their cognitive architecture and intuition while drawing their self-portrait. The test
subjects may also have experienced what is called the “minds eye”. In other words the test
subject(s) may have visualised their physical actions of drawing their self-portrait in their minds
eye. The results of the subjects drawings should be of poor quality. Especially if we keep in mind
just how difficult it is to recall all the details that are included in a image and just how difficult
the task is for the cognitive architecture. The difficulty factor for the cognitive architecture is also
increased if the test subject(s) lift their pen from the paper. This would result with that the test
10 (12)
Clive Leddy (dvacly@cs.umu.se)
Lab
16-02-12
subject imagines that they are replacing their pen at the correct position on the paper for to
continue they’re drawing.
The underlining intention of this test is to act as an introduction into mental imagery and the
theory that surrounds this subject.
11 (12)
Clive Leddy (dvacly@cs.umu.se)
Lab
16-02-12
Literature
Cooper, L. A. (1975). Mental rotation of random two-dimensional shapes. Cognitive Pyschology
7, 20-43.
Finke, R. A. (1985) Theories relating mental imagery to perception. Psychological Bulletin 98,
236-259.
Finke, R. A, and Pinker, S. (1992). Spontaneous imagery scanning in mental extrapolation.
Journal of Experimental Psychology: Learning Memory and Cognition 8, 142-147.
Kosslyn, S. (1980) Imagery and mind. Cambridge, Mass.: Harvard University Press.
Kosslyn, S & Koenig, O, 1992 Wet mind: The new cognitive neuroscience.
Kosslyn, S, Ball, T. M, and Reiser, B. J. (1978) Visual images preserve metric spatial
information: Evidence from studies of image scanning. Journal of Experimental Psychology:
Human Perception and Performance 4, 47-60.
Pylyshyn, Z. (1984) Computation and cognition: Toward a foundation for cognitive science.
Cambridge, Mass. : MIT Press.
Simon, H. (1972) What is visual imagery. An information processing interperation. In L. W.
Gregg, ed., Cognitive learning and memory. New York: Wiley.
Stillings, Neil A. (editor) (1995) Cognitive Science an introduction (Second edition)
Massachusetts Institute of Technology.
12 (12)