Given the similarly of vision and visual
imagery, does this suggest that imaging
involves a mental picture?
 Reasoning from the involvement of the visual
system in mental imagery to the conclusion
that the format of the image must be pictorial,
is based on a thoroughly discredited theory of
vision (the picture theory of vision).
• There are serious reasons to doubt that
vision involves the construction of a
pictorial display
Conscious experience and the picture-theory
The picture theory was meant to explain why our perceptual experience
is panoramic and stable while the visual inputs are partial and constantly
changing. This assumes that the content of experience is represented.
But the picture theory of vision has been thoroughly
discredited: There is no rich panoramic display in vision
(e.g., see change blindness, superposition studies, …)
Vision appears to fill in missing
parts of a familiar pattern
Find three of these …
In here….
But superposition of images has been discredited
O'Regan & Lévy-Schoen, 1983
We encode far less visual information
than we generally believe we do – far
from what any picture would hold
Harborside
Airplane
Signature properties of vision do
not apply to off-retinal information
Signature properties of vision do
not apply to off-retinal information
Off-retinal information uses memory
Effectiveness of anorthoscopic presentation depends on memory load
There are many questions about what goes
on when we have the experience of
“seeing with the mind’s eye”



Is mental imagery a special form of thought? If so, in what
way is it special?
 Are mental images sensory-based and modality-specific?
 Are mental images like pictures? In what respect?
 Are images different from other forms of thought? Do
they resemble what they represent?
Does mental imagery use the visual system? If so, what does
that tell us about the format of images?
Is there neurophysiological evidence for a pictorial “display”
in visual cortex?
 What if a display were found in human visual cortex?
A central methodological point:
The difference between appeals to
properties of the architecture or to
properties of what is represented
Suppose we observe some robust behavioral regularity.
What might it tell us about the nature of the system that
exhibits the regularity?
 e.g., in the case of human behavior, what does it tell
us about the cognitive architecture or the format of
representations?
An illustrative example: Behavior of a mystery box
What does this behavior pattern tell us about the nature of the box?
The moral of this example:
Regularities in behavior may be due
to one of two very different causes:
1.
2.
The inherent nature of the system
(to its structure or wiring), or
The nature of what the system
represents (what it “knows”).
The “imagery debate”
The major difference between picturetheorists and the rest of us (me) is in
how we answer the following question:
1. Do experiments on mental imagery (such as those I will
review) tell us anything about the format of images or
about the properties of a special imagery architecture?
Or …
2. Do the experiments tell us about the knowledge that
people have about how things would look if they were
actually to see them (together with some common
psychophysical skills)?

While these are the main alternatives, there are also
other reasons why experiments come out as they do.
Examples to probe your intuition
Imagine various events unfolding
before your “mind’s eye” –





Imagine a bicyclist racing up a hill. Down a hill?
Imagine turning a large heavy wheel. A light wheel.
Imagine a baseball being hit. What shape trajectory
does it trace out? Where would you run to catch it?
Imagine a coin dropping and whirling on its edge as
it eventually settles. How does it behave?
Imagine a heavy ball (a shot-put) being dropped at
the same time as a light ball (a tennis ball). Indicate
when they hit the floor. Repeat for different heights.
Why did things unfold the way
they did in your imagination?

Was it because of the format of your image or your
cognitive architecture? Or because of what you know?



Did it reveal a capacity of mind?
Or was it because you made it do what it did?
Could you make your image have any properties you
choose? Or behave in any way you want? Why not?



How about imagining an object from all directions at once, or
from no particular direction?
How about imagining a 4-dimensional object?
Can you imagine a printed letter which is neither upper nor
lower case? A triangle that is not a particular type?
More examples of the behavior of images:
Do they tell us about tacit knowledge or
about the nature of the architecture or format


Color mixing
Conservation of volume
Or, more relevant to the present discussion:
 The effect of image size
 Scanning mental images
Studies of mental scanning
Does it show that images have metrical space?
2
1.8
1.6
Latency (secs)
1.4
scan image
imagine lights
show direction
1.2
1
0.8
0.6
0.4
0.2
0
Relative distance on image
(Pylyshyn & Bannon. See Pylyshyn, 1981)
Conclusion: The image scanning effect is Cognitively Penetrable
 i.e., it depends on goals and beliefs, or on Tacit Knowledge.
Another methodological note

What is tacit knowledge?




Tacit knowledge of what?


A central construct in cognitive science: Cognitive
phenomena turn on goals and beliefs
Tacit knowledge need not be consciously accessible but
may be inferred indirectly
Access to tacit knowledge is context-dependent
It does not refer to knowledge of what the experimenter wants nor
the expectation of how the experiments will turn out (cf Dennis). It
refers to knowledge of what it would be like to see the relevant
situation as it unfolds.
Cognitive penetrability as a litmus
Major Question 2: Does visual
imagery use the visual system?

It depends on what you mean by “visual” and by “use”




Let’s assume that “the visual system” is “active” in some
way during episodes of mental imagery.


vision can involve all of cognition
use vision can mean actually perceive the image
The reason this question is of interest to picture theorists is
that a positive answer is thought to entail that images are
picture-like.
What follows from that?
Why should we think that vision may be involved?

Note that the main support for the assumption that vision is involved
comes from neuroscience – to be considered later.
Reasons for thinking that images are
interpreted through the visual system




Similar phenomenology of imagining & seeing
 This is the absolutely central reason which
overshadows all others
Superposition & interference studies
Visual illusions with projected images
 The ubiquitous role of attention
Reperceiving and novel reconstruals
 A large but very problematic literature
 Chambers, Finke, Peterson, Slezak
More persuasive demonstrations
Images constructed from descriptions
 The umbrella example(s)
 The two-parallelogram example
 Amodal completion
 Reconstruals: Peterson vs Slezak

Imagine two parallelograms
Imagine two parallelograms
Imagine two parallelograms
Amodal completion by imagery?
Amodal completion by imagery?
Vision is involved when images are
superimposed onto visual displays
 Many experiments show that when you project
an image onto a display the image acts very
much like a superimposed display
• Shepard & Podgorny, Hayes, …
• Interference effects (Brooks)
• Mental scanning
• Interaction with the motor system (Finke)
Can images be visually reinterpreted?

There have been many claims that people can visually
reinterpret images (but remember the parallelogram example!)



These have all been cases where one could easily figure
out what the combined image would look like without
actually seeing it (e.g., the J – D superposition).
Pederson’s careful examination of visual “reconstruals”
showed (contrary to her own conclusion) that images
are never ambiguous (no Necker cube or figure-ground
reversals) and when new construals were achieved from
images they were quite different from the ones achieved
in vision (more variable, more guessing from partial
cues, etc).
The best evidence comes from a philosopher, Peter
Slezak!
Slezak figures
Pick one (or two) of these animals and
memorize what they look like. Now
rotate it in your mind by 90 degrees
clockwise and see what it looks like.
Slezak figures rotated 90o
And what if visual imagery did
involve the “visual system”?


Even if there was credible evidence that vision
is involved in visual imagery, it would tell us
nothing about the format of imagery because,
There is strong evidence that visual
representations are not pictorial either!
The picture theory of vision
is a non-starter, even for cats
(Cartoon by Kliban)
A more plausible theory of vision
(even for cats)