Conscious Thought as Simulation
of Action and Perception
Germund Hesslow
Toward a Science of Consciousness
Skövde 2001
Problems of consciousness
• How does the inner world arise?
• What are mental objects?
• What is the function of the inner world?
• Can animals and robots have inner worlds?
Notice that the problems are phrased in terms of ’the inner
world’ rather than ’consciousness’. Although the inner world is
a part of consciousness, some would hold that the latter also
requires emotions, meanings, qualia etc.
The simulation hypothesis
1) Simulation of behaviour
We can simulate behaviour by covert preparatory action. Thinking of
doing something is similar to actually doing it. It is covert behaviour.
2) Simulation of perception
We can simulate perception by reactivation of sensory cortex
Imagining that one is perceiving something is similar to actually
perceiving it and activates the same brain structures.
3) Anticipation
Simulation of behaviour can elicit other perceptual activity. This
entails that simulation of behaviour can elicit perceptual activity which
resembles the activity which probably would have occurred if the the
simulated actions had actually been performed.
David Hume
(1711-76)
Alexander Bain
(1818-1903)
B.F. Skinner
(1904-1990)
Simulation of behaviour:
covert, incipient behaviour
’The tendency of the idea of an action to produce the fact,
shows that the idea is already the fact in a weaker form.
Thinking is restrained speaking or acting.’ (Bain, 1868 p 340)
’In speech we have a series of actions fixed in trains by
association, and performable either actually or mentally at
pleasure; the mental action being nothing else than a sort of
whisper, or approach to a whisper, instead of the full-spoken
utterance.’ (Bain, 1868, p.347)
Hierarchical organisation of action
Draw triangle
Get pen Get paper Draw
Draw horizontal line Draw sloping ….
Contract m brachioradialis
Contract ....
Main signal flow
Global motor commands are generated rostrally in the frontal lobes.
The more caudal we go, the more specific the commands until we
reach the pyramidal tract neurons of the primary motor cortex. When
simulating a movement, the early rostral parts are reproduced but not
the final specific motor commands.
Simulation of perception:
sensory reactivation
‘What is the manner of occupation of the brain with a
resuscitated feeling of resistance, a smell or a sound?
There is only one answer that seems admissible. The
renewed feeling occupies the very same parts, and in
the same manner, as the original feeling, and no other
parts, nor in any other assignable manner. ‘ ( Bain,
1868, p. 338)
Pain perception
Phantom pain
Seeing
Imaging
Perceptual simulation
A signal reaching the brain from the outside will appear the same,
to the brain, and will generate the same reactions regardless of
whether the signal starts at a peripheral receptor or on the way to
the brain. A signal reaching the frontal lobe from the visual cortex
will appear the same, to the frontal lobe, even if it was not triggered
by an external stimulus.
When I press the key A on my computer keyboard, the processor
perceives the key press. If the signal was being generated
somewhere else, but fed into the processor in the same way, it
would appear to the processor as if I had pressed A.
If the computer can react to the internal stimulus, it has an inner
world.
MRI signal intensity in primary visual
cortex during external vs imagined stimulus
Le Bihan et al. PNAS 90:11802-11805, 1993
MRI activity with external and imagined stimulus
Tootell et al, TINS, 2: 174-183, 1998
Classical ’Pavlovian’ conditioning
Tone
Air puff
Air puff
Tone
Air puff
Tone
Tone
Air puff
Is the CS associated with the stimulus or with
the response?
CS - R:
P
p
b
B
p
b
B
T
CS - S:
P
T
Sensory preconditioning
Experimental paradigm
Training
Test
Stage 1:
Light - Tone
No overt response
Stage 2:
Tone - foot shock
Tone
Flexion
Light
Flexion
Stage 3:
Mechanism: is CS associated with S or with R?
T
L
t
f
F
T
L
t
f
F
Anticipation:
action-sensation associations
’The succession designated as cause and effect, are fixed in
the mind by Contiguity. The simplest activity is where our
own activity is the cause. We strike a blow, and there
comes a noise and a fracture. The voluntary energy put
forth in the act, becomes thenceforth associated with the
sound and the breakage. Hardly any bond of association
arrives sooner at maturity, than the bond between our own
actions and the sensible effects that follow from them.’
(Bain, 1868, p. 427)
Anticipation
Actions, and hence preparations for actions, often have
predictable consequences which generate new predictable
stimuli. We will assume that an organism can learn such relations
in the form of associations such that the early preparatory phase
of an action can elicit the perceptual activity in sensory cortex that
will probably result from the completed movement. Such
anticipation would enable the organism to avoid potentially
harmful consequences of an act before it is performed.
It follows that simulated movements can also elicit perceptual
activity.
Predictable consequence
S1
S2
s1
s2
r1
r2
R1
R2
Anticipation of consequence
Do we need cognitive maps?
Tolman & Gleitman (1949) J Exp Psych 39: 810-819.
Simulation of behavioural chains
Given the ability to simulate actions and perceptions and an
anticipation mechanism, the organism can simulate long chains of
actions and perceptions without any external input. This simulated
interaction with the external world will inevitably appear as an inner
world.
Behavioural chain
Simulation of behavioural chain
Conversation
Talking to oneself
Simulating conversation
Strong points of the simulation hypothesis
Ontological parsimony: no representations, mental events…
No evolutionary leaps: same structures underlying inner world
as are used for perception and movement
Explains absence of specific brain structure (see above)
Explains relationship between cognitive and motor functions:
simulated movement will require same structures (premotor
ctx, cerebellum and basal ganglia) as overt movement
Problems of consciousness
• How does the inner world arise?
By simulation of behaviour and perception
• What are mental objects?
Source of image is not object but simulated seeing
• What is the function of the inner world?
Inevitable consequence of simulation
• Can animals and robots have inner worlds?
Yes, if their brains can generate their own input
References
A previous version of the simulation hypothesis was presented in
Hesslow G. (1994) Will neuroscience explain consciousness? J Theor Biol 171:29-39.
Many of the critical ideas can be found in the behaviourist literature, for instance
Bain A (1855, 1868) The Senses and the Intellect
Skinner BF (1953) Science and Human Behavior. Macmillan, New York
Skinner BF (1974) About Behaviorism. Knopf, New York
Much of the empirical evidence for covert behaviour cited in the lecture can be found in
Jeannerod M (1994) The representing brain: Neural correlates of motor intention and
imagery. Behav Brain Sci 17: 187-245
Evidence for simulation of perception is reviewed in
Kosslyn SM (1994) Image and Brain: The Resolution of the Imagery Debate. MIT Press,
Cambridge
Further details will be added to my website shortly
www.mphy.lu.se/avd/nf/hesslow