Cognitive Computing 2012
The computer and the mind
ASSOCIATIONISM
Professor Mark Bishop
Idealism

The ‘ideal’ is the realm of mental ideas.


Ontological idealism


Contra ‘realism’ in which the ‘real’ is said to have an absolute
existence prior to - and independent of – [our] knowledge.
insists that the only things that really exist are ideas.
Epistemological idealism

23/03/2016
insists that the only things of which we can be certain of are ideas.
Platonic Idealism

Plato proposed an ‘idealist theory’ as a solution to the problem of universals:

A universal is that which things share in virtue of having some particular
property


Cf. ‘Chair-iness’ and the archetypal chair.
Plato argued that it is only the universals - ‘the forms’ / ‘Platonic Ideals’ - that
are really real …

23/03/2016
… not particular individual things – instances – such as ‘this red chair’.
The ‘allegory of the cave’

Plato imagines …




Plato subsequently explains how the philosopher is like a prisoner who is freed from the cave who
comes to understand that the shadows on the wall are not constitutive of reality at all


… a group of people who have lived all of their lives chained in a cave, facing a blank wall.
The people watch shadows projected on the wall by things passing in front of a fire behind them, and begin to ascribe forms to
these shadows.
[According to Plato] the shadows are as close as the prisoners get to seeing reality.
as he can perceive the true form of reality rather than the mere shadows seen by the prisoners.
The allegory is related to Plato's ‘theory of forms’, wherein Plato asserts that “forms” (or “ideas”) and not the material world of change known to us through sensation - possess the highest and most
fundamental kind of reality.


23/03/2016
Only knowledge of ‘the forms’ constitutes real knowledge.
In addition, the allegory of the cave is an attempt to explain the philosopher's place in society.
Nominalism & Idealism

Confusingly, because Plato’s asserts that ‘the forms are genuinely real’, it is also
called Platonic realism.

In this sense Platonic realism contrasts with nominalism


the notion that ‘mental abstractions’ are merely names without an independent
existence;

the only thing that different chairs have in common is the name ‘chair’.
However Platonic realism is also seen as an early form of ‘Idealism’

23/03/2016
as it asserts the primacy of the idea of universals over actual instances of
particular ‘material’ things.
British Empiricism:
Hobbes (1588-1679); Locke (1632-1704); Hume (1711-1776)

The view that all ‘knowledge’ comes from
‘experience’.

That knowledge is causally dependent on
experience.

That knowledge is justified solely by experience.
23/03/2016
Subjective Idealism:
Bishop Berkeley (1685-1753)

Like the Empiricists George Berkeley believed that all
‘knowledge’ comes from ‘experience’ (of our perceptions).

However unlike the Empiricists, Berkeley believed that only
perceptions are ‘real’; if a tree falls in a forest with no one
around, does it fall with a sound?


Scientifically: “Sound is vibration, transmitted to our senses through
the mechanism of the ear, and recognized as sound only at our nerve
centers. The falling of the tree or any other disturbance will produce
vibration of the air.”

Philosophically: “To be is to be perceived”; is sound only sound if a person
hears it? I.e. if no one is around to see, hear or smell the tree as it falls, how could
the sound be said to exist?
But then why do different people share the same kinds of
perceptions?

23/03/2016
Berkeley, ‘God as the ultimate cause of all perceptions’.
Associationism and the British
Empiricists


Associationism

The view that the mind is organized, at least in part, by
principles of association.

Associationism is closely linked with the British Empiricist
movement.
The core idea of Associationism is that items that ‘go
together’ in experience will ‘go together’ in thought.
23/03/2016
Associationism
Items are associated (go together) in the mind through a
process of experience:

complex ideas are constructed from simples;
simple ‘ideas’ are derived from sensations/perceptions.


Sensations are not governed by association, but are caused
from something outside the head:



23/03/2016
for Hobbes, Locke, Hume, the ultimate cause of sensation are things
in the world;
for Berkeley the ultimate cause of sensation is God.
Varieties of Associationism

Pure Associationism


Holds that only Associationist principles govern the
operation of the mind.
Mixed Associationism

23/03/2016
Holds that principles other than Association can have an
influence on the mind.
The association or ‘going together’ of
mental items

Mental ‘items’ can be associated by:



spatial contiguity;
temporal contiguity;
more abstract principles such as:



cause & effect;
similarity and contrast.
Suggested ‘items of association’ in the mind are:

23/03/2016
memories; ideas; images; thoughts etc.
A Blank Slate?

For Locke in, An Essay concerning human
understanding, (1690), the mind at birth is a ‘tabula
rasa’ (blank slate).

23/03/2016
There are no innate ideas, “Let us then suppose the mind
to be, as we say, white paper, void of all character, without
any ideas. How comes it to be furnished? I answer, in one
word, from experience.”
Primary qualities

Example of primary qualities include: solidity; extension;
figure; number; motion; rest.

The primary qualities have a direct link to their bearer; a primary quality
‘says’ something about its bearer.


Primary qualities are ‘essential’ to their bearers and are intrinsic
qualities of their bearer.

23/03/2016
E.g. If an object instantiates the primary quality of ‘rest’ then the object
much be ‘at rest’.
I.e. Primary qualities are independent of the perceiving mind.
Secondary qualities

Example of secondary qualities include: colour; sound; taste;
smell.

Secondary qualities are the powers of objects - by configurations of
their primary qualities - to cause experience.

For Locke primary qualities exist in the world but secondary qualities
exist only in the mind of the perceiver.

Thus, there is no blueness or sweetness in the world, only extension in
motion.
23/03/2016
Associationist Processes

Sequencing


Compounding


Associations follow one another (eg. in time).
Complex items are formed from simple items.
Decomposition

23/03/2016
Complex items can be broken down into their simple
elements.
Sequencing

Groups of items, (eg. memories), follow one another in one of two ways:

Intrinsic associations
Some items have a natural connection, (i.e. a connection independent of the
observer).


E.g. Chilli peppers and ‘hotness’.
Extrinsic associations


And some items have an observer dependent connection, (either voluntary or
by chance).

23/03/2016
E.g. If the first person one fell madly in love with had long ginger hair then one
may thereafter associate these qualities with beauty, sex and love.
Locke’s three properties of extrinsic
association

Extrinsic associations are either voluntary or happen by chance.

The strength of the ‘impression’ of ideas can reinforce the association.


Powerful ideas may be forever linked in the mind (cf. perception of beauty).
Some items (contingent on say, genes) will ‘go together’ more easily than
others


23/03/2016
For example, some find associations with maths easy; some with football; others with
singing etc.
So although at birth the mind is a blank slate (empty of ideas), individuals may find some
ideas easier to associate than others.
Associationism

Mental representations are ideas.

We ‘entertain ideas’ when in mental states.

Hume suggested that mental processes are sequences of associated mental ‘ideas’,
(‘associationism’).

But what are such mental ‘ideas’ really about?


The problem of intentionality: how do ‘mental ideas’ connect with [become to be about]
things in the world?
Hume suggested that mental ‘ideas’ are fundamentally like ‘images’.

23/03/2016
Hence positing a ‘pictorial resemblance’ between idea and world; but this has problems.
Problems with Hume’s associationism

1. Hume’s theory is both (a) too general; (b) not general enough.

a. Not all ideas are pictures


b. Consider a picture of Eiffel Tower.


The image may look like the Eiffel Tower, from a certain perspective, but the image is not necessary to
the notion of the Eiffel Tower (the tower that Gustave Eiffel built).
2. No account of ‘mental reference’.

Resemblance is not sufficient for representation


Justice?
A cartoon may look more like its creator than its subject, but still represent the subject.
3. No account of truth and falsity


23/03/2016
[Contra early Wittgenstein] Images are not propositions;
Images [in themselves] are neither true or false.
Associationism and representational
theory

Cognitive processes are defined as, “associations between
representations”

Associationism easily accommodates [folk] psychological explanations.

However Associationism lacks a workable account of ‘representational
content’.

23/03/2016
Representational content is whatever it is that constitutes a ‘representation of a
dog’ as representing a dog rather than as ‘representing something else’; or
rather than not being representational at all…
William James (1878)
- The Principles of Psychology

For James thinking (all aspects of conscious life) had the following key properties:

Thinking is conscious.

Thinking is open to introspective examination.

Thinking is private.


‘My thought belongs with my other thoughts and your thought with your other thoughts’.
Thinking ‘flows like a stream’

The ‘stream of consciousness’; affect on literature – Woolf; Joyce etc.

Thinking is ‘about something’ (i.e. it is fundamentally “intentional”).

Thinking has evolved (thinking not as a ‘gift of god’).
23/03/2016
What do we think next?

For James the key question in psychology is, ‘how
does the mind solve the problem of what to think
next?’

His answer is that it operates on general principles of
association which James attempted to illustrate on a
deeper, quasi-neurological basis.
23/03/2016
James, ‘On thinking’ (1)

Principle 1:

When two elementary brain processes have been active
together or in immediate succession, one of them, on
reoccurring, tends to propagate its excitement into the
other.

23/03/2016
NB. This principle is similar to the learning scheme outlined some
years later by Donald Hebb.
James, ‘On thinking’ (2)

Second principle:


The amount of activity at any given point in the brain cortex is the sum tendencies of all
the other points that discharge into it, such tendencies being proportionate:

to the number of times the excitement of each point may have accompanied the point in
question;

to the intensity of the excitements;

to the absence of any rival point, functionally disconnected with the first, into which the
discharges may have been diverted.
This principle is similar to the mathematical neural model outlined a few years later by
McCulloch & Pitts.
23/03/2016
Spontaneous thought
Spontaneous thought


1. Unrestricted ‘total association’ between arbitrary concepts:

E.g. [the memory of] a walk followed by a romantic dinner.

2. Partial association(s)

Not all memories are associated.

Why are some memories linked and not others?

3. Focussed association / recall by similarity

Why might, say, thoughts of the moon lead to thoughts of football?
23/03/2016
1. Unrestricted associations

Consider the association of ‘walk’
followed by ‘romantic thoughts’.




Let [A] be the memory of the ‘walk’.
 The pattern of excitement distributed
across neurons [a,b,c,d,e].
Let [B] be ‘romantic thoughts’.
 The pattern of excitement distributed
across neurons [l,m,n,o,p].
Thus [A] must excite [B]

And [B] must excite [A].
Hence James suggests the memories [A]
and [B] ‘vibrate in unison’.
23/03/2016
2. Partial association

Only some past experiences have associated consequences.

James suggests four principles which determine which experiences are associated
together; these are:
Habit: the more often something done the more likely it is to be associated.
Recency: more recent events are more likely to be recalled.
Vividness: the more intense an experience the more likely it is to be recalled.
Emotional congruity: Similar emotion backgrounds are more likely to be associated
together.





E.g. Feeling miserable makes it more difficult to recall times of joy.
How these principles of association work:



23/03/2016
If one is thinking A and A is associated with B, (by say habit), then one will
subsequently think B unless a stronger principle applies.
At any time the strongest principle of association is that which pertains.
3: Focalised recall / association by similarity
Think of the ‘gas-flame’…

And subsequent thoughts of the ‘moon’ [via a pale-whiteness; similarity of
colour]…



Which, in turn, is linked to the thoughts of ‘football’ [via roundness; similarity of
shape]..
So via focussed recall thought can moves from [A] to [B] (from
gas-flame to football) even though neither [A] nor [B] have any
properties in common.
23/03/2016
Voluntary thought

Typically this is difficult for associationists; James tries to
address the issue by showing how associationism can
perform recall of a forgotten thing [and via the same
mechanism] perform a form of ‘means-end’ analysis.

To recall a forgotten item:



Suppose memories [a,b,c] and [l,m,n] are all associated (&
fully interconnected) with forgotten item [Z].
Activation of [a,b,c] will eventually propagate to [l,m,n] and
together [a,b,c, l,m,n] will activate [Z].
Means-end analysis is performed in a similar way.


23/03/2016
If [a,b,c] is the goal this will eventually excite suggestions
[l,m,n] which together will excite the solution [Z].
NB. James never considers if all reasoning is performed via
means end or even if his version of means-end analysis will
work in practise; consider balancing a bank account ...
The link between Connectionism and
Associationism

Let input nodes to a ANN be sensory
transducers (producing 'sensations').

Let internal (hidden) network nodes encode
ideas

Let inter-node weights indicate strengths
between ideas
.
Let output nodes define behaviour.



… Then there appears to be a simple
correspondence between Connectionism
and Associationism.
23/03/2016
Analysing the link between Connectionism and
Associationism

Strictly speaking there are several ways in which this link breaks down:

In ‘Associationist Networks’ the items that get associated are specific idea(s). In local connectionist networks where one node encodes an item this
also holds.
However, in ANNs that use distributed encoding of knowledge this connection clearly does not hold.


In pure associationism ‘ideas’ are pure copies of sensations. In networks where weights from the input node are not unity [+1] this is not true information is scaled by the weight value..
NB. Associationism and connectionism can most easily represent the ‘arity zero’ predicates (propositions) of first order
logic, (cf. Stochastic Diffusion Networks/NESTER).
23/03/2016