Neural Networks
Dr. Peter Phillips
The Human Brain (Recap of week 1)
A Classic Artificial Neuron (Recap cont.)
X1
W1
X2
W2
f(Sj)
Output
W3
X3
Sj
Unsupervised Learning
• Today’s lecture will consider the use of
Self Organising Map (SOM) and
Unsupervised Learning
• Recall that Supervised Learning
matches inputs to outputs.
• Unsupervised Learning Classifies the
data into classes
The Biological Basis for Unsupervised
Neural Networks
• Major sensory and motor systems
are ‘topographically mapped’ in the
brain
–Vision: retinotopic map
–Hearing: tonotopic map
–Touch: somatotopic map
Kohonen Self-Organising Maps
• The most famous unsupervised
learning network is the Kohonen
Network.
• Neural network algorithm using
unsupervised competitive
learning
• Primarily used for organization
and visualization of complex data
Teuvo Kohonen
Understanding the Data Set
• A good understanding of the data set is
essential to use a SOM – or any network for
that matter
• A ‘distance measure’ and/or suitable
rescaling must be defined to allow
meaningful comparison
• The data must be of good quality and must
be representative of the application area
SOM - Architecture
j
2d array of neurons
wj1
wj2 wj3
x1
x2
x3
wjn
Weighted synapses
...
xn
Set of input signals
(connected to all neurons in lattice)
Finding a Winner (2)
• Euclidean distance between two vectors a and b, a
= (a1,a2,…,an), b = (b1,b2,…bn), is calculated as:
d a, b 
 a
 bi 
2
i
i
• i.e. Pythagoras’ Theorem
• Other distance measures
could be used, e.g.
Manhattan distance
Euclidean distance
SOM Parameters
• The learning rate and neighbourhood
function define to what extent the
weights of each node are adjusted
Neighbourhood function
Degree of
neighbourhood
Degree of
neighbourhood
1
1
Time
0.5
0.5
8
Distance from winner
Time
10
6
4
2
-2
-4
-6
-8
-1
0
10
8
6
4
2
0
-2
-4
-8
-1
0
-6
Distance from winner
0
0
0
Data For Tutorial Work
• Data collected from a UHT plant at
Leatherhead
• Consists of 300 cases use 150 for Training
and 150 for Testing
• Data collected with plant running in normal
state, during cleaning of exchangers and
with fault
Tutorial 2 (UHT Plant Data)
UHT plant cleaned and after a period of norm al operation returning to fault condition
1.2
1
0.8
0.6
0.4
0.2
0
1
15
29
43
57
71
85
99
113 127 141 155 169 183 197 211 225 239 253 267 281 295 309 323 337 351 365 379 393
-0.2
Cleaning
Normal
Fault
My settings
First run 70 epochs – learning rate 0.6 to 0.1
Second run 50 epochs – learning rate constant
0.1
First run Neighbourhood kept to 1
Second run Neighbourhood start 1 end 0
Trajan Classification
SOM – New Data
• A trained SOM can be used to classify new
input data
• The input data is classified to the node with
the ‘best’ or ‘closest’ weights
• Previous knowledge of other data samples
assigned to the same class enable inferences
to be made about the ‘new’ input sample