Part 3: Autonomous Agents
11/5/04
Positive Coupling
Negative Coupling
• Positive sense (sign)
• Large strength
11/5/04
• Negative sense (sign)
• Large strength
1
11/5/04
2
State = –1 & Local Field < 0
Weak Coupling
• Either sense (sign)
• Little strength
h<0
11/5/04
3
11/5/04
4
1
Part 3: Autonomous Agents
11/5/04
State = –1 & Local Field > 0
State Reverses
h>0
h>0
11/5/04
11/5/04
5
11/5/04
6
State = +1 & Local Field > 0
State = +1 & Local Field < 0
h>0
h<0
7
11/5/04
8
2
Part 3: Autonomous Agents
11/5/04
Hopfield Net as Soft Constraint
Satisfaction System
State Reverses
• States of neurons as yes/no decisions
• Weights represent soft constraints between
decisions
h<0
– hard constraints must be respected
– soft constraints have degrees of importance
• Decisions change to better respect
constraints
• Is there an optimal set of decisions that best
respects all constraints?
11/5/04
9
10
Quantifying “Tension”
Convergence
• Does such a system converge to a stable
state?
• Under what conditions does it converge?
• There is a sense in which each step relaxes
the “tension” in the system
• But could a relaxation of one neuron lead to
greater tension in other places?
11/5/04
11/5/04
11
• If wij > 0, then si and sj want to have the same sign
(si sj = +1)
• If wij < 0, then si and sj want to have opposite
signs (si sj = –1)
• If wij = 0, their signs are independent
• Strength of interaction varies with |wij|
• Define disharmony (“tension”) Dij between
neurons i and j:
Dij = – si w ij s j
Dij < 0 they are happy
Dij > 0 they are unhappy
11/5/04
12
3
Part 3: Autonomous Agents
11/5/04
Review of Some Vector Notation
Total Energy of System
x1 T
x = M = ( x1 L x n )
xn The “energy” of the system is the total
“tension” (disharmony) in it:
E {s} = Dij = si w ij s j
ij
n
x T y = xi yi = x y
= 12 si w ij s j
x1 y1
xy = M
x m y1
= 12 si w ij s j
j
x T My = = 12 sT Ws
11/5/04
13
Another View of Energy
=
=
n
j=1
x i M ij y j
14
E = E {s} E {s}
= siw ij sj + si w ij s j
ij
ij
= sk w kj s j + sk w kj s j
s w s
i
(quadratic form)
• Suppose that neuron k changes state
• Change of energy:
ij j
jk
j
jk
= ( sk sk ) w kj s j
s h
i i
i
= sk hk
1 T
2
= s h
11/5/04
i=1
j
i
1
2
m
(outer product)
11/5/04
E {s} = 12 si w ij s j
1
2
L
Do State Changes Decrease Energy?
The energy measures the number of neurons
whose states are in disharmony with their
local fields (i.e. of opposite sign):
i
x1 y n M xm yn L
O
T
ji
i
(inner product)
i=1
ij
i
(column vectors)
15
11/5/04
<0
jk
16
4
Part 3: Autonomous Agents
11/5/04
Proof of Convergence
in Finite Time
Energy Does Not Increase
• In each step in which a neuron is considered
for update:
E{s(t + 1)} – E{s(t)} 0
• Energy cannot increase
• Energy decreases if any neuron changes
• Must it stop?
11/5/04
17
• There is a minimum possible energy:
– The number of possible states s {–1, +1}n is
finite
– Hence Emin = min {E(s) | s {±1}n} exists
• Must show it is reached in a finite number
of steps
11/5/04
Steps are of a Certain Minimum Size
If hk > 0, then (let hmin = min of possible positive h)
n
hk minh h = w kj s j s {±1} h > 0 = df hmin
jk
E = sk hk = 2hk 2hmin
If hk < 0, then (let hmax = max of possible negative h)
n
hk maxh h = w kj s j s {±1} h < 0 = df hmax
jk
18
Conclusion
• If we do asynchronous updating, the
Hopfield net must reach a stable, minimum
energy state in a finite number of updates
• This does not imply that it is a global
minimum
E = sk hk = 2hk 2hmax
11/5/04
19
11/5/04
20
5
Part 3: Autonomous Agents
11/5/04
Lyapunov Functions
Example Limit Cycle with
Synchronous Updating
• A way of showing the convergence of discreteor continuous-time dynamical systems
• For discrete-time system:
– need a Lyapunov function E (“energy” of the state)
– E is bounded below (E{s} > Emin)
– E < (E)max 0 (energy decreases a certain
minimum amount each step)
– then the system will converge in finite time
w>0
w>0
• Problem: finding a suitable Lyapunov function
11/5/04
21
The Hopfield Energy Function is
Even
• A function f is odd if f (–x) = – f (x),
for all x
• A function f is even if f (–x) = f (x),
for all x
• Observe:
11/5/04
22
Conceptual
Picture of
Descent on
Energy
Surface
E {s} = 12 (s) T W(s) = 12 sT Ws = E {s}
11/5/04
23
11/5/04
(fig. from Solé & Goodwin)
24
6
Part 3: Autonomous Agents
11/5/04
Energy
Surface
+
Flow
Lines
Energy
Surface
11/5/04
(fig. from Haykin Neur. Netw.)
25
11/5/04
Flow
Lines
(fig. from Haykin Neur. Netw.)
27
26
Bipolar
State
Space
Basins of
Attraction
11/5/04
(fig. from Haykin Neur. Netw.)
11/5/04
28
7
Part 3: Autonomous Agents
11/5/04
Basins
in
Bipolar
State
Space
energy decreasing paths
11/5/04
29
8