An iterative construction of solutions of
the TAP equations.
Erwin Bolthausen, University of Zürich
Disordered Media, Warwick, Sept. 7, 2011
An iterative construction of solutions of the TAP equations.
2
Sherrington-Kirkpatrick model:
Random interactions: Independent centered Gaussians gij ; i < j; with variance
1=N;de ned on ( ; F; P) :
Inverse temperature > 0; strength h 0 of the external eld.
Hamiltonian:
def
HN;! ( ) =
X
gij (!)
i j
+h
i=1
1 i<j N
Partition function:
def
ZN;! =
Gibbs measure on
N
:
N
X
X
exp [HN;! ( )] :
exp [HN;! ( )]
:
ZN;!
Gibbs-expectations ( xed ! ) are written as h i :
def
mi (!) = h ii :
i;
i
2 f 1; 1g
An iterative construction of solutions of the TAP equations.
3
TAP equations (Thouless-Anderson-Palmer):
XN
gij mj
mi = tanh h +
2
j=1
(1
q) mi ;
where gij = gji; gii = 0; and q = q ( ; h) is the (unique for h > 0) solutions of
Z
1
p
2
def
q = tanh2 (h +
qx) (dx) ; (dx) = p e x =2dx:
2
Has to be understood in limiting N ! 1 sense.
The Onsager term 2 (1 q) mi is an Itô type correction: Standard mean- eld theory:
X
mi ' tanh h +
gij mj :
j
i
For SK correct by replacing mj on the rhs by mcut
from the system with the connecj
tions to i cut. Expanding in rst order for the gij :
i
mcut
' mj
j
Mathematical proofs of TAP only for small
1
m2j migij :
: Talagrand, Chatterjee.
An iterative construction of solutions of the TAP equations.
4
Free energy:
1
1
f ( ; h) = lim
log ZN = lim
E log ZN :
N !1 N
N !1 N
The replica symmetric “solution”, given by SK:
Z
p
RS ( ; h) =
log cosh (h +
qz) (dz)
def
2
+
4
(1
q)2 + log 2:
Aizenman-Lebowitz-Ruelle (h = 0), Talagrand (h 6= 0):
Theorem For small enough :
f ( ; h) = RS ( ; h)
“Small enough
satis ed:
” is believed to mean that the de Almayda–Thouless-condition is
(AT) :
2
Z
(dz)
p
cosh4 h +
qz
1:
An iterative construction of solutions of the TAP equations.
5
Proposal for a direct
construction of “solutions” of TAP: Assume h > 0 : Recursive
n
o
(k)
approximations mi
: Given (gij )
1 i N
(0) def
(k) def
mi =
tanh h +
(1) def
mi
= 0; mi
X
gij mj
=
(k 1)
j
p
2
q;
(1
(k 2)
q) mi
Questions:
Structure of the dependence of m(k) on (gij )?
Convergence as k ! 1?
Relation to SK?
Second point:
Theorem: (AT) is satis ed iff
N
1 X
(k)
lim
lim
E
m
i
k;k 0 !1 N !1 N
i=1
(k 0 )
mi
2
= 0:
; k
2:
An iterative construction of solutions of the TAP equations.
6
Proof by an evaluation of
Theorem
where
N
X
1
def
(k) (j)
mi mi :
(j; k) = lim E
N !1 N
i=1
(k; k) = q; 8k;
j+1
=
: [0; q] ! (0; q] de ned by
Z
p
p
(t) = tanh h +
tx1 +
q
j
;
(j; k) =
j;
8j < k;
p
= q (0) < q:
def
1
tx2 tanh h +
(q) = q:
p
tx1 +
p
q
tx3
3
(dx) :
An iterative construction of solutions of the TAP equations.
Lemma
7
is increasing and convex on [0; q] : Furthermore
Z
(dx)
0
2
(q) =
:
p
4
cosh h +
qx
n
( 1) ! q () (AT)
An iterative construction of solutions of the TAP equations.
8
Structure of the (gij )-dependence of the m(k) : Alternative representation:
X (k 1) (k 1)
Xk 2
(k)
(r)
mi ' tanh h +
gij mj
+
;
r i
j
def
r
r
= q
q
Pr
1
j=1
Pr
1
j=1
r=1
2
j
2
j
;
(r) def
i =
X
(r)
(r)
^j :
gij m
j
^ (1); m
^ (2); : : : come from m(1); m(2); : : : via Gram-Schmidt in RN w.r.t. the inner product
m
1 X
hx; yi =
xi yi :
i
N
(1)
Let Fk =
L g (k) Fk
2
;:::;
(k)
^ (k) are Fk 1-measurable.
: m(k); m
Gaussian, and g (k) is conditionally independent of Fk 1:
X1
2
(AT) ()
r =q
r=1
An iterative construction of solutions of the TAP equations.
1 X hX (k
gij
(AT) ()
j
N i
9
1)
(k 1)
mj
i2
!k!1 0:
Using this representation, one can prove the claims by iteratively applying the LLN,
conditionally successively on Fk 2; Fk 3; : : : .
For that one needs the conditional covariance structure of the (k) :
(
1
1
+
O
N
for i = j;
(k) (k)
(r)
(k)
E i j Fk 1 = m^ (k)
Pk 1 m^ (r)
^j
^j
i m
i m
1
+ O N 2 for i 6= j:
r=1
N
N
N
Illustration from the rst steps:
X
p
p (1)
(2)
(1) def
(1)
q i ; i =
gij ;
mi = q; mi = tanh h +
j
Z
1 XN
p
(2)
mi ' tanh (h +
qx) (dx) = 1;
i=1
N
Z
X
N
1
p
(2)2
mi ' tanh2 (h +
qx) (dx) = q;
i=1
N
are evident from LLN.
An iterative construction of solutions of the TAP equations.
(3)
mi
10
X
= tanh h +
(2)
j
2
gij mj
(1
q)
p
(1)
Here one does the shift from g to g (2) which is independent of
(2)
gij ' gij
N
1
(1)
i
q :
+
(1)
j
Z
x tanh (h +
: Essentially
:
(3)
The correction inside the tanh of mi :
X (1)
(2)
(1)
1
m
'
N
+
j
i
j
(1)
1 i
+
j
=
=)
(3)
mi
' tanh h +
In the general case: g ! g (2) ! g (3)
produces the r (r) terms.
(1)
1 i
+
X
2p
(2)
j
q (1
(2)
gij mj +
p
qx) (dx)
q) :
(1)
i
:
successively eats up the Onsager term and
An iterative construction of solutions of the TAP equations.
11
Open problem: Behavior beyond the (AT) line of
X (k 1) (k 1)
gij mj
:
j
Original motivation: (Re)prove f ( ; h) = RS ( ; h) hopefully up to (AT), by a change
of measure argument:
X
N
2 ZN =
exp [ H ( )] P coin toss ( ) :
ptilt
mi ( i ) =
exp [hi i] coin
p
cosh (hi)
toss
( i) ; mi = tanh (hi) :
The mi from the TAP approximations.
h
X
1
1
log ZN =
log
exp H ( )
N
N
1 X
+
log cosh (hi):
N i
|R
{z
}
' log cosh(h+ x) (dx)
X
i
hi
i
i
P tilt ( ) + log 2
An iterative construction of solutions of the TAP equations.
After some computations
h
X
exp H ( )
X
def
where ^ i =
2
exp 4
12
X
X
i<j
i
hi
i
i
gij ^ i ^ j
2
P tilt ( ) ' exp
2
2N
X
i<j
3
N
(1
4
^ 2i ^ 2j 5 P tilt ( ) ;
q)2
mi: For the latter factor one should get (by second moment)
1
log [ ] = 0:
lim
N !1 N
For small ; this is o.k., but not up to (AT).
i
An iterative construction of solutions of the TAP equations.
13
My favorite spin glass: Perceptron: gij ; 1
N 1:
def
H( ) =
N
X
f
i=1
XN
|
gij j = N
{z
}
j=1
LN;
N i.i.d. Gaussian with variance
i; j
y
;i
N
1 X
=
N i=1
Z
def
y
;i
f (x) LN; (dx)
:
Question: Is there a quenched LDP in the sense that
# f : LN;
g
2N exp [ N J ( )] ; a:s:
A natural question: Given the gij ; is there a typical law of those 's for which LN;
?