Supplement for
“New Insights into Laplacian Similarity Search”
1
Xiao-Ming Wu1 Zhenguo Li2 Shih-Fu Chang1
Department of Electrical Engineering, Columbia University
2
Huawei Noah’s Ark Lab, Hong Kong
{xmwu,sfchang}@ee.columbia.edu
Abstract
li.zhenguo@huawei.com
Proof. By definition,
M
This is the supplement for our main paper “New Insights
into Laplacian Similarity Search” [3]. Here, we show the
proofs of all the theoretical arguments in the main paper.
=
(L + αΛ)−1
=
Λ− 2 (Λ− 2 LΛ− 2 + αI)−1 Λ− 2
!−1
n
X
1
1
(γi + α)ui ui >
Λ− 2
Λ− 2
=
Proof of Statements in Sec. 2.1: M is positive and symmetric, i.e., ∀i, j, mij > 0, and mij = mji . Regardless of
Λ, mii is always the unique largest element in the i-th column and row of M .
Proof. (a) Since L + αΛ is symmetric, M = (L + αΛ)
is symmetric.
(b) Note that
1
1
1
1
i=1
=
Λ
n
X
− 12
i=1
=
−1
α
1
P
i
1
ui ui >
γi + α
>
λi
11 + Λ
!
1
Λ− 2
n
X
− 21
i=2
1
ui u>
i
γi + α
1
M
=
(L + αΛ)
−1
= (D + αΛ − W )
!
1
Λ− 2 .
1
Proof of Corollary 2.2: lim E = Λ− 2 L̄† Λ− 2 .
−1
α→0
Proof. It follows from L̄† =
(I − (D + αΛ)−1 W )−1 (D + αΛ)−1
!
∞
X
=
[((D + αΛ)−1 )W ]k (D + αΛ)−1 ,
=
Pn
1
>
i=2 γi ui ui .
Proof of Statements in Sec. 2.1:
Ranking by (hij )i=1,...,n is equivalent as ranking by the j-th
1
1
column of D− 2 L†sym D− 2 .
k=0
from which we can see that M is positive since the graph is
connected.
(c) Now we show that mjj is the unique largest in its column. Assume, to the contrary, there exists i, j, i 6= j, such
that mjj ≤ mij . Denote k = arg maxi6=j mij . Note that
M is symmetric and M > 0. Let B = (bij ) := D + αΛ −
W . Note that B is symmetric
and strictly diagonally domiP
nant, i.e., ∀k, bkk >
|b
|. By BM =P
I, we have 0 =
ki
P i6=k
B(k, :)M (:,P
j) = i bki mij = bkk mP
kj +
i6=k bki mij ≥
bkk mkj − ( i6=k |bki |)mkj = (bkk − i6=k |bki |)mkj > 0,
which contradicts the assumption.
Proof. Let ei denote the i-th unit vector in Rn . The hitting
time that a random walk from vertex i to hit vertex j can be
computed by [1]:
1
1
1
Hij = d(V)h p ej , L†sym ( p ej − √ ei )i
di
dj
dj
1 > †
1
ej Lsym ej − p
ei > L†sym ej
dj
di dj
= d(V)
!
.
Thus given j, ranking by (hij )i=1,...,n is determined
by − √ 1 ei > L†sym ej . Denote by B = (bij ) :=
di dj
1
1
D− 2 L†sym D− 2 . Then bij = √ 1
Proof of Theorem 2.1:
1
M = C + E, where C = P
11> , and E =
α
λ
i
i
!
n
X
1
>
− 12
− 12
ui ui Λ .
Λ
γ +α
i=2 i
di dj
ei > L†sym ej . This
shows that ranking by (hij )i=1,...,n in ascending order is
the same as ranking by (bij )i=1,...,n in descending order.
Note that a smaller hij means vertices i and j are closer on
the graph.
1
P
Proof of Lemma 3.1: (a) [2] Lf (Sk ) =
j∈S̄k a1j ,
(b) limα→0 Lf (Sk ) = λ(S¯k )/λ(V), 1 ≤ k ≤ n.
Proof. (a) Recall that f is the first column of M = (L +
αΛ)−1 . We have (L + αΛ)f = e1 , which can be written as:
X
w1j (f1 − fj ) = 1 − αλ1 f1 ,
(1)
j6=1
X
wij (fi − fj ) = −αλi fi , i 6= 1.
(2)
1
=
limα→0 Rd (Sc )
Pc−1
¯
k=1 (d(Sc \Sk )+d(Sc ))
= c −
d(S¯c )
d(Sc \Sk )
d(Sc )
→ 0, we have d(S¯c )
d(S¯c )
Pc−1
Proof.
1 +
¯
k=1 d(Sk )
d(S¯ )
Pc−1c
k=1 d(Sc \Sk )
.
d(S¯c )
=
As
→ 0 for k < c, which
completes the proof.
lim Rd (Sc ) = 0,
lim
Proof of Lemma 3.7:
d(Sc )/d(S¯c )→∞ α→0
if d1 < td(Sc ) for a fixed scalar t, 0 < t < 1.
j6=i
Pc−1
¯
1
k=1 d(Sk )
=
limα→0 Rd (Sc )
d(S¯c )
d(S¯1 )+d1 −d1
c )−d1
c)
≥ d(S
≥ (1−t)d(S
→
d(S¯c )
d(S¯c )
d(S¯c )
Proof.
By Eq. (1) and Eq. (2), we have
Lf (Sk ) =
k X
X
wij (fi − fj ) = 1 −
i=1 j6=i
=1−
X
k
X
=
αλi fi
a1j =
X
a1j .
(3)
j∈S̄k
(L + αΛ)−1 αΛ
1
P
i
1
λi
11> Λ + αΛ− 2
n
X
i=2
1
ui u>
i
γi + α
!
1
Λ2 .
Therefore, limα→0 A = P1 λi 11> Λ. By Eq. (3), we have
i
limα→0 Lf (Sk ) = λ(S¯k )/λ(V).
Proof of Theorem 3.4: Rf (Sc ) < 1/(c − 1).
P
Proof. Since Lf (Sk ) = j∈S̄k a1j strictly decreases when
k increases, ∀k < c, Lf (Sc ) < Lf (Sk ).
Proof of Theorem 3.5:
(a) If di = b, ∀i, for some constant b, then
limα→0 Ri (Sc ) = limα→0 Rd (Sc ).
d(S¯c )
c \Sk )
(b) Suppose for 1 ≤ k < c, d(S
|Sc \Sk | > |S¯c | . Then
limα→0 Ri (Sc ) > limα→0 Rd (Sc ).
d(S¯c )
c \Sk )
(c) Suppose for 1 ≤ k < c, d(S
|Sc \Sk | < |S¯c | . Then
limα→0 Ri (Sc ) < limα→0 Rd (Sc ).
Proof. (a) It follows from d(S¯k ) = b|S¯k |, for 1 ≤ k ≤ c.
c \Sk )
k|
(b) Since for 1 ≤ k < c, d(S
> |S|cS\S
¯c | , we have
d(S¯c )
d(Sc \Sk )+d(S¯c )
d(S¯c )
|Sc \Sk |+|S¯c |
|S¯c |
=
>
=
(c) The proof is similar to that of (b).
Proof of Lemma 3.6:
lim
lim Rd (Sc ) =
d(Sc )/d(S¯c )→0 α→0
→
Proof of Theorem 3.8: Suppose for 1 ≤ k <
0
ˆ
c )+τ d
c \Sk )
< d(S \S
. Then limα→0 Ri (Sc ) <
c, d(S
|Sc \Sk |
|S¯c |
limα→0 Rh (Sc ).
Proof. Since for 1
|Sc \Sk |
, we have
|S¯c |
|Sc \Sk |+|S¯c |
=
|S¯c |
d(S 0 \Sc )+τ dˆ
>
d(S 0 \Sk )+τ dˆ
1
.
c−1
|S¯k |
.
|S¯c |
≤
d(Sc \Sk )
d(S 0 \Sc )+τ dˆ
d(Sc \Sk )+d(S 0 \Sc )+τ dˆ
d(S 0 \Sc )+τ dˆ
k
<
c,
d(S 0 \Sk )+τ dˆ
=
d(S 0 \Sc )+τ dˆ
|S¯k |
. Therefore, for 1 ≤
|S¯c |
|S¯c |
. This proves limα→0
|S¯k |
<
<
k < c,
Ri (Sc ) <
limα→0 Rh (Sc ).
Proof of Lemma 3.9:
lim
lim Rh (Sc ) =
α→0
ˆ
maxi∈Sc di /d→0
Proof.
1
limα→0 Rh (Sc )
Pc−1
d(S¯k )
d(S¯c )
≥
∞.
Note that in Eq. (3), since A = (aij )P
= (L + αΛ)−1 αΛ,
αλi fi = a1i . We also use the fact that j a1j = 1.
(b) By Theorem 2.1.,
=
∞,
d(S¯1 )
d(S¯c )
c)
as d(S
d(S¯c )
i=1
j∈Sk
A
≥
1
.
c−1
d(S 0 \Sk )+τ dˆ
d(S 0 \Sc )+τ dˆ
Pc−1
=
0
=
ˆ
k=1 d(Sc \Sk )
k=1 (d(Sc \Sk )+d(S \Sc )+τ d)
= c − 1 + d(S
0 \S )+τ d
ˆ .
d(S 0 \Sc )+τ dˆ
c
maxi∈Sc di
d(Sc \Sk )
As
→ 0, we have d(S 0 \S )+τ dˆ → 0 for k < c,
dˆ
c
which completes the proof.
Proof of Theorem 3.10: Suppose for 1 ≤ k < c,
S¯c )
> |S ∗ \Sd(|+d(
. Then limα→0 Rh (Sc ) >
S¯ )/dˆ
d(Sc \Sk )
|Sc \Sk |
c
∗
limα→0 Rd (Sc ).
Proof. Since for 1
≤
|Sc \Sk |dˆ
, we have
ˆ
|S ∗ \Sc |d+d(
S¯∗ )
∗
ˆ
ˆ
|Sc \Sk |d+|S \Sc |d+d(
S¯∗ )
=
ˆ
|S ∗ \Sc |d+d(
S¯∗ )
¯
d(Sc )
1 ≤ k < c, d(
<
S¯k )
d(Sc \Sk )
>
d(S¯c )
d(Sc \Sk )+d(S¯c )
d(S¯k )
=
>
d(S¯c )
d(S¯c )
∗
ˆ
¯
|S \Sk |d+d(S∗ )
. Therefore, for
ˆ
|S ∗ \Sc |d+d(
S¯∗ )
∗
ˆ
¯
|S \Sc |d+d(S∗ )
. This proves
ˆ
|S ∗ \Sk |d+d(
S¯∗ )
k
<
limα→0 Rd (Sc ) < limα→0 Rh (Sc ).
c,
References
[1] U. von Luxburg, A. Radl, and M. Hein. Hitting and
commute times in large random neighborhood graphs.
Journal of Machine Learning Research, 15:1751–1798,
2014. 1
[2] X.-M. Wu, Z. Li, and S.-F. Chang. Analyzing the harmonic structure in graph-based learning. In NIPS, 2013.
2
[3] X.-M. Wu, Z. Li, and S.-F. Chang. New insights into
laplacian similarity search. In CVPR, 2015. 1