Supplement for “New Insights into Laplacian Similarity Search” 1 Xiao-Ming Wu1 Zhenguo Li2 Shih-Fu Chang1 Department of Electrical Engineering, Columbia University 2 Huawei Noah’s Ark Lab, Hong Kong {xmwu,sfchang}@ee.columbia.edu Abstract li.zhenguo@huawei.com Proof. By definition, M This is the supplement for our main paper “New Insights into Laplacian Similarity Search” [3]. Here, we show the proofs of all the theoretical arguments in the main paper. = (L + αΛ)−1 = Λ− 2 (Λ− 2 LΛ− 2 + αI)−1 Λ− 2 !−1 n X 1 1 (γi + α)ui ui > Λ− 2 Λ− 2 = Proof of Statements in Sec. 2.1: M is positive and symmetric, i.e., ∀i, j, mij > 0, and mij = mji . Regardless of Λ, mii is always the unique largest element in the i-th column and row of M . Proof. (a) Since L + αΛ is symmetric, M = (L + αΛ) is symmetric. (b) Note that 1 1 1 1 i=1 = Λ n X − 12 i=1 = −1 α 1 P i 1 ui ui > γi + α > λi 11 + Λ ! 1 Λ− 2 n X − 21 i=2 1 ui u> i γi + α 1 M = (L + αΛ) −1 = (D + αΛ − W ) ! 1 Λ− 2 . 1 Proof of Corollary 2.2: lim E = Λ− 2 L̄† Λ− 2 . −1 α→0 Proof. It follows from L̄† = (I − (D + αΛ)−1 W )−1 (D + αΛ)−1 ! ∞ X = [((D + αΛ)−1 )W ]k (D + αΛ)−1 , = Pn 1 > i=2 γi ui ui . Proof of Statements in Sec. 2.1: Ranking by (hij )i=1,...,n is equivalent as ranking by the j-th 1 1 column of D− 2 L†sym D− 2 . k=0 from which we can see that M is positive since the graph is connected. (c) Now we show that mjj is the unique largest in its column. Assume, to the contrary, there exists i, j, i 6= j, such that mjj ≤ mij . Denote k = arg maxi6=j mij . Note that M is symmetric and M > 0. Let B = (bij ) := D + αΛ − W . Note that B is symmetric and strictly diagonally domiP nant, i.e., ∀k, bkk > |b |. By BM =P I, we have 0 = ki P i6=k B(k, :)M (:,P j) = i bki mij = bkk mP kj + i6=k bki mij ≥ bkk mkj − ( i6=k |bki |)mkj = (bkk − i6=k |bki |)mkj > 0, which contradicts the assumption. Proof. Let ei denote the i-th unit vector in Rn . The hitting time that a random walk from vertex i to hit vertex j can be computed by [1]: 1 1 1 Hij = d(V)h p ej , L†sym ( p ej − √ ei )i di dj dj 1 > † 1 ej Lsym ej − p ei > L†sym ej dj di dj = d(V) ! . Thus given j, ranking by (hij )i=1,...,n is determined by − √ 1 ei > L†sym ej . Denote by B = (bij ) := di dj 1 1 D− 2 L†sym D− 2 . Then bij = √ 1 Proof of Theorem 2.1: 1 M = C + E, where C = P 11> , and E = α λ i i ! n X 1 > − 12 − 12 ui ui Λ . Λ γ +α i=2 i di dj ei > L†sym ej . This shows that ranking by (hij )i=1,...,n in ascending order is the same as ranking by (bij )i=1,...,n in descending order. Note that a smaller hij means vertices i and j are closer on the graph. 1 P Proof of Lemma 3.1: (a) [2] Lf (Sk ) = j∈S̄k a1j , (b) limα→0 Lf (Sk ) = λ(S¯k )/λ(V), 1 ≤ k ≤ n. Proof. (a) Recall that f is the first column of M = (L + αΛ)−1 . We have (L + αΛ)f = e1 , which can be written as: X w1j (f1 − fj ) = 1 − αλ1 f1 , (1) j6=1 X wij (fi − fj ) = −αλi fi , i 6= 1. (2) 1 = limα→0 Rd (Sc ) Pc−1 ¯ k=1 (d(Sc \Sk )+d(Sc )) = c − d(S¯c ) d(Sc \Sk ) d(Sc ) → 0, we have d(S¯c ) d(S¯c ) Pc−1 Proof. 1 + ¯ k=1 d(Sk ) d(S¯ ) Pc−1c k=1 d(Sc \Sk ) . d(S¯c ) = As → 0 for k < c, which completes the proof. lim Rd (Sc ) = 0, lim Proof of Lemma 3.7: d(Sc )/d(S¯c )→∞ α→0 if d1 < td(Sc ) for a fixed scalar t, 0 < t < 1. j6=i Pc−1 ¯ 1 k=1 d(Sk ) = limα→0 Rd (Sc ) d(S¯c ) d(S¯1 )+d1 −d1 c )−d1 c) ≥ d(S ≥ (1−t)d(S → d(S¯c ) d(S¯c ) d(S¯c ) Proof. By Eq. (1) and Eq. (2), we have Lf (Sk ) = k X X wij (fi − fj ) = 1 − i=1 j6=i =1− X k X = αλi fi a1j = X a1j . (3) j∈S̄k (L + αΛ)−1 αΛ 1 P i 1 λi 11> Λ + αΛ− 2 n X i=2 1 ui u> i γi + α ! 1 Λ2 . Therefore, limα→0 A = P1 λi 11> Λ. By Eq. (3), we have i limα→0 Lf (Sk ) = λ(S¯k )/λ(V). Proof of Theorem 3.4: Rf (Sc ) < 1/(c − 1). P Proof. Since Lf (Sk ) = j∈S̄k a1j strictly decreases when k increases, ∀k < c, Lf (Sc ) < Lf (Sk ). Proof of Theorem 3.5: (a) If di = b, ∀i, for some constant b, then limα→0 Ri (Sc ) = limα→0 Rd (Sc ). d(S¯c ) c \Sk ) (b) Suppose for 1 ≤ k < c, d(S |Sc \Sk | > |S¯c | . Then limα→0 Ri (Sc ) > limα→0 Rd (Sc ). d(S¯c ) c \Sk ) (c) Suppose for 1 ≤ k < c, d(S |Sc \Sk | < |S¯c | . Then limα→0 Ri (Sc ) < limα→0 Rd (Sc ). Proof. (a) It follows from d(S¯k ) = b|S¯k |, for 1 ≤ k ≤ c. c \Sk ) k| (b) Since for 1 ≤ k < c, d(S > |S|cS\S ¯c | , we have d(S¯c ) d(Sc \Sk )+d(S¯c ) d(S¯c ) |Sc \Sk |+|S¯c | |S¯c | = > = (c) The proof is similar to that of (b). Proof of Lemma 3.6: lim lim Rd (Sc ) = d(Sc )/d(S¯c )→0 α→0 → Proof of Theorem 3.8: Suppose for 1 ≤ k < 0 ˆ c )+τ d c \Sk ) < d(S \S . Then limα→0 Ri (Sc ) < c, d(S |Sc \Sk | |S¯c | limα→0 Rh (Sc ). Proof. Since for 1 |Sc \Sk | , we have |S¯c | |Sc \Sk |+|S¯c | = |S¯c | d(S 0 \Sc )+τ dˆ > d(S 0 \Sk )+τ dˆ 1 . c−1 |S¯k | . |S¯c | ≤ d(Sc \Sk ) d(S 0 \Sc )+τ dˆ d(Sc \Sk )+d(S 0 \Sc )+τ dˆ d(S 0 \Sc )+τ dˆ k < c, d(S 0 \Sk )+τ dˆ = d(S 0 \Sc )+τ dˆ |S¯k | . Therefore, for 1 ≤ |S¯c | |S¯c | . This proves limα→0 |S¯k | < < k < c, Ri (Sc ) < limα→0 Rh (Sc ). Proof of Lemma 3.9: lim lim Rh (Sc ) = α→0 ˆ maxi∈Sc di /d→0 Proof. 1 limα→0 Rh (Sc ) Pc−1 d(S¯k ) d(S¯c ) ≥ ∞. Note that in Eq. (3), since A = (aij )P = (L + αΛ)−1 αΛ, αλi fi = a1i . We also use the fact that j a1j = 1. (b) By Theorem 2.1., = ∞, d(S¯1 ) d(S¯c ) c) as d(S d(S¯c ) i=1 j∈Sk A ≥ 1 . c−1 d(S 0 \Sk )+τ dˆ d(S 0 \Sc )+τ dˆ Pc−1 = 0 = ˆ k=1 d(Sc \Sk ) k=1 (d(Sc \Sk )+d(S \Sc )+τ d) = c − 1 + d(S 0 \S )+τ d ˆ . d(S 0 \Sc )+τ dˆ c maxi∈Sc di d(Sc \Sk ) As → 0, we have d(S 0 \S )+τ dˆ → 0 for k < c, dˆ c which completes the proof. Proof of Theorem 3.10: Suppose for 1 ≤ k < c, S¯c ) > |S ∗ \Sd(|+d( . Then limα→0 Rh (Sc ) > S¯ )/dˆ d(Sc \Sk ) |Sc \Sk | c ∗ limα→0 Rd (Sc ). Proof. Since for 1 ≤ |Sc \Sk |dˆ , we have ˆ |S ∗ \Sc |d+d( S¯∗ ) ∗ ˆ ˆ |Sc \Sk |d+|S \Sc |d+d( S¯∗ ) = ˆ |S ∗ \Sc |d+d( S¯∗ ) ¯ d(Sc ) 1 ≤ k < c, d( < S¯k ) d(Sc \Sk ) > d(S¯c ) d(Sc \Sk )+d(S¯c ) d(S¯k ) = > d(S¯c ) d(S¯c ) ∗ ˆ ¯ |S \Sk |d+d(S∗ ) . Therefore, for ˆ |S ∗ \Sc |d+d( S¯∗ ) ∗ ˆ ¯ |S \Sc |d+d(S∗ ) . This proves ˆ |S ∗ \Sk |d+d( S¯∗ ) k < limα→0 Rd (Sc ) < limα→0 Rh (Sc ). c, References [1] U. von Luxburg, A. Radl, and M. Hein. Hitting and commute times in large random neighborhood graphs. Journal of Machine Learning Research, 15:1751–1798, 2014. 1 [2] X.-M. Wu, Z. Li, and S.-F. Chang. Analyzing the harmonic structure in graph-based learning. In NIPS, 2013. 2 [3] X.-M. Wu, Z. Li, and S.-F. Chang. New insights into laplacian similarity search. In CVPR, 2015. 1