advertisement

Supplemental Material for “Can Visual Recognition Benefit from Auxiliary Information in Training?” Qilin Zhang1 , Gang Hua1 , Wei Liu2 , Zicheng Liu3 , Zhengyou Zhang3 1 2 Stevens Institute of Technolog, Hoboken, NJ IBM Thomas J. Watson Research Center, Yorktown Heights, NY 3 Microsoft Research, Redmond, WA In this supplemental material, we present the proof of Proposition 1 in Section 1 and additional experimental results with the the RGBD Object dataset [1] in Section 2. 1 Proof of Proposition 1 In this section, we present the proof of the Proposition 1 for Paper 498. First, the existence of the upper bound is proven in Section 1.1, then the proof that the sequence f (W(s)), s = 1, 2, · · · is monotonic is presented in Section 1.2. With the Bolzano-Weierstrass theorem and the conclusions of Section 1.1 and Section 1.2, the Proposition 1 is proven. 1.1 Proof of the existence of the upper bound Cu From the constraint in Eq. (6) of the paper: 1 T T Wj (1 − τj ) Rj Rj + τj In Wj = I, n (1) where τj denotes the pre-specified regularization parameter, 0 < τj < 1 (j = 1, 2, · · · , J), we have 1 T Wj Rj RTj Wj + τj tr WjT Wj = n, (2) (1 − τj )tr n and therefore tr WjT Rj RTj Wj ≤ 1 , 0 < τj < 1, ∀j = 1, 2, · · · , J. 1 − τj In addition, we have 1h i tr WjT Rj RTk Wk ≤ tr WjT Rj RTj Wj + tr WkT Rk RTk Wk 2 1 1 1 + < ∞, ≤ 2 1 − τj 1 − τk (3) (4) (5) 2 Q. Zhang, G. Hua, W. Liu, Z. Liu, Z. Zhang where the inequality (4) follows the property 1 tr(AT A) + tr(BT B) , (6) 2 T which comes from the fact that tr (A − B) (A− B) ≥ 0, because the matrix (A − B)T (A − B) 0. Since tr WjT Rj RTk Wk is bounded, , g(x) = x or x2 , we have 2 1 1 1 < ∞. (7) g tr WjT Rj RTk Wk + ≤ 4 1 − τj 1 − τk tr(BT A) ≤ Considering cjk = 0 or 1, we have J X j6=k 2 J 1 T 1 X 1 1 cjk g tr < ∞, Wj Rj RTk Wk ≤ + n 4n 1 − τj 1 − τk (8) j6=k which shows that the sequence f (W(s)), s = 1, 2, · · · is upper bounded by Cu = 2 J 1 X 1 1 + < ∞. 4n 1 − τj 1 − τk (9) j6=k t u 1.2 Proof that the sequence f (W(s)), s = 1, 2, · · · is monotonically increasing In this section, the monotonic property of the sequence f (W(s)), s = 1, 2, · · · is presented. Following [2–4], we first present a Lemma, and then prove that f (W(s)) ≤ f (W(s + 1)), s = 1, 2, · · · , (10) where s is the iteration index, s = 1, 2, · · · . def Define the function r(Yj , Yk ) = tr( n1 YjT Yk ) = tr( n1 WjT Rj RTk Wk ), therefore, f (W1 (s), · · · , WJ (s)) = J X cjk g[r(Yj (s), Yk (s))]. (11) j,k=1,k6=j Using this notation, we have the following Lemma: Lemma 1. Define def fj (Wj ) = j−1 X J X cjk g r(RTj Wj , Yk (s + 1)) + cjk g r(RTj Wj , Yk (s)) k=1 k=j+1 (12) s.t. WjT Nj Wj = I, Visual Recognition with Auxiliary Information in Training 3 then fj (Wj (s)) ≤ fj (Wj (s + 1)), j = 1, · · · , J. (13) Proof. We prove Lemma 1 in two cases, i.e., g(x) = x2 and g(x) = x. Case 1: when g(x) = x2 , we have fj (Wj ) in the following form, fj (Wj ) = j−1 X J 2 2 X cjk r(RTj Wj , Yk (s + 1)) + cjk r(RTj Wj , Yk (s)) , k=1 k=j+1 (14) which can be written as j−1 1X (s) cjk θjk tr(Wj (s)T Rj Yk (s + 1)) fj (Wj (s)) = n k=1 J 1 X (s) cjk θjk tr(Wj (s)T Rj Yk (s)) (15) n k=j+1 j−1 J X X 1 (s) (s) cjk θjk Yk (s) , = tr Wj (s)T Rj cjk θjk Yk (s + 1) + n + k=1 k=j+1 (16) (s) where θjk are defined as (s) θjk ( r (Yj (s), Yk (s + 1)) if k = 1, · · · , j − 1 = . r (Yj (s), Yk (s)) if k = j + 1, · · · , J Note that in Eq. (16), the term P (s) j−1 k=1 cjk θjk Yk (s + 1) + (17) (s) k=j+1 cjk θjk Yk (s) PJ is equivalent to the definition of Zj (s), hence fj (Wj ) can be simplified as 1 T n tr(Wj Rj Zj (s)). Considering the following optimization problem: 1 maxWj tr(WjT Rj Zj (s)), s.t. WjT Nj Wj = I, n (18) whose solution is exactly † Wj (s + 1) = Nj−1 Rj Zj (s) [Zj (s)T RTj Nj−1 Rj Zj (s)]1/2 , (19) tr(Wj (s)T Rj Zj (s)) ≤ tr(Wj (s + 1)T Rj Zj (s)). (20) we have 4 Q. Zhang, G. Hua, W. Liu, Z. Liu, Z. Zhang Similarly, the following equations can be obtained: fj (Wj (s)) = j−1 X k=1 ≤ j−1 X J X (s) cjk θjk r(Yj (s), Yk (s + 1)) + (s) cjk θjk r(Yj (s), Yk (s)) k=j+1 (s) cjk θjk r(Yj (s + 1), Yk (s + 1)) k=1 J X + (s) cjk θjk r(Yj (s + 1), Yk (s)). (21) k=j+1 Considering that cjk is either 0 or 1, we hence have cjk = c2jk . Applying the Cauchy-Schwartz inequality, we have fj (Wj (s)) ≤ j−1 X (s) c2jk θjk r(Yj (s + 1), Yk (s + 1)) + k=1 J X (s) c2jk θjk r(Yj (s + 1), Yk (s)) k=j+1 v uj−1 J 2 2 X uX (s) (s) cjk θjk + cjk θjk ≤t k=1 k=j+1 v uj−1 J X uX 2 2 cjk (r(Yj (s + 1), Yk (s))) ·t cjk (r(Yj (s + 1), Yk (s + 1))) + k=1 k=j+1 v uj−1 J X uX 2 2 =t cjk (r(Yj (s), Yk (s + 1))) + cjk (r(Yj (s), Yk (s))) k=1 k=j+1 v uj−1 J X uX 2 2 ·t cjk (r(Yj (s + 1), Yk (s + 1))) + cjk (r(Yj (s + 1), Yk (s))) k=1 = q k=j+1 fj (Wj (s)) · q fj (Wj (s + 1)). (22) We immediately have fj (Wj (s)) ≤ fj (Wj (s + 1)). This concludes the case 1 scenario. Case 2: when g(x) = x, we have fj (Wj ) = j−1 X cjk r(RTj Wj , Yk (s + 1)) k=1 + J X cjk r(RTj Wj , Yk (s)). (23) k=j+1 (s) Therefore, we can have exactly the same equation as Eq. (16), except that θjk ≡ 1 for all the cases. The same equation as in Eq. (20) can be obtained, which directly implies that fj (Wj (s)) ≤ fj (Wj (s + 1)). This concludes both the case 2 scenario and the entire proof of Lemma. t u Visual Recognition with Auxiliary Information in Training 5 With the conclusion in Lemma 1, we proceed with the proof that the sequence f (W(s)), s = 1, 2, · · · is monotonically increasing. Consider the following subtraction J X [fj (Wj (s + 1)) − fj (Wj (s))] j=1 = j−1 J X X j=1 k=1 + 1 T T Wj (s + 1) Rj Rk Wk (s + 1) cjk g tr n J J X X j=1 k=j+1 − j−1 J X X j=1 k=1 − j−1 J X X j=1 k=1 = − 1h 2 1 T T Wj (s + 1) Rj Rk Wk (s) cjk g tr n 1 T T Wj (s) Rj Rk Wk (s + 1) cjk g tr n 1 cjk g tr Wj (s + 1)T Rj RTk Wk (s + 1) n J X cjk g tr j,k=1,k6=j J X cjk g tr j,k=1,k6=j 1 Wj (s + 1)T Rj RTk Wk (s + 1) n i 1 Wj (s)T Rj RTk Wk (s) ≥ 0. n (24) The last equation in Eq. (24) follows the Lemma 1. This implies that f (W1 (s), · · · , WJ (s)) ≤ f (W1 (s + 1), · · · , WJ (s + 1)) i.e., f (W(s)) ≤ f (W(s + 1)), s = 1, 2, · · · . (25) (26) Using Eq. (8), Eq. (26), the bounded sequence f (W(s)), s = 1, 2, · · · is monotonically increasing. According to the Bolzano-Weierstrass theorem, the sequence will converge, i.e., Proposition 1 is proven. 2 Additional Experimental Results With the same RGBD Object dataset and experimental settings from [1] and [5], we have also conducted additional experiments with the state-of-the-art HMPbased features [5]. These additional results are summarized in Table 1–2 (too many entries to fit in a single page). As is seen in Table 1–2, the overall recognition accuracy improves significantly across all methods, as compared to the EMK-based features [1]. In a large portion of the categories, perfect recognition is achieved even with the naive baseline 6 Q. Zhang, G. Hua, W. Liu, Z. Liu, Z. Zhang Table 1. Accuracy Table Part 1 for the Multi-View RGBD Object Instance recognition with HMP features, the highest and second highest values are colored red and blue, respectively. The remaining part is in Table 2. Category SVM SVM2K KCCA apple ball banana bell pepper binder bowl calculator camera cap cellphone cereal box coffee mug comb dry battery flashlight food bag food box food can food cup food jar garlic glue stick greens hand towel instant noodles keyboard kleenex lemon light bulb lime marker mushroom notebook onion orange peach pear pitcher plate pliers potato rubber eraser scissors shampoo soda can 87.62 100 73.74 82.07 100 83.08 100 99.17 100 98.43 100 92.88 96.67 86.34 97.34 100 100 98.08 96.32 98.73 90.91 100 76.76 99.63 100 94.55 98.87 45.02 87.67 67.78 100 99.42 100 98.74 72.46 100 90.04 100 99.66 92.58 63.71 69.61 100 99.35 100 93.33 100 77.78 84.06 100 86.54 100 100 100 95.29 100 85.45 97.33 84.14 97.34 100 99.84 93.02 98.90 100 98.40 100 89.73 100 99.51 95.05 97.36 47.41 97.26 62.22 93.93 100 100 94.64 97.58 100 93.24 100 100 89.96 71.81 75.49 100 99.68 100 92.86 100 81.31 78.88 100 86.54 100 100 100 95.81 100 96.90 97.33 85.46 97.87 100 100 94.39 97.06 100 98.40 100 89.19 100 99.76 95.05 97.36 47.41 96.58 61.11 98.26 100 100 95.90 99.03 100 90.75 100 100 90.39 74.13 71.57 100 99.68 99.55 KCCA RGCCA RGCCA RGCCA +L +L +AL 77.14 92.38 87.62 93.33 100 100 100 100 81.31 80.30 75.25 80.30 77.29 73.31 80.88 73.31 100 100 100 100 86.15 87.31 77.69 87.69 99.44 100 100 100 100 99.17 99.17 99.17 100 100 97.08 100 94.76 95.81 98.43 95.81 100 100 100 100 94.43 99.69 91.64 99.69 96.67 98.00 95.33 98.00 76.65 87.67 85.90 88.55 95.21 100 97.34 100 100 100 96.88 100 99.84 100 100 100 81.67 99.18 98.08 99.32 96.32 97.43 97.06 97.43 91.46 100 97.15 100 96.26 95.99 91.98 95.99 100 100 99.68 100 85.41 85.41 81.08 90.27 100 100 99.63 100 99.76 100 100 100 94.55 96.04 94.55 96.53 96.98 98.87 98.87 98.87 45.82 47.81 44.62 47.81 95.21 91.78 89.04 92.47 58.33 62.78 68.33 62.78 96.75 99.57 100 99.78 100 100 99.42 100 100 99.62 100 100 95.90 96.53 99.37 98.74 98.07 96.62 69.57 99.03 100 100 100 100 90.04 91.81 88.61 91.81 100 100 100 100 99.32 95.59 100 99.32 90.39 93.45 92.58 93.45 65.64 65.25 67.57 71.04 69.61 70.59 74.02 70.59 100 100 100 100 98.39 99.68 94.19 99.68 99.10 100 100 100 DCCA 93.33 100 80.30 83.27 100 87.69 100 99.17 100 95.81 100 100 98.00 88.55 100 100 100 99.32 98.53 100 95.99 100 91.35 100 100 96.53 98.87 47.81 92.47 62.22 99.57 100 100 98.74 99.03 100 93.95 100 99.32 93.45 71.43 71.08 100 99.68 100 Visual Recognition with Auxiliary Information in Training 7 Table 2. Continued from Table 1: Accuracy Table Part 2 for the Multi-View RGBD Object Instance recognition with HMP features, the highest and second highest values are colored red and blue, respectively. Category sponge stapler tomato tooth brush tooth paste water bottle average SVM 99.49 80.12 69.69 99.48 100 97.32 92.62 SVM2K 100 73.59 81.59 100 100 95.98 93.35 KCCA KCCA+L RGCCA RGCCA+L RGCCA+AL DCCA 100 99.83 99.83 99.83 99.83 99.83 76.56 76.56 81.90 81.90 82.20 82.49 83.85 83.85 80.45 68.56 80.45 80.45 99.48 96.91 98.97 99.48 98.97 98.97 100 100 100 100 100 100 95.71 94.64 96.25 96.78 96.25 96.25 93.87 91.85 93.93 92.40 94.35 94.62 SVM algorithm, However, the advantage of the proposed method is revealed in some of the challenging categories. Overall, with the HMP-based features, “KCCA+L” and the “RGCCA+L” algorithms cannot match the SVM baseline, and the “SVM2K” and “KCCA” algorithms are only marginally better than the baseline. The “RGCCA” and “RGCCA+AL” algorithms offer some improvements, while the proposed DCCA algorithm achieves the highest overall recognition accuracy. Considering there are more than 13,000 testing samples, the two percent performance improvement means correctly classifying an additional amount of more than 200 samples. References 1. Lai, K., Bo, L., Ren, X., Fox, D.: A large-scale hierarchical multi-view rgb-d object dataset. In: Robotics and Automation (ICRA), 2011 IEEE International Conference on, IEEE (2011) 1817–1824 2. Tenenhaus, A., Tenenhaus, M.: Regularized generalized canonical correlation analysis. Psychometrika 76 (2011) 257–284 3. Tenenhaus, A.: Kernel generalized canonical correlation analysis. Actes des 42è journées de Statistique (2010) 4. Hanafi, M.: Pls path modelling: computation of latent variables with the estimation mode b. Computational Statistics 22 (2007) 275–292 5. Bo, L., Ren, X., Fox, D.: Unsupervised feature learning for rgb-d based object recognition. ISER, June (2012)