advertisement

Complex Support Vector Machines for Regression and Quaternary Classification P.Bouboulis, S.Theodoridis, C.Mavroforakis, L.Dalla Learning (MKL) 19, 2010 (?) (I got it from arXiv) Coffeetalk, 1 May 2014 donderdag 1 mei 2014 Classification of complex data • Classifying bird songs: ï3 8 8000 x 10 6 7000 4 6000 2 frequency 5000 0 ï2 4000 3000 ï4 2000 ï6 ï8 1000 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0 1 2 3 4 5 time (s) 6 7 8 9 • But... the spectrogram contains complex values! • Can I classify a complex-valued image? 2 donderdag 1 mei 2014 Complex support vector machine • Inputs are complex vectors: • Output can be: r ˜ = x + ix x i • a complex number (regression) • a class label (classification) • To output a label: � For real valued SVM: ≥ 0 assign to ω1 f (x) = �w, x� + b < 0 assign to ω2 For complex SVM: ˜ x ˜ �H + b f (x) = �w, 3 donderdag 1 mei 2014 Complex inner product • You can define inner product in different ways: • Simple: r r i i ˜ x ˜ �H = �w , x � + �w , x � �w, • Just a concatenation of the real and imaginary spaces. • You can use the same SVM as always... • Complex: � � i r r i r r i i ˜ x ˜ �H = �w , x � + �w , x � + i �w , x � − �w , x � �w, • Now the output is not so clear: ˜ x ˜ �H + b) sign (�w, sign of a complex number? donderdag 1 mei 2014 4 v∗ Not 2-class, but 4-class classifier! Fig. 4. A complex couple of hyperplanes separates the space of complex numbers (i.e., H = C) into four parts. 2 ∂L =w− ∗ c and for n = 1, . . . finally take tha 5. A complex coupledistinguish of hyperplanes thatFOUR separates classes: the four given • Now theFig. classifier can classes. The hyperplanes are chosen so that to maximize the margin between the classes. ˜ x ˜ �H + b)) ≷ 0 sign (Re(�w, ˜ ˜ sign (Im(� w, x � + b)) ≷ 0 H !" $" #$ r r $2 ! w +v ! $ ! wi − v i $ 2 H n=1 #$2 and $ −(w + v ) $ $ +$ $ wr − v r $ 2 = H i i "wr + v r "2H + "wi − v i "2H + "(wi + v i )"2H + "wr − v r "2H = donderdag 1 mei 2014 r 2 i 2 N % r 2 i 2 5 Further things in the paper... • To find the Quadratic program (or, find derivatives of real valued functions in a complex space), use “Wirtinger’s calculus (it appears you can write it as two independent quadratic programming problems) • Derive for both regression as classification • Propose kernel mappings for complex data: ΦC (˜ x) 6 donderdag 1 mei 2014 w "H + 2"v "H + 2"v "H = 2("w"2H + "v"2H ). N for n = 1, . . . , N . Following a similar procedure as in complex SVR case, it turns out that the dual problem ca split into two separate maximization tasks: For your information: SVM optimization problem H + N ! • It C N looks like: (ξnr + ξni ) a n=1 Φ∗C (z n ), v#H + c) Φ∗C (z n ), w#H + c) maximize ≥ ≥ ≥ 1− 1− 0 ξnr ξni • Labels d = ±1 ± i 1, . . . , N. n (four classes) (28) es N ! r i 2 •C the !H + QPs(ξfor + ξ n n) N n=1 real and the complex part of the + "Φ∗C (z n ), v#H + c) − 1 + ξnr ) label + "Φ∗C (z n ), w#H + c) − 1 + ξni & N % an − n=1 subject to an drn = 0 n=1 C 0 ≤ an ≤ N for n = 1, . . . , N and ˆ a an am drn drm κrC (z m , z n ) n,m=1 N % subject to maximize N % N % bn − n=1 N % bn bm din dim κrC (z m , z n ) n,m=1 N % bn din = 0 n=1 C 0 ≤ bn ≤ N for n = 1, . . . , N. Observe that, similar to the regression case, these 7 lems are equivalent with two distinct real SVM (dual) donderdag 1 mei 2014 Conclusion r ˜ = x + ix • Treating the real and imaginary part of x as independent features is OK! i • In the fully complex treatment, we get a four-class classifier explo • ... and we can define new kernels 3 ! n, 2 1 0 1.0 2 0.5 1 0.0 0 !0.5 !1 Henc !1.0 !2 Fig. 1. The element κrC (·, (0, 0)T ) of the induced real feature space of the complex Gaussian kernel. 8 donderdag 1 mei 2014 As a on C