Optimization Problem Based on L2,1-norms
Xiaohong Chen
19-10-2012
1
Outline
Efficient and robust feature selection via
joint l2,1-norm minimzation
Robust and discriminative distance for
multi-instance learning
 Its application…
2
Outline
Efficient and robust feature selection via
joint l2,1-norm minimization
Robust and discriminative distance for
multi-instance learning
 Its application…
3
Efficient and robust feature selection
via joint l2,1-norm minimzation
4
Robust Feature Selection Based on l21-norm
Given training data {x1, x2,…, xn} and the associated class
labels {y1,y2,…, yn}
Least square regression solves the following optimizaiton
problem to obtain the projection matrix W
Add a regularization R(W) to the robust version of LS,
5
Robust Feature Selection Based on l21-norm
Possible regularizations
Ridge regularization
Lasso regularization
Lasso regularization
Penalize all c regression coefficients
corresponding to a single feature as a whole
6
Robust Feature Selection Based on l21-norm
7
Robust Feature Selection Based on l21-norm
Denote
(14)
8
Robust Feature Selection Based on l21-norm
Then we have
(19)
9
The iterative algorithm to solve problem (14)
Theorem1: The algorithm will monotonically decrease the objective
of the problem in Eq.(14) in each iteration, and converge to the global
optimum of the problem.
10
Proof of theorem1
2ab  a  b
2
2
a 2 b2
a

2b 2b
u
a2
b2
a
b
2b
2b
u
11
Proof of theorem1
12
(1)
(2)
(1)+(2)
13
Outline
Efficient and robust feature selection via
joint l2,1-norm minimization
Robust and discriminative distance for
multi-instance learning
 Its application…
14
Robust and discriminative distance for
multi-instance learning
15
Multi-instance learning
多示例学习中，训练集由若干个具有概念标记的包(bag)组成，
每个包包含若干个没有概念标记的示例。若一个包中至少有
一个正例，则该包被标记为正(positive)，若一个包中所以示
例都是反例，则该包被标记为反(negative),通过对训练包的学
习，希望学习系统尽可能正确地对训练集之外的包的概念标
记进行预测。
16
The illustration of MIL
17
Notations
Given N training bags
classes.
and K conceptual
Each bag contains a number of instances
Given the class memberships of the input data, denoted as
18
Notations
First, we represent every class as a super-bag that comprises the
instances of all its training
, where
19
Objective to learn class specific distance metrics
For a given class, Ck,, we solve the following optimization problem:
20
Algorithm and its analysis
21
Algorithm and its analysis
22
Algorithm and its analysis
23
Algorithm and its analysis
On the other hand,
24
Algorithm and its analysis
25
Algorithm and its analysis
26
Algorithm and its analysis
Therefore, the objective value of the problem of (6) is decreased in
each iteration till convergences.
27
Outline
Efficient and robust feature selection via
joint l2,1-norm minimzation
Robust and discriminative distance for
multi-instance learning
 Its application…
28
Its application
For example:

min
W T ( xi  x j )( xi  x j )TW
xi , x j同类

min
W T ( xi  x j )( xi  x j )TW
W
xi , x j 不同类
min
W
AW
BW
2,1
AW
BW
2,1
2,1
2,1
  CW
2,1
29
Reference
[1]F.Nie, D.Xu, X.Cai, and C.Ding. Efficient and robust feature selection via
joint l2,1-norm minimzation. NIPS 2010.
[2] H.Wang, F.Nie and H.Huang. Robust and discriminative distance for multiinstance learning, CVPR 2012: 2919-2924
30
Thanks! Q&A
31