An Extended Level Method for Multiple Kernel Learning
Zenglin
The Chinese University of Hong Kong
1
Xu ,
Rong
2
Jin ,
Irwin
1
King ,
1 {zlxu, king, lyu}@cse.cuhk.edu.hk
Department of Computer Science and Engineering
The Chinese University of Hong Kong
Shatin, N.T., Hong Kong
1. Multiple Kernel Learning (MKL)
Given a list of base kernel functions/matrices Ki , i = 1, . . . ,m, MKL
searches for a linear combination of the base kernel functions that
maximizes a generalized performance measure.
3. Level Method for MKL
3.1 Motivation
Michael R.
1
Lyu
2 rongjin@cse.msu.edu
Department of Computer Science and Engineering
Michigan State University
East Lansing, MI, 48824
Michigan State University
4. Experiments
4.1 Settings
Kernel combination parameters
4.2 Results
Kernel weights evolution
SVM parameters
Properties
SILP / breast
3.2 Algorithms
Question?
How to efficiently solve the optimization problem?
2. Optimization Method
Small or medium scale:
Time-saving-ratio
Properties:
Evolution of objective values
 Semi-definite Programming (SDP) [Lanckriet et al., 2004]
 Second Order Cone Programming (SOCP) [Bach et al., 2004]
SD / breast
Breast data set
Large scale:
 Semi-Infinite Linear Programming (SILP) [Sonnenburg et al., 2006]
 Subgradient Descent (SD) [Rakotomamonjy et al., 2008]
Properties of Bounds
Properties of Gap
Steps of SILP and SD
4.3 Discussion
SILP
pro
con
SD
pro
3.3 Level method illustration
SILP:
 oscillation of solutions
SD:
 a large number of calls to SVM required to compute
the optimal step size via a line search
 e.g., for “iono”, 1231 times for SD, while 47 for level
method
Level method:
 the cutting plane model
 the projection to level sets ensures the stability of solutions
5 Conclusion
An efficient method for large scale multiple kernel learning
con
 utilize the gradients of all the past solutions
 introduce a projection step for regularization
Level / breast