Sparse Inverse Covariance
Estimation with Graphical LASSO
J. Friedman, T. Hastie, R. Tibshirani
Biostatistics, 2008
Presented by Minhua Chen
1
Outline
•
•
•
•
•
Motivation
Mathematical Model
Mathematical Tools
Graphical LASSO
Related papers
2
Motivation
(M. Choi, V. Chandrasekaran and A.S. Willsky, 2009)
(O. Banerjee, L. Ghaoui,
3
and A. d’Aspremont, 2008)
Mathematical Model
• The optimization problem is concave (M. Yuan and Y. Lin, 2007).
• Various optimization algorithms have been proposed
(M. Yuan and Y. Lin, 2007; O. Banerjee, L. Ghaoui, and A. d’Aspremont, 2008;
N. Meinshausen and P. Buhlmann, 2006).
• The Graphical LASSO algorithm, built on a previous paper
(O. Banerjee, L. Ghaoui, and A. d’Aspremont, 2008) , is widely used due to its
computational efficiency.
• It transforms the above optimization to LASSO regressions.
4
Mathematical Tools (1)
• Subgradient (J. Tropp, 2006)
Example 1:
Example 2:
5
Mathematical Tools (2)
• Matrix inversion identity:
• The above equations reveal the relationship between the inverse
covariance matrix and the covariance matrix.
6
Graphical LASSO (1)
7
Graphical LASSO (2)
8
Graphical LASSO (3)
9
Graphical LASSO (4)
Ground Truth
Inferred
10
Related papers:
• N. Stadler and P. Buhlmann, Missing Values: Sparse Inverse
Covariance Estimation and an Extension to Sparse Regression
Proposed a MissGLasso algorithm to impute the missing data
and infer the inverse covariance matrix simultaneously.
• O. Banerjee, L. El Ghaoui and A. d’Aspremont, Model Selection
Through Sparse Maximum Likelihood Estimation for
Multivariate Gaussian or Binary Data
Used a constrained quadratic programming algorithm
(COVSEL) to solve the same optimization problem as Graphical
LASSO.
• N. Meinshausen and P. Buhlmann, High-Dimensional Graphs
and Variable Selection with the Lasso
Proposed a neighborhood selection method to approximate
the Gaussian Graph.
11