Jon Dattorro
convexoptimization.com
prototypical cardinality problem
find
x
subject to Ax  b
x0
card x  k
Perspectives:
 Combinatorial
 Geometric
2
Euclidean bodies
Permutation Polyhedron
P  {X  Rn  n
X T 1  1, X 1  1, X  0 }
• n! permutation matrices are vertices in (n-1)2 dimensions.
• permutaton matrices are minimum cardinality doubly stochastic matrices.
H
Hyperplane
H  {x  R n
x T 1  1}
3
Geometrical perspective
Compressed Sensing
minimize
|| x ||1
subject to Ax  b
1-norm ball: 2n vertices, 2n facets
Candes/Donoho (2004)
4
Candes demo

%Emmanuel Candes, California Institute of Technology, June 6 2007, IMA Summerschool.



clear all, close all
n = 512;
m = 64;





k = 0:n-1; t = 0:n-1;
F = exp(-i*2*pi*k'*t/n)/sqrt(n);
freq = randsample(n,m);
A = [real(F(freq,:));
imag(F(freq,:))];




S = 28;
support = randsample(n,S);
x0 = zeros(n,1); x0(support) = randn(S,1);
b = A*x0;







% Solve l1 using CVX
cvx_quiet(true);
cvx_begin
variable x(n);
minimize(norm(x,1));
A*x == b;
cvx_end


norm(x - x0)/norm(x0)
figure, plot(1:n,x0,'b*',1:n,x,'ro'), legend('original','decoded')
% Size of signal
% Number of samples (undersample by a factor 8)
% Fourier matrix
% Incomplete Fourier matrix
wikimization.org
5
Candes demo
find
x
subject to Fx  Fx0
A F  R m  n
b
Fx0  R
m  64  2
n  512
k  28
 binary mask
F Fourier matrix
x is sparse
6
m
k-sparse sampling theorem
• Donoho/Tanner (2005)
A R
m n
7
two geometrical interpretations
find
x
subject to Ax  b
x0
K  {Ax x  0}
8
motivation to study cones
 convex cones generalize orthogonal subspaces
K  K  R
*
n
 Projection on K determinable from projection on -K*
and vice versa. (Moreau)
 Dual cone:
K  { y x y  0 x  K}
*
T
9
application - LP presolver
 Delete rows and columns of matrix A
 columns: smallest face F of cone K containing b

F( K  b)  {a  K a  K  b }
*
 A holds generators for K
find
subject to
z
bT z  0
AT z  0
A(: , i ) T z  1
 If feasible, throw A(: , i) away
K  {Ax x  0}
10
application - Cartography
11
list reconstruction from distance D
a.k.a
 metric multidimensional scaling
 principal component analysis
 Karhunen-Loeve transform
 cartography: projection on semidefinite cone
12
minimize
DS h
|| V ( D  H ) Vn ||F
T
n
subject to  V D Vn  0
T
n
projection on semidefinite cone because
S  V S h Vn
T
n
subspace of symmetric matrices
is isomorphic with
subspace of symmetric hollow matrices
13
minimize
DS h
|| V ( D  H ) Vn ||F
T
n
subject to  V D Vn  0
T
n
(EY)
rank V D Vn  
T
n
 is convex problem (Eckart & Young) (§7.1.4 CO&EDG)
 optimal list X from
 VnT D Vn
(§5.12 CO&EDG)
14
ordinal reconstruction
minimize
DS h
|| VnT ( D  O ) Vn ||F
subject to  VnT D Vn  0
rank V D Vn  
T
n
 vect D  K M 
• nonconvex
• strategy: break into two problems: (EY) and convex problem
minimize

||    vect D ||
subject to   K M 
• fast projection on monotone nonnegative cone KM+ (Nemeth, 2009)
•
 R
50202
15
Cardinality heuristics

y 
minimize
||  ||0
4M4M
R
subject to E vec   t
0

minimize
vec  , y
*
subject to E vec   t
0
16
Rank heuristics
 trace is convex envelope of rank on PSD matrices
 rank function is quasiconcave
17
 Idea behind convex iteration
tr G  G , I
(vector inner product)
18
Convex Iteration
19
application (Recht, Fazel, Parrilo, 2007)
(Rice University 2005)
20
one-pixel camera - MIT
21
one-pixel camera - MIT
22
application - MRI phantom
• Led directly to sparse sampling theorem
MATLAB>>
phantom(256)
Candes, Romberg, Tao 2004
23
application - MRI phantom
• MRI raw data called k-space
• aliasing at 4% subsampling
• Raw data in Fourier domain
vect 1 f
24
application - MRI phantom

P C
216 × 216
(projection matrix)
• hard to compute
y is direction vector from convex iteration
25
application - MRI phantom
26
application - MRI phantom
reconstruction error: -103dB
27