COM524500 Optimization for Comm.
SDP
COM524500
Optimization for Communications
8. Semidefinite Programming
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
1
COM524500 Optimization for Comm.
SDP
Semidefinite Programming (SDP)
Inequality form:
min cT x
s.t. F (x) 0
where F (x) = F0 + x1 F1 + . . . + xn Fn , Fi ∈ Sp .
Standard form:
min tr(CX)
s.t. X 0
tr(Ai X) = bi ,
i = 1, . . . , m
where Ai ∈ Sn , and C ∈ Sn .
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
2
COM524500 Optimization for Comm.
SDP
• The inequality & standard forms can be shown to be
equiv.
• F (x) 0 is commonly known as a linear matrix
inequality (LMI).
• An SDP with multiple LMIs
min cT x
s.t. Fi (x) 0,
i = 1, . . . , m
can be reduced to an SDP with one LMI since
Fi (x) 0, i = 1, . . . , m ⇔ blkdiag(F1 (x), . . . , Fm (x)) 0
where blkdiag is the block diagonal operator.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
3
COM524500 Optimization for Comm.
SDP
Example: Max. eigenvalue minimization
Let λmax (X) denote the maximum eigenvalue of a matrix X.
Max. eigenvalue minimization problem:
min λmax (A(x))
x
where A(x) = A0 + x1 A1 + . . . + xn An ∈ Sn .
We note that fixing x,
λmax (A(x)) ≤ t ⇐⇒ A(x) − tI 0
Hence, the problem is equiv. to
min t
x,t
s.t. A(x) − tI 0
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
4
COM524500 Optimization for Comm.
SDP
LP as SDP
Standard LP:
min cT x
s.t. x 0,
aTi x = bi ,
i = 1, . . . , m
Let C = diag(c), & Ai = diag(ai ). The standard SDP
min tr(CX)
s.t. X 0
tr(Ai X) = bi ,
i = 1, . . . , m
is equiv. to the LP since X 0 =⇒ diag(X) 0.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
5
COM524500 Optimization for Comm.
SDP
Inequality form LP
min cT x
s.t. Ax b
is equiv. to the SDP
min cT x
s.t. diag(Ax − b) 0
because X 0 =⇒ Xii ≥ 0 for all i.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
6
COM524500 Optimization for Comm.
SDP
Schur Complements
Let X ∈ Sn and partition

X=
A
B
BT C


S = C − B T A−1 B is called the Schur complement of A
in X (provided A 0).
Important facts:
• X 0 iff A 0 and S 0.
• If A 0, then X 0 iff S 0.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
7
COM524500 Optimization for Comm.
SDP
Schur complements are useful in turning some nonlinear
constraints into LMIs:
Example: The convex quadratic inequality
(Ax + b)T (Ax + b) − cT x − d ≤ 0
is equivalent to


I
Ax + b
(Ax + b)T cT x + d

0
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
8
COM524500 Optimization for Comm.
SDP
QCQP as SDP
A convex QCQP can always be written as
min kA0 x + b0 k22 − cT0 x − d0
s.t. kAi x + bi k22 − cTi x − di ≤ 0,
i = 1, . . . , L
By Schur complement, the QCQP is equiv. to
min t
"
s.t.
"
A0 x + b0
I
(A0 x + b0
)T
+ d0 + t
Ai x + bi
I
(Ai x + bi
cT0 x
)T
#
cTi x +
di
#
0
0,
i = 1, . . . , L
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
9
COM524500 Optimization for Comm.
SDP
Example: The second order cone inequality:
kAx + bk2 ≤ f T x + d
If the domain is such that f T x + d > 0, the inequality can
be re-expressed as
1
f x+d− T
(Ax + b)T (Ax + b) ≥ 0.
f x+d
T
By Schur complement, the inequality is equiv. to


(f T x + d)I Ax + b

0
(Ax + b)T f T x + d
• This result indicates that SOCP can be turned to an SDP.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
10
COM524500 Optimization for Comm.
SDP
Application in Combinatorial Optimization
Problem Statement:
Consider the Boolean quadratic program (BQP):
max xT Cx
s.t. xi ∈ {−1, +1},
i = 1, . . . , n
where C ∈ Sn .
• The BQP is (very) nonconvex since the equality
constraints x2i = 1 are nonconvex.
• In fact the BQP is NP-hard in general.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
11
COM524500 Optimization for Comm.
SDP
Practical Example I: MAXCUT [GW95]
• Input: A graph G = (V, E) with weights wij for
(i, j) ∈ E. Assume wij ≥ 0 and wij = 0 if (i, j) ∈
/ E.
• Goal: Divide nodes into two parts so as to maximize
the weight of the edges whose nodes are in different
parts.
w13
3
w14
4
1
w25
2
5
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
12
COM524500 Optimization for Comm.
SDP
Suppose V = {1, 2, . . . , n}. The MAXCUT problem takes
the form
max
n
n X
X
i=1
1 − xi xj
wij
2
j=i+1
s.t. xi ∈ {−1, +1},
i = 1, . . . , n
which can be rewritten as
max xT Cx
s.t. xi ∈ {−1, +1},
where Cij = Cji =
− 14 wij
i = 1, . . . , n
for i 6= j, & Cii =
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
1
2
Pn
j=1
wij .
13
COM524500 Optimization for Comm.
SDP
Practical Example II: ML MIMO detection [MDW+ 02]
H11
H1,n
replacements
data
symbols
Hm,1
Multiplexer
Demultiplexer
Hm,n
• The tx & rx sides have n & m antennas, respectively.
• We consider a spatial multiplexing system, where each
tx antenna transmits its own sequence of symbols.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
14
COM524500 Optimization for Comm.
SDP
Assume frequency flat fading, & antipodal modulation.
The received signal model may be expressed as
y = Hs + v
where
y ∈ Rm
multi-receiver output vector;
s ∈ {−1, +1}n transmitted symbols;
H ∈ Rm×n
MIMO (or multi-antenna) channel;
v ∈ Rn
Gaussian noise with zero mean &
covariance σ 2 I.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
15
COM524500 Optimization for Comm.
SDP
Maximum-likelihood (ML) detection of s:
min n ky − Hsk22
s∈{±1}
= min n sT H T Hs − 2sT H T y
s∈{±1}
• The ML problem is a nonhomogeneous BQP.
• But it can be reformulated as a homogeneous BQP.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
16
COM524500 Optimization for Comm.
SDP
The ML problem can be homogenized as follows:
min n sT H T Hs − 2sT H T y
s∈{±1}
= min n (cs̃)T H T H(cs̃) − 2(cs̃)T H T y
(s = cs̃)
= min n s̃T H T H s̃ − 2cs̃T H T y
(c2 = 1)
s̃∈{±1}
c∈{±1}
s̃∈{±1}
c∈{±1}
=
min
(s̃,c)∈{±1}n+1
h
s̃T
 
i H T H −H T y
s̃
 
c  T
c
−y H
0

Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
17
COM524500 Optimization for Comm.
SDP
BQP Approximation by SDP:
The BQP can be reformulated as
max tr(CX)
x,X
s.t. X = xxT
Xii = 1,
i = 1, . . . , n
Now,
• the objective function is linear for any C;
• the {±1} constraints on x becomes linear in X; but
• X = xxT is nonconvex.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
18
COM524500 Optimization for Comm.
SDP
Using the fact that
X = xxT =⇒ X 0
we approximate the BQP by solving the following SDP:
max tr(CX)
s.t. X 0
Xii = 1,
i = 1, . . . , n
Once the SDP is solved, its solution is used to approximate
the BQP solution (e.g., by applying rank-1 approximation to
the SDP solution).
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
19
COM524500 Optimization for Comm.
SDP
SDP approximation looks heuristic, but it has been shown to
provide good approx. accuracy. To see this, let
?
fBQP
= max n xT Cx,
x∈{±1}
?
fSDP
=
max
X0,Xii =1 ∀i
trCX
• Goemans & Williamson [GW95] showed that for
MAXCUT where Cij ≤ 0 for i 6= j & C 0,
?
?
?
0.87856fSDP
≤ fBQP
≤ fSDP
• Nesterov [N98] showed that for C 0,
2 ?
f
π SDP
?
?
≤ fBQP
≤ fSDP
• Zhang [Z00] showed that if Cij ≥ 0 for i 6= j,
?
?
fBQP
= fSDP
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
20
COM524500 Optimization for Comm.
SDP
Tx Downlink Beamforming for Broadcasting
• The problem is the same as that in the last lecture,
except that the transmitter sends common information
to all receivers.
• The received signal remains the same:
yi = hTi x + vi ,
i = 1, . . . , n
but the transmitted signal is now given by
x = fs
where f ∈ Cm is the tx beamformer vector & s ∈ C is
the information bearing signal.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
21
COM524500 Optimization for Comm.
SDP
• Problem: [SDL06] Minimize the tx power, subject to
constraints that the received SNR is no less than some
pre-specified threshold:
khTi f k22
≥ γo ,
2
σi
i = 1, . . . , n
where γo is the SNR threshold, & σi2 is the noise
variance.
• Let Qi =
1
∗ T
h
h .
2
γo σi i i
The problem can be written as
min kf k22
s.t. tr(f f H Qi ) ≥ 1,
i = 1, . . . , n
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
22
COM524500 Optimization for Comm.
SDP
• Unfortunately the tx beamformer design problem here is
nonconvex.
• Even worse, the problem is shown to be NP-hard
[SDL06].
• However, we can approximate the problem using SDP.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
23
COM524500 Optimization for Comm.
SDP
• An equivalent form of the problem:
min
m
f ∈C
,F ∈Hm
tr(F )
s.t. F = f f H
tr(F Qi ) ≥ 1,
i = 1, . . . , n
• SDP approximation:
min tr(F )
s.t. F 0
tr(F Qi ) ≥ 1,
i = 1, . . . , n
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
24
COM524500 Optimization for Comm.
SDP
• Another Problem: Maximize the weakest received
SNR, subject to a constraint that the tx power is no
greater than a threshold Po :
1
max min 2 |hi f |2
i=1,...,n σ
i
s.t. kf k22 ≤ Po
• This problem is also NP-hard.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
25
COM524500 Optimization for Comm.
SDP
• The problem can be re-expressed as
max t
1 T 2
s.t. 2 |hi f | ≥ t,
σi
kf k22 ≤ Po
i = 1, . . . , n
• The associated SDP approximation:
max t
1
s.t. 2 tr(h∗i hTi F ) ≥ t,
σi
F 0
i = 1, . . . , n
tr(F ) ≤ Po
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
26
COM524500 Optimization for Comm.
SDP
References
[MDW+ 02]
W.-K. Ma, T.N. Davidson, K.M. Wong, P.C. Ching, & Z.-Q.
Luo, “Quasi-ML multiuser detection using SDR with
application to sync. CDMA,” IEEE Trans. Signal Proc.,
2002.
[GW95]
M.X. Goemans & D.P. Williamson, “Improved approx. alg.
for max. cut & satisfiability problem using SDP,” J. ACM,
1995.
[N98]
Y.E. Nesterov, “SDR and nonconvex quadratic opt.,” Opt.
Meth. Software, 1998.
[Z00]
S. Zhang, “Quadratic max. and SDR,” Math. Program.,
2000.
[SDL06]
N.D. Sidiropoulos, T.N. Davidson, & Z.-Q. Luo, “Transmit
beamforming for physical layer multicasting,” IEEE Trans.
Signal Proc., 2006.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
27
COM524500 Optimization for Comm.
SDP
Additional Reading on SDP Applications
[DLW00]
T.N. Davidson, Z.-Q. Luo, & K.M. Wong, “Design of
orthogonal pulse shapes for communications via SDP,” IEEE
Trans. Signal Proc., 2000.
[DTS01]
B. Dumitrescu, I. Tabus, & P. Stoica, “On the parameterization
of positive real sequences & MA parameter estimation,” IEEE
Trans. Signal Proc., 2001.
Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University
28