Lecture 14 Quadratic Optimization and its SDP Relaxation
• QCQP & motivation
• Complexity of QCQP
• SDP relaxation for QCQP
• When is SDP relaxation tight?
• Approximation upper and lower bounds
• Proof ideas
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Quadratic Optimization Problem
the generic QCQP can be written:
minimize
x†P 0x + r0
subject to x†P ix + ri ≤ 0,
i = 1, . . . , m
• x can be real or complex
• if all Pi are (Hermitian) p.s.d., convex problem,
• here, we suppose at least one Pi is not p.s.d.
• can easily include linear terms
• vast range of applications...
1
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Example: Boolean Least Squares
Boolean least-squares problem:
minimize kAx − bk2
subject to x2i = 1, i = 1, . . . , n
• basic problem in digital communications
• could check all 2n possible values of x . . .
• an NP-hard problem, and very hard in practice
• many heuristics for approximate solution
2
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Example: Partition Problem
minimize
subject to
xT W x
x2i = 1,
i = 1, . . . , n
where W ∈ S n, with W ii = 0.
• a feasible x corresponds to the partition
{1, . . . , n} = {i | xi = −1} ∪ {i | xi = 1}
• the matrix coefficient W ij can be interpreted as the cost of having the elements i and j in the
same partition.
• the objective is to find the partition with least total cost
• particular instance: MAXCUT (W ij ≥ 0)
3
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Motivation: Transmit Beamforming
• Downlink transmission: base station
has K antennas; m receivers
• n multicast groups,
{G1, . . . , Gn}, Gk ∩ Gl = ∅
• ∪k Gk = {1, . . . , m},
Pn
Gk := |Gk |,
k=1 Gk = m.
• wk : beamforming vector for Gk
• sk : signal sent to group Gk
• transmitted signal:
s(t) =
n
X
sk (t)wk
k=1
4
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Transmit Beamforming
• assume each receiver has one antenna, with
channel vector hi
• For ULA, LoS, far field,
†
−jφi
−j(K−1)φi
hi = 1, e
, ..., e
where φi = 2π λd sin θi; Vandermonde.
• signal at receiver i ∈ Gk :
†
s hi + v i =
†
sk wk hi
+
X
†
sl wl hi + v i
l6=k
• SINR for receiver i ∈ Gk
|w†k hi|2
†
2
2
|w
l6=k
l hi | + σi
P
5
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Problem Description: Transmit Beamforming
• Transmit beamforming problem: minimize transmit power, subject to QoS constraint for
each receiver in each group.
minimize
n
X
kwk k
k=1
subject to
2
|w†k hi|2
†
2
|w
l6=k
l hi |
P
+
σi2
≥ ci, ∀i ∈ Gk , ∀k ∈ {1, . . . , n}
m
minimize
subject to
n
X
kwk k
k=1
†
2
|wk hi|
2
− ci
X
†
2
2
|wl hi| ≥ ciσi , ∀i ∈ Gk , ∀k ∈ {1, . . . , n}
l6=k
⇑
separable homogeneous QCQP
6
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Complexity of QCQP
• QCQP is in general NP-hard: contains binary LP/QP as special cases
quadratic constraint x2i = 1 ⇐⇒ binary constraint xi = ±1
• “Simple” cases of QCQP: Single group multicast transmit beamforming
minimize kwk2
subject to |w†hi|2 ≥ 1, ∀i ∈ {1, . . . , n}
remains NP-hard.
7
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Complexity of QCQP over Rm
• A known NP-complete problem Π: Given integers P
a1, · · · , am, do there exist
m
m
m
binary variables {wi}i=1 ∈ {+1, −1} , such that n=1 aiwi = 0?
• A polynomial reduction to the single group multicast transmit beamforming
over the reals:
2
w12 + · · · + wm
+
minimize
m
w∈R
m
X
!2
aiwi
= w†Qw
i=1
subject to |wi| ≥ 1, ∀i ∈ {1, . . . , m}
†
where a := [a1, · · · , am] , and Q := I + aa†.
• “Yes” for Π iff the optimal objective of QCQP = m.
8
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Complexity of QCQP over Cn
• Let n = 2m + 1 and let the complex-valued decision vector be
w = [w0, w1, ..., wm, wm+1, . . . , w2m]T ∈ Cn.
• Let us denote
a :=
A :=
T
a1 a2 . . . am
−1m
Im Im
− 21 1†ma a† 0†m
,
,
Q := AT A + I m,
where 1m, 0m denote the all-one, the zero vectors in Cm.
9
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
• Claim: “Yes” for Π iff the following QCQP over Cn has a minimum value of n
minimize
n
w∈C
w†Qw
subject to |wi| ≥ 1, i = 1, ..., n,
• Since w†Qw = kAwk22 +
2m
X
|wi|2 ≥ 2m + 1 = n, for |wi| ≥ 1 ∀i, it follows
i=0
that
w†Qw = n,
|wi| ≥ 1 ∀ i ⇐⇒ Aw = 0, |wi| = 1 ∀ i.
• This gives rise to a set of linear equations:
− w0 + wi + wm+i = 0,
m
m
X
1 X −
ai w0 +
aiwi = 0.
2 i=1
i=1
i = 1, . . . , m,
(1)
(2)
10
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
• Since |wi| = 1, we can let wi/w0 = eiθi for i = 1, . . . , 2m. Using (1) we have
cos θi + cos θm+i = 1,
sin θi + sin θm+i = 0,
where i = 1, . . . , m. These two equations imply that θi ∈ {−π/3, π/3} for all i.
This in particular means that cos θi = cos θm+i = 1/2 for i = 1, . . . , m,
implying that
(
!
)
m
m
X
1 X
Re −
ai +
aiwi/w0 = 0.
2 i=1
i=1
• Therefore, (2) is satisfied if and only if
(
1
Im −
2
m
X
i=1
!
ai
+
m
X
)
aiwi/w0
i=1
= Im
(m
X
i=1
)
aiwi/w0
=
m
X
ai sin θi = 0,
i=1
which is equivalent to “Yes” for Π.
11
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Convex Relaxation of QCQP
Using a fundamental observation:
X := xxT
⇔
X 0, rank(X) = 1,
and noting xT Pix = Tr (XPi), the original QCQP:
minimize xT P 0x + r0
subject to xT P ix + ri ≤ 0,
i = 1, . . . , m
12
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
can be rewritten:
minimize Tr (XP 0) + r0
subject to Tr (XP i) + ri ≤ 0,
X0
rank(X) = 1
i = 1, . . . , m
the only nonconvex constraint is now rank(X) = 1...
13
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Convex Relaxation: Semidefinite Relaxation
• we can directly relax this last constraint, i.e. drop the nonconvex rank(X) = 1
to keep only X 0
• the resulting program gives a lower bound on the optimal value
minimize Tr (XP 0) + r0
subject to Tr (XP i) + ri ≤ 0,
X0
i = 1, . . . , m
⇒ SDP
14
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
How to Generate a Feasible Solution?
Let X ∗ be the optimal solution of
• pick x as a Gaussian variable with x ∼ N (0, X ∗)
• Since Tr (X ∗P i) + ri = E[xT P ix + ri], x will solve the QCQP “on average” over this
distribution
in other words:
minimize
E[xT P 0x + r0]
subject to E[xT P ix + ri] ≤ 0, i = 1, . . . , m
a good feasible point can then be obtained by sampling enough x. . .
Key question:
• how good is the approximate solution x?
• can we bound f (x)/f ∗ by a constant?
15
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Problem Description: Separable Homogeneous QCQP
Let C i, Aij be Hermitian matrices, and H = R or C.
minimize
wi ∈HKi
subject to
n
X
†
w i Ci w i
i=1
†
†
w1 A11 w1 + w2 A12 w2 + · · · + w†n A1n wn ≥ b1
†
†
w1 A21 w1 + w2 A22 w2 + · · · + w†n A2n wn ≥ b2
⇐ NP-hard
..
.
†
†
w1 Am1 w1 + w2 Am2 w2 + · · · + w†n Amn wn ≥ bm
minimize
n
X
Tr(C i W i )
i=1
subject to
Tr(A11 W 1 + A12 W 2 + · · · + A1n W n ) ≥ b1
..
.
⇐ SDP relaxation
Tr(Am1 W 1 + Am2 W 2 + · · · + Amn W n ) ≥ bm
W i 0.
Question: How good is the SDP relaxation?
16
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Empirical Behavior
• Measured VDSL channel data by
France Telecom R&D;
• 17 measured channel scenarios, 28
loops, 300m, 21.5-30 Mhz;
• Compared SDP solution vs
precoding (ignore crosstalk)
no
• SDP yields nearly doubling of
minimum received signal power
• SDP relaxation is tight in over 50%
of instances.
• SDP solved by the base station
Transmit precoding for VDSL multicasting.
• Question: ∃ a theoretical performance bound?
17
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Problem Description: Separable Homogeneous QCQP
Let C i, Aij be Hermitian matrices, and H = R or C.
minimize
wi ∈HKi
subject to
n
X
†
w i Ci w i
i=1
†
†
w1 A11 w1 + w2 A12 w2 + · · · + w†n A1n wn ≥ b1
†
†
w1 A21 w1 + w2 A22 w2 + · · · + w†n A2n wn ≥ b2
⇐ NP-hard
..
.
†
†
w1 Am1 w1 + w2 Am2 w2 + · · · + w†n Amn wn ≥ bm
minimize
n
X
Tr(Ci W i )
i=1
subject to
Tr(A11 W 1 + A12 W 2 + · · · + A1n W n ) ≥ b1
..
.
⇐ SDP relaxation
Tr(Am1 W 1 + Am2 W 2 + · · · + Amn W n ) ≥ bm
W i 0.
Question: How good is the SDP relaxation?
18
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Worst-case SDP Approximation Performance
In general, SDP relaxation for separable homogeneous QCQP is not tight.
Example: For any M > 0, consider
υqp := min
s.t.
x2 + y 2
y 2 ≥ 1, x2 − M xy ≥ 1, x2 + M xy ≥ 1.
• Notice that the last two constraints of QCQP imply x2 ≥ M |x||y| + 1, x2 ≥ 1, which, in
light of the constraints y 2 ≥ 1, further imply x2 ≥ M + 1. ⇒ υqp ≥ M + 2.
• However, for the SDP relaxation
υsdp := min
s.t.
X(11) + X(22)
X(22) ≥ 1, X(11) − M X(12) ≥ 1, X(22) + M X(12) ≥ 1, X 0.
• X = I is clearly a feasible solution, implying υsdp ≤ 2.
⇒ the performance ratio υqp /υsdp is at least (M + 2)/2, which can be arbitrarily large.
19
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Simple Cases
1. Ki = 1, for all i.
• Then, wi is a scalar, implying W i ≥ 0 ⇔ W i = w2i for some wi.
• The SDP relaxation is a LP, and is equivalent to the original nonconvex QCQP.
2. m = n = 1
• Then the separable homogeneous QCQP becomes
†
minimize w Cw,
†
subject to w Aw ≥ b.
• This is a generalized eigenvalue problem.
• For this case, SDP relaxation is tight. ⇒ SDP admits a rank-1 solution.
20
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Unicast
• When each group Gk has one receiver, then m = n, and the transmit beamforming becomes
minimize
subject to
n
X
kw i k
i=1
†
2
|wi hi|
2
− ci
X
†
2
2
|wl hi| ≥ ciσi , ∀i ∈ {1, . . . , n}
l6=i
• Surprise: SDP relaxation is always tight in this case! (Bengtsson-Ottersten, 2001).
• Proof relies on Lagrangian duality and Perron-Frobenius theorem.
• Is there a more general statement?
21
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Existence of Low Rank SDP Solution
• (Strengthened) Pataki’s result: The separable SDP has a solution {W ∗i }ni=1 satisfying
n
X
r(W ∗)(r(W ∗ + 1)
i
i
2
i=1
≤ m.
• If W ∗i 6= 0 for all i, then r(W ∗i ) ≥ 1, implying
n≤
n
X
r(W ∗)(r(W ∗ + 1)
i
i=1
i
2
≤ m.
• If in addition m ≤ n, then m = n and each W ∗i must be rank-1.
• This shows why SDP relaxation is tight for the unicast case.
(All feasible beamforming solutions have W i 6= 0.)
22
SDP Relaxation for Nonconvex QP
Zhi-Quan Luo
Yet Another Case: Vandermonde hi
• For uniform linear array, line of sight and far-field propagation, hi is Vandermonde
†
−jφi
−j(K−1)φi
hi = 1, e
, ..., e
:= a(φi),
Result:
If all hi = a(φi) for some φi ∈ [0, π), then the SDP relaxation of the
separable homogeneous QCQP has a rank-1 solution. (m > n allowed)
minimize
subject to
n
X
2
k wk k
k=1
X † 2
†
2
2
|wl hi | ≥ ci σi , ∀i ∈ Gk , ∀k ∈ {1, . . . , n}
| w k hi | − c i
l6=k
m
minimize
n
X
Tr(W k )
k=1
subject to
†
Tr(W k hi hi ) − ci
X
†
2
Tr(W l hi hi ) ≥ ci σi , ∀i ∈ Gk , ∀k ∈ {1, . . . , n}
l6=k
W k 0.
23