ELEC/IEDA 5470, Fall 2023
TA: Yifan Yu
Homework #1
TA: Yifan Yu
Due: September 18, 2023 at 5:50 p.m.
Problem 1 (20 points)
Are the following sets convex? Give a brief justification for each of the following cases:
1. (5 points) C = {x ∈ R | x2 − 3x + 2 > 0}.
2. (5 points) C = x ∈ Rn aT1 x ≤ b1 , aT2 x ≤ b2 .
3. (5 points) C = x ∈ Rn x + S2 ⊆ S1 , where S1 , S2 ⊆ Rn with S1 convex.
4. (5 points) C = x ∈ R2+ x1 x2 > 1 .
Answer 1
1. {x ∈ R | x2 − 3x + 2 > 0} ⇐⇒ x ∈ (−∞, 1) ∪ (2, +∞). The set is nonconvex.
2. The set is an intersection of two halfspaces, so it is a convex set.
3. This set is convex. x + S2 ⊆ S1 if x + y ∈ S1 for all y ∈ S2 . Therefore,
{x | x + S2 ⊆ S1 } = ∩ {x | x + y ∈ S1 } = ∩ {x | x ∈ S1 − y} .
y∈S2
y∈S2
The intersection of convex sets S1 − y is convex.
4. x1 x2 > 1 can be rewritten as
Therefore, the set is convex.
1
x1
− x2 < 0. The function f (x1 , x2 ) =
1
x1
− x2 is convex.
Problem 2 (20 points)
1. (15 points) Determine the convexity of the following functions with x ∈ Rn and Y ≻ 0.
Justify your answers.
(a) f (Y) = xT Y−1 x.
(b) f (x) = xT Y−1 x.
(c) f (x, Y) = xT Y−1 x. Hint: the Schur complement. What can you infer about the convexity
of functions f (Y) and f (x) from that of f (x, Y)?
p√
aT x + b.
2. (5 points) Study the convexity of the function g(x) =
Answer 2
1.
(a) The function f (Y) = xT Y−1 x is convex. To see this, suppose Y = Z + tV, where Z ≻ 0,
1
−1
−1
n
T
ELEC/IEDA V
5470,
2023
∈ SFall
. We
define g(t) = xT (Z + tV) x = Tr X (Z + tV)
with X = xxTA:
. Yifan Yu
−1
g(t) = Tr X (Z + tV)
−1 −1/2
−1/2
−1/2
−1/2
I + tZ
VZ
= Tr Z
XZ
−1
= Tr Z−1/2 XZ−1/2 Q (I + tΛ) QT
−1
= Tr QT Z−1/2 XZ−1/2 Q (I + tΛ)
=
n X
−1
QT Z−1/2 XZ−1/2 Q (1 + tλi ) ,
ii
i=1
where we used the eigenvalue decomposition Z−1/2 VZ−1/2 = QΛQT .
Since QT Z−1/2 XZ−1/2 Q is symmetric and positive semidefinite, QT Z−1/2 XZ−1/2 Q ii ≥
0, i = 1, . . . , n. The function g is thus a nonnegative weighted sum of convex functions
1/(1 + tλi ), hence it is convex.
(b) The function f (x) = xT Y−1 x is convex since it is quadratic with Y−1 ≻ 0.
(c) The function f (x, Y) = xT Y−1 x is convex. The epigraph of f is
epif = {(x, Y, t) | Y ≻ 0, xT Y−1 x ≤ t}.
Using the Schur complement condition for positive semidefiniteness of a block matrix,
)
(
Y x
⪰0 .
epif = (x, Y, t) | Y ≻ 0,
xT t
Since the condition is a linear matrix inequality in (x, Y, t), epif is convex, and the
function f is also convex.
Since f (x, Y) = xT Y−1 x is jointly convex on x and Y, it is convex in each variable
separately. Thus, from the convexity of f (x, Y) we can infer the convexity of (a) and (b).
p√
p√
· is a concave function, it follows that
aT x + b is concave.
2. Since aT x + b is affine and
Problem 3 (13 points)
1. Rewrite the following optimization problem as an LP (assuming d > ||c||1 ):
minimize ||Ax − b||1 /(cT x + d)
x
subject to ||x||∞ ≤ 1.
Answer 3
1. With ||x||∞ ≤ 1, cT x + d > 0 by the assumption d > ||c||1 . Define
y = x/(cT x + d)
T
t = 1/(c x + d).
(1)
(2)
Then ||Ax−b||1 /(cT x+d) = ||Ay−bt||1 , ||x||∞ ≤ 1 ⇒ ||y/t||∞ ≤ 1 ⇒ ||y||∞ ≤ t, and cT y+dt = 1.
2
ELEC/IEDA
5470,
Fall
2023
Now we write
the
equivalent
LP:
TA: Yifan Yu
minimize sT 1
y,t,s
subject to − t1 ≤ y ≤ t1
− s ≤ Ay − bt ≤ s
(3)
cT y + dt = 1.
Problem 3 (13 points)
2. Rewrite the following constraint as an SOC constraint:
(x, y, z) ∈ Rn+2 ∥x∥2 ≤ yz, y ≥ 0, z ≥ 0 .
Hint: You may need the equality: yz = 41 (y + z)2 − (y − z)2 .
(4)
Answer 3
2. Since
yz =
1
(y + z)2 − (y − z)2 ,
4
the constraint ∥x∥2 ≤ yz, y ≥ 0, z ≥ 0 can be rewritten as
2x
≤ y + z.
y−z
Thus, (4) can be rewritten as the following SOC constraint:
)
(
2x
n+2
≤y+z .
(x, y, z) ∈ R
y−z
Problem 3 (14 points)
3. Rewrite the following problem as a second-order cone program (SOCP):
minimize xT Ax + aT x
x
subject to Bx ≤ b,
(5)
where A = AT ⪰ 0.
Answer 3
3. Introduce variable t, then the problem (5) can be reformulated as
minimize t + aT x
x,t
subject to Bx ≤ b,
xT Ax ≤ t.
1
Notice that xT Ax ≤ t is equivalent to ∥A 2 x∥2 ≤ t. According to (4), the above problem can be
3
ELEC/IEDA
5470,
Fall 2023
written in an
SOCP:
TA: Yifan Yu
minimize t + aT x
x,t
subject to Bx
≤ 1b,
2A 2 x
≤ t + 1.
t−1
Problem 4 (20 points)
Let f : Rn → R be twice continuously differentiable. Suppose f satisfies
m∥y∥2 ≤ yT ∇2 f (x)y ≤ M ∥y∥2 , ∀ x, y ∈ Rn ,
where m and M are some positive scalars.
1. Prove the following equation:
n
o
α
1
minn ∇f (x)T (w − x) + ∥w − x∥2 = − ∥∇f (x)∥2 , ∀ α > 0.
w∈R
2
2α
2. Prove that f has a unique global minimum x∗ .
3. (advanced) Prove that the global minimum x∗ satisfies
1
1
∥∇f (x)∥2 ≤ f (x) − f (x∗ ) ≤
∥∇f (x)∥2 , ∀ x ∈ Rn .
2M
2m
4. (advanced)Prove that the global minimum x∗ satisfies
m
M
∥x − x∗ ∥2 ≤ f (x) − f (x∗ ) ≤
∥x − x∗ ∥2 , ∀ x ∈ Rn .
2
2
Answer 4
1. Since the function h(w) = ∇f (x)T (w − x) + α2 ∥w − x∥2 , α > 0 is convex, we obtain
x−
1
∇f (x) = arg min h(w).
w
α
(6)
Now we take w = x − α1 ∇f (x) in (6) and get
o
n
α
1
minn ∇f (x)T (w − x) + ∥w − x∥2 = − ∥∇f (x)∥2 , ∀ α > 0.
w∈R
2
2α
(7)
2. Since m∥y∥2 ≤ yT ∇2 f (x)y, m > 0, the Hessian matrix of f (x) is positive definite. Thus f
has a unique global minimum.
It can also be proved by contradiction.
Suppose x1 and x2 are two global minima with x1 ̸= x2 . Since m∥y∥2 ≤ yT ∇2 f (x)y ≤
M ∥y∥2 for any x, y ∈ Rn , we have
mI ⪯ ∇2 f (x) ⪯ M I, ∀ x ∈ Rn .
(8)
By ∇2 f (x) ⪰ mI for any x ∈ Rn , we have
f (y) ≥ f (x) + ∇f (x)T (y − x) +
4
m
∥y − x∥2 , ∀ x, y ∈ Rn .
2
(9)
ELEC/IEDA
5470,yFall
By taking
= x2023
1 and x = x2 in Eq. (9), we obtain
TA: Yifan Yu
f (x1 ) ≥ f (x2 ) + ∇f (x2 )T (x1 − x2 ) +
m
∥x1 − x2 ∥2 .
2
(10)
Since x1 and x2 are two global minima, we know f (x1 ) = f (x2 ) and ∇f (x2 ) = 0. Thus (10)
can be written as
m
∥x1 − x2 ∥2 ≤ 0.
(11)
2
We can see (11) is contradicted with x1 ̸= x2 since m > 0. Therefore f has a unique global
minimum.
3.
1
∥∇f (x)∥2 .
(a) Now we prove f (x) − f (x∗ ) ≤ 2m
According to the equation in theProblem 4.1
n
o
α
1
minn ∇f (x)T (w − x) + ∥w − x∥2 = − ∥∇f (x)∥2 , ∀ α > 0,
w∈R
2
2α
(12)
we take α = m and get
∇f (x)T (y − x) +
1
m
∥y − x∥2 ≥ −
∥∇f (x)∥2 .
2
2m
(13)
Together with Eq. (9) and Eq. (13), we obtain
f (y) − f (x) ≥ ∇f (x)T (y − x) +
m
1
∥y − x∥2 ≥ −
∥∇f (x)∥2 , ∀ x, y ∈ Rn .
2
2m
(14)
By taking y = x∗ , we have
f (x) − f (x∗ ) ≤
1
∥∇f (x)∥2 .
2m
(15)
1
(b) We prove f (x) − f (x∗ ) ≥ 2M
∥∇f (x)∥2 .
2
By ∇ f (x) ⪯ M I for any x ∈ Rn , we have
f (y) ≤ f (x) + ∇f (x)T (y − x) +
Let g(y) = f (x) + ∇f (x)T (y − x) +
x−
Now we take y = x −
f (x −
1
M ∇f (x)
M
2 ∥y
M
∥y − x∥2 , ∀ x, y ∈ Rn .
2
(16)
− x∥2 . By the convexity of g(y), we obtain
1
∇f (x) = arg min g(y).
y
M
(17)
in (16) and get
1
1
1
∇f (x)) ≤ f (x) −
∥∇f (x)∥2 +
∥∇f (x)∥2 , ∀ x ∈ Rn .
M
M
2M
(18)
Then we have
f (x) − f (x −
1
1
∇f (x)) ≥
∥∇f (x)∥2 , ∀ x ∈ Rn .
M
2M
Together with the fact that f (x) − f (x∗ ) ≥ f (x) − f (x −
f (x) − f (x∗ ) ≥
5
1
M ∇f (x)),
1
∥∇f (x)∥2 , ∀ x ∈ Rn .
2M
(19)
we obtain
(20)
∗ 2
ELEC/IEDA
5470,
Fall f2023
4. (a) We
prove
(x) − f (x∗ ) ≤ M
2 ∥x − x ∥ .
2
By ∇ f (x) ⪯ M I, we have
TA: Yifan Yu
f (x) ≤ f (x∗ ) + ∇f (x∗ )T (x − x∗ ) +
M
∥x − x∗ ∥2 , ∀ x ∈ Rn .
2
(21)
Because of ∇f (x∗ ) = 0, we obtain
f (x) ≤ f (x∗ ) +
M
∥x − x∗ ∥2 , ∀ x ∈ Rn .
2
(22)
∗ 2
(b) We prove f (x) − f (x∗ ) ≥ m
2 ∥x − x ∥ .
2
By ∇ f (x) ⪰ mI, we have
f (x) ≥ f (x∗ ) + ∇f (x∗ )T (x − x∗ ) +
m
∥x − x∗ ∥2 , ∀ x ∈ Rn .
2
(23)
By ∇f (x∗ ) = 0, we obtain
f (x) ≥ f (x∗ ) +
6
m
∥x − x∗ ∥2 , ∀ x ∈ Rn .
2
(24)