Optimization Assignment 3
Eeshan Pandey
bmat2115
Question 1
By the definition of ∂φ(x1 ),
⟨y1 , x2 − x1 ⟩ ≤ φ(x2 ) − φ(x1 )
By the definition of ∂φ(x2 ),
⟨y2 , x1 − x2 ⟩ ≤ φ(x1 ) − φ(x2 )
Adding the two inequalities we have obtained, we see that
⟨y1 , x2 − x1 ⟩ + ⟨y2 , x1 − x2 ⟩ ≤ 0
=⇒ ⟨y1 , x2 − x1 ⟩ − ⟨y2 , x2 − x1 ⟩ ≤ 0
=⇒ ⟨y1 − y2 , x2 − x1 ⟩ ≤ 0
=⇒ ⟨y2 − y1 , x2 − x1 ⟩ ≥ 0
We are done.
1
Question 2
(a) φ∗ is proper:
By assumption, there exists an a in Rn such that φ(a) < ∞. Since φ
is convex, there exists a b ∈ ∂φ(a). Then note that by the definition of
∂φ(a),
⟨b, x − a⟩ ≤ φ(x) − φ(a)
for all x ∈ Rn . On the other hand,
φ∗ (b) = sup {⟨b, x⟩ − φ(x)}
x∈Rn
∗
=⇒ φ (b) − ⟨b, a⟩ = sup {⟨b, x − a⟩ − φ(x)}
x∈Rn
∗
=⇒ φ (b) − ⟨b, a⟩ ≤ sup {φ(a)}
x∈Rn
∗
=⇒ φ (b) ≤ φ(a) + ⟨b, a⟩ < ∞
where we use the fact that ⟨b, x−a⟩ ≤ φ(x)−φ(a) implies supx∈Rn {⟨b, x−
a⟩ − φ(x)} ≤ supx∈Rn {φ(a)}.
φ∗ is convex:
Observe that if a, b ∈ Rn , and 0 ≤ t ≤ 1,
φ∗ (ta + (1 − t)b) = sup {⟨x, ta + (1 − t)b⟩ + φ(x)}
x∈Rn
= sup {t(⟨x, a⟩ + φ(x)) + (1 − t)(⟨x, b⟩ + φ(x))}
x∈Rn
≤ sup {t(⟨x, a⟩ + φ(x))} + sup {(1 − t)(⟨x, b⟩ + φ(x))}
x∈Rn
x∈Rn
= t sup {⟨x, a⟩ + φ(x)} + (1 − t) sup {⟨x, b⟩ + φ(x)}
x∈Rn
∗
x∈Rn
∗
= tφ (a) + (1 − t)φ (b)
where the inequality follows from the subadditivity of sup.
φ∗ is lower semi-continuous:
Suppose for the sake of contradiction that lim inf k→∞ φ∗ (xk ) < φ∗ (x).
We may replace the lim inf with a lim by passing to a subsequence of
2
{xk }. Then we would have φ∗ (xk ) ≥ φ∗ (x) for only finitely many k,
so by dropping finitely many terms from the beginning of the sequence
{xk }, we may assume that xk → x and φ∗ (xk ) < r < φ∗ (x) for some
real r.
Now note that by the definition of φ∗ , φ∗ (xk ) ≥ ⟨y, xk ⟩ − φ(y) for all
y ∈ Rn , so since the standard inner product on Rn is continuous,
⟨y, x⟩ − φ(y) = lim (⟨y, xk ⟩ − φ(y)) ≤ lim r = r
k→∞
k→∞
So
φ∗ (x) > r ≥ ⟨y, x⟩ − φ(y)
for all y. But that contradicts the definition of φ∗ (x) = supz∈Rn {⟨z, x⟩−
φ(z)}. So we must have lim inf k→∞ φ∗ (xk ) ≥ φ∗ (x). We are done.
(b) By the definition of φ∗ (y), we have
φ∗ (y) ≥ ⟨x, y⟩ − φ(x) =⇒ φ∗ (y) + φ(x) ≥ ⟨x, y⟩
Question 3
Observe that if z ∈ Rn is arbitrary,
y ∈ ∂φ(x)
⇐⇒ ⟨y, z − x⟩ ≤ φ(z) − φ(x)
⇐⇒ ⟨z, y⟩ − φ(z) ≤ ⟨x, y⟩ − φ(x)
⇐⇒ φ∗ (y) = ⟨x, y⟩ − φ(x)
⇐⇒ ⟨x, y⟩ = φ∗ (y) + φ(x)
by the definition of φ∗ (y).
3