MTH 377/577 CONVEX OPTIMIZATION
Winter Semester 2022
Indraprastha Institute of Information Technology Delhi
ASSESSMENT 3 (Time: 1 hour, Total Points: 20)
Please Note:
1. Submission received after the deadline will incur a penalty.
2. First attempt submitting a good amount of time before the deadline via Google Classroom.
If you encounter technical problems in submitting via Google Classroom, send it to me
directly via email.
3. While you can consult all the material at hand, discussing the problems with any person
is a violation of academic integrity.
Q1. (10 points) Using the KKT conditions, write down a detailed solution to solve the
following optimization problem framed in R2
√
√
max 2 x + 2 y
subject to 2x + y ≤ 3
x + 2y ≤ 3
A1. Reformulate the problem as a minimization problem.
√
√
min f0 (x, y) := −2 x − 2 y
subject to f1 (x, y) := 2x + y − 3 ≤ 0
f2 (x, y) := x + 2y − 3 ≤ 0
Df0 =
h
− √1x
− √1y
i
2
D f0 =
(λ1 )
(λ2 )
1
2x3/2
0
0
1
2y 3/2
D2 f0 is a positive semidefinite matrix because both its eigenvalues (the diagonal entries) are
nonnegative and so f0 is a convex functions on R2+ . The feasible set, as a solution set of linear
inequalities, is a polyhedron and so a convex set. This means this is a convex optimization
problem.
1
Slater’s condition is satisfied because (1/2, 1/2) is a strictly feasible point. Therefore, Strong
Duality holds. This further implies that KKT conditions are necessary and sufficient for a
primal-dual pair (x, y, λ1 , λ2 ) to be optimal. These conditions look like the following
2x + y − 3 ≤ 0
x + 2y − 3 ≤ 0
λ1 ≥ 0
λ2 ≥ 0
λ1 (x + 2y − 3) = 0
λ2 (2x + y − 3) = 0
1
− √ + 2λ1 + λ2 = 0
x
1
− √ + λ1 + 2λ2 = 0
y
(P F 1)
(P F 2)
(DF 1)
(DF 2)
(CS1)
(CS2)
(LO1)
(LO2)
The analysis of the KKT system reduces to analyzing four combinatorial possibilities :
Case 1. (PF1) and (PF2) both bind at the optimum. In this case, the primal optimum is
the unique solution to the equations
2x + y − 3 = 0,
x + 2y − 3 = 0
which is (x, y) = (1, 1). Substituting the primal optimal values in (LO1) and (LO2) gives
the dual optimum as the solution to the equations
2λ1 + λ2 = 1,
λ1 + 2λ2 = 1
which is (λ1 , λ2 ) = (1/3, 1/3).
Case 2. (PF1) binds at the optimum but (PF2) is slack. (CS2) then implies λ2 = 0.
(LO1) and (LO2) then imply λ1 = √1y = 2√1 x , which implies the equation y = 4x for primal
variables. This, together with a binding (PF1) then gives (x, y) = (1/2, 2). However, this
point is not even primal feasible, as it violates (PF2).
Case 3. (PF2) binds at the optimum but (PF1) is slack. One can rule out any solution in
this case on the lines of Case 2.
Case 4. Both (PF1) and (PF2) are slack at the optimum. (CS1) and (CS2) then implies
λ1 = λ2 = 0 which then gives x = y = 0 from (LO1) and (LO2). However, this cannot be
an optimum because the point (1/2, 1/2) achieves a lower f0 value.
Therefore, (x, y) = (1, 1) solves the convex optimization problem.
Grading Comment: 2 points for showing that the problem is convex, 2 points for pointing
out why Strong Duality holds, 2 points for writing down KKT conditions and 4 points for
correctly solving them, including analyzing these cases.
2
Q2. (5 points) Find the Lagrangian dual function if we regard the following problem as our
primal problem:
min x21 + x22
subject to 2x1 + x2 ≥ 1
x1 ≥ 0, x2 ≥ 0
0 is an obvious lower bound on the value of this problem. Using the Lagrangian dual function,
find a lower bound that is better than 0.
A2. Rewrite the primal problem as
min x21 + x22
subject to 1 − 2x1 − x2 ≤ 0
−x1 ≤ 0
−x2 ≤ 0
(λ1 )
(λ2 )
(λ3 )
Let λ = (λ1 , λ2 , λ3 ) and x = (x1 , x2 ). The Lagrangian and its gradient (read as column
vector) at x are
L(x, λ) = x21 + x22 + λ1 (1 − 2x1 − x2 ) − λ2 x1 − λ3 x2
∇x L(x, λ) = (2x1 − 2λ1 − λ2 , 2x2 − λ1 − λ3 )
Since the Lagrangian dual function g(λ) = minx L(x, λ), we can find the point x that minimizes the Lagrangian. Since this is an unconstrained problem, this point would be found
3
by setting ∇x L(x, λ) = 0 and is given by x1 = λ1 + λ22 , x2 = λ1 +λ
. The Lagrangian dual
2
function is found by simply substituting this value in the Lagrangian and is given by
g(λ) = λ1 +
λ3 λ1 + λ3 2
5λ1
λ2 λ1 + λ3 λ2 2
− λ2 −
+
+ λ1 1 −
− λ2 λ1 +
− λ3
2
2
2
2
2
2
We know that for every λ ≥ 0, g(λ) is a lower bound on the value of this problem. One
really has to check with some intuition to find a positive lower bound. For a value (λ1 =
1/2, λ2 = 0, λ3 = 0), we find that g(λ) = 3/16 > 0.
Grading Comment: 1 point for formulating the Lagrangian correctly, 1.5 points for defining the dual function and solving the problem that defines the dual, 1 point for computing
g(λ) and 1.5 points for lower bound as well as displaying the dual variables that achieve that
lower bound.
Q3. (5 points) Consider a convex optimization (minimization) problem in standard form.
Given a feasible point x for the primal problem and a feasible point (λ, µ) for the corresponding dual problem, define the duality gap at (x, λ, µ). Is the duality gap informative about
3
the optimality of x for the primal problem ? If yes, establish mathematically the sense in
which it is informative. If not, argue why not.
A3. The duality gap at (x, λ, µ) is defined to be f0 (x) − g(λ, µ) where f0 is the primal
objective and g is the dual objective. It gives the degree of primal sub-optimality of x. By
weak duality, we know g(λ, µ) ≤ p? where p? is the primal optimal value. Multiplying the
inequality by -1 and adding f0 (x) to both sides gives
0≤
f0 (x) − p?
| {z }
degree of primal suboptimality
≤ f0 (x) − g(λ, µ)
|
{z
}
duality gap
A zero duality gap implies primal optimality (and in fact dual optimality as well).
Grading Comment: 2 points for the correct definition. 2 points for informativeness regarding primal sub-optimality, 1 point for informativeness about primal optimality.
4