MA 321 (Optimization)
1. (P): Minimize f (x1 , x2 ) = x21 x2 − x1 + 2x2 − x1 x2
subject to
x21 + x22 ≤ 1,
2x1 + x2 ≤ 2,
−x1 ≤ 0, −x2 ≤ 0.
(a) Find all x∗ ∈ Fea(P) on the boundary of Fea(P) for which G0 (x∗ ) ̸= Dx∗ .
Soln: (a) All x ∈ F ea(P ) such that x1 = 0 have [0, 1] or [0, −1] as a feasible direction but since g3 (x) = 0 is a binding constraint for all such points and
∇(g3 (x∗ ))[0, ∓1] = 0, so G0 (x∗ ) ̸= Dx∗ . Similarly All x ∈ F ea(P ) such that
x2 = 0 have [1, 0] or [−1, 0] is a feasible direction but since g4 (x) = 0 is a binding constraint for all such points and ∇(g4 (x∗ ))[∓1, 0] = 0, so G0 (x∗ ) ̸= Dx∗ .
Similarly All x ∈ F ea(P ) such that 2x1 + x2 = 2 have [−1, 2] or [1, −2] as a feasible direction but since g2 (x) = 0 is a binding constraint for all such points and
∇(g2 (x∗ ))[−1, 2] = 0, so G0 (x∗ ) ̸= Dx∗ . All points x ∈ F ea(P ) with x21 + x22 = 1,
there is no d feasible direction such that ∇(g2 (x∗ ))d = 0 since the tangential
lines at each of these points are not feasible directions at x∗ so for all such points
G0 (x∗ ) = Dx∗ .
So the answer is all points on the boundary except the boundary points on the
[2]
circle x21 + x22 = 1.
(b) Check whether x∗ = [0, 1]T satisfies the first order necessary conditions for a local
minimum of (P). If yes justify, if no then give a feasible direction d at x∗ , such
that f (x∗ + td) < f (x∗ ) for t > 0, sufficiently small.
Soln: (b) The first order necessary condition for a local minimum x∗ is ∇(f (x∗ ))d ≥
0 for all feasible directions d at x∗ . ∇(f (x)) = [2x1 x2 − 1 − x2 , 2 + x21 − x1 ], so
∇(f (x∗ )) = [−2, 2] at x∗ = [0, 1]T For d = [0, −1]T which is a feasible direction at
x∗ = [0, 1]T ∇(f (x∗ ))d = −2 < 0 so it does not satisfy the first order necessary
conditions.
Since by Taylor’s formula
f (x∗ + td) = f (x∗ ) + t∇(f (x∗ ))d + o(t), since ∇(f (x∗ ))d < 0 for d = [0, −1]T so
for t > 0, small f (x∗ + td) < f (x∗ ).
[2]
(c) Is x∗ = [ 53 , 45 ]T an FJ point of (P)?
Soln: If x∗ = [ 53 , 54 ]T is an FJ point then there exists nonnegative ui s not all
zeroes such that u0 ∇(f (x∗ )) + u1 ∇(g1 (x∗ )) + u2 ∇(g2 (x∗ )) = 0
where g1 , g2 is binding at x∗ = [ 53 , 45 ]T . ⇒ u0 [− 21
, 44 ] + u1 [2 53 , 2 45 + u2 [2, 1] = [0, 0].
25 25
But for nonnegative ui not all of which are not zeroes, the second component of
the LHS vector is strictly positive,
so x∗ = [ 53 , 45 ]T is not an FJ point.
[3]
(d) Are all the FJ points of (P) also KKT points?
Soln: Since at every x∗ ∈ Fea(P), G0 (x∗ ) ̸= ϕ so by Gordon’s theorem
P
∗
i∈Ix∗ ui ∇(gi (x )) = [0, 0] does not have a solution in nonnegative ui such that
not all are zeroes. (**)
So if x∗ is an FJ point then since
P
u0 ∇(f (x∗ )) + i∈Ix∗ ui ∇(g1 (x∗ )) = 0
has a nonnegative non trivial solution, and in all such solutions from (**) u0 ̸= 0
which implies x∗ is a KKT point.
[2]
(e) Does x∗ = [1, 0]T satisfy the second order necessary conditions for a local minimum?
Soln: The second order necessary conditions for a local minimum x∗ is that for
all feasible directions d at x∗
(i)∇(f (x∗ ))d ≥ 0
(ii) If ∇(f (x∗ ))d = 0 then d2 ∇2 (f (x∗ ))d ≥ 0.
∇(f (x∗ )) = [−1, 2] and for all d ∈ Dx∗ , d1 ≤ 0, d1 ̸= 0 and d2 ≥ 0.
So it satisfies ∇(f (x∗ ))d ≥ 0, also since ∇(f (x∗ ))d = −d1 + 2d2 > 0 for all
d ∈ Dx∗ so the first condition is satisfied and second condition is also vacuously
satisfied.
[3]
2. (P): Minimize ex21 + 2x22 − x1 x2 − x1 + 3x2
subject to ax1 +bx2 ≤ 2,
cx1 +dx2 ≥ 4,
−x1 ≤ 0, −x2 ≤ 0, (a, b, c, d, e ̸= 0).
(a) If possible give values of a, b, c, d, e, such that every KKT point (there exists
atleast one) of (P) is a global minimum.
1
1
−1
2
which is positive
Soln: If we take e = then the hessian matrix is
−1 4
4
semidefinite since the determinant and the diagonal entries are all nonnegative.
So f is convex and then every KKT point is a global minimum.
Note that since ∇(f (x)) = [2ex1 − x2 − 1, 4x2 − x1 + 3] if we take a = b = 1
and c = d = 2 (e = 14 ) then the feasible region is a line segment x1 + bx2 = 2,
−x1 ≤ 0, −x2 ≤ 0. The point [2, 0]T which is a feasible point and which satisfies
−x2 = 0 is a KKT point since
∇(f (x∗ )) + 1[0, −1] = 0.
[3]
(b) If possible give values of a, b, c, d, e and an x∗ ∈ Fea(P) such that there exists an
FJ point which is not a local minimum point.
Soln: For the same choice of a, b, c, d, e, note that every feasible point is an FJ
point however [0, 2]T is a feasible point but not a local minimum since d = [1, −1]
is a feasible direction at [0, 2]T and ∇(f (x∗ ))d < 0.
[3]
3. Using the Big(M) method find an initial BFS of (P) given below:
Minimize −x1 + 4x2 + 2x3
subject to
2x1 + x2 − 3x3 ≤ 2,
x1 − x2 + 4x3 ≥ 4,
x1 ≥ 0, x2 ≥ 0, x3 ≥ 0.
Soln: By adding artificial variable w2 variable we get LP(M) as
Minimize −x1 + 4x2 + 2x3 + M w2
subject to
2x1 + x2 − 3x3 + s1 = 2,
[3]
x1 − x2 + 4x3 − s2 + w2 = 4,
x1 ≥ 0, x2 ≥ 0, x3 , s1 , s2 , w2 ≥ 0.
The table corresponding to the BFS with basic variables s1 , w2 is given by
cj − zj
s1
w2
B −1 a˜1 B −1 a˜2 B −1 a˜3 B −1 s1 B −1 s2 B −1 w2 B −1 b
−1 − M 4 + M 2 − 4M
0
M
0
2
1
−3
1
0
0
2
1
−1
4
0
−1
1
4
Then the entering variable is x3 and the leaving variable is w2 so in the next iteration
we get the BFS [0, 0, 1, 5, 0]T of (P).
[3]
4.
4 7 1 5 20
9 8 13 7 30
4 5 9 3 40
30 35 10 15
(a) Check whether the initial BFS x0 with basic cells
B = {(1, 1), (2, 2), (3, 1), (3, 2), (3, 3), (3, 4)}, is optimal for (P) (by taking u3 = 0,
where u3 is the dual variable corresponding to supply s3 ). If it is not optimal,
then by using a θ-loop, get a BFS with a lower value of the objective function.
Soln: The initial BFS is given by
20
20
30
30
10 5 10 15 40
30 35 10 15
we take u3 = 0 so u1 = 0 and u2 = 3.
v1 = 4, v2 = 5, v3 = 9, v4 = 3.
The cij − ui − vj values are indicated in the cells in the following table:
2 −8 2
2
1 1
Clearly x13 s the entering variable and the θ-loop in B∪{(1, 3)} is {(1, 1), (1, 3), (3, 1), (3, 3)}.
The θ value of the entering variable is 10 so the new BFS is
10
10
20
30
30
20 5
15 40
30 35 10 15
The cost associated with the new BFS is 440 which is strictly less than 520 the
cost associated with the initial BFS.
[4]
(b) If cost cij is changed to c′ij for some xij such that there exists an optimal solution
of new (P) with x11 = 17, then does this new(P) have a unique optimal solution?
P
P
Soln: If x is a BFS then for any basic variable xij for x, xij = i αi si + j βj dj ,
where αi , βj = 0, 1, −1 for all i, j. Since in this (P), all the si , dj is a multiple of
5 so value of every basic variable in a BFS will be a multiple of 5. Since x11 = 17
in an optimal solution so this optimal solution is not a BFS, since there must
exist an optimal BFS so the total number of optimal solutions is infinite.
[2]
5. Let B be a basic set of cells in a 5 × 5 transportation array and let θ1 , θ2 be the θ-loops
in B ∪ {(p, q)}, B ∪ {(r, s)}, respectively, such that θ1 ∩ θ2 = ϕ. Let (u, v) ∈ θ1 .
(a) Is B ∪ {(p, q)} \ {(u, v)} LI?
Soln: Note that there exists exactly one θ-loop in B ∪ {(p, q)} which is given as
θ1 . Since (u, v) ∈ θ1 , and no subset of a θ-loop is a θ-loop so B ∪ {(p, q)} \ {(u, v)}
does not contain any θ-loop. Since any collection of cells which does not contain
a θ-loop is LI so B ∪ {(p, q)} \ {(u, v)} is LI.
[2]
(b) What are the possible number of distinct θ-loops in B ∪ {(p, q), (r, s)}?
Soln: We know there are atleast two distinct θ-loops in B ∪ {(p, q), (r, s)}, given
by θ1 , θ2 .
Suppose there exists some other θ-loop in B ∪ {(p, q), (r, s)} then that θ-loop must
include both the cells (p, q), (r, s).
Then α1 (p, q) + α2 (r, s) = 1(i1 , j1 ) + ..... + (±1)(ik , jk ) (1),
where (i1 , j1 ), . . . , (ik , jk ) are the other cells of the θ-loop and as an abuse of notation (p, q) represents both the cell and the column in the coefficient matrixA
corresponding to the variable xpq . WLOG let us assume that α1 = α2 = 1.
Also from the θ1 , θ2 loops we get
(p, q) = 1(i′1 , j1′ ) + ..... + 1(i′r , jr′ ) (2)
(r, s) = 1(i′′1 , j1′′ ) + ..... + 1(i′′t , jt′′ ) (3)
where (i′1 , j1′ ), ....., (i′r , jr′ ), and (i′′1 , j1′′ ), ....., (i′′t , jt′′ ) are the other cells of θ1 ,θ2 , respectively.
So if we subtract (2)+(3) from (1) then in the LHS we get the zero vector and in
the RHS we get a linear combination of the LI vectors corresponding to the basic
cells, so the linear combination in the RHS must be the trivial linear combination.
Since θ1 ∩ θ2 = ϕ, so in the RHS of (1) each of the vectors must be present which
are present in RHS of (2) and (3), so the θ-loop representing (1) must contain
all the cells of θ1 , θ2 loops which contradicts the third property in the definition
of a θ-loop.
[3]
6. Consider a linear programming problem (P) of the form,
Minimize cT x
subject to Am×n x = b, x ≥ 0.
Let LP(M) be obtained from (P) by adding atleast one artificial variable to (P) with
cost M ( M large). Let x0 be an optimal BFS for (P) and let y0 be optimal for the
dual of (P).
Check the correctness of the following statements with proper justification.
(a) If Fea(P) is unbounded then Fea(LP(M)) is unbounded.
Soln: Since Fea(P) is unbounded there exist a d a direction of Fea(P) then
check that [dT , 0]T is a direction of Fea(LP(M)).
[2]
(b) If Fea(LP(M)) is unbounded then Fea(P) is unbounded.
Soln: The above statement is False and the following is an example:
Consider (P) with feasible region given by
x1 − x2 + s1 = 1,
x1 − 2x2 − s2 = 1,
x1 ≥ 0, x2 ≥ 0,
s1 ≥ 0,
s2 ≥ 0. Since F ea(P )
consists only of a single point so it is bounded and (P) has an optimal solution.
Also Fea(LP(M)) is then given by
x1 − x2 + s1 = 1,
x1 − 2x2 − s2 + w = 1,
x1 ≥ 0, x2 ≥ 0,
s1 ≥ 0,
s2 ≥ 0, w ≥ 0.
T
Fea(LP(M)) is unbounded and has direction [1, 1, 0, 0, 1] .
(c) If a variable xn+1 is added to (P) and y0 satisfies the newly added constraint to
the dual of new (P) as an equality then y0 is optimal for the dual of new (P).
Soln: Since y0 satisfies the newly added constraint to the dual of new (P) as an
equality it implies that y0 is feasible for the new Dual of new (P). The feasible
region of the Dual of new (P) is a subset of the feasible region of the dual of
the old (P) since the addition of a variable in (P) results in the addition of a
constraint in the dual of (P). Since b has not changed and given that bT y0 ≥ bT y
for all y in the feasible region of the dual of (P) so bT y0 ≥ bT y for all y in the
feasible region of the dual of new (P).
[2]
(d) If c1 is changed to c′1 such that the optimal value of the new (P) is strictly greater
than cT x0 then x1 must be a basic variable in BFS x0 .
Soln: If c1 is changed to c′1 and let x1 be a nonbasic variable in BFS x0 . If
c′1 − z1 ≥ 0, then x0 is optimal for the new (P) (since the other cj − zj are
unchanged in the new table and may be assumed to be ≥ 0 and the B −1 b is also
a nonnegative vector) and since x1 = 0 so the new optimal value c′T x0 = cT x0 .
If c′1 −z1 < 0 then x1 enters the basis in the next iteration and we get a new optimal
solution (since it exists) say x1 , where the optimal value is c′T x1 ≤ c′T x0 = cT x0
(the old optimal value)
So given that optimal value of the new (P) is strictly greater than cT x0 , implies
that x1 must be a basic variable in BFS x0 .
[3]