OR-ST-MA 706
Page |1
COMMENTS CONCERNING LEMMA 1.22
Comments concerning Lemma 1.22:
 

Suppose that x 0 is a solution to Problem (P). Then f ( x0 )  S X , x0  .
Many results concerning optimality conditions for nonlinear programming problems
(e.g. Karush-Kuhn-Tucker conditions, Lagrange multipliers) are special cases of Lemma 1.22.
 

For special problems, the form of S X , x0  must the determined. This will be developed
below.
Lemma 1.25 Suppose g : E n  E m is differentiable at x 0 , suppose there exists z  E n such
that g ( x 0 ) z  0 , and let B   x : g ( x)  g ( x 0 ) . Then S ( B; x 0 )   x  E n : g ( x 0 ) x  0 , and
 
 

thus S B; x0   g ( x0 )T  ,   E m ,   0 .
Proof: Part 1 Let z  E n be such that g ( x 0 ) z  0 ; then g ( x0 )( z )  0 and
g ( x0   ( z ))  g ( x0 ),  such that 0     , for some   0 .
Define x k  x 0   ( z ) ;  x k   B and x k  x 0 as k   .
k
k
Let   k ; then  k ( x k  x0 )   z , thus  k ( x k  x0 )   z .

Then  z  S ( B, x0 ) and  y : g ( x 0 ) y  0  S(B,x 0 ) .
Now let x be such that g ( x0 ) x  0 .
Then g ( x0 )  ( z)  (1   ) x  0,   (0,1)
 ( z )  (1   ) x  S  B; x 0  ,   (0,1)
Letting   0 , we have x  S  B; x 0  . Thus far we have shown
x : g ( x ) x  0  S  B; x  .
0
0
Part 2 Let x  S  B, x 0  ; then there exist  x k   B such that x k  x 0 and  k  0
such that  k ( x k  x0 )  x .
Since g is differentiable at x 0 , g ( x k )  g ( x 0 )  g ( x 0 )( x k  x 0 )  x k  x 0  ( x k ; x 0 ) ,
where  ( x k  x0 )  0 as k   . Since x k  B, g ( x k )  g ( x0 ) , and
0  lim  k  g ( x k )  g ( x0 )   lim g ( x0 )  k ( x k  x0 )   g ( x0 ) x .
k 
k 
Hence S  B; x 0    x : g ( x 0 ) x  0 . QED
OR-ST-MA 706
Page |2
Remark: g ( x 0 ) z  0 says that the gradients g1 ( x 0 ), , g m ( x 0 ) are in an open halfspace. In particular, this condition holds if the gradient vectors are linearly independent ( i.e. if
g ( x0 ) has full rank). This follows from Lemma 1.19.
Consider Problem (P’’): Min f ( x)
x  En
s.t. gi ( x)  0, i  1,, m
 g1 
Notation: g   
 
 g m 

X  x : g ( x)  0

I  i : gi ( x 0 )  0
g I ( x 0 ) k n : rows are the gradients of the active constraints
 

Lemma 1.22 If x0 is a solution to Problem (P’’), then f ( x0 )  S X , x0  .


Now S X , x0  S
x : g ( x)  g ( x )  0 , x  .
0
I
0
I
By Lemma 1.25, if there exist y such that g I ( x 0 ) y  0 , then

 

S X , x 0  x : g I ( x 0 ) x  0
 
and
 S  X , x    g ( x ) ,
0
0 T
I
  0 .

So f ( x0 )  S X , x0   f ( x0 )  g I ( x0 )T ˆ , ˆ  0, ˆ  E k
Define   0, i  I . Then f ( x0 )  g ( x 0 )T    0 , i gi ( x 0 )  0 , and    0 (which
are just the K-K-T conditions).

i
Remark: The multiplier corresponding to f ( x 0 ) is positive since we are assuming there
exist y such that g I ( x 0 ) y  0 . If no such y exists, then we have only the following:

 

S X , x 0  x : g I ( x 0 ) x  0

  
(see Part 2 of the proof for Lemma 1.25)


  

This implies x : g I ( x0 ) x  0   S X , x0   g I ( x0 )T  ,   0  S X , x 0  .
If no such y exists, the by Lemma 1.19, there exist nonzero    0 such that
g I ( x 0 )T    0 and one obtains the Fritz-John Conditions (Theorem 1.20) : there exist
0  0,    0, (0 ,   )  0, 0f ( x 0 )  g I ( x 0 )T    0 .
EQUALITY CONSTRAINTS
Consider the equality constrained problem: Min f ( x) x  E n
s.t. h j ( x)  0, j  1, , p
OR-ST-MA 706
Page |3
f, hj’s continuously differentiable
 h1 
 
h 
X   x  E n : h( x)  0
 hp 
 
Lemma 1.26 Let h : E n  E p be continuously differentiable at x 0 , and let
C   x  E n : h( x)  h( x 0 ) . If h( x 0 ) has full rank, then S (C ; x 0 )   x : h( x 0 ) x  0 and hence
 S C; x    h( x ) ,
0
  E p .
0 T
 

Thus, Lemma 1.22 says f ( x0 )  S X ; x0  . This means that f ( x0 )  h( x0 )T   .
p
In other words, f ( x0 )    j h j ( x 0 )  0 (which is just Lagrange Multipliers).
j 1
Consider the general nonlinear programming problem:
Min f ( x)
x  En
s.t. gi ( x)  0, i  1,, m
h j ( x)  0, j  1, , p
f, gi’s, hj’s are continuously differentiable
g   g1
gm 
T

I  i : g ( x )  0
h   h1
hp 
T

X  x  E n : gi ( x)  0, i  1, , m; h j ( x)  0, j  1, , p
0
i
Lemma 1.27 Let v1 : E n  E m and v2 : E n  E p be differentiable and continuously
differentiable, respectively, at x 0 . If v2 ( x 0 ) is of full rank, if there exist y such that
v1 ( x 0 ) y  0 and v2 ( x 0 ) y  0 , and if C   x : v1 ( x)  v1 ( x 0 ), v2 ( x)  v2 ( x 0 ) , then


S (C; x 0 )  x  E n : v1 ( x 0 ) x  0, v2 ( x 0 ) x  0
And thus
 S (C; x )    v ( x )
0
0 T
1
  v2 ( x 0 )T  ,   0
 

Now Lemma 1.22 says that if x 0 solves Problem (P), then f ( x0 )  S X ; x0  . Note
that S ( X , x0 )  S
x  E
n

: g I ( x)  g ( x0 )  0, h( x)  h( x0 )  0 , x0 .
OR-ST-MA 706
Page |4
By Lemma 1.27, if there exist y  E n such that g I ( x 0 ) y  0 and h( x0 ) y  0 , and if
h( x 0 ) is full rank; then S  X , x 0    x : g I ( x 0 ) x  0, h( x 0 ) x  0 and
 S  X , x    g ( x )   h( x ) ,
0
0 T
0 T
I
 
  0 .

So f ( x0 )  S X , x0  implies that there exist ˆ  E k , ˆ  0,    E p such that
f ( x0 )  g I ( x0 )T ˆ  h( x0 )T   . Define i  0, i  I ; then you get the following conditions
(K-K-T).
f ( x0 )  g ( x0 )T    h( x 0 )T    0
i gi ( x 0 )  0, i  1, , m
  0
Remark: The assumption that there exist y  E n such that g I ( x 0 ) y  0 and
h( x0 ) y  0 , and h( x 0 ) is at full rank comprise a constraint qualification. It holds if
gi ( x 0 ), i  I , and h j ( x0 ) are linearly independent.