Fin500J: Mathematical Foundations in Finance
Topic 4: Unconstrained and Constrained
Optimization
Philip H. Dybvig
Reference: Mathematics for Economists, Carl Simon and Lawrence Blume,
Chapter 17, 18, 19
Slides designed by Yajun Wang
Fin500J Topic 4
Fall 2010 Olin Business School
1
Outline
 Unconstrained Optimization

Functions of One Variable
o General Ideas of Optimization
o First and Second Order Conditions
o Local v.s. Global Extremum

Functions of Several Variables
o First and Second Order Conditions
o Local v.s. Global Extremum
 Constrained Optimization

Kuhn-Tucker Conditions

Sensitivity Analysis

Second Order Conditions
Fin500J Topic 4
Fall 2010 Olin Business School
2
Unconstrained Optimization
 An unconstrained optimization problem is one where
you only have to be concerned with the objective
function you are trying to optimize.
 An objective function is a function that you are trying to
optimize.

None of the variables in the objective function are
constrained.
Fin500J Topic 4
Fall 2010 Olin Business School
3
General Ideas of Optimization
 There are two ways of examining optimization.
 Maximization (example: maximize profit)

In this case you are looking for the highest point on the
function.
 Minimization (example: minimize cost)

In this case you are looking for the lowest point on the
function.
 Maximization f(x) is equivalent to minimization –f(x)
Fin500J Topic 4
Fall 2010 Olin Business School
4
Graphical Representation of a Maximum
y
16
y = f(x) = -x2 + 8x
4
Fin500J Topic 4
Fall 2010 Olin Business School
8
x
5
Questions Regarding the Maximum
 What is the sign of f '(x) when x < x*?
 Note: x* denotes the point where the function is at a
maximum.
 What is the sign of f '(x) when x > x*?
 What is f '(x) when x = x*?
Definition:
 A point x* on a function is said to be a critical point if
f ' (x*) = 0.
 This is the first order condition for x* to be a
maximum/minimum.
Fin500J Topic 4
Fall 2010 Olin Business School
6
Second Order Conditions
 If x* is a critical point of function f(x), can we decide whether




it is a max, a min or neither?
Yes! Examine the second derivative of f(x) at x*, f ' '(x*);
x* is a maximum of f(x) if f ' '(x*) < 0;
x* is a minimum of f(x) if f ' '(x*) > 0;
x* can be a maximum, a minimum or neither if f ' '(x*) = 0;
Fin500J Topic 4
Fall 2010 Olin Business School
7
An Example of f''(x*)=0
 Suppose y = f(x) = x3, then f '(x) = 3x2 and f ''(x) =6x,
 This implies that x* = 0 and f ''(x*=0) = 0.
y=f(x)=x3
y
x
x*=0 is a saddle point where the point is neither a maximum nor a
minimum
Fin500J Topic 4
Fall 2010 Olin Business School
8
Example of Using First and Second
Order Conditions
 Suppose you have the following function:
 f(x) = x3 – 6x2 + 9x
 Then the first order condition to find the critical
points is:
 f’(x) = 3x2 - 12x + 9 = 0
 This implies that the critical points are at x = 1 and x = 3.
4
2
0
-2
-4
-6
Fin500J Topic 4
-8
-0.5
0
0.5
1
1.5
2
Fall 2010 Olin Business School
2.5
3
3.5
4
9
Example of Using First and Second
Order Conditions (Cont.)
 The next step is to determine whether the critical points
are maximums or minimums.
 These can be found by using the second order condition.
 f
' '(x) = 6x – 12 = 6(x-2)
 Testing x = 1 implies:
 f ' '(1) = 6(1-2) = -6 < 0.
 Hence at x =1, we have a maximum.
 Testing x = 3 implies:
 f ' '(3) = 6(3-2) = 6 > 0.
 Hence at x =3, we have a minimum.
 Are these the ultimate maximum and minimum of the
function f(x)?
Fin500J Topic 4
Fall 2010 Olin Business School
10
Local Vs. Global Maxima/Minima
 A local maximum is a point that f(x*) ≥ f(x) for all x in some
open interval containing x* and a local minimum is a point
that f(x*) ≤ f(x) for all x in some open interval containing x*;
 A global maximum is a point that f(x*) ≥ f(x) for all x in the
domain of f and a global minimum is a point that f(x*) ≤ f(x)
for all x in the domain of f.
 For the previous example, f(x)   as x   and f(x)  -
as x  -. Neither critical point is a global max or min of f(x).
Fin500J Topic 4
Fall 2010 Olin Business School
11
Local Vs. Global Maxima/Minima (Cont.)
 When f ''(x)≥0 for all x, i.e., f(x) is a convex function, then
the local minimum x* is the global minimum of f(x)
 When f ''(x)≤0 for all x, i.e., f(x) is a concave function, then
the local maximum x* is the global maximum of f(x)
Fin500J Topic 4
Fall 2010 Olin Business School
12
Conditions for a Minimum or a Maximum
Value of a Function of Several Variables
 Correspondingly, for a function f(x) of several
independent variables x
 Calculate
f x and set it to zero.
 Solve the equation set to get a solution vector x*.
 Calculate 2 f x .
 Evaluate it at x*.
 Inspect the Hessian matrix at point x*.
Hx  2 f x
Fin500J Topic 4
Fall 2010 Olin Business School
13
Hessian Matrix of f(x)
f x  is a C 2 functionof n variables,
  2 f x 


2
x1

Hx    2 f x    

  2 f x 


 xn x1
 2 f x 

x1xn 
.

 2 f x 

2
xn 
Since cross- partialsare equal for a C 2 function,H(x)
is a symmetricmatrix.
Fin500J Topic 4
Fall 2010 Olin Business School
14
Conditions for a Minimum or a Maximum Value of a
Function of Several Variables (cont.)

Let f(x) be a C2 function in Rn. Suppose that x* is a critical
point of f(x), i.e., f x *  0.
1.
If the Hessian Hx * is a positive definite matrix, then x*
is a local minimum of f(x);
If the Hessian Hx * is a negative definite matrix, then
x* is a local maximum of f(x).
If the Hessian Hx * is an indefinite matrix, then x* is
neither a local maximum nor a local minimum of f(x).
2.
3.
Fin500J Topic 4
Fall 2010 Olin Business School
15
Example
 Find the local maxs and mins of f(x,y)
f ( x, y)  x  y  9 xy
3
3
 Firstly, computing the first order partial derivatives (i.e.,
gradient of f(x,y)) and setting them to zero
 f 
 
2

3
x
 9y 

x


0
f ( x, y ) 
 
 f   3 y 2  9 x 

  
 y 
 criticalpointsx*, y * is (0,0)and ( 3, -3 ).
Fin500J Topic 4
Fall 2010 Olin Business School
16
Example (Cont.)
 We now compute the Hessian of f(x,y)
 2 f 2 f 
 2

xy   6 x 9 
 x
2
 f ( x, y )   2
   9  6 y .
2

 f  f  
 yx y 2 


 The first order leading principal minor is 6x and the second order
principal minor is -36xy-81.
 At (0,0), these two minors are 0 and -81, respectively. Since the
second order leading principal minor is negative, (0,0) is a saddle
of f(x,y), i.e., neither a max nor a min.
 At (3, -3), these two minors are 18 and 243. So, the Hessian is
positive definite and (3,-3) is a local min of f(x,y).
 Is (3, -3) a global min?
Fin500J Topic 4
Fall 2010 Olin Business School
17
Global Maxima and Minima of a Function
of Several Variables
 Let f(x) be a C2 function in Rn, then
 When f(x) is a concave function, i.e., 2 f x is negative
semidefinite for all x and f x *  0 , then x* is a global
max of f(x);
 When f(x) is a convex function, i.e., 2 f x is positive
semidefinite for all x and f x *  0 , then x* is a global
min of f(x);
Fin500J Topic 4
Fall 2010 Olin Business School
18
Example (Discriminating Monopolist)
 A monopolist producing a single output has two types of customers. If it
produces q1 units for type 1, then these customers are willing to pay a price
of 50-5q1 per unit. If it produces q2 units for type 2, then these customers
are willing to pay a price of 100-10q2 per unit.
 The monopolist’s cost of manufacturing q units of output is 90+20q.
 In order to maximize profits, how much should the monopolist produce
for each market?
 Profit is:
f (q1 , q2 )  q1 (50  5q1 )  q2 (100 10q2 )  (90  20(q1  q2 )).
T hecriticalpointsare
f
f
 50  10q1  20  0  q1  3,
 100 20q2  20  0  q2  4.
q1
q 2
2 f
2 f
2 f
2 f
 10,
 20,

 0.
2
2
q1
q2
q1q2 q2 q1
  2 f is negativedefinite (3,4)is theprofit- maximizingsupply plan.
Fin500J Topic 4
Fall 2010 Olin Business School
19
Constrained Optimization
 Examples:
 Individuals maximizing utility will be subject to a
budget constraint
 Firms maximising output will be subject to a cost
constraint
 The function we want to maximize/minimize is called
the objective function
 The restriction is called the constraint
Fin500J Topic 4
Fall 2010 Olin Business School
20
Constrained Optimization (General Form)
 A general mixed constrained multi-dimensional
maximization problem is
max f (x )  f (x 1 ,..., x n )
subject to
g1 (x 1 ,..., x n )  b1 , g 2 (x 1 ,..., x n )  b2 ,
, g k (x 1 ,..., x n )  bk ,
h1 (x 1 ,..., x n )  c 1 , h2 (x 1 ,..., x n )  c 2 ,
, hm (x 1 ,..., x n )  c m .
Fin500J Topic 4
Fall 2010 Olin Business School
21
Constrained Optimization (Lagrangian Form)
 The Lagrangian approach is to associate a Lagrange
multiplier i with the i th inequality constraint and μi with
the i th equality constraint.
 We then form the Lagrangian
L (x 1 ,..., x n , 1 ,..., k , 1 ,..., m ) 
k
f (x 1 ,..., x n )   i  g i (x 1 ,..., x n )  bi 
i 1
m
  i hi (x 1 ,..., x n )  c i .
i 1
Fin500J Topic 4
Fall 2010 Olin Business School
22
Constrained Optimization (Kuhn-Tucker
Conditions)
 If x * is a local maximum of f on the constraint set
defined by the k inequalities and m equalities, then,
there exists multipliers * , , * , * , , * satisfying
k
1
1
m
L( x* ,  * ,  * ) f ( x* ) k * gi ( x* ) m * hi ( x* )

  i
  i
 0, j  1,
x j
x j
x j
x j
i 1
i 1
hi ( x* )  ci , i  1,
,n
,m
i*  gi ( x* )  bi   0, i  1, , k
gi ( x* )  bi , i  1,
,k
i*  0, i  1, , k
Fin500J Topic
Topic44
Fall 2010
2008 Olin Business School
23
Constrained Optimization (Kuhn-Tucker
Conditions)
 The first set of KT conditions generalizes the
unconstrained critical point condition
 The second set of KT conditions says that x needs to
satisfy the equality constraints
 The third set of KT conditions is
i*  gi ( x* )  bi   0, i  1,
,k
 That is to say
if i*  0 then gi (x * )  bi
if gi (x * )  bi then i*  0
Fin500J Topic
Topic44
Fall 2010
2008 Olin Business School
24
Constrained Optimization (Kuhn-Tucker
Conditions)
 This can be interpreted as follows:
 Additional units of the resource bi only have value if the
available units are used fully in the optimal solution, i.e., if
gi (x * )  bi the constraint is not binding thus it does not
make difference in the optimal solution and i*=0.
 Finally, note that increasing bi enlarges the feasible region,
and therefore increases the objective value
 Therefore, i0 for all i
Fin500J Topic
Topic44
Fall 2010
2008 Olin Business School
25
Example
max x  y
2
subject to
x y 4
x  0, y  0.
2
2
Form the Lagrangian
L=x-y 2   (x 2  y 2  4)  1 x  2 y .
The first order conditions become:
L
 1  2  x  1  0,
x
L
(2)
 2y  2  y  2  0,
y
(1)
(3)x 2  y 2  4  0
Fin500J Topic
Topic44
(4)1 x  0,
(5)2 y  0,
(6)1  0,
(7)2  0,
(8)x  0,
(8)y  0.
Fall 2010
2008 Olin Business School
26
Example (cont.)
By (1), 1  1  2  x , since 1  0, 1  1  0
   0 and x  0, by (4), 1  0.
by (2), 2  2y (1   ), since 1    0
 either both y and 2 are zero, or both
are positive, from (5)  y  0, 2  0.
By (3) and (8)  x=2, from (4), 1  0,
1
1
by (1),   . So, (x , y ,  , 1 , 2 )  (2, 0, , 0, 0).
4
4
Fin500J Topic
Topic44
Fall 2010
2008 Olin Business School
27
Sensitivity Analysis
 We notice that
L(x *, *, *)  f (x *)   *  g (x *)  b    * '(h (x *)  c )  f (x *)
 What happens to the optimal solution value if the
right-hand side of constraint i is changed by a small
amount, say bi , i=1…k. or ci , i=1….m.
 It changes by approximately  * b or i* c i
i
i

i
*is
the shadow price of ith inequality constraint and i*
is the shadow price of ith equality constraint
Fin500J Topic 4
Fall 2010 Olin Business School
28
Sensitivity Analysis (Example)
• In the previous example, if we change the first
constraint to x2+y2=3.9, then we predict that the new
optimal value would be 2+1/4(-0.1)=1.975.
• If we compute that problem with this new constraint,
then x-y2= 3.9  1.9748.
• If, instead, we change the second constraint from x≥0
to x≥0.1, we do not change the solution or the
optimum value since 1*  0.
Fin500J Topic 4
Fall 2010 Olin Business School
29
Utility Maximization Example
T heutilityderivedfromexercise(X) and watchingmovies(M)
is described by thefunction
U  X,M   100 e  2 X  e  M
Four hours per day are available to watch moviesand exercise.
Our Lagrangianfunctionis
L(X,M,λ)  100 e  2 X  e  M    X  M  4 
First - Order Conditions:
LX  2e  2 X  λ  0
LM  e  M  λ  0
Lλ  X  M  4  0
Fin500J Topic 4
Fall 2010 Olin Business School
30
Utility Max Example Continued
First - Order Conditions:
LX  2e  2 X  λ  0, (1) LM  e  M  λ  0, (2) Lλ  X  M  4  0 (3)
From (2) we get thatλ  e -M . Subst it uting into(1), we get
2e - 2 X  e  M  0 Solving (3) for M and substituting, we get
2e-2X  e ( 4 X )  0  ln(2)- 2X  -4  X or 3X  ln(2) 4
ln(2)  4
8  ln(2)
X* 
and M * 
3
3
Fin500J Topic 4
Fall 2010 Olin Business School
31