OPTIMIZATION
ENGR 351 Numerical Methods
Instructor: Dr. L.R. Chevalier
Recall, when determining the root,
we were seeking x where f(x) = 0
f(x)
f(x) = 0
x
With optimization, however we are
seeking f '(x) = 0
f(x)
f '(x) = 0
f '(x) = 0
x
The maximum occurs when f "(x)<0
f(x)
f '(x) = 0
f "(x) < 0
x
f(x)
The minimum occurs when f "(x)>0
f '(x) = 0
f "(x)> 0
x
Optimization
 In some techniques, we determine the
optima by solving the root problem f
'(x) =0
 If f ’(x) is not available analytically, we
may use a finite difference
approximation to estimate the
derivative
Examples of Optimization
Problems
 Design structures for minimum cost
 Design water resource project to
mitigate flood damage while yielding
maximum hydropower
 Design pump and heat transfer
equipment for maximum efficiency
 Inventory control
 Optimize planning and scheduling
Methods Presented
 One-dimensional Unconstrained
Optimization

Golden Search Method
 Constrained Optimization


Graphically
Using Excel
Specific Study Objectives
 Understand why and where optimization
occurs in engineering problem solving
 Understand the major elements of the
general optimization problem: objective
function, decision variables, and constraints
Specific Study Objectives
 Be able to distinguish between linear and
nonlinear optimization, and between
constrained and unconstrained problems
 Be able to define the golden ratio and
understand how it makes 1-D optimization
efficient
 Be able to solve a 2-D linear programming
problem graphically
Mathematical Background
An optimization or mathematical
programming problem is generally stated
as:
find x which minimizes or maximizes f(x)
subject to
di(x)  ai
ei(x) = bi
i = 1,2,……m
i=1,2,……p
Mathematical Background
find x which minimizes or maximizes f(x)
subject to
di(x)  ai
ei(x) = bi
i = 1,2,……m
i=1,2,……p
x is the design vector (n-dimensions)
f(x) is the objective function
Mathematical Background
find x which minimizes or maximizes f(x)
subject to
di(x)  ai
ei(x) = bi
i = 1,2,……m
i=1,2,……p
di(x) are inequality constraints
ei(x) are equality constraints
Mathematical Background
 If f(x) and the constraints are linear, we
have linear programming
 If f(x) is quadratic, and the constraints
are linear, we have quadratic
programming
 If f(x) in not linear or quadratic, and/or
the constraints are nonlinear, we have
nonlinear programming
Mathematical Background
find x which minimizes or maximizes f(x)
subject to
di(x)  ai
ei(x) = bi
i = 1,2,……m
i=1,2,……p
Without these, we have
unconstrained optimization
Mathematical Background
find x which minimizes or maximizes f(x)
subject to
di(x)  ai
ei(x) = bi
i = 1,2,……m
i=1,2,……p
With them, we have
constrained optimization
1-D Unconstrained Optimization
Here we see a multimodal case…
however we want the global max or min!
global maximum
f(x)
local maximum
x
local minimum
global minimum
1-D Unconstrained Optimization
We will consider the Golden Section
Search method which is based on the
Golden Ratio
5 1
 0.61803......
2
Golden Ratio and Fibonacci
Numbers
The Parthenon
5th century BC
This proportion was
considered
aesthetically pleasing
by the Greeks
O.61803
1
Golden Ratio and Fibonacci
Numbers
The Golden Ratio is related to an
important mathematical series known as
the Fibonacci numbers
0,1,1,2,3,5,8,13,21,34…..
Each number after the first two
represents the sum of the preceding two.
Note the ratio of consecutive numbers
Golden Ratio and Fibonacci
Numbers
0,1,1,2,3,5,8,13,21,34…..
0/1=0
1/1=1
1/2=0.5
2/3=0.667
3/5==0.6
5/8=0.625
8/13=0.615
Continue and
the ratio
approaches the
golden ratio!
Golden Ratio and Fibonacci
Numbers
0
1
1
2
3
5
8
13
21
34
55
89
144
233
0.00000
1.00000
0.50000
0.66667
0.60000
0.62500
0.61538
0.61905
0.61765
0.61818
0.61798
0.61806
0.61803
5 1
 0.61803......
2
1-D Unconstrained Optimization: The
Golden-Section Search
f(x)
Pick two points,
xu and xl
xu
xl
lo = xu-xl
x
1-D Unconstrained Optimization: The
Golden-Section Search
f(x)
xu
xl
lo = xu-xl
x
We will now
need a two
new points
based on the
constraints
l0 = l1 + l2
l1/l0 = l2/l1
1-D Unconstrained Optimization: The
Golden-Section Search
f(x)
Substituting
l1/(l1+l2) = l2/l1
xu
xl
lo = xu-xl
x
If the
reciprocal is
taken, and
R = l2/l1
1+R = 1/R
R2 + R - 1 = 0
1-D Unconstrained Optimization: The
Golden-Section Search
f(x)
R2 + R - 1 = 0
xu
xl
lo = xu-xl
x
This can be
solved for the
positive root
 1  1  4 1
5 1

 0.61803
2
2
1-D Unconstrained Optimization: The
Golden-Section Search
5 1
xu  xl 
d
2
x1  xl  d
f(x)
xu
xl
lo = xu-xl
x
x2  xu  d
Evaluate the
function at these
points. Two
results can
occur.
1-D Unconstrained Optimization: The
Golden-Section Search
f(x)
xl
d
x2
x1
d
xu
x
1-D Unconstrained Optimization: The
Golden-Section Search
f(x)
xl
d
x1
x2
d
Here, f(x1) > f(x2)
xu
x
1-D Unconstrained Optimization: The
Golden-Section Search
f(x)
xl
d
xu
x
x1
x2
d
Eliminate the domain to the left of x2
1-D Unconstrained Optimization: The
Golden-Section Search
f(x)
xl
d
xu
x1
x2
d
x2 becomes xl for the next round
x
1-D Unconstrained Optimization: The
Golden-Section Search
f(x)
xl
d
x1
x2
xu
x
d
If f(x1)<f(x2), eliminate points to the right of x1
1-D Unconstrained Optimization: The
Golden-Section Search
f(x)
xl
d
xu
x
x1
x2
d
Back to the first case, here the new xl is x2
1-D Unconstrained Optimization: The
Golden-Section Search
f(x)
x
d xu
old
x
=x
1
Because of the Golden Ratio,2 the previous
x1 becomes the current x2
xl
xl
1-D Unconstrained Optimization: The
Golden-Section Search
f(x)
xl
x2
d xu
5 1
xu  xl 
x1  xl 
2
x
1-D Unconstrained Optimization: The
Golden-Section Search
f(x)
xl
x2
d xu
x
Repeat this algorithm until f(x) stabilizes
1-D Unconstrained Optimization: The
Golden-Section Search
2
x
Find the maximum of 2 sin x 
10
xl
f(x l)
0.0000
0.0000
0.9443
0.9443
1.3050
1.3050
0.0000
0.0000
1.5310
1.5310
1.7595
1.7595
x2
f(x 2)
1.5279
0.9443
1.5279
1.3050
1.5279
1.4427
1.7647
1.5310
1.7647
1.7595
1.7647
1.7755
x1
f(x 1)
2.4721
1.5279
1.8885
1.5279
1.6656
1.5279
0.6300
1.7647
1.5432
1.7647
1.7136
1.7647
xu
f(x u)
4.0000
2.4721
2.4721
1.8885
1.8885
1.6656
Let’s review the spreadsheet file
opt-a.xls
-3.1136
0.6300
0.6300
1.5432
1.5432
1.7136
d
2.4721
1.5279
0.9443
0.5836
0.3607
0.2229
Example
Perform three iterations of
the golden section search to
maximize
f(x) = -1.5x6 - 2x4 +12x
8
f(x)
using the initial
guesses
xl=0 and xu =2
10
6
4
2
0
0.0
0.5
1.0
x
1.5
Solution
xl
f(x l)
0.0000
0.0000
0.4721
0.0000
0.0000
5.5496
x2
f(x 2)
0.7639
0.4721
0.7639
x1
8.1879
5.5496
8.1879
Reference opt-a.xls
f(x 1)
1.2361
0.7639
0.9443
4.8142
8.1879
8.6778
xu
f(x u)
2.0000 -104.0000
1.2361
4.8142
1.2361
4.8142
d
1.2361
0.7639
0.4721
Use of Solver
 If SOLVER is not under Tools, you’ll
have to add it



Use <TOOLS - ADD INS> command
Choose SOLVER ADD-IN
If not available as an option, you will need
to install it from the original MS Office CD
Reference opt-a.xls
Constrained Optimization
Linear programming (LP) is an
optimization approach that deals with
meeting a desired objective
- maximizing profit
- minimizing cost
Both the objective function and the
constraints are linear in this case
Constrained Optimization
Objective function
Maximize Z = c1x1 +c2x2 +…..cnxn
or
Minimize Z = c1x1 +c2x2 +…..cnxn
where ci = payoff of each unit of the jth activity
xi = magnitude of the jth activity
Constrained Optimization
Objective function
Maximize Z = c1x1 +c2x2 +…..cnxn
or
Minimize Z = c1x1 +c2x2 +…..cnxn
where ci = payoff of each unit of the jth activity
xi = magnitude of the jth activity
Hence, Z is the total payoff
due to the total number of
activities, n
Constrained Optimization
The constraints can be represented by:
ai1x1 +bi2x2+…..ainxn  bi
where aij = amount of the ith resource
that is consumed for each unit of the jth
activity
bi = amount of the ith resource available
Constrained Optimization
Finally, we add the constraint that all
activities must have a positive value
xi  0
Setting up the general problem
 Gas processing plant that receives a
fixed amount of raw gas each week
 Capable of processing two grades of
heating gas (regular and premium)
 High demand for the product (I.e.
guaranteed to sell)
 Each grade yields a different profit
Similar to Problem 15.1 p. 377
Setting up the general problem
 Each grade has different production
time and on-site storage constraints
 Facility is only open 120hrs/week
 Using the factors in the table on the
next page, develop a linear
programming formulation to maximize
profits for this operation.
Parameters
Product
Regular Premium Resource Availability
7
11
77
Resource
Raw Gas
3
(m /tonne)
Production Time 10
(hr/tonne)
Storage
9
Profit (/tonne)
150
8
120
6
175
Note: a metric ton, or tonne, is equal to 1000 kg)
Parameters
Product
Regular Premium Resource Availability
7
11
77
Resource
Raw Gas
3
(m /tonne)
Production Time 10
(hr/tonne)
Storage
9
Profit (/tonne)
150
8
120
6
175
Let x1 = amount of regular and
x2 = amount of premium
Objective Function
Product
Regular Premium Resource Availability
7
11
77
Resource
Raw Gas
3
(m /tonne)
Production Time 10
(hr/tonne)
Storage
9
Profit (/tonne)
150
8
120
6
175
Total Profit = 150 x1 + 175 x2
Maximize Z = 150 x1 + 175 x2
Objective Function
Product
Regular Premium Resource Availability
7
11
77
Resource
Raw Gas
3
(m /tonne)
Production Time 10
(hr/tonne)
Storage
9
Profit (/tonne)
150
8
120
6
175
Total Profit = 150 x1 + 175 x2
Maximize Z = 150 x1 + 175 x2
Objective
function
Constraints
Product
Regular Premium Resource Availability
7
11
77
Resource
Raw Gas
3
(m /tonne)
Production Time 10
(hr/tonne)
Storage
9
Profit (/tonne)
150
8
120
6
175
7x1 + 11x2  77
10x1 + 8x2  120
x1  9
x2  6
x1,x2  0
(material constraint)
(time constraint)
(storage constraint)
(storage constraint)
(positivity constraint)
Graphical Solution
14
12
10
x2
8
6
4
2
0
0
5
10
x1
15
Graphical Solution
Now we need to add the objective
function to the plot. Start with
Z = 0 (0=150x1 + 175x2)
and
Z = 500 (500=150x1 + 175x2)
Graphical Solution
Z=1550
12
10
8
6
x2
Still in
feasible
region
x1 8
x2 2
14
4
2
0
-5
-2
0
5
-4
x1
10
15
Excel Solution: Using Solver
Solver Parameters
Note: See Example 15.3 p. 388
Solver Solution
Recall
graphical
solution
x1 8
x2 2
Example
Develop the equations (objective
function and constraints) needed to
optimize the problem on the next
slide.
Example
A construction site requires a minimum of
10,000 yd3 of sand and gravel mixture. The
mixture must contain no less than 5000 yd3
of sand and no more than 6000 yd3 of
gravel. The material may be obtained from
two sites
Site
Delivery Cost % Sand
($/yd3)
% Gravel
1
2
5
7
70
40
30
60
Excel Solution
C
x1
x2
0.00
0.00
0.00
Objective Function
Constraints
x1 + x2
0.3x1 +0.6x2
0.7x1+0.4x2
x1
x2
0.00
0.00
0.00
0.00
0.00
>=
>=
<=
>=
>=
10,000
5000
6000
0
0
Excel Solution
C
x1
x2
63333.33
3333.33
6666.67
Objective Function
Constraints
x1 + x2
0.3x1 +0.6x2
0.7x1+0.4x2
x1
x2
10000.00
5000.00
5000.00
3333.33
6666.67
>=
>=
<=
>=
>=
10,000
5000
6000
0
0