Optimization
from Prof. Goldsman’s lecture notes
Outline
What is a Model?
Operation Research
Optimization Example
Shortest Path
Introduction to Modeling
What is a Model?
Abstraction, representation
Infinitely many models of the same reality
Often a model is created for a purpose
A good model discards the irrelevant
A good model retains what is crucial
Often we believe we understand something better after
modeling it
We trust a model if it gives accurate predictions
(qualitative or quantitative)
Note that…
When you want to create a model…
Collect the necessary data
Distinguish between the model input
and model output
Lengthy notes are worthless
KISS Rule
Operation Research
Higher level of theoretical and mathematical
orientation
Categorization of Operation Research:
Wants to model the problem
Encounter the problems of estimating the values of the
parameters
Parameters may not be constant over time
2 Approaches:
Forget the fact that they are RV and use the model w/o
RV taken into consideration – Deterministic Approach
Use model w/ RV – Probabilistic Approach
Optimization Example
Shortest Path of Auto Travel Routes
180
a
c
20
200
100
t
S
100
150
b
80
Distances are in miles
d
70
Shortest Path (Cont’d)
Optimal Solution has length 270
miles
180
a
c
20
200
100
t
S
100
150
b
80
d
70
But it did not go to node d and a
Shortest Path (Cont’d)
Algorithm actually finds a tree giving shortest
paths from s to every
graph
a node in180
c
20
200
100
t
S
100
150
b
80
d
70
Shortest Path: Definitions
Graph G= (V,E)
V: vertex set, contains special vertices s and t
E: edge set
Costs Cij on edges (i, j) in E
Cij >= 0
no cycles with negative total cost
Cost of a path = sum of edge costs
Objective: find min cost path from s to t
Shortest Path (Cont’d)
A Mathematical Problem
An Optimization Problem
It has:
A set of possible solutions (paths from s to t)
An objective function (min the sum of edge costs)
An algorithm that correctly and quickly solves cases of the
shortest path problem, provided that
The instances satisfy Cij >= 0
The instances are not too huge
Example
Goal: use a car for 4 years at min cost
Purchase 1st year 1 year 2nd year 2 year 3rd year 3rd year
Cost
maint. Resale maint
Resale maint Resale
New
Car
15000
Used 5000
Car
1000
11000
1000
9000
1500
8000
2000
4000
3000
3000
3500
2500
Example (Cont’d)
Vertices of graph need not represent physical
locations
V= {0,1,2,3,4}
time 0, 1,...,4 in years
Seek least expensive path from 0 to 4
Edge cost from i to j: cost of buying a car at time
i, using it, and selling it at time j
for each edge, pick cheapest alternative (new or
used)
Example (Cont’d)
0
2
1
Buy at 0
sell at 1
3
Buy at 1,
sell at 2
Buy at time 0, keep
2 years, sell at time 2
Keep the new car for 1 year:
Keep the used car for 1 year:
Keep the new car for 2 years:
Keep the used car for 2 years:
5000
3000
8000
7000
4
Example (Cont’d)
Continue and find the shortest path
Note that: shortest path does allow
directed graph
i.e. you cannot go from node 2 to node 1
at a cost of 3000
Optimization Model Usage
Data
Real
problem
Math Problem
(Optimization Model)
Algorithm
Conceptual
Model
Solution to
Math Problem
What is A Successful Model?
Models must fit the real problem
Realism and Generality
Able to solve the model
Solvability and Tractability
Modeling FAQ
Tradeoff between realism and solvability
Good modelers know …
Different models limitations
Fitting a model into a wider range of real problems
Fitting a real problem into a model
Advanced modelers know how to …
Solve a wider range of models
Extend the range of cases that can be solved with
software tools
Optimization Models Spectrum
Networks
Shortest Path
Min Span Tree
Max Flow
Assignment
Transportation
Min Cost Flow
LP
Convex QP
blending
portfolio
optimization
planning
production/distribution
flow of materials
IP
NLP
logistics
scheduling
chemical
processes
materials
design
Mathematical Programming
Linear Programming
Nonlinear Programming
Integer Programming
Binary Programming
Quadratic Programming
Geometric Programming
Dynamic Programming
Mixed Integer Programming
Mathematical Programming
(Cont’d)
Three important characteristics:
Decision Variables
Objective Function
Constraints
Solving:
Graphically
Simplex
A Piece
of
Cake
Example
Farmer Jones must determine how many acres of
corn and wheat to plant this year. An acre of wheat
yields 25 bushels of wheat and requires 10 hours of
labor per week. an acre of corn yields 10 bushels of
corn and requires 4 hours of labor per week. All
wheat can be sold at $4 a bushel and all corn can be
sold at $3 a bushel. Seven acres of land and 40
hours per week of labor are available. Government
regulations require that at least 30 bushels of corn
can be produced during the current year.
Formulate an LP whose solution will tell Farmer
Jones how to maximize the total revenue from
wheat and corn.
Find the solution using graphical method.
Assignment Problem
Assignment Problem
Simplest and easiest
Def: there are N items or services available at
N locations and N other locations that require
one and only one of these N items or services
Example: 4 types of product and 4 machines.
Each machine can only produce one type of
products.
Objective: assign jobs to machines that will
minimize the total cost
Assignment Problem (Cont’d)
For unbalanced assignment problems
Add dummy rows or columns
Set the cost of these rows or columns to
be all zeros
Solving Assignment Problem
Using Hungarian Method
Subtract the smallest number in each row from every
entry in that row
Subtract the smallest number in each column from
every entry in that column
A zero cost assignment (if it can be made) is optimal. If
not, complete the Hungarian Method
Draw the min # of horizontal and vertical lines (no
diagonals) that intersect all the zeros
Subtract the smallest uncrossed number from every
other uncrossed number and add it to all elements
where the vertical and horizontal lines intersect
If a zero cost assignment can be made, it is optimal
O’w repeat steps 4 and 5 until one can be made
Solving Assignment Problem
Draw the min # of
(Cont’d)
horizontal and vertical lines
(no diagonals) that intersect
all the zeros
Smallest no = 1
Row 4 & 5: Subtract 1
Intersection: add 1
Subtract the smallest
uncrossed number from
every other uncrossed
number and add it to all
elements where the vertical
and horizontal lines
intersect
1
2
3
4
5
1
7
3
2
0
0
2
4
0
1
1
1
3
0
0
0
1
1
4
1
1
1
3
0
5
1
1
1
6
0
1
2
3
4
5
1
7
3
2
0
1
2
4
0
1
1
2
3
0
0
0
1
2
4
0
0
0
2
0
5
0
0
0
5
0
Solving Assignment Problem
(Cont’d)
1
2
3
4
5
1
7
3
2
0
1
2
4
0
1
1
2
3
0
0
0
1
2
4
0
0
0
2
0
5
0
0
0
5
0
If a zero cost assignment can be made  optimal
Solution?
4-1, 2-2, 3-3, 1-4, 5-5
Hungarian Method: Max
Objective: maximize
Note that max f(x) = min –f(x)
Solve:
Replace all the cost elements by their
negative
Continue w/ the minimization procedure
Hungarian Method: Max
Replace all the cost elements by their negative
(Cont’d)
1
2
3
4
1
5
3
5
4
2
2
1
6
3
3
5
4
6
2
4
6
3
1
4
1
2
3
4
1
-5
-3
-5
-4
2
-2
-1
-6
-3
3
-5
-4
-6
-2
4
-6
-3
-1
-4
Find the
smallest
number
per row
Subtract the smallest number in each row from every entry
in that row (perform similar calculation for column)
1
2
3
4
1
2
3
4
1
0
2
0
1
1
0
0
0
0
2
4
5
0
3
2
4
3
0
2
3
1
2
0
4
3
1
0
0
3
4
0
3
5
2
4
0
1
5
1
Find the
smallest
number
per
column
Transportation Problem
There are M sources with something
available and N destinations needing
something
Difference:
Assignment Problem:
f(x): x  y
Transportation problem:
f(x): x  y1, y2,…,yn
f(x): x1, x2,…,xm  y
Transportation Problem
Mathematical Formulation:
(Cont’d)
M
Optimize
N
c
z=
ij
x ij
i=1 j=1
subject to
M
x
ij
= bj
j = 1,…, N
ij
= ai
i = 1,…, M
i=1
N
x
j=1
N
where
b
j=1
M
j
=
a
i
i=1
ai = number of units available at i
bj = #units needed at j
cij = cost to ship 1 unit from i to j
xij = #units shipped from source i to destination j
More Example
Smallco, Inc.
Smallco, Inc. manufactures two products,
wooden toy cars and wooden toy trucks, made
from mahogany. The profit margin for the two
toys is $1.10 and $0.70, respectively. Based on
careful market analysis, it appears that Smallco
can sell about 2000 of each toy each week.
However, Smallco can only obtain a limited
amount of mahogany, roughly 3000 board-feet
per week. Producing either toy requires 1 board
foot of wood.
What is the best mix of toys for Smallco to
produce if the goal is to maximize total profit
margin?
More Example (Cont’d)
What is the objective?
What can be controlled?
How do the decisions affect the
objective?
What limits the decisions?
The LP model is …
Solve it!
Example…
Again
Dorial Auto manufactures luxury cars and trucks.
The company believes that its most likely
customers are high-income women and men. To
reach these groups, Dorial Auto has embarked on
an ambitious TV advertising campaign, and has
decided to purchase 1-minute ad spots on two
types of programs; comedy shows and football
games. Each comedy commercial is seen by 7
million high-income women and 2 million highincome man.
A 1-minute comedy spot costs $50,000 and a 1minute football spot costs $100,000. Dorial wants
to reach at least 28 million high-income women
and 24 million high-income man. How should
Dorian buy commercial time to reach their target
at the lowest possible cost?
Model With Discrete Variables
Model with specific (discrete)
values, is not a linear programming
model
Types:
Integer Programming
Mixed Integer Programming
Binary Integer Programming
Model With Discrete Variables
(Cont’d)
When do you need discrete variables?
Continuous problems
“yes/no” aspects to the decision
“How many” not “How much”
How to model it?
Non-obvious “trick”
Treat each linear function as a separate cost function,
with its own “variable”
Associate a binary variable with each of the linear
function
Enforce the requirement that you cannot do this if you
don’t have this (TRICK)
Model With Discrete Variables
(Cont’d)
It is tricky
It is not obvious and most of us
(including myself) would have a hard
time “inventing” this ourselves
Can be very difficult to solve
Example: selecting among investment
alternatives, designing supply chains,
sourcing products, production/inventory
planning, crew scheduling, vehicle
routing, etc