Knapsack Model

Notes 5
Intuitive idea: what is the most valuable
collection of items that can be fit into a
backpack?
IE 312
1
Race Car Features
Cost (thousand)
Speed increase (mph)
$
1
10.2 $
8
Proposed Feature (j )
2
3
4
6.0 $ 23.0 $ 11.1 $
3
15
7
5
9.8 $
10
6
31.6
12
Budget of $35,000
Which features should be added?
Notes 5
IE 312
2
Formulation

Decision variables
xj 

ILP
1 if feature j is added
0
otherwise
max
8 x1  3x2  15 x3  7 x4  10 x5  12 x6
s.t. 10.2 x1  6.0 x2  23.0 x3  11.1x4  9.8 x5  31.6 x6  35
x j  0 or 1
Notes 5
IE 312
3
LINGO Formulation
Variables indexed by
this set
MODEL:
SETS:
Specify
index sets
FEATURES /F1,F2,F3,F4,F5,F6/: INCLUDE,SPEED_INC,COST;
ENDSETS
DATA:
SPEED_INC = 8 3 15 7 10 12;
All the
constants
COST = 10.2 6.0 23.0 11.1 9.8 31.6;
Note
; to end command
: to begin an
environment
BUDGET = 35;
ENDDATA
Objective MAX = @SUM( FEATURES: SPEED_INC * INCLUDE);
Constraints
@SUM( FEATURES: COST * INCLUDE) <= BUDGET;
@FOR( FEATURES: @BIN( INCLUDE));
END
Notes 5
IE 312
Decision variables
are binary
4
Solve using Branch & Bound
x5  1
x5  0
Candidate Problem
max
8 x1  3 x2  15 x3  7 x4  12 x6
s.t. 10.2 x1  6.0 x2  23.0 x3  11.1x4  31.6 x6  35
x j  0 or 1
Relaxed Problem
max
8 x1  3x2  15 x3  7 x4  10 x5  12 x6
s.t. 10.2 x1  6.0 x2  23.0 x3  11.1x4  9.8 x5  31.6 x6  35
0  x j  1
Notes 5
IE 312
Solution?
5
What is the Relative Worth?
x1 : 8
 0.78
Want to add this
feature second
10.2
x2 : 3  0.5
6
x3 : 15  0.65
23
x4 : 7
 0.63
11.1
x5 : 10
 1.02
9.8
x6 : 12
 0.38
31.6
Notes 5
Want to add this
feature first
IE 312
6
Solve Relaxed Problem
x5  1
x5  0
Relaxed Problem
max
8 x1  3x2  15 x3  7 x4  10 x5  12 x6
s.t. 10.2 x1  6.0 x2  23.0 x3  11.1x4  9.8 x5  31.6 x6  35
0  x j  1
8 1 15 1  7  0.257
Solution:
x1  1, still have $35 - $10.2  $24.8
x3  1, still have $24.8 - $23.0  $1.8
x4  1.8 7 , out of money
Notes 5
IE 312
Objective <= 24.8
7
Now the other node…
x5  1
x5  0
Relaxed Problem
max
8 x1  3 x2  15 x3  7 x4  10  12 x6
s.t. 10.2 x1  6.0 x2  23.0 x3  11.1x4  31.6 x6  35  9.8
0  x j  1
Solution:
x1  1  Still have $25.2  $10.2  $15
x3  15
Notes 5
23
 0.652
Objective <= 27.8
IE 312
8
Next Step?
x5  1
x5  0
Objective <= 24.8
Notes 5
Objective <= 27.8
IE 312
9
Rule of Thumb: Better Value
Relaxed Problem
x5  1
x5  0
max
s.t.
Obj <= 24.8
x1  0
x1  1
Obj. <=26.4
18  3 x2  15 x3  7 x4  12 x6
6.0 x2  23.0 x3  11.1x4  31.6 x6  15.2
0  x j  1
Obj. <= 27.8
Relaxed Problem
max
10  3x2  15 x3  7 x4  12 x6
s.t. 6.0 x2  23.0 x3  11.1x4  31.6 x6  25.2
0  x j  1
Solution:
x3  15
23
 0.652
Solution: x3  1  Still have $25.2  $23  $2.2
x4  2.2
Notes 5
11.1
 0.198
IE 312
10
Next Level
x5  1
x5  0
Obj <= 24.8
x1  1
x1  0
Obj. <=26.4
x3  1
x3  0
Obj. = 25
Infeasible
Now What?
Notes 5
IE 312
11
Next Steps …
Still need to continue
branching here.
x5  1
x5  0
Obj <= 24.8
x1  0
Obj. <= 26.4
x1  1
x3  1
x3  0
Obj. = 25
Notes 5
Infeasible
IE 312
Finally we will
have accounted
for every
solution!
12
Capital Budgeting


Notes 5
Multidimensional knapsack problems
are often called capital budgeting
problems
Idea: select collection of projects,
investments, etc, so that the value is
maximized (subject to some resource
constraints)
IE 312
13
NASA Capital Budgeting
Budget Requirements
2000- 2005- 2010- 2015- 2020Not Depends
j Mission
2004 2009 2014 2019 2024 Value With
On
1 Communication satellite
6
200
2 Orbital microwave
2
3
3
3 Io lander
3
5
20
4 Uranus orbiter 2020
10
50
5
3
5 Uranus orbiter 2010
5
8
70
4
3
6 Mercury probe
1
8
4
20
3
7 Saturn probe
1
8
5
3
8 Infrared imaging
5
10
11
9 Ground-based SETI
4
5
200
14
10 Large orbital structures
8
4
150
11 Color imaging
2
7
18
8
2
12 Medical technology
5
7
8
13 Polar orbital platform
1
4
1
1
300
14 Geosynchronous SETI
4
5
3
3
185
9
Budget
10
12
14
14
14
Notes 5
IE 312
14
Formulation

Decision variables
xj 

1
if mission j is chosen
0 otherwise
Budget constraints
6 x1  2 x2  3x3  1x7  4 x9  5x12  10 (year 1)
Notes 5
IE 312
15
Formulation


Mutually exclusive choices
x4  x5  1
x8  x11  1
x9  x14  1
Dependencies
x11  x2
x4  x3

Notes 5
IE 312
16
Assignment Problems



Notes 5
Assignment problems deal with optimal
pairing or matching of objects in two
distinct sets
Decision variable
1 if i is assigned to j
xij 
0 otherwise
Let A be the set of allowed assignments
and cij be the cost of assigning i to j.
IE 312
17
Formulation
cij xij
min
( i , j ) in A
xij  1, for all i  N
s.t.
j:( i , j ) in A
xij  1, for all j  N 2
i:( i , j ) in A
xij  0,1 for all (i,j ) in A
Notes 5
IE 312
18
Example

Notes 5
We must determine how jobs should be
assigned to machines to minimize setup
times, which are given below:
Job 1
Job 2
Job 3
Job 4
Machine 1
14
5
8
7
Machine 2
2
12
6
5
Machine 3
7
8
3
9
Machine 4
2
4
6
10
IE 312
19
Hungarian Algorithm

Notes 5
Step 1: (a) Find the minimum element
in each row of the cost matrix. Form a
new matrix by subtracting this cost
from each row. (b) Find the minimum
cost in each column of the new matrix,
and subtract this from each column.
This is the reduced cost matrix.
IE 312
20
Example: Step 1(a)
Job 1
Job 2
Job 3
Job 4
Machine 1
14
5
8
7
Machine 2
2
12
6
5
Machine 3
7
8
3
9
Machine 4
2
4
6
10
Notes 5
Job 1
Job 2
Job 3
Job 4
Machine 1
9
0
3
2
Machine 2
0
10
4
3
Machine 3
4
5
0
6
Machine 4
0
2
4
8
IE 312
21
Example: Step 1(b)
Job 1
Job 2
Job 3
Job 4
Machine 1
9
0
3
2
Machine 2
0
10
4
3
Machine 3
4
5
0
6
Machine 4
0
2
4
8
Job 1
Job 2
Job 3
Job 4
Machine 1
9
0
3
0
Machine 2
0
10
4
1
Machine 3
4
5
0
4
Machine 4
0
2
4
6
Notes 5
IE 312
22
Hungarian Algorithm

Notes 5
Step 2: Draw the minimum number of
lines that are needed to cover all the
zeros in the reduced cost matrix. If m
lines are required, then an optimal
solution is available among the covered
zeros in the matrix. Otherwise, continue
to Step 3.
IE 312
23
Example: Step 2
Job 1
Job 2
Job 3
Job 4
Machine 1
9
0
3
0
Machine 2
0
10
4
1
Machine 3
4
5
0
4
Machine 4
0
2
4
6
We need 3<4 lines, so
continue to Step 3
Notes 5
IE 312
24
Hungarian Algorithm

Notes 5
Step 3: Find the smallest nonzero
element (say, k) in the reduced cost
matrix that is uncovered by the lines.
Subtract k from each uncovered
element, and add k to each element
that is covered by two lines. Return to
Step 2.
IE 312
25
Example: Step 3
Job 1
Job 2
Job 3
Job 4
Machine 1
9
0
3
0
Machine 2
0
10
4
1
Machine 3
4
5
0
4
Machine 4
0
2
4
6
Notes 5
Job 1
Job 2
Job 3
Job 4
Machine 1
10
0
3
0
Machine 2
0
9
3
0
Machine 3
5
5
0
4
Machine 4
0
1
3
5
IE 312
26
Example: Step 2 (again)
Job 1
Job 2
Job 3
Job 4
Machine 1
10
0
3
0
Machine 2
0
9
3
0
Machine 3
5
5
0
4
Machine 4
0
1
3
5
Need 4 lines, so we have the
optimal assignment and we stop
Notes 5
IE 312
27
Example: Final Solution
Job 1
Machine 1
10
Machine 2
Machine 3
Machine 4
Job 2
0
0
5
9
Job 3
3
0
3
0
5
0
1
Job 4
0
3
4
5
Optimal assignment
x12  1, x33  1, x41  1, x24  1
Notes 5
IE 312
28
Travelling Salesman Problem
(TSP)
Fort Dodge
Waterloo
Carroll
Boone
Ames
Marshalltown
West Des Moines
Notes 5
IE 312
What is the shortest route,
starting in Ames, that visits
each city exactly ones?
29
TSP Solution
Fort Dodge
Waterloo
Carroll
Boone
Ames
Marshalltown
West Des Moines
Notes 5
IE 312
30
Not a TSP Solution
Fort Dodge
Waterloo
Carroll
Boone
Ames
Marshalltown
West Des Moines
Notes 5
IE 312
31
Applications

Routing of vehicles (planes, trucks, etc.)

Routing of postal workers

Drilling holes on printed circuit boards

Routing robots through a warehouse,
etc.
Notes 5
IE 312
32
Formulating TSP

A TSP is symmetric if you can go both
ways on every arc
1 if the route includes leg between i and j
xi , j  
otherwise
0
min
 c
i
j i
x
j i
Notes 5
i, j
x
i, j i, j
2
IE 312
33
Example
10
1
1
1
1
5
2
3
10
10
4
1
1
1
6
Formulate a TSP
Notes 5
IE 312
34
Subtours




Notes 5
It is not sufficient to have two arcs
connected to each node
Why?
Must eliminate all subtours
Every subset of points must be exited
IE 312
35
How do we eliminate
subtours?
10
1
1
1
1
5
Notes 5
2
3
10
10
IE 312
4
1
1
1
6
36
Asymmetric TSP

Now we have decision variables
1 if the route goes from i to j
xi , j  
otherwise
0

Constraints
x
j i
i, j
x
j i
Notes 5
j ,i
1
(leave i )
1
(enter i )
IE 312
37
Asymmetric TSP (cont.)

Each tour must enter and leave every
subset of points
 x
iS jS

Notes 5
i, j
1
Along with all variables being 0 or 1,
this is a complete formulation
IE 312
38
Example
10
1
1
1
1
5
2
3
10
4
10
1
1
1
6
Assume a two unit penalty for passing from a high to lower
numbered node.
This is now an asymmetric TSP. Why?
Notes 5
IE 312
39
Subtour Elimination


Making sure there are no subtours
involves a very large number of
constraints
Can obtain simpler constraints if we go
with a nonlinear objective function
y k ,i
Notes 5
1 if the kth point visi ted is i

otherwise
0
IE 312
40
Quadratic Assignment
Formulation
min
 d  y
 y 1
i, j
i
j
k ,i
yk 1, j
k
k ,i
i
y
k ,i
1
k
yk ,i  0,1
Notes 5
IE 312
41
Example: reformulate
10
1
1
1
1
5
Notes 5
2
3
10
10
IE 312
4
1
1
1
6
42
Solving TSP



Notes 5
We can use branch-and-bound to get
an exact solution to the TSP problem
As always, the key to implementing
branch-and-bound is to relax the
problem so that we can easily solve the
relaxed problem, but we still get good
bounds
How can we relax the TSP?
IE 312
43
Relaxing the TSP
min
ci , j xi , j
i j not i
xi , j  1
(leave i )
x j ,i  1
(enter i )
j not i
j not i
Reduces to the
assignment
problem
xi , j  1
i in S i not in S
xi , j  0,1
Notes 5
IE 312
44
Branching
xi , j  0
x j ,i  0
Which cities should be selected?
What is the most important variable?
Notes 5
IE 312
45
Example: Solution to
Assignment Problem
x Ames, DSM  xDSM , Ames  xCR , IC  xIC ,CR  1
Ames
CR
IC
DSM
Notes 5
IE 312
46
Branching
xDSM , Ames  0
x Ames, DSM  0
Always branch to split up the subtours!
Solving the assignment problem will
hopefully yield a feasible solution soon.
Notes 5
IE 312
47
Solution to New Assignment
Problem (Left Branch)
This makes both subtours impossible, so we get
Ames
CR
IC
DSM
Thus, optimal solution is
found by solving a total of
three assignment problems!
Notes 5
IE 312
48
Poor Branching Examples
Notes 5
xCR , Ames  0
x Ames,CR  0
xCR , Ames  0
xCR , Ames  1
IE 312
49
Set Packing, Covering, and
Partitioning
Notes 5
IE 312
50
Select Locations
Notes 5
IE 312
51
Ways of Splitting the Set

Set covering constraints
x
jJ

1
Set packing constraints
x
jJ

j
j
1
Set partitioning constraints
x
jJ
Notes 5
j
1
IE 312
52
Example: Choosing OR
Software
Software Package (j )
Algorithm 1
2
3
4
LP
Yes Yes Yes Yes
IP
Yes
Yes
NLP
Yes Yes
Cost
3
4
6
14
Formulate a set covering problem to acquire the minimum cost
software with LP, IP, and NLP capabilities.
Formulate set partitioning and set packing problems. What goals
do they meet?
Notes 5
IE 312
53
Maximum Coverage




Perhaps the budget only allows $9000
What can we then do
 Maximum coverage
How do we now formulate the problem?
Need new variables
1 if ALG not provided
yALG  
otherwise
0
Notes 5
IE 312
54
Useful Formulation Tricks!

Integer variables can be used to model
numerous things that seem difficult




Notes 5
Fixed-charge IP
Either-or constraints
If-then constraints
All of these involve introducing a
(dummy) binary variable and a “very
large number”
IE 312
55
Fixed-Charge Constraint

We need to determine how many Gadgets to make
x  Number of Gadgets made


We have variable cost of $7/ Gadget made, but also a fixed cost
of $500 if we make any Gadgets (but we could also choose not
to make any, in which case we would have no cost
How do we represent the cost?
1 if we make Gadgets
y
0 otherwise
Cost  7 x  500 y
xMy

Notes 5
Here, M is a very large number, e.g. M = 1000
IE 312
56
What-If Constraints
Need to model (continuous variable):
x  0 or x  100
Trick: add a binary variable y, and a very large M
x  My
100  x  M (1  y )
y  0,1
Here, M = 200 would do nicely
Notes 5
IE 312
57
If-Then Constraints


We want to represent
x1  1 then x2  x3  x4  0
Similar to before:
x2  x3  x4  M  y
x1  M (1  y)
y  0,1

Notes 5
Here, M=3 would be big enough!
IE 312
58