EE 553
Integer Programming
J. McCalley
1
Solving IPs
Minimize f(x1, x2, x3)
Subject to
h(x1, x2, x3)=c
g(x1, x2, x3)≤b
x1, x2, x3 are binary (1 or 0)
How to solve it?
Two immediate ideas:
1. Check every possible solution (exhaustive enumeration),
A huge computational burden.
2. Solve it as an LP and then round to closest binary value.
Under such an approximation, optimality cannot
guaranteed.
2
Solving IPs: Tree search methods
• Tree-search methods conceptualize
the problem as a huge tree of
solutions, and then they try to do
some smart things to avoid
searching the entire tree.
• Most popular IP (and MIP) solver
today is a tree-search method
called branch and bound. CPLEX
uses this method, in combination
with cutting planes.
• We will study the branch and
bound method.
●
●
●
●
●
●
●
● ● ● ● ● ● ● ●
3
Solving IPs: Branch & Bound
4
Solving IPs: Branch & Bound
Definitions
• Predecessor problem
• Successor problem
Problem Pj is predecessor to Problem Pk and
Problem Pk is successor to Problem Pj if:
they are identical with the exception
that one continuous-valued variable
in Pj is constrained to be integer in Pk.
●
P
● j
●
●
●
● Pk
●
● ● ● ● ● ● ● ●
5
“Zeta”
max
  17 x1  12 x 2
Example
s.t.
P0
Pose a problem P1 to be exactly like P0
except that we will constrain x1≤1.
10 x1  7 x 2  40
max
x1  x 2  5
s.t.
x1 , x 2  0
P1
Solution (as an LP) using CPLEX yields
P0
10 x1  7 x 2  40
x1  x 2  5
x1  1
Solution :
x1  1 . 667 , x 2  3 . 333 ,
  17 x1  12 x 2
x1 , x 2  0
  68 . 333
What do you expect the value of x1 to be in
the optimal solution?
Because the solution without the
constraint x1≤1 wanted 1.667 of x1, we can be
sure that the solution with the constraint
x1≤1 will want as much of x1 as it can get, i.e.,
it will want x1=1.
Solution (as an LP) using CPLEX yields
P1 Solution :
x1  1 . 0 , x 2  4 . 0 ,
  65 . 0
It worked: x1 did in fact become integer. In
fact, x2 became integer as well, but this is by
6
coincidence.
Example
Let’s consider the same problem but
with (x1,x2) constrained to be integers.
max
  68 . 333
P1 Solution :
10 x1  7 x 2  40
x1  1 . 0 , x 2  4 . 0 ,
x1  x 2  5
  65 . 0
x1 , x 2  0
x1 , x 2 integers.
Is the P1 solution we obtained,
which by chance is a feasible
solution to IP1, also optimal to IP1?
Solution :
x1  1 . 667 , x 2  3 . 333 ,
  17 x1  12 x 2
s.t.
IP1
P0
●
x1<1
●
P2
●
●
●
●
●
● ● ● ● ● ● ● ●
P1 has to be as good as or
better than its successors
because these will be more
heavily constrained, so we
need not look at P1 successors.
But what about P2? What is P2?
7
Example
P2 is the P0 problem with x1>2.
It is the remaining part of the
space we need to search.
max
  17 x1  12 x 2
s.t.
P2
10 x1  7 x 2  40
x1  x 2  5
x1  2
x1 , x 2  0
P0
Solution :
x1  1 . 667 , x 2  3 . 333 ,
  68 . 333
●
P1 Solution :
x1  1 . 0 , x 2  4 . 0 ,
x1<1 x1>2
●
  65 . 0
●
●
●
●
P2
Solution :
x1  2 . 0 , x 2  2 . 857 ,
  68 . 284
●
● ● ● ● ● ● ● ●
Use CPLEX to solve P2. (see solution on the tree). Observe P2 is infeasible!
(infeasible to the IP, that is, because x2 is non-integer).
So can we draw any useful conclusion from this observation?
YES!
But
why?
1. The P2 solution, 68.284, is better than our best feasible solution so
far, 65.0 (65.0 establishes a lower boundsolution is no less than 65).
2. Although P2 is infeasible, we can add constraints and find feasible
solutions in successor problems.
3. We are not sure any of those successor problems will be “better than
our best” (of 65), but because 68.284>65, we know it is worth trying. 8
Yes, Why? (In more detail…)
Compare the objective function
value of P2, 68.8286, with the
P Solution :
objective function value of P1,
x  1 . 667 , x  3 . 333 ,
  68 . 333
65. Since we are maximizing,
●
P Solution :
the objective function of P2 is
x1<1 x1>2 P Solution :
x  2 . 0 , x  2 . 857 ,
better. But the P2 solution is not x  1 .0 , x  4 .0 ,●
●
  68 . 284
  65 . 0
feasible. But we can constrain x2
so that we get a feasible
●
●
●
●
solution. Whether that feasible
● ● ● ● ● ● ● ●
solution will have better
objective function value we do
not know.
What we do know is, because the objective function value of P2 (68.8286)
is better than the objective function value of P1 (65), it is worthwhile to
check it. Although the objective function value of successor problems to
P2 can only get worse (lower), they might be better than P1, and if we can
find a successor (or a successor’s successor,…) that is feasible, it might be
better than our best current feasible solution, which is P1.
9
Example
0
1
2
1
1
2
2
1
2
Example
P0
Solution :
x1  1 . 667 , x 2  3 . 333 ,
  68 . 333
●
P1 Solution :
x1  1 . 0 , x 2  4 . 0 ,
●
  65 . 0
Question: What if the P2
solution would have been 64?
Would you have searched its
successor nodes?
x1<1 x1>2
●
●
●
●
P2
Solution :
x1  2 . 0 , x 2  2 . 857 ,
  68 . 284
●
● ● ● ● ● ● ● ●
NO! Why not?
Because successor nodes, even if feasible, would necessarily be
more constrained than P2 and therefore no better than its solution.
Since we already have a feasible solution of 65, and P2 successors
could be no better than 64, there is no use searching them. Again,
10
the value of 65 establishes a lower bound on the problem solution.
But because P2 solution is
better than our current best
P Solution :
feasible solution, we should
x  1 . 667 , x  3 . 333 ,
pursue P2 successor problems.
P Solution :
  68 . 333
P Solution :
So what constraint should we x  1 .0 , x  4 .0 ,
x  2 . 0 , x  2 . 857 ,
x1>2   68 .284
add to P2?
x1<1
  65 . 0
x2<2 x2>3
Our choices are x2<2 and x2>3.
Let’s try x2<2, using P3.
P Solution :
Example
0
1
2
2
1
1
1
2
2
3
max
  17 x1  12 x 2
x1  2 . 6 , x 2  2 . 0 ,
  68 . 2
s.t.
10 x1  7 x 2  40
P3
x1  x 2  5
x1  2
x2  2
The P3 solution is not feasible.
Should we branch further?
YES!
x1 , x 2  0
1. The P3 solution, 68.2, is better than our best feasible solution so far, 65.0.
2. Although P2 is infeasible, we can add constraints and find feasible solutions in
successor problems.
3. We are not sure any of those successor problems will be “better than our best”
11
(of 65), but because 68.2>65, we know it is worth trying.
So what constraint should we
add to P3?
Our choices are x1<2 and x1>3.
Let’s try x1<2, using P4.
max
  17 x1  12 x 2
P0
Solution :
x1  1 . 667 , x 2  3 . 333 ,
  68 . 333
P1 Solution :
x1<1
x1>2 
Solution : x2<2
10 x1  7 x 2  40
P3
x1  x 2  5
x1  2 . 6 , x 2  2 . 0 ,
x1  2
  68 . 2
x2  2
x1 , x 2  0
Should we branch further?
P4
P2
Solution :
x1  2 . 0 , x 2  2 . 857 ,
x1  1 . 0 , x 2  4 . 0 ,
  65 . 0
s.t.
P4
Example
x1<2
Solution :
 68 . 284
x2>3
x1>3
x1  2 . 0 , x 2  2 . 0
  58 . 0
No! Why?
Two reasons, either one of which is enough:
• The P4 solution is feasible! And so we will not find another better feasible
solution that is successor to P4.
• The P4 objective is 58, worse than our best (65). So P4 and any further
successor nodes are of no interest.
12
Now what? We have to decide on going
back to P3 or P2. Choose P3.
Example
P0
We impose x1>3, using P5.
max
  17 x1  12 x 2
10 x1  7 x 2  40
P5
x1  1 . 667 , x 2  3 . 333 ,
  68 . 333
P1 Solution :
x1<1
  65 . 0
P2
x1>2 
Solution : x2<2
x1  x 2  5
P3
x1  3
x1  2 . 6 , x 2  2 . 0 ,
x2  2
  68 . 2
x1 , x 2  0
Should we branch further?
P4
Solution :
x1  2 . 0 , x 2  2 . 857 ,
x1  1 . 0 , x 2  4 . 0 ,
s.t.
Solution :
x1<2
Solution :
x1>3 P
5
 68 . 284
x2>3
Solution :
x1  2 . 0 , x 2  2 . 0
x1  3 . 0 , x 2  1 . 4286
  58 . 0
  68 . 1429
Yes! Why?
1. The P5 solution, 68.1429, is better than our best feasible solution so far, 65.0.
2. Although P5 is infeasible, we can add constraints and find feasible solutions in
successor problems.
3. We are not sure any of those successor problems will be “better than our best”
13
(of 65), but because 68.1429>65, we know it is worth trying.
So what constraint should we
add to P5?
Our choices are x2<1 and x2>2.
Let’s try x2<1, using P6.
  17 x1  12 x 2
max
P0
Solution :
x1  1 . 667 , x 2  3 . 333 ,
  68 . 333
P1 Solution :
P2
x1<1
x1>2 
Solution : x2<2
10 x1  7 x 2  40
P3
x1  x 2  5
x1  2 . 6 , x 2  2 . 0 ,
x1  3
  68 . 2
x2  1
x1 , x 2  0
Should we branch further?
Yes! Why?
P4
Solution :
x1  2 . 0 , x 2  2 . 857 ,
x1  1 . 0 , x 2  4 . 0 ,
  65 . 0
s.t.
P6
Example
x1<2
Solution :
x1>3 P
5
 68 . 284
x2>3
Solution :
x1  2 . 0 , x 2  2 . 0
x1  3 . 0 , x 2  1 . 4286
  58 . 0
  68 . 1429
P6
Solution :
x2<1
x2>2
x1  3 . 3 , x 2  1 . 0
  68 . 1
1. The P6 solution, 68.1, is better than our best feasible solution so far, 65.0.
2. Although P6 is infeasible, we can add constraints and find feasible solutions in
successor problems.
3. We are not sure any of those successor problems will be “better than our best”
14
(of 65), but because 68.1>65, we know it is worth trying.
So what constraint should we
add to P6?
Our choices are x1<3 and x1>4.
Let’s try x1<3, using P7.
max
  17 x1  12 x 2
P0
  68 . 333
P1 Solution :
x1<1
x1>2 
Solution : x2<2
10 x1  7 x 2  40
P3
x1  x 2  5
x1  2 . 6 , x 2  2 . 0 ,
x1  3
  68 . 2
P4
x1 , x 2  0
Should we branch further?
No! Why?
x1<2
Solution :
The P7 solution is feasible! We
will not find another better
successor to P7
P7 objective is 63, worse than
our best (65), so P7’s successor
nodes are of no interest.
x1>3 P
5
 68 . 284
x2>3
Solution :
x1  3 . 0 , x 2  1 . 4286
  58 . 0
  68 . 1429
P6
Solution :
x1  3 . 0 , x 2  1 . 0
  63 . 0
Solution :
x2<1
x2>2
x1  3 . 3 , x 2  1 . 0
  68 . 1
x1<3
P7
Solution :
x1  2 . 0 , x 2  2 . 0
Two reasons, either one of which is enough:
•
P2
x1  2 . 0 , x 2  2 . 857 ,
x1  1 . 0 , x 2  4 . 0 ,
x2  1
•
Solution :
x1  1 . 667 , x 2  3 . 333 ,
  65 . 0
s.t.
P7
Example
x1>4
Now let’s
try here.
15
So now add the x1>4 constraint
to P6, to obtain P8.
max
P0
  17 x1  12 x 2
s.t.
P8
Example
x1  1 . 667 , x 2  3 . 333 ,
  68 . 333
P1 Solution :
10 x1  7 x 2  40
x1  1 . 0 , x 2  4 . 0 ,
x1  x 2  5
  65 . 0
x1<1
P3
x2  1
Solution :
x1>2 
Solution : x2<2
 68 . 284
x2>3
x1  2 . 6 , x 2  2 . 0 ,
x1 , x 2  0
  68 . 2
Should we branch further?
No! Why?
P4
The P8 solution is feasible! We
will not find another better
successor to P8
Note the P8 objective is 68, which is
better than our best (65)! So P8
solution becomes our new best, i.e.,
it becomes our new lower bound on
the solution. That is, the objective at
the solution must be at least 68.
P2
x1  2 . 0 , x 2  2 . 857 ,
x1  4
•
Solution :
x1<2
Solution :
x1>3 P
5
Solution :
x1  2 . 0 , x 2  2 . 0
x1  3 . 0 , x 2  1 . 4286
  58 . 0
  68 . 1429
P6
Solution :
x2<1
x2>2
x1  3 . 3 , x 2  1 . 0
  68 . 1
x1<3
P7
Solution :
x1>4
P8
Solution :
x1  3 . 0 , x 2  1 . 0
x1  4 . 0 , x 2  0
  63 . 0
  68 . 0
16
Example
Question: Do we need to
check the other branch to
P5 and P2?
P0
x1  1 . 667 , x 2  3 . 333 ,
  68 . 333
P1 Solution :
Answer: Yes! Why?
Solution :
Because the objective value
for P5 and P2 at greater
than our bound of 68, so a
successor node could be
better than 68 as well.
Solution :
x1  2 . 0 , x 2  2 . 857 ,
x1  1 . 0 , x 2  4 . 0 ,
x1<1
  65 . 0
P2
P3
x1>2 
Solution : x2<2
 68 . 284
x2>3
x1  2 . 6 , x 2  2 . 0 ,
  68 . 2
P4
x1<2
Solution :
x1>3 P
5
Solution :
x1  2 . 0 , x 2  2 . 0
x1  3 . 0 , x 2  1 . 4286
  58 . 0
  68 . 1429
P6
Solution :
x2<1
x2>2
x1  3 . 3 , x 2  1 . 0
  68 . 1
x1<3
P7
Solution :
x1>4
P8
Solution :
x1  3 . 0 , x 2  1 . 0
x1  4 . 0 , x 2  0
  63 . 0
  68 . 0
17
max
Example
  17 x1  12 x 2
s.t.
P0
10 x1  7 x 2  40
x1  x 2  5
P9
x1  2
x2  3
x1 , x 2  0
Solution :
x1  1 . 667 , x 2  3 . 333 ,
  68 . 333
P1 Solution :
Solution :
x1  2 . 0 , x 2  2 . 857 ,
x1  1 . 0 , x 2  4 . 0 ,
x1<1
  65 . 0
P2
P3
x1>2 
Solution : x2<2
 68 . 284
x1  2 . 6 , x 2  2 . 0 ,
max
  68 . 2
  17 x1  12 x 2
P4
s.t.
10 x1  7 x 2  40
P10
x1  x 2  5
x1<2
Solution :
x1>3 P
5
P9: infeasible
to the LP
Solution :
x1  2 . 0 , x 2  2 . 0
x1  3 . 0 , x 2  1 . 4286
  58 . 0
  68 . 1429
P6
x1  3
Solution :
x2<1
  68 . 1
x1 , x 2  0
x1<3
Solution :
x1>4
P8
x2>2
P10: infeasible
to the LP
x1  3 . 3 , x 2  1 . 0
x2  2
P7
x2>3
Solution :
x1  3 . 0 , x 2  1 . 0
x1  4 . 0 , x 2  0
  63 . 0
  68 . 0
18
Central ideas to branch & bound
Branch, force integrality on one variable by adding a constraint to an LP-relaxation:
Use LP-relaxation to decide how to branch. Each branch adds a constraint to
previous LP-relaxation to enforce integrality on one variable that was not
integer in the predecessor solution.
Bound, continue branching only if the objective function value of the current solution is
better than the objective function value of the best feasible solution obtained so far:
Maintain the best feasible solution obtained so far as a bound on tree-paths that
should still be searched.
• If any tree node has an objective value less optimal than the identified bound,
no further searching from that node is necessary, since adding constraints can
never improve an objective.
• If any tree node has an objective value more optimal than the identified
bound, then additional searching from that node is necessary.
19
Using CPLEX to solve MIPS directly
1. Created CPLEX source within a
text file called mip.lp as follows:
maximize
17 x1 + 12 x2
subject to
10 x1 + 7 x2 <= 40
x1 + x2 <= 5
Bounds
0<= x1 <= 1000
0<= x2 <= 1000
Integer
x1 x2
end
2. Used WinSCP to port the file to
server (linux-7.ece.iastate.edu).
4. Called cplex.
5. Typed read mip.lp to read in
problem statement.
6. Typed mipopt to call the MIPsolver. The result was as follows…
20
Using CPLEX to solve MIPS directly
1. Created CPLEX source within a
text file called mip.lp as follows:
7. Typed
maximize
display solution variables 17 x1 + 12 x2
The result was:
subject to
10 x1 + 7 x2 <= 40
Variable Name
Solution Value
x1 + x2 <= 5
x1
4.000000
Bounds
All other variables in the range 1-2 are 0.
0<= x1 <= 1000
0<= x2 <= 1000
Integer
x1 x2
end
21
Recall this point in our procedure:
The question can be posed like this:
P Solution :
Depth: Do we continue from P3,
x  1 . 667 , x  3 . 333 ,
requiring x1≤2, for example? or
P Solution :
  68 . 333
P Solution :
Breadth: Do we go back to P2 to
x  2 . 0 , x  2 . 857 ,
x  1 .0 , x  4 .0 ,
examine its other branch, x2≥3?
x1>2   68 .284
x1<1
  65 . 0
x2<2 x2>3
Depth vs breadth
0
1
2
2
1
1
1
2
P3
2
Solution :
x1  2 . 6 , x 2  2 . 0 ,
  68 . 2
For high-dimensional IPs, it is usually the case that feasible solutions are
more likely to occur deep in a tree than at nodes near the root.
Finding multiple feasible solutions early in B&B is important because it
tightens the bound (in the above example, it increases the bound), and
therefore enables termination of branching at more nodes (and therefore
decreases computation). One can see that this will be the case if we consider
the bound before we find a feasible solution: the bound is infinite! (-∞ for
maximization problems and +∞ for minimization problems).
22
For high-dimensional IPs, is
there any benefit for selecting
one non-integer variable over
another when branching?
Branching variable selection
P0
x1  1 . 667 , x 2  3 . 333 ,
  68 . 333
P1 Solution :
In our problem, except initially,
there was never a decision to
make in this way because there
was never more than one noninteger variable.
Solution :
Solution :
x1  2 . 0 , x 2  2 . 857 ,
x1  1 . 0 , x 2  4 . 0 ,
x1<1
  65 . 0
P2
P3
x1>2 
Solution : x2<2
x1  2 . 6 , x 2  2 . 0 ,
 68 . 284
x2>3
P9: infeasible
  68 . 2
P4
x1<2
Solution :
x1>3 P
5
Solution :
x1  3 . 0 , x 2  1 . 4286
x  2 .0 , x  2 .0
A rich research question.
  68 . 1429
  58 . 0
For specific problems, you can
x2>2
P Solution : x2<1
pre-specify an ordering of the
P10: infeasible
x  3 .3, x  1 .0
variables required to be integer.
  68 . 1
Good orderings become apparent
x1<3 x1>4
P Solution :
P Solution :
based on experience with
x  4 .0 , x  0
running the algorithm or based x  3 . 0 , x  1 . 0
  68 . 0
on physical understanding, e.g.,   63 . 0
23
largest unit.
1
2
6
1
7
1
2
8
2
1
2
Our example was a pure integer
problem. What about a MIP?
max
Mixed integer problems
P0
  17 x1  12 x 2
x1  1 . 667 , x 2  3 . 333 ,
s.t.
IP 2
10 x1  7 x 2  40
x1  x 2  5
x1 , x 2  0
Solution :
  68 . 333
P1 Solution :
Solution :
x1  2 . 0 , x 2  2 . 857 ,
x1  1 . 0 , x 2  4 . 0 ,
x1<1
  65 . 0
P2
P3
x1 integer.
x1>2 
Solution : x2<2
 68 . 284
x1  2 . 6 , x 2  2 . 0 ,
x2>3
P9: infeasible
  68 . 2
The solution to IP2 is obtained as
soon as we solve P1 and P2.
P4
x1<2
Solution :
x1>3 P
5
x1  2 . 0 , x 2  2 . 0
x1  3 . 0 , x 2  1 . 4286
  58 . 0
  68 . 1429
Conclusion: We can easily solve
P6
MIP within our LP-relaxation
branch and bound scheme by
simply allowing the non-integer P
variables to remain relaxed.
x
x1  3 . 3 , x 2  1 . 0
7
1
Solution :
Solution :
x2<1
x2>2
P10: infeasible
  68 . 1
x1<3
Solution :
 3 .0 , x 2  1 .0
  63 . 0
x1>4
P8
Solution :
x1  4 . 0 , x 2  0
  68 . 0
24