Integer Programming
Algorithms
Branch and Bound Method
Section 12.5, 12.6
Solving IP Problems is not easy (Section 11.5)
 Exponential growth of problem difficulty

Each time n is increased by one, the number of solutions
doubles
 Advances in BIP and linear programming
algorithms as well as computing speed make
solving large problems possible today

Would have taken years of computing time 25 years ago
2
Solving IP Problems is not easy (Section 11.5)

Most successful integer programming algorithms


For any given IP problem:


Incorporate a linear programming algorithm
The corresponding linear programming problem is called its
“LP relaxation”
IP problems frequently have some special structure

Can be exploited to simplify the problem
3
Solving IP Problems is not easy (Section 12.5)

Primary determinants of computational difficulty of
an IP problem
The number of integer variables
 Whether the variables are binary integer variables or general
integer variables
 Whether there is any special structure


IP problems are generally more difficult to solve than
linear programming problems
4
Solving IP Problems is not easy (Section 12.5)

Potential approach
Apply simplex method to LP relaxation and round the
answer to the nearest integer
 Approach may be adequate if variable values are quite large


Rounding will introduce only a small error
Downside 1: optimal linear programming solution not
necessarily feasible if rounded
 Downside 2: rounded solution may be far from the optimal
integer solution

5
How different can it be from LP ?
 Rounding cannot help!
LP Solution
Objective line
x2
Feasible region
x1
Integer Solution
How different can it be from LP ?
x2
The LP problem can
be feasible, whereas
its ILP version is not.
Feasible region
x1
Solving IP Problems is not easy (Section 11.5)
 Better approach

Heuristic algorithms
 Developing IP algorithms

An active area of research
8
First…Enumeration procedure
 Enumeration procedure

Natural to consider due to the finite number of
feasible solutions


The finite number can be very large
Should be structured so that only a very few
solutions are examined
9
Enumeration… a naïve solution strategy
Assuming that all variables are finite.
 Enumerate all solutions with a tree

x1=0
x2=0
x2=1
x2=2
x1=2
x1=1
x2=0
x2=1
x2=2
x2=0
x2=1
x2=2
Guaranteed to find a feasible solution if it exists
 But, exponential growth in the size of the tree/computation
time

Principle of the IP Algorithms
The ILP algorithms are based on exploiting the computational success of
LP. The strategy of these algorithms involves three steps:
 Step 1. Relax the solution of the ILP by deleting the integer restriction
on all integer variables and replacing any binary by the continuous
range 0 ≤ y ≤ 1. The result of the relaxation is a regular LP.

Step 2. Solve the LP, and identify its continuous optimum.

Step 3. Starting from the continuous optimum point, add special
constraints that iteratively modify the LP solution space in a manner
that will eventually render an optimum extreme point satisfying the
integer requirements.
Principle of the IP Algorithms…
Several general methods have been developed for generating the
special constraints in step 3
1.
2.
Branch and Bound Method
Cutting plane method
Before…..
1. Show how the non-convex shaded solution spaces can be
represented by a set of simultaneous constraints.
Before…..
1.
2. Which of two solution spaces is larger?
a.
b.
ILP solution space
LP relaxation solution space
3. What can we say about their respective optimal solutions?
a.
b.
For a maximization problem
For a minimization problem
The Branch-and-Bound Algorithm
and its Application (section 12.6)
A family of algorithms: Branch & Bound (B&B)

Probably the most popular method for solving Integer Linear
Programming problems (First presented in 1960).

Branch-and-bound technique

Basic approach: divide and conquer
Branch and Bound Method

Systematically partitions the solution space into mutually
exclusive subspaces to find the optimal solution for the
original problem.

Implicitly (and iteratively) searches all the solution space
until it finds the optimum, performing the search in an
intelligent way.
Branch and Bound Main Idea
Solve the ‘relaxed’ problem, i.e. no integrality constraints.
 If the relaxed problem is infeasible – backtrack (there is no integer
solution in this branch)
 If the solution is integral – terminate (‘feasible’).
 Otherwise split on a variable for which the assignment is nonintegral, and repeat for each case.

Divide and Conquer Principle

Since the original “large” problem is too difficult to be solved directly,
it is divided into smaller and smaller sub-problems until these subproblems can be conquered.

The dividing (branching) is done by partitioning the entire set of
feasible solutions into smaller and smaller subsets.

The conquering (fathoming) is done partially by bounding how good
the best solution in the subset can be and then discarding the subset if
its bound indicates that it cannot possibly contain an optimal solution
for the original problem.

Application example and general process

See Pages 502-509 in the text
Example
Max
1- LP solution
Z = 5X1 + 4X2
Subject to:
X1 + X2 ≤ 5
10X1 + 6X2 ≤ 45
X1, X2  0, integers.
Dividing (branching)
2- Adding some constraints or Branching
Dividing (branching)
Conquering (fathoming)

LP2 cannot lead to a better
integer solution then LP1
2- Adding some constraints or Branching
In fact, LP1 is optimal…
Set the bounds
UB=23.75
LB=-∞
Dividing (branching)
LP1 solved
UB=23.75
LB=23
Conquering (fathoming)

LP2 cannot lead to a better
integer solution then LP1
LP2 solved
UB=23
LB=23
LP1 is optimal
B&B Method Terminology

Branching variable: The variable that is used to branch to the
new sub-problem.

Fathomed node: A node that cannot produce a better solution
than the current best feasible one.

Lower bound: The best current feasible solution that can be used
to discard unexplored nodes (non-promising). It is the worst case
scenario for the solution of the problem.

Upper bound: An optimal but infeasible solution that represents
the best-case scenario for the problem’s solution.

The range between the upper and lower bounds is where lies the
feasible optimal solution.
The Method…One more time

It divides the space into sub-spaces, this is known as
branching. The sub-problem is simple enough that it can be
solved and its solution can be compared to the best current
solution (lower bound).

If it is a better solution, it becomes the new current “best”
solution (new lower bound) and the one before is fathomed.
(This is one type of fathoming, prune,…)

The
B&B
method
only
explores
promising
sub-problems
(branches).
It
does
not
explore
unpromising branches which are removed, another type of
fathoming.
B&B Method Steps
1.
Obtain an upper bound of the problem, usually by using LP
relaxation. If the optimal solution is integer, stop, otherwise
continue
2.
Identify the most promising (?) decision variable to use as the
basis for branching. Use this variable to divide the solution
space into two sub-problems.
B&B Method Steps (continued)
Continue subdividing the solution space until either you find
a feasible solution or it is proved that the sub-problem is not
promising (either because it does not have a feasible integer
solution or its best possible solution is not better than the
current lower bound). These are three ways to fathom a
branch (dismissed from further consideration) if:
3.



Test 1 - Its bound  Z*.
Test 2 - Its LP Relaxation has no feasible solutions.
Test 3 - The optimal solution for its LP relaxation is an integer (if this
solution is better than the current lower bound, it becomes the new lower
bound and Test 1 is reapplied to all the unfathomed sub-problems with
the new lower bound).
B&B Method Steps (continued)
4.
If the solution is feasible, compare it to the current lower
bound and if the solution is better, replace the lower bound
with the new solution. If the solution is not promising, then
fathom the branch.
5.
Backtrack until a promising unexplored branch is found.
Continue branching, updating bounds, and fathoming
branches until you have explored it completely
Example 1
Max
Z = X1 + X2
Subject to:
- X1 + X2 ≤ 2
8X1 + 2X2 ≤ 19
X1, X2  0, integer
L1
L2
 At the optimum X1 = 1.5, X2 = 3.5 and Z0 = 5
 The solution is not integer,
we branch on either X1 or X2.
Set the bounds
UB=5
LB=-∞
Example 1… we branch on X1
P1
Max
Z = X1 + X2
Subject to:
- X1 + X2 ≤ 2
8X1 + 2X2 ≤ 19
X1 ≤ 1
X1, X2  0.
P2
Max
Z = X1 + X2
Subject to:
1.
2.
3.
We update
our LB
- X1 + X2 ≤ 2
8X1 + 2X2 ≤ 19
X1  2
X1, X2  0.
We fathom the node (Z < LB)
We update the UB (Least among highest so far)
LB=UB, (1,3) is optimal.
Example 2
z  Max 4 x1  x2
7 x1  2 x2  14
x2  3
2 x1  2 x2  3
x1  0, x 2  0
x1 , x2 integers
X2
( x1, x2 )  (
X1
20
,3)
7
Example 2
• Bounding: solve linear programming relaxation: ( x1, x2 )  ( 20 ,3)
7
• Branching: divide into 2 sub-problems: x1  2, x1  3
• Node selection: select left node
x1  2 x1  3
upper = 59/7
lower =  
x1  2
X2
x1  3
X1
Example 2
• Bounding: solve linear programming relaxation LP1:
UB = 59/7
x1  2
Z1  15 / 2
x1  3
1
( x1, x2 )  (2, )
2
Example 2…
• Bounding: solve linear programming relaxation LP2: infeasible
• Node is thus cut by infeasibility, Update upper bound
• Node selection: select (lowest) right node
UB = 59/7
x1  2
UB = 15/2
x1  3
infeasible
Example 2
• Bounding: solve linear programming relaxation LP1:
1
( x1, x2 )  (2, )
2
• Branching: divide into 2 sub-problems:
x2  0, x2  1
UB = 59/7
x1  2
x1  3
Example 2…
• Bounding: solve linear programming relaxation LP3:
• Branching: divide into 2 sub-problems:
• Node selection: select (highest) right node
1
( x1, x2 )  (2, )
2
x2  0, x2  1
x1  2
UB = 59/7
x1  2
x1  3
UB = 15/2
x2  0
x2  1
x1  3
Example 2…
• Bounding: solve linear programming relaxation:
• Node is thus trimmed by optimality
• Node selection: select (lowest) left node
( x1, x2 )  (2,1)
UB = 59/7
x1  2
x1  3
infeasible
UB = 15/2
x2  0
x2  1
LB = 7
X1
Example 2…
3
• Bounding: solve linear programming relaxation:
( x1, x2 )  ( ,0)
2
• Solution is 6, because 6 < 7 , node is trimmed by bound
upper = 59/7
x1  2
x1  3
infeasible
upper = 15/2
x2  0
x2  1
LB = 7
Z3= 6
We fathom the node (Z3 < LB)
X2
x1  2 x1  3
x2  1
Update UB to 7
UB = LB = 7
x2  0
X1
Example 3.
Develop the B&B tree of the following problem. For convenience,
select X1 as the branching variable at node 0.
Max
Z = 3X1 + 2 X2
Subject to:
2X1 + 5X2 ≤ 9
4X1 + 2X2 ≤ 9
X1, X2  0, integer
𝐿𝑃0
𝐿𝑃1
𝐿𝑃2
𝐿𝑃3
𝐿𝑃5
𝐿𝑃4
𝐿𝑃6
Example 4.
Solve the following problems using the branch and bound method.
Max
Subject to:
Z = X1 + X2
2X1 + 5X2 ≤ 16
6X1 + 5X2 ≤ 27
X1, X2  0, and integer
Branch on x2
Example 5.
Solve the following problems using the branch and bound method.
Max
Z = 2 X1 + 3X2
Subject to:
5X1 + 7X2 ≤ 35
4X1 + 9X2 ≤ 36
X1, X2  0, and integer
Z = 14.47
(3.7,2.35)
Z = 14
(3,2.666667)
Z = 14.428
(4,2.142857)
Minimization Example
Min z= 6x1 +8x2
s.t.
3x1 +x2 ≥4
x1 +2x2 ≥4
x1, x2 ≥ 0 and integer
LB = 17.6
We define UB
and LB
Set the bounds,
the branch
UB = +∞
LB = 17.6
(0.8,1.6) Z0 = 17.6
x2  1
x2  2
Minimization Example 1
Min z= 6x1 +8x2
s.t.
3x1 +x2 ≥4
x1 +2x2 ≥4
x2 ≤ 1
x1, x2 ≥ 0 and integer
Can we update
UB and/or LB?
(2,1) Z1 = 20
Set the bounds
UB = 20
LB = 17.6
Minimization Example 1
Min z= 6x1 +8x2
s.t.
3x1 +x2 ≥4
x1 +2x2 ≥4
x2 ≥ 2
x1, x2 ≥ 0 and integer
LB = 17.6
Can we conclude?
x2  1
(0.666,2) Z2 = 20
If we keep branching (x1), any
solution would be more than Z2
UB = 20
LB = 17.6
x2  2
Z2 = 20
Minimization Example 1
If we keep branching
Min z= 6x1 +8x2
s.t.
3x1 +x2 ≥4
x1 +2x2 ≥4
x2 ≥ 2
x1 = 0
x1, x2 ≥ 0 and integer
(0,4) Z4 = 32
Min z= 6x1 +8x2
s.t.
3x1 +x2 ≥4
x1 +2x2 ≥4
x2 ≥ 2
x1 ≥ 1
x1, x2 ≥ 0 and integer
(1,2) Z5 = 22
Minimization Example 2
Min z= 5x1 +4x2
s.t.
3x1 + 2x2 ≥ 5
2x1 +3x2 ≥ 7
x1, x2 ≥ 0 and integer
LP0, LP1 and LP 2 (Branching on x1)
LP3 and LP4 (Branching on x2, from LP1)
LP5 and LP6 (Branching on x1, from LP4)
Suggested Questions
 Formulation

12.1- 6; 12.3-1; 12.3-5; 12.4-5; ; 12.4-6;
 Branch and Bound

12.5-3; 12.6-1; 12.6-6
More Example 1
Solve the following problems using the branch and bound method.
Max
Subject to:
Z = 5 X1 + 8 X2
X1 + X2 ≤ 6
5X1 + 9 X2 ≤ 45
X1, X2  0, and integer
More Example 1
More Example 1
Example More Example 2