Water Resources Development and Management
Optimization
(Integer Programming)
CVEN 5393
Mar 11, 2013
Acknowledgements
• Dr. Yicheng Wang (Visiting
Researcher, CADSWES during Fall
2009 – early Spring 2010) for
slides from his Optimization
course during Fall 2009
• Introduction to Operations
Research by Hillier and Lieberman,
McGraw Hill
Today’s Lecture
• Integer Programming
– Examples
• R-resources / demonstration
INTEGER PROGRAMMING
Integer Programming
In many practical problems, the decision variables actually make sense only
if they have integer values.
If some or all of the decision variables in a linear programming formulation
are required to have integer values, then it is an Integer Programming (IP)
problem.
The mathematical model for integer programming is the linear programming
model with one additional restriction that some or all of the decision
variables must have integer values.
If only some of the decision variables are required to have integer values,
then the model is called Mixed Integer Programming (MIP) problem.
If all of the decision variables are required to have integer values, then the
model is called Pure Integer Programming problem.
In some decision-making problems, the only two possible choices for decisions are
yes and no.
1. Examples of Integer Programming
(1) Example of BIP
All the decision variables have the binary form
x1: building a factory in Los Angeles? Yes (x1=1) or No (x1=0)
x2: building a factory in San Francisco? Yes (x2=1) or No (x2=0)
x3: building a warehouse in Los Angeles? Yes (x3=1) or No (x3=0)
x4: building a warehouse in San Francisco? Yes (x4=1) or No (x4=0)
Because the last two decisions represent mutually exclusive alternatives (the
company wants at most one new warehouse), we need the constraint
Furthermore, decisions 3 and 4 are contingent decisions, because they are
contingent on decisions 1 and 2, respectively (the company would consider
building a warehouse in a city only if a new factory also were going there). Thus,
in the case of decision 3, we require that x3 = 0 if x1 = 0. This restriction on x3
(when x1 = 0) imposed by adding the constraint
Similarly, the requirement that x4 = 0 if x2 = 0 is imposed by adding the constraint
The complete BIP model for this problem is
(2) Example of MIP
2. Some Perspectives on Solving Integer Programming Problems
subject to
Question 1: Pure IP problems have a finite number of feasible solutions. Is it
possible to solve pure IP problems by exhaustive enumeration ?
Consider the simple case of BIP problems.
With n variables, there are 2n solutions to be considered. Each time n is
increased by 1, the number of solutions is doubled.
n=10, 1024 solutions
n=20, more than 1 million solutions
n=30; more than 1 billion solutions.
Question 2: Is IP easier to solve than LP
because IP tends to have much fewer feasible
solutions than LP ?
Question 3: Can we use the approximate
procedure of simply applying the simplex method
to get an LP optimal solution and then rounding
the noninteger values of the solution to integers ?
A
B
subject to
C
A
B
3. The Branch-and-Bound Technique and Its Application to BIP
The Branch-and-Bound Technique is the most popular mode for IP
algorithms.
The basic concept underlying the branch-and-bound technique is to
divide and conquer. The original problem is divided into smaller
and smaller subproblems until these subproblems can be conquered.
The dividing (branching) is done by partitioning the entire set of
feasible solutions into smaller and smaller subsets. The conquering
(fathoming) is done partially by bounding how good the best
solution in the subset can be and then discarding the subset if its
bound indicates that it cannot possibly contain an optimal solution
for the original problem.
California Manufacturing Co. Example
Branching
Original problem
Solution Tree
The variable used to do this branching
at any iteration by assigning values to
the variable is called the branching
variable
Bounding
LP relaxation of the whole problem
LP relaxation of subproblem 1
LP relaxation of subproblem 2
Fathoming
Three cases where a subproblem is conquered (fathomed).
(1) A subproblem is conquered if its LP relaxation has an integer optimal solution
(2) A subproblem is conquered if it is inferior to the current incumbent.
Since Z*=9, there is no reason to consider further any subproblem whose
bound ≤ 9, since such a subproblem cannot have a feasible solution better than
the incumbent. Stated more generally, a subproblem is fathomed whenever its
(3) If the simplex method finds that a subproblem’s LP relaxation has no
feasible solutions, then the subproblem itself must have no feasible solutions, so
it can be dismissed (fathomed).
Using the BIP Branch-and Bround Algorithm to
Solve the California Manufacturing Co. Example
(1) Initiliazation
Set Z*= − ∞. Solve the relaxation of the whole problem by the
simplex method. The optimal solution of the relaxation is
Subproblem 1
1)The bound of the whole problem is less than Z*.
2)The relaxation of the whole problem has feasible solution.
3)The optimal solution includes a noninteger value of x1.
So the whole problem can not be fathomed and should be divided
(branched) into subproblems.
Subproblem 2
(2) Iteration 1
Subproblem 1 with x1=0. The optimal solution of its relaxation is (0,1,0,1) with Z=9.
The optimal solution is integer, which is the best feasible solution found so far. So this
integer solution with Z*=9 is stored as the first incumbent. Since the optimal solution of
its relaxation is integer, subproblem 1 is fathomed
Subproblem 2 with x1=1. Because subproblem 2 is not fathomed, it should be divided
into subproblems.
(3) Iteration 2
The only remaining subproblem corresponds to the x1=1 node, so we shall branch from
this node to create the two new subproblems.
Subproblem 3 with x1=1, x2=0.
LP relaxation of subproblem 3
LP relaxation of subproblem 4
Bound for subproblem 3 :
Bound for subproblem 4 :
Subproblem 4 with x1=1, x2=1.
Subproblem 1
Subproblem 2
Subproblem 4
Subproblem 3
(4) Iteration 3
So far, the algorithm has created 4 subproblems. Subproblem 1 has been fathomed, and
subproblem 2 has been replaced by subproblems 3 and 4, but these last two remain under
consideration. Because they were created simultaneously, but subproblem 4 has the larger
bound, the next branching is done from subproblem 4, which creates the following new
subproblems
LP relaxation of subproblem 5 :
Subproblem 5 with x =1, x =1, x =0.
1
2
3
LP relaxation of subproblem 6 :
No feasible solutions
Bound for subproblem 5 : Z≤ 16
Subproblem 1
Subproblem 6 with x1=1, x2=1, x3=1.
Subproblem 3
Subproblem 5
Subproblem 2
Subproblem 4
Subproblem 6
(5) Iteration 4
The subproblems 3 and 5 corresponding to nodes (1,0) and (1,1,0) remain under
consideration. Since subproblem 5 was created most recently, so it is selected for
branching.
Since x4 is the last variable, fixing its value at either 0 or 1 actually creates a single
solution rather than subproblems. These single solutions are
(1,1,0,0) with Z=14 is better than
the incumbent with Z*=9, so
(1,1,0,0) with Z*=14 becomes the
new incumbent.
Because a new incumbent has been
found, we reapply fathoming test 1
with the new incumbent to the only
remaining subproblem 3.
Subproblem 1 Subproblem 3
Subproblem 2
Subproblem 5
Subproblem 4
Subproblem 6
Subproblem3:
There are no remaining (unfathomed) subproblems. Therefore, the optimality test
indicates that the current incumbent
is optimal.
4. The Branch-and-Bound Algorithm for MIP
An MIP Example
Change 1:The bounding step
BIP algorithm: With integer coefficients in the objective function of BIP, the value of Z for
the optimal solution for the subproblem’s LP relaxation is rounded down to obtain the
bound, because any feasible solution for the subproblem must have an integer Z.
MIP algorithm: With some of the variables not integer-restricted, the bound is the value of
Z without rounding down.
Change 2: The fathoming test
BIP algorithm: With a BIP problem, one of the fathoming tests is that the optimal solution
for the subproblem’s LP relaxation is integer, since this ensures that the solution is feasible,
and therefore optimal, for the subproblem..
MIP algorithm: With a MIP problem, the test requires only that the integer-restricted
variables be integer in the optimal solution for the subproblem’s LP relaxation, because this
suffices to ensure that the solution is feasible, and therefore optimal, for the subproblem..
Subproblem 1
Subproblem 2
Change 3:Choice of the branching variable
BIP algorithm: The next variable in the natural ordering x1, x2, …, xn is chosen
automatically.
MIP algorithm: The only variables considered are the integer-restricted variables that
have a noninteger value in the optimal solution for the LP relaxation of the current
subproblem. The rule for choosing among these variables is to select the first one in the
natural ordering.
Change 4:The values assigned to the branching variables
BIP algorithm: The binary variable is fixed at 0 and 1, respectively, for the 2 new
subproblems.
MIP algorithm: The general integer-restricted variable is given two ranges for the 2 new
subproblems.
Subproblem 3
Subproblem 1
Subproblem 2
Subproblem 4
Subproblem 3
Subproblem 5
Subproblem 6
Subproblem 5
Subproblem 3
Subproblem 1
Subproblem 6
Subproblem 4
Subproblem 2
INTEGER / MIXED-INTEGER
PROGRAMMING
(PROF. MCMAKEE NOTES)
INTRODUCTION & EXAMPLES
HUGHES-MCMAKEE-NOTES\CHAPTER06.PDF
Hughes-McMakee-notes\chapter-07.pdf