6-1 Chapter 6 Integer Programming Section 6.1 Introduction to Integer Programming Integer programming problem: a linear programming problem in which the domains of all (or some) of the variables are restricted to be integral. For example 1, Transportation problem The units to be shipped are automobiles, refrigerators and so on. 2, Production problem The units to be produced are homes, swimming pools and so on. Difficulties A most obvious possible solution technique: Initially ignore the integral restriction, solve the linear programming problem first, then round off the calculated solution to integral values. This method fails in the following example. Example Maximize f ( x1 , x2 ) 3x1 13x2 Subject to 2 x1 9 x2 40 11x1 8 x2 82 x1 , x2 0 and integral Ignoring the integral restriction, we apply the simplex algorithm. 6-2 Table 6.1 x3 x4 x2 x4 x1 x2 x3 x4 2 11 -3 2 9 9 -8 -13 1 0 0 1 9 8 9 13 9 11 115 8 115 167 115 0 1 0 1 115 9 0 1 9 0 x2 0 1 x1 1 0 0 0 0 1 0 2 115 9 115 1 115 Slack variables: x 3 , x4 40 82 0 4 4 9 5 9 7 57 9 2 2 5 1 9 5 4 58 5 117 4 Maximum value of f for the simple linear programming problem = 58 , 5 1 2 attained at (9 , 2 ) 5 5 round off to (9, 2), (10, 2), (10, 3) or (9, 3). all are not feasible The simplex algorithm has provided us with no useful information. optimal solution infeasible 6-3 For the problem with integral restriction, the actual solution (only two variables) can be obtained graphically (see Figure 6.1). All of feasible solutions: 36 lattice points (both coordinates are integrals). The coefficients (in f): 3 > 0, 13 > 0 possible optimal solution points: (2,4), (6,3), (8,2) f (2,4) = 58 f (6,3) = 57 f (8,2) = 50 optimal solution point = (2,4). Problem Set 6.1: 2; 4. 6-4 Section 6.2 Models with Integer Programming Formulations The Allocation of Resource in Discrete Amounts Problem: to utilize a given amount of a resource in a way that maximizes profit or gain. Supposition: to each of alternative methods of utilizing the resource, only multiples of fixed sized lots can be allocated. Integrally restricted variables have to be introduced to represent the number of lots. Example 1 An investment firm has $100,000 to invest: A construction project Portfolio of stocks Shares of a stock Question: Investment $48,000 $19,000/unit $1750/share How would the firm invest to maximize profit? Variables: 1, if the constructi on project is invested x1 0, if not x2 number of units of the portfolio bought x3 number of shares of the stock bought Integer programming problem: 2900 x1 1100 x2 95x3 Maximize Subject to 48,000 x1 19,000 x2 1,750 x3 100,000 0 x1 1, x2 , x3 0 and x1, x2 , x3 integral . Profit $2,900 $1,100/unit $95/share 6-5 Example 2 (cargo loading problem) An airplane can carry up to w lb of cargo. There are n different items: Item i: weight ai lb, profit c i dollars. Question: What items should be shipped to maximize profit? Variables 1, if the ith item is to be shipped xi 0, if not Integer programming problem n c x Maximize: i 1 i i Subject to: n a x i 1 i i w 0 xi 1 and integral, i 1,..., n . Two more restrictions: 1. Item 2 cannot be shipped unless Item 1 is also shipped. Add a constraint: x2 x1 . 2. Item 3 and Item 4 cannot be shipped together. Add a constraint: x3 x4 1 . Modification of the Objective Function An example – The Fixed Charge Problem In an operation, there is a fixed charge or cost for a process only if that process is used. For Example If A machine is to be used A new market is to be opened Warehouses are to be used there could be A setup cost An initial market development cost A rental cost To formulate models, introduce for each fixed charge 1, if the fixed charge is to be assessed an auxiliary variable= 0, if not Then (the fixed charge)(the auxiliary variable) is added to the objective function,and an inequality is added to the constraints in order to maintain the “on or off” role of the auxiliary variable. 6-6 Example 3 A machine shop has three processes to produce two parts. There are weekly limits on raw material, and labor; weekly fixed demands for two parts; operation cost (per hour) for Process i : c i ; setup and maintenance charge (fixed charge) d i for Process i if Process i is utilized. Question: What is the weekly operation to minimize the total cost which meets the weekly demands? Variables: xi = number of hours per week that Process i is used. Auxiliary variables: 0, xi 0 yi 1, xi 0 Objective function: c1 x1 c2 x2 c3 x3 d1 y1 d 2 y2 d 3 y3 Constraints for restrictions on raw material and labor, impose “ ” type inequalities in the xi ' s ; for weekly demands for two parts, impose “ ” type inequalities in the xi ' s ; in order to regulate the yi ' s , suppose xi 40 , estimate an obvious upper bound for xi then we add to constraints: xi 40 0 yi 1 and integral, i 1,2,3 yi Note If Process i is utilized in the optimal operation x i 0 yi 1 . 40 If Process i is not used yi is permitted to be 0 or 1. Minimization of the objective function yi = 0 the “on or off” role of yi is achieved. 6-7 Example 4 A distributor supplies n retail outlets, and he may consider to use some warehouses. Known data: d j (units) = monthly requirement of Outlet j s i (units) = storage capacity of Warehouse i ri (dollars) = monthly rental fee of Warehouse i cij (dollars) = cost to ship 1 unit from Warehouse i to Outlet j (it is known that the total capacity of warehouses is larger than the total demand of retail outlets.) Question: What warehouse should be utilized to minimize the total cost? Variables: x ij = number of units shipped monthly from Warehouse i to Outlet j. Auxiliary variables: 0, warehouse i is not utilized yi 1, warehouse i is utilized And we know that warehouse i is used if and only if x ij 0. j Integer programming problem: Minimize: cij xij ri yi i, j i Subject to: x ij si , i 1,..., m j x ij i d j , j 1,..., n x ij yi j si , i 1,..., m 0 xij , 0 yi 1 and integral. (1) Example 4 (continued) In Example 4, if both Warehouses 1 and 2 are utilized, then the monthly rent can be reduced by $R. Introduce another “on or off” variable 0, y1 y2 2 z 1, y1 y2 2 Then -Rz is added to the objective function 6-8 y1 y2 2 0 z 1 and integral z are added to the constraints. Constraints. Example 5a A company produces products A&B A B M(material) (units) 5 8 Machine Time (min) 2 4 weekly available 110min of machine time at no cost weekly purchase price of material M $2/unit for 1st 150 units $3/unit for additional 100 units Problem: Formulate a model for maximizing profit. Variables: x1 = number of A’s to be produced x2 = number of B’s to be produced x3 = number of units of M purchased at $2/unit x4 = number of units of M purchased at $3/unit Linear programming problem 25 x1 45 x2 2 x3 3x4 Maximize: Subject to: 5 x1 8 x2 x3 x4 2 x1 4 x2 110 x3 150 x4 100 x1 , x2 , x3 , x4 0 Note (2) Here we do not use auxiliary discrete variables. The purchase price for the first 150 units is cheaper in any optimal solution, x4 0 only if x3 150 . Selling Price (dollars) 25 45 6-9 Example 5b The purchase price of M is changed to be $2/unit for 1st 150 units $1.50/unit for next 100 units Variables: x1 , x2 , x3 as before x4 = number of units of M purchased at $1.50/unit Objective function in (2) is changed to be 25x1 45x2 2 x3 1.50 x4 In order to force the use of the first 150 units of M before the use of the less expensive last 100 units. Introduce an auxiliary variable 0, x3 150 (cannot use the cheap material) y1 1, x3 150 (can use the cheap material) and x4 0 only if y1 1 . Integer programming problem: 25x1 45x2 2 x3 1.5x4 Maximize: Subject to: 5 x1 2 x1 8 x2 4 x2 x3 y1 x4 x3 x 4 110 150 x3 / 150 100 y1 x4 0 only if y1 1 x1 , x2 , x3 , x4 0 0 y1 1 and integral only if x3 150 Example 5c Suppose the amount of M is unlimited, and the price at $2/unit for 1st 150 units $1.50/unit for next 100 units $1/unit for any number of units purchased over the first 250 units Variables: x1 , x2 as before x3 = number of units of M purchased at $2/unit x4 = number of units of M purchased at $1.5/unit x5 = number of units of M purchased at $1/unit 6-10 Auxiliary variables: 0, y1 1, x3 150 x3 150 0, y2 1, , x4 100 x4 100 Integer programming problem: Maximize: 25 x1 45 x2 2 x3 1.5 x4 1x5 Subject to: 5 x1 8 x2 x3 x4 x5 2 x1 4 x2 110 x3 150 y1 x3 / 150, y 2 x4 / 100 x4 100 y1 , x5 245 y 2 0 x1 , x2 , x3 , x4 , x5 0 y1 , y 2 1 and integral Note Induction of the constraint: x5 245y2 We need to have a upper bound for x5 . 1st constraint x5 5 x1 8 x2 x3 x4 nd 2 constraint x1 55, x2 27.5 x1 , x2 0 x5 5(55) 8(27.5) x3 495 x3 On the other hand x5 0 only if x3 150 and x4 100 x5 495 150 100 245 Notice y2 1 only if x4 100 then the constraint x5 245y2 allows x5 to be a positive value ( 245 ). x4 x4 245 is an upper bound of x5 6-11 The Problem of Alternative Constraints Situation: variables must satisfy one (or more) set (sets) from two (or more) alternative sets of constraints For example, In the manufacture of a certain product the bidding for various contracts the utilization of some resources Possibilities to produce either none or an amount exceeding some minimal batch size the bids are either 0 or some lower bound One of several methods may be employed, each method corresponds to a system of constraints reflecting the limited supplies and required demands To formulate models, introduce auxiliary variables equal to either 0 or 1, use these variables to modify the system of constraints. Example 6 Suppose a variable xi must be either 0 or bi (lower bound). Introduce an “on or off” variable yi , and ask that xi 0 if yi 0, and xi bi if yi 1 . This requirement can be realized by adding constraints: xi mi yi Suppose in any feasible solution xi mi xi bi yi 0 yi 1 and integral Note yi can equal only 0 or 1. If yi 0 xi 0 xi is properly restricted If yi 1 xi bi (when yi 1 , the 1st inequality xi mi has no significance) In fact if only one variable, say x1 , has such restriction, then we may not need to introduce the on and off variable y1 , because we can first solve the problem with the request x1 0 , then solve the problem again by replacing x1 0 with the constraint x1 b1 , and at last compare the optimal values of the two cases and choose the better solution. However, if many variables, say x1 , x2 ,..., x10 , have such restriction, then we should introduce 0-1 variables y1 , y2 ,..., y10 by the above method, for otherwise there would be 210 1024 possible cases to compare. Problem Set 6.2 : 6; 11. 6-12 Section 6.3 Gomory’s Cutting Plane Algorithm Pure integer programming problem: a linear programming problem with integral restrictions on all of the variables. Fundamental idea underlying Gemory’s Cutting Plane Algorithm: add one constraint to the problem at a time if in the solution some variables do not have integer value; eventually have a linear programming problem with an optimal solution with integral coordinates. Process of the algorithm: First, solve the original linear programming problem ignoring the integral restrictions. Second, if this solution has all integral coordinates, it is also an optimal solution to the integer programming problem. If not, generate a new constraint with two fundamental properties: (1) (2) the non-integral optimal solution will not satisfy this constraint. all integral feasible solution will satisfy this constraint. Note The new constraint essentially cuts off a subset of the feasible region for the case of continuous variables. Contains no feasible integral solutions Then, add the new constraint to the problem and proceed to solve the expanded problem as before. (again ignore the integral requirement) Example 1 Minimize: x1 3x2 Subject to: x1 x2 2 2 x1 4 x2 15 x1 , x2 0 and integral Introduce integrally restricted slack variables: x3 , x4 . Use the simplex algorithm to solve the problem ignoring the integral restrictions. 6-13 Table 6.2 x3 x4 x3 x2 x1 x2 x3 x4 1 2 1 3 2 1 2 5 2 -1 4 -3 1 0 0 0 1 1 0 0 0 0 1 0 1 4 1 4 3 4 2 15 0 23 4 15 4 45 4 Minimal value is attained at 15 3 3 x1 0, x2 3 , ( x3 5 , x4 0) 4 4 4 Non-integral Generate a new constraint: Select a constraint equation with non-integral constant term, say the first one: 3 1 23 x1 x3 x4 . 2 4 4 Separate all constants into their integral and fractional parts: 1 1 3 (1 ) x1 x3 x4 5 2 4 4 1 1 3 x1 x4 5 x1 x3 2 4 4 For integral solutions, the right side and therefore the left side must be integral. Notice that all variables 0 1 1 3 3 x1 x4 2 4 4 4 1 1 3 x1 x4 0 and integral. 2 4 4 Let this be the new constraint, and change it to an equation by introducing a slack variable x5 , i.e., 1 1 3 x5 x1 x4 , 2 4 4 then x5 0 and integral. 6-14 The expanded problem is: 5 3 45 z x1 x4 Minimize: 2 4 4 Subject to 3 1 x1 x3 x4 2 4 1 1 x1 x2 x4 2 4 1 1 x1 x4 x5 2 4 x1 , x2 , x3 , x4 , x5 0 and integral 23 4 15 4 3 4 former optimal solution 15 23 x1 x4 0, x2 , x3 4 4 does not satisfy Table 6.3 x3 x2 x5 x3 x2 x4 x1 3 2 1 2 1 2 5 2 1 0 2 1 x2 x3 0 1 1 0 0 0 0 0 0 1 0 0 1 0 0 0 x5 x4 1 4 1 4 1 4 3 4 0 0 1 0 0 0 1 0 1 1 -4 3 23 4 15 4 3 4 45 4 5 3 3 9 Proceed as above: Solve the expanded problem ignoring integral restriction with the Dual Simplex Algorithm (Table 6.3). Multiplying the new constraint by (-1), the basic variables are x3 , x2 , x5 . After one pivot operation, we obtain minimal value = -9, attained at x1 0, x2 3 . Geometrically, x2 integral solution x1 6-15 The added constraint 2 x1 x4 3 0 1 1 3 x1 x4 0 2 4 4 x4 15 2 x1 4 x2 (2nd row in Table 6.2) 2 x1 (15 2 x1 4 x2 ) 3 4 x2 12 x2 3. The new constraint x2 3 cuts off 3 (0, 3 ) the original non-integral optimal solution 4 from the feasible set, but does not exclude any feasible integral points. We now consider general case. Generation of new constraints: Suppose bi is not integral. The ith constraint is a x ij j bi (1) j Let [a] be the greatest integer in a (i.e., not exceeding a), then a-[a] is the fractional part of a: 0 a [a] 1 . 6-16 Denote (1) f ij aij [aij ] f i bi [bi ] [a ij ] f ij x j bi f i j or f ij x j f i bi [aij ]x j j (2) j For any integral solution to the original constraints the right side of (2) is integral. So, the left side of (2) is also integral. Notice that all x j 0, 0 f ij 1, and 0 f i 1 , then any integral feasible solution must satisfy: fij x j f i f i 1 j f ij x j f i 0 and integral (3) j ~this is the new constraint Note 1 Note 2 Note 3 Adding (3), we still have a pure integer programming problem. Any feasible integral solution to the original constraints satisfies (3). The optimal basic feasible solution from the final tableau does not satisfy (3). In fact f i 0 (since bi is not integral). For any basic variable x j , in column j, aij 0 or 1, hence f ij 0 ; For any nonbasic variable x j , x j 0 ; f ij x j fi fi 0 . j Note 4 If several of bi are not integral, in practice, choosing bi with the largest fractional value is easy to apply and usually quite effective. Gomory’s Cutting Plane Algorithm for solving the pure integer programming problem Minimize c x (or Maximize c x ) Subject to Ax b x 0 and integral Algorithm 1. Solve the linear programming problem ignoring the integral restriction on x. If this solution has all integral coordinates it is an optimal solution to the original problem. 6-17 2. Otherwise a new constraint is added to the problem. a. Select a non-integral bi with the largest fractional value. n b. Suppose bi is selected and the corresponding equation is a x j 1 f ij x j x f i ij j bi j f ij aij [aij ] , j 1,2,..., n where f i bi bi x is a new slack variable, x 0 and integral. Add this constraint to the problem and return to step 1. Use the Dual Simplex Algorithm to solve the corresponding linear programming problem. c. Example 2 Minimize: x1 2x2 Subject to: 2 x1 x2 5 4 x1 4 x2 5 x1 , x2 0 and integral Integrally restricted slack variables: x3 , x4 Apply the Simplex Algorithm Table 6.4 x1 x2 x3 x4 x4 2 -4 1 1 4 ○ -2 1 0 0 0 1 0 x3 3 0 1 x2 -1 1 0 -1 0 0 x1 1 0 x2 0 1 0 0 x3 Add a new constraint 1 1 1 x3 x4 x5 . 3 6 2 slack variable 1 3 1 3 1 3 5 5 0 1 4 15 4 1 4 1 2 1 12 1 6 5 12 5 4 5 2 5 4 5 2 15 4 Non-integral solution (select the second row) 6-18 Apply the Dual Simplex Algorithm detailed steps: 1 1 5 2nd equation : 1x2 x3 x4 3 6 2 1 1 1 fractional 0 x2 x3 x4 0 3 6 2 part of the 1 1 1 coefficients x3 x4 x5 , 3 6 2 & integral x5 0, integral Table 6.5 x3 1 3 1 3 1 3 1 3 x1 x2 x1 1 0 x2 0 1 x5 0 0 0 0 x1 1 0 0 x2 0 1 0 x3 0 0 1 0 0 0 Add a new constraint (select the first row) 3 3 x4 x6 4 4 slack variable 1 4 1 1 1 4 4 3 4 Apply the Dual Simplex Algorithm. x4 1 12 1 6 1 6 5 12 1 4 0 1 2 1 4 x5 0 0 1 0 1 1 -3 1 5 4 5 2 1 2 15 4 3 4 2 3 2 13 4 Non-integral solution 6-19 Table 6.6 x1 x2 x3 x1 1 0 0 x2 0 1 0 x.3 0 0 1 x6 0 0 0 0 0 0 x1 1 0 x2 0 x3 x4 x5 x6 1 0 1 0 -3 0 0 1 1 0 x4 1 4 0 1 2 3 4 1 4 0 0 1 1 0 0 1 0 0 1 0 -3 0 0 0 1 0 0 0 0 0 1 Minimal value = -3 attained at x1 1, x2 2 1 3 0 2 3 4 3 1 3 3 4 2 3 2 3 4 13 4 1 2 1 1 3 Optimal solution to the original problem all are integers Geometrically 1st added constraint 1 1 1 x3 x4 0 3 6 2 1 1 1 (5 2 x1 x2 ) (5 4 x1 4 x2 ) 0 3 6 2 x2 2 second point first point optimal integral solution (third point) first cutting plane second cutting plane 6-20 2nd added constraint 3 3 x4 0 4 4 3 3 (5 4 x1 4 x2 ) 0 4 4 x1 x2 1 Problem Set 6.3 : 1; 4; 5. 6-21 Section 6.4 A Branch and Bound Algorithm Basic process of the Branch and Bound Algorithm (for solving integer programming problem): First, solve the problem ignoring the integral restriction. If there is no feasible solution, the original problem also has no feasible solution. Stop. If the optimal solution is integral, it is also optimal for the original problem. Stop. If the optimal solution has non-integral coordinates, then proceed the following iterations: 1. Generate two new problems (branching process): each is obtained by adding a new constraint to the previous problem. arbitrarily select a variable, say x j , x j bi non-integral two new constraints: x j [bi ] x j [bi ] 1 Note 1 Note 2 Note 3 2. The previous non-integral optimal solution is not feasible for both new problems. Any previous integral feasible solution is feasible for one of these two new problems. The integral optimal solution to the previous problem could be contained in either problem. Solve the new problems ignoring the integral restriction by using the Dual Simplex Algorithm. If either problem has no feasible solution eliminate from considering this problem. If either problem has an integral optimal solution this solution is feasible for the original integer programming problem, and it provides a bound for the optimal value of the original problem (a lower bound for Max problem, an upper bound for Min problem). Eliminate from consideration any problem generated through the branching process with an optimal value no better than this bound. If there is no other problem generated through the branching process with optimal value better than this bound, 6-22 this solution is optimal for the original integer programming problem. 3. Go to the next iteration with the remaining problems. Notes 4 If two or more branches remain open, two frequently used rules to select the problem are (1) (2) the problem with the most favorable optimal value. the problem most recently generated. Example 1 Example 2, Section 6.3 Minimize: z x1 2x2 Subject to: 2 x1 x2 5 4 x1 4 x2 5 x1 , x2 0 and integral Ignoring the integral restriction, we have 3 Min z 3 4 1 1 attained at x1 1 , x2 2 4 2 Non-integral Min x1 2, x2 1 is an integral feasible solution to Problem 1 an upper bound for the optimal value of Problem 1 is 0 6-23 Arbitrarily select x1 and use x1 1 x1 2 to form two new problems. Use the Dual Simplex Algorithm to solve these two new problems ignoring the integral restrictions (see Figure 6.7). 1 Min z in problem 2 A 3 0 Min z in problem 2 B 2 better integral solutions may be contained in Problem 2A, branch again off Problem 2A using x2 , we get problems 3A and 3B: Continuing by branching at Problem 3A leads to Figure 6.9. 6-24 0 3 eliminate this problem 1 3 2 eliminate this problem 2 x1 1, x2 2 is an integral feasible solution upper bound = -3 the optimal value of the original integer programming problem = -3, attained at x1 1, x2 2 . Example 2 Maximize: Subject to: 10 x1 2 x1 z 8x1 15x2 21x2 x2 156 22 x1 , x2 0 and integral Use the Branch and Bound Algorithm to solve this problem. See Figure 6.10 6-25 Max 110 117 eliminate this problem lower bound 117 3 Max z in problem 3B 117 117 Max z in problem 3 A, but only by a fractional 5 number 3 value of z at any feasible solution to Problem 3B 117 . 5 As all coefficients of z are integrals value of z at any integral feasible solution to Problem 3B 117 . Algorithm terminates. The optimal value for the original integer programming problem = 117, attained at x1 9, x2 3 . Example 3 Maximize: Subject to: 2 x1 11x1 The example in Section 6.1 z 3x1 13x2 9 x2 8 x2 40 82 x1 , x2 0 and integral Use the Branch and Bound Algorithm to solve this problem. See Figure 6.11. 6-26 Max 1 8 52 2 11 Hold 2A in abeyance for future consideration 58 57 58 Eliminate 4A 8 Max z in problem 2 A 11 further consideration of problem 2A is unnecessary terminate at x1 2, x2 4 with optimal value =58. Max z in problem 4 B 58 52 Advantages of the Branch and Bound Algorithm 1. The algorithm generates feasible integral solutions as it proceeds, along with estimates on how close to the optimal value these solutions might be for a large and complicated problem, if we are unable to complete the computation (maybe because of limited computer capacities), the algorithm may still provide an integral feasible solution, which is not necessarily optimal but adequate for our purposes. 2. The method can be used to handle many different cases, for example it can solve some problems in which only a part of variables should have integral values, or some variables are 0-1 binary variables. 6-27 When we stop further branching at a particular problem? 1. the problem has no feasible solution; 2. the problem has an optimal integral solution; 3. the bounds so far we obtained show that this problem cannot contain the optimal solution of the original problem. Problem Set 6.4: 1(b), (d), (e). (use LINGO to solve each sub-problem)