Linear Programming
Examples of linear programming problems: single source shortest paths, maximum flow,
minimum flow, multicommodity path.
For example:
A: amount of time spent on school work
B: amount of time spent on fun
C: amount of time spent on pay work
Maximize 24-A-B - C
given the constraints A+B + C  24, B +C 8 and A  0, B  0, C  0.
N
Linear function f(x1, x2, …, xN ) is a function in form f(x1, x2, …, xN) =
 ax
i i
i=1
Linear constraints are linear inequalities and/or linear equalities. For example, a-b  4.
We saw them in ch.24, when we were solving the difference constraint problems.
A linear programming problem is the problem of either minimizing or maximizing a
linear function subject to a set of linear constraints. The function that we are trying to
maximize or minimize is called objective function. A set of any values of variables that
satisfies all constraints of a linear program is called a feasible solution is. Since there are
many values possible, all solutions span a feasible region.
For example, this is a linear programming problem:
Maximize
a+b
//this is objective function
Subject to:
// these are the constraints
4a – b  8
2a + b  10
5a – 2b  -2
a, b  0
All solutions to this set of inequalities lie in the area bounded by the above inequalities.
(Draw it). The optimal solution is the highest point of that area that intersects with the
line a+b=z. The value of z should be as high as possible.
When there are more variables, then each inequality forms a shape in space. The
intersection of the area bounded by those spaces is called a simplex. The objective
function is now a multi-dimensional plane The intersection of the simplex and the
objective function is again through multiple points, however, one point has the highest
value for the objective function. The simplex algorithm starts with some feasible
solution and optimizes it until it finds the highest value of the objective function. In this
case, that local maximum is also the global maximum.
Simplex algorithm requires that the linear programming problem be set up in slack form.
Linear program can be converted from any form into standard form, and then into slack
form.
Linear programming problem in standard form is represented as:
N
Maximize
 cx
//objective function
j j
j=1
N
Given the constraints
ax b
ij j
i
for i = 1, 2, …, m
j=1
xj  0
for j = 1, …, N
//non-negativity constraint
Converting any linear program into standard form:
1. if the objective function is minimization, just “negate” the objective function.
For example. “minimize –a +b subject to a-b  2, a, b  0 ” is equivalent to
“maximize a-b subject to a-b  2, a, b  0 ”.
2. If a variable does not have non-negativity constraints (i.e. it could be negative
too), convert it into a difference of two positive variables.
For example, if variable x does not have non-negativity constraint, then express it
as x = y – z, where y, z  0.
3. If there is an equality in the constraints, replace it by two inequalities. Equality
x=y is replaced by two inequalities: x  y and x  y.
4. If some constraints are  rather than , multiply the entire inequality by –1.
a x b
ij j
i
is equivalent to
 -a x  -b
ij j
i
Linear program in slack form uses only equalities for constraints, except for the
non-negativity constraints.
Converting linear program in standard form into linear program in slack
form:
N
Each constraint
N
a x b
ij j
j=1
i
is represented as xN+i= bi
- a x
ij j
j=1
and xN+i  0.
xN+i are basic variables, or slack variables. The original set of xi are non-basic
variables.
29.1-4 Convert to standard form:
Minimize
2a + 7b
Subject to
a=7
3a + b  24
b0
c0
a  7 and a  7
-3a – b  -24
b0
-c  0 c= - f, f  0
a = d-e, d, e  0
Maximize 2e–2d-7b
d – e  7, d-e  7
-3d + 3e – b  -24
b, d, e,f  0
Maximize 2e-2d -7b
d–e7
-d + e  -7
-3d + 3e – b  -24
b, d, e,f  0
29.1-5 Convert to slack form:
Maximize
2a – 6c
Subject to
a+b-c  7
3a –b  8
-a + 2b + 2c  0
a,b,c  0
Maximize
Subject to:
2a – 6c
d = 7 – a – b +c
-3a + b  -8  e = -8 +3a -b
f = 0 – a + 2b + 2c
a,b,c, d, e,f  0
The Simplex Algorithm
The basic solution to a linear program is obtained by setting all non-basic variables to 0
and calculating basic variables.
Simplex(L) {
while the objective value for the basic solution can be increased {
reformulate lin.prog. so that the basic solution has greater objective value
}
return the basic solution
}
Simplex(L) {
while there are variables to pivot {
pivot
}
return the basic solution
}
//assume that L is a linear program in slack form
Simplex(L) {
//while there are variables to pivot
while objective function has positive coefficients{
//pivot
//select non-basic variable x whose coefficient in the
objective function is positive
//increase its value without violating any constraints
find basic solution, find how much we can increase,
pick the equation that allows that increase
//”swap” x with another basic variable: set x as basic, and
make the other variable non-basic
//rewrite all constraints with x as basic
}
return the basic solution
}
Example p.791:
Maximize z = 3a + b + 2c
Subject to: 1.
a + b + 3c  30
2.
2a + 2b + 5c  24
3.
4a + b + 2c  36
4.
a,b,c  0
In slack form:
Maximize z = 3a + b + 2c
Subject to: d = 30 – a – b - 3c
e = 24 - 2a - 2b - 5c
f = 36 - 4a - b - 2c
a,b,c, d,e,f  0
The basic solution is a = 0, b = 0, c = 0, d = 30, e = 24, f = 36, i.e. (0,0,0, 30,24,36). z = 0.
1. Pivot using a.
When we substitute the basic solution into inequality 1, a  30. From 2., a  24/2
= 12. From 3., a  9. Therefore, we can increase a only by 9. So we swap a and f by
rewriting 4. as a = 9 – b/4 – c/2 – f/4 and substituting it into all other constraints.
Maximize z = 3(9-b/4 – c/2 –f/4) + b + 2c
Subject to: d = 30 – (9-b/4 – c/2 –f/4) – b - 3c
e = 24 - 2(9-b/4 – c/2 –f/4) - 2b - 5c
a = 9 -b/4 – c/2 –f/4
a,b,c, d,e,f  0
Maximize z = 27 + b/4 + c/2 – 3f/4
Subject to: 1.
a = 9 -b/4 – c/2 –f/4
2.
d = 21 – 3b/4 – 5c/2 + f/4
3.
4.
e = 6 - 3b/2 – 4c + f/2
a,b,c, d,e,f  0
The basic solution is b = 0, c = 0, f=0, a = 9, d = 21, e = 6 i.e. (9,0,0, 21, 6,0). z = 27.
1. Pivot using b or c. Pick c.
Constraint 1 limits c to 18, 2 limits c to 42/5, and 3 limits it to 3/2. Therefore, rewrite 3
and swap e and c.
Maximize z = 111/4 + b/16 – e/8 – 11f/16
Subject to: 1.
a = 33/4 – b/16 – e/8 – 5f/16
2.
c = 3/2 – 3b/8 – e/4 + f/8
3.
d = 69/4 + 3b/16 + 5e/8 – f/16
4.
a,b,c, d,e,f  0
The basic solution is b = 0, e = 0, f=0, a = 33/4, c = 3/2, d=69/4 i.e. (33/4,0,3/2, 69/4, 0).
z = 111/4.
1. Pivot using b.
The three constraints give bounds of 132,4, and infinity (because d increases as b
increases). Therefore rewrite 2 and swap b and c.
Maximize z = 28 -c/6 – e/6 – 2f/3
Subject to: 1.
a = 8 + c/6 + e/6 – f/3
2.
c = 4 – 8c/3 – 2e/3 + f/3
3.
d = 18 – c/2 + e/2
4.
a,b,c, d,e,f  0
The basic solution is (8,4,0,18,0,0). z = 28.
Since the objective function has no non-negative coefficients, this is the optimal solution.
Issues:
1. how do we determine if a linear program is feasible
2. what if the linear program is feasible but the initial basic solution is not feasible
3. how do we determine if a linear program is unbounded (i.e. optimal objective
value is infinity)
4. how do we chose which variables to swap.
Some answers:
1. Determine if a linear program is feasible by formulating an auxiliary linear
program.
Example from the book:
Maximize 2a – b
Subject to 2a – b  2
a – 5b  -4
a,b  0
The initial basic solution is a=b=0, but it violates the second constraint, so it is not a
feasible solution.
N
The original linear problem is in form:
a x b
ij j
i
, all xj  0.
j=1
Formulate an auxiliary linear problem:
Maximize
-x0
Subject to:
N
 aijxj
- x0  bi and all xj  0
for all j
j=1
The original linear problem is feasible iff the optimal objective value of the auxiliary
problem is 0.
Continuing the example:
Maximize –x0
Subject to 2a – b -x0  2
a – 5b -x0  -4
a,b, x0  0
z = –x0
Subject to c = 2 - 2a + b + x0
d = -4 - a + 5b + x0
a,b, x0, c, d  0
Basic solution: x0=a=b=0, c=2, d=-4 which is not feasible.
2. find an initial basic solution that is feasible by swapping x0 with the basic variable
whose value in the basic solution is most negative.
In this case, d:
z = -4 –a + 5b – d
x0 = 4 +a -5b + d
c = 6 – a – 4b + d
a,b, x0, c, d  0
Basic solution: a=b=d=0, z=4, c=6, which is feasible. So, keep on running PIVOT until
the optimal solution to the auxiliary problem is found.
Swap x0 and b:
z = -x0
b = 4/5 – x0/5 + a/5 + d/5
c = 14/5 + 4x0/5 – 9a/5 + d/5
a,b, x0, c, d  0
No more swaps, so this is the optimal solution, z=0. Therefore, the initial linear problem
was feasible.
So, now go back to the original linear problem, but using only non-basic variables from
the auxiliary problem:
z = 2a – b where b = 4/5 – x0/5 + a/5 + d/5 and x0=0
Subject to 2a – b  2
a – 5b  -4
a,b  0
So now the problem becomes:
z = 4/5 + 9a/5 – d/5
b = 4/5 – x0/5 + a/5
c = 14/5 -9a/5 + d/5
a,b, x0, c, d  0
The basic solution is a=d =0, b=4/5, c=14/5, z=4/5, which is feasible, so then keep on
running the SIMPLEX.
The book continues with this example for one more iteration, which has a feasible basic
solution.
The next iteration does not have a feasible basic solution, so the whole process of
formulating an auxiliary problem has to be repeated…
If we draw the problem, we can see that there is no finite solution anyways:
Maximize
2a – b
Subject to
2a – b  2
a
– 5b  -4
a,b  0
a 2a-b=2
0
1
2
3
4
5
6
7
8
10
20
The acceptable region is above or on the blue line
the acceptable region is above or on the purple line
and only in the first quadrant
a-5b=-4
-2
0.8
0
1
2
1.2
4
1.4
6
1.6
8
1.8
10
2
12
2.2
14
2.4
18
2.8
38
4.8
Therefore, there is no finite solution to this problem. The solution is on the blue line as the blue line reaches infinity.
Why? Because there are only two inequalities, and then do not define an area that is bounded.
If the second constraint was such that the lines formed a bounded area, then there would be a finite optimal solution.
ANOTHER EXAMPLE:
Solve the following linear program using SIMPLEX algorithm:
Minimize z = a + b + c
Subject to:
1.
2.
3.
4.
a-b-c 0
a+b+c≥4
a+b-c=2
a, b  0
How will you check if your solution is correct? Do so.
First, convert to standard form:
Maximize z = -a - b - c
Subject to:
1.
2.
3.
4.
5.
6.
a-b-c 0
-a - b - c  -4
a+b-c2
-a-b+c  -2
a, b  0
c = d-f
d, e  0
So, then problem becomes:
Maximize z = -a - b – d + f
Subject to: 1.
a-b–d+f 0
2.
-a - b – d + f  -4
3.
a+b–d+f2
4.
-a-b+ d - f  -2
5.
a, b, d, f  0
highlighting minuses hard to see
In slack form:
Maximize z = -a - b – d + f
Subject to: 1.
g = -a + b + d - f
2.
h = -4 + a + b + d - f
3.
j=2-a-b+d-f
4.
k = -2 + a + b - d + f
5.
a, b, d, f , g, h, j, k  0
Basic solution: a=0, b=0, d=0, f=0, z=0; g=0, h=-4, j=2, k=-2
This is not possible, because all variables have to be non-negative. So, we
cannot do anything starting with the basic solution of all 0s. We would
have to start with another basic solution.
So, this is as far as we would get with the high-level pseudocode of Simplex
algorithm. In order to fully solve it, we would have to actually trace the
pseudocode in detail.
Tracing the SIMPLEX algorithm from the book:
Maximize –x0
Subject to: 1.
2.
3.
4.
5.
In slack form:
Maximize z = -x0
Subject to: 1.
a - b – d + f –x0  0
-a - b – d + f -x0  -4
a + b – d + f –x0  2
-a-b+ d - f – x0  -2
a, b, d, f, x0  0
g = -a + b + d - f +x0
2.
3.
4.
5.
h = -4 + a + b + d - f + x0
j = 2 - a - b + d - f + x0
k = -2 + a + b - d + f + x0
a, b, d, f , g, h, j, k, x0  0
Switch x0 and h:
x0 = 4-a-b-d+f +h
Basic solution: a=b=d=f=h=0, x0=4, g=4, j=2, k=2. This solution is feasible.
Maximize z = -4+a+b+d-f-h
Subject to: 1.
g = -a + b + d - f + 4-a-b-d+f +h = 4-2a +h
2.
x0 = 4-a-b-d+f +h
3.
j = 2 - a - b + d - f + 4-a-b-d+f +h = 6-2a –2b +h
4.
k = -2 + a + b - d + f + 4-a-b-d+f +h = 6 –2d +2f +h
5.
a, b, d, f , g, h, j, k, x0  0
Pivot on a and switch a and g:
Etc.
Questions and Answers:
>About the example from p.791: How are different iterations of “while” loop
>different?
>Assume the basic solution b=0,c=0,f=0,a=9,d=21,e=6,f=0.
>
>The naive approach to pivoting on c at this stage would involve plugging in
basic solution values to each equation, on _both_ the left and the right sides,
except for c, which we're trying to solve for. However, if you do this, you get
(for constraint 1):
>
>9 = 9 - 0/4 - c/2 -0/4, c = -2.
>
>If I did my arithmetic correctly, this only works out if you set a to 0, but it's
not clear to me why we should do that.
>
>0 = 9 - 0/4 - c/2 - 0/4, c = 18
>
>The first time around we can plug into the original ****inequality****, and we
don't have this problem.
That's it. The first time it was ***inequality**. So, we didn't have the slack
variables d,e,f.
Now we have equalities, and the left side is the slack. That slack can be
varied. The goal is to have that slack as low as possible, ideally 0, i.e. to
come as close as possible to the = side of the original inequalities. In fact,
slack=0 is ideal, that is the maximum, but then other constraints will limit how
much we can "creep up".
So, you plug in on the right side, set slack to 0, and see how much you can
"creep up":
So that's why we do constraint 1 as:
0 = 9 - 0/4 - c/2 - 0/4, c = 18
etc., each slack is set to 0;
Since we are trying to get the slack to be as close to 0 as possible, taking into
account all constraints, the next step is to determine which equation allows us
to reduce slack as much as possible without violating any other constraints.
So, we pick the smallest increase in c.
In summary: the procedure for every iteration of while loop is the same. The
main idea is to “creep up” towards reducing the slack and thus keep on
coming to the edge, i.e. the equality part, of the original inequalities. This is a
very commonly used approach: start from some solution, and try to come
closer and closer to the optimal solution.