Linear programming – The Simplex Algorithm
The Simplex algorithm for solving LP’s requires that all constraints are equations (with exception of sign constraints on the variables) and all variables be nonnegative. An LP in this form is said to be in standard form .
Any LP can be converted into an equivalent one in standard form.
Example: max z

20 x
1

15 x
2 s.t.
x
1
50 x
1
20 x
1

 x
2
35 x
2
15 x
2 x x
2





100
100
6000
2000
0 max z

20 x
1

15 x
2 s.t.
x
1
50 x
1
20 x
1

 x
2
35 x
2
15 x
2
 s
1
 s
2


 s
3

 
4
100
100
6000
2000 all variables

0
 s variables are called slack variables
 e variables are called excess or surplus variables
We usually refer to both the slack and surplus variables as ‘slacks’.
We can also convert an LP with unrestricted or nonpositive variables into standard form.
1

Suppose we have an LP in standard form with n variables and m constraints.
For convenience, we label the variables x x
1 2
, , x n
.
max (or min) z
 c x
 c x
 
s.t. a x
 a x
2 a x
 a x
2
 
  a x n
 n n
 b
1 b
2 a x m 1 1
 a x m 2 2
  a x mn n
 b m x x
1 2
, , x n

0
If we define A





 a a a

11
21 m 1 a
12 a
22
 a m 2


 a a
1 n
2
 a mn n





, x x
1
  x
2
 
  
 
 
  x n
, and b
 b
  b
2
 
, then b m the equation constraints of the LP can be written in matrix form as Ax
 b .
Usually, A is called the constraint matrix and b is the right hand side vector.
We can write all the constraints of an LP in standard form as follows:
Ax
 b x

0
Before we can present the Simplex method for solving LP’s we need to review some fundamental definitions and results.
2

Consider here a system of equations Ax
 b with n variables and m constraints, where we assume n
 m (i.e., more variables than equations).
Definition: A basic solution to the system of equations Ax
 b is obtained by:
1.
setting n
 m variables equal to 0,
2.
solving for the m remaining variables, and
3.
confirming that the solution for these m remaining variables is unique.
Comment: condition 3 is equivalent to requiring that the m columns of matrix
A that correspond to these m variables be linearly independent . This means that none of these m columns from A can be expressed as a linear combination of the remaining m

1 columns.
The n
 m variables that are fixed equal to 0 are called nonbasic variables
(NBV). The remaining m variables are called basic variables (BV). Note that, in general, different choices of nonbasic and basic variables will yield different
(basic) solutions to the system of equations Ax
 b .
Example: Find all basic solutions to the system x
1
 x
2

3
 x
2
 x
3
 
1

NBV =

NBV =

NBV = x
3 x
2
, BV=
, BV= x x x x
2
} x
1 x
1 x
1



2, x
2

1, x
3

0
3, x
2

0, x
3
 
1
0, x
2

3, x
3

2 x
1
, BV= x x
Definition: A nonnegative basic solution is a called a basic feasible solution
(bfs) since it is feasible to Ax
 b x

0 and basic for Ax
 b .
Theorem: the basic feasible solutions of Ax
 b are the extreme points of the polyhedral set defined by Ax
 b x

0 .
3

Example: x
1
2 x
1
 x
2
 x
2

40

60

0 x x
1 2
By drawing the polyhedron in x x
2
) space, we see that its extreme points are:
(0,0), (30,0), (20,20), (0,40).
Now we want to identify the basic feasible solutions and see their correspondence with the extreme points. To do this, we need to convert the constraint into standard form by adding slacks. x
1
 x
2 x
1
 x
2
2
 s
1

40
 s
2

60 x x s s
2

0 x
2
BV NBV x
1 s
1 s
2 bfs?
Definition: Two basic feasible solutions are called adjacent if their sets of basic variables differ in only one variable.
Theorem: If a linear program has an optimal solution, then there is one that is an extreme point.
Corollary: we can solve an LP in standard form by considering only its basic feasible solutions.
4

The Simplex Method
1.
Convert the LP to standard form.
2.
Obtain a bfs (if possible).
3.
Determine whether the current bfs is optimal
4.
If current bfs is not optimal, determine which nonbasic variables should become basic and which basic variable should become nonbasic, so that a new (adjacent) bfs with better objective function value is obtained.
5.
Compute the values of the variables in the new bfs by using elementary row operations (ero’s). Go back to step 3.
Example: max z

60 x
1

30 x
2

20 x
3
s.t.
8 x
1
4 x
1
2 x
1

6 x
2

2 x
2

1.5
x
2 x
2
 x
3

1.5
x
3


48
20

0.5
x
3

8

5 x x x
3

0
We first write the objective function in the following form: z

60 x
1

30 x
2

20 x
3

0 and we convert the constraints to standard form:
5

Row 0 z - 60 x
1
- 30 x
2
- 20 x
3
Basic variable
= 0 z

0
Row 1 8 x
1
+ 6 x
2
+ x
3
+ s
1
= 48 s
1
= 48
Row 2 4 x
1
+ 2 x
2
+ 1.5
x
3
+
Row 3 2 x
1
+ 1.5
x
2
+ 0.5
x
3
+
Row 4 x
2 s
2
= 20 s
3
+ s
4
= 8
= 5 s
2 s
3 s
4
= 20
= 8
= 5
We need an initial bfs. Note BV = { bfs s
1

48, s
2

20, s
3

8, s
4

5 , s s s s
4 x
1
 x
2
 x
3

}, NBV = {
0 . The x x x
3 value
} gives the
of this bfs is z = 0.
The constraints in the above tabular formulation of our LP are not only in standard form, they are also in Simplex tableau form with respect to the choice we made of BV and NBV. Simplex tableau form means that each row is associated to a unique basic variable which appears in its row with a coefficient of 1 (and coefficient 0 in all other rows).
We will consider also z to be the basic variable associated with row 0 (however, we don’t require z to be nonnegative like the decision variables).
Is this bfs ( s
1

48, s
2

20, s
3

8, s
4

5 , x
1
 x
2
 x
3

0 ) an optimal solution?
Note that, in this case, the value of z will improve if we can make x x
2
, or x
3 positive. For a maximization LP, we will choose to increase the value of the nonbasic variable with the most negative coefficient in row 0 (in this case, x
1
) .
We say we choose x
1
to enter the set of basic variables. Note that we choose to increase the value of only one nonbasic variable per iteration, the others will remain fixed at value 0. This will have the effect of moving from one extreme point of the feasible region to a (better) adjacent one.
6

We need to determine now which (currently) basic variable should become nonbasic. Note that as we increase the value of x
1
, the values of the current basic variables must update according to:
8 x
1
+ s = 48
1
4 x
1
+ s
2
= 20
2 x
1
+ s
3
= 8 s
4
= 5
That is, s
1
= 48 - 8 x
1 s
2
= 20 - 4 x
1 s
3
= 8 - 2 x
1 s
4
= 5
We need to maintain the nonnegativity of all variables that are currently basic: s
1 s
2 s
3
= 48 - 8
= 20 - 4
= 8 - 2 x
1

0 x
1

0 x
1

0 s
4
= 5 0 ( s
4
is not affected here by changes in value of x
1
)
To preserve the nonnegativity of s s
2 s
3
, s
1
nonnegative if: x
1

48

6
8 s
2
nonnegative if: x
 
1
20
4
5 s
3
nonnegative if: x
1
8
4
2
7

Ratio test :
The largest value we can assign to x
1
now is the minimum of
48 20 8
, ,
8 4 2
= 4.
Note that if we set x
1

4 , then s
3
becomes 0. Variable s
3
is the one that we select to leave the set of basic variables and become nonbasic.
Now we need to compute the values of the new bfs. In the same process we can update our Simplex tableau to a canonical form that represents the new choice of BV and NBV.
The row of the entering variable ( x
1
) in the new tableau will be the one of the leaving variable ( s
3
).
We transform our current tableau to obtain the new one by a sequence of elementary row operations.
Row 0 x
1
enters BV
 s
3
leaves BV
 z - 60 x
1
- 30 x
2
- 20 x
3
= 0
Row 1 8 x + 6
1 x +
2 x +
3 s = 48
1
Row 2 4 x
1
+ 2 x
2
+ 1.5
x
3
+ s
2
= 20
Row 3 2 x
1
+ 1.5
x
2
+ 0.5
x
3
+
Row 4 x
2 s
3
+ s
4
= 8
= 5
The row of the leaving variable (in this case Row 3) is called the pivot row.
The coefficient of x
1
in the pivot row is called the pivot term.
We will compute the values of the variables in the new basic feasible solution and update the Simplex tableau to reflect the new choice of BV and NBV. We will do this using elementary row operations (ero’s) on the current tableau.
8

Henceforth, we will save effort by dropping the variables from the Simplex tableau and show only the coefficient entries: z
Row 0
Row 1
Row 2
Row 3
Row 4
1
0
0
0
0 x
1 x
2 x
3
- 60 - 30 - 20
8
4
2
0
6
1
1
2 1.5
1.5 0.5
0 s
1
0
1
0
0
0 s
2
0
0
1
0
0 s
3
0
0
0
1
0 s
4 rhs
0
0
0
0
1
0
48
20
8
5 z

0 s
1

48 s
2

20 s
3

8 s

4
5
Elementary row operations to update the Simplex tableau:
1.
Divide all entries in the pivot row (here, row 3) by the pivot term so that the entering variable (here, x
1
) has a coefficient of 1 in the pivot row.
(new row 3) = 0.5 * (old row 3) z x
1 x
2 x
3
Row 0
Row 1
Row 2
Row 3
Row 4 s
1 s
2 s
3 s
4 rhs
1 - 60 - 30 - 20 0 0 0 0 0
0 8 6 1 1 0 0 0 48
0
0
0
4
0
2 1.5
1 .75 .25
1 0
0
0
0
1
0
0
0
.5
0
0 20
0 4
1 5
2.
Add appropriate multiples of the pivot row to eliminate the entering variable from all other rows.
(new row 0) = (old row 0) + 60 * (new row 3)
(new row 1) = (old row 1) - 8 * (new row 3)
(new row 2) = (old row 2) - 4 * (new row 3)
(new row 4) = (old row 4) - 0 * (new row 3)
9

The (updated) Simplex tableau for BV={ s s x s
4
} is:
Row 0
Row 1
Row 2
Row 3
Row 4
1
0
0
0
0 z x
1 x
2 x
3 s
1 s
2 s
3 s
4 rhs
0 15 - 5 0 0 30 0 240
0 0 -1 1 0 -4 0 16
0
1 .75 .25
0
-1
1
.5
0
0
0
0
1
0
0
-2
.5
0
0
0
1
4
4
5 z

240 s
1

16 s

2
4 x
1

4 s
4

5
We have completed one iteration of the Simplex Method. Now we need to determine if the current bfs is an optimal solution for the LP.
Since the coefficient of x
3
in row 0 is negative, we can improve z if we make x
3
positive. Thus, we can not conclude that we have an optimal solution and we decide to have x
3
become basic.
We do the ratio test to determine which variable leaves the basis. We only consider rows with positive entries in the column of the entering variable ( x
3
), since these correspond to the basic variables that decrease as x
3
increases. x
3 rhs ratios
Row 0
Row 1 ( s
1
) -1 16 no
.5 4 4/.5 Row 2 ( s
2
)
Row 3 ( x
1
)
Row 4 ( s
4
)
.25
0
4
5
4/.25 no
Minimum ratio attained at Row 2, thus s
2
leaves basis.
New set of basic variables is: BV={ s x x s
4
} . We use elementary row operations to update the tableau (see next page).
10

Row 0
Row 1
Row 2
Row 3
Row 4
1
0
0
0
0 z x
1 x
2 x
3 s
1 s
2 s
3 s
4 rhs
0 5 0 0 10 10 0 280
0 -2 0 1 2 -8 0 24
0
1 1.25
0
-2
1
1
0
0
0
0
0
2
-.5 1.5
0
-4
0
0
0
1
8
2
5 z

280 s
1

24 x
3

8 x
1

2 s
4

5
Since the coefficients of the decision variables in row 0 are all nonnegative, we can conclude that the current solution is optimal.
Remaining issues:
Finding an initial bfs:
It is easy for LP’s were all original constraints are 
and the right hand side data is nonnegative (as in the LP we just solved). In such case we can use the slacks as initial BV. In the general case, finding an initial bfs is not trivial and requires solving another LP (called the Phase I LP, for which we find an initial bfs by construction).
Unbounded LP’s:
We conclude that an LP is unbounded if at some iteration we select a variable to enter the basis but there are no upper bound on its value since all coefficients in its column has no positive coefficients (i.e., we can’t do the ratio test)
Convergence:
There are at most n
 
possible basic solutions. Assuming we don’t repeat a basis, the Simplex method will terminate in a finite number of iterations.
Repeating a basis is called “cycling” and can only occur if the constraints are degenerate.
There are anti-cycling rules that can be used with the Simplex method, but these are not implemented in commercial codes since cycling is rarely a problem in practice.
11