Matrices in Simplex Algorithm
Let an LP model is
Max/Min z = CX
s.t.
AX=b
X>=0
Max/Min z = CBXB + CNXN
s.t.
ABXB + ANXN = b
XB, XN >=0
(1)
(2)
Let’s consider only (1) and (2):
z = CBXB + CNXN
ABXB + ANXN = b
(1)
(2)
Solving (2) means that multiplying the equation system by AB-1 from the left side:
AB-1ABXB + AB-1ANXN = AB-1b
(3)
After this multiplication (3) is equivalent to:
IXB + AB-1ANXN = AB-1b
(4)
Then simply XB is expressed in terms of XN:
XB = AB-1b - AB-1ANXN
(5)
When XB is substituted in (1) z is expressed in terms of XN:
z = CB(AB-1b - AB-1ANXN) + CNXN
(6)
After simple computations (6) is equivalent to:
z = CBAB-1b + (CN - CBAB-1AN)XN (7)
z + (CBAB-1AN - CN)XN = CBAB-1b (8)
(8) and (4) are gives the Simplex Table:
z + (CBAB-1AN - CN)XN = CBAB-1b
(8)
-1
-1
AB ANXN + IXB = AB b
(4)
Basic
XN
XB
RHS
z
CBAB-1AN – CN
0
CBAB-1b
XB
AB-1AN
I
AB-1b
 This row is called as “objective row”
 These rows are called as “constraint rows”
The above rules are always valid in a Simplex Table.
Important Property 4: For any simplex table, AB-1 is the mxm matrix consisting of the
columns in the current (or optimal) table that correspond to the initial table’s set of basic
variables taken in the same order.
Reduced Cost: For any nonbasic variable, the reduced cost for the variable is the amount by
which the nonbasic variable’s objective function coefficient must be improved before that
variable will become a basic in some optimal solution to the LP model.
Let xi be a nonbasic variable in the optimal solution of an LP model and ZCi be the objective
row coefficient of this variable in the optimal Simplex Table. Reduced cost of x i is equal to
|ZCi|.
Example:
Developing the equivalent standard model.
Max Z = 5 X1 + 4 X2
s.t.
6 X1 + 4 X2 + X3
= 24
X1 + 2 X2
+ X4
=6
- X1 + X2
+ X5
=1
X2
+ X6 = 2
X1, X2, X3, X4, X5, X6 ≥ 0
b=(24 6 1 2)T
X=(X1 X2 X3 X4 X5 X6)  XB=(X3 X4 X5 X6), XN=(X1 X2)
C=(5 4 0 0 0 0)  CB=(0 0 0 0), CN=(5 4)
6

A 1
1
0

4
2
1
1
1
0
0
0
0
1
0
0
0
0
1
0
0
1

0  A  0
0
B
0

0
1

0
1
0
0
0
0
1
0
0
6

0 , A  1
0 N  1
0
1 

Starting Simplex Table for XB=(X3 X4 X5 X6):
Basic
X1
X2
X3
X4
X5
X6
RHS
Z
-5
-4
0
0
0
0
0
X3
6
4
1
0
0
0
24
X4
1
2
0
1
0
0
6
4
2
1
1 
X5
-1
1
0
0
1
0
1
X6
0
1
0
0
0
1
2
Second Simplex Table for new XB = (X1 X4 X5 X6).
Basic
X1
X2
X3
X4
X5
X6
RHS
Z
0
-4/6
5/6
0
0
0
20
X1
1
4/6
1/6
0
0
0
4
X4
0
4/3
-1/6
1
0
0
2
X5
0
5/3
1/6
0
1
0
5
X6
0
1
0
0
0
1
2
Third Simplex Table for the new XB = (X1 X2 X5 X6) which is OPTIMAL SOLUTION
Basic
X1
X2
X3
X4
X5
X6
RHS
Z
0
0
3/4
1/2
0
0
21
X1
1
0
1/4
-1/2
0
0
3
X2
0
1
-1/8
3/4
0
0
3/2
X5
0
0
3/8
-5/4
1
0
5/2
X6
0
0
1/8
-3/4
0
1
1/2
Reduced cost of X3 is 3/4, reduced cost of X4 is 1/2.
Duality
Example:
Consider the following LP model.
Max z = 10x1 + 4x2
s.t.
5x1 + 2x2 <= 30
x1, x2 >= 0
Now, let’s try to find an upper bound for the value of z. It means that z cannot be more than
this upper bound. One can say that 100000000 is an upper bound for z. Yes, it seems that z
cannot exceed 100000000. So, this is a valid upper bound for z. Now, let’s try to find a better
upper bound for z (in maximization problems better upper bound means smaller upper
bound). According to the constraints and the parameters of the model it seems that z cannot
exceed 10000. So, compared to the first upper bound now we have much more better upper
bound. Now, although we did not solve the model we have an idea about the optimal solution.
We know that in any feasible solution z cannot be more than 10000. At this point the question
is what is the best upper bound for z? One can easily see that the maximum value of z can be
60 in a feasible solution. According to the constraint in a feasible solution 5x1 + 2x2 can be at
most 30. Since the objective function is two times of (5x1 + 2x2) it can be at most 60.
Example:
Consider the following LP model.
Min z = 10x1 + 4x2
s.t.
5x1 + 2x2 => 30
x1, x2 >= 0
Now, let’s try to find a lower bound for the value of z. It means that z cannot be less than this
lower bound. One can say that -100000000 is a lower bound for z. Yes, it seems that z cannot
be less than -100000000. So, this is a valid lower bound for z. Now, let’s try to find a better
lower bound for z (in minimization problems better lower bound means higher lower bound).
According to the constraints and the parameters of the model it seems that z cannot be less
than 0. So, compared to the first lower bound now we have much more better lower bound.
Now, although we did not solve the model we have an idea about the optimal solution. We
know that in any feasible solution z cannot be less than 0. At this point the question is what is
the best lower bound for z? One can easily see that the minimum value of z can be 60 in a
feasible solution. According to the constraint in a feasible solution 5x1 + 2x2 can be at least
30. Since the objective function is two times of (5x1 + 2x2) it can be at least 60.
Example:
Consider the following LP model.
Max z = 11x1 + 12x2
s.t.
5x1 + 2x2 <= 30
x1 + 8x2 <= 20
x1, x2 >= 0
Now, let’s try to find a good upper bound for the value of z as much as possible. This time the
situation is not as much easy as the previous example. If you multiply (5x1 + 2x2) by 2 and
add (x1 + 8x2) you obtain 11x1 + 12x2 which is the objective function. Now, see the
following operation:
5x1 + 2x2 <= 30 (multiply both side by 2)
x1 + 8x2 <= 20 (multiply both side by 1)
and then sum them. The resultant expression is:
11x1 + 12x2 <= 80.
So, 80 is an upper bound for z.
Example:
Consider the following LP model.
Min z = 11x1 + 12x2
s.t.
5x1 + 2x2 >= 30
x1 + 8x2 >= 20
x1, x2 >= 0
Now, let’s try to find a good lower bound for the value of z as much as possible. This time the
situation is not as much easy as the previous example. If you multiply (5x1 + 2x2) by 2 and
add (x1 + 8x2) you obtain 11x1 + 12x2 which is the objective function. Now, see the
following operation:
5x1 + 2x2 >= 30 (multiply both side by 2)
x1 + 8x2 >= 20 (multiply both side by 1)
and then sum them. The resultant expression is:
11x1 + 12x2 >= 80.
So, 80 is a lower bound for z.
Example:
Consider the following LP model.
Max z = 11x1 + 12x2
s.t.
5x1 + 2x2 <= 30
2x1 + 9x2 <= 20
x1, x2 >= 0
Now, let’s try to find a good upper bound for the value of z as much as possible. This time the
situation is not as much easy as the previous example. If you multiply (5x1 + 2x2) by 2 and
add (x1 + 8x2) you obtain 12x1 + 13x2. Now, see the following operation:
5x1 + 2x2 <= 30 (multiply both side by 2)
2x1 + 9x2 <= 20 (multiply both side by 1)
and then sum them. The resultant expression is:
12x1 + 13x2 <= 80.
Since x1>=0, 12x1>=11x1, and since x2>=0, 13x2>=12x2. So,
z = 11x1 + 12x2 <=12x1 + 13x2 <= 80. As a result,
z <= 80.
80 is an upper bound for z.
Example:
Consider the following LP model.
Min z = 13x1 + 15x2
s.t.
5x1 + 2x2 >= 30
2x1 + 9x2 >= 20
x1, x2 >= 0
Now, let’s try to find a good lower bound for the value of z as much as possible. This time the
situation is not as much easy as the previous example. If you multiply (5x1 + 2x2) by 2 and
add (x1 + 8x2) you obtain 12x1 + 13x2. Now, see the following operation:
5x1 + 2x2 >= 30 (multiply both side by 2)
2x1 + 9x2 >= 20 (multiply both side by 1)
and then sum them. The resultant expression is:
12x1 + 13x2 >= 80.
Since x1>=0, 13x1>=12x1, and since x2>=0, 15x2>=13x2. So,
z = 13x1 + 15x2 >=12x1 + 13x2 >= 80. As a result,
z >= 80.
80 is a lower bound for z.
When we have a general model in order to find the best upper (lower) bound to the objective
function value of a maximization (minimization) problem determining the multipliers of the
constraints emerges as another problem. We should determine the minimum (maximum)
multipliers that yields a function which is greater (less) than or equal to the objective function
of the maximization (minimization) model. This is another LP problem and yields another LP
model. The original model is called primal the new model is called dual and the relation
between these models and their solutions are called duality. Multipliers of the constraints are
called dual variables. Duality is vital to understanding advanced topics in LP and NLP. The
relationship gives us interesting economic insights and provide additional insights into
sensitivity analysis.
Example:
Develop the dual of the following LP model.
Primal model:
Max z = 10x1 + 4x2
s.t.
5x1 + 2x2 <= 30
x1, x2 >= 0
(corresponding dual variable is y1)
Now, let’s multiply both side of the primal constraint by multiplier y1.
(5y1)x1 + (2y1)x2 <= 30y1
From the examples at the beginning of duality you can understand that we try to determine the
value of y1 such that
z = 10x1 + 4x2 <= (5y1)x1 + (2y1)x2 <= 30y1
In order to have such a relation the following conditions must be satisfied:
10x1 <= (5y1)x1  10 <= 5y1
4x2 <= (2y1)x2  4 <= 2y1
If these two conditions are satisfied by the multiplier y1 then it is guaranteed that
z = 10x1 + 4x2 <= (5y1)x1 + (2y1)x2 <= 30y1
When y1=100 these two conditions are satisfied and we obtain
z = 10x1 + 4x2 <= (5*100)x1 + (2*100)x2 <= 30*100  z <= 3000
So, 3000 is an upper bound for z. Because the better upper bound means smaller upper bound
the question is that for which value of y1 the above two conditions are satisfied and the upper
bound (30y1) is minimized. This yields following LP problem:
Dual model:
Min w = 30y1
s.t.
5y1 >= 10
2y1 >= 4
y1 >= 0
(dual constraint corresponding to x1)
(dual constraint corresponding to x2)
Example:
Develop the dual of the following LP model.
Primal model:
Min z = 13x1 + 15x2
s.t.
5x1 + 2x2 >= 30
2x1 + 9x2 >= 20
x1, x2 >= 0
(corresponding dual variable is y1)
(corresponding dual variable is y2)
Now, let’s multiply both sides of the primal constraints by corresponding multipliers y1 and
y2.
(5y1)x1 + (2y1)x2 >= (30y1)
(2y2)x1 + (9y2)x2 >= (20y2)
Now, let’s sum these two inequality:
(5y1 + 2y2)x1 + (2y1 + 9y2)x2 >= (30y1 + 20y2)
We try to determine the values of y1 and y2 such that the following relation is satisfied.
z = 13x1 + 15x2 >= (5y1 + 2y2)x1 + (2y1 + 9y2)x2 >= (30y1 + 20y2)
In order to have such a relation the following conditions must be satisfied:
13x1 >= (5y1 + 2y2)x1  13 >= 5y1 + 2y2
15x2 >= (2y1 + 9y2)x2  15 >= 2y1 + 9y2
If these two conditions are satisfied by the multipliers y1 and y2 then it is guaranteed that
z = 13x1 + 15x2 >= (5y1 + 2y2)x1 + (2y1 + 9y2)x2 >= (30y1 + 20y2)
When y1= 0 and y2=0 these two conditions are satisfied and we obtain
z = 13x1 + 15x2 >= (5*0 + 2*0)x1 + (2*0 + 9*0)x2 >= (30*0 + 20*0)  z >= 0
So, 0 is a lower bound for z. Because the better lower bound means higher upper bound the
question is that for which values of y1 and y2 the above two conditions are satisfied and the
lower bound (30y1 + 20y2) is maximized. This yields following LP problem:
Dual model:
Max w = 30y1 + 20y2
s.t.
5y1 + 2y2 <= 13
2y1 + 9y2 <= 15
y1, y2 >= 0
(dual constraint corresponding to x1)
(dual constraint corresponding to x2)
The relation between the primal and dual models are given in the following table.
Primal Model
Primal objective
maximization
Primal Constraint
>=
<=
=
Primal Variable
>=0
<=0
Unrestricted









Dual Model
Dual objective
minimization
Dual Variable
<= 0
>= 0
Unrestricted
Dual Constraint
>=
<=
=
Primal Model
Primal objective
minimization
Primal Variable
<= 0
>= 0
Unrestricted
Primal Constraint
>=
<=
=









Dual Model
Dual objective
maximization
Dual Constraint
>=
<=
=
Dual Variable
>= 0
<= 0
Unrestricted
Note that dual model of the dual model of a primal model is again the primal model. For each
constraint in the primal model we have a corresponding dual variable in the dual model. For
each primal variable in the primal model we have a corresponding constraint in the dual
model.
Example:
Primal model:
Max z = 5x1 – 14x2 + 6x3 + 8x4
s.t.
7x1 + 2x2 – 3x3
>= 35
2x1 – 3x2 + 5x3 + x4 <= 135
4x1
+ 6x3 + 2x4 = 76
x1>=0
x2>=0
x3<=0
x4 : unrestricted
Dual model:
Min w = 35y1 + 135y2 +76y3
s.t.
7y1 + 2y2 + 4y3 >= 5
2y1 – 3y2
>= -14
-3y1 + 5y2 + 6y3 <= 6
y2 + 2y3 = 8
y1 <= 0
y2 >= 0
y3 : unrestricted
Relationship Between Primal solutions and Dual Solutions:
1) (weak duality property) If primal is a maximization type problem and dual is a
minimization problem:
Z = Cx <= W=yb
If primal is a minimization type problem and dual is a maximization problem:
W=yb <= Z = Cx
2) If one of them has an unbounded solution then the other has infeasible solution
3) If one of them has an infeasible solution then the other has either unbounded solution
or infeasible solution
4) (strong duality property) If one of them has an optimal solution then the other also
has an optimal solution and the optimal objective function values are equal.
Let X be the optimal solution of the primal model and Y be the optimal solution of
dual model.
Z=CX=W=Yb
Z=CBAB-1b=W=YbYb=CBAB-1bY = CBAB-1
According to this property the optimal solution of dual model can be directly
determined from the optimal solution of the primal model. (or vice versa)
5) (complementary slackness property)
Let xsi be the slack or surplus variable added to the ith constraint of the primal model
and yi be the dual variable corresponding to this constraint. In the optimal solutions of
the models
yi*xsi=0
Similarly, let ysi be the slack or surplus variable added to the ith constraint of the dual
model and xi be the primal variable corresponding to this constraint. In the optimal
solutions of the models
xi*ysi=0
Economic Interpretation of the Dual Variables
The shadow price (or dual price) of the ith constraint is the amount by which the optimal Zvalue is improved (increased in a max problem and decreased in a min problem) if we
increase bi by 1 (from bi to bi + 1).
In a maximization problem the shadow price of the ith constraint of a problem is the optimal
value of the ith dual variable.
In a minimization problem the shadow price of the ith constraint of a problem is negative of
the optimal value of the ith dual variable.
So, dual variables show the price we are willing to pay for a unit of each of our resources.
Negative shadow price implies deterioration in the optimal Z-value by increasing bi by 1.
How to Read the Optimal Dual Solution from Objective Row of Optimal Table
Optimal value of dual variable yi:
if constraint is a ≤ constraint
yi = coefficient of Si in optimal row 0
if constraint is a ≥ constraint
yi = - (coefficient of ei in optimal row 0)
if constraint is an equality constraint
yi = (coefficient of ai in optimal row 0) – M
Dual Simplex Algorithm
In a Simplex Table if the optimality condition is satisfied (i.e., ZCi values of all nonbasic decision variables are non-positive (<=0) in a minimization problem, non-negative
(>=0) in a maximization problem) but feasibility condition is violated (i.e., there are basic
decision variables with negative values) we may use the Dual Simplex Algorithm. These
solutions are called “dual feasible solution”. We may have such a Simplex Table if some new
constraints are added to the model and the optimal Simplex table of the old model is updated
according to the new constraints, or if all of the objective function coefficients are positive in
a minimization problem or negative in a maximization problem.
Suppose that you have an LP model and you solved it by the Simplex Algorithm. Then
some new constraints are added to the model. You may follow two alternate ways: (1)
cancelling the previous solution and searching the new optimal solution by the Simplex
Algorithm from the very beginning, (2) updating the optimal Simplex Table of the old model
according to the new constraints and continue by the Dual Simplex Algorithm.
Step 1: Find a Simplex Table that the optimality condition is satisfied but feasibility condition
is violated.
Step 2: If feasibility condition is satisfied STOP, the optimal solution.
Step 4: (Finding the basic decision variable that will leave the basic decision variable set XB
and move to the non-basic decision variable set XN)
Let RHSj be the value of the jth basic decision variable.
Let
k  arg min {RHS j }
j 1, 2 ,..., m
RHS j 0
Then, Xl is the basic decision variable in the kth constraint row of the current Simplex Table.
Note that arg min returns the value of j that yields the searched minimum value.
Step 3: (Finding the non-basic decision variable that will enter to the basic decision variable
set XB)
For all non-basic decision variable Xi, having negative caik value, compute: |
one having the minimum ratio as the entering variable: Xe.
If there is no non-basic with negative caik value STOP: Infeasible Solution.
ZCi
| . Select the
caik
Step 5: (Constructing the Simplex Table of the next basic decision variable set)
The column of entering variable Xe is called “pivot column”, the row of leaving variable Xl is
called “pivot row”, the coefficient caek in the intersection of pivot column and pivot row is
called “pivot element”.
Step 5-a) Replace Xl with Xe in the “Basic” column of the next Simplex Table.
Step 5-b) New pivot row = (current pivot row) / pivot element
Step 5-c) All other rows, including objective row,
New row = (current row) – (its pivot column coefficient in the current table)*(New pivot row)
Go to Step 2.
Example:
Max z = 4x1 + 3x2 + x3
s.t.
3x1 + 2x2 + x3 + x4 = 7
2x1 + x2 + 2x3 + x5 = 11
x1, x2, x3, x4, x5 >= 0
The optimal Simplex Table is given below.
Basic
X1
X2
X3
X4
X5
RHS
Z
1/2
0
1/2
3/2
0
21/2
X2
3/2
1
1/2
1/2
0
7/2
X5
1/2
0
3/2
-1/2
1
15/2
Assume that the following constraint is added to the model.
x2 <= 3
Search the new optimal solution.
Step 1: In order to add this constraint to the Simplex Table it should be transformed to
the equality type by adding a new slack variable (x6).
x2 + x6 = 3
Basic
X1
X2
X3
X4
X5
X6
RHS
Z
1/2
0
1/2
3/2
0
0
21/2
X2
3/2
1
1/2
1/2
0
0
7/2
X5
1/2
0
3/2
-1/2
1
0
15/2
X6
0
1
0
0
0
1
3
After adding the new constraint to the table one of the properties of the Simplex Table is
violated. The bold “1” in column of X2 must be zero. By the row operations we should make
it zero (multiply the X2 row by -1 and add it to the X6 row).
Basic
X1
X2
X3
X4
X5
X6
RHS
Z
1/2
0
1/2
3/2
0
0
21/2
X2
3/2
1
1/2
1/2
0
0
7/2
X5
1/2
0
3/2
-1/2
1
0
15/2
X6
-3/2
0
-1/2
-1/2
0
1
-1/2
Step 2: Feasibility condition is not satisfied. Go to Step 3.
Step 3: X6 is the leaving variable.
Step 4: The ratio for X1 is | 1/2 / -3/2 | = 1/3
The ratio for X3 is | 1/2 / -1/2 | = 1
The ratio for X4 is | 3/2 / -1/2 | = 3
Minimum ratio belongs to X1. Hence, X1 is the entering variable.
Step 5: After row operations the next Simplex Table is
Basic
X1
X2
X3
X4
X5
X6
RHS
Z
0
0
1/3
4/3
0
1/3
31/3
X2
0
1
0
0
0
1
3
X5
0
0
4/3
-2/3
1
1/3
22/3
X1
1
0
1/3
1/3
0
-2/3
1/3
Step 2: Feasibility condition is satisfied: New optimal solution. STOP.