Module MA3484: Methods of Mathematical
Economics
Hilary Term 2015
D. R. Wilkins
c David R. Wilkins 2015
Copyright Contents
1 Introduction to Linear Programming
1.1 A Furniture Retailing Problem . . . . . . . . . . . . . . . . . .
1.2 Linear Programming Problems in Von Neumann Maximizing
Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
1
2 Bases of Finite-Dimensional Vector Spaces
2.1 Real Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Linear Dependence and Bases . . . . . . . . . . . . . . . . . .
2.3 Dual Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
6
6
9
3 The
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
3.9
Transportation Problem
Transportation in the Dairy Industry . . . . . . . . . . . . .
The General Transportation Problem . . . . . . . . . . . . .
Transportation Problems in which Total Supply equals Total
Demand . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Row Sums and Column Sums of Matrices . . . . . . . . . . .
Bases for the Transportation Problem . . . . . . . . . . . . .
Basic Feasible Solutions of Transportation Problems . . . . .
An Example illustrating the Procedure for finding an Initial
Basic Feasible Solution to a Transportation Problem using the
Minimum Cost Method . . . . . . . . . . . . . . . . . . . . .
An Example illustrating the Procedure for finding a Basic Optimal Solution to a Transportation Problem . . . . . . . . .
A Result concerning the Construction of Bases for the Transportation Problem . . . . . . . . . . . . . . . . . . . . . . .
i
4
13
. 13
. 14
.
.
.
.
14
16
19
23
. 28
. 30
. 37
3.10 The Minimum Cost Method . . . . . . . . . . . . . . . . . . . 39
3.11 The Northwest Corner Method . . . . . . . . . . . . . . . . . 41
3.12 The Iterative Procedure for Solving the Transportation Problem, given an Initial Basic Feasible Solution . . . . . . . . . . 41
4 The Simplex Method
49
4.1 Vector Inequalities and Notational Conventions . . . . . . . . 49
4.2 Feasible and Optimal Solutions of General Linear Programming Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.3 Linear Programming Problems in Dantzig Standard Form . . . 50
4.4 Basic Feasible Solutions to Linear Programming Problems in
Dantzig Standard Form . . . . . . . . . . . . . . . . . . . . . . 52
4.5 A Simplex Method Example with Five Variables and Two
Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.6 A Linear Tableau Example with Five Variables and Three
Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.7 Some Results concerning Finite-Dimensional Real Vector Spaces 77
4.8 The Extended Simplex Tableau for solving Linear Programming Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.9 The Simplex Tableau Algorithm . . . . . . . . . . . . . . . . . 92
4.10 The Revised Simplex Algorithm . . . . . . . . . . . . . . . . . 97
4.11 Finding an Initial Basic Solution to a Linear Programming
Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
5 General Linear Programming Problems, Duality and Complementary Slackness
102
5.1 General Linear Programming Problems . . . . . . . . . . . . . 102
5.2 Complementary Slackness and the Weak Duality Theorem . . 105
5.3 Open and Closed Sets in Euclidean Spaces . . . . . . . . . . . 113
5.4 A Separating Hyperplane Theorem . . . . . . . . . . . . . . . 114
5.5 Convex Cones . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
5.6 Farkas’ Lemma . . . . . . . . . . . . . . . . . . . . . . . . . . 117
5.7 Strong Duality . . . . . . . . . . . . . . . . . . . . . . . . . . 122
ii
1
Introduction to Linear Programming
1.1
A Furniture Retailing Problem
A retail business is planning to devote a number of retail outlets to the sale
of armchairs and sofas.
The retail prices of armchairs and sofas are determined by fierce competition in the furniture retailing business. Armchairs sell for e700 and sofas
sell for e1000.
However
• the amount of floor space (and warehouse space) available for stocking
the sofas and armchairs is limited;
• the amount of capital available for purchasing the initial stock of sofas
and armchairs is limited;
• market research shows that the ratio of armchairs to sofas in stores
should neither be too low nor too high.
Specifically:
• there are 1000 square metres of floor space available for stocking the
initial purchase of sofas and armchairs;
• each armchair takes up 1 square metre;
• each sofa takes up 2 square metres;
• the amount of capital available for purchasing the initial stock of armchairs and sofas is e351,000;
• the wholesale price of an armchair is e400;
• the wholesale price of a sofa is e600;
• market research shows that between 4 and 9 armchairs should be in
stock for each 3 sofas in stock.
We suppose that the retail outlets are stocked with x armchairs and y
sofas.
The armchairs (taking up 1 sq. metre each) and the sofas (taking up 2
sq. metres each) cannot altogether take up more than 1000 sq. metres of
floor space. Therefore
x + 2y ≤ 1000 (Floor space constraint).
1
The cost of stocking the retail outlets with armchairs (costing e400 each)
and sofas (costing e600 each) cannot exceed the available capital of e351000.
Therefore
4x + 6y ≤ 3510 (Capital constraint).
Consumer research indicates that x and y should satisfy
4y ≤ 3x ≤ 9y
(Armchair/Sofa ratio).
An ordered pair (x, y) of real numbers is said to specify a feasible solution
to the linear programming problem if this pair of values meets all the relevant
constraints.
An ordered pair (x, y) constitutes a feasible solution to the the Furniture
Retailing problem if and only if all the following constraints are satisfied:
x − 3y
4y − 3x
x + 2y
4x + 6y
x
y
≤
≤
≤
≤
≥
≥
0;
0;
1000;
3510;
0;
0;
The feasible region for the Furniture Retailing problem is depicted below:
We identify the vertices (or corners) of the feasible region for the Furniture Retailing problem. There are four of these:
• there is a vertex at (0, 0);
• there is a vertex at (400, 300) where the line 4y = 3x intersects the line
x + 2y = 1000;
2
• there is a vertex at (510, 245) where the line x + 2y = 1000 intersects
the line 4x + 6y = 3510;
• there is a vertex at (585, 195) where the line 3y = x intersects the line
4x + 6y = 3510.
These vertices are identified by inspection of the graph that depicts the
constraints that determine the feasible region.
The furniture retail business obviously wants to confirm that the business
will make a profit, and will wish to determine how many armchairs and sofas
to purchase from the wholesaler to maximize expected profit.
There are fixed costs for wages, rental etc., and we assume that these are
independent of the number of armchairs and sofas sold.
The gross margin on the sale of an armchair or sofa is the difference
between the wholesale and retail prices of that item of furniture.
Armchairs cost e400 wholesale and sell for e700, and thus provide a gross
margin of e300.
Sofas cost e600 wholesale and sell for e1000, and thus provide a gross
margin of e400.
In a typical linear programming problem, one wishes to determine not
merely feasible solutions to the problem. One wishes to determine an optimal
solution that maximizes some objective function amongst all feasible solutions
to the problem.
The objective function for the Furniture Retailing problem is the gross
profit that would accrue from selling the furniture in stock. This gross profit
is the difference between the cost of purchasing the furniture from the wholesaler and the return from selling that furniture.
This objective function is thus f (x, y), where
f (x, y) = 300x + 400y.
We should determine the maximum value of this function on the feasible
region.
Because the objective function f (x, y) = 300x + 400y is linear in x and
y, its maximum value on the feasable region must be achieved at one of the
vertices of the region.
Clearly this function is not maximized at the origin (0, 0)!
Now the remaining vertices of the feasible region are located at (400, 300),
(510, 245) and (585, 195), and
f (400, 300) = 240, 000,
f (510, 245) = 251, 000,
3
f (585, 195) = 253, 500.
It follows that the objective function is maximized at (585, 195).
The furniture retail business should therefore use up the available capital,
stocking 3 armchairs for every sofa, despite the fact that this will not utilize
the full amount of floor space available.
1.2
Linear Programming Problems in Von Neumann
Maximizing Form
A linear programming problem may be presented in Von Neumann maximizing form as follows:
given real numbers ci , Ai,j and bj for
i = 1, 2, . . . , m and j = 1, 2, . . . , n,
find real numbers x1 , x2 , . . . , xn so as to
maximize c1 x1 + c2 x2 + · · · + cn xn
subject to constraints
xj ≥ 0 for j = 1, 2, . . . , n, and
Ai,1 x1 + Ai,2 x2 + · · · + Ai,n xn ≤ bi for i = 1, 2, . . . , m.
The furniture retailing problem may be presented in von Neumann maximizing form with n = 2, m = 4,
(c1 , c2 ) = (300, 400),



 
1 −3
b1
0
 −3 4 
 b2   0
, 
 
A=
 1
 b3  =  1000
2 
4
6
b4
3510


.

Here A represents the m × n whose coefficient in the ith row and jth
column is Ai,j .
Linear programming problems may be presented in matrix form. We
adopt the following notational conventions with regard to transposes, row
and column vectors and vector inequalities:—
• vectors in Rm and Rn are represented as column vectors;
4
• we denote by M T the n × m matrix that is the transpose of an m × n
matrix M ;
• in particular, given b ∈ Rm and c ∈ Rn , where b and c are represented as
column vectors, we denote by bT and cT the corresponding row vectors
obtained on transposing the column vectors representing b and c;
• given vectors u and v in Rn for some positive integer n, we write u ≤ v
(and v ≥ u) if and only if uj ≤ vj for j = 1, 2, . . . , n.
A linear programming problem in von Neumann maximizing form may
be presented in matrix notation as follows:—
Given an m × n matrix A with real coefficients,
and given column vectors b ∈ Rm and c ∈ Rn ,
find x ∈ Rn so as to
maximize cT x
subject to constraints Ax ≤ b and x ≥ 0.
5
2
2.1
Bases of Finite-Dimensional Vector Spaces
Real Vector Spaces
Definition A real vector space consists of a set V on which there is defined
an operation of vector addition, yielding an element v + w of V for each pair
v, w of elements of V , and an operation of multiplication-by-scalars that
yields an element λv of V for each v ∈ V and for each real number λ. The
operation of vector addition is required to be commutative and associative.
There must exist a zero element 0V of V that satisfies v + 0V = v for all
v ∈ V , and, for each v ∈ V there must exist an element −v of V for which
v+(−v) = 0V . The following identities must also be satisfied for all v, w ∈ V
and for all real numbers λ and µ:
(λ + µ)v = λv + µv,
λ(v + w) = λv + λw,
λ(µv) = (λµ)v,
1v = v.
Let n be a positive integer. The set Rn consisting of all n-tuples of real
numbers is then a real vector space, with addition and multiplication-byscalars defined such that
(x1 , x2 , . . . , xn ) + (y1 , y2 , . . . , yn ) = (x1 + y1 , x2 + y2 , . . . xn + yn )
and
λ(x1 , x2 , . . . , xn ) = (λx1 , λx2 , . . . , λxn )
for all (x1 , x2 , . . . , xn ), (y1 , y2 , . . . , yn ) ∈ R and for all real numbers λ.
The set Mm,n (R) of all m × n matrices is a real vector space with respect
to the usual operations of matrix addition and multiplication of matrices by
real numbers.
2.2
Linear Dependence and Bases
Elements u1 , u2 , . . . , um of a real vector space V are said to be linearly dependent if there exist real numbers λ1 , λ2 , . . . , λm , not all zero, such that
λ1 u1 + λ2 u2 + · · · + λm um = 0V .
If elements u1 , u2 , . . . , um of real vector space V are not linearly dependent,
then they are said to be linearly independent.
Elements u1 , u2 , . . . , un of a real vector space V are said to span V if,
given any element v of V , there exist real numbers λ1 , λ2 , . . . , λn such that
v = λ1 u1 + λ2 u2 + · · · + λn un .
6
A vector space is said to be finite-dimensional if there exists a finite
subset of V whose members span V .
Elements u1 , u2 , . . . , un of a finite-dimensional real vector space V are
said to constitute a basis of V if they are linearly independent and span V .
Lemma 2.1 Elements u1 , u2 , . . . , un of a real vector space V constitute a
basis of V if and only if, given any element v of V , there exist uniquelydetermined real numbers λ1 , λ2 , . . . , λn such that
v = λ1 u1 + λ2 u2 + · · · + λn un .
Proof Suppose that u1 , u2 , . . . , un is a basis of V . Let v be an element V .
The requirement that u1 , u2 , . . . , un span V ensures that there exist real
numbers λ1 , λ2 , . . . , λn such that
v = λ1 u1 + λ2 u2 + · · · + λn un .
If µ1 , µ2 , . . . , µn are real numbers for which
v = µ1 u1 + µ2 u2 + · · · + µn un ,
then
(µ1 − λ1 )u1 + (µ2 − λ2 )u2 + · · · + (µn − λn )un = 0V .
It then follows from the linear independence of u1 , u2 , . . . , un that µi −λi = 0
for i = 1, 2, . . . , n, and thus µi = λi for i = 1, 2, . . . , n. This proves that the
coefficients λ1 , λ2 , . . . , λn are uniquely-determined.
Conversely suppose that u1 , u2 , . . . , un is a list of elements of V with the
property that, given any element v of V , there exist uniquely-determined
real numbers λ1 , λ2 , . . . , λn such that
v = λ1 u1 + λ2 u2 + · · · + λn un .
Then u1 , u2 , . . . , un span V . Moreover we can apply this criterion when
v = 0. The uniqueness of the coefficients λ1 , λ2 , . . . , λn then ensures that if
λ1 u1 + λ2 u2 + · · · + λn un = 0V
then λi = 0 for i = 1, 2, . . . , n. Thus u1 , u2 , . . . , un are linearly independent.
This proves that u1 , u2 , . . . , un is a basis of V , as required.
Proposition 2.2 Let V be a finite-dimensional real vector space, let
u1 , u2 , . . . , un
be elements of V that span V , and let K be a subset of {1, 2, . . . , n}. Suppose
either that K = ∅ or else that those elements ui for which i ∈ K are linearly
independent. Then there exists a basis of V whose members belong to the list
u1 , u2 , . . . , un which includes all the vectors ui for which i ∈ K.
7
Proof We prove the result by induction on the number of elements in the
list u1 , u2 , . . . , un of vectors that span V . The result is clearly true when
n = 1. Thus suppose, as the induction hypothesis, that the result is true for
all lists of elements of V that span V and that have fewer than n members.
If the elements u1 , u2 , . . . , un are linearly independent, then they constitute the required basis. If not, then there exist real numbers λ1 , λ2 , . . . , λn ,
not all zero, such that
λ1 u1 + λ2 u2 + · · · + λn un = 0V .
Now there cannot exist real numbers λ1 , λ2 , . . . , λn , not all zero, such
n
P
that both
λi ui = 0V and also λi = 0 whenever i 6= K. Indeed, in the
i=1
case where K = ∅, this conclusion follows from the requirement that the real
numbers λi cannot all be zero, and, in the case where K 6= ∅, the conclusion
follows from the linear independence of those ui for which i ∈ K. Therefore
there must exist some integer i satisfying 1 ≤ i ≤ n for which λi 6= 0 and
i 6∈ K. Without loss of generality, we may suppose that u1 , u2 , . . . , un are
ordered so that n 6∈ K and λn 6= 0. Then
n−1
X
λi
ui .
un = −
λ
i=1 n
Let v be an element of V . Then there exist real numbers µ1 , µ2 , . . . , µn
n
P
such that v =
µi ui , because u1 , u2 , . . . , un span V . But then
i=1
n−1 X
µn λi
v=
µi −
ui .
λ
n
i=1
We conclude that u1 , u2 , . . . , un−1 span the vector space V . The induction
hypothesis then ensures that there exists a basis of V consisting of members
of this list that includes the linearly independent elements u1 , u2 , . . . , um , as
required.
Corollary 2.3 Let V be a finite-dimensional real vector space, and let
u1 , u2 , . . . , un
be elements of V that span the vector space V . Then there exists a basis of
V whose elements are members of the list u1 , u2 , . . . , un .
Proof This result is a restatement of Proposition 2.2 in the special case
where the set K in the statement of that proposition is the empty set.
8
2.3
Dual Spaces
Definition Let V be a real vector space. A linear functional ϕ: V → R on
V is a linear transformation from the vector space V to the field R of real
numbers.
Given linear functionals ϕ: V → R and ψ: V → R on a real vector space V ,
and given any real number λ, we define ϕ + ψ and λϕ to be the linear
functionals on V defined such that (ϕ + ψ)(v) = ϕ(v) + ψ(v) and (λϕ)(v) =
λ ϕ(v) for all v ∈ V .
The set V ∗ of linear functionals on a real vector space V is itself a
real vector space with respect to the algebraic operations of addition and
multiplication-by-scalars defined above.
Definition Let V be a real vector space. The dual space V ∗ of V is the
vector space whose elements are the linear functionals on the vector space V .
Now suppose that the real vector space V is finite-dimensional. Let
u1 , u2 , . . . , un be a basis of V , where n = dim V . Given any v ∈ V there
n
P
exist uniquely-determined real numbers λ1 , λ2 , . . . , λn such that v =
λj uj .
j=1
It follows that there are well-defined functions ε1 , ε2 , . . . , εn from V to the
field R defined such that
!
n
X
εi
λj uj = λi
j=1
for i = 1, 2, . . . , n and for all real numbers λ1 , λ2 , . . . , λn . These functions are
linear transformations, and are thus linear functionals on V .
Lemma 2.4 Let V be a finite-dimensional real vector space, let
u1 , u2 , . . . , un
be a basis of V , and let ε1 , ε2 , . . . , εn be the linear functionals on V defined
such that
!
n
X
εi
λj uj = λi
j=1
for i = 1, 2, . . . , n and for all real numbers λ1 , λ2 , . . . , λn . Then ε1 , ε2 , . . . , εn
constitute a basis of the dual space V ∗ of V . Moreover
ϕ=
n
X
i=1
∗
for all ϕ ∈ V .
9
ϕ(ui )εi
Proof Let µ1 , µ2 , . . . , µn be real numbers with the property that
n
P
µ i εi =
i=1
0V ∗ . Then
0=
n
X
!
µ i εi
(uj ) =
n
X
i=1
µi εi (uj ) = µj
i=1
for j = 1, 2, . . . , n. Thus the linear functionals ε1 , ε2 , . . . , εn on V are linearly
independent elements of the dual space V ∗ .
Now let ϕ: V → R be a linear functional on V , and let µi = ϕ(ui ) for
i = 1, 2, . . . , n. Now
1 if i = j;
εi (uj ) =
0 if i 6= j.
It follows that
n
X
i=1
!
µ i εi
n
X
!
λj uj
=
j=1
n X
n
X
µi λj εi (uj ) =
i=1 j=1
=
n
X
λj ϕ(uj ) = ϕ
j=1
n
X
µj λj
j=1
n
X
!
λj uj
j=1
for all real numbers λ1 , λ2 , . . . , λn .
It follows that
n
n
X
X
ϕ=
µ i εi =
ϕ(ui )εi .
i=1
i=1
We conclude from this that every linear functional on V can be expressed as
a linear combination of ε1 , ε2 , . . . , εn . Thus these linear functionals span V ∗ .
We have previously shown that they are linearly independent. It follows that
n
P
they constitute a basis of V ∗ . Moreover we have verified that ϕ =
ϕ(ui )εi
i=1
for all ϕ ∈ V ∗ , as required.
Definition Let V be a finite-dimensional real vector space, let u1 , u2 , . . . , un
be a basis of V . The corresponding dual basis of the dual space V ∗ of V
consists of the linear functionals ε1 , ε2 , . . . , εn on V , where
!
n
X
εi
λj uj = λi
j=1
for i = 1, 2, . . . , n and for all real numbers λ1 , λ2 , . . . , λn .
Corollary 2.5 Let V be a finite-dimensional real vector space, and let V ∗
be the dual space of V . Then dim V ∗ = dim V .
10
Proof We have shown that any basis of V gives rise to a dual basis of V ∗ ,
where the dual basis of V has the same number of elements as the basis
of V to which it corresponds. The result follows immediately from the fact
that the dimension of a finite-dimensional real vector space is the number of
elements in any basis of that vector space.
Let V be a real-vector space, and let V ∗ be the dual space of V . Then
V ∗ is itself a real vector space, and therefore has a dual space V ∗∗ . Now each
element v of V determines a corresponding linear functional Ev : V ∗ → R on
V ∗ , where Ev (ϕ) = ϕ(v) for all ϕ ∈ V ∗ . It follows that there exists a function
ι: V → V ∗∗ defined so that ι(v) = Ev for all v ∈ V . Then ι(v)(ϕ) = ϕ(v)
for all v ∈ V and ϕ ∈ V ∗ .
Now
ι(v + w)(ϕ) = ϕ(v + w) = ϕ(v) + ϕ(w) = (ι(v) + ι(w))(ϕ)
and
ι(λv)(ϕ) = ϕ(λv) = λϕ(v) = (λι(v))(ϕ)
for all v, w ∈ V and ϕ ∈ V ∗ and for all real numbers λ. It follows that
ι(v + w) = ι(v) + ι(w) and ι(λv) = λι(v) for all v, w ∈ V and for all real
numbers λ. Thus ι: V → V ∗∗ is a linear transformation.
Proposition 2.6 Let V be a finite-dimensional real vector space, and let
ι: V → V ∗∗ be the linear transformation defined such that ι(v)(ϕ) = ϕ(v) for
all v ∈ V and ϕ ∈ V ∗ . Then ι: V → V ∗∗ is an isomorphism of real vector
spaces.
Proof Let u1 , u2 , . . . , un be a basis of V , let ε1 , ε2 , . . . , εn be the dual basis
of V ∗ , where
1 if i = j,
εi (uj ) =
0 if i 6= j,
and let v ∈ V . Then there exist real numbers λ1 , λ2 , . . . , λn such that v =
n
P
λi ui .
i=1
Suppose that ι(v) = 0V ∗∗ . Then ϕ(v) = Ev (ϕ) = 0 for all ϕ ∈ V ∗ . In
particular λi = εi (v) = 0 for i = 1, 2, . . . , n, and therefore v = 0V . We
conclude that ι: V → V ∗∗ is injective.
Now let F : V ∗ → R be a linear functional on V ∗ , let λi = F (εi ) for
n
n
P
P
i = 1, 2, . . . , n, let v =
λi ui , and let ϕ ∈ V ∗ . Then ϕ =
ϕ(ui )εi (see
i=1
i=1
11
Lemma 2.4), and therefore
ι(v)(ϕ) = ϕ(v) =
n
X
λi ϕ(ui ) =
i=1
= F
n
X
n
X
F (εi )ϕ(ui )
i=1
!
ϕ(ui )εi
= F (ϕ).
i=1
Thus ι(v) = F . We conclude that the linear transformation ι: V → V ∗∗
is surjective. We have previously shown that this linear transformation is
injective. There ι: V → V ∗∗ is an isomorphism between the real vector spaces
V and V ∗∗ as required.
The following corollary is an immediate consequence of Proposition 2.6.
Corollary 2.7 Let V be a finite-dimensional real vector space, and let V ∗
be the dual space of V . Then, given any linear functional F : V ∗ → R, there
exists some v ∈ V such that F (ϕ) = ϕ(v) for all ϕ ∈ V ∗ .
12
3
3.1
The Transportation Problem
Transportation in the Dairy Industry
We discuss an example of the Transportation Problem of Linear Programming, as it might be applied to optimize transportation costs in the dairy
industry.
A food business has milk-processing plants located in various towns in a
small country. We shall refer to these plants as dairies. Raw milk is supplied
by numerous farmers with farms located throughout that country, and is
transported by milk tanker from the farms to the dairies. The problem is
to determine the catchment areas of the dairies so as to minimize transport
costs.
We suppose that there are m farms, labelled by integers from 1 to m that
supply milk to n dairies, labelled by integers from 1 to n. Suppose that, in
a given year, the ith farm has the capacity to produce and supply a si litres
of milk for i = 1, 2, . . . , m, and that the jth dairy needs to receive at least dj
litres of milk for j = 1, 2, . . . , n to satisfy the business obligations.
m
P
The quantity
si then represents that total supply of milk, and the
quantity
n
P
i=1
dj represents the total demand for milk.
j=1
We suppose that xi,j litres of milk are to be transported from the ith farm
to the jth dairy, and that ci,j represents the cost per litre of transporting this
milk.
Then the total cost of transporting milk from the farms to the dairies is
m X
n
X
ci,j xi,j .
i=1 j=1
The quantities xi,j of milk to be transported from the farms to the dairies
should then be determined for i = 1, 2, . . . , m and j = 1, 2, . . . , n so as to
minimize the total cost of transporting milk.
However the ith farm can supply no more than si litres of milk in a given
year, and that jth dairy requires at least dj litres of milk in that year. It
follows that the quantities xi,j of milk to be transported between farms and
dairy are constrained by the requirements that
n
X
xi,j ≤ si
for i = 1, 2, . . . , m
j=1
13
and
m
X
xi,j ≥ dj
for j = 1, 2, . . . , m.
i=1
3.2
The General Transportation Problem
The Transportation Problem can be expressed generally in the following
form. Some commodity is supplied by m suppliers and is transported from
those suppliers to n recipients. The ith supplier can supply at most to si
units of the commodity, and the jth recipient requires at least dj units of the
commodity. The cost of transporting a unit of the commodity from the ith
supplier to the jth recipient is ci,j .
The total transport cost is then
m X
n
X
ci,j xi,j .
i=1 j=1
where xi,j denote the number of units of the commodity transported from
the ith supplier to the jth recipient.
The Transportation Problem can then be presented as follows:
determineP
xi,j for i = 1, 2, . . . , m and j = 1, 2, . . . , n so as
minimize
ci,j xi,j subject to the constraints xi,j ≥ 0 for all i
i,j
and j,
n
P
xi,j ≤ si and
j=1
m
P
xi,j ≥ dj , where si ≥ 0 for all i,
i=1
dj ≥ 0 for all i, and
m
P
si ≥
i=1
3.3
n
P
dj .
j=1
Transportation Problems in which Total Supply
equals Total Demand
Consider an instance of the Transportation Problem with m suppliers and
n recipients. The following proposition shows that a solution to the Transportation Problem can only exist if total supply of the relevant commodity
exceeds total demand for that commodity.
Proposition 3.1 Let s1 , s2 , . . . , sm and d1 , d2 , . . . , dn be non-negative real
numbers. Suppose that there exist non-negative real numbers xi,j be for i =
1, 2, . . . , m and j = 1, 2, . . . , n that satisfy the inequalities
n
X
xi,j ≤ si
and
j=1
m
X
i=1
14
xi,j ≥ dj .
Then
n
X
m
X
dj ≤
j=1
si .
i=1
Moreover if it is the case that
n
X
m
X
dj =
j=1
then
n
X
si .
i=1
xi,j = si
for i = 1, 2, . . . , m
xi,j = dj
for j = 1, 2, . . . , n.
j=1
and
m
X
i=1
Proof The inequalities satisfied by the non-negative real numbers xi,j ensure
that
n
m X
n
m
X
X
X
dj ≤
xi,j ≤
si .
j=1
i=1 j=1
i=1
Thus the total supply must equal or exceed the total demand.
n
m P
n
P
P
If it is the case that
xi,j < si for at least one value of i then
xi,j <
j=1
Pm
i=1
i=1 j=1
m
P
si . Similarly if it is the case that
j then
m P
n
P
i=1 j=1
xi,j > dj for at least one value of
i=1
xi,j >
Pn
j=1
dj .
It follows that if total supply equals total demand, so that
m
X
si =
n
X
i=1
then
n
X
dj ,
j=1
xi,j = si
for i = 1, 2, . . . , m
xi,j = dj
for j = 1, 2, . . . , n,
j=1
and
m
X
i=1
as required.
15
We analyse the Transportation Problem in the case where total supply
equals total demand. The optimization problem in this case can then be
stated as follows:—
determineP
xi,j for i = 1, 2, . . . , m and j = 1, 2, . . . , n so as
minimize
ci,j xi,j subject to the constraints xi,j ≥ 0 for all i
i,j
and j,
n
P
m
P
xi,j = si and
j=1
all i and j, and
m
P
i=1
xi,j = dj , where si ≥ 0 and dj ≥ 0 for
i=1
n
P
si =
dj .
j=1
Definition A feasible solution to the Transportation Problem (with equality
of total supply and total demand) takes the form of real numbers xi,j , where
• xi,j ≥ 0 for i = 1, 2, . . . , m and j = 1, 2, . . . , n;
•
n
P
xi,j = si ;
j=1
•
m
P
xi,j = dj .
i=1
Definition A feasible solution (xi,j ) of the Transportation Problem is said
to be optimal if it minimizes cost amongst all feasible solutions of the Transportation Problem.
3.4
Row Sums and Column Sums of Matrices
We commence the analysis of the Transportation Problem by studying the
interrelationships between the various real vector spaces and linear transformations that arise naturally from the statement of the Transportation
Problem.
The quantities xi,j to be determined are coefficients of an m×n matrix X.
This matrix X is represented as an element of the real vector space Mm,n (R)
that consists of all m × n matrices with real coefficients.
The non-negative quantities s1 , s2 , . . . , sm that specify the sums of the
coefficients in the rows of the unknown matrix X are the components of a
supply vector s belonging to the m-dimensional real vector space Rm .
Similarly the non-negative quantities d1 , d2 , . . . , dn that specify the sums
of the coefficients in the columns of the unknown matrix X are the components of a demand vector d belonging to the n-dimensional space Rn .
16
The requirement that total supply equals total demand translates into a
m
P
requirement that the sum
(s)i of the components of the supply vector s
must equal the sum
n
P
i=1
(d)j of the components of the demand vector d.
j=1
Accordingly we introduce a real vector space W consisting of all ordered
m
n
P
P
pairs (y, z) for which y ∈ Rm , z ∈ Rn and (y)i =
(z)j .
i=1
j=1
Lemma 3.2 Let m and n be positive integers, and let
(
)
m
n
X
X
W = (y, z) ∈ Rm × Rn :
(y)i =
(z)j .
i=1
j=1
Then the dimension of the real vector space W is m + n − 1.
Proof It is easy to see that the vector space W is isomorphic to Rm when
n = 1. The result then follows directly in the case when n = 1. Thus suppose
that n > 1.
Given real numbers y1 , y2 , . . . , yn and z1 , z2 , . . . , zn−1 , there exists exactly
one element (y, z) of W that satisfies (y)i = yi for i = 1, 2, . . . , m and
(z)j = zj for j = 1, 2, . . . , n − 1. The remaining component (z)n of the
n-dimensional vector z is then determined by the equation
(z)n =
m
X
yi −
i=1
m−1
X
zj .
j=1
It follows from this that dim W = m + n − 1, as required.
The supply and demand constraints on the sums of the rows and columns
of the unknown matrix X can then be specified by means of linear transformations
ρ: Mm,n (R) → Rm
and
σ: Mm,n (R) → Rn ,
where, for each X ∈ Mm,n (R), the components of the m-dimensional vector
ρ(X) are the sums of the coefficients along each row of X, and the components
of the n-dimensional vector σ(X) are the sums of the coefficients along each
column of X.
17
Let X ∈ Mm,n (R). Then the ith component ρ(X)i of the vector ρ(X) is
determined by the equation
ρ(X)i =
n
X
(X)i,j
for i = 1, 2, . . . , m,
j=1
for i = 1, 2, . . . , m, and the jth component σ(X)j of σ(X) is determined by
the equation
m
X
σ(X)j =
(X)i,j for j = 1, 2, . . . , n.
i=1
for j = 1, 2, . . . , n.
The costs ci,j are the components of an m × n matrix C, the cost matrix,
that in turn determines a linear functional
f : Mm,n (R) → R
on the vector space Mm,n (R) defined such that
T
f (X) = trace(C X) =
m X
n
X
(C)i,j Xi,j
i=1 j=1
for all X ∈ Mm,n (R).
An instance of the problem is specified by specifying a supply vector s,
demand vector d and cost matrix C. The components of s and d are required
to be non-negative real numbers. Moreover (s, d) ∈ W , where W is the real
vector space consisting of all ordered pairs (s, d) with s ∈ Rm and d ∈ Rn
for which the sum of the components of the vector s equals the sum of the
components of the vector d.
A feasible solution of the Transportation Problem with given supply vector s, demand vector d and cost matrix C is represented by an m×n matrix X
satisfying the following three conditions:—
• The coefficients of X are all non-negative;
• ρ(X) = s;
• σ(X) = d.
The cost functional f : Mm,n (R) → R is defined so that
f (X) = trace(C T X)
for all X ∈ Mm,n (R).
A feasible solution X of the Transportation problem is optimal if and
only if f (X) ≤ f (X) for all feasible solutions X of that problem.
18
Lemma 3.3 Let Mm,n (R) be the real vector space consisting of all m × n
matrices with real coefficients, let ρ: Mm,n (R) → Rm and σ: Mm,n (R) → Rn
be the linear transformations defined so that the ith component ρ(X)i of ρ(X)
satisfies
n
X
ρ(X)i =
(X)i,j for i = 1, 2, . . . , m,
j=1
for i = 1, 2, . . . , m, and the jth component σ(X)j of σ(X) satisfies
m
X
σ(X)j =
(X)i,j
for j = 1, 2, . . . , n.
i=1
for j = 1, 2, . . . , n. Then (ρ(X), σ(X)) ∈ W for all X ∈ Mm,n (R), where
)
(
m
n
X
X
(y)i =
(z)j .
W = (y, z) ∈ Rm × Rn :
i=1
j=1
Proof Let X ∈ Mm,n (R). Then
m
X
i=1
m X
n
n
X
X
ρ(X)i =
(X)i,j =
σ(X)j .
i=1 j=1
j=1
It follows that (ρ(X), σ(X)) ∈ W for all X ∈ Mm,n (R), as required.
3.5
Bases for the Transportation Problem
The real vector space Mm,n (R) consisting of all m×n matrices with real coefficients has a natural basis consisting of the matrices E (i,j) for i = 1, 2, . . . , m
and j = 1, 2, . . . , n, where, for each i and j, the coefficient of the matrix E (i,j)
in the ith row and jth column has the value 1, and all other coefficients are
zero. Indeed
m X
n
X
X=
(X)i,j E (i,j)
i=1 j=1
for all X ∈ Mm,n (R).
Let ρ: Mm,n (R) → Rm and σ: Mm,n (R) → Rn be the linear transforman
P
tions defined such that (ρ(X))i =
(X)i,j for i = 1, 2, . . . , m and (σ(X))j =
j=1
m
P
(X)i,j for j = 1, 2, . . . , n. Then ρ(E (i,j) ) = b(i) for i = 1, 2, . . . , m, where
i=1
b(i) denotes the ith vector in the standard basis of Rm , defined such that
1 if i = k;
(i)
(b )k =
0 if i 6= k.
19
Similarly σ(E (i,j) ) = b(j) for j = 1, 2, . . . , n, where b(j) denotes the jth vector
in the standard basis of Rn , defined such that
1 if j = l;
(j)
(b )l =
0 if j 6= l.
Now (ρ(X), σ(X)) ∈ W for all X ∈ Mm,n (R), where
(
)
m
n
X
X
W = (y, z) ∈ Rm × Rn :
(y)i =
(z)j .
i=1
j=1
Let I = {1, 2, . . . , m} and J = {1, 2, . . . , n}, and let
β (i,j) = (b(i) , b(j) )
for all (i, j) ∈ I × J. Then the elements β (i,j) span the vector space W . It
follows from basic linear algebra that there exist subsets B of I × J such that
the elements β (i,j) of W for which (i, j) ∈ B constitute a basis of the real
vector space W (see Corollary 2.3).
Definition Let I = {1, 2, . . . , m} and J = {1, 2, . . . , n}, where m and n are
positive integers, let
(
)
m
n
X
X
W = (y, z) ∈ Rm × Rn :
(y)i =
(z)j ,
i=1
j=1
and, for each (i, j) ∈ I × J, let β (i,j) = (b(i) , b(j) ), where b(i) ∈ Rm and b(j) ∈
Rn are defined so that the ith component of b(i) and that jth component
of b(j) are equal to 1 and the other components of these vectors are zero.
A subset B of I × J is said to be a basis for the Transportation Problem
with m suppliers and n recipients if and only if the elements β (i,j) for which
(i, j) ∈ B constitute a basis of the real vector space W .
The real vector space W is of dimension m+n−1, where m is the number
of suppliers and n is the number of recipients. It follows that any basis for
the Transportation Problem with m suppliers and n recipients has m + n − 1
members.
Proposition 3.4 Let I = {1, 2, . . . , m} and J = {1, 2, . . . , n}, where m
and n are positive integers. Then a subset B of I × J is a basis for the
transportation problem if and only if, given any vectors y ∈ Rm and z ∈ Rn
m
n
P
P
satisfying (y)m =
(z)n , there exists a unique m × n matrix X with real
i=1
j=1
coefficients satisfying the following properties:—
20
(i)
n
P
(X)i,j = (y)i for i = 1, 2, . . . , m;
j=1
(ii)
m
P
(X)i,j = (z)j for j = 1, 2, . . . , n;
i=1
(ii) (X)i,j = 0 unless (i, j) ∈ B.
Proof For each (i, j) ∈ I × J, let E (i,j) denote the matrix whose coefficient
in the ith row and jth column are equal to 1 and whose other coefficients are
zero, and let ρ(X) ∈ Rm and σ(X) ∈ Rn be defined for all m × n matrices X
n
m
P
P
with real coefficients so that (ρ(X))i =
(X)i,j and (σ(X))j =
(X)i,j .
j=1
i=1
Then ρ(E ) = b for i = 1, 2, . . . , m, where b denotes the vector in Rm
whose ith component is equal to 1 and whose other components are zero.
Similarly σ(E (i,j) ) = b(j) for j = 1, 2, . . . , n, where b(j) denotes the vector
in Rm whose jth component is equal to 1 and whose other components are
zero.
Let
(
)
m
n
X
X
W = (y, z) ∈ Rm × Rn :
(y)i =
(z)j .
(i,j)
(i)
(i)
i=1
j=1
Then (ρ(X), σ(X)) ∈ W for all X ∈ Mm,n (R), and
(ρ(E (i,j) ), σ(E (i,j) ) = β (i,j)
for all (i, j) ∈ I × J where
β (i,j) = (b(i) , b(j) ).
Let B be a subset of I × J, let y and z be elements of Rm and Rn
respectively that satisfy (y, z) ∈ W , and let X be an m × n matrix with real
coefficients with the property that (X)i,j = 0 unless (i, j) ∈ B. Then
X
X
ρ(X) =
(X)(i,j) ρ(E (i,j) ) =
(X)(i,j) b(i)
(i,j)∈B
(i,j)∈B
X
X
and
σ(X) =
(X)(i,j) σ(E (i,j) ) =
(i,j)∈B
(X)(i,j) b(j) ,
(i,j)∈B
and therefore
(ρ(X), σ(X)) =
X
(i,j)∈B
21
(X)(i,j) β (i,j) .
Suppose that, given any y ∈ Rm and z ∈ Rn for which (y, z) ∈ W , there
exists a unique m×n matrix X such that y = ρ(X), z = σ(X) and (X)i,j = 0
for all (i, j) ∈ B. Then the elements β (i,j) of W for which (i, j) ∈ B must span
W and must also be linearly independent. These elements must therefore
constitute a basis for the vector space B. It then follows that the subset B
of I × J must be a basis for the Transportation Problem.
Conversely if B is a basis for the Transportation Problem then, given any
(y, z) ∈ W , there must
m × n matrix X with real coefficients
Pexist a unique
(i,j)
such that (y, z) =
(X)i,j β
and (X)i,j = 0 unless (i, j) ∈ B. The
(i,j)∈B
result follows.
Lemma 3.5 Let m and n be positive integers, let I = {1, 2, . . . , m} and
J = {1, 2, . . . , n}, and let K be a subset of I × J. Suppose that there is no
basis B of the Transportation Problem for which K ⊂ B. Then there exists
a non-zero m × n matrix Y with real coefficients which satisfies the following
conditions:
•
n
P
(Y )i,j = 0 for i = 1, 2, . . . , m;
j=1
•
m
P
(Y )i,j = 0 for j = 1, 2, . . . , n;
i=1
• (Y )i,j = 0 when (i, j) 6∈ K.
Proof Let
(
W =
m
n
X
X
m
n
(y, z) ∈ R × R :
(y)i =
(z)j
i=1
)
,
j=1
let b(1) , b(2) , . . . , b(m) be the standard basis of Rm and let b(1) , b(2) , . . . , b(n)
be the standard basis of Rn , where the ith component of b(i) and the jth
component of b(j) are equal to 1 and the other components of these vectors
are zero, and let β (i,j) = (b(i) , b(j) ) for all (i, j) ∈ I × J.
Now follows from Proposition 2.2 that if the elements β (i,j) for which
(i, j) ∈ K were linearly independent then there would exist a subset B of
I × J satisfying K ⊂ B such that the elements β (i,j) for which (i, j) ∈ B
would constitute a basis of W . This subset B of I × J would then be a
basis for the Transportation Problem. But the subset K is not contained in
any basis for the Transportation Problem. It follows that the elements β (i,j)
for which (i, j) ∈ K must be linearly dependent. Therefore there exists a
22
non-zero m × n matrix Y with real coefficients such that (Y )i,j = 0 when
(i, j) 6∈ K and
m X
n
X
(Y )i,j β (i,j) = 0W .
i=1 j=1
Now β (i,j) = (b(i) , b(j) ) for all i ∈ I and j ∈ J. It follows that
m X
n
X
(Y )i,j b(i) = 0
i=1 j=1
and
m X
n
X
(Y )i,j b(j) = 0,
i=1 j=1
and therefore
n
X
(Y )i,j = 0 (i = 1, 2, . . . , m)
j=1
and
m
X
(Y )i,j = 0 (j = 1, 2, . . . , n),
i=1
as required.
3.6
Basic Feasible Solutions of Transportation Problems
Consider the Transportation Problem with m suppliers and n recipients,
where the ith supplier can provide at most si units of some given commodity, where si ≥ 0, and the jth recipient requires at least dj units of that
commodity, where dj ≥ 0. We suppose also that total supply equals total
demand, so that
m
n
X
X
si =
dj ,
i=1
j=1
The cost of transporting the commodity from the ith supplier to the jth
recipient is ci,j .
The concept of a basis for the Transportation Problem was introduced in
Subsection 3.5. We recall some results established in that subsection.
Let I = {1, 2, . . . , m} and J = {1, 2, . . . , n}. A subset B of I ×J is a basis
for the Transportation Problem if and only if, given any vectors y ∈ Rm and
23
z ∈ Rn satisfying
m
P
(y)i =
i=1
coefficients such that
n
P
(z)j , there exists a unique matrix X with real
j=1
n
P
(X)i,j = (y)i for i = 1, 2, . . . , m,
j=1
m
P
(X)i,j = (z)j for
i=1
j = 1, 2, . . . , n and (X)i,j = 0 unless (i, j) ∈ B (see Proposition 3.4). A basis
for the transportation problem has m + n − 1 elements.
Also if K is a subset of I × J that is not contained in any basis for the
Transportation Problem then there exists a non-zero m × n matrix Y such
n
m
P
P
that
(Y )i,j = 0 for i = 1, 2, . . . , m,
(X)i,j = 0 for j = 1, 2, . . . , n and
j=1
i=1
(Y )i,j = 0 unless (i, j) ∈ K (see Lemma 3.5).
Definition A feasible solution (xi,j ) of a Transportation Problem is said to
be basic if there exists a basis B for that Transportation Problem such that
xi,j = 0 whenever (i, j) 6∈ B.
Example Consider the instance of the Transportation Problem where m =
n = 2, s1 = 8, s2 = 3, d1 = 2, d2 = 9, c1,1 = 2, c1,2 = 3, c2,1 = 4 and c2,2 = 1.
A feasible solution takes the form of a 2 × 2 matrix
x1,1 x1,2
x2,1 x2,2
with non-negative components which satisfies the two matrix equations
x1,1 x1,2
1
8
=
x2,1 x2,2
1
3
and
1 1
x1,1 x1,2
x2,1 x2,2
=
2 9
.
A basic feasible solution will have at least one component equal to zero.
There are four matrices with at least one zero component which satisfy the
required equations. They are the following:—
0 8
8 0
2 6
−1 9
,
,
,
.
2 1
−6 9
0 3
3 0
The first and third of these matrices have non-negative components.
These two matrices represent basic feasible solutions to the problem, and
moreover they are the only basic feasible solutions.
The costs associated with the components of the matrices are c1,1 = 2,
c1,2 = 3, c2,1 = 4 and c2,2 = 1.
24
The cost of the basic feasible solution
0 8
2 1
is
8c1,2 + 2c2,1 + c2,2 = 24 + 8 + 1 = 33.
2 6
The cost of the basic feasible solution
is
0 3
2c1,1 + 6c1,2 + 3c2,2 = 4 + 18 + 3 = 25.
x1,1 x1,2
Now any 2 × 2 matrix
satisfying the two matrix equations
x2,1 x2,2
x1,1 x1,2
1
8
=
,
x2,1 x2,2
1
3
x1,1 x1,2
1 1
= 2 9
x2,1 x2,2
must be of the form
x1,1 x1,2
x2,1 x2,2
=
λ
8−λ
2−λ 1+λ
for some real number
λ.
λ
8−λ
But the matrix
has non-negative components if and
2−λ 1+λ
only if 0 ≤ λ ≤ 2. It follows that the set of feasible solutions of this instance
of the transportation problem is
λ
8−λ
: λ ∈ R and 0 ≤ λ ≤ 2 .
2−λ 1+λ
The costs associated with the components of the matrices are c1,1 = 2,
c1,2 = 3, c2,1 = 4 and c2,2 = 1. Therefore, for each real number λ satisfying
λ
8−λ
0 ≤ λ ≤ 2, the cost f (λ) of the feasible solution
is given
2−λ 1+λ
by
f (λ) = 2λ + 3(8 − λ) + 4(2 − λ) + (1 + λ) = 33 − 4λ.
2 6
Cost is minimized when λ = 2, and thus
is the optimal solution
0 3
of this instance of the Transportation Problem. The cost of this optimal
solution is 25.
25
Proposition 3.6 Given any feasible solution of the Transportation Problem,
there exists a basic feasible solution with whose cost does not exceed that of
the given solution.
Proof Let m and n be positive integers, and let s and d be elements of
Rm and Rn respectively that satisfy (s)i ≥ 0 for i = 1, 2, . . . , m, (d)i ≥ 0
m
n
P
P
for j = 1, 2, . . . , n and
(s)i =
(d)j , let C be an m × n matrix whose
i=1
j=1
components are non-negative real numbers, and let X be a feasible solution
of the resulting instance of the Transportation Problem with cost matrix C.
Let si = (s)i , dj = (d)j , xi,j = (X)i,j and ci,j = (C)i,j for i = 1, 2, . . . , m
n
P
and j = 1, 2, . . . , n. Then xi,j ≥ 0 for all i and j,
xi,j = si for i =
j=1
1, 2, . . . , m and
m
P
i=1
solution X is then
xi,j = dj for j = 1, 2, . . . , n. The cost of the feasible
m P
n
P
ci,j xi,j .
i=1 j=1
If the feasible solution X is itself basic then there is nothing to prove.
Suppose therefore that X is not a basic solution. We show that there then
exists a feasible solution X with fewer non-zero components than the given
feasible solution.
Let I = {1, 2, . . . , m} and J = {1, 2, . . . , n}, and let
K = {(i, j) ∈ I × J : xi,j > 0}.
Because X is not a basic solution to the Transportation Problem, there does
not exist any basis B for the Transportation Problem satisfying K ⊂ B. It
therefore follows from Lemma 3.5 that there exists a non-zero m×n matrix Y
which satisfies the following conditions:—
n
P
•
(Y )i,j = 0 for i = 1, 2, . . . , m;
j=1
•
m
P
(Y )i,j = 0 for j = 1, 2, . . . , n;
i=1
• (Y )i,j = 0 when (i, j) 6∈ K.
We can assume without loss of generality that
m P
n
P
ci,j (Y )i,j ≥ 0, be-
i=1 j=1
cause otherwise we can replace Y with −Y .
Let Zλ = X − λY for all real numbers λ. Then (Zλ )i,j = xi,j − λyi,j for
i = 1, 2, . . . , m and j = 1, 2, . . . , n, where xi,j = (X)i,j and yi,j = (Y )i,j .
Moreover the matrix Zλ has the following properties:—
26
•
n
P
(Zλ )i,j = si ;
j=1
•
m
P
(Zλ )i,j = dj ;
i=1
• (Zλ )i,j = 0 whenever (i, j) 6∈ K;
•
m X
n
X
i=1 j=1
ci,j (Zλ )i,j ≤
m X
n
X
ci,j (X)i,j whenever λ ≥ 0.
i=1 j=1
Now the matrix Y is a non-zero matrix whose rows and columns all sum
to zero. It follows that at least one of its coefficients must be strictly positive.
Thus there exists at least one ordered pair (i, j) belonging to the set K for
which yi,j > 0. Let
xi,j
λ0 = minimum
: (i, j) ∈ K and yi,j > 0 .
yi,j
Then λ0 > 0. Moreover if 0 ≤ λ < λ0 then xi,j − λyi,j > 0 for all (i, j) ∈ K,
and if λ > λ0 then there exists at least one element (i0 , j0 ) of K for which
xi0 ,j0 − λyi0 ,j0 < 0. It follows that xi,j − λ0 yi,j ≥ 0 for all (i, j) ∈ K, and
xi0 ,j0 − λ0 yi0 ,j0 = 0.
Thus Zλ0 is a feasible solution of the given Transportation Problem whose
cost does not exceed that of the given feasible solution X. Moreover Zλ0 has
fewer non-zero components than the given feasible solution X.
If Zλ0 is itself a basic feasible solution, then we have found the required
basic feasible solution whose cost does not exceed that of the given feasible
solution. Otherwise we can iterate the process until we arrive at the required
basic feasible solution whose cost does not exceed that of the given feasible
solution.
A given instance of the Transportation Problem has only finitely many
basic feasible solutions. Indeed there are only finitely many bases for the
problem, and any basis is associated with at most one basic feasible solution.
Therefore there exists a basic feasible solution whose cost does not exceed the
cost of any other basic feasible solution. It then follows from Proposition 3.6
that the cost of this basic feasible solution cannot exceed the cost of any other
feasible solution of the given instance of the Transportation Problem. This
basic feasible solution is thus a basic optimal solution of the Transportation
Problem.
27
The Transportation Problem determined by the supply vector, demand
vector and cost matrix has only finitely many basic feasible solutions, because there are only finitely many bases for the problem, and each basis can
determine at most one basic feasible solution. Nevertheless the number of
basic feasible solutions may be quite large.
But it can be shown that the Transportation Problem always has a basic
optimal solution. It can be found using an algorithm that implements the
Simplex Method devised by George B. Dantzig in the 1940s. This algorithm
involves passing from one basis to another, lowering the cost at each stage,
until one eventually finds a basis that can be shown to determine a basic
optimal solution of the Transportation Problem.
3.7
An Example illustrating the Procedure for finding
an Initial Basic Feasible Solution to a Transportation Problem using the Minimum Cost Method
We discuss the method for finding a basic optimal solution of the Transportation Problem by working through a particular example. First we find
an initial basic feasible solution using a method known as the Minimum Cost
Method. Then we test whether or not this initial basic feasible solution is
optimal. It turns out that, in this example, the initial basic solutions is not
optimal. We then commence an iterative process for finding a basic optimal
solution.
Let ci,j be the coefficient in the ith row and jth column of the cost matrix C, where


8 4 16
 3 7 2 

C=
 13 8 6  .
5 7 8
and let
s1 = 13,
s2 = 8,
d1 = 19,
s3 = 11,
d2 = 12,
s4 = 13,
d3 = 14.
We seek to non-negative real numbers xi,j for i = 1, 2, 3, 4 and j = 1, 2, 3
4 P
3
P
that minimize
ci,j xi,j subject to the following constraints:
i=1 j=1
3
X
xi,j = si
for i = 1, 2, 3, 4,
j=1
28
4
X
xi,j = dj
for j = 1, 2, 3,
i=1
and xi,j ≥ 0 for all i and j.
For this problem the supply vector is (13, 8, 11, 13) and the demand vector
is (19, 12, 14). The components of both the supply vector and the demand
vector add up to 45.
In order to start the process of finding an initial basic solution for this
problems, we set up a tableau that records the row sums (or supplies), the
column sums (or demands) and the costs ci,j for the given problem, whilst
leaving cells to be filled in with the values of the non-negative real numbers
xi,j that will specify the initial basic feasible solution. The resultant tableau
is structured as follows:—
ci,j & xi,j
1
1
8
2
4
?
2
?
3
7
?
3
13
8
dj
7
?
19
?
13
?
8
?
11
6
?
5
si
2
?
?
4
3
16
8
?
12
? 13
14 45
We apply the minimum cost method to find an initial basic solution.
The cell with lowest cost is the cell (2, 3). We assign to this cell the
maximum value possible, which is the minimum of s2 , which is 8, and d3 ,
which is 14. Thus we set x2,3 = 8. This forces x2,1 = 0 and x2,2 = 0. The
pair (2, 3) is added to the current basis.
The next undetermined cell of lowest cost is (1, 2). We assign to this cell
the minimum of s1 , which is 13, and d2 − x2,2 , which is 12. Thus we set
x1,2 = 12. This forces x3,2 = 0 and x4,2 = 0. The pair (1, 2) is added to the
current basis.
The next undetermined cell of lowest cost is (4, 1). We assign to this cell
the minimum of s4 − x4,2 , which is 13, and d1 − x2,1 , which is 19. Thus we set
x4,1 = 13. This forces x4,3 = 0. The pair (4, 1) is added to the current basis.
The next undetermined cell of lowest cost is (3, 3). We assign to this
cell the minimum of s3 − x3,2 , which is 11, and d3 − x2,3 − x4,3 , which is 6
(= 14 − 8 − 0). Thus we set x3,3 = 6. This forces x1,3 = 0. The pair (3, 3) is
added to the current basis.
29
The next undetermined cell of lowest cost is (1, 1). We assign to this cell
the minimum of s1 − x1,2 − x1,3 , which is 1, and d1 − x2,1 − x4,1 , which is 6.
Thus we set x1,1 = 1. The pair (1, 1) is added to the current basis.
The final undetermined cell is (3, 1). We assign to this cell the common
value of s3 − x3,2 − x3,3 and d1 − x1,1 − x2,1 − x4,1 , which is 5. Thus we set
x3,1 = 5. The pair (3, 1) is added to the current basis.
The values of the elements xi,j of the initial basic feasible solution are
tabulated (with basis elements marked by the • symbol) as follows:—
ci,j & xi,j
1
1
8
2
3
3
4
dj
•
1
2
4
•
12
7
0
0
13 • 8
5
0
5 • 7
13
0
19
12
3
16
2
6
si
0
•
8
•
6
13
8
11
8
0 13
14 45
Thus the initial basis is B where
B = {(1, 1), (1, 2), (2, 3), (3, 1), (3, 3), (4, 1)}.
The basic feasible solution is represented by the 6 × 5 matrix X, where


1 12 0
 0 0 8 

X=
 5 0 6 .
13 0 0
The cost of this initial feasible basic solution is
8 × 1 + 4 × 12 + 2 × 8 + 13 × 5 + 6 × 6
+ 5 × 13
= 8 + 48 + 16 + 65 + 36 + 65
= 238.
3.8
An Example illustrating the Procedure for finding a Basic Optimal Solution to a Transportation
Problem
We continue with the study of the optimization problem discussed in the
previous section.
30
We seek to determine non-negative real numbers xi,j for i = 1, 2, 3, 4 and
4 P
3
P
j = 1, 2, 3 that minimize
ci,j xi,j , where ci,j is the coefficient in the ith
i=1 j=1
row and jth column of the cost matrix C, where


8 4 16
 3 7 2 

C=
 13 8 6  .
5 7 8
subject to the constraints
3
X
xi,j = si
(i = 1, 2, 3, 4)
j=1
and
4
X
xi,j = dj
(j = 1, 2, 3),
s2 = 8,
s3 = 11,
i=1
where
s1 = 13,
d1 = 19,
d2 = 12,
s4 = 13,
d3 = 14.
We have found an initial basic feasible solution by the Minimum Cost
Method. This solution satisfies xi,j = (X)i,j for all i and j, where


1 12 0
 0 0 8 

X=
 5 0 6 .
13 0 0
We next determine whether this initial basic feasible solution is an optimal
solution, and, if not, how to adjust the basis to obtain a solution of lower
cost.
We determine u1 , u2 , u3 , u4 and v1 , v2 , v3 such that ci,j = vj − ui for all
(i, j) ∈ B, where B is the initial basis.
We seek a solution with u1 = 0. We then determine qi,j so that ci,j =
vj − ui + qi,j for all i and j.
31
We therefore complete the following tableau:—
ci,j & qi,j
1
2
3
4
vj
1
8
2
3
ui
• 4 • 16
0
0
0
?
3
7
2 • ?
?
?
0
13 • 8
6 • ?
0
?
0
5 • 7
8
?
0
?
?
?
?
?
Now u1 = 0, (1, 1) ∈ B and (1, 2) ∈ B force v1 = 8 and v2 = 4.
Then v1 = 8, (3, 1) ∈ B and (4, 1) ∈ B force u3 = −5 and u4 = 3.
Then u3 = −5 and (3, 3) ∈ B force v3 = 1.
Then v3 = 1 and (2, 3) ∈ B force u2 = −1.
After entering the numbers ui and vj , the tableau is as follows:—
ci,j & qi,j
1
2
3
4
vj
1
8
2
3
ui
• 4 • 16
0
0
0
?
3
7
2 • −1
?
?
0
13 • 8
6 • −5
0
?
0
5 • 7
8
3
0
?
?
8
4
1
Computing the numbers qi,j such that ci,j + ui = vj + qi,j , we find that
q1,3 = 15, q2,1 = −6, q2,2 = 2, q3,2 = −1, q4,2 = 6 and q4,3 = 10.
32
The completed tableau is as follows:—
ci,j & qi,j
1
1
8
2
3
3
13
4
5
vj
8
2
4
•
0
3
16
•
0
7
2
−6
2
• 8
0
−1
• 7
0
6
4
6
8
ui
0
15
• −1
0
• −5
0
3
10
1
The initial basic feasible solution is not optimal because some of the
quantities qi,j are negative. To see this, suppose that the numbers xi,j for
i = 1, 2, 3, 4 and j = 1, 2, 3 constitute a feasible solution to the given problem.
3
4
P
P
Then
xi,j = si for i = 1, 2, 3 and
xi,j = dj for j = 1, 2, 3, 4. It follows
j=1
i=1
that
4 X
3
X
ci,j xi,j =
i=1 j=1
4 X
3
X
(vj − ui + qi,j )xi,j
i=1 j=1
=
3
X
vj dj −
j=1
4
X
i=1
u i si +
4 X
3
X
qi,j xi,j .
i=1 j=1
Applying
thisPidentity to the initial basic feasible solution, we find that
P3
4
v
d
i=1 ui si = 238, given that 238 is the cost of the initial baj=1 j j −
sic feasible solution. Thus the cost C of any feasible solution (xi,j ) satisfies
C = 238 + 15x1,3 − 6x2,1 + 2x2,2 − x3,2 + 6x4,2 + 10x4,3 .
One could construct feasible solutions with x2,1 < 0 and xi,j = 0 for
(i, j) 6∈ B ∪ {(2, 1)}, and the cost of such feasible solutions would be lower
than that of the initial basic solution. We therefore seek to bring (2, 1) into
the basis, removing some other element of the basis to ensure that the new
basis corresponds to a feasible basic solution.
The procedure for achieving this requires us to determine a 4×3 matrix Y
satisfying the following conditions:—
• y2,1 = 1;
• yi,j = 0 when (i, j) 6∈ B ∪ {(2, 1)};
33
• all rows and columns of the matrix Y sum to zero.
Accordingly we fill in the following tableau with those coefficients yi,j of
the matrix Y that correspond to cells in the current basis (marked with the
• symbol), so that all rows sum to zero and all columns sum to zero:—
yi,j
1
2
3
4
1
?
1
?
?
0
2
• ? •
◦
•
•
0
3
0
? • 0
? • 0
0
0
0
The constraints that y2,1 = 1, yi,j = 0 when (i, j) 6∈ B and the constraints
requiring the rows and columns to sum to zero determine the values of yi,j
for all yi,j ∈ B. These values are recorded in the following tableau:—
yi,j
1
2
3
4
1
0
1
−1
0
0
2
3
• 0 •
◦
−1 •
•
1 •
•
0
0
0
0
0
0
0
We now determine those values of λ for which X+λY is a feasible solution,
where


1
12
0
 λ
0 8−λ 

X + λY = 
 5 − λ 0 6 + λ .
13
0
0
In order to drive down the cost as far as possible, we should make λ as
large as possible, subject to the requirement that all the coefficients of the
above matrix should be non-negative numbers. Accordingly we take λ = 5.
Our new basic feasible solution X is then as follows:—


1 12 0
 5 0 3 

X=
 0 0 11  .
13 0 0
34
We regard X of as the current feasible basic solution.
The cost of the current feasible basic solution X is
8 × 1 + 4 × 12 + 3 × 5 + 2 × 3 + 6 × 11
+ 5 × 13
= 8 + 48 + 15 + 6 + 66 + 65
= 208.
The cost has gone down by 30, as one would expect (the reduction in the
cost being −λq2,1 where λ = 5 and q2,1 = −6).
The current basic feasible solution X is associated with the basis B where
B = {(1, 1), (1, 2), (2, 1), (2, 3), (3, 3), (4, 1)}.
We now compute, for the current feasible basic solution We determine, for
the current basis B values u1 , u2 , u3 , u4 and v1 , v2 , v3 such that ci,j = vj − ui
for all (i, j) ∈ B. the initial basis.
We seek a solution with u1 = 0. We then determine qi,j so that ci,j =
vj − ui + qi,j for all i and j.
We therefore complete the following tableau:—
ci,j & qi,j
1
2
3
4
vj
1
8
2
3
ui
• 4 • 16
0
0
0
?
3 • 7
2 • ?
0
?
0
13
8
6 • ?
?
?
0
5 • 7
8
?
0
?
?
?
?
?
Now u1 = 0, (1, 1) ∈ B and (1, 2) ∈ B force v1 = 8 and v2 = 4.
Then v1 = 8, (2, 1) ∈ B and (4, 1) ∈ B force u2 = 5 and u4 = 3.
Then u2 = 5 and (3, 3) ∈ B force v3 = 7.
Then v3 = 7 and (3, 3) ∈ B force u3 = 1.
Computing the numbers qi,j such that ci,j + ui = vj + qi,j , we find that
q1,3 = 9, q2,2 = 8, q3,1 = 6, q3,2 = 5, q4,2 = 6 and q4,3 = 4.
35
The completed tableau is as follows:—
ci,j & qi,j
1
2
3
4
vj
1
8
2
3
ui
• 4 • 16
0
0
0
9
3 • 7
2 • 5
0
8
0
13
8
6 • 1
6
5
0
5 • 7
8
3
0
6
4
8
4
7
All numbers qi,j are non-negative for the current feasible basic solution.
This solution is therefore optimal. Indeed, arguing as before we find that the
cost C of any feasible solution (xi,j ) satisfies
C = 208 + 9x1,3 + 8x2,2 + 6x3,1 + 5x3,2 + 6x4,2 + 4x4,3 .
We conclude that X is an basic optimal solution, where


1 12 0
 5 0 3 

X=
 0 0 11  .
13 0 0
36
3.9
A Result concerning the Construction of Bases for
the Transportation Problem
The following general proposition ensures that certain standard methods for
determining an initial basic solution of the Transportation Problem, including
the Northwest Corner Method and the Minimum Cost Method will succeed
in determining a basic feasible solution to the Transportation Problem.
Proposition 3.7 Let I = {1, 2, . . . , m} and J = {1, 2, . . . , n}, where m
and n are positive integers, let i1 , i2 , . . . , im+n−1 be elements of I and let
j1 , j2 , . . . , jm+n−1 be elements of J, and let
B = {(ik , jk ) : k = 1, 2, . . . , m + n − 1}.
Suppose that there exist subsets I0 , I1 , . . . , Im+n−1 of I and J0 , J1 , . . . , Jm+n+1
of J such that I0 = I, J0 = J, and such that, for each integer k between 1
and m + n − 1, exactly one of the following two conditions is satisfied:—
(i) ik 6∈ Ik , jk ∈ Jk , Ik−1 = Ik ∪ {ik } and Jk−1 = Jk ;
(ii) ik ∈ Ik , jk 6∈ Jk , Ik−1 = Ik and Jk−1 = Jk ∪ {jk };
Then, given any real numbers a1 , a1 , . . . , am and b1 , b2 , . . . , bn satisfying
m
X
ai =
i=1
n
X
bj ,
j=1
thereP
exist uniquely-determined
Preal numbers xi,j for all i ∈ I and j ∈ J such
that
xi,j = ai for all i ∈ I,
xi,j = bj for all j ∈ J, and xi,j = 0 whenever
j∈J
i∈I
(i, j) 6∈ B.
Proof We prove the result by induction on m + n. The result is easily seen
to be true when m = n = 1. Thus suppose as our inductive hypothesis that
the corresponding results are true when I and J are replaced by I1 and J1 ,
0
0
so
given
P that,
P 0 any real numbers ai for i ∈ I1 and bj for j ∈ J1 satisfying
0
ai =
bj , there exist uniquely-determined real numbers xi,j for i ∈ I1
i∈I1
j∈J1
P
P
and j ∈ J1 such that
xi,j = ai for all i ∈ I1 and
xi,j = bj for all j ∈ I1 .
j∈J1
i∈I1
We prove that the corresponding results are true for the given sets I and J.
Now the conditions in the statement of the Proposition ensure that either
i1 6∈ I1 or else j1 6∈ J1 .
Suppose that i1 6∈ I1 . Then I = I1 ∪ {i1 } and J1 = J. Now Ik and Jk are
subsets of I1 and J1 for k = 1, 2, . . . , m+n−1. Moreover (ik , jk ) ∈ Ik−1 ×Jk−1
37
for all integers k satisfying 1 ≤ k ≤ m + n + 1. It follows that ik ∈ I1 and
therefore ik 6= i1 whenever 2 ≤ k ≤ m. It follows that the conclusions of
the proposition are true if and only if there exist uniquely-determined real
numbers xi,j for i ∈ I and j ∈ I such that
xi1 ,j1 = ai1 ,
xi ,j = 0 whenever j 6= j1 ,
X 1
xi,j = ai whenever i 6= i1 ,
j∈J
X
xi,j1 = bj1 − ai1 ,
i∈I1
X
xi,j = bj whenever j 6= j1 ,
i∈I1
xi,j = 0 whenever (i, j) 6∈ B
The induction hypothesis ensures the existence and uniqueness of the real
numbers xi,j for i ∈ I1 and j ∈ J determined so as to satisfy the above
conditions. Thus the induction hypothesis ensures that the required result
is true in the case where i1 6∈ I1 .
An analogous argument shows that the required result is true in the case
where j1 6∈ J1 . The result follows.
Proposition 3.7 ensures that if I = {1, 2, . . . , m} and J = {1, 2, . . . , n}
and if a subset B of I × J is determined so as to satisfy the requirements of
Proposition 3.7, then that subset B of I × J is a basis for the Transportation
Problem with m suppliers and n recipients.
The algorithms underlying the Minimal Cost Method and the Northwest
Corner Method give rise to subsets Ik and Jk of I and J respectively for
k = 0, 1, 2, . . . , m + n − 1 that satisfy the conditions of Proposition 3.7. This
proposition therefore ensures that Minimal Cost Method and the Northwest
Corner Method do indeed determine basic feasible solutions to the Transportation Problem.
Remark One can prove a converse result to Proposition 3.7 which establishes that, given any basis B for an instance of the Transportation Problem
with m suppliers and n recipients, there exist subsets Ik of I and Jk of J,
for i = 1, 2, . . . , m + n − 1, where I = {1, 2, . . . , m} and J = {1, 2, . . . , n}, so
that these subsets Ik and Jk of I and J are related to one another and to the
basis B in the manner described in the statement of Proposition 3.7.
38
3.10
The Minimum Cost Method
We describe the Minimum Cost Method for finding an initial basic feasible
solution to the Transportation Problem.
Consider an instance of the Transportation Problem specified by positive integers m and n and non-negative real numbers s1 , s2 , . . . , sm and
m
n
P
P
d1 , d2 , . . . , dn , where
si =
dj . Let I = {1, 2, . . . , m} and let J =
j=1
i=1
{1, 2, . . . , n}. A feasible solution consists of an array of P
non-negative real
numbers xi,j for i ∈ I and j ∈ J with the property that
xi,j = si for all
j∈J
P
i ∈ I and
xi,j = dj for all j ∈ J. The objective of the problem is to find
i∈I
a feasible solution that minimizes
cost, where the cost of a feasible solution
PP
(xi,j : i ∈ I andj ∈ J) is
ci,j xi,j .
i∈I j∈J
In applying the Minimal Cost Method to find an initial basic solution to
the Transportation we apply an algorithm that corresponds to the determination of elements (i1 , j1 ), (i2 , j2 ), . . . , (im+n−1 , jm+n−1 ) of I ×J and of subsets
I0 , I1 , . . . , Im+n−1 of I and J0 , J1 , . . . , Jm+n−1 of J such that the conditions of
Proposition 3.7 are satisfied.
Indeed let I0 = I, J0 = J and B0 = {0}. The Minimal Cost Method
algorithm is accomplished in m + n − 1 stages.
Let k be an integer satisfying 1 ≤ k ≤ m + n − 1 and that subsets Ik−1 of
I, Jk−1 of J and Bk−1 of I × J have been determined in accordance with the
rules that apply at previous stages of the Minimal Cost algorithm. Suppose
also that non-negative real numbers xi,j have been determined for all ordered
pairs (i, j) in I × J that satisfy either i 6∈ Ik−1 or j 6∈ Jk−1 so as to satisfy
the following conditions:—
P
•
xi,j ≤ si whenever i ∈ Ik−1 ;
j∈J\Jk−1
•
P
xi,j = si whenever i 6∈ Ik−1 ;
j∈J
•
P
xi,j ≤ dj whenever j ∈ Jk−1 ;
i∈I\Ik−1
•
P
xi,j = dj whenever j 6∈ Jk−1 .
i∈I
The Minimal Cost Method specifies that one should choose (ik , jk ) ∈ Ik−1 ×
Jk−1 so that
cik ,jk ≤ ci,j for all (i, j) ∈ Ik−1 × Jk−1 ,
39
and set Bk = Bk−1 ∪ {(ik , jk )}. Having chosen (ik , jk ), the non-negative real
number xik ,jk is then determined so that


X
X
xik ,jk = min sik −
xik ,j , djk −
xi,jk  .
j∈J\Jk−1
i∈I\Ik−1
The subsets Ik and Jk of I and J respectively are then determined, along
with appropriate values of xi,j , according to the following rules:—
(i) if
sik −
X
xik ,j < djk −
j∈J\Jk−1
X
xi,jk
i∈I\Ik−1
then we set Ik = Ik−1 \ {ik } and Jk = Jk−1 , and we also let xik ,j = 0
for all j ∈ Jk−1 \ {jk };
(ii) if
sik −
X
xik ,j > djk −
X
xi,jk
i∈I\Ik−1
j∈J\Jk−1
then we set Jk = Jk−1 \ {jk } and Ik = Ik−1 , and we also let xi,jk = 0
for all i ∈ Ik−1 \ {ik };
(iii) if
sik −
X
xik ,j = djk −
j∈J\Jk−1
X
xi,jk
i∈I\Ik−1
then we determine Ik and Jk and the corresponding values of xi,j either in accordance with the specification in rule (i) above or else in
accordance with the specification in rule (ii) above.
These rules ensure that the real numbers xi,j determined at this stage are all
non-negative, and that the following conditions are satisfied at the conclusion
of the kth stage of the Minimal Cost Method algorithm:—
P
•
xi,j ≤ si whenever i ∈ Ik ;
j∈J\Jk
•
P
xi,j = si whenever i 6∈ Ik ;
j∈J
•
P
xi,j ≤ dj whenever j ∈ Jk ;
i∈I\Ik
•
P
xi,j = dj whenever j 6∈ Jk .
i∈I
40
At the completion of the final stage (for which k = m + n − 1) we have
determined a subset B of I × J, where B = Bm+n−1 , together with nonnegative real numbers xi,j for i ∈ I and j ∈ I that constitute a feasible
solution to the given instance of the Transportation Problem. Moreover
Proposition 3.7 ensures that this feasible solution is a basic feasible solution
of the problem with associated basis B.
3.11
The Northwest Corner Method
The Northwest Corner Method for finding a basic feasible solution proceeds
according to the stages of the Minimum Cost Method above, differing only
from that method in the choice of the ordered pair (ik , jk ) at the kth stage
of the method. In the Minimum Cost Method, the ordered pair (ik , jk ) is
chosen such that (ik , jk ) ∈ Ik−1 × Jk−1 and
cik ,jk ≤ ci,j
for all (i, j) ∈ Ik−1 × Jk−1
(where the sets Ik−1 , Jk−1 are determined as in the specification of the Minimum Cost Method). In applying the Northwest Corner Method, costs associated with ordered pairs (i, j) in I × J are not taken into account. Instead
(ik , jk ) is chosen so that ik is the minimum of the integers in Ik−1 and jk is the
minimum of the integers in Jk−1 . Otherwise the specification of the Northwest Corner Method corresponds to that of the Minimum Cost Method, and
results in a basic feasible solution of the given instance of the Transportation
Problem.
3.12
The Iterative Procedure for Solving the Transportation Problem, given an Initial Basic Feasible
Solution
We now describe in general terms the method for solving the Transportation
Problem, in the case where total supply equals total demand.
We suppose that an initial basic feasible solution has been obtained. We
apply an iterative method (based on the general Simplex Method for the
solution of linear programming problems) that will test a basic feasible solution for optimality and, in the event that the feasible solution is shown not
to be optimal, establishes information that (with the exception of certain
‘degenerate’ cases of the Transportation Problem) enables one to find a basic
feasible solution with lower cost. Iterating this procedure a finite number of
times, one should arrive at a basic feasible solution that is optimal for the
given instance of the the Transportation Problem.
41
We suppose that the given instance of the Transportation Problem involves m suppliers and n recipients. The required supplies are specified by
non-negative real numbers s1 , s2 , . . . , sm , and the required demands are specified by non-negative real numbers d1 , d2 , . . . , dn . We further suppose that
m
P
Pn
si =
j=1 dj . A feasible solution is represented by non-negative real
i=1
numbers xi,j for i = 1, 2, . . . , m and j = 1, 2, . . . , n, where
n
P
xi,j = si for
j=1
i = 1, 2, . . . , m and
m
P
xi,j = dj for j = 1, 2, . . . , n.
i=1
Let I = {1, 2, . . . , m} and J = {1, 2, . . . , n}. A subset B of I × J is a
basis for the Transportation Problem if and only if, given any real numbers
y1 , y2 , . . . , ym and z1 , z2 , . . . , zn , there exist uniquely determined real numbers
n
m
P
P
xi,j for i ∈ I and j ∈ J such that
xi,j = yi for i ∈ I,
xi,j = zj for j ∈ J,
j=1
i=1
where xi,j = 0 whenever (i, j) 6∈ B (see Proposition 3.4).
A feasible solution (xi,j ) is said to be a basic feasible solution associated
with the basis B if and only if xi,j = 0 for all i ∈ I and j ∈ J for which
(i, j) 6∈ B.
Let xi,j be a non-negative real number for each i ∈ I and j ∈ J. Suppose that (xi,j ) is a basic feasible solution to the Transportation Problem
associated with basis B, where B ⊂ I × J.
m P
n
P
The cost associated with a feasible solution (xi,j is given by
ci,j xi,j ,
i=1 j=1
where the constants ci,j are real numbers for all i ∈ I and j ∈ J. A feasible
solution for the given instance of the Transportation Problems is an optimal
solution if and only if it minimizes cost amongst all feasible solutions to the
problem.
In order to test for optimality of a basic feasible solution (xi,j ) associated
with a basis B, we determine real numbers u1 , u2 , . . . , um and v1 , v2 , . . . , vn
with the property that ci,j = vj − ui for all (i, j) ∈ B. (Proposition 3.10
below guarantees that, given any basis B, it is always possible to find the
required quantities ui and vj .) Having calculated these quantities ui and vj
we determine the values of qi,j , where qi,j = ci,j − vj + ui for all i ∈ I and
j ∈ J. Then qi,j = 0 whenever (i, j) ∈ B.
We claim that a basic feasible solution (xi,j ) associated with the basis B
is optimal if and only if qi,j ≥ 0 for all i ∈ I and j ∈ J. This is a consequence
of the identity established in the following proposition.
Proposition 3.8 Let xi,j , ci,j and qi,j be real numbers defined for i = 1, 2, . . . , m
and j = 1, 2, . . . , n, and let u1 , u2 , . . . , um and v1 , v2 , . . . , vn be real numbers.
42
Suppose that
ci,j = vj − ui + qi,j
for i = 1, 2, . . . , m and j = 1, 2, . . . , n. Then
m X
n
X
ci,j xi,j =
i=1 j=1
where si =
Pn
j=1
m
X
vj dj −
i=1
n
X
m X
n
X
ui si +
j=1
qi,j xi,j ,
i=1 j=1
Pm
xi,j for i = 1, 2, . . . , m and dj =
i=1
xi,j for j = 1, 2, . . . , n.
Proof The definitions of the relevant quantities ensure that
m X
n
X
ci,j xi,j =
i=1 j=1
m X
n
X
(vj − ui + qi,j )xi,j
i=1 j=1
=
n
X
vj
j=1
+
=
m
X
m
X
!
xi,j
i=1
m
n
XX
ui
n
X
i=1
!
xi,j
j=1
qi,j xi,j
i=1 j=1
n
X
v j dj −
i=1
−
m
X
ui si +
j=1
m X
n
X
qi,j xi,j ,
i=1 j=1
as required.
Corollary 3.9 Let m and n be integers, and let I = {1, 2, . . . , m} and J =
{1, 2, . . . , n}. Let xi,j and ci,j be real numbers defined for all i ∈ I and
j ∈ I, and let u1 , u2 , . . . , um and v1 , v2 , . . . , vn be real numbers. Suppose that
ci,j = vj − ui for all (i, j) ∈ I × J for which xi,j 6= 0. Then
m X
n
X
ci,j xi,j =
i=1 j=1
where si =
n
P
m
X
dj vj −
i=1
xi,j for i = 1, 2, . . . , m and dj =
j=1
n
X
si ui ,
j=1
m
P
xi,j for j = 1, 2, . . . , n.
i=1
Proof Let qi,j = ci,j + ui − vj for all i ∈ I and j ∈ J. Then qi,j = 0 whenever
xi,j 6= 0. It follows from this that
m X
n
X
qi,j xi,j = 0.
i=1 j=1
43
It then follows from Proposition 3.8 that
n
m X
X
ci,j xi,j =
i=1 j=1
m X
n
m
n
X
X
X
(vj − ui + qi,j )xi,j =
dj vj −
si u i ,
i=1 j=1
i=1
j=1
as required.
Let m and n be positive integers, let I = {1, 2, . . . , m} and J = {1, 2, . . . , n},
and let the subset B of I × J be a basis for an instance of the Transportation
Problem with m suppliers and n recipients. Let the cost of a feasible solution
m P
n
n
m
P
P
P
(xi,j ) be
ci,j xi,j . Now
xi,j = si and
xi,j = dj , where the quantii=1 j=1
j=1
i=1
ties si and dj are determined by the specification of the problem and are the
same for all feasible solutions of the problem. Let quantities ui for i ∈ I and
vj for j ∈ J be determined such that ci,j = vj − ui for all (i, j) ∈ B, and let
qi,j = ci,j + ui − vj for all i ∈ I and j ∈ J. Then qi,j = 0 for all (i, j) ∈ B.
It follows from Proposition 3.8 that
m X
n
X
ci,j xi,j =
i=1 j=1
m
X
vj dj −
i=1
n
X
ui si +
m X
n
X
j=1
qi,j xi,j .
i=1 j=1
Now if the quantities xi,j for i ∈ I and j ∈ J constitute a basic feasible
solution associated with the basis B then xi,j = 0 whenever (i, j) 6∈ B. It
m P
n
P
follows that
qi,j xi,j = 0, and therefore
i=1 j=1
m
X
v j dj −
i=1
n
X
ui si = C,
j=1
where
C=
m X
n
X
ci,j xi,j .
i=1 j=1
The cost C of the feasible solution (xi,j ) then satisfies the equation
C=
m X
n
X
ci,j xi,j = C +
i=1 j=1
m X
n
X
qi,j xi,j .
i=1 j=1
If qi,j ≥ 0 for all i ∈ I and j ∈ J, then the cost C of any feasible solution
(xi,j ) is bounded below by the cost of the basic feasible solution (xi,j ). It
follows that, in this case, the basic feasible solution (xi,j ) is optimal.
44
Suppose that (i0 , j0 ) is an element of I × J for which qi0 ,j0 < 0. Then
(i0 , j0 ) 6∈ B. There is no basis for the Transportation Problem that includes
the set B ∪{(i0 , j0 )}. A straightforward application of Lemma 3.5 establishes
the existence of quantities yi,j for i ∈ I and j ∈ J such that yi0 ,j0 = 1 and
yi,j = 0 for all i ∈ I and j ∈ J for which (i, j) 6∈ B ∪ {(i0 , j0 )}.
Let the m × n matrices X and Y be defined so that (X)i,j = xi,j and
(Y )i,j = yi,j for all i ∈ I and j ∈ J. Suppose that xi,j > 0 for all (i, j) ∈
B. Then the components of X in the basis positions are strictly positive.
It follows that, if λ is positive but sufficiently small, then the components
of the matrix X + λY in the basis positions are also strictly positive, and
therefore the components of the matrix X + λY are non-negative for all
sufficiently small non-negative values of λ. There will then exist a maximum
value λ0 that is an upper bound on the values of λ for which all components
of the matrix X + λY are non-negative. It is then a straightforward exercise
in linear algebra to verify that X + λ0 Y is another basic feasible solution
associated with a basis that includes (i0 , j0 ) together with all but one of the
elements of the basis B. Moreover the cost of this new basic feasible solution
is C + λ0 qi0 ,j0 , where C is the cost of the basic feasible solution represented
by the matrix X. Thus if qi0 ,j0 < 0 then the cost of the new basic feasible
solution is lower than that of the basic feasible solution X from which it was
derived.
Suppose that, for all basic feasible solutions of the given Transportation
problem, the coefficients of the matrix specifying the basic feasible solution
are strictly positive at the basis positions. Then a finite number of iterations
of the procedure discussed above with result in an basic optimal solution of
the given instance of the Transportation Problem. Such problems are said
to be non-degenerate.
However if it turns out that a basic feasible solution (xi,j ) associated with
a basis B satisfies xi,j = 0 for some (i, j) ∈ B, then we are in a degenerate
case of the Transportation Problem. The theory of degenerate cases of linear
programming problems is discussed in detail in textbooks that discuss the
details of linear programming algorithms.
We now establish the proposition that guarantees that, given any basis B,
there exist quantities u1 , u2 , . . . , um and v1 , v2 , . . . , vn such that the costs ci,j
associated with the given instance of the Transportation Problem satisfy
ci,j = vj − ui for all (i, j) ∈ B. This result is an essential component of
the method described here for testing basic feasible solutions to determine
whether or not they are optimal.
Proposition 3.10 Let m and n be integers, let I = {1, 2, . . . , m} and J =
{1, 2, . . . , n}, and let B be a subset of I × J that is a basis for the transporta45
tion problem with m suppliers and n recipients. For each (i, j) ∈ B let ci,j be
a corresponding real number. Then there exist real numbers ui for i ∈ I and
vj for j ∈ J such that ci,j = vj − ui for all (i, j) ∈ B. Moreover if ui and v j
are real numbers for i ∈ I and j ∈ J that satisfy the equations ci,j = v j − ui
for all (i, j) ∈ B, then there exists some real number k such that ui = ui + k
for all i ∈ I and v j = vj + k for all j ∈ J.
Proof Let
(
W =
(y, z) ∈ Rm × Rn :
m
P
(y)i =
i=1
n
P
)
(z)j
,
j=1
let ρ: Mm,n (R) → Rm and σ: Mm,n (R) → Rn be the linear transformations
n
m
P
P
(X)i,j for i = 1, 2, . . . , m and σ(X)j = (X)i,j
defined such that ρ(X)i =
j=1
i=1
for j = 1, 2, . . . , n, let
MB = {X ∈ Mm,n (R) : (X)i,j = 0 whenever (i, j) 6∈ B},
and let C be the m × n matrix defined such that (C)i,j = ci,j for all i ∈ I
and j ∈ J.
Now, given any element (y, z) of W , there exists a uniquely-determined
n
m
P
P
m×n matrix X such that (X)i,j = (y)i for i = 1, 2, . . . , m, (X)i,j = (z)j
j=1
i=1
for j = 1, 2, . . . , n and (X)i,j = 0 unless (i, j) ∈ B (see Proposition 3.4).
Then X is the unique matrix belonging to MB that satisfies ρ(X) = y and
σ(X) = z. We define
T
g(y, z) = trace(C X) =
m X
n
X
ci,j (X)i,j .
i=1 j=1
We obtain in this way a well-defined function g: W → R characterized by the
property that
g(ρ(X), σ(X)) = trace(C T X)
for all X ∈ MB . Now ρ(λX) = λy and σ(λX) = λz for all real numbers λ.
It follows that
g(λ(y, z)) = λg(y, z)
for all real numbers λ. Also, given elements (y0 , z0 ) and (y00 , z00 ) of W , there
exist unique matrices X 0 and X 00 belonging to MB such that ρ(X 0 ) = y0 ,
46
ρ(X 00 ) = y00 , σ(X 0 ) = z0 and σ(X 00 ) = z00 . Then ρ(X 0 + X 00 ) = y0 + y00 and
σ(X 0 + X 00 ) = z0 + z00 , and therefore
g((y0 , z0 ) + (y00 , z00 )) = trace(C T (X 0 + X 00 ))
= trace(C T X 0 ) + trace(C T X 00 )
= g(y0 , z0 ) + g(y00 , z00 ).
It follows that the function g: W → R is a linear transformation. It is thus a
linear functional on the real vector space W .
For each integer i between 1 and m, let b(i) denote the vector in Rm whose
ith component is equal to 1 and whose other components are zero, and, for
each integer j between 1 and n, let b(j) denote the vector in Rn whose
jth component is equal to 1 and whose other components are zero. Then
(b(1) −b(i) , 0) ∈ W for i = 1, 2, . . . , m and (b(1) , b(j) ) ∈ W for j = 1, 2, . . . , n.
We define ui = g(b(1) − b(i) , 0) for i = 1, 2, . . . , m and vj = g(b(1) , b(j) ) for
j = 1, 2, . . . , n. Then u1 = 0 and
vj − ui = g(b(1) , b(j) ) − g(b(1) − b(i) , 0)
= g (b(1) , b(j) ) − (b(1) − b(i) , 0)
= g(b(i) , b(j) )
for all i ∈ I and j ∈ J.
If (i, j) ∈ B then b(i) = ρ(E (i,j) ) and b(j) = σ(E (i,j) ), where E (i,j) is the
m × n matrix whose coefficient in the ith row and jth column is equal to 1
and whose other coefficients are zero. Moreover E (i,j) ∈ MB for all (i, j) ∈ B.
It follows from the definition of the linear functional g that
vj − ui = g(b(i) , b(j) ) = trace(C T E (i,j) ) = ci,j
for all (i, j) ∈ B.
Now let ui and v j be real numbers for i ∈ I and j ∈ J that satisfy the
equations ci,j = v j − ui for all (i, j) ∈ B. Let
g(y, z) =
n
X
vj (z)j −
j=1
m
X
ui (y)i
i=1
for all y ∈ Rm and z ∈ Rn . Then
g(ρ(E (i,j) ), σ(E (i,j) )) = g(b(i) , b(j) ) = v j − ui
for all (i, j) ∈ I × J. It follows that
g(ρ(E (i,j) ), σ(E (i,j) )) = v j −ui = ci,j = g(ρ(E (i,j) ), σ(E (i,j) )) for all (i, j) ∈ B.
47
Now the matrices E (i,j) for all (i, j) ∈ B constitute a basis of the vector space
MK . It follows that
g(ρ(X), σ(X)) = g(ρ(X), σ(X))
for all X ∈ MB . But every element of the vector space W is of the form
(ρ(X), σ(X)) for some X ∈ MB . (This follows Proposition 3.4, as discussed
earlier in the proof.) Thus
g(y, z) = g(y, z)
for all (y, z) ∈ W . In particular
ui − u1 = g(b(1) − b(i) , 0) = g(b(1) − b(i) , 0) = ui − u1
for all i ∈ I, and
v j − u1 = g(b(1) , b(j) ) = g(b(1) , b(j) ) = vj − u1
for all j ∈ J. Let k = u1 − u1 . Then ui = ui + k for all i ∈ I and v j = vj + k
for all j ∈ J, as required.
48
4
4.1
The Simplex Method
Vector Inequalities and Notational Conventions
Let v be an element of the real vector space Rn . We denote by (v)j the jth
component of the vector v. The vector v can be represented in the usual
fashion as an n-tuple (v1 , v2 , . . . , vn ), where vj = (v)j for j = 1, 2, . . . , n.
However where an n-dimensional vector appears in matrix equations it will
usually be considered to be an n × 1 column vector. The row vector corresponding to an element v of Rn will be denoted by vT because, considered as
a matrix, it is the transpose of the column vector representing v. We denote
the zero vector (in the appropriate dimension) by 0.
Let x and y be vectors belonging to the real vector space Rn for some
positive integer n. We write x ≤ y (and y ≥ x) when (x)j ≤ (y)j for
j = 1, 2, . . . , n. Also we write x y (and y x) when (x)j < (y)j for
j = 1, 2, . . . , n.
These notational conventions ensure that x ≥ 0 if and only if (x)j ≥ 0
for j = 1, 2, . . . , n.
The scalar product of two n-dimensional vectors u and v can be represented as the matrix product uT v. Thus
uT v = u1 v1 + u2 v2 + · · · + un vn
for all u, v ∈ Rn , where uj = (u)j and vj = (v)j for j = 1, 2, . . . , n.
Given an m × n matrix A, where m and n are positive integers, we denote
by (A)i,j the coefficient in the ith row and jth column of the matrix A.
4.2
Feasible and Optimal Solutions of General Linear
Programming Problems
A general linear programming problem is one that seeks values of real variables x1 , x2 , . . . , xn that maximize or minimize some objective function
c1 x 1 + c2 x 2 + · · · cn x n
that is a linear functional of x1 , x2 , . . . , xn determined by real constants
c1 , c2 , . . . , cn , where the variables x1 , x2 , . . . , xn are subject to a finite number
of constraints that each place bounds on the value of some linear functional
of the variables. These constraints can then be numbered from 1 to m, for
an appropriate value of m such that, for each value of i between 1 and m,
the ith constraint takes the form of an equation or inequality that can be
expressed in one of the following three forms:—
ai,1 x1 + ai,2 x2 + · · · + ai,n xn = bi ,
49
ai,1 x1 + ai,2 x2 + · · · + ai,n xn ≥ bi ,
ai,1 x1 + ai,2 x2 + · · · + ai,n xn ≤ bi
for appropriate values of the real constants ai,1 , ai,2 , . . . , ai,n and bi . In addition some, but not necessarily all, of the variables x1 , x2 , . . . , xn may be
required to be non-negative. (Of course a constraint requiring a variable to
be non-negative can be expressed by an inequality that conforms to one of
the three forms described above. Nevertheless constraints that simply require some of the variables to be non-negative are usually listed separately
from the other constraints.)
Definition Consider a general linear programming problem with n real variables x1 , x2 , . . . , xn whose objective is to maximize or minimize some objective function subject to appropriate constraints. A feasible solution of this
linear programming problem is specified by an n-dimensional vector x whose
components satisfy the constraints but do not necessarily maximize or minimize the objective function.
Definition Consider a general linear programming problem with n real variables x1 , x2 , . . . , xn whose objective is to maximize or minimize some objective function subject to appropriate constraints. A optimal solution of this
linear programming problem is specified by an n-dimensional vector x that is
a feasible solution that optimizes the value of the objective function amongst
all feasible solutions to the linear programming problem.
4.3
Linear Programming Problems in Dantzig Standard Form
Let A be an m × n matrix of rank m with real coefficients, where m ≤ n,
and let b ∈ Rm and c ∈ Rn be vectors of dimensions m and n respectively.
We consider the following linear programming problem:—
Determine an n-dimensional vector x so as to minimize cT x subject to the constraints Ax = b and x ≥ 0.
We refer to linear programming problems presented in this form as being in
Dantzig standard form. We refer to the m × n matrix A, the m-dimensional
vector b and the n-dimensional vector c as the constraint matrix, target vector
and cost vector for the linear programming problem.
Remark Nomenclature in Linear Programming textbooks varies. Problems
presented in the above form are those to which the basic algorithms of George
50
B. Dantzig’s Simplex Method are applicable. In the series of textbooks by
George B. Dantzig and Mukund N. Thapa entitled Linear Programming,
such problems are said to be in standard form. In the textbook Introduction to Linear Programming by Richard B. Darst, such problems are said
to be standard-form LP. On the other hand, in the textbook Methods of
Mathematical Economics by Joel N. Franklin, such problems are said to be
in canonical form, and the term standard form is used for problems which
match the form above, except that the vector equality Ax = b is replaced
by a vector inequality Ax ≥ b. Accordingly the term Danztig standard form
is used in these notes both to indicate that such problems are in standard
form at that term is used by textbooks of which Dantzig is the author, and
also to emphasize the connection with the contribution of Dantzig in creating
and popularizing the Simplex Method for the solution of linear programming
problems.
A linear programming problem in Dantzig standard form specified by an
m × n constraint matrix A of rank m, an m-dimensional target vector b
and an n-dimensional cost vector c has the objective of finding values of real
variables x1 , x2 , . . . , xn that minimize the value of the cost
c1 x 1 + c2 x 2 + · · · + cn x n
subject to constraints
A1,1 x1 + A1,2 x2 + · · · + A1,n xn = b1 ,
A2,1 x1 + A2,2 x2 + · · · + A2,n xn = b2 ,
..
.
Am,1 x1 + Am,2 x2 + · · · + Am,n xn = bm
and
x1 ≥ 0,
x2 ≥ 0, . . . ,
xn ≥ 0.
In this above programming problem, the function sending the n-dimensional
vector x to the corresponding cost cT x is the objective function for the problem. A feasible solution to the problem consists of an n-dimensional vector
(x1 , x2 , . . . , xn ) whose components satisfy the above constraints but do not
necessarily minimize cost. An optimal solution is a feasible solution whose
cost does not exceed that of any other feasible solution.
51
4.4
Basic Feasible Solutions to Linear Programming
Problems in Dantzig Standard Form
We define the notion of a basis for a linear programming problem in Dantzig
standard form.
Definition Let A be an m×n matrix of rank m with real coefficients, where
m ≤ n, let b ∈ Rm be an m-dimensional column vector, let c ∈ Rn be an
n-dimensional column vector. Consider the following programming problem
in Dantzig standard form:
find x ∈ Rn so as to minimize cT x subject to constraints Ax = b
and x ≥ 0.
For each integer j between 1 and n, let a(j) denote the m-dimensional vector
determined by the jth column of the matrix A, so that (a(j) )i = (A)i,j for
i = 1, 2, . . . , m and j = 1, 2, . . . , n. A basis for this linear programming
problem is a set consisting of m distinct integers j1 , j2 , . . . , jm between 1 and
n for which the corresponding vectors
a(j1 ) , a(j2 ) , . . . , a(jm )
constitute a basis of the vector space Rm .
We next define what is meant by saying that a feasible solution of a
programming problem Dantzig standard form is a basic feasible solution for
the programming problem.
Definition Let A be an m×n matrix of rank m with real coefficients, where
m ≤ n, let b ∈ Rm be an m-dimensional column vector, let c ∈ Rn be an
n-dimensional column vector. Consider the following programming problem
in Dantzig standard form:—
find x ∈ Rn so as to minimize cT x subject to constraints Ax = b
and x ≥ 0.
A feasible solution x for this programming problem is said to be basic if there
exists a basis B for the linear programming problem such that (x)j = 0 when
j 6∈ B.
Lemma 4.1 Let A be an m×n matrix of rank m with real coefficients, where
m ≤ n, let b ∈ Rm be an m-dimensional column vector, let c ∈ Rn be an
n-dimensional column vector. Consider the following programming problem
in Dantzig standard form:
52
find x ∈ Rn so as to minimize cT x subject to constraints Ax = b
and x ≥ 0.
Let a(j) denote the vector specified by the jth column of the matrix A for
j = 1, 2, . . . , n. Let x be a feasible solution of the linear programming problem.
Suppose that the m-dimensional vectors a(j) for which (x)j > 0 are linearly
independent. Then x is a basic feasible solution of the linear programming
problem.
Proof Let x be a feasible solution to the programming problem, let xj = (x)j
for all j ∈ J, where J = {1, 2, . . . , n}, and let K = {j ∈ J : xj > 0}. If the
vectors a(j) for which j ∈ K are linearly independent then basic linear algebra
ensures that further vectors a(j) can be added to the linearly independent set
{a(j) : j ∈ K} so as to obtain a finite subset of Rm whose elements constitute
a basis of that vector space (see Proposition 2.2). Thus exists a subset B of
J satisfying K ⊂ B ⊂ J such that the m-dimensional vectors a(j) for which
j ∈ B constitute a basis of the real vector space Rm . Moreover (x)j = 0
for all j ∈ J \ B. It follows that x is a basic feasible solution to the linear
programming problem, as required.
Theorem 4.2 Let A be an m × n matrix of rank m with real coefficients,
where m ≤ n, let b ∈ Rm be an m-dimensional column vector, let c ∈ Rn
be an n-dimensional column vector. Consider the following programming
problem in Dantzig standard form:
find x ∈ Rn so as to minimize cT x subject to constraints Ax = b
and x ≥ 0.
If there exists a feasible solution to this programming problem then there exists
a basic feasible solution to the problem. Moreover if there exists an optimal
solution to the programming problem then there exists a basic optimal solution
to the problem.
Proof Let J = {1, 2, . . . , n}, and let a(j) denote the vector specified by the
jth column of the matrix A for all j ∈ J.
Let x be a feasible solution to the programming problem, let xj = (x)j
for all j ∈ J, and let K = {j ∈ J : xj > 0}. Suppose that x is not basic.
Then the vectors a(j) for which j ∈ K must be linearly dependent. We show
that there then exists a feasible solution with fewer non-zero components
than the given feasible solution x.
for j ∈ K, not all zero, such that
P Now(j)there exist real numbers yj (j)
yj a = 0, because the vectors a for j ∈ K are linearly dependent.
j∈K
53
Let yj = 0 for all j ∈ J \ K, and let y ∈ Rn be the n-dimensional vector
satisfying (y)j = yj for j = 1, 2, . . . , n. Then
X
X
Ay =
yj a(j) =
yj a(j) = 0.
j∈J
j∈K
It follows that A(x − λy) = b for all real numbers λ, and thus x − λy is
a feasible solution to the programming problem for all real numbers λ for
which x − λy ≥ 0.
Now y is non-zero vector. Replacing y by −y, if necessary, we can assume,
without loss of generality, that at least one component of the vector y is
positive. Let
xj
: j ∈ K and yj > 0 ,
λ0 = minimum
yj
xj
≥ λ0 for
yj
all j ∈ J for which yj > 0. Multiplying by the positive number yj , we find
that xj ≥ λ0 yj and thus xj − λ0 yj ≥ 0 when yj > 0. Also λ0 > 0 and
xj ≥ 0, and therefore xj − λ0 yj ≥ 0 when yj ≤ 0. Thus xj − λ0 yj ≥ 0 for
all j ∈ J. Also xj0 − λ0 yj0 = 0, and xj − λ0 yj = 0 for all j ∈ J \ K. Let
x0 = x − λ0 y. Then x0 ≥ 0 and Ax0 = b, and thus x0 is a feasible solution to
the linear programming problem with fewer non-zero components than the
given feasible solution.
Suppose in particular that the feasible solution x is optimal. Now there
exist both positive and negative values of λ for which x − λy ≥ 0. If it
were the case that cT y 6= 0 then there would exist values of λ for which
both x − λy ≥ 0 and λcT y > 0. But then cT (x − λy) < cT x, contradicting
the optimality of x. It follows that cT y = 0, and therefore x − λy is an
optimal solution of the linear programming problem for all values of λ for
which x − λy ≥ 0. The previous argument then shows that there exists a
real number λ0 for which x − λ0 y is an optimal solution with fewer non-zero
components than the given optimal solution x.
We have shown that if there exists a feasible solution x which is not basic
then there exists a feasible solution with fewer non-zero components than x.
It follows that if a feasible solution x is chosen such that it has the smallest
possible number of non-zero components then it is a basic feasible solution
of the linear programming problem.
Similarly we have shown that if there exists an optimal solution x which
is not basic then there exists an optimal solution with fewer non-zero components than x. It follows that if an optimal solution x is chosen such that
and let j0 be an element of K for which λ0 = xj0 /yj0 . Then
54
it has the smallest possible number of non-zero components then it is a basic
optimal solution of the linear programming problem.
4.5
A Simplex Method Example with Five Variables
and Two Constraints
We consider the following linear programming problem:—
minimize
3x1 + 4x2 + 2x3 + 9x4 + 5x5
subject to the following constraints:
5x1 + 3x2 + 4x3 + 7x4 + 3x5 = 11;
4x1 + x2 + 3x3 + 8x4 + 4x5 = 6;
xj ≥ 0 for j = 1, 2, 3, 4, 5.
The constraints require that x1 , x2 , x3 , x4 , x5 be non-negative real numbers
satisfying the matrix equation


x1
 x2  
5 3 4 7 3 
11
 x3  =
.

4 1 3 8 4 
6
 x4 
x5
Thus we are required to find a (column) vector x with components x1 ,
x2 , x3 , x4 and x5 satisfying the equation Ax = b, where
5 3 4 7 3
11
A=
, b=
.
4 1 3 8 4
6
Let
(1)
a
=
5
4
(4)
a
,
=
(2)
a
7
8
=
3
1
(3)
,
(5)
and a
a
=
=
3
4
4
3
,
.
For a feasible solution to the problem we must find non-negative real numbers
x1 , x2 , x3 , x4 , x5 such that
x1 a(1) + x2 a(2) + x3 a(3) + x4 a(4) + x5 a(5) = b.
An optimal solution to the problem is a feasible solution that minimizes
c1 x 1 + c2 x 2 + c3 x 3 + c4 x 4 + c5 x 5
55
amongst all feasible solutions to the problem, where c1 = 3, c2 = 4, c3 = 2,
c4 = 9 and c5 = 5.
Let c denote the column vector whose ith component is ci respectively.
Then
cT = 3 4 2 9 5 ,
and an optimal solution is a feasible solution that minimizes cT x amongst all
feasible solutions to the problem. We refer to the quantity cT x as the cost
of the feasible solution x.
Let I = {1, 2, 3, 4, 5}. A basis for this optimization problem is a subset
{j1 , j2 } of I, where j1 6= j2 , for which the corresponding vectors a(j1 ) , a(j2 )
constitute a basis of R2 . By inspection we see that each pair of vectors taken
from the list a(1) , a(2) , a(3) , a(4) , a(5) consists of linearly independent vectors,
and therefore each pair of vectors from this list constitutes a basis of R2 . It
follows that every subset of I with exactly two elements is a basis for the
optimization problem.
A feasible solution (x1 , x2 , x3 , x4 , x5 ) to this optimization problem is a
basic feasible solution if there exists a basis B for the optimization problem
such that xj = 0 when j 6= B.
In the case of the present problem, all subsets of {1, 2, 3, 4, 5} with exactly
two elements are bases for the problem. It follows that a feasible solution to
the problem is a basic feasible solution if and only if the number of non-zero
components of the solution does not exceed 2.
We take as given the following initial basic feasible solution x1 = 1,
x2 = 2, x3 = x4 = x5 = 0. One can readily verify that a(1) + 2a(2) = b. This
initial basic feasible solution is associated with the basis {1, 2}. The cost of
this solution is 11.
We apply the procedures of the simplex method to test whether or not
this basic feasible solution is optimal, and, if not, determine how to improve
it.
The basis {1, 2} determines a 2 × 2 minor MB of A consisting of the first
two columns of A. Thus
5 3
MB =
.
4 1
We now determine
the components of the vector p ∈ R2 whose transpose
p1 p2 satisfies the matrix equation
c1 c2 = p1 p2 MB .
Now
MB−1
1
=−
7
56
1 −3
−4 5
.
It follows that
p1 p2 = c1 c2 MB−1
1
1 −3
3 4
= −
−4 5
7
13
− 11
=
.
7
7
pT =
We next compute a vector q ∈ R5 , where qT = cT − pT A. Solving the
equivalent matrix equation for the transpose qT of the column vector q, we
find that
qT = cT − pT A
=
3 4 2 9 5
=
=
3 4 2 9 5
0 0 − 75 60
7
−
−
40
7
− 11
7
13
7
3 4
.
19
7
3
7
5 3 4 7 3
4 1 3 8 4
− 75
We denote the jth component of the vector j by qj .
Now q3 < 0. We show that this implies that the initial basic feasible
solution is not optimal, and that it can be improved by bringing 3 (the index
of the third column of A) into the basis.
Suppose that x is a feasible solution of this optimization problem. Then
Ax = b, and therefore
cT x = pT Ax + qT x = pT b + qT x.
The initial basic feasible solution x satisfies
T
q x=
5
X
qj xj = 0,
j=1
because q1 = q2 = 0 and x3 = x4 = x5 = 0. This comes about because the
manner in which we determined first p then q ensures that qj = 0 for all
j ∈ B, whereas the components of the basic feasible solution x associated
with the basis B satisfy xj = 0 for j 6∈ B. We find therefore that pT b is the
cost of the initial basic feasible solution.
The cost of the initial basic feasible solution is 11, and this is equal to
the value of pT b. The cost cT x of any other basic feasible solution satisfies
cT x = 11 − 75 x3 +
60
x
7 4
where xj denotes the jth component of x.
57
+
40
x,
7 5
We seek to determine a new basic feasible solution x for which x3 > 0,
x4 = 0 and x5 = 0. The cost of such a basic feasible solution will then be
less than that of our initial basic feasible solution.
In order to find our new basic feasible solution we determine the relationships between the coefficients of a feasible solution x for which x4 = 0 and
x5 = 0. Now such a feasible solution must satisfy
x1 a(1) + x2 a(2) + x3 a(3) = b = x1 a(1) + x2 a(2) ,
where x1 and x2 are the non-zero coefficients of the initial basic feasible
solution. Now the vectors a(1) and a(2) constitute a basis of the real vector
space R2 . It follows that there exist real numbers t1,3 and t2,3 such that
a(3) = t1,3 a(1) + t2,3 a(2) . It follows that
(x1 + t1,3 x3 )a(1) + (x2 + t2,3 x3 )a(2) = x1 a(1) + x2 a(2) .
The linear independence of a(1) and a(2) then ensures that x1 +t1,3 x3 = x1
and x2 + t2,3 x3 = x2 . Thus if x3 = λ, where λ ≥ 0 then
x1 = x1 − λt1,3 ,
x2 = x2 − λt2,3 .
Thus, once t1,3 and t2,3 have been determined, we can determine the range
of values of λ that ensure that x1 ≥ 0 and x2 ≥ 0.
In order to determine the values of t1,3 and t2,3 we note that
5
5 3
1
(1)
a
=
=
4
4 1
0
3
5 3
0
a(2) =
=
1
4 1
1
and therefore
(3)
a
(1)
(2)
= t3,1 a + t3,2 a
t3,1
= MB
,
t3,2
where
MB =
=
5 3
4 1
5 3
4 1
t3,1
t3,2
.
It follows that
1
t3,1
1 −3
4
−1 (3)
=
= MB a = −
3
t3,2
7 −4 5
58
5
7
1
7
.
Thus t3,1 = 57 and t3,2 = 71 .
We now determine the feasible solutions x of this optimization problem
that satisfy x3 = λ and x4 = x5 = 0. we have already shown that
x1 = x1 − λt1,3 ,
x2 = x2 − λt2,3 .
Now x1 = 1, x2 = 2, t1,3 = 57 and t2,3 = 17 . It follows that x1 = 1 − 57 λ and
x2 = 2 − 71 λ. Now the components of a feasible solution must satisfy x1 ≥ 0
and x2 ≥ 0. it follows that 0 ≤ λ ≤ 75 . Moreover on setting λ = 75 we find
that x1 = 0 and x2 = 95 . We thus obtain a new basic feasible solution x
associated to the basis {2, 3}, where
xT = 0 59 75 0 0 .
The cost of this new basic feasible solution is 10.
We now let B 0 and x0 denote the new basic and new associated basic
feasible solution respectively, so that B 0 = {2, 3} and
x0T = 0 95 57 0 0 .
We also let MB 0 be the 2 × 2 minor of the matrix A with columns indexed
by the new basis B, so that
1
3 4
3 −4
−1
MB 0 =
and MB 0 =
.
1 3
5 −1 3
0
2
We now
determine the components of the vector p ∈ R whose transpose
0
p2 satisfies the matrix equation
c2 c3 = p01 p02 MB 0 .
p01
We find that
p01 p02
MB−10
1
3 −4
4 2
=
−1 3
5
2 −2 .
=
c2 c3
=
We next compute the components of the vector q0 ∈ R5 so as to ensure
that
q0T = cT − p0T A
−
=
3 4 2 9 5
=
=
3 4 2 9 5 −
1 0 0 11 7 .
59
5 3 4 7 3
2 −2
4 1 3 8 4
2 4 2 −2 −2
The components of the vector q0 determined using the new basis {2, 3} are all
non-negative. This ensures that the new basic feasible solution is an optimal
solution.
Indeed let x be a feasible solution of this optimization problem. Then
Ax0 = b, and therefore
cT x = p0T Ax + q0T x0 = p0T b + q0T x.
Moreover p0T b = 10. It follows that
cT x = 10 + x1 + 11x4 + 7x5 ≥ 10,
and thus the new basic feasible solution x0 is optimal.
We summarize the result we have obtained. The optimization problem
was the following:—
minimize
3x1 + 4x2 + 2x3 + 9x4 + 5x5
subject to the following constraints:
5x1 + 3x2 + 4x3 + 7x4 + 3x5 = 11;
4x1 + x2 + 3x3 + 8x4 + 4x5 = 6;
xj ≥ 0 for j = 1, 2, 3, 4, 5.
We have found the following basic optimal solution to the problem:
x1 = 0,
9
x2 = ,
5
7
x3 = ,
5
x4 = 0,
x5 = 0.
We now investigate all bases for this linear programming problem in order
to determine which bases are associated with basic feasible solutions.
The problem is to find x ∈ R5 that minimizes cT x subject to the constraints Ax = b and x ≥ 0, where
5 3 4 7 3
11
A=
, b=
4 1 3 8 4
6
and
cT =
3 4 2 9 5
.
For each two-element subset B of {1, 2, 3, 4, 5} we compute MB , MB−1
and MB−1 b, where MB is the 2 × 2 minor of the matrix A whose columns are
indexed by the elements of B. We find the following:—
60
B
5 3
4 1
5 4
4 3
5 7
4 8
5 3
4 4
3 4
1 3
3 7
1 8
3 3
1 4
4 7
3 8
4 3
3 4
7 3
8 4
{1, 2}
{1, 3}
{1, 4}
{1, 5}
{2, 3}
{2, 4}
{2, 5}
{3, 4}
{3, 5}
{4, 5}
MB−1
MB
1 −3
−4 5
3 −4
−4 5
8 −7
−4 5
4 −3
−4 5
3 −4
−1 3
8 −7
−1 3
4 −3
−1 3
8 −7
−3 4
4 −3
−3 4
4 −3
−8 7
− 17
−
1
12
1
8
1
5
1
17
1
9
1
11
1
7
1
4
MB−1 b
1
2
−9
14
23 6
− 67
13 4
− 47
9 5
7
5
46 17
7
17
26 9
7
9
46 11
9
− 11
26 7
− 79
13 2
− 23
2
cT MB−1 b
11
1
1
1
10
247
17
139
9
1
1
1
From this data, we see that there are four basic feasible solutions to the
problem. We tabulate them below:—
B
{1, 2}
{2, 3}
{2, 4}
{2, 5}
x
(1, 2, 0, 0, 0)
(0, 95 , 75 , 0, 0)
7
(0, 46
, 0, 17
, 0)
17
26
(0, 9 , 0, 0, 97 )
247
17
139
9
Cost
11
10
= 14.529 . . .
= 15.444 . . .
61
4.6
A Linear Tableau Example with Five Variables and
Three Constraints
Example Consider the problem
Ax = b and x ≥ 0, where

1 2 3
A= 2 3 1
4 2 5
cT =
of minimizing cT x subject to constraints

3 5
2 3 ,
1 4


13
b =  13  ,
20
2 4 3 1 4
.
As usual, we denote by Ai,j the coefficient of the matrix A in the ith row
and jth column, we denote by bi the ith component of the m-dimensional
vector b, and we denote by cj the jth component of the n-dimensional vector c.
We let a(j) be the m-dimensional vector specified by the jth column of
the matrix A for j = 1, 2, 3, 4, 5. Then
 
 
 
1
2
3
(1)
(2)
(3)





2 , a =
3 , a =
1 ,
a =
5
4
2
 
 
3
5
a(4) =  2  and a(5) =  3  .
1
4
A basis B for this linear programming problem is a subset of {1, 2, 3, 4, 5}
consisting of distinct integers j1 , j2 , j3 for which the corresponding vectors
a(j1 ) , a(j2 ) , a(j3 ) constitute a basis of the real vector space R3 .
Given a basis B for the linear programming programming problem, where
B = {j1 , j2 , j3 }, we denote by MB the matrix whose columns are specified
by the vectors a(j1 ) , a(j2 ) and a(j3 ) . Thus (MB )i,k = Ai,jk for i = 1, 2, 3 and
k = 1, 2, 3. We also denote by cB the 3-dimensional vector defined such that
cTB = cj1 cj2 cj3 .
The ordering of the columns of MB and cB is determined by the ordering
of the elements j1 , j2 and j3 of the basis. However we shall proceed on the
basis that some ordering of the elements of a given basis has been chosen,
and the matrix MB and vector cB will be determined so as to match the
chosen ordering.
62
Let j1 = 1, j2 = 2 and j3 = 3, and let B = {j1 , j2 , j3 } = {1, 2, 3}. Then
B is a basis of the linear programming problem, and the invertible matrix
MB determined by a(jk ) for k = 1, 2, 3 is the following 3 × 3 matrix:—


1 2 3
MB =  2 3 1  .
4 2 5
This matrix has determinant −23, and
  13


4
7
− 23 23
13 −4 −7
23
−1 
  6
7
5 
− 23
MB−1 =
.
 −6 −7 5  =  23
23
23
8
6
1
−8 6 −1
− 23 23
23
Then

MB−1 a(1)

1
=  0 ,
0

MB−1 a(4)


0
=  1 ,
0


0
MB−1 a(2)
MB−1 a(3) =  0  ,
1

 25 
− 23
− 24
23
 31 
 27 
−1 (5)
=  23  and MB a =  23  .
13
23
Also
26
23


1
MB−1 b =  3  .
2
It follows that x is a basic feasible solution of the linear programming problem, where
xT = 1 3 2 0 0 .
The vectors a(1) , a(2) , a(3) , a(4) , a(5) , b, e(1) , e(2) and e(3) can then be expressed as linear combinations of a(1) , a(2) , a(3) with coefficients as recorded
in the following tableau:—
a(1)
a(2)
a(3)
a(1)
1
0
0
a(2)
0
1
0
a(3)
0
0
1
a(4)
− 24
23
a(5)
25
− 23
27
23
13
23
31
23
26
23
·
·
·
·
·
63
b e(1)
1 − 13
23
6
3
23
8
2
23
·
·
e(2)
e(3)
4
23
7
23
6
− 23
7
23
5
− 23
1
23
·
·
There is an additional row at the bottom of the tableau. This row is
the criterion row of the tableau. The values in this row have not yet been
calculated, but, when calculated according to the rules described below, the
values in the criterion row will establish whether the current basic feasible
solution is optimal and, if not, how it can be improved.
Ignoring the criterion row, we can represent the structure of the remainder
of the tableau in block form as follows:—
a(1)
a(j1 )
..
.
a(j3 )
a(5)
···
e(1)
b
···
MB−1 A
MB−1 b
MB−1
·
·
·
e(3)
We now employ the principles of the Simplex Method in order to determine whether or not the current basic feasible solution is optimal and, if not,
how to improve it by changing the basis.
Let p be the 3-dimensional vector determined so that
pT = cTB MB−1 .
Then pT MB = cTB , and therefore pT a(jk ) = cjk for k = 1, 2, 3. It follows that
(pT A)j = cj whenever j ∈ B. Putting in the relevant numerical values, we
find that
pT MB = cTB = cj1 cj2 cj3 = c1 c2 c3 = 2 4 3 ,
and therefore
pT =
2 4 3
MB−1 =
22
23
18
23
−3
23
.
We enter the values of p1 , p2 and p3 into the cells of the criterion row
in the columns labelled by e(1) , e(2) and e(3) respectively. The tableau with
these values entered is then as follows:—
a(1)
a(2)
a(3)
a(1)
1
0
0
a(2)
0
1
0
a(3)
0
0
1
a(4)
− 24
23
a(5)
25
− 23
27
23
13
23
31
23
26
23
·
·
·
·
·
64
b e(1)
1 − 13
23
6
3
23
8
2
23
·
22
23
e(2)
e(3)
4
23
7
23
6
− 23
18
23
7
23
5
− 23
1
23
3
− 23
The values in the criterion row in the columns labelled by e(1) , e(2) and
e can be calculated from the components of the cost vector c and the values
in these columns of the tableau. Indeed Let ri,k = (MB−1 )i,k for i = 1, 2, 3 and
k = 1, 2, 3. Then each ri,k is equal to the value of the tableau element located
in the row labelled by a(ji ) and the column labelled by e(k) . The definition
of the vector p then ensures that
(3)
pk = cj1 r1,k + cj2 r2,k + cj3 r3,k
for k = 1, 2, 3, where, for the current basis, j1 = 1, j2 = 2 and j3 = 3.
The cost C of the current basic feasible solution x satisfies C = cT x.
Now (pT A)j = cj for all j ∈ B, where B = {1, 2, 3}. Moreover the current
basic feasible solution x satisfies xj = 0 when j 6∈ B, where xj = (x)j for
j = 1, 2, 3, 4, 5. It follows that
C − pT b = cT x − pT Ax =
5
X
(cj − (pT A)j )xj
j=1
=
X
T
(cj − (p A)j )xj = 0,
j∈B
and thus
C = cT x = pT b.
Putting in the numerical values, we find that C = 20.
We enter the cost C into the criterion row of the tableau in the column
labelled by the vector b. The resultant tableau is then as follows:—
a(1)
a(2)
a(3)
a(1)
1
0
0
a(2)
0
1
0
a(3)
0
0
1
a(4)
− 24
23
a(5)
25
− 23
27
23
13
23
31
23
26
23
b
1
3
2
·
·
·
·
·
20
e(1)
− 13
23
6
23
8
23
22
23
e(2)
e(3)
4
23
7
23
6
− 23
18
23
7
23
5
− 23
1
23
3
− 23
Let si denote the value recorded in the tableau in the row labelled by a(ji )
and the column labelled by b for i = 1, 2, 3. Then the construction of the
tableau ensures that
b = s1 a(j1 ) + s2 a(j2 ) + s3 a(j3 ) ,
and thus si = xji for i = 1, 2, 3, where (x1 , x2 , x3 , x4 , x5 ) is the current basic
feasible solution. It follows that
C = cj1 s1 + cj2 s2 + cj3 s3 ,
65
where, for the current basis, j1 = 1, j2 = 2 and j3 = 3. Thus the cost of the
current basic feasible solution can be calculated from the components of the
cost vector c and the values recorded in the rows above the criterion row of
the tableau in the column labelled by the vector b.
We next determine a 5-dimensional vector q such that cT = pT A + qT .
We find that
−qT = pT A − cT

=
=
=
1 2 3 3
22
18
−3
 2 3 1 2
23
23
23
4 2 5 1
− 2 4 3 1 4
152
99
2 4 3 23
− 2 4
23 60
76
0 0 0 23 23

5
3 
4
3 1 4
Thus
q1 = 0,
q2 = 0,
q3 = 0,
,
q4 = − 76
23
q5 = − 60
.
23
The 4th and 5th components of the vector q are negative. It follows that
the current basic feasible solution is not optimal. Indeed let x be a basic
feasible solution to the problem, and let xj = (x)j for j = 1, 2, 3, 4, 5. Then
the cost C of the feasible solution x satisfies
C = cT x = pT Ax + qT x = pT b + qT x = C + qT x
76
60
= C − x4 − x 5 .
23
23
It follows that the basic feasible solution x will have lower cost if either x4 > 0
or x5 > 0.
We enter the value of −qj into the criterion row of the tableau in the
column labelled by a(j) for j = 1, 2, 3, 4, 5. The completed tableau associated
with basis {1, 2, 3} is then as follows:—
a(1)
a(2)
a(3)
a(1)
1
0
0
a(2)
0
1
0
a(3)
0
0
1
0
0
0
a(4)
− 24
23
a(5)
25
− 23
27
23
13
23
76
23
31
23
26
23
60
23
b
1
3
2
20
e(1)
− 13
23
6
23
8
23
22
23
e(2)
e(3)
4
23
7
23
6
− 23
18
23
7
23
5
− 23
1
23
3
− 23
We refer to this tableau as the extended simplex tableau associated with
the basis {1, 2, 3}.
The general structure of the extended simplex tableau is then as follows:—
66
a(j1 )
a(j2 )
a(j3 )
a(1)
t1,1
t2,1
t3,1
−q1
a(2)
t1,2
t2,2
t3,2
−q2
a(3)
t1,3
t2,3
t3,3
−q3
a(4)
t1,4
t2,4
t3,4
−q4
a(5)
t1,5
t2,5
t3,5
−q5
e(1)
r1,1
r2,1
r3,1
p1
b
s1
s2
s3
C
e(2)
r1,2
r2,2
r3,2
p2
e(3)
r1,3
r2,3
r3,3
p3
where j1 , j2 and j3 are the elements of the current basis, and where the
coefficients ti,j si and ri,k are determined so that
a(j) =
3
X
ti,j a(ji ) ,
b=
i=1
3
X
si a(ji ) ,
e(k) =
i=1
3
X
ri,k a(ji )
i=1
for j = 1, 2, 3, 4, 5 and k = 1, 2, 3.
The coefficients of the criterion row can then be calculated according to
the following formulae:—
pk =
3
X
cji ri,k ,
C=
i=1
3
X
p i bi ,
−qj =
i=1
3
X
pi Ai,j − cj .
i=1
The extended simplex tableau can then be represented in block form as
follows:—
a(1)
a(j1 )
..
.
(j3 )
a(5)
···
b
e(1)
···
MB−1 A
MB−1 b
MB−1
pT A − cT
pT b
pT
e(3)
a
The values in the criterion row in any column labelled by some a(j) can
also be calculated from the values in the relevant column in the rows above
the criterion row.
To see this we note that the value entered into the tableau in the row
labelled by a(ji ) and the column labelled by a(j) is equal to ti,j , where ti,j
is the coefficient in the ith row and jth column of the matrix MB−1 A. Also
pT = cTB MB−1 , where (cB )i = cji for i = 1, 2, 3. It follows that
T
p A=
cTB MB−1 A
=
3
X
cji ti,j .
i=1
Therefore
−qj = (pT A)j − cj
= cj1 t1,j + cj2 t2,j + cj3 t3,j − cj
67
for j = 1, 2, 3, 4, 5.
The coefficients of the criterion row can then be calculated according to
the formulae
pk =
3
X
cji ri,k ,
C=
i=1
3
X
cj i s i ,
−qj =
i=1
3
X
cji ti,j − cj .
i=1
The extended simplex tableau can therefore also be represented in block
form as follows:—
a(1)
a(j1 )
..
.
a(5)
···
e(1)
b
···
MB−1 A
MB−1 b
MB−1
cTB MB−1 A − cT
cTB MB−1 b
cTB MB−1
a(j3 )
e(3)
We now carry through procedures for adjusting the basis and calculating
the extended simplex tableau associated with the new basis.
We recall that the extended simplex tableau corresponding to the old
basis {1, 2, 3} is as follows:—
a(1)
a(2)
a(3)
a(1)
1
0
0
a(2)
0
1
0
a(3)
0
0
1
0
0
0
a(4)
− 24
23
a(5)
25
− 23
27
23
13
23
76
23
31
23
26
23
60
23
e(1)
− 13
23
b
1
3
2
6
23
8
23
22
23
20
e(2)
e(3)
4
23
7
23
6
− 23
18
23
7
23
5
− 23
1
23
3
− 23
We now consider which of the indices 4 and 5 to bring into the basis.
Suppose we look for a basis which includes the vector a(4) together with
two of the vectors a(1) , a(2) and a(3) . A feasible solution x with x5 = 0 will
satisfy
24
13
λ 3 − 27
λ
2
−
λ
λ
0
xT = 1 + 23
,
23
23
where λ = x4 . Indeed A(x − x) = 0, where x is the current basic feasible
solution, and therefore
(x1 − 1)a(1) + (x2 − 3)a(2) + (x3 − 2)a(3) + x4 a(4) = 0.
Now
24 (1)
a(4) = − 23
a +
68
27 (2)
a
23
+
13 (3)
a ,
23
It follows that
(x1 − 1 −
24
x )a(1)
23 4
+ (x2 − 3 +
27
x )a(2)
33 4
+ (x3 − 2 +
13
x )a(3)
23 4
= 0.
But the vectors a(1) , a(2) and a(3) are linearly independent. Thus if x4 = λ
and x5 = 0 then
x1 − 1 −
24
λ
23
= 0,
x2 − 3 +
27
λ
23
= 0,
x3 − 2 +
13
λ
23
= 0,
and thus
x1 = 1 +
24
λ,
23
x2 = 3 −
27
λ,
23
x3 = 2 −
13
λ.
23
For the solution x to be feasible the components of x must all be nonnegative, and therefore λ must satisfy
, 2 × 23
.
λ ≤ min 3 × 23
27
13
69
46
Now 3 × 23
= 27
≈ 2.56 and 2 × 23
= 13
≈ 3.54. It follows that the maximum
27
13
possible value of λ is 69
.
The
feasible
solution
corresponding to this value of
27
λ is a basic feasible solution with basis {1, 3, 4}, and passing from the current
basic feasible solution x to the new feasible basic solution would lower the
69
76
× 27
= 228
≈ 8.44.
cost by −q4 λ, where −q4 λ = 23
27
We examine this argument in more generality to see how to calculate the
change in the cost that arises if an index j not in the current basis is brought
into that basis. Let the current basis be {j1 , j2 , j3 }. Then
b = s1 a(j1 ) + s2 a(j2 ) + s3 a(j3 )
and
a(j) = t1,j a(j1 ) + t2,j a(j2 ) + t3,j a(j3 ) .
Thus if x is a feasible solution, and if (x)j 0 = 0 for j 0 6∈ {j1 , j2 , j3 , j}, then
xj1 a(j1 ) + xj2 a(j2 ) + xj3 a(j3 ) + xj a(j) − b = 0.
Let λ = xj . Then
(xj1 + λt1,j − s1 )a(j1 ) + (xj2 + λt2,j − s2 )a(j2 ) + (xj3 + λt3,j − s3 )a(j3 ) = 0.
But the vectors a(j1 ) , a(j2 ) , a(j3 ) are linearly independent, because {j1 , j2 , j3 }
is a basis for the linear programming problem. It follows that
xji = si − λti,j
for i = 1, 2, 3.
69
For a feasible solution we require λ ≥ 0 and si − λti,j ≥ 0 for i = 1, 2, 3.
We therefore require
si
0 ≤ λ ≤ min
: ti,j > 0 .
ti,j
We could therefore obtain a new basic feasible solution by ejecting from the
si
current basis an index ji for which the ratio
has its minimum value, where
ti,j
this minimum is taken over those values of i for which ti,j > 0. If we set λ
equal to this minimum value, then the cost is then reduced by −qj λ.
and s3 /t4,3 = 46
. Now
With the current basis we find that s2 /t4,2 = 69
27
13
69
46
< 13 . It follows that we could bring the index 4 into the basis, obtaining
27
, given that
a new basis {1, 3, 4}, to obtain a cost reduction equal to 228
13
76
69
228
× 27 = 13 ≈ 8.44.
23
We now calculate the analogous cost reduction that would result from
69
and s3 /t5,3 = 46
.
bringing the index 5 into the basis. Now s2 /t5,2 = 31
26
46
69
Moreover 26 < 31 . It follows that we could bring the index 5 into the basis,
46
× 26
=
obtaining a new basis {1, 2, 5}, to obtain a cost reduction equal to 60
23
120
≈
4.62.
26
We thus obtain the better cost reduction by changing basis to {1, 3, 4}.
We need to calculate the tableau associated with the basis {1, 3, 4}. We
will initially ignore the change to the criterion row, and calculate the updated
values in the cells of the other rows. The current tableau with the values in
the criterion row deleted is as follows:—
a(1)
a(2)
a(3)
a(1)
1
0
0
a(2)
0
1
0
a(3)
0
0
1
a(4)
− 24
23
a(5)
25
− 23
27
23
13
23
31
23
26
23
·
·
·
·
·
b e(1)
1 − 13
23
6
3
23
8
2
23
·
e(2)
e(3)
4
23
7
23
6
− 23
7
23
5
− 23
1
23
·
·
·
Let v be a vector in R3 and suppose that
v = µ1 a(1) + µ2 a(2) + µ3 a(3) = µ01 a(1) + µ02 a(4) + µ03 a(3) .
Now
24 (1)
a(4) = − 23
a +
On multiplying this equation by
23 (4)
a
27
23
,
27
27 (2)
a
23
+
13 (3)
a .
23
we find that
= − 24
a(1) + a(2) +
27
70
13 (3)
a ,
27
and therefore
a(2) =
24 (1)
a
27
+
23 (4)
a
27
−
13 (3)
a .
27
It follows that
v = (µ1 +
24
µ )a(1)
27 2
+
23
µ a(4)
27 2
13
µ )a(3) ,
27 2
+ (µ3 −
and thus
µ01 = µ1 +
µ02 =
24
µ,
27 2
23
µ,
27 2
µ03 = µ3 −
13
µ.
27 2
Now each column of the tableau specifies the coefficients of the vector
labelling the column of the tableau with respect to the basis specified by the
vectors labelling the rows of the tableau.
The pivot row of the old tableau is that labelled by the vector a(2) that
is being ejected from the basis. The pivot column of the old tableau is that
labelled by the vector a(4) that is being brought into the basis. The pivot
element of the tableau is the element or value in both the pivot row and the
27
.
pivot column. In this example the pivot element has the value 23
We see from the calculations above that the values in the pivot row of the
of the
old tableau are transformed by multiplying them by the reciprocal 23
27
pivot element; the entries in the first row of the old tableau are transformed
by adding to them the entries below them in the pivot row multiplied by the
factor 24
; the values in the third row of the old tableau are transformed by
27
subtracting from them the entries above them in the pivot row multiplied by
13
.
the factor 27
0
Indeed the coefficients ti,j , si , ri,k , t0i,j , s0i and ri,k
are defined for i = 1, 2, 3,
j = 1, 2, 3, 4, 5 and k = 1, 2, 3 so that
(j)
a
=
b =
e(k) =
3
X
i=1
3
X
i=1
3
X
(ji )
ti,j a
3
X
=
si a(ji ) =
i=1
3
X
0
s0i a(ji ) ,
i=1
3
X
ri,k a(ji ) =
i=1
0
t0i,j a(ji ) ,
0
0
ri,k
a(ji ) ,
i=1
where j1 = j10 = 1, j3 = j30 = 3, j2 = 2 and j20 = 4.
The general rule for transforming the coefficients of a vector when changing from the basis a(1) , a(2) , a(3) to the basis a(1) , a(4) , a(3) ensure that
t02,j =
1
t2,4
t2,j ,
71
t0i,j = ti,j −
s02 =
1
t2,4
1
t2,4
(i = 1, 3).
s2 ,
s0i = si −
0
r2,k
=
ti,4
t2,j
t2,4
ti,4
s2
t2,4
(i = 1, 3).
r2,j ,
0
ri,k
= ri,k −
ti,4
r2,k
t2,4
(i = 1, 3).
The quantity t2,4 is the value of the pivot element of the old tableau. The
quantities t2,j , s2 and r2,k are those that are recorded in the pivot row of
that tableau, and the quantities ti,4 are those that are recorded in the pivot
column of the tableau.
We thus obtain the following tableau:–
a(1)
a(4)
a(3)
a(1)
1
0
0
·
a(2)
a(4)
0
1
0
a(5)
b
24
27
23
27
13
− 27
a(3)
0
0
1
3
27
31
27
13
27
99
27
69
27
15
27
·
·
·
·
·
e(1)
9
− 27
e(2)
e(3)
6
27
6
27
12
27
7
27
11
− 27
3
27
5
− 27
4
27
·
·
·
The values in the column of the tableau labelled by the vector b give
us the components of a new basic feasible solution x0 . Indeed the column
specifies that
69 (4)
b = 99
a(1) + 27
a + 15
a(2) ,
27
27
and thus Ax0 = b where
x0T =
99
27
0
15
27
69
27
0
.
We now calculate the new values for the criterion row. The new basis B 0
is given by B 0 = {j10 , j20 , j30 }, where j10 = 1, j20 = 4 and j30 = 3. The values p01 ,
p02 and p03 that are to be recorded in the criterion row of the new tableau in
the columns labelled by e(1) , e(2) and e(3) respectively are determined by the
equation
0
0
0
p0k = cj10 r1,k
+ cj20 r2,k
+ cj30 r3,k
72
for k = 1, 2, 3, where
cj10 = c1 = 2,
cj20 = c4 = 1,
cj30 = c3 = 3,
0
denotes the ith component of the vector e(k) with respect to
and where ri,k
the basis a(1) , a(4) , a(3) of R3 .
We find that
0
0
0
p01 = cj10 r1,1
+ cj20 r2,1
+ cj30 r3,1
9
6
= 2 × − 27
+ 1 × 27
+3×
0
0
0
0
p2 = cj10 r1,2 + cj20 r2,2 + cj30 r3,2
p03
6
27
=
6
,
27
12
2
7
= − 27
= 2 × 27
+ 1 × 27
+ 3 × − 11
,
27
0
0
0
= cj10 r1,3 + cj20 r2,3 + cj30 r3,3
5
4
3
= 2 × 27
+ 1 × − 27
+ 3 × 27
= 13
.
27
We next calculate the cost C 0 of the new basic feasible solution. The
quantities s01 , s02 and s03 satisfy s0i = x0ji for i = 1, 2, 3, where (x01 , x02 , x03 , x04 , x05 )
is the new basic feasible solution. It follows that
C 0 = cj1 s01 + cj2 s02 + cj3 s03 ,
where s1 , s2 and s3 are determined so that
0
0
0
b = s01 a(j1 ) + s02 a(j2 ) + s03 a(j3 ) .
The values of s01 , s02 and s03 have already been determined, and have been
recorded in the column of the new tableau labelled by the vector b.
We can therefore calculate C 0 as follows:—
C 0 = cj10 s01 + cj2 s02 + cj3 s03 = c1 s01 + c4 s02 + c3 s03
15
99
+ 69
+ 3 × 27
= 312
.
= 2 × 27
27
27
Alternatively we can use the identity C 0 = p0T b to calculate C 0 as follows:
C 0 = p01 b1 + p02 b2 + p03 b3 =
6
27
× 13 −
2
27
× 13 +
13
27
× 20 =
312
.
27
We now enter the values of p01 , p02 , p03 and C 0 into the tableau associated
with basis {1, 4, 3}. The tableau then takes the following form:—
a(1)
a(4)
a(3)
a(1)
1
0
0
·
a(2)
a(4)
0
1
0
a(5)
b
24
27
23
27
13
− 27
a(3)
0
0
1
3
27
31
27
13
27
·
·
·
·
99
27
69
27
15
27
312
27
73
e(1)
9
− 27
6
27
6
27
6
27
e(2)
e(3)
12
27
7
27
− 11
27
2
− 23
3
27
5
− 27
4
27
13
23
In order to complete the extended tableau, it remains to calculate the
values −qj0 for j = 1, 2, 3, 4, 5, where qj0 satisfies the equation −qj0 = p0T aj −cj
for j = 1, 2, 3, 4, 5.
Now qj0 is the jth component of the vector q0 that satisfies the matrix
equation −q0T = p0T A − cT . It follows that
−q0
T
= p0T A − cT

1 2 3 3
6
−2
13
 2 3 1 2
27
27
27
4 2 5 1
− 2 4 3 1 4
3 1 76
2 32
− 2 4
27
27
76
0 − 27
0 0 − 32
27
=
=
=

5
3 
4
3 1 4
Thus
q20 =
q10 = 0,
q40 = 0,
q30 = 0,
76
,
27
q50 =
32
.
27
The value of each qj0 can also be calculated from the other values recorded
in the column of the extended simplex tableau labelled by the vector a(j) .
0
Indeed the vector p0 is determined so as to satisfy the equation p0T a(j ) = cj 0
for all j 0 ∈ B 0 . It follows that
0T
(j)
p a
=
3
X
0T
(ji0 )
ti,j p a
=
i=1
and therefore
−qj0
=
3
X
cji0 t0i,j ,
i=1
3
X
cji0 t0i,j − cj .
i=1
The extended simplex tableau for the basis {1, 4, 3} has now been computed, and the completed tableau is as follows:—
a(1)
a(4)
a(3)
a(1)
1
0
0
0
a(2)
24
27
23
27
13
− 27
76
− 27
a(3)
0
0
1
a(4)
0
1
0
0
0
a(5)
b
3
27
31
27
13
27
− 32
27
99
27
69
27
15
27
312
27
e(1)
9
− 27
6
27
6
27
6
27
e(2)
e(3)
12
27
7
27
− 11
27
2
− 23
3
27
5
− 27
4
27
13
23
The fact that qj0 ≥ 0 for j = 1, 2, 3, 4, 5 shows that we have now found our
basic optimal solution. Indeed the cost C of any feasible solution x satisfies
C = cT x = p0T Ax + q0T x = p0T b + q0T x
74
= C 0 + q0T x
76
32
= C 0 + x2 + x5 ,
27
27
where x2 = (x)2 and x5 = (x)5 .
Therefore x0 is a basic optimal solution to the linear programming problem, where
69
0 15
0 .
x0T = 99
27
27
27
It is instructive to compare the pivot row and criterion row of the tableau
for the basis {1, 2, 3} with the corresponding rows of the tableau for the basis
{1, 4, 3}.
These rows in the old tableau for the basis {1, 2, 3} contain the following
values:—
a(2)
a(1)
0
a(2)
1
a(3)
0
0
0
0
a(4)
a(5)
27
23
76
23
31
23
60
23
b
3
20
e(1)
e(2)
6
23
22
23
7
23
18
23
e(3)
5
− 23
3
− 23
The corresponding rows in the new tableau for the basis {1, 4, 3} contain
the following values:—
a(4)
a(1)
0
0
a(2)
23
27
76
− 27
a(3)
0
a(4)
1
0
0
a(5)
b
e(1)
e(2)
31
27
− 32
27
69
27
312
27
6
27
6
27
7
27
2
− 23
e(3)
5
− 27
13
23
If we examine the values of the criterion row in the new tableau we find
that they are obtained from corresponding values in the criterion row of the
old tableau by subtracting off the corresponding elements of the pivot row of
. As a result, the new tableau has
the old tableau multiplied by the factor 76
27
value 0 in the cell of the criterion row in column a(4) . Thus the same rule
used to calculate values in other rows of the new tableau would also have
yielded the correct elements in the criterion row of the tableau.
We now investigate the reasons why this is so.
First we consider the transformation of the elements of the criterion row
in the columns labelled by a(j) for j = 1, 2, 3, 4, 5. Now the coefficients ti,j
and t0i,j are defined for i = 1, 2, 3 and j = 1, 2, 3, 4, 5 so that
a(j) =
3
X
ti,j a(ji ) =
i=1
3
X
i=1
75
0
t0i,j a(ji ) ,
where j1 = j10 = 1, j3 = j30 = 3, j2 = 2 and j20 = 4. Moreover
t02,j =
and
t0i,j = ti,j −
1
t2,4
t2,j
ti,4
t2,j
t2,4
(i = 1, 3).
Now
−qj =
3
X
cji ti,j − cj
i=1
−qj0
= c1 t1,j + c2 t2,j + c3 t3,j − cj ,
3
X
=
cji0 t0i,j − cj .
=
i=1
c1 t01,j
+ c4 t02,j + c3 t03,j − cj .
Therefore
qj − qj0 = c1 (t01,j − t1,j ) + c4 t02,j − c2 t2,j + c3 (t03,j − t3,j )
1
=
(−c1 t1,4 + c4 − c2 t2,4 − c3 t3,4 ) t2,j
t2,4
q4
=
t2,j
t2,4
and thus
−qj0 = −qj +
q4
t2,j
t2,4
for j = 1, 2, 3, 4, 5.
Next we note that
C =
C0 =
3
X
i=1
3
X
cji si = c1 s1 + c2 s2 + c3 s3 ,
cji0 s0i = c1 s01 + c4 s02 + c3 s03 .
i=1
Therefore
C 0 − C = c1 (s01 − s1 ) + c4 s02 − c2 s2 + c3 (s03 − s3 )
1
=
(−c1 t1,4 + c4 − c2 t2,4 − c3 t3,4 ) s2
t2,4
q4
s2
=
t2,4
76
and thus
C 0 = qk +
q4
s2
t2,4
for k = 1, 2, 3.
To complete the verification that the criterion row of the extended simplex
tableau transforms according to the same rule as the other rows we note that
pk =
p0k =
3
X
i=1
3
X
cji ri,k = c1 r1,k + c2 r2,k + c3 r3,k ,
0
0
0
0
cji0 ri,k
= c1 r1,k
+ c4 r2,k
+ c3 r3,k
.
i=1
Therefore
0
0
0
p0k − pk = c1 (r1,k
− r1,k ) + c4 r2,k
− c2 r2,k + c3 (r3,k
− r3,k )
1
(−c1 t1,4 + c4 − c2 t2,4 − c3 t3,4 ) r2,k
=
t2,4
q4
=
r2,k
t2,4
and thus
p0k = pk +
q4
r2,k
t2,4
for k = 1, 2, 3.
This completes the discussion of the structure and properties of the extended simplex tableau associated with the optimization problem under discussion.
4.7
Some Results concerning Finite-Dimensional Real
Vector Spaces
We consider the representation of vectors belonging to the m-dimensional
vector space Rm as linear combinations of basis vectors belonging to some
chosen basis of this m-dimensional real vector space.
Elements of Rm are normally considered to be column vectors represented
by m × 1 matrices. Given any v ∈ Rm , we denote by (v)i the ith component
of the vector v, and we denote by vT the 1 × m row vector that is the
transpose of the column vector representing v ∈ Rm . Thus
vT = v1 v2 · · · vm ,
77
where vi = (v)i for i = 1, 2, . . . , m.
We define the standard basis of the real vector space Rm to be the basis
e(1) , e(2) , . . . , e(m)
defined such that
(k)
(e )i =
It follows that v =
m
P
1 if k = i;
0 if k 6= i.
(v)i e(i) for all v ∈ Rm .
i=1
Let u(1) , u(2) , . . . , u(m) be a basis of the real vector space Rm , and let
e(1) , e(2) , . . . , e(m) denote the standard basis of Rm . Then there exists an
invertible m × m matrix M with the property that
(k)
u
m
X
=
(M )i,k e(i)
i=1
for k = 1, 2, . . . , m.
The product M v is defined in the usual fashion for any m-dimensional
vector v: the vector v is expressed as an m × 1 column vector, and the
matrix product is then calculated according to the usual rules of matrix
multiplication, so that
(M v)i =
m
X
(M )i,k (v)k ,
k=1
and thus
Mv =
m X
m
X
(i)
Mi,k (v)k e
=
i=1 k=1
m
X
(v)k u(k) .
k=1
Then M e(i) = u(i) for i = 1, 2, . . . , m. The inverse matrix M −1 of M then
satisfies M −1 u(i) = e(i) for i = 1, 2, . . . , m.
Lemma 4.3 Let m be a positive integer, let u(1) , u(2) , . . . , u(m) be a basis of
Rm , let e(1) , e(2) , . . . , e(m) denote the standard basis of Rm , and let M be the
non-singular matrix that satisfies M e(i) = u(i) for i = 1, 2, . . . , m. Let v be
a vector in Rm , and let λ1 , λ2 , . . . , λm be the unique real numbers for which
v=
m
X
λi u(i) .
i=1
Then λi is the ith component of the vector M −1 v for i = 1, 2, . . . , m.
78
Proof The inverse matrix M −1 of M satisfies M −1 u(k) = e(k) for k =
1, 2, . . . , m. It follows that
M −1 v =
m
X
λk M −1 u(k) =
k=1
m
X
λk e(k) ,
k=1
and thus λ1 , λ2 , . . . , λm are the components of the column vector M −1 v, as
required.
Lemma 4.4 Let m be a positive integer, let u(1) , u(2) , . . . , u(m) be a basis of
Rm , let e(1) , e(2) , . . . , e(m) denote the standard basis of Rm , and let M be the
non-singular matrix that satisfies M e(i) = u(i) for i = 1, 2, . . . , m. Then
e
(k)
=
m
X
ri,k u(i) ,
i=1
where ri,k is the coefficient (M −1 )i,k in the ith row and kth column of the
inverse M −1 of the matrix M .
Proof It follows from Lemma 4.3 that e(k) =
m
P
ri,k u(i) , where the coeffi-
i=1
cients ri,k satisfy ri,k = (M −1 e(k) )i for i = 1, 2, . . . , m. But M −1 e(k) is the
column vector whose components are those of the kth column of the matrix M −1 . The result follows.
Lemma 4.5 Let m be a positive integer, let u(1) , u(2) , . . . , u(m) be a basis of
Rm , let e(1) , e(2) , . . . , e(m) denote the standard basis of Rm , let M be the nonsingular matrix that satisfies M e(i) = u(i) for i = 1, 2, . . . , m, and let ri,k =
(M −1 )i,k for i = 1, 2, . . . , m and k = 1, 2, . . . , m. Let g1 , g2 , . . . , gm be real
m
P
P
(k)
numbers, and let p = m
p
e
,
where
p
=
gi ri,k for k = 1, 2, . . . , m.
k
k=1 k
i=1
Then pT u(i) = gi for i = 1, 2, . . . , m.
Proof It follows from the definition of the matrix M that (u(i) )k = (M )k,i
for all integers i and k between 1 and m. It follows that the ith component of
the row vector pT M is equal to pT u(i) for i = 1, 2, . . . , m. But the definition
of the vector p ensures that pi is the ith component of the row vector gT M −1 ,
where g ∈ R is defined so that
gT = g1 g2 · · · gm .
It follows that pT = gT M −1 , and therefore pT M = gT . Taking the ith
component of the row vectors on both sides of this equality, we find that
pT u(i) = gi , as required.
79
Lemma 4.6 Let m be a positive integer, let
u(1) , u(2) , . . . , u(m)
and
û(1) , û(2) , . . . , û(m)
be bases of Rm , and let v be an element of Rm . Let h be an integer between 1
m
P
and m. Suppose that û(h) =
µi u(i) , where µ1 , µ2 , . . . , µm are real numbers,
i=1
and that u(i) = û(i) for all integers i between 1 and m satisfying i 6= h. Let
m
m
P
P
v=
λi u(i) =
λ̂i û(i) . where λi ∈ R and λ̂i ∈ R for i = 1, 2, . . . , m. Then
i=1
i=1
λ̂(i)

1


λh
if i = h;
µh
=
µi

 λi −
λh if i 6= h.
µh
Proof From the representation of û(h) as a linear combination of the basis
vectors u(1) , u(2) , . . . , u(m) we find that
X µi
1 (h)
û = u(h) +
u(i) .
µh
µ
h
1≤i≤m
i6=h
Moreover û(i) = u(i) for all integers i between 1 and m satisfying i 6= h. It
follows that
X µi
1 (h)
û −
û(i) .
u(h) =
µh
µh
1≤i≤m
i6=h
It follows that
n
n
X
X
(i)
λ̂i û
= v=
λi u(i)
i=1
i=1
=
X
λi û(i) +
1≤i≤m
i6=h
=
X 1≤i≤m
i6=h
X µi
1
λh û(h) −
λh u(i) .
µh
µ
h
1≤i≤m
i6=h
λi −
µi
1
λh û(i) +
λh û(h)
µh
µh
Therefore, equating coefficients of û(i) for i = 1, 2, . . . , n, we find that

1


λh
if i = h,
µh
λ̂(i) =
µi

 λi −
λh if i 6= h,
µh
as required.
80
4.8
The Extended Simplex Tableau for solving Linear
Programming Problems
We now consider the construction of a tableau for a linear programming
problem in Dantzig standard form. Such a problem is specified by an m × n
matrix A, an m-dimensional target vector b ∈ Rm and an n-dimensional
cost vector c ∈ Rn . We suppose moreover that the matrix A is of rank m.
We consider procedures for solving the following linear program in Danzig
standard form.
Determine x ∈ Rn so as to minimize cT x subject to the constraints Ax = b and x ≥ 0.
We denote by Ai,j the component of the matrix A in the ith row and
jth column, we denote by bi the ith component of the target vector b for
i = 1, 2, . . . , m, and we denote by cj the jth component of the cost vector c
for j = 1, 2, . . . , n.
We recall that a feasible solution to this problem consists of an n-dimensional
vector x that satisfies the constraints Ax = b and x ≥ 0 (see Subsection 4.2).
A feasible solution of the linear programming problem then consists of nonnegative real numbers x1 , x2 , . . . , xn for which
n
X
xj a(j) = b.
j=1
A feasible solution determined by x1 , x2 , . . . , xn is optimal if it minimizes cost
n
P
cj xj amongst all feasible solutions to the linear programming problem.
j=1
Let j1 , j2 , . . . , jm be distinct integers between 1 and n that are the elements of a basis B for the linear programming problem. Then the vectors a(j)
for j ∈ B constitute a basis of the real vector space Rm . (see Subsection 4.4).
We denote by MB the invertible m×m matrix whose component (M )i,k in
the ith row and jth column satisfies (MB )i,k = (A)i,jk for i, k = 1, 2, . . . , m.
Then the kth column of the matrix MB is specified by the column vector
a(jk ) for k = 1, 2, . . . , m, and thus the columns of the matrix MB coincide
with those columns of the matrix A that are determined by elements of the
basis B.
Every vector in Rm can be expressed as a linear combination of the basis
vectors a(j1 ) , a(j2 ) , . . . , a(jm ) . It follows that there exist uniquely determined
real numbers ti,j and si for i = 1, 2, . . . , m and j = 1, 2, . . . , n such that
(j)
a
=
m
X
(ji )
ti,j a
and b =
i=1
m
X
i=1
81
si a(ji ) .
It follows from Lemma 4.3 that
ti,j = (MB−1 a(j) )i
and si = (MB−1 b)i
for i = 1, 2, . . . , m.
The standard basis e(1) , e(2) , . . . , e(m) of Rm is defined such that
1 if k = i;
(k)
(e )i =
0 if k 6= i.
It follows from Lemma 4.4 that
e
(k)
=
m
X
ri,k u(i) ,
i=1
where ri,k is the coefficient (MB−1 )i,k in the ith row and kth column of the
inverse MB−1 of the matrix MB .
We can record the coefficients of the m-dimensional vectors
a(1) , a(2) , . . . , a(n) , b, e(1) , e(2) , . . . , e(m)
with respect to the basis a(j1 ) , a(j2 ) , . . . , a(jm ) , of Rm in a tableau of the following form:—
a(j1 )
a(j2 )
..
.
a(jm )
a(1)
t1,1
t2,1
..
.
a(2)
t1,2
t2,2
..
.
···
···
···
..
.
a(n)
t1,n
t2,n
..
.
b
s1
s2
..
.
e(1)
r1,1
r2,1
..
.
e(2)
r1,2
r2,2
..
.
···
···
···
..
.
e(m)
r1,m
r2,m
..
.
tm,1
·
tm,2
·
· · · tm,n
···
·
sm
·
rm,1
·
rm,2
·
···
···
rm,m
·
The definition of the quantities ti,j ensures that
1 if i = k;
ti,jk =
0 if i 6= k.
Also
ti,j =
m
X
ri,k Ai,j
k=1
for i = 1, 2, . . . , m and j = 1, 2, . . . , n, and
si =
m
X
k=1
82
ri,k bk
for i = 1, 2, . . . , m.
If the quantities s1 , s2 , . . . , sm are all non-negative then they determine a
basic feasible solution x of the linear programming problem associated with
the basis B with components x1 , x2 , . . . , xn , where xji = si for i = 1, 2, . . . , m
and xj = 0 for all integers j between 1 and n that do not belong to the
basis B. Indeed
n
X
(j)
xj a
=
j=1
m
X
(ji )
xji a
m
X
=
i=1
si a(ji ) .
i=1
The cost C of the basic feasible solution x is defined to be the value cT x of
the objective function. The definition of the quantities s1 , s2 , . . . , sm ensures
that
n
m
X
X
C=
cj x j =
cj i s i .
j=1
i=1
If the quantities s1 , s2 , . . . , sn are not all non-negative then there is no
basic feasible solution associated with the basis B.
The criterion row at the bottom of the tableau has cells to record quantities p1 , p2 , . . . , pm associated with the vectors that constitute the standard
basis e(1) , e(2) , . . . , e(m) of Rm . These quantities are defined so that
pk =
m
X
cji ri,k
i=1
for k = 1, 2, . . . , m, where cji is the cost associated with the basis vector a(ji )
for i = 1, 2, . . . , k,
An application of Lemma 4.5 establishes that
m
X
pk Ak,ji = cji
k=1
for i = 1, 2, . . . , k.
On combining the identities
si =
m
X
ri,k bk ,
pk =
m
X
cji ri,k
and C =
i=1
k=1
m
X
i=1
derived above, we find that
C=
m
X
i=1
cji si =
m X
m
X
i=1 k=1
83
cji ri,k bk =
m
X
k=1
p k bk .
cji si
The tableau also has cells in the criterion row to record quantities
−q1 , −q2 , . . . , −qn ,
where q1 , q2 , . . . , qn are the components of the unique n-dimensional vector q
characterized by the following properties:
• qji = 0 for i = 1, 2, . . . , m;
• cT x = C + qT x for all x ∈ Rm satisfying the matrix equation Ax = b.
First we show that if q ∈ Rn is defined such that qT = cT − pT A then
the vector q has the required properties.
The definition of p1 , p2 , . . . , pk ensures (as a consequence of Lemma 4.5,
as noted above) that
m
X
pk Ak,ji = cji
k=1
for i = 1, 2, . . . , k. It follows that
T
qji = cji − (p A)ji = cji −
m
X
pk Ak,ji = 0
k=1
for i = 1, 2, . . . , n.
Also pT b = C. It follows that if x ∈ Rn satisfies Ax = b then
cT x = pT Ax + qT x = pT b + qT x = C + qT x.
Thus if qT = cT − pT A then the vector q satisfies the properties specified
above.
We next show that
m
X
(pT A)j =
cji ti,j
i=1
for j = 1, 2, . . . , n.
Now
ti,j =
m
X
ri,k Ak,j
k=1
for i = 1, 2, . . . , m and j = 1, 2, . . . , n. (This is a consequence of the identities
(j)
a
=
m
X
Ak,j e
(k)
=
m X
m
X
i=1 k=1
k=1
84
ri,k Ak,j a(ji ) ,
as noted earlier.)
Also the definition of pk ensures that
pk =
m
X
cji ri,k
i=1
for k = 1, 2, . . . , m. These results ensure that
m
X
cji ti,j =
m X
m
X
i=1
cji ri,k Ak,j =
m
X
i=1 k=1
k=1
m
X
m
X
It follows that
−qj =
pk Ak,j − cj =
pk Ak,j = (pT A)j .
cji ti,k − cj
i=1
k=1
for j = 1, 2, . . . , n.
The extended simplex tableau associated with the basis B is obtained
by entering the values of the quantities −qj (for j = 1, 2, . . . , n), C and pk
(for k = 1, 2, . . . , m) into the bottom row to complete the tableau described
previously. The extended simplex tableau has the following structure:—
a(j1 )
a(j2 )
..
.
a(jm )
a(1)
t1,1
t2,1
..
.
a(2)
t1,2
t2,2
..
.
···
···
···
..
.
a(n)
t1,n
t2,n
..
.
b
s1
s2
..
.
e(1)
r1,1
r2,1
..
.
e(2)
r1,2
r2,2
..
.
···
···
···
..
.
e(m)
r1,m
r2,m
..
.
tm,1
−q1
tm,2
−q2
···
···
tm,n
−qn
sm
C
rm,1
p1
rm,2
p2
···
···
rm,m
pm
The extended simplex tableau can be represented in block form as follows:—
a(1)
a(j1 )
..
.
a(jm )
···
a(n)
e(1)
b
···
MB−1 A
MB−1 b
MB−1
pT A − cT
pT b
pT
e(m)
Let cB denote the m-dimensional vector defined so that
cTB = cj1 cj2 · · · cjm .
The identities we have verified ensure that the extended simplex tableau can
therefore also be represented in block form as follows:—
85
a(1)
a(j1 )
..
.
(jm )
···
a(n)
e(1)
b
···
e(m)
MB−1 A
MB−1 b
MB−1
cTB MB−1 A − cT
cTB MB−1 b
cTB MB−1
a
Given an m × n matrix A of rank m, an m-dimensional target vector b,
and an n-dimensional cost vector c, there exists an extended simplex tableau
associated with any basis B for the linear programming problem, irrespective
of whether or not there exists a basic feasible solution associated with the
given basis B.
The crucial requirement that enables the construction of the tableau is
that the basis B should consist of m distinct integers j1 , j2 , . . . , jm between
1 and m for which the corresponding columns of the matrix A constitute a
basis of the vector space Rm .
A basis B is associated with a basic feasible solution of the linear programming problem if and only if the values in the column labelled by the
target vector b and the rows labelled by a(j1 ) , a(j2 ) , . . . , a(jm ) should be nonnegative. If so, those values will include the non-zero components of the basic
feasible solution associated with the basis.
If there exists a basic feasible solution associated with the basis B then
that solution is optimal if and only if all the values in the criterion row in
the columns labelled by a(1) , a(2) , . . . , a(n) are all non-positive.
Versions of the Simplex Tableau Algorithm for determining a basic optimal solution to the linear programmming problem, given an initial basic
feasible solution, rely on the transformation rules that determine how the
values in the body of the extended simplex tableau are transformed on passing from an old basis B to an new basis B 0 , where the new basis B 0 contains
all but one of the members of the old basis B.
Let us refer to the rows of the extended simplex tableau labelled by the
basis vectors a(1) , a(2) , . . . , a(n) as the basis rows of the tableau.
Lemma 4.6 determines how entries in the basis rows of the extended
simplex tableau transform which one element of the basis is replaced by an
element not belonging to the basis.
Let the old basis B consist of distinct integers j1 , j2 , . . . , jm between 1
0
and n, and let the new basis B 0 also consist of distinct integers j10 , j20 , . . . , jm
0
between 1 and n. We suppose that the new basis B is obtained from the
old basis by replacing an element jh of the old basis B by some integer jh0
between 1 and n that does not belong to the old basis. We suppose therefore
that ji = ji0 when i 6= h, and that jh0 is some integer between 1 and n that
86
does not belong to the basis B.
0
Let the coefficients ti,j , t0i,j , si , s0i , ri,k and ri,k
be determined for i =
1, 2, . . . , m, j = 1, 2, . . . , n and k = 1, 2, . . . , m so that
(j)
a
=
m
X
(ji )
ti,j a
m
X
=
i=1
0
t0i,j a(ji )
i=1
for j = 1, 2, . . . , n,
b=
m
X
(ji )
si a
i=1
and
e(k) =
m
X
=
m
X
0
s0i a(ji )
i=1
ri,k a(ji ) =
i=1
m
X
0
0
a(ji )
ri,k
i=1
for k = 1, 2, . . . , m.
It then follows from Lemma 4.6 that
1
th,j ,
t0h,j =
th,jh0
ti,j 0
t0i,j = ti,j − h th,j (i 6= h).
th,jh0
1
s0h =
sh ,
th,jh0
ti,j 0
s0i = si − h sh (i 6= h),
th,jh0
1
0
rh,k
=
rh,k ,
th,jh0
ti,j 0
0
ri,k
= ri,k − h rh,k (i 6= h).
th,jh0
The pivot row of the extended simplex tableau for this change of basis
from B to B 0 is the row labelled by the basis vector a(jh ) that is to be removed
from the current basis. The pivot column of the extended simplex tableau
0
for this change of basis is the column labelled by the vector a(jh ) that is to
be added to the current basis. The pivot element for this change of basis is
the element th,jh0 of the tableau located in the pivot row and pivot column of
the tableau.
The identities relating the components of a(j) , b and e(k) with respect
to the old basis to the components of those vectors with respect to the new
basis ensure that the rules for transforming the rows of the tableau other
than the criterion row can be stated as follows:—
87
• a value recorded in the pivot row is transformed by dividing it by the
pivot element;
• an value recorded in a basis row other than the pivot row is transformed
by substracting from it a constant multiple of the value in the same
column that is located in the pivot row, where this constant multiple
is the ratio of the values in the basis row and pivot row located in the
pivot column.
In order to complete the discussion of the rules for transforming the values recorded in the extended simplex tableau under a change of basis that
replaces an element of the old basis by an element not in that basis, it remains to analyse the rule that determines how the elements of the criterion
row are transformed under this change of basis.
First we consider the transformation of the elements of the criterion row
in the columns labelled by a(j) for j = 1, 2, . . . , n. Now the coefficients ti,j
and t0i,j are defined for i = 1, 2, . . . , m and j = 1, 2, . . . , n so that
a(j) =
m
X
ti,j a(ji ) =
i=1
m
X
0
t0i,j a(ji ) ,
i=1
where j1 = j10 = 1, j3 = j30 = 3, j2 = 2 and j20 = 4. Moreover
1
t0h,j =
th,jh0
and
t0i,j = ti,j −
th,j
ti,jh0
th,j
th,jh0
for all integers i between 1 and m for which i 6= h.
Now
m
X
−qj =
cji ti,j − cj
i=1
and
−qj0 =
m
X
cji0 t0i,j − cj .
i=1
Therefore
qj − qj0 =
X
cji (t0i,j − ti,j ) + cjh0 t0h,j − cjh th,j
1≤i≤m
i6=h
88
1
−
th,jh0
qjh0
=
th,j
th,jh0
=
!
m
X
cji ti,jh0 + cjh0
th,j
i=1
and thus
qjh0
th,j
th,jh0
−qj0 = −qj +
for j = 1, 2, . . . , n.
Next we note that
C=
m
X
cj i s i
i=1
and
0
C =
m
X
cji0 s0i .
i=1
Therefore
C0 − C =
X
cji (s0i − si ) + cjh0 s0h − cjh sh
1≤i≤m
i6=h
=
1
th,jh0
qjh0
=
sh
th,jh0
−
m
X
!
cji ti,jh0 + cjh0
sh
i=1
and thus
C 0 = qk +
qjh0
sh
th,jh0
for k = 1, 2, . . . , m.
To complete the verification that the criterion row of the extended simplex
tableau transforms according to the same rule as the other rows we note that
pk =
m
X
cji ri,k
i=1
and
p0k
=
m
X
i=1
89
0
cji0 ri,k
.
Therefore
p0k − pk =
X
0
0
cji (ri,k
− ri,k ) + cjh0 rh,k
− cjh rh,k
1≤i≤m
i6=h
1
−
th,jh0
qjh0
=
rh,k
th,jh0
=
m
X
!
cji ti,jh0 + cjh0
rh,k
i=1
and thus
p0k = pk +
qjh0
rh,k
th,jh0
for k = 1, 2, . . . , m.
We conclude that the criterion row of the extended simplex tableau transforms under changes of basis that replace one element of the basis according
to a rule analogous to that which applies to the basis rows. Indeed an element of the criterion row is transformed by subtracting from it a constant
multiple of the element in the pivot row that belongs to the same column,
where the multiplying factor is the ratio of the elements in the criterion row
and pivot row of the pivot column.
We have now discussed how the extended simplex tableau associated with
a given basis B is constructed from the constraint matrix A, target vector b
and cost vector c that characterizes the linear programming problem. We
have also discussed how the tableau transforms when one element of the given
basis is replaced.
It remains how to replace an element of a basis associated with a nonoptimal feasible solution so as to obtain a basic feasible solution of lower cost
where this is possible.
We use the notation previously established. Let j1 , j2 , . . . , jm be the
elements of a basis B that is associated with some basic feasible solution
of the linear programming problem. Then there are non-negative numbers
s1 , s2 , . . . , sm such that
m
X
b=
si a(ji ) ,
i=1
(ji )
where a is the m-dimensional vector determined by column ji of the constraint matrix A.
Let j0 be an integer between 1 and n that does not belong to the basis B.
Then
m
X
(j0 )
a −
ti,j0 a(ji ) = 0.
i=1
90
and therefore
(j0 )
λa
+
m
X
(si − λti,j0 )a(ji ) = b.
i=1
This expression representing b as a linear combination of the basis vectors
a(j0 ) , a(j1 ) , a(j2 ) , . . . , a(jm ) determines an n-dimensional vector x(λ) satisfying
the matrix equation Ax(λ) = b. Let xj (λ) denote the jth component of the
vector x(λ) for j = 1, 2, . . . , n. Then
• xj0 (λ) = λ;
• xji (λ) = si − λti,j0 for i = 1, 2, . . . , m;
• xj = 0 when j 6∈ {j0 , j1 , j2 , . . . , jm }.
The n-dimensional vector x(λ) represents a feasible solution of the linear
programming problem if and only if all its coefficients are non-negative. The
cost is then C + qj0 λ, where C is the cost of the basic feasible solution
determined by the basis B.
Suppose that qj0 < 0 and that ti,j0 ≤ 0 for i = 1, 2, . . . , m. Then x(λ) is
a feasible solution with cost C + qj0 λ for all non-negative real numbers λ. In
this situation there is no optimal solution to the linear programming problem,
because, given any real number K, it is possible to choose λ so that C +qj0 λ <
K, thereby obtaining a feasible solution whose cost is less than K.
If there does exist an optimal solution to the linear programming problem
then there must exist at least one integer i between 1 and m for which
ti,j0 > 0. We suppose that this is the case. Then x(λ) is a feasible solution
if and only if λ satisfies 0 ≤ λ ≤ λ0 , where
si
λ0 = minimum
: ti,j0 > 0 .
ti,j0
We can then choose some integer h between 1 and n for which
sh
= λ0 .
th,j0
0
Let ji0 = ji for i 6= h, and let jh0 = j0 , and let B 0 = {j10 , j20 , . . . , jm
}. Then x(λ0 )
is a basic feasible solution of the linear programming problem associated with
the basis B 0 . The cost of this basic feasible solution is
C+
sh qj0
.
th,j0
91
It makes sense to select the replacement column so as to obtain the
greatest cost reduction. The procedure for finding this information from
the tableau can be described as follows.
We suppose that the simplex tableau for a basic feasible solution has
been prepared. Examine the values in the criterion row in the columns
labelled by a(1) , a(2) , . . . , a(n) . If all those are non-positive then the basic
feasible solution is optimal. If not, then consider in turn those columns a(j0 )
for which the value −qj0 in the criterion row is positive. For each of these
columns, examine the coefficients recorded in the column in the basis rows.
If these coefficients are all non-positive then there is no optimal solution to
the linear programming problem. Otherwise choose h to be the value of i
si
amongst those values of i for which ti,j0 > 0.
that minimizes the ratio
ti,j0
The row labelled by a(jh ) would then be the pivot row if the column a(j0 )
were used as the pivot column.
sh (−qj0 )
that would result if
Calculate the value of the cost reduction
th,j0
the column labelled by a(j0 ) were used as the pivot column. Then choose the
pivot column to maximize the cost reduction amongst all columns a(j0 ) for
which −qj0 > 0. Choose the row labelled by a(jh ) , where h is determined
as described above. Then apply the procedures for transforming the simplex
tableau to that determined by the new basis B 0 , where B 0 includes j0 together
with ji for all integers i between 1 and m satisfying i 6= h.
4.9
The Simplex Tableau Algorithm
In describing the Simplex Tableau Algorithm, we adopt notation previously
introduced. Thus we are concerned with the solution of a linear programming
problem in Dantzig standard form, specified by positive integers m and n,
an m × n constraint matrix A of rank m, a target vector b ∈ Rm and a cost
vector c ∈ Rn . The optimization problem requires us to find a vector x ∈ Rn
that minimizes cT x amongst all vectors x ∈ Rn that satisfy the constraints
Ax = b and x ≥ 0.
We denote by Ai,j the coefficient in the ith row and jth column of the
matrix A, we denote the ith component of the target vector b by bi and we
denote the jth component of the cost vector c by cj for i = 1, 2, . . . , m and
j = 1, 2, . . . , n.
As usual, we define vectors a(j) ∈ Rm for j = 1, 2, . . . , n such that (a(j) )i =
Ai,j for i = 1, 2, . . . , m and j = 1, 2, . . . , n.
Distinct integers j1 , j2 , . . . , jm between 1 and n determine a basis B, where
B = {j1 , j2 , . . . , jm },
92
if and only if the corresponding vectors a(j1 ) , a(j2 ) , . . . , a(jm ) constitute a basis
of Rm . Given such a basis B we let MB denote the invertible m × m matrix
defined such that (MB )i,k = Ai,jk for all integers i and k between 1 and m.
We let ti,j = (MB−1 A)i,j and si = (MB−1 b)i for i = 1, 2, . . . , m and j =
1, 2, . . . , n. Then
m
X
(j)
a =
ti,j a(ji )
i=1
for j = 1, 2, . . . , n, and
m
X
b=
si a(ji ) .
i=1
A basis B determines an associated basic feasible solution if and only if
si ≥ 0 for i = 1, 2, . . . , m. We suppose in what follows that the basis B
determines a basic feasible solution.
Let
m
X
C=
cj i s i .
i=1
Then C is the cost of the basic feasible solution associated with the basis B.
Let
m
X
−qj =
cji ti,j − cj .
i=1
Then qj = 0 for all j ∈ {j1 , j2 , . . . , jm }. Also the cost of any feasible solution
(x1 , x2 , . . . , xn ) of the linear programming problem is
C+
n
X
q j xj .
j=1
The simplex tableau associated with the basis B is that portion of the extended simplex tableau that omits the columns labelled by e(1) , e(2) , . . . , e(m) .
The simplex table has the following structure:
(j1 )
a
a(j2 )
..
.
a(jm )
a(1)
t1,1
t2,1
..
.
a(2)
t1,2
t2,2
..
.
···
···
···
..
.
a(n)
t1,n
t2,n
..
.
b
s1
s2
..
.
tm,1
−q1
tm,2
−q2
···
···
tm,n
−qn
sm
C
93
Let cB denote the m-dimensional vector defined such that
cTB = cj1 cj2 · · · cjm .
Then the simplex tableau can be represented in block form as follows:—
a(1)
a(n)
···
b
(j1 )
a
..
.
MB−1 A
MB−1 b
cTB MB−1 A − cT
cTB MB−1 b
a(jm )
Example We consider again the following linear programming problem:—
minimize
3x1 + 4x2 + 2x3 + 9x4 + 5x5
subject to the following constraints:
5x1 + 3x2 + 4x3 + 7x4 + 3x5 = 11;
4x1 + x2 + 3x3 + 8x4 + 4x5 = 6;
xj ≥ 0 for j = 1, 2, 3, 4, 5.
We are given the following initial basic feasible solution (1, 2, 0, 0, 0). We
need to determine whether this initial basic feasible solution is optimal and,
if not, how to improve it till we obtain an optimal solution.
The constraints require that x1 , x2 , x3 , x4 , x5 be non-negative real numbers
satisfying the matrix equation


x1
 x2  
11
5 3 4 7 3 
 x3  =
.

4 1 3 8 4 
6
 x4 
x5
Thus we are required to find a (column) vector x with components x1 ,
x2 , x3 , x4 and x5 that maximizes cT x subject to the constraints Ax = b and
x ≥ 0, where
5 3 4 7 3
11
A=
, b=
,
4 1 3 8 4
6
and
cT =
3 4 2 9 5
94
.
Our initial basis B satisfies B = {j1 , j2 }, where j1 = 1 and j2 = 2. The
first two columns of the matrix A provide the corresponding invertible 2 × 2
matrix MB . Thus
5 3
MB =
.
4 1
Inverting this matrix, we find that
MB−1
1
=−
7
1 −3
−4 5
.
For each integer j between 1 and 5, let a(j) denote the m-dimensional
vector whose ith component is Ai,j for i = 1, 2. Then
a(j) =
2
X
ti,j a(ji )
and b =
i=1
2
X
si a(ji ) ,
i=1
where ti,j = (MB−1 A)i,j and si = (MB−1 b)i for j = 1, 2, 3, 4, 5 and i = 1, 2.
Calculating MB−1 A we find that
9 1 0 75 17
−1
7
7
.
MB A =
8
0 1 17 − 12
−
7
7
Also
MB−1 b
=
1
2
.
The coefficients of these matrices determine the values of ti,j and si to be
entered into the appropriate cells of the simplex tableau.
The basis rows of the simplex tableau corresponding to the basis {1, 2}
are thus as follows:—
(1)
a
a(2)
a(1)
1
0
a(2)
0
1
a(3)
a(4)
a(5)
5
7
1
7
17
7
− 12
7
9
7
− 87
b
1
2
·
·
·
·
·
·
Now the cost C of the current feasible solution satisfies the equation
C=
2
X
cji si = c1 s1 + c2 s2 ,
i=1
where c1 = 3, c2 = 4, s1 = 1 and s2 = 2. It follows that C = 11.
95
To complete the simplex tableau, we need to compute −qj for j =
1, 2, 3, 4, 5, where
2
X
−qj =
cji ti,j − cj .
i=1
Let cB denote the 2-dimensional vector whose ith component is (cji ). Then
cB = (3, 4). Let q denote the 5-dimensional vector whose jth component is
qj for j = 1, 2, 3, 4, 5. Then
−qT = cTB MB−1 A − cT .
It follows that
−q
T
5
7
1
7
1 0
0 1
17
7
− 12
7
=
3 4
=
− 3 4 2 9 5
0 0 75 − 60
− 40
.
7
7
9
7
− 87
The simplex tableau corresponding to basis {1, 2} is therefore completed
as follows:—
(1)
a
a(2)
a(1)
1
0
a(2)
0
1
0
0
a(3)
a(4)
a(5)
5
7
1
7
5
7
17
7
− 12
7
− 60
7
9
7
− 87
b
1
2
− 40
7
11
The values of −qj for j = 1, 2, 3, 4, 5 are not all non-positive ensures that
the initial basic feasible solution is not optimal. Indeed the cost of a feasible
solution (x1 , x2 , x3 , x4 , x5 ) is
11 − 57 x3 +
60
x
7 4
+
40
x.
7 5
Thus a feasible solution with x3 > 0 and x4 = x5 = 0 will have lower cost
than the initial feasible basic solution. We therefore implement a change of
basis whose pivot column is that labelled by the vector a(3) .
We must determine which row to use as the pivot row. We need to detersi
mine the value of i that minimizes the ratio
, subject to the requirement
ti,3
that ti,3 > 0. This ratio has the value 75 when i = 1 and 14 when i = 2.
Therefore the pivot row is the row labelled by a(1) . The pivot element t1,3
then has the value 75 .
The simplex tableau corresponding to basis {2, 3} is then obtained by
subtracting the pivot row multiplied by 51 from the row labelled by a(2) ,
96
subtracting the pivot row from the criterion row, and finally dividing all
values in the pivot row by the pivot element 57 .
The simplex tableau for the basis {2, 3} is thus the following:—
a(1)
(3)
a
a(2)
a(3)
1
0
a(4)
a(5)
b
17
5
− 11
5
9
5
− 15
a(2)
0
1
− 75
7
5
9
5
−1
0
0
−11
−7
10
7
5
All the values in the criterion row to the left of the new cost are nonpositive. It follows that we have found a basic optimal solution to the linear
programming problem. The values recorded in the column labelled by b
show that this basic optimal solution is
(0, 59 , 75 , 0, 0).
4.10
The Revised Simplex Algorithm
The Simplex Tableau Algorithm restricts attention to the columns to the left
of the extended simplex tableau. The Revised Simplex Algorithm proceeds
by maintaining the columns to the right of the extended simplex tableau,
calculating values in the columns to the left of that tableau only as required.
We show how the Revised Simplex Algorithm is implemented by applying
it to the example used to demonstrate the implementation of the Simplex
Algorithm.
Example We apply the Revised Simplex Algorithm to determine a basic
optimal solution to the the following linear programming problem:—
minimize
3x1 + 4x2 + 2x3 + 9x4 + 5x5
subject to the following constraints:
5x1 + 3x2 + 4x3 + 7x4 + 3x5 = 11;
4x1 + x2 + 3x3 + 8x4 + 4x5 = 6;
xj ≥ 0 for j = 1, 2, 3, 4, 5.
We are given the following initial basic feasible solution (1, 2, 0, 0, 0). We
need to determine whether this initial basic feasible solution is optimal and,
if not, how to improve it till we obtain an optimal solution.
97
The constraints require that x1 , x2 , x3 , x4 , x5 be non-negative real numbers
satisfying the matrix equation


x1
 x2  
5 3 4 7 3 
11
 x3  =
.

4 1 3 8 4 
6
 x4 
x5
Thus we are required to find a (column) vector x with components x1 ,
x2 , x3 , x4 and x5 that maximizes cT x subject to the constraints Ax = b and
x ≥ 0, where
5 3 4 7 3
11
A=
, b=
,
4 1 3 8 4
6
and
cT =
3 4 2 9 5
.
Our initial basis B satisfies B = {j1 , j2 }, where j1 = 1 and j2 = 2. The
first two columns of the matrix A provide the corresponding invertible 2 × 2
matrix MB . Thus
5 3
MB =
.
4 1
Inverting this matrix, we find that
MB−1
1
=−
7
1 −3
−4 5
.
For each integer j between 1 and 5, let a( (j) denote the m-dimensional
vector whose ith component is Ai,j for i = 1, 2. Then
(j)
a
=
2
X
(ji )
ti,j a
and b =
i=1
2
X
si a(ji ) ,
i=1
where ti,j = (MB−1 A)i,j and si = (MB−1 b)i for j = 1, 2, 3, 4, 5 and i = 1, 2.
Let ri,k = (MB−1 )i,k for i = 1, 2 and k = 1, 2, and let
C = cj1 s1 + cj2 s2 = c1 s1 + c2 s2 = 11
p1 = cj1 r1,1 + cj2 r2,1 = c1 r1,1 + c2 r2,1 = 13
7
p2 = cj1 r1,2 + cj2 r2,2 = c1 r1,2 + c2 r2,2 = − 11
7
The values of si , ri,k , C and pk are inserted into the following tableau,
which consists of the columns to the right of the extended simplex tableau:—
98
a(1)
a(2)
b
1
2
11
e(1)
− 17
e(2)
4
7
13
7
− 57
3
7
− 11
7
To proceed with the algorithm, one computes values −qj for j 6∈ B using
the formula
−qj = p1 A1,j + p2 A2,j − cj ,
seeking a value of j for which −qj > 0. Were all the values −qj are nonpositive (i.e., if all the qj are non-negative), then the initial solution would
be optimal. Computing −qj for j = 5, 4, 3, we find that
× 3 − 11
× 4 − 5 = − 40
−q5 = 13
7
7
7
13
11
−q4 = = 7 × 7 − 7 × 8 − 9 = − 60
7
11
5
−q3 = = 13
×
4
−
×
3
−
2
=
7
7
7
The inequality q3 > 0 shows that the initial basic feasible solution is not
optimal, and we should seek to change basis so as to include the vector a(3) .
Let
t1,3 = r1,1 A1,3 + r1,2 A2,3 = − 17 × 4 + 73 × 3 =
t2,3 = r2,1 A1,3 + r2,2 A2,3 =
4
7
× 4 − 57 × 3 =
5
7
1
7
Then
a(3) = t1,3 a(j1 ) + t2,3 a(j2 ) = 75 a(1) + 71 a(2) .
We introduce a column representing the vector a(3) into the tableau to
serve as a pivot column. The resultant tableau is as follows:—
a(3)
a(1)
a(2)
5
7
1
7
5
7
b
1
2
11
e(1)
− 17
e(2)
4
7
13
7
− 75
3
7
− 11
7
To determine a pivot row we must pick the row index i so as to minimize
si
, subject to the requirement that ti,3 > 0. In the context of this
the ratio
ti,3
example, we should pick i = 1. Accordingly the row labelled by the vector a(1)
is the pivot row. To implement the change of basis we must subtract from
the second row the values above them in the pivot row, multiplied by 15 ; we
must subtract the values in the pivot row from the values below them in the
99
criterion row, and we must divide the values in the pivot row itself by the
pivot element 57 .
The resultant tableau corresponding to the basis 2, 3 is then as follows:—
a(3)
a(2)
a(3)
1
0
b
0
10
7
5
9
5
e(1)
− 15
e(2)
3
5
− 47
2
−2
3
5
A straightforward computation then shows that if
pT = 2 −2
then
pT A − cT =
−1 0 0 −11 −7
.
The components of this row vector are all non-positive. It follows that the
basis {2, 3} determines a basic optimal solution
(0, 59 , 75 , 0, 0).
4.11
Finding an Initial Basic Solution to a Linear Programming Problem
Suppose that we are given a linear programming problem in Dantzig standard
form, specified by positive integers m and n, an m × n matrix A of rank m,
an m-dimensional target vector b ∈ Rm and an n-dimensional cost vector
c ∈ Rn . The problem requires us to find an n-dimensional vector x that
minimizes the objective function cT x subject to the constraints Ax = b and
x ≥ 0.
The Simplex Tableau Algorithm and the Revised Simplex Algorithm provided methods for passing from an initial basic feasible solution to a basic
optimal solution, provided that such a basic optimal solution exists. However, we need first to find an initial basic feasible solution for this linear
programming problem.
One can find such an initial basic feasible solution by solving an auxiliary
linear programming problem. This auxiliary problem requires us to find nn
P
dimensional vectors x and z that minimize the objective function
(z)j
j=1
subject to the constraints Ax + z = b, x ≥ 0 and z ≥ 0.
This auxiliary linear programming problem is itself in Dantzig standard
form. Moreover it has an initial basic feasible solution specified by the simultaneous equations x = 0 and z = b. The objective function of a feasible
100
solution is always non-negative. Applications of algorithms based on the
Simplex Method should identify a basic optimal solution (x, z) for this probn
P
lem. If the cost
(z)j of this basic optimal solution is equal to zero then
j=1
Ax = b and x ≥ 0. If the cost of the basic optimal solution is positive then
the problem does not have any basic feasible solutions.
The process of solving a linear programming problem in Dantzig standard
form thus typically consists of two phases. The Phase I calculation aims to
solve the auxiliary linear programming problem of seeking n-dimensional
n
P
vectors x and z that minimize (z)j subject to the constraints Ax + z = b,
i=1
x ≥ 0 and z ≥ 0. If the optimal solution (x, z) of the auxiliary problem
satisfies z 6= 0 then there is no initial basic solution of the original linear
programming problem. But if z = 0 then Ax = b and x ≥ 0, and thus
the Phase I calculation has identified an initial basic feasible solution of
the original linear programmming problem. The Phase II calculation is the
process of successively changing bases to lower the cost of the corresponding
basic feasible solutions until either a basic optimal solution has been found
or else it has been demonstated that no such basic optimal solution exists.
101
5
5.1
General Linear Programming Problems, Duality and Complementary Slackness
General Linear Programming Problems
Linear programming is concerned with problems seeking to maximize or minimize a linear functional of several real variables subject to a finite collection
of constraints, where each constraint either fixes the values of some linear
function of the variables or else requires those values to be bounded, above
or below, by some fixed quantity.
The objective of such a problem involving n real variables x1 , x2 , . . . , xn
is to maximize or minimize an objective function of those variables that is of
the form
c1 x 1 + c2 x 2 + · · · + cn x n ,
subject to appropriate constraints. The coefficients c1 , c2 , . . . , cn that determine the objective function are then fixed real numbers.
Now such an optimization problem may be presented as a minimization
problem, because simply changing the signs of all the coefficients c1 , c2 , . . . , cn
converts any maximization problem into a minimization problem. We therefore suppose, without loss of generality, that the objective of the linear programming problem is to find a feasible solution satisfying appropriate constraints which minimizes the value of the objective function amongst all such
feasible solutions to the problem.
Some of the constraints may simply require specific variables to be nonnegative or non-positive. Now a constraint that requires a particular variable xj to be non-positive can be reformulated as one requiring a variable to
be non-negative by substituting xj for −xj in the statement of the problem.
Thus, without loss of generality, we may suppose that all constraints that
simply specify the sign of a variable xj will require that variable to be nonnegative. Then all such constraints can be specified by specifying a subset
J + of {1, 2, . . . , n}: the constraints then require that xj ≥ 0 for all j ∈ J + .
There may be further constraints in addition to those that simply specify
whether one of the variables is required to be non-positive or non-negative.
Suppose that there are m such additional constraints, and let them be numbered between 1 and m. Then, for each integer i between 1 and m, there
exist real numbers Ai,1 , Ai,2 , . . . , Ai,n and bi that allow the ith constraint to
be expressed either as an inequality constraint of the form
Ai,1 x1 + Ai,2 x2 + . . . + Ai,n xn ≥ bi
102
or else as an equality constraint of the form
Ai,1 x1 + Ai,2 x2 + . . . + Ai,n xn = bi .
It follows from the previous discussion that the statement of a general
linear programming problem can be transformed, by changing the signs of
some of the variables and constants in the statement of the problem, so
as to ensure that the statement of the problem conforms to the following
restrictions:—
• the objective function is to be minimized;
• some of the variables may be required to be non-negative;
• other constraints are either inequality constraints placing a lower bound
on the value of some linear function of the variables or else equality
constraints fixing the value of some linear function of the variables.
Let us describe the statement of a linear programming problem as being
in general primal form if it conforms to the restrictions just described.
A linear programming problem is expressed in general primal form if the
specification of the problem conforms to the following restrictions:—
• the objective of the problem is to find an optimal solution minimizing
the objective function amongst all feasible solutions to the problem;
• any variables whose sign is prescribed are required to be non-negative,
not non-positive;
• all inequality constraints are expressed by prescribing a lower bound
on the value on some linear function of the variables.
A linear programming problem in general primal form can be specified
by specifying the following data: an m × n matrix A with real coefficients,
an m-dimensional vector b with real components; an n-dimensional vector c with real components; a subset I + of {1, 2, . . . , m}; and a subset J +
of {1, 2, . . . , n}. The linear programming programming problem specified by
this data is the following:—
seek x ∈ Rn that minimizes the objective function cT x subject to
the following constraints:—
• Ax ≥ b;
• (Ax)i = (b)i unless i ∈ I + ;
103
• (x)j ≥ 0 for all j ∈ J + .
We refer to the m × n matrix A, the m-dimensional vector b and the ndimensional vector c employed in specifying a linear programming problem
in general primal form as the constraint matrix, target vector and cost vector
respectively for the linear programming problem. Let us refer to the subset
I + of {1, 2, . . . , m} specifying those constraints that are inequality constraints
as the inequality constraint specifier for the problem, and let us refer to the
subset J + of {1, 2, . . . , n} that specifies those variables that are required to
be non-negative for a feasible solution as the variable sign specifier for the
problem.
We denote by Primal(A, b, c, I + , J + ) the linear programming problem
whose specification in general primal form is determined by a constraint
matrix A, target vector b, cost vector c, inequality constraint specifier I +
and variable sign specifier J + .
A linear programming problem formulated in general primal form can be
reformulated as a problem in Dantzig standard form, thus enabling the use
of the Simplex Method to find solutions to the problem.
Indeed consider a linear programming problem Primal(A, b, c, I + , J + )
where the constraint matrix A is an m × n matrix with real coefficients,
the target vector b and the cost vector c are vectors of dimension m and n
respectively with real coefficients. Then the inequality constraint specifier I +
is a subset of {1, 2, . . . , m} and the variable sign specifier J + is a subset of
{1, 2, . . . , n}. The problem is already in Dantzig standard form if and only
if I + = ∅ and J + = {1, 2, . . . , n}.
If the problem is not in Dantzig standard form, then each variable xj
−
for j 6∈ J + can be replaced by a pair of variables x+
j and xj satisfying the
−
+
−
constraints x+
j ≥ 0 and xj ≥ 0: the difference xj − xj of these new variables
is substituted for xj in the objective function and the constraints. Also a
slack variable zi can be introduced for each i ∈ I + , where zi is required to
satisfy the sign constraint zi ≥ 0, and the inequality constraint
Ai,1 x1 + Ai,2 x2 + . . . + Ai,n xn ≥ bi
is then replaced by the corresponding equality constraint
Ai,1 x1 + Ai,2 x2 + . . . + Ai,n xn − zi = bi .
The linear programming problem Primal(A, b, c, I + , J + ) in general primal form can therefore be reformulated as a linear programming problem in
Dantzig standard form as follows:—
104
−
0
determine values of xj for all j ∈ J + , x+
j and xj for all j ∈ J ,
0
+
+
where J = {1, 2, . . . , n} \ J , and zi for all i ∈ I so as to
minimize the objective function
X
X
X
cj x j +
cj x +
−
cj x −
j
j
j∈J +
j∈J 0
j∈J 0
subject to the following constraints:—
(i)
P
j∈J +
+
Ai,j xj +
P
Ai,j x+
j −
j∈J 0
P
Ai,j x−
j = bi for each i ∈ {1, 2, . . . , n}\
j∈J 0
I ;
1.
P
j∈J +
Ai,j xj +
P
Ai,j x+
j −
j∈J 0
P
+
Ai,j x−
j −zi = bi for each i ∈ I ;
j∈J 0
(ii) xj ≥ 0 for all j ∈ J + ;
−
0
(iii) x+
j ≥ 0 and xj ≥ 0 for all j ∈ J ;
(iv) zi ≥ 0 for all i ∈ I + .
Once the problem has been reformulated in Dantzig standard form, techiques
based on the Simplex Method can be employed in the search for solutions to
the problem.
5.2
Complementary Slackness and the Weak Duality
Theorem
Every linear programming problem Primal(A, b, c, I + , J + ) in general primal
form determines a corresponding linear programming problem Dual(A, b, c, I + , J + )
in general dual form. The second linear programming problem is referred to
as the dual of the first, and the first linear programming problem is referred
to as the primal of its dual.
We shall give the definition of the dual problem associated with a given
linear programming problem, and investigate some important relationships
between the primal linear programming problem and its dual.
Let Primal(A, b, c, I + , J + ) be a linear programming problem in general
primal form specified in terms of an m×n constraint matrix A, m-dimensional
target vector b, n-dimensional cost vector c, inequality constraint specifier I + and variable sign specifier J + . The corresponding dual problem
Dual(A, b, c, I + , J + ) may be specified in general dual form as follows:
105
seek p ∈ Rm that maximizes the objective function pT b subject
to the following constraints:—
• pT A ≤ cT ;
• (p)i ≥ 0 for all i ∈ I + ;
• (pT A)j = (c)j unless j ∈ J + .
Lemma 5.1 Let Primal(A, b, c, I + , J + ) be a linear programming problem expressed in general primal form with constraint matrix A with m rows and n
columns, target vector b, cost vector c, inequality constraint specifier I + and
variable sign specifier J + . Then the feasible and optimal solutions of the corresponding dual linear programming problem Dual(A, b, c, I + , J + ) are those
of the problem Primal(−AT , −c, −b, J + , I + ).
Proof An m-dimensional vector p satisfies the constraints of the dual linear
programming problem Dual(A, b, c, I + , J + ) if and only if pT A ≤ cT , (p)i ≥ 0
for all i ∈ I + and (pT A)j = (c)j unless j ∈ J + . On taking the transposes of
the relevant matrix equations and inequalities, we see that these conditions
are satisfied if and only if −AT p ≥ −c, (p)i ≥ 0 for all i ∈ I + and (−AT p)j =
(−c)j unless j ∈ J + . But these are the requirements that the vector p
must satisfy in order to be a feasible solution of the linear programming
problem Primal(−AT , −c, −b, J + , I + ). Moreover p is an optimal solution of
Dual(A, b, c, I + , J + ) if and only if it maximizes the value of pT b, and this is
the case if and only if it minimizes the value of −bT p. The result follows.
A linear programming problem in Dantzig standard form is specified by
specifying integers m and n a constraint matrix A which is an m × n matrix
with real coefficients, a target vector b belonging to the real vector space Rm
and a cost vector c belonging to the real vector space Rm . The objective
of the problem is to find a feasible solution to the problem that minimizes
the quantity cT x amongst all n-dimensional vectors x for which Ax = b and
x ≥ 0.
The objective of the dual problem is then to find some feasible solution
to the problem that maximizes the quantity pT b amongst all m-dimensional
vectors p for which pT A ≤ c.
Theorem 5.2 (Weak Duality Theorem for Linear Programming Problems
in Dantzig Standard Form)
Let m and n be integers, let A be an m × n matrix with real coefficients, let
b ∈ Rm and let c ∈ Rn . Let x ∈ Rn satisfy the constraints Ax = b and
x ≥ 0, and let p ∈ Rm satisfy the constraint pT A ≤ c. Then pT b ≤ cT x.
Moreover pT b = cT x if and only if the following complementary slackness
condition is satisfied:
106
• (pT A)j = (c)j for all integers j between 1 and n for which (x)j > 0.
Proof The constraints satisfied by the vectors x and p ensure that
cT x − pT b = (cT − pT A)x + pT (Ax − b)
= (cT − pT A)x,
because Ax − b = 0. But also x ≥ 0 and cT − pT A) ≥ 0, and therefore
(cT − pT A)x ≥ 0. Moreover (cT − pT A)x = 0 if and only if (cT − pT A)j = 0
for all integers j between 1 and n for which (x)j > 0. The result follows.
Corollary 5.3 Let a linear programming problem in Dantzig standard form
be specified by an m × n constraint matrix A, and m-dimensional target
vector b and an n-dimensional cost vector c. Let x∗ be a feasible solution
of this primal problem, and let p∗ be a solution of the dual problem. Then
p∗T A ≤ cT . Suppose that the complementary slackness conditions for this
primal-dual pair are satisfied, so that (p∗T A)j = (c)j for all integers j between
1 and n for which (x∗ )j > 0. Then x∗ is an optimal solution of the primal
problem, and p∗ is an optimal solution of the dual problem.
Proof Because the complementary slackness conditions for this primal-dual
pair are satisfied, it follows from the Weak Duality Theorem that cT x∗ =
p∗T b (see Theorem 5.2). But it then also follows from the Weak Duality
Theorem that
cT x ≥ p∗T b = cT x∗
for all feasible solutions x of the primal problem. It follows that x∗ is an
optimal solution of the primal problem. Similarly
pT b ≤ cT x∗ = p∗T b
for all feasible solutions p of the dual problem. It follows that p∗ is an
optimal solution of the dual problem, as required.
Another special case of duality in linear programming is exemplified by a
primal-dual pair of problems in Von Neumann Symmetric Form. In this case
the primal and dual problems are specified in terms of an m × n constraint
matrix A, an m-dimensional target vector b and an n-dimensional cost vector c. The objective of the problem is minimize cT x amongst n-dimensional
vectors x that satisfy the constraints Ax ≥ b and x ≥ 0. The dual problem is to maximize pT b amongst m-dimensional vectors p that satisfy the
constraints pT A ≤ cT and p ≥ 0.
107
Theorem 5.4 (Weak Duality Theorem for Linear Programming Problems
in Von Neumann Symmetric Form)
Let m and n be integers, let A be an m × n matrix with real coefficients,
let b ∈ Rm and let c ∈ Rn . Let x ∈ Rn satisfy the constraints Ax ≥ b and
x ≥ 0, and let p ∈ Rm satisfy the constraints pT A ≤ c and pT ≥ 0. Then
pT b ≤ cT x. Moreover pT b = cT x if and only if the following complementary
slackness conditions are satisfied:
• (Ax)i = (b)i for all integers i between 1 and m for which (p)i > 0;
• (pT A)j = (c)j for all integers j between 1 and n for which (x)j > 0;
Proof The constraints satisfied by the vectors x and p ensure that
cT x − pT b = (cT − pT A)x + pT (Ax − b).
But x ≥ 0, p ≥ 0, Ax − b ≥ 0 and cT − pT A ≥ 0. It follows that
cT x − pT b ≥ 0. and therefore cT x ≥ pT b. Moreover cT x − pT b = 0 if and
only if (cT − pT A)j (x)j = 0 for j = 1, 2, . . . , n and (p)i (Ax − b)i = 0, and
therefore cT x = pT b if and only if the complementary slackness conditions
are satisfied.
Theorem 5.5 (Weak Duality Theorem for Linear Programming Problems
in General Primal Form)
Let x ∈ Rn be a feasible solution to a linear programming problem
Primal(A, b, c, I + , J + )
expressed in general primal form with constraint matrix A with m rows and
n columns, target vector b, cost vector c, inequality constraint specifier I +
and variable sign specifier J + , and let p ∈ Rm be a feasible solution to the
corresponding dual programming problem
Dual(A, b, c, I + , J + ).
Then pT b ≤ cT x. Moreover pT b = cT x if and only if the following complementary slackness conditions are satisfied:—
• (Ax)i = bi whenever (p)i 6= 0;
• (pT A)j = (c)j whenever (x)j 6= 0.
Proof The feasible solution x to the primal problem satisfies the following
constraints:—
108
• Ax ≥ b;
• (Ax)i = (b)i unless i ∈ I + ;
• (x)j ≥ 0 for all j ∈ J + .
The feasible solution p to the dual problem satisfies the following constraints:—
• pT A ≤ cT ;
• (p)i ≥ 0 for all i ∈ I + ;
• (pT A)j = (c)j unless j ∈ J + .
Now
cT x − pT b = (cT − pT A)x + pT (Ax − b)
n
m
X
X
T
T
=
(c − p A)j (x)j +
(p)i (Ax − b)i .
j=1
i=1
Let j be an integer between 1 and n. If j ∈ J + then (x)j ≥ 0 and (cT −
pT A)j ≥ 0, and therefore (cT −pT A)j (x)j ≥ 0. If j 6∈ J + then (pT A)j = (c)j ,
and therefore (cT − pT A)j (x)j = 0, irrespective of whether (x)j is positive,
negative or zero. It follows that
n
X
(cT − pT A)j (x)j ≥ 0.
j=1
Moreover
n
X
(cT − pT A)j (x)j = 0
j=1
if and only if (pT A)j = (c)j for all indices j for which (x)j 6= 0.
Next let i be an index between 1 and m. If i ∈ I + then (p)i ≥ 0 and
(Ax − b)i ≥ 0, and therefore (p)i (Ax − b)i ≥ 0. If i 6∈ I + then (Ax)i = (b)i ,
and therefore (p)i (Ax − b)i = 0, irrespective of whether (p)i is positive,
negative or zero. It follows that
m
X
(p)i (Ax − p)i ≥ 0.
i=1
Moreover
m
X
(p)i (Ax − p)i = 0.
i=1
109
if and only if (Ax)i = (b)i for all indices i for which (p)i 6= 0. The result
follows.
Corollary 5.6 Let x∗ ∈ Rn be a feasible solution to a linear programming
problem Primal(A, b, c, I + , J + ) expressed in general primal form with constraint matrix A with m rows and n columns, target vector b, cost vector c, inequality constraint specifier I + and variable sign specifier J + , and let
p∗ ∈ Rm be a feasible solution to the corresponding dual programming problem
Dual(A, b, c, I + , J + ). Suppose that the complementary slackness conditions
are satisfied for this pair of problems, so that (Ax)i = bi whenever (p)i 6= 0,
and (pT A)j = (c)j whenever (x)j 6= 0. Then x∗ is an optimal solution for
the primal problem and p∗ is an optimal solution for the dual problem.
Proof Because the complementary slackness conditions for this primal-dual
pair are satisfied, it follows from the Weak Duality Theorem that cT x∗ =
p∗T b (see Theorem 5.5). But it then also follows from the Weak Duality
Theorem that
cT x ≥ p∗T b = cT x∗
for all feasible solutions x of the primal problem. It follows that x∗ is an
optimal solution of the primal problem. Similarly
pT b ≤ cT x∗ = p∗T b
for all feasible solutions p of the dual problem. It follows that p∗ is an
optimal solution of the dual problem, as required.
Example Consider the following linear programming problem in general
primal form:—
find values of x1 , x2 , x3 and x4 so as to minimize the objective
function
c1 x1 + c2 x2 + c3 x3 + c4 x4
subject to the following constraints:—
• a1,1 x1 + a1,2 x2 + a1,3 x3 + a1,4 x4 = b1 ;
• a2,1 x1 + a2,2 x2 + a2,3 x3 + a2,4 x4 = b2 ;
• a3,1 x1 + a3,2 x2 + a3,3 x3 + a3,4 x4 ≥ b3 ;
• x1 ≥ 0 and x3 ≥ 0.
Here ai,j , bi and cj are constants for i = 1, 2, 3 and j = 1, 2, 3, 4. The dual
problem is the following:—
110
find values of p1 , p2 and p3 so as to maximize the objective function
p 1 b1 + p 2 b2 + p 3 b3
subject to the following constraints:—
• p1 a1,1 + p2 a2,1 + p3 a3,1 ≤ c1 ;
• p1 a1,2 + p2 a2,2 + p3 a3,2 = c2 ;
• p1 a1,3 + p2 a2,3 + p3 a3,3 ≤ c3 ;
• p1 a1,4 + p2 a2,4 + p3 a3,4 = c4 ;
• p3 ≥ 0.
We refer to the first and second problems as the primal problem and
the dual problem respectively. Let (x1 , x2 , x3 , x4 ) be a feasible solution of the
primal problem, and let (p1 , p2 , p3 ) be a feasible solution of the dual problem.
Then
!
3
4
3
4
X
X
X
X
cj −
pi ai,j xj
cj x j −
p i bi =
j=1
i=1
i=1
j=1
+
3
X
4
X
pi
i=1
Now the quantity cj −
3
P
!
ai,j xj − bi
.
j=1
pi ai,j = 0 for j = 2 and j = 4, and
i=1
4
P
j=1
0 for i = 1 and i = 2. It follows that
4
X
cj x j −
j=1
3
X
c1 −
p i bi =
i=1
3
X
!
pi ai,1
x1
i=1
+
c3 −
3
X
!
pi ai,3
x3
i=1
+ p3
4
X
!
a3,j xj − b3 .
j=1
Now x1 ≥ 0, x3 ≥ 0 and p3 ≥ 0. Also
c1 −
3
X
pi ai,1 ≥ 0,
c3 −
i=1
3
X
i=1
111
pi ai,3 ≥ 0
ai,j xj −bi =
and
4
X
a3,j xj − b3 ≥ 0.
j=1
It follows that
4
X
cj x j −
j=1
and thus
3
X
pi bi ≥ 0.
i=1
4
X
cj x j ≥
3
X
j=1
i=1
4
X
3
X
p i bi .
Now suppose that
cj x j =
j=1
p i bi .
i=1
Then
c1 −
c3 −
p3
3
X
i=1
3
X
!
pi ai,1 x1 = 0,
!
pi ai,3 x3 = 0,
i=1
4
X
!
a3,j xj − b3
= 0,
j=1
because a sum of three non-negative quantities is equal to zero if and only if
each of those quantities is equal to zero.
It follows that
4
3
X
X
cj x j =
p i bi
j=1
i=1
if and only if the following three complementary slackness conditions are
satisfied:—
•
3
P
pi ai,1 = c1 if x1 > 0;
i=1
•
3
P
pi ai,3 = c3 if x3 > 0;
i=1
•
P4
j=1
a3,j xj = b3 if p3 > 0.
112
5.3
Open and Closed Sets in Euclidean Spaces
Let m be a positive integer. The Euclidean norm |x| of an element x of Rm
is defined such that
m
X
2
(x)2i .
|x| =
i=1
The Euclidean distance function d on Rm is defined such that
d(x, y) = |y − x|
for all x, y ∈ Rm . The Euclidean distance function satisfies the Triangle
Inequality, together with all the other basic properties required of a distance
function on a metric space, and therefore Rm with the Euclidean distance
function is a metric space.
A subset U of Rm is said to be open in Rm if, given any point b of U ,
there exists some real number ε satisfying ε > 0 such that
{x ∈ Rm : |x − b| < ε} ⊂ U.
A subset of Rm is closed in Rm if and only if its complement is open in Rm .
Every union of open sets in Rm is open in Rm , and every finite intersection
of open sets in Rm is open in Rm .
Every intersection of closed sets in Rm is closed in Rm , and every finite
union of closed sets in Rm is closed in Rm .
Lemma 5.7 Let m be a positive integer, let u(1) , u(2) , . . . , u(m) be a basis of
Rm , and let
( m
)
X
F =
si u(i) : si ≥ 0 for i = 1, 2, . . . , m .
i=1
Then F is a closed set in Rm .
Proof Let T : Rm → Rm be defined such that
T (s1 , s2 , . . . , sm ) =
m
X
si u(i)
i=1
for all real numbers s1 , s2 , . . . , sm . Then T is an invertible linear operator on
Rm , and F = T (G), where
G = {x ∈ Rm : (x)i ≥ 0 for i = 1, 2, . . . , m}.
113
Moreover the subset G of Rm is closed in Rm .
Now it is a standard result of real analysis that every linear operator on
a finite-dimensional vector space is continuous. Therefore T −1 : Rm → Rm is
continuous. Moreover T (G) is the preimage of the closed set G under the
continuous map T −1 , and the preimage of any closed set under a continuous
map is itself closed. It follows that T (G) is closed in Rm . Thus F is closed
in Rm , as required.
Lemma 5.8 Let m be a positive integer, let F be a non-empty closed set in
Rm , and let b be a vector in Rm . Then there exists an element g of F such
that |x − b| ≥ |g − b| for all x ∈ F .
Proof Let R be a positive real number chosen large enough to ensure that
the set F0 is non-empty, where
F0 = F ∩ {x ∈ Rm : |x − b| ≤ R}.
Then F0 is a closed bounded subset of Rm . Let f : F0 → R be defined such
that f (x) = |x − b| for all x ∈ F . Then f : F0 → R is a continuous function
on F0 .
Now it is a standard result of real analysis that any continuous real-valued
function on a closed bounded subset of a finite-dimensional Euclidean space
attains a minimum value at some point of that set. It follows that there
exists an element g of F0 such that
|x − b| ≥ |g − b|
for all x ∈ F0 . If x ∈ F \ F0 then
|x − b| ≥ R ≥ |g − b|.
It follows that
|x − b| ≥ |g − b|
for all x ∈ F , as required.
5.4
A Separating Hyperplane Theorem
Definition A subset K of Rm is said to be convex if (1 − µ)x + µx0 ∈ K for
all elements x and x0 of K and for all real numbers µ satisfying 0 ≤ µ ≤ 1.
It follows from the above definition that a subset K of Rm is a convex
subset of Rm if and only if, given any two points of K, the line segment
joining those two points is wholly contained in K.
114
Theorem 5.9 Let m be a positive integer, let K be a closed convex set in
Rm , and let b be a vector in Rm , where b 6∈ K. Then there exists a linear
functional ϕ: Rm → R and a real number c such that ϕ(x) > c for all x ∈ K
and ϕ(b) < c.
Proof It follows from Lemma 5.8 that there exists a point g of K such that
|x − b| ≥ |g − b| for all x ∈ K. Let x ∈ K. Then (1 − λ)g + λx ∈ K for
all real numbers λ satisfying 0 ≤ λ ≤ 1, because the set K is convex, and
therefore
|(1 − λ)g + λx − b| ≥ |g − b|
for all real numbers λ satisfying 0 ≤ λ ≤ 1. Now
(1 − λ)g + λx − b = g − b + λ(x − g).
It follows by a straightforward calculation from the definition of the Euclidean norm that
|g − b|2 ≤ |(1 − λ)g + λx − b|2
= |g − b|2 + 2λ(g − b)T (x − g)
+ λ2 |x − g|2
for all real numbers λ satisfying 0 ≤ λ ≤ 1. In particular, this inequality
holds for all sufficiently small positive values of λ, and therefore
(g − b)T (x − g) ≥ 0
for all x ∈ K.
Let
ϕ(x) = (g − b)T x
for all x ∈ Rm . Then ϕ: Rm → R is a linear functional on Rm , and ϕ(x) ≥
ϕ(g) for all x ∈ K. Moreover
ϕ(g) − ϕ(b) = |g − b|2 > 0,
and therefore ϕ(g) > ϕ(b). It follows that ϕ(x) > c for all x ∈ K, where
c = 12 ϕ(b) + 12 ϕ(g), and that ϕ(b) < c. The result follows.
5.5
Convex Cones
Definition Let m be a positive integer. A subset C of Rm is said to be a
convex cone in Rm if λv + µw ∈ C for all v, w ∈ C and for all real numbers
λ and µ satisfying λ ≥ 0 and µ ≥ 0.
115
Lemma 5.10 Let m be a positive integer. Then every convex cone in Rm is
a convex subset of Rm .
Proof Let C be a convex cone in Rm and let v, w ∈ C. Then λv + µw ∈ C
for all non-negative real numbers λ and µ. In particular (1 − λ)w + λv ∈ C.
whenever 0 ≤ λ ≤ 1, and thus the convex cone C is a convex set in Rm , as
required.
Lemma 5.11 Let S be a subset of Rm , and let C be the set of all elements
of Rm that can be expressed as a linear combination of the form
s1 a(1) + s2 a(2) + · · · + sn a(n) ,
where a(1) , a(2) , . . . , a(n) are vectors belonging to S and s1 , s2 , . . . , sn are nonnegative real numbers. Then C is a convex cone in Rm .
Proof Let v and w be elements of C. Then there exist finite subsets S1
and S2 of S such that v can be expressed as a linear combination of the
elements of S1 with non-negative coefficients and w can be expressed as a
linear combination of the elements of S2 with non-negative coefficients. Let
S1 ∪ S2 = {a(1) , a(2) , . . . , a(n) }.
Then there exist non-negative real numbers s1 , s2 , . . . , sn and t1 , t2 , . . . , tn
such that
n
n
X
X
v=
sj a(j) and w =
tj a(j) .
j=1
j=1
Let λ and µ be non-negative real numbers. Then
λv + µw =
n
X
(λsj + µtj )a(j) ,
j=1
and λsj + µtj ≥ 0 for j = 1, 2, . . . , n. It follows that λv + µw ∈ S, as
required.
Proposition 5.12 Let m be a positive integer, let a(1) , a(2) , . . . , a(n) ∈ Rm ,
and let C be the subset of Rm defined such that
( n
)
X
C=
tj a(j) : tj ≥ 0 for j = 1, 2, . . . , n .
j=1
Then C is a closed convex cone in Rm .
116
Proof It follows from Lemma 5.11 that C is a convex cone in Rm . We must
prove that this convex cone is a closed set.
The vectors a(1) , a(2) , . . . , a(n) span a vector subspace V of Rm that is
isomorphic as a real vector space to Rk for some integer k satisfying 0 ≤ k ≤
m. This vector subspace V of Rm is a closed subset of Rm , and therefore any
subset of V that is closed in V will also be closed in Rm . Replacing Rm by
Rk , if necessary, we may assume, without loss of generality that the vectors
a(1) , a(2) , . . . , a(n) span the vector space Rm . Thus if A is the m × n matrix
defined such that (A)i,j = (a(j) )i for i = 1, 2, . . . , m and j = 1, 2, . . . , n then
the matrix A is of rank m.
Let B be the collection consisting of all subsets B of {1, 2, . . . , n} for
which the members of the set {a(j) : j ∈ B} constitute a basis of the real
vector space Rm and, for each B ∈ B, let
( m
)
X
CB =
si a(ji ) : si ≥ 0 for i = 1, 2, . . . , m ,
i=1
where j1 , j2 , . . . , jm are distinct and are the elements of the set B. It follows
from Lemma 5.7 that the set CB is closed in Rm for all B ∈ B.
Let b ∈ C. The definition of C then ensures that there exists some
x ∈ Rn that satisfies Ax = b and x ≥ 0. Thus the problem of determining
x ∈ Rn such that Ax = b and x ≥ 0 has a feasible solution. It follows
from Theorem 4.2 that there exists a basic feasible solution to this problem,
and thus there exist distinct integers j1 , j2 , . . . , jm between 1 and n and nonnegative real numbers s1 , s2 , . . . , sm such that a(j1 ) , a(j2 ) , . . . , a(jm ) are linearly
independent and
m
X
b=
si a(ji ) .
i=1
Therefore b ∈ CB where
B = {j1 , j2 , . . . , jm }.
We have thus shown that, given any element b of C, there exists a subset
B of {1, 2, . . . , n} belonging to B for which b ∈ CB . It follows from this that
the subset C of Rm is the union of the closed sets CB taken over all elements
B of the finite set B. Thus C is a finite union of closed subsets of Rm , and
is thus itself a closed subset of Rm , as required.
5.6
Farkas’ Lemma
Proposition 5.13 Let C be a closed convex cone in Rm and let b be a vector
in Rm . Suppose that b 6∈ C. Then there exists a linear functional ϕ: Rm → R
117
such that ϕ(v) ≥ 0 for all v ∈ C and ϕ(b) < 0.
Proof Suppose that b 6∈ C. The cone C is a closed convex set. It follows
from Theorem 5.9 that there exists a linear functional ϕ: Rn → R and a real
number c such that ϕ(v) > c for all v ∈ C and ϕ(b) < c.
Now 0 ∈ C, and ϕ(0) = 0. It follows that c < 0, and therefore ϕ(b) ≤
c < 0.
Let v ∈ C. Then λv ∈ C for all real numbers λ satisfying λ > 0. It
c
follows that λϕ(v) = ϕ(λv) > c and thus ϕ(v) > for all real numbers λ
λ
satisfying λ > 0, and therefore
c
= 0.
λ→+∞ λ
ϕ(v) ≥ lim
We conclude that ϕ(v) ≥ 0 for all v ∈ C.
Thus ϕ(v) ≥ 0 for all v ∈ C and ϕ(b) < 0, as required.
Lemma 5.14 (Farkas’ Lemma) Let A be a m × n matrix with real coefficients, and let b ∈ Rm be an m-dimensional real vector. Then exactly one of
the following two statements is true:—
(i) there exists x ∈ Rn such that Ax = b and x ≥ 0;
(ii) there exists y ∈ Rm such that yT A ≥ 0 and yT b < 0.
Proof Let a(1) , a(2) , . . . , a(n) be the vectors in Rm determined by the columns
of the matrix A, so that (a(j) )i = (A)i,j for i = 1, 2, . . . , m and j = 1, 2, . . . , n,
and let
( n
)
X
C=
xj a(j) : xj ≥ 0 for j = 1, 2, . . . , n .
j=1
It follows from Proposition 5.12 that C is a closed convex cone in Rm . Moreover
C = {Ax : x ∈ Rn and x ≥ 0}.
Thus b ∈ C if and only if there exists x ∈ Rn such that b = Ax and x ≥ 0.
Therefore statement (i) in the statement of Farkas’ Lemma is true if and only
if b ∈ C.
If b 6∈ C then it follows from Proposition 5.13 that there exists a linear
functional ϕ: Rm → R such that ϕ(v) ≥ 0 for all v ∈ C and ϕ(b) < 0. Then
there exists y ∈ Rm with the property that ϕ(v) = yT v for all v ∈ Rm .
Now Ax ∈ C for all x ∈ Rn satisfying x ≥ 0. It follows that yT Ax ≥ 0
for all x ∈ Rn satisfying x ≥ 0. In particular (yT A)i = yT Ae(i) ≥ 0 for
118
i = 1, 2, . . . , m, where e(i) is the vector in Rm whose ith component is equal
to 1 and whose other components are zero. Thus if b 6∈ C then there exists
y ∈ Rm for which yT A ≥ 0 and yT b < 0.
Conversely suppose that there exists y ∈ Rm such that yT A ≥ 0 and
yT b < 0. Then yT Ax ≥ 0 for all x ∈ Rn satisfying x ≥ 0, and therefore
yT v ≥ 0 for all v ∈ C. But yT b < 0. It follows that b 6∈ C. Thus statement
(ii) in the statement of Farkas’s Lemma is true if and only if b 6∈ C. The
result follows.
Corollary 5.15 Let A be a m × n matrix with real coefficients, and let c ∈
Rn be an n-dimensional real vector. Then exactly one of the following two
statements is true:—
(i) there exists y ∈ Rm such that yT A = cT and y ≥ 0;
(ii) there exists v ∈ Rn such that Av ≥ 0 and cT v < 0.
Proof It follows on applying Farkas’s Lemma to the transpose of the matrix A that exactly one of the following statements is true:—
(i) there exists y ∈ Rm such that AT y = c and y ≥ 0;
(ii) there exists v ∈ Rm such that vT AT ≥ 0 and vT c < 0.
But vT c = cT v. Also AT y = c if and only if yT A = cT , and vT AT ≥ 0 if
and only if Av ≥ 0. The result follows.
Corollary 5.16 Let A be a m×n matrix with real coefficients, and let c ∈ Rn
be an n-dimensional real vector. Suppose that cT v ≥ 0 for all v ∈ Rn
satisfying Av ≥ 0. Then there exists some there exists y ∈ Rm such that
yT A = cT and y ≥ 0.
Proof Statement (ii) in the statement of Corollary 5.15 is false, by assumption, and therefore statement (i) in the statement of that corollary must be
true. The result follows.
Proposition 5.17 Let n be a positive integer, let I be a non-empty finite
set, let ϕ: Rn → R be a linear functional on Rn , and, for each i ∈ I, let
ηi : Rn → R be a linear functional on Rn . Suppose that ϕ(v) ≥ 0 for all
v ∈ Rn with the property that ηi (v) ≥ 0 for all i ∈ I.P Then there exist
non-negative real numbers gi for all i ∈ I such that ϕ =
gi ηi .
i∈I
119
Proof We may suppose that I = {1, 2, . . . , m} for some positive integer m.
For each i ∈ I there exist real numbers Ai,1 , Ai,2 , . . . , Ai,n such that
ηi (v1 , v2 , . . . , vn ) =
n
X
Ai,j vj
j=1
for i = 1, 2, . . . , m and for all real numbers v1 , v2 , . . . , vn . Let A be the m × n
matrix whose coefficient in the ith row and jth column is the real number
Ai,j for i = 1, 2, . . . , m and j = 1, 2, . . . , n. Then an n-dimensional vector
v ∈ Rn satisfies ηi (v) ≥ 0 for all i ∈ I if and only if Av ≥ 0.
There exists an n-dimensional vector c ∈ Rn such that ϕ(v) = cT v for all
v ∈ Rn . Then cT v ≥ 0 for all v ∈ Rn satisfying Av ≥ 0. It then follows from
Corollary 5.16 that there exists y ∈ Rm such that yT A = cT andP
y ≥ 0. Let
gi = (y)i for i = 1, 2, . . . , m. Then gi ≥ 0 for i = 1, 2, . . . , m and
gi ηi = ϕ,
i∈I
as required.
Remark The result of Proposition 5.17 can also be viewed as a consequence
of Proposition 5.13 applied to the convex cone in the dual space Rn∗ of the
real vector space Rn generated by the linear functionals ηi for i ∈ I. Indeed
let C be the subset of Rn∗ defined such that
(
)
X
C=
gi ηi : gi ≥ 0 for all i ∈ I .
i∈I
It follows from Proposition 5.12 that C is a closed convex cone in the dual
space Rn∗ of Rn . If the linear functional ϕ did not belong to this cone then it
would follow from Proposition 5.13 that there would exist a linear functional
V : Rn∗ → R with the property that V (ηi ) ≥ 0 for all i ∈ I and V (ϕ) < 0.
But given any linear functional on the dual space of a given finite-dimensional
vector space, there exists some vector belonging to the given vector space such
that the linear functional on the dual space evaluates elements of the dual
space at that vector (see Corollary 2.7). It follows that there would exist
v ∈ Rn such that V (ψ) = ψ(v) for all ψ ∈ Rn∗ . But then ηi (v) ≥ 0 for all
i ∈ I and ϕ(v) < 0. This contradicts the requirement that ϕ(v) ≥ 0 for
all v ∈ Rn satisfying ηi (v) ≥ 0 for all i ∈ I. To avoid this contradiction it
must be the case that ϕ ∈ C, and therefore
P there must exist non-negative
real numbers gi for all i ∈ I such that ϕ = i∈I gi ηi .
Corollary 5.18 Let n be a positive integer, let I be a non-empty finite set, let
ϕ: Rn → R be a linear functional on Rn , and, for each i ∈ I, let ηi : Rn → R be
a linear functional on Rn . Suppose that there exists a subset I0 of I such that
120
ϕ(v) ≥ 0 for all v ∈ Rn with the property that ηi (v) ≥ 0 for all i ∈ I0 .P
Then
there exist non-negative real numbers gi for all i ∈ I such that ϕ =
gi ηi
i∈I
and gi = 0 when i 6∈ I0 .
Proof It follows directly from Proposition P
5.17 that there exist non-negative
real numbers gi for all i ∈ I0 such that ϕ =
gi ηi . Let gi = 0 for all i ∈ I\I0 .
i∈I0
P
Then ϕ =
gi ηi , as required.
i∈I0
Definition A subset X is said to be a convex polytope if there exist linear
functionals η1 , η2 , . . . , ηm on Rn and real numbers s1 , s2 , . . . , sm such that
X = {x ∈ Rn : ηi (x) ≥ si for i = 1, 2, . . . , m}.
Let (ηi : i ∈ I) be a finite collection of linear functionals on Rn indexed
by a finite set I, let si be a real number for all i ∈ I, and let
\
X = {x ∈ R : ηi (x) ≥ si }.
i∈I
Then X is a convex polytope in Rn . A point x of Rn belongs to the convex
polytope X if and only if ηi (x) ≥ si for all i ∈ I.
Proposition 5.19 Let n be a positive integer, let I be a non-empty finite
set, and, for each i ∈ I, let ηi : Rn → R be non-zero linear functional and let
si be a real number. Let X be the convex polytope defined such that
\
X = {x ∈ R : ηi (x) ≥ si }.
i∈I
(Thus a point x of Rn belongs to the convex polytope X if and only if ηi (x) ≥
si for all i ∈ I.) Let ϕ: Rn → R be a non-zero linear functional on Rn , and
let x∗ ∈ X. Then ϕ(x∗ ) ≤ ϕ(x) for all x ∈ X if and P
only if there exist
non-negative real numbers gi for all i ∈ I such that ϕ =
gi ηi and gi = 0
i∈I
whenever ηi (x∗ ) > si .
Proof Let K = {i ∈ I : ηi (x∗ ) > si }. Suppose that there
P do not exist
non-negative real numbers gi for all i ∈ I such that ϕ =
gi ηi and gi = 0
i∈I
when i ∈ K. Corollary 5.18 then ensures that there must exist some v ∈ Rn
such that ηi (v) ≥ 0 for all i ∈ I \ K and ϕ(v) < 0. Then
ηi (x∗ + λv) = ηi (x∗ ) + ληi (v) ≥ si
121
for all i ∈ I \ K and for all λ ≥ 0. If i ∈ K then ηi (x∗ ) > si . The set K is
finite. It follows that there must exist some real number λ0 satisfying λ0 > 0
such that ηi (x∗ + λv) ≥ si for all i ∈ K and for all real numbers λ satisfying
0 ≤ λ ≤ λ0 .
Combining the results in the cases when i ∈ I \ K and when i ∈ K,
we find that ηi (x∗ + λv) ≥ si for all i ∈ I and λ ∈ [0, λ0 ], and therefore
x∗ + λv ∈ X for all real numbers λ satisfying 0 ≤ λ ≤ λ0 . But
ϕ(x∗ + λv) = ϕ(x∗ ) + λϕ(v) < ϕ(x∗ )
whenever λ > 0. It follows that the linear functional ϕ cannot attain a
minimum value in X at any point x∗ for which either K = I or for which K
is a proper subset of I but
Pthere exist non-negative real numbers gi for all
i ∈ I \ K such that ϕ =
gi ηi . The result follows.
i∈I\K
5.7
Strong Duality
Example Consider again the following linear programming problem in general primal form:—
find values of x1 , x2 , x3 and x4 so as to minimize the objective
function
c1 x1 + c2 x2 + c3 x3 + c4 x4
subject to the following constraints:—
• a1,1 x1 + a1,2 x2 + a1,3 x3 + a1,4 x4 = b1 ;
• a2,1 x1 + a2,2 x2 + a2,3 x3 + a2,4 x4 = b2 ;
• a3,1 x1 + a3,2 x2 + a3,3 x3 + a3,4 x4 ≥ b3 ;
• x1 ≥ 0 and x3 ≥ 0.
Now the constraint
a1,1 x1 + a1,2 x2 + a1,3 x3 + a1,4 x4 = b1
can be expressed as a pair of inequality constraints as follows:
a1,1 x1 + a1,2 x2 + a1,3 x3 + a1,4 x4 ≥ b1
−a1,1 x1 − a1,2 x2 − a1,3 x3 − a1,4 x4 ≥ −b1 .
Similarly the equality constraint involving b2 can be expressed as a pair or
inequality constraints.
Therefore the problem can be reformulated as follows:—
122
find values of x1 , x2 , x3 and x4 so as to minimize the objective
function
c1 x 1 + c2 x 2 + c3 x 3 + c4 x 4
subject to the following constraints:—
• a1,1 x1 + a1,2 x2 + a1,3 x3 + a1,4 x4 ≥ b1 ;
• −a1,1 x1 − a1,2 x2 − a1,3 x3 − a1,4 x4 ≥ −b1 ;
• a2,1 x1 + a2,2 x2 + a2,3 x3 + a2,4 x4 ≥ b2 ;
• −a2,1 x1 − a2,2 x2 − a2,3 x3 − a2,4 x4 ≥ −b2 ;
• a3,1 x1 + a3,2 x2 + a3,3 x3 + a3,4 x4 ≥ b3 ;
• x1 ≥ 0;
• x3 ≥ 0.
Let
ϕ(x1 , x2 , x3 , x4 ) = c1 x1 + c2 x2 + c3 x3 + c4 x4 ,
and let
η1+ (x1 , x2 , x3 , x4 )
η1− (x1 , x2 , x3 , x4 )
η2+ (x1 , x2 , x3 , x4 )
η2− (x1 , x2 , x3 , x4 )
η3 (x1 , x2 , x3 , x4 )
ζ1 (x1 , x2 , x3 , x4 )
ζ3 (x1 , x2 , x3 , x4 )
=
=
=
=
=
=
=
a1,1 x1 + a1,2 x2 + a1,3 x3 + a1,4 x4 ,
−η1 (x1 , x2 , x3 , x4 ),
a2,1 x1 + a2,2 x2 + a2,3 x3 + a2,4 x4 ,
−η3 (x1 , x2 , x3 , x4 ),
a3,1 x1 + a3,2 x2 + a3,3 x3 + a3,4 x4 ,
x1 ,
x3 ,
Then (x1 , x2 , x3 , x4 ) is a feasible solution to the primal problem if and
only if this element of R4 belongs to the convex polytope X, where X is
the subset of R4 consisting of all points x of R4 that satisfy the following
constraints:—
• η1+ (x) ≥ b1 ;
• η1− (x) ≥ −b1 ;
• η2+ (x) ≥ b2 ;
• η2− (x) ≥ −b2 ;
• η3 (x) ≥ b3 ;
123
• ζ1 (x) ≥ 0;
• ζ3 (x) ≥ 0.
An inequality constraint is said to be binding for a particular feasible
solution x if equality holds in that constraint at the feasible solution. Thus
the constraints on the values of η1+ , η1− , η2+ and η2− are always binding at
points of the convex polytope X, but the constraints determined by η3 , ζ1
and ζ3 need not be binding.
Suppose that the linear functional ϕ attains its minimum value at a point
∗
x of X, where x∗ = (x∗1 , x∗2 , x∗3 , x∗4 ). It then follows from Proposition 5.19
−
+
−
that there exist non-negative real numbers p+
1 , p1 , p2 , p2 , p3 , q1 and q3 such
that
− −
+ +
− −
+
p+
1 η1 + p1 η1 + p2 η2 + p2 η2 + p3 η3 + q1 ζ1 + q3 ζ3 = ϕ.
Moreover p3 = 0 if η3 (x∗ ) > b3 , q1 = 0 if ζ1 (x∗ ) > 0, and q3 = 0 if ζ3 (x∗ ) > 0.
Now η1− = −η1+ and η2− = −η2+ . It follows that
p1 η1+ + p2 η2+ + p3 η3 + q1 ζ1 + q3 ζ3 = ϕ,
−
+
−
where p1 = p+
1 − p1 and p2 = p2 − p2 . Moreover p3 = 0 if
p1 a1,1 + p2 a2,1 + p3 a3,1
p1 a1,2 + p2 a2,2 + p3 a3,2
p1 a1,3 + p2 a2,3 + p3 a3,3
p1 a1,4 + p2 a2,4 + p3 a3,4
p3
4
P
i=1
a3,j x∗j > b3 ,
3
P
a3,j x∗j > b3 ,
i=1
q1 = 0 if x∗1 > 0, and q3 = 0 if x∗3 > 0.
It follows that
Moreover p3 = 0 if
4
P
≤
=
≤
=
≥
c1 ,
c2 ,
c3 ,
c4 ,
0.
pi ai,1 = c1 if x∗1 > 0, and
i=1
3
P
pi ai,3 = c3
i=1
if x∗3 > 0. It follows that (p1 , p2 , p3 ) is a feasible solution of the dual problem
to the feasible primal problem.
Moreover the complementary slackness conditions determined by the primal problem are satisfied. It therefore follows from the Weak Duality Theorem (Theorem 5.5) that (p1 , p2 , p3 ) is an optimal solution to the dual problem.
Theorem 5.20 (Strong Duality for Linear Programming Problems with Optimal Solutions)
Let x∗ ∈ Rn be an optimal solution to a linear programming problem
Primal(A, b, c, I + , J + )
124
expressed in general primal form with constraint matrix A with m rows and
n columns, target vector b, cost vector c, inequality constraint specifier I +
and variable sign specifier J + . Then there exists an optimal solution p∗ to
the corresponding dual programming problem
Dual(A, b, c, I + , J + ),
and moreover p∗T b = cT x∗ .
Proof Let x∗ = (x∗1 , x∗2 , . . . , x∗n ), and let Ai,j = (A)i,j , bi = (b)i and cj = (c)j
for i = 1, 2, . . . , m and j = 1, 2, . . . , n. Then optimal solution x∗ minimizes
cT x∗ subject to the following constraints:—
• Ax∗ ≥ b;
• (Ax∗ )i = bi unless i ∈ I + ;
• x∗j ≥ 0 for all j ∈ J + .
Let p be a feasible solution to the dual problem, and let p = (p1 , p2 , . . . , pm ).
Then p must satisfy the following constraints:—
• pT A ≤ cT ;
• pi ≥ 0 for all i ∈ I + ;
• (pT A)j = cj unless j ∈ J + .
Now the constraints of the primal problem can be expressed in inequality
form as follows:—
• (Ax∗ )i ≥ bi for all i ∈ I + ;
• (Ax∗ )i ≥ bi for all i ∈ I \ I + ; (−Ax∗ )i ≥ −bi for all i ∈ I \ I + ;
• x∗j ≥ 0 for all j ∈ J + .
Let
ϕ(x1 , x2 , . . . , xn ) =
n
X
cj x j ,
j=1
ηi (x1 , x2 , . . . , xn ) =
n
X
(A)i,j xj
(i = 1, 2, . . . , m)
j=1
ζj (x1 , x2 , . . . , xn ) = xj
125
(j = 1, 2, . . . , n)
It follows from Proposition 5.19 that if there exists an optimal solution to
the primal problem then there exist non-negative quantities pi for all i ∈ I + ,
+
+
p+
such that
i and p− for all i ∈ I \ I and qj for all j ∈ J
X
X
X
−
ϕ=
pi η i +
(p+
−
p
)η
+
qj ζj .
i
i
i
i∈I +
i∈I\I +
j∈J +
Moreover pi = 0 whenever i ∈ I + and ηi (x∗1 , x∗2 , . . . , x∗n )i > bi and qj = 0
whenever x∗j > 0. Let p∗ ∈ Rm be defined such that (p∗ )i = pi for all i ∈ I +
−
+
∗T
and (p∗ )i = p+
i − pi for all i ∈ I \ I . Then (p A)j ≤ cj for j = 1, 2, . . . , n,
(p∗ )i ≥ 0 for all i ∈ I + , and (p∗T A)j = cj unless j ∈ J + . Moreover (p∗ )i = 0
whenever (Ax∗ )i > bi and qi = 0 whenever xj > 0. It follows that p∗ is a
feasible solution of the dual problem. Moreover the relevant complementary
slackness conditions are satisfied by x∗ and p∗ . It is then a consequence
of the Weak Duality Theorem that cT x∗ = p∗T b, and that therefore p∗
is an optimal solution of the dual problem (see Corollary 5.6). The result
follows.
126