15.081J/6.251J Introduction to Mathematical
Programming
Lecture 3: Geometry of Linear Optimization II
1 Outline
Slide 1
BFS for standard form polyhedra
Deeper understanding of degeneracy
Existence of extreme points
Optimality of Extreme Points
Representation of Polyhedra
2 BFS for standard form polyhedra
Ax = b and x 0
m n matrix A has linearly independent rows
x 2 <n is a basic solution if and only if Ax = b, and there exist indices
Slide 2
B(1) : : : B (m) such that:
{ The columns AB(1) : : : AB(m) are linearly independent
{ If i 6= B(1) : : : B (m), then xi = 0
2.1 Construction of BFS
Slide 3
Procedure for constructing basic solutions
1. Choose m linearly independent columns AB(1) : : : AB(m)
2. Let xi = 0 for all i =
6 B(1) : : : B (m)
3. Solve Ax = b for xB(1) : : : xB(m)
Ax = b ! BxB + NxN = b
xN = 0 xB = B; b
1
2.2 Example 1
21 1 2 1 0 0 03 2 83
66 0 1 6 0 1 0 0 77 x = 66 12 77
41 0 0 0 0 1 05 4 45
0 1 0 0 0 0 1
A A A A basic columns
4
5
6
7
1
6
Slide 4
Solution: x = (0 0 0 8 12 4 6), a BFS
Another basis: A3 A5 A6 A7 basic columns.
Solution: x = (0 0 4 0 ;12 4 6), not a BFS
2.3 Geometric intuition
Slide 5
A3
b
A1
A2
A4 = - A1
2.4 Example 2
General form
Slide 6
Slide 7
x1 + x2 + x3
x1
x3
3x2 + x3
x1 x2 x3
2
4
2
3
6
0
Standard form
x1 + x2 + x3 + s1 = 4
x1 + s2
= 2
x3 + s3
= 3
3x2 + x3 + s4
= 6
x1 x2 x3 s1 : : : s4 0
Using the denition for BFS in polyhedra in general form :
9
x1 + x2 + x3 = 4 =
Choose tight constraints: x3
= 3 ) (1 0 3)
x2
=0
011 001 001
Check if @ 1 A @ 0 A @ 1 A span <3 (they do)
1
1
0
Slide 8
Slide 9
Using the denition for BFS in polyhedra in standard form :
Pick the basic variables: x1 x3 s2 s3 : xB = (x1 x3 s2 s3)
Pick the nonbasic variables: x2 s1 s4 : xN = (x2 s1 s4 )
Slide 10
Partition A:
x1
1
1
0
0
2
A = 664
x2
1
0
3
0
x3
1
0
1
1
s1
1
0
0
0
s2
0
1
0
0
21 1 0 03
21
6
7
6
B = 64 10 01 10 01 57 N = 64 03
0 1 0 0
0 x 1 0 1 10
xN = 0 xB = B; b ) BB@ xs CCA = BB@ 13 CCA
s3
0
0
1
0
s4
0
0
0
1
1
0
0
0
0
0 77 B non-singular
05
1
3
77 = B N ]
5
3
Slide 11
1
1
3
2
s3
3
3 Degeneracy for standard form polyhedra
3.1 Denition
A BFS x of P = fx 2 <n : Ax = b A : n n x 0g is called degenerate
if it contains more than n ; m zeros.
x is non-degenerate if it contains exactly n ; m zeros.
3
Slide 12
3.2 Example 2, revisited
Slide 13
In previous example:
(2 2 0 0 0 3 0)degenerate : n = 7
m=4
More than n ; m = 7 ; 4 = 3 zeros.
Ambiguity about which are basic variables.
(x1 x2 x3 x6) one choice
(x1 x2 x6 x7) another choice
3.3 Extreme points and BFS
Slide 14
Consider again the extreme point (2 2 0 0 0 6 0)
How do we construct the basis?
8
<
B=
:
0
@
1 0
A @
1
1
0
0
A
1
0
0
3
A
1
2
1 0
A @
1
A
0
0
1
0
A
9
=
6
Slide 15
Columns in B are linearly independent.
Rank (A) = 4
jBj = 3 < 4
Can we augment B?
Choices:
{ B0 = B fA3 g basic variables x1 x2 x3 x6
{ B0 = B fA7 g basic variables x1 x2 x6 x7
{ How many choices do we have?
3.4 Degeneracy and geometry
Whether a BFS is degenerate may depend on the particular representation
of a polyhedron.
n
o
P = (x1 x2 x3) x1 ; x2 = 0 x1 + x2 + 2x3 = 2 x1 x2 x3 0 :
4
Slide 16
n = 3, m = 2 and n ; m = 1. (1 1 0) is nondegenerate, while (0,0,1) is
degenerate.
n
Consider the representation P = (x1 x2 x3) x1 ;x2 = 0 x1 +x2 +2x3 =
o
2 x1 0 x3 0 : (0,0,1) is now nondegenerate.
3.5 Conclusions
An extreme point corresponds to possibly many bases in the presence of
degeneracy.
A basic feasible solution, however, corresponds to a unique extreme point.
Degeneracy is not a purely geometric property.
4 Existence of
extreme points
Slide 17
Slide 18
P
Q
Note that P = f(x1 x2) : 0 x1 1g does not have an extreme point, while
P 0 = f(x1 x2) : x1 x2 x1 0 x2 0g has one. Why?
4.1 Denition
Slide 19
4.2 Theorem
Slide 20
A polyhedron P <n contains a line if there exists a vector x 2 P and a
nonzero vector d 2 <n such that x + d 2 P for all scalars .
Suppose that the polyhedron P = fx 2 <n j ai 0 x bi i = 1 : : : mg is
nonempty. Then, the following are equivalent:
(a) The polyhedron P has at least one extreme point.
(b) The polyhedron P does not contain a line.
(c) There exist n vectors out of the family a1 : : : am , which are linearly
independent.
5
4.3 Corollary
Slide 21
Polyhedra in standard form contain an extreme point.
Bounded polyhedra contain an extreme point.
4.4 Proof
Let P = fx j Ax = b x 0g 6= rank(A) = m. If there exists a feasible
solution in P, then there is an extreme point.
Proof
Slide 22
Let x = (x1 : : : xt 0 : : : 0), s.t. x 2 P. Consider B = fA1 A2 : : : At g
If fA1 A2 : : : At g are linearly independent we can augment, to nd a
basis, and thus a BFS exists.
If fA1 A2 : : : Atg are dependent
d1A1 + + dtAt = 0 (di 6= 0)
But x1A1 + + xtAt = b
) (x1 + d1)A1 + + (xt + dt )At = b
Consider xj () =
Clearly A x() = b
Let:
x + d
j
j
j = 1 : : :t
otherwise.
0
1 = max
d >0
n
j
; xdjj
j
j
j
n x
2 = dmin
>0 ; d
o
o
For
Slide 23
(if all dj 0)
1 = ;1)
(if all dj 0)
2 = +1
(suciently small)
1 2
x() 0
Since at least one (d1 : : : dt) 6= 0 ) at least one from 1 2 is nite, say 1 .
But then x(1 ) 0 and number of nonzeros decreased.
xj + dj 0 ) xj ;dj
Slide 24
4.5 Example 3
P = fx j x1 + x2 + x3
=2
x1
+x4 = 1 x1 : : : x4 0g
x = ; 12 21 1 21 6
Slide 25
1
1 1 1 0 B= 1 0 0 1
1 1 1 0 0
1
;
+0
0
1
0
;
=
1
0
Slide 26
Consider: x() = 2 + 12 ; 1 12 ; for ; 12 12 :
x() 2 P.
Note x(; 12 ) = (0 1 1 1) and x( 21 ) = (1 0 1 0).
5 Optimality of
Extreme Points
5.1 Theorem
Slide 27
Consider
min c x
s:t: x 2 P = fx 2 <n j Ax bg:
P has no line and it has an optimal solution.
Then, there exists an optimal solution which is an extreme point of P.
0
5.2 Proof
Slide 28
v optimal value of the cost c x.
Q : set of optimal solutions, i.e.,
Q = fx j c x = v Ax bg
Q P and P contains no lines, Q does not contain any lines, hence is has
an extreme point x� .
0
0
Slide 29
Claim: x� is an extreme point of P.
Suppose not 9 y w 6= x� : x� = y + (1 ; )w y w 2 P 0 < < 1:
v = c x� = c y + (1 ; )c w
cyv )
c w v ) c y = c w = v ) y w 2 Q
) x� is NOT an extreme point of Q, CONTRADICTION.
0
0
0
0
0
0
0
7
6 Representation
of Polyhedra
6.1 Theorem
A nonempty and bounded polyhedron is the convex hull of its extreme points.
.
y
.
z
.
P
u
Q
8
a 'i*x = bi*
Slide 30
MIT OpenCourseWare
http://ocw.mit.edu
6.251J / 15.081J Introduction to Mathematical Programming
Fall 2009
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.