15.081J/6.251J Introduction to Mathematical
Programming
Lecture 2: Geometry of Linear Optimization I
1 Outline:
1.
2.
3.
4.
5.
What is the central problem?
Standard Form.
Preliminary Geometric Insights.
Geometric Concepts (Polyhedra, \Corners").
Equivalence of algebraic and geometric concepts.
2 Central Problem
minimize
subject to
cx
ai x =
ai x ai x Slide 2
0
0
0
0
2 M1
2 M2
i 2 M3
j 2 N1
j 2 N2
i
i
bi
b
i
b
i
j0
>
j <0
x
x
2.1 Standard Form
Slide 3
cx
Ax = b
x0
minimize
subject to
0
Characteristics
Slide 1
Minimization problem
Equality constraints
Non-negative variables
2.2 Transformations
max c x
0
ai x 0
ai x 0
>
xj < 0
; min(;c x)
ai x + si = bi si 0
0
i
0
b
i
b
,
ai x ; i =
0
s
i
b j = xj ; xj
;
j 0 xj 0
+
x
+
x
1
;
i0
s
Slide 4
2.3 Example
maximize
subject to
x1
Slide 5
; x2
+ 21
+2 2 1
>
1 <0 2 0
x1
x
x1
x
x
x
+
;minimize ;x+1 + x1 + x2
subject to x+1 ; x1 + x2 + s1
=1
+
x1 ; x1 + 2x2
; s2 = 1
+
x1 x1 x2 s1 s2 0
;
;
;
;
3 Preliminary Insights
Slide 6
minimize ; 1 ; 2
subject to 1 + 2 2 3
2 1+ 23
1 20
x
x
x
x
x
x
x x
x2
3
- x1 - x2 = - 2
1.5
- x1 - x2 = z
(1,1)
- x1 - x2 = 0
3
1.5
c
x1
x1 + 2x2 < 3
2x1 + x2 < 3
;x1 + x2 1
x1 0
x2 0
There exists a unique optimal solution.
There exist multiple optimal solutions in this case, the set of optimal
solutions can be either bounded or unbounded.
2
Slide 7
Slide 8
x2
c = (1,0)
c = (- 1,- 1)
1
c = (0,1)
c = (1,1)
x1
The optimal cost is ;1, and no feasible solution is optimal.
The feasible set is empty.
4 Polyhedra
4.1 Denitions
Slide 9
The set fx j a x = g is called a hyperplane.
The set fx j a x g is called a halfspace.
The intersection of many halfspaces is called a polyhedron.
A polyhedron is a convex set, i.e., if x y 2 , then x + (1 ; )y 2 .
0
b
0
b
P
P
= b3
x=
a '2
a '3x
b2
a3
a2
a' x < b
a1
1
a 4' x = b4
=b
=b
a4
a5
a
a5' x = b
5
(b)
(a)
3
1x
a' x
a' x > b
a'
P
5 Corners
5.1 Extreme Points
Slide 10
Polyhedron P = fx j Ax bg
x 2 P is an extreme point of P
if 6 9 y z 2 P (y =
6 x z 6= x):
x = y + (1 ; )z 0 < < 1
.
.
.
u
w
v
P
.
.
.
z
x
y
5.2 Vertex
Slide 11
x 2 is a vertex of if 9 c:
x is the unique optimum
P
P
minimize
subject to
5.3 Basic Feasible Solution
P
= f(
x1 x2 x3
)j
x1
+
x2
+
cy
y2
0
x3
P
=1
x1 x2 x3
0g
Slide 12
Slide 13
Points A,B,C : 3 constraints active
Point E: 2 constraints active
suppose we add 2 1 + 2 2 + 2 3 = 2.
x
x
x
4
.
w
{y |
c 'y
=c
'w }
P
c
.
c
x
'x }
'y = c
{y | c
x3
.
A
E
D
.
.
.
P
x2
C
.
B
x1
Then 3 hyperplanes are tight, but constraints are not linearly independent.
Intuition: a point at which inequalities are tight and corresponding equations
are linearly independent.
= fx 2 <n j Ax bg
Slide 14
n
P
a1 : : : am rows of A
x2P
I = fi j ai x = bi g
0
Denition x is a basic feasible solution if subspace spanned by fai 2 g
i
is <n .
5.3.1 Degeneracy
If jI j = n, then ai i 2 I are linearly independent x nondegenerate.
5
I
Slide 15
If jI j > n, then there exist n linearly independent fai i 2 I g x degener-
ate.
A
D
C
P
B
E
(a)
5.3.2 Example
(b)
min
st
: :
+ 52
1+
2+
x1
x
x
x
Slide 16
;2x3
4
2
3
6
0
x3
x1
3 2+
x1 x3
x
x3
x2 x3
6 Equivalence of denitions
Slide 18
Theorem: = fx j Ax bg. Let x 2 .
P
Slide 17
P
x is a vertex , x is an extreme point , x is a BFS.
6.1 Proof
1. Vertex ) extreme point
9c : c x c y 8y 2
If x is not an extreme point 9y z 6= x:
x = y + (1 ; )z . But c x c y c x c z
) c x = c y + (1 ; )c z c x contradiction
0
<
0
Slide 19
P
0
0
0
0
0
<
<
0
<
0
0
2. Extreme point ) BFS
Suppose x is not a BFS.
6
Slide 20
Let = f : ai x = ig. But ai do not
span all of <n ) 9z 2 <n : ai z = 0 2
Let
x1 = x + z
x2 = x ; z )
aix1 = i
aix2 = i 2
I
0
i
b
0
i
I
0
b
0
i
i
b
I
62 I : aix < bi ) ai (x + z ) < bi 0
ai (x ; z)
0
0
Slide 21
i
< b
for small enough.
) x1 x2 2 : yet x = x1 +2 x2 )
x not an extreme point: contradiction
3. BFS ) vertex
x BFS
= f : ai x = i g
Let i = 10 262
c = ;d A
m
P
P
P .
Then c x = ;d Ax = ;
iai x = ; ai x = ;
i
i�1
i I
i I
But 8x 2 P : ai x i )
x optimum
min
cx
c x = ; P aix ; P i = c x
P
Slide 22
�
I
0
i
�
b
d
i
I
i
I:
0
�
0
0
0
�
0
0
d
�
0
�
b
2
2
b
�
0
i2I
0
Why unique?
i2I
b
0
�
0
x2
7
P:
Slide 23
Equality holds if ai x = i 2 since ai spans <n
solution x = x .
0
b i
I
�
8
aix = i has a unique
0
b
MIT OpenCourseWare
http://ocw.mit.edu
6.251J / 15.081J Introduction to Mathematical Programming
Fall 2009
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.