UMass Lowell Computer Science 91.503
Analysis of Algorithms
Prof. Karen Daniels
Fall, 2006
Lecture 9
Wednesday, 11/15/06
Linear Programming
Overview




Motivation & Basics
Standard & Slack Forms
Formulating Problems as Linear Programs
Simplex Algorithm



Example
High-Level Algorithm
Correctness


Roadmap
Key Concepts



Initial Basic Feasible Solution
Duality
Literature Case Study
Motivation & Basics
Motivation: A Political Problem
Goal: Win election by winning majority of votes in each region.
100,000
voters
200,000 50,000
voters
voters
Thousands of
voters who could
be won with
$1,000 of ads
Subgoal: Win majority of votes in each region while
minimizing advertising cost.
source: 91.503 textbook Cormen et al.
Motivation: A Political Problem
(continued)
urban
suburban
rural
Thousands of
voters
representing
majority.
source: 91.503 textbook Cormen et al.
General Linear Programs
real numbers
Linear function
variables
Linear
constraints
Linear inequalities
source: 91.503 textbook Cormen et al.
Overview of Linear Programming
Objective function
Convex feasible
region
Objective value
source: 91.503 textbook Cormen et al.
Standard & Slack Forms
Standard Form
objective function
constraints
source: 91.503 textbook Cormen et al.
Standard Form (compact)
n-dimensional
vectors
mxn matrix
m-dimensional
vector
Can specify linear program in standard form by (A,b,c).
source: 91.503 textbook Cormen et al.
Converting to Standard Form
source: 91.503 textbook Cormen et al.
Converting to Standard Form
(continued)
Transforming minimization to maximization
Negate coefficients
source: 91.503 textbook Cormen et al.
Converting to Standard Form
(continued)
Giving each variable a non-negativity constraint
New non-negativity constraints
If xj has no non-negativity constraint,
replace each occurrence of xj with xj’ – xj”.
source: 91.503 textbook Cormen et al.
Converting to Standard Form
(continued)
Transforming equality constraints to inequality constraints
source: 91.503 textbook Cormen et al.
Converting to Standard Form
(continued)
Changing sense of an inequality constraint
Rationale:
source: 91.503 textbook Cormen et al.
Converting Linear Programs
into Slack Form
for algorithmic ease, transform all constraints except
slack variable
non-negativity ones into equalities
for inequality
constraint:
define
slack
instead of s
non-basic variables
basic
variables
source: 91.503 textbook Cormen et al.
Converting Linear Programs
into Slack Form (continued)
objective
function
source: 91.503 textbook Cormen et al.
Converting Linear Programs
into Slack Form (continued)
set of indices of basic variables
set of indices of non-basic variables
Compact Form: (N, B, A, b, c, v)
Slack Form Example
Compact Form
negative of slack form coefficients
source: 91.503 textbook Cormen et al.
Formulating Problems
as Linear Programs
Shortest Paths
Single-pair shortest path: minimize “distance” from source s to sink t.
Can we replace maximize with minimize here? Why or why not?
source: 91.503 textbook Cormen et al.
Maximum Flow
source: 91.503 textbook Cormen et al.
Minimum Cost Flow
source: 91.503 textbook Cormen et al.
Multicommodity Flow
should be si
source: 91.503 textbook Cormen et al.
Simplex Method
Solving a Linear Program

Simplex algorithm




Ellipsoid method


Geometric interpretation
 Visit vertices on the boundary of the simplex
representing the convex feasible region
Transforms set of inequalities using process similar
to Gaussian elimination
Run-time
 not polynomial in worst-case
 often very fast in practice
Run-time
 polynomial
 slow in practice
Interior-Point methods


Run-time
 polynomial
 for large inputs, performance can be competitive
with simplex method
source: 91.503 textbook Cormen et al.
Moves through interior of feasible region
Simplex Algorithm: Example
Basic Solution
Standard
Form
Slack
Form
Basic Solution:
( x1 , x2  x6 )  (0,0,0,30,24,36)
Basic Solution: set each nonbasic variable to 0.
source: 91.503 textbook Cormen et al.
Simplex Algorithm: Example
Reformulating the LP Model
Main Idea: In each iteration, reformulate the LP
model so basic solution has larger objective value
Select a nonbasic variable whose
objective coefficient is positive: x1
Increase its value as much as possible.
Identify tightest constraint on increase.
For basic variable x6 of that constraint,
swap role with x1.
Rewrite other equations with x6 on RHS.
new objective value
entering variable
PIVOT
leaving variable
source: 91.503 textbook Cormen et al.
Simplex Algorithm: Example
Reformulating the LP Model
Next Iteration: select x3 as entering variable.
entering variable
new objective value
PIVOT
leaving variable
New Basic Solution: ( x1 , x2  x6 )  (33 / 4,0,3 / 2,69 / 4,0,0)
source: 91.503 textbook Cormen et al.
Simplex Algorithm: Example
Reformulating the LP Model
Next Iteration: select x2 as entering variable.
new objective value
entering variable
PIVOT
leaving variable
New Basic Solution: ( x1 , x2  x6 )  (8,4,0,18,0,0)
source: 91.503 textbook Cormen et al.
Simplex Algorithm: Pivoting
leaving variable
entering variable
Rewrite the
equation that has
xl on LHS to have
xe on LHS
Update remaining
equations by substituting
RHS of new equation for
each occurrence of xe.
Do the same for
objective function.
Update sets of
nonbasic, basic
variables.
source: 91.503 textbook Cormen et al.
Simplex Algorithm: Pseudocode
initial basic solution
to be defined later
(detects infeasibility)
detects unboundedness
optimal solution
source: 91.503 textbook Cormen et al.
Correctness: Roadmap
(Key Pieces)
Lemma 29.3: Algebraic lemma
Lemma 29.8:
Weak LP duality
Lemma 29.4: Slack form uniqueness
Lemma 29.1: Pivot results
Lemma 29.11: Laux
Lemma 29.12:
Infeasibility
detection
n  m
29.5:  m 


Lemma 29.6:
Lemma
Tie-breaking
Iteration bound, cycling
Lemma 29.2:
Basic solution
feasible -> if
SIMPLEX finds
solution it is
feasible; if reports
unbounded, then
model is
unbounded
Corollary 29.9:
Conditions for
which feasible
solutions for primal,
dual programs are
optimal
Lemma 29.7: Basic solution
feasible -> SIMPLEX either
reports unbounded or finds
feasible solution in  n  m 
 m 


iterations
Theorem 29.10:
LP duality:
SIMPLEX primal
result is optimal &
dual is optimal
Theorem 29.13: Fundamental Theorem of Linear Programming
For LP model in standard form, either:
1. exists optimal solution with finite objective function value & SIMPLEX returns one, or
2. infeasible & SIMPLEX returns INFEASIBLE, or
3. Unbounded & SIMPLEX returns UNBOUNDED
Proof of Correctness Key Concepts
Initial Basic Feasible Solution
Finding an Initial Solution
An LP model whose initial basic solution is not feasible
source: 91.503 textbook Cormen et al.
Finding an Initial Solution
(continued)
Auxiliary LP model Laux:
source: 91.503 textbook Cormen et al.
Finding an Initial Solution
(continued)
source: 91.503 textbook Cormen et al.
Finding an Initial Solution
(continued)
Original LP model
Laux
Laux in slack form
source: 91.503 textbook Cormen et al.
Finding an Initial Solution
(continued)
PIVOT
PIVOT
source: 91.503 textbook Cormen et al.
Finding an Initial Solution
(continued)
source: 91.503 textbook Cormen et al.
Proof of Correctness Key Concepts
Duality
Linear Programming Duality
max becomes min
x variables go away
RHS coefficients
swap places with
objective function
coefficients
y variables appear
sense changes
source: 91.503 textbook Cormen et al.
Duality Example
source: 91.503 textbook Cormen et al.
Weak Linear Programming Duality
Any feasible solution to primal LP has value no greater
than that of any feasible solution to the dual LP.
source: 91.503 textbook Cormen et al.
Weak Linear Programming Duality
(continued)
source: 91.503 textbook Cormen et al.
Finding a Dual Solution
Finding a dual solution whose value is equal to that of an
optimal primal solution…
source: 91.503 textbook Cormen et al.
Optimality
source: 91.503 textbook Cormen et al.
Literature Case Study