Maximum Flow and Minimum Cut
Max Flow, Min Cut
Max flow and min cut.
Two very rich algorithmic problems.
n
Cornerstone problems in combinatorial optimization.
n
Beautiful mathematical duality.
Nontrivial applications / reductions.
Minimum cut
Maximum flow
Max-flow min-cut theorem
Ford-Fulkerson augmenting path algorithm
Edmonds-Karp heuristics
Bipartite matching
Princeton University • COS 226 • Algorithms and Data Structures • Spring 2004 • Kevin Wayne •
n
n
Network connectivity.
n
Bipartite matching.
n
Data mining.
n
Open-pit mining.
n
Airline scheduling.
n
Image processing.
n
Project selection.
n
Baseball elimination.
n
Network reliability.
n
Security of statistical data.
n
Distributed computing.
n
Egalitarian stable matching.
n
Distributed computing.
n
Many many more . . .
http://www.Princeton.EDU/~cos226
2
Soviet Rail Network, 1955
Minimum Cut Problem
Network: abstraction for material FLOWING through the edges.
n
Directed graph.
n
Capacities on edges.
n
Source node s, sink node t.
Min cut problem. Delete "best" set of edges to disconnect t from s.
source
s
capacity
Source: On the history of the transportation and maximum flow problems.
Alexander Schrijver in Math Programming, 91: 3, 2002.
3
2
9
5
10
4
15
15
10
5
3
8
6
10
15
4
6
15
10
4
30
7
t
sink
4
Cuts
Cuts
A cut is a node partition (S, T) such that s is in S and t is in T.
n
A cut is a node partition (S, T) such that s is in S and t is in T.
capacity(S, T) = sum of weights of edges leaving S.
10
5
s
n
2
9
5
4
15
15
10
3
8
6
10
4
6
15
10
4
30
7
capacity(S, T) = sum of weights of edges leaving S.
10
t
S
15
5
s
S
15
Capacity = 30
2
9
5
4
15
15
10
3
8
6
10
4
6
15
10
4
30
7
t
Capacity = 62
5
6
Minimum Cut Problem
Maximum Flow Problem
A cut is a node partition (S, T) such that s is in S and t is in T.
n
Network: abstraction for material FLOWING through the edges.
capacity(S, T) = sum of weights of edges leaving S.
Min cut problem. Find an s-t cut of minimum capacity.
n
Directed graph.
n
Capacities on edges.
n
Source node s, sink node t.
same input as min cut problem
Max flow problem. Assign flow to edges so as to:
10
5
s
2
9
5
4
15
15
3
8
6
n
Equalize inflow and outflow at every intermediate vertex.
n
Maximize flow sent from s to t.
10
t
source
S
15
4
6
15
4
30
7
2
9
5
10
4
15
15
10
5
3
8
6
10
15
4
6
15
10
4
30
7
10
10
s
capacity
Capacity = 28
7
t
sink
8
Flows
Flows
A flow f is an assignment of weights to edges so that:
n
n
A flow f is an assignment of weights to edges so that:
Capacity: 0 £ f(e) £ u(e).
n
Flow conservation: flow leaving v = flow entering v.
n
Capacity: 0 £ f(e) £ u(e).
Flow conservation: flow leaving v = flow entering v.
except at s or t
4
10
s
capacity
flow
0
5
15
0
except at s or t
2
0
9
5
4 4
0
15
15 0
0
10
3
4
8
6
4
10
4 0
0
6
15 0
0
10
4
0
30
7
10
10
t
s
capacity
flow
Value = 4
3
5
15
11
2
6
9
5
4 4
0
15
15 0
6
10
3
8
8
6
8
10
4 0
1
6
15 0
10
10
4
11
30
t
Value = 24
7
9
10
Maximum Flow Problem
Flows and Cuts
Max flow problem: find flow that maximizes net flow into sink.
10
10
s
capacity
flow
4
5
15
14
2
9
9
4 0
1
15
15 0
9
10
3
8
8
6
9
10
4 0
4
6
15 0
10
10
4
14
30
7
Observation 1. Let f be a flow, and let (S, T) be any s-t cut. Then, the
net flow sent across the cut is equal to the amount reaching t.
5
10
10
t
s
S
Value = 28
11
4
5
10
15
2
6
9
5
4 4
0
15
15 0
6
10
3
8
8
6
8
10
4 0
0
6
15 0
10
10
4
10
30
7
t
Value = 24
12
Flows and Cuts
Flows and Cuts
Observation 1. Let f be a flow, and let (S, T) be any s-t cut. Then, the
net flow sent across the cut is equal to the amount reaching t.
Observation 1. Let f be a flow, and let (S, T) be any s-t cut. Then, the
net flow sent across the cut is equal to the amount reaching t.
10
10
s
S
4
5
10
15
2
6
9
5
4 4
0
15
15 0
6
10
3
8
8
6
8
10
4 0
0
6
15 0
10
10
4
10
30
7
10
10
t
s
S
4
5
10
15
Value = 24
2
6
9
5
4 4
0
15
15 0
6
10
3
8
8
6
8
10
4 0
0
6
15 0
10
10
4
10
30
7
t
Value = 24
13
14
Flows and Cuts
Max Flow and Min Cut
Observation 2. Let f be a flow, and let (S, T) be any s-t cut. Then the
value of the flow is at most the capacity of the cut.
Cut capacity = 30 Þ
2
10
s
5
4
3
9
15
8
Observation 3. Let f be a flow, and let (S, T) be an s-t cut whose capacity
equals the value of f. Then f is a max flow and (S, T) is a min cut.
Cut capacity = 28 Þ
Flow value £ 30
5
15
6
10
10
10
10
s
t
S
S
15
4
4
6
30
15
10
7
15
4
5
15
15
Flow value £ 28
Flow value = 28
2
9
9
5
4 0
1
15
15 0
9
10
3
8
8
6
9
10
4 0
4
6
15 0
10
10
4
15
30
7
t
16
Max-Flow Min-Cut Theorem
Towards an Algorithm
Max-flow min-cut theorem. (Ford-Fulkerson, 1956): In any network,
the value of max flow equals capacity of min cut.
n
Find s-t path where each arc has f(e) < u(e) and "augment" flow along it.
Proof IOU: we find flow and cut such that Observation 3 applies.
Min cut capacity = 28
10
10
s
S
4
5
15
15
4
2
9
9
5
4 0
1
15
15 0
9
10
3
8
8
6
9
10
4 0
4
6
15 0
10
10
4
15
30
7
Flow value = 0
flow
Û Max flow value = 28
0
4
s
0
10
0
4
0
4
2
capacity
5
0
4
0
13
3
0
10
t
t
17
18
Towards an Algorithm
Towards an Algorithm
Find s-t path where each arc has f(e) < u(e) and "augment" flow along it.
Find s-t path where each arc has f(e) < u(e) and "augment" flow along it.
n
Greedy algorithm: repeat until you get stuck.
0
4
s
X
0 10
10
0
4
0
4
2
Greedy algorithm: repeat until you get stuck.
n
Fails: need to be able to "backtrack."
Flow value = 10
flow
4
n
capacity
5
4
0
4
0
4
X
0 10
13
3
X
0 10
10
Flow value = 10
flow
t
s
X
0 10
10
0
4
2
4
Bottleneck capacity of path = 10
4
4
19
s
10
10
0
4
5
0
4
X
0 10
13
3
5
4
4
4
4
2
capacity
X
0 10
10
t
Flow value = 14
4
4
6
13
3
10
10
t
20
Residual Graph
Augmenting Paths
Augmenting path = path in residual graph.
flow = f(e)
Original graph.
n
Flow f(e).
n
Edge e = v-w
6
17
v
w
Increase flow along forward edges.
n
Decrease flow along backward edges.
capacity = u(e)
4
n
Edge e = v-w or w-v.
n
"Undo" flow sent.
5
4
4
residual
Residual edge.
4
4
v
All the edges that have
strictly positive residual capacity.
10
s
residual capacity = u(e) – f(e)
Residual graph.
n
n
10
2
3
10
t
3
w
11
4
6
4 0
X
residual capacity = f(e)
4X
0
4
10
10
s
X
0 4
4
4
original
5
4 X
0
2
4
10
10
X 6
13
3
10
t
21
22
Augmenting Paths
Ford-Fulkerson Augmenting Path Algorithm
Observation 4. If augmenting path, then not yet a max flow.
Q. If no augmenting path, is it a max flow?
4
while (there exists an augmenting path) {
Find augmenting path P
Compute bottleneck capacity of P
Augment flow along P
}
5
4
4
residual
Ford-Fulkerson algorithm. Generic method for solving max flow.
4
4
s
10
6
2
3
10
t
7
4
4 0
X
4X
0
4
10
s
10
5
4 X
0
2
Flow value = 14
X
0 4
4
4
original
Questions.
4
10
X 6
13
10
3
10
n
Does this lead to a maximum flow?
yes
n
How do we find an augmenting path?
s-t path in residual graph
n
How many augmenting paths does it take?
n
How much effort do we spending finding a path?
t
23
24
Max-Flow Min-Cut Theorem
Proof of Max-Flow Min-Cut Theorem
Augmenting path theorem. A flow f is a max flow if and only if there
are no augmenting paths.
(ii) Þ (iii). If there is no augmenting path relative to f, then there
exists a cut whose capacity equals the value of f.
Proof.
Max-flow min-cut theorem. The value of the max
flow is equal to the capacity of the min cut.
n
n
We prove both simultaneously by showing the following are equivalent:
(i) f is a max flow.
Let f be a flow with no augmenting paths.
Let S be set of vertices reachable from s in residual graph.
– S contains s; since no augmenting paths, S does not contain t
– all edges e leaving S in original network have f(e) = u(e)
– all edges e entering S in original network have f(e) = 0
(ii) There is no augmenting path relative to f.
(iii) There exists a cut whose capacity equals the value of f.
(i) Þ (ii)
(ii) Þ (iii)
(iii) Þ (i)
f
equivalent to not (ii) Þ not (i), which was Observation 4
next slide
this was Observation 3
=
=
S
å f (e ) - å f (e )
e out of S
T
e in to S
t
å u (e )
e out of S
= capacity (S, T)
s
residual network
25
26
Max Flow Network Implementation
Ford-Fulkerson Algorithm: Implementation
Edge in original graph may correspond to 1 or 2 residual edges.
Ford-Fulkerson main loop.
n
May need to traverse edge e = v-w in forward or reverse direction.
n
Flow = f(e), capacity = u(e).
n
Insert two copies of each edge, one in adjacency list of v and one in w.
public class Edge {
private int v, w;
private int cap;
private int flow;
// while there exists an augmenting path, use it
while (augpath()) {
// compute bottleneck capacity
int bottle = INFINITY;
for (int v = t; v != s; v = ST(v))
bottle = Math.min(bottle, pred[v].capRto(v));
// from, to
// capacity from v to w
// flow from v to w
// augment flow
for (int v = t; v != s; v = ST(v))
pred[v].addflowRto(v, bottle);
public Edge(int v, int w, int cap) { ... }
public int cap() { return cap; }
public int flow() { return flow; }
public
public
public
public
boolean from(int v)
int other(int v)
int capRto(int v)
void addflowRto(int
{ return this.v == v;
{ return from(v) ? this.w : this.v;
{ return from(v) ? flow : cap - flow;
v, int d) { flow += from(v) ? -d : d;
// keep track of total flow sent from s to t
value += bottle;
}
}
}
}
}
}
27
28
Ford-Fulkerson Algorithm: Analysis
Choosing Good Augmenting Paths
Assumption: all capacities are integers between 1 and U.
Use care when selecting augmenting paths.
Invariant: every flow value and every residual capacities remain an
integer throughout the algorithm.
4
0
100
Theorem: the algorithm terminates in at most | f * | £ V U iterations.
Corollary: if U = 1, then algorithm runs in £ V iterations.
not polynomial
in input size!
0
100
1 0
s
0
100
Integrality theorem: if all arc capacities are integers, then there
exists a max flow f for which every flow value is an integer.
t
0
100
2
Original Network
29
30
Choosing Good Augmenting Paths
Choosing Good Augmenting Paths
Use care when selecting augmenting paths.
1
0
X
100
4
0
100
0
1 X
100
0
100
Original Network
4
1
1 X
0
s
Use care when selecting augmenting paths.
0
100
1
100
1 1
s
t
0
100
2
Original Network
31
t
1
100
2
32
Choosing Good Augmenting Paths
Choosing Good Augmenting Paths
Use care when selecting augmenting paths.
Use care when selecting augmenting paths.
4
4
1
100
0 1
X
100
0
1 X
1
s
X
0 1
100
1
100
1 0
s
t
1
100
1
100
2
Original Network
1
100
t
1
100
2
Original Network
200 iterations possible!
33
34
Choosing Good Augmenting Paths
Shortest Augmenting Path
Use care when selecting augmenting paths.
Shortest augmenting path.
n
Some choices lead to exponential algorithms.
n
Easy to implement with BFS.
n
Clever choices lead to polynomial algorithms.
n
Finds augmenting path with fewest number of arcs.
n
Optimal choices for real world problems ???
while (!q.isEmpty()) {
int v = q.dequeue();
IntIterator i = G.neighbors(v);
while(i.hasNext()) {
Edge e = i.next();
int w = e.other(v);
if (e.capRto(w) > 0) { // is v-w a residual edge?
if (wt[w] > wt[v] + 1) {
wt[w] = wt[v] + 1;
pred[w] = e;
// keep track of shortest path
q.enqueue(w);
}
}
}
}
return (wt[t] < INFINITY);
// is there an augmenting path?
Design goal is to choose augmenting paths so that:
n
Can find augmenting paths efficiently.
n
Few iterations.
Choose augmenting path with:
Edmonds-Karp (1972)
n
Fewest number of arcs.
(shortest path)
n
Max bottleneck capacity.
(fattest path)
35
36
Shortest Augmenting Path Analysis
Fattest Augmenting Path
Length of shortest augmenting path increases monotonically.
Fattest augmenting path.
n
Strictly increases after at most E augmentations.
n
Finds augmenting path whose bottleneck capacity is maximum.
n
At most E V total augmenting paths.
n
Delivers most amount of flow to sink.
n
O(E2 V)
n
Solve using Dijkstra-style (PFS) algorithm.
running time.
12
v
X 10
9
w
10
residual capacity
if (wt[w] < Math.min(wt[v], e.capRto(w)) {
wt[w] = Math.min(wt[v], e.capRto(w));
pred[w] = v;
}
Finding a fattest path. O(E log V) per augmentation with binary heap.
Fact. O(E log U) augmentations if capacities are between 1 and U.
37
38
Choosing an Augmenting Path
History of Worst-Case Running Times
Choosing an augmenting path.
n
n
n
n
Any path will do Þ wide latitude in implementing Ford-Fulkerson.
Generic priority first search.
Some choices lead to good worst-case performance.
– shortest augmenting path
– fattest augmenting path
– variation on a theme: PFS
Average case not well understood.
Research challenges.
n
Practice: solve max flow problems on real networks in linear time.
n
Theory: prove it for worst-case networks.
Year
Discoverer
Method
Asymptotic Time
1951
Dantzig
Simplex
E V2 U
†
1955
Ford, Fulkerson
Augmenting path
EVU
†
1970
Edmonds-Karp
Shortest path
E2 V
1970
Edmonds-Karp
Max capacity
E log U (E + V log V) †
1970
Dinitz
Improved shortest path
E V2
1972
Edmonds-Karp, Dinitz
Capacity scaling
1973
Dinitz-Gabow
Improved capacity scaling
1974
Karzanov
Preflow-push
V3
1983
Sleator-Tarjan
Dynamic trees
E V log V
1986
Goldberg-Tarjan
FIFO preflow-push
E V log (V2 / E)
...
...
...
1997
Goldberg-Rao
Length function
E2
log U †
E V log U
†
...
E3/2
(V2
log
/ E) log U †
EV2/3 log (V2 / E) log U †
† Arc capacities are between 1 and U.
39
40
An Application
Bipartite Matching
Jon placement.
Bipartite matching.
n
Companies make job offers.
n
Input: undirected and bipartite graph G.
n
Students have job choices.
n
Set of edges M is a matching if each vertex appears at most once.
n
Max matching: find a max cardinality matching.
Can we fill every job?
Can we employ every student?
Alice-Adobe
Bob-Yahoo
Carol-HP
Dave-Apple
Eliza-IBM
Frank-Sun
1
A
2
B
Matching M
1-B, 3-A, 4-E
L
3
C
4
D
5
E
R
41
42
Bipartite Matching
Bipartite Matching
Bipartite matching.
Reduces to max flow.
n
Input: undirected and bipartite graph G.
n
Create a directed graph G'.
n
Set of edges M is a matching if each vertex appears at most once.
n
Direct all arcs from L to R, and give infinite (or unit) capacity.
n
Max matching: find a max cardinality matching.
n
Add source s, and unit capacity arcs from s to each node in L.
n
Add sink t, and unit capacity arcs from each node in R to t.
1
A
2
B
L
Matching M
1
A
2
B
1
1
1-A, 2-B, 3-C, 4-D
L
B
3
C
D
C
3
C
4
D
4
D
4
5
E
E
5
5
R
43
G
A
2
3
s
¥
R
1
t
E
G'
44
Bipartite Matching: Proof of Correctness
Bipartite Matching: Proof of Correctness
Claim. Matching in G of cardinality k induces flow in G' of value k.
Claim. Flow f of value k in G' induces matching of cardinality k in G.
Given matching M = { 1-B, 3-A, 4-E } of cardinality 3.
n
Consider flow f that sends 1 unit along each of 3 paths:
s-1-B-t s-3-A-t s-4-E-t.
n
By integrality theorem, there exists 0/1 valued flow f of value k.
n
Consider M = set of edges from L to R with f(e) = 1.
– each node in L and R incident to at most one edge in M
– |M| = k
n
f is a flow, and has cardinality 3.
n
L
1
A
2
B
3
C
4
5
1
1
¥
R
L
A
1
1
A
2
B
3
C
2
B
3
C
D
4
D
4
E
5
E
5
s
G
t
G'
45
Reduction
Reduction.
n
Given an instance of bipartite matching.
n
Transform it to a max flow problem.
n
Solve max flow problem.
n
Transform max flow solution to bipartite matching solution.
Issues.
n
How expensive is transformation?
O(E + V)
n
Is it better to solve problem directly? O(E V1/2) bipartite matching
Bottom line: max flow is an extremely rich problem-solving model.
n
Many important practical problems reduce to max flow.
n
We know good algorithms for solving max flow problems.
47
G
1
1
¥
R
A
2
B
3
C
D
4
D
E
5
E
s
G'
1
t
46