On Partitioning Graphs via
Single Commodity Flows
Lorenzo Orecchia
UC Berkeley
Leonard J. Schulman
Caltech
Umesh V. Vazirani
UC Berkeley
Nisheeth K. Vishnoi
IBM Delhi – work done while visiting UC Berkeley
STOC 2008 , Victoria
The SPARSEST CUT problem
Given a graph G=(V,E) and partition (S, S )
| E(S, S ) |
Expansion of (S, S ) =
min{| S |,| S |}
E(S, S )
S
S
The SPARSEST CUT problem
Given a graph G=(V,E) and partition (S, S )
| E(S, S ) |
Expansion of (S, S ) =
min{| S |,| S |}
SPARSEST CUT:
find (S, S ) with minimum expansion !(G).
Applications: Divide-and-Conquer, Image Segmentation,
VLSI design, Clustering.
Theoretical Importance: Metric Embeddings, Spectral
Methods.
The SPARSEST CUT problem
Given a graph G=(V,E) and partition (S, S )
| E(S, S ) |
Expansion of (S, S ) =
min{| S |,| S |}
SPARSEST CUT:
find (S, S ) with minimum expansion !(G).
Applications: Divide-and-Conquer, Image Segmentation,
VLSI design, Clustering.
Theoretical Importance: Metric Embeddings, Spectral
Methods.
The SPARSEST CUT problem is NP-hard.
Approximation Algorithms for
SPARSEST CUT
Algorithm
Output Expansion
**
Running Time *
& d2n # **
O$$ 2 !!
% ' "
Spectral
2 d!
Leigthon-Rao
! log n
~ 2
O(n )
! log n
[ ARV ] poly(n)
~
[ AHK ] O(n2 )
ARV
~
*All graphs have been sparsified to O(n) edges.
** For a d-regular graph G.
Approximation Algorithms for
SPARSEST CUT
Algorithm
Output Expansion
**
Running Time *
& d2n # **
O$$ 2 !!
% ' "
Spectral
2 d!
Leigthon-Rao
! log n
~ 2
O(n )
! log n
[ ARV ] poly(n)
~
[ AHK ] O(n2 )
ARV
IN PRACTICE: Too slow for massive data sets.
Spectral and heuristics like METIS used instead.
~
*All graphs have been sparsified to O(n) edges.
** For a d-regular graph G.
Fast Approximation Algorithms
Algorithm
Output Expansion
**
Running Time *
& d2n # **
O$$ 2 !!
% ' "
Spectral
2 d!
Leigthon-Rao
! log n
~ 2
O(n )
! log n
[ ARV ] poly(n)
~
[ AHK ] O(n2 )
ARV
KRV
!(log n)
2
~
O(n3 / 2 )
CUT-MATCHING GAME: FRAMEWORK FOR COMPUTING APPROX USING
s-t MAXFLOW COMPUTATIONS
~
*All graphs have been sparsified to O(n) edges. ** For a d-regular graph G.
Fast Approximation Algorithms
Algorithm
Output Expansion
Leigthon-Rao
! log n
Running Time
~ 2
O(n )
ARV
! log n
[ ARV ] poly(n)
~
[ AHK ] O(n2 )
KRV
!(log n)2
~
O(n3 / 2 )
! log n
~ 3/2
O(n )
AK
~
*All graphs have been sparsified to O(n) edges.
** For a d-regular graph G.
Our Contribution
Algorithm
Output Expansion
Running Time
KRV
!(log n)2
~
O(n3 / 2 )
! log n
~
O(n3 / 2 )
! log n
~
O(n3 / 2 )
AK
THIS PAPER
IN KRV CUT-MATCHING
GAME FRAMEWORK
Our Contribution
Algorithm
Output Expansion
Running Time
KRV
!(log n)2
~
O(n3 / 2 )
! log n
~
O(n3 / 2 )
! log n
~
O(n3 / 2 )
AK
THIS PAPER
IN KRV CUT-MATCHING
GAME FRAMEWORK
LOWER BOUND
No better approx than Ω((log n)1/2) in KRV framework.
Our Contribution
Algorithm
Output Expansion
KRV
!(log n)2
~ 3/2
O(n )
! log n
~ 3/2
O(n )
! log n
~ 3/2
O(n )
AK
THIS PAPER
Running Time
LOWER BOUND
No better approx than Ω((log n)1/2) in KRV framework.
Best integrality gap for SDP is Ω(loglog(n)).
Our Contribution
Algorithm
Output Expansion
KRV
!(log n)2
~ 3/2
O(n )
! log n
~ 3/2
O(n )
! log n
~ 3/2
O(n )
AK
THIS PAPER
Running Time
LOWER BOUND
No better approx than Ω((log n)1/2) in KRV framework.
Best integrality gap for SDP is Ω(loglog(n)).
CUT-MATCHING RIGHT ABSTRACTION?
The KRV Cut-Matching Game
CUT PLAYER
H0
MATCHING PLAYER
The KRV Cut-Matching Game
CUT PLAYER
H0
(S1, S1 )
50-50 Cut
MATCHING PLAYER
The KRV Cut-Matching Game
CUT PLAYER
H0
MATCHING PLAYER
(S1, S1 )
M1
50-50 Cut
Perfect Matching
The KRV Cut-Matching Game
CUT PLAYER
H1
MATCHING PLAYER
(S1, S1 )
M1
50-50 Cut
Perfect Matching
The KRV Cut-Matching Game
CUT PLAYER
H1
MATCHING PLAYER
(S1, S1 )
M1
50-50 Cut
Perfect Matching
(S 2 , S2 )
50-50 Cut
The KRV Cut-Matching Game
CUT PLAYER
H1
(S1, S1 )
50-50 Cut
MATCHING PLAYER
M1
Perfect Matching
(S 2 , S2 )
M1
50-50 Cut
Perfect Matching
The KRV Cut-Matching Game
CUT PLAYER
H2
MATCHING PLAYER
(S1, S1 )
M1
50-50 Cut
Perfect Matching
(S 2 , S2 )
M1
50-50 Cut
Perfect Matching
…
The KRV Cut-Matching Game
CUT PLAYER
H2
MATCHING PLAYER
(S1, S1 )
M1
50-50 Cut
Perfect Matching
(S 2 , S2 )
M1
50-50 Cut
Perfect Matching
…
Go until time T when
1
"(HT ) !
4
GOAL:
GOAL:
Minimize T
Maximize T
The KRV Cut-Matching Game
CUT PLAYER
H2
MATCHING PLAYER
(S1, S1 )
M1
50-50 Cut
Perfect Matching
(S 2 , S2 )
M1
50-50 Cut
Perfect Matching
…
Go until time T when
1
"(HT ) !
4
GOAL:
GOAL:
Minimize T
Maximize T
KRV: there exists a cut strategy achieving T = O((log n)2).
The KRV Cut-Matching Game
Runs in time c(n)
per iteration
CUT PLAYER STRATEGY
T = t(n)
The KRV Cut-Matching Game
Runs in time c(n)
per iteration
Running time:
t(n)· (Tmaxflow + c(n))
CUT PLAYER STRATEGY
APPROXIMATION
ALGORITHM
T = t(n)
Approx Ratio:
t(n)
The KRV Cut-Matching Game
Runs in time c(n)
per iteration
CUT PLAYER STRATEGY
Running time:
t(n)· (Tmaxflow + c(n))
Time to
compute s-t
maxflow in G
~
O(n3 / 2 )
APPROXIMATION
ALGORITHM
T = t(n)
Approx Ratio:
t(n)
The KRV Cut-Matching Game
Runs in time c(n)
per iteration
CUT PLAYER STRATEGY
Running time:
t(n)· (Tmaxflow + c(n))
APPROXIMATION
ALGORITHM
T = t(n)
Approx Ratio:
t(n)
~
KRV strategy has c(n) = O(n) and t(n)= O((log n)2).
~
O(n3 / 2 )
TOTAL RUNNING TIME:
~ 3/2
O(n )
Our Version of the Cut-Matching Game
• MODIFIED GAME
1.No Stopping Condition
2.Value of Game is
!(HT )
T
Our Version of the Cut-Matching Game
• MODIFIED GAME
1.No Stopping Condition
2.Value of Game is
• STILL YIELDS
APPROX ALGORITHM
!(HT )
T
Approximation Ratio =
!(HT )
T
Our Version of the Cut-Matching Game
• MODIFIED GAME
1.No Stopping Condition
2.Value of Game is
• STILL YIELDS
APPROX ALGORITHM
!(HT )
T
Approximation Ratio =
!(HT )
T
• RESULTS
CUT STRATEGY:
MATCHING STRATEGY:
& 1 #
((HT ) '(log n)
!!
=
= '$$
2
T
O(log n)
% log n "
& 1 #
'(HT )
!
= O$
$ log n !
T
%
"
Cut Strategies: Finding Cuts Quickly
After t iterations, Ht = { M1, M2, …, Mt }.
= +1 charge
= –1 charge
Random assignment of charge
Cut Strategies: Finding Cuts Quickly
After t iterations, Ht = { M1, M2, …, Mt }.
= +1 charge
Random assignment of charge
= –1 charge
(x+y)/2
x
Mix the charges along
the matchings { M1, M2, …, Mt}
y
(x+y)/2
Cut Strategies: Finding Cuts Quickly
After t iterations, Ht = { M1, M2, …, Mt }.
= +1 charge
= –1 charge
Random assignment of charge
Cut Strategies: Finding Cuts Quickly
After t iterations, Ht = { M1, M2, …, Mt }.
= +1 charge
= –1 charge
Random assignment of charge
Cut Strategies: Finding Cuts Quickly
After t iterations, Ht = { M1, M2, …, Mt }.
= +1 charge
Random assignment of charge
= –1 charge
Mix the charges along
the matchings { M1, M2, …, Mt }
Cut Strategies: Finding Cuts Quickly
After t iterations, Ht = { M1, M2, …, Mt }.
= +1 charge
Random assignment of charge
= –1 charge
If cut is small,
unbalance remains.
Mix the charges along
the matchings { M1, M2, …, Mt }
Cut Strategies: Finding Cuts Quickly
Order the vertices according to the final charge present
and cut in half.
S
n/2
S
n/2
The KRV mixing walk
KRV-walk
.
At round t:
& I + Mt '1 #& I + Mt '2 # & I + M1 #
P( t ) = $
!$
!...$
!
% 2 "% 2 " % 2 "
Lazy random walk traversing matchings in order.
0
0
M1
M2
1
0
The KRV mixing walk
KRV-walk
.
At round t:
& I + Mt '1 #& I + Mt '2 # & I + M1 #
P( t ) = $
!$
!...$
!
% 2 "% 2 " % 2 "
Lazy random walk traversing matchings in order.
0
0
M1
Averaging
along M1
M2
1/2
1/2
The KRV mixing walk
KRV-walk
.
At round t:
& I + Mt '1 #& I + Mt '2 # & I + M1 #
P( t ) = $
!$
!...$
!
% 2 "% 2 " % 2 "
Lazy random walk traversing matchings in order.
1/4
1/4
M1
Averaging
along M2
M2
1/4
1/4
Sketch of KRV Analysis
1.
Mixing of P(t) measured by potential function
2
F
"t =|| P( t ) ! J / n ||
2.
If P(t) mixes well, Ht has good expansion.
Possible to embed Kn in Ht.
3.
Potential Reduction at every iteration
#t = #t "1 " L(Mt ) ! P( t )
Mixing due to
matching Mt
Decomposition possible as KRV walks matchings in order.
4.
Cut-finding procedure reduces potential by a fixed factor
&
1 #
!!
(t = (t '1$$1 '
% log n "
Yields expander in
O((log n)2) rounds
Why KRV cannot do better
Recall:
Approximation
is
!(HT )
T
Why KRV cannot do better
Recall:
Approximation
is
Can KRV get
better than O(1)
expansion?
Why KRV cannot do better
Recall:
Approximation
is
S
V-S
SUPPOSE:
Can KRV get
better than O(1)
expansion?
Walk on M1, M2, …, Mk mixes perfectly on S and V-S
and
no edge cross (S,V-S)
Why KRV cannot do better
Recall:
Mk+1
Approximation
is
Now walk mixes
perfectly.
Expansion is O(1).
S
V-S
SUPPOSE:
Can KRV get
better than O(1)
expansion?
Walk on M1, M2, …, Mk mixes perfectly on S and V-S
and
no edge cross (S,V-S)
Our Cut Strategy: a Different Walk
IDEA: use lazy natural random walk
I M1 + M2 + ... + Mt !1
P( t ) = +
2
2( t ! 1)
ADVANTAGES:
–
–
Eliminates bad case: possible to get better expansion.
Better handle on expansion through mixing by Cheeger’s Inequality.
CHALLENGE:
–
Impossible to decompose potential as in KRV.
#t = #t "1 " L(Mt ) ! P( t )
Additional matching modifies all steps of walk.
Our Cut Strategy: a Different Walk
IDEA: use lazy natural random walk
& I M1 + M2 + ... + Mt '1 #
!!
P( t ) = $$ +
2( t ' 1)
%2
"
d
ADVANTAGES:
–
–
Eliminates bad case: possible to get better expansion.
Better handle on expansion through mixing by Cheeger’s Inequality.
CHALLENGE:
–
Impossible to decompose potential as in KRV.
#t = #t "1 " L(Mt ) ! P( t )
Additional matching modifies all steps of walk.
Modified Walk and Matrix Inequalities
CHALLENGE:
Impossible to decompose potential as in
KRV.
Additional matching modifies all steps of walk.
SOLUTION:
Use round-robin walk close to natural walk:
Ni =
d
1
I+
Mi
d +1
d +1
P( t ) = (N1N2 ...Nt !1Nt !1Nt !2 ...N1 )
Apply matrix inequality:
t
( ABA ) ! A tB t A t
d
Modified Walk and Matrix Inequalities
CHALLENGE:
Impossible to decompose potential as in
KRV.
Additional matching modifies all steps of walk.
SOLUTION:
Use round-robin walk close to natural walk:
Ni =
d
1
I+
Mi
d +1
d +1
P( t ) = (N1N2 ...Nt !1Nt !1Nt !2 ...N1 )
Apply matrix inequality:
t
( ABA ) ! A tB t A t
#t = #t "1 " L(Mt ) ! P( t )
d
Modified Walk and Matrix Inequalities
CHALLENGE:
Impossible to decompose potential as in
KRV.
Additional matching modifies all steps of walk.
SOLUTION:
Use round-robin walk close to natural walk:
Ni =
d
1
I+
Mi
d +1
d +1
P( t ) = (N1N2 ...Nt !1Nt !1Nt !2 ...N1 )
Apply matrix inequality:
t
( ABA ) ! A tB t A t
#t = #t "1 " L(Mt ) ! P( t )
d
Modified Walk and Matrix Inequalities
SOLUTION:
Use round-robin walk close to natural.
Apply matrix inequality.
Yields same potential reduction as KRV.
But our walk is better related to expansion:
In O((log n)2) rounds,
conductance (1\log n) by Cheeger.
Modified Walk and Matrix Inequalities
SOLUTION:
Use round-robin walk close to natural.
Apply matrix inequality.
Yields same potential reduction as KRV.
But our walk is better related to expansion:
In O((log n)2) rounds,
conductance (1\log n) by Cheeger.
Ω(log n) expansion in O((log n)2) rounds.
Modified Walk and Matrix Inequalities
SOLUTION:
Use round-robin walk close to natural.
Apply matrix inequality.
Yields same potential reduction as KRV.
But our walk is better related to expansion:
In O((log n)2) rounds,
conductance (1\log n) by Cheeger.
Ω(log n) expansion in O((log n)2) rounds.
TIME: only polylog factors worse than KRV
Lower Bound
Matching player yielding
& 1 #
'(HT )
!
= O$
$ log n !
T
%
"
against any Cut player.
No better approximation than O((log n)1/2)
in KRV Cut-Matching game
Lower Bound Idea
A NAÏVE MATCHING PLAYER:
Fix a cut (S,V-S). Keep it as sparse as possible.
S
Lower Bound Idea
A NAÏVE MATCHING PLAYER:
Fix a cut (S,V-S). Keep it sparse.
Cut player
plays…
S
Lower Bound Idea
A NAÏVE MATCHING PLAYER:
Fix a cut (S,V-S). Keep it sparse.
Cut player
plays…
S
GAME OVER
Lower Bound Idea
A NAÏVE MATCHING PLAYER:
Fix a cut (S,V-S). Keep it sparse.
Cut player
plays…
S
GAME OVER
IDEA: hedge over many cuts
Lower Bound Idea
THE REAL PLAYER - AT START:
Matching player selects log(n) ‘orthogonal’ 50-50 cuts in V.
Orthogonal 50-50 cuts
Minimum correlation
Cut player cannot ‘kill’
many cuts at once
THE REAL PLAYER - THROUGHOUT THE GAME:
Matching player adds matchings to minimize average expansion.
Main Lemma
8 50-50 cut (S,V-S),
d
Hd = {! 1,+1}
Main Lemma
8 50-50 cut (S,V-S),
9 a perfect matching M, s.t.
d
Hd = {! 1,+1}
Main Lemma
8 50-50 cut (S,V-S),
9 a perfect matching M, s.t.
"| u ! v |
1
( u,v )#M
d
Hd = {! 1,+1}
( )
=O d
Conclusion and Open Problems
POWER OF CUT-MATCHING GAME:
Simple yet powerful framework for SPARSEST CUT.
OPEN QUESTION:
Can we use Cut-Matching to get fast (log n)1/2 approximation?