All-Pairs Shortest Paths with Matrix Multiplication
Chandler Burfield
March 15, 2013
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
1 / 19
Outline
1
Seidel’s algorithm
2
Faster Matrix Mulitplicaltion
Strassen’s Algorithm
Speeding up Seidel’s Algorithm
3
More Advanced APSP Algorithms that Use Matrix Multiplication
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
2 / 19
All-Pairs Shortest Paths
Problem
To find the shortest path
between all vertices v 2 V
for a graph G = (V , E ).
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
3 / 19
Definitions
A: the input to Seidel’s algorithm
Let A be the adjacency matrix, an n x n boolean matrix where a 1
represents an edge between node i and node j in the graph G.
D: the output of Seidel’s algorithm
Let D be the distance matrix, an n x n integer matrix with dij representing
the length of the shortest path from vertex i to vertex j in the graph G.
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
4 / 19
Seidel’s Algorithm
Seidel(A)
Let Z = A · A
Let B be an n x⇢n 0-1 matrix,
1 if i 6= j and (aij = 1 or zij > 0)
where bij =
0 otherwise
IF 8i, j, i 6= j bij = 1 then
Return D = 2B-A
Let T = Seidel(B)
Let X = T · A
Return n x n matrix
⇢ D,
2tij
if xij tij · degree(j)
where dij =
2tij 1 if xij < tij · degree(j)
Function
Seidel’s algorithm is an APSP algorithm that can be used on unweighted,
undirected graphs.
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
5 / 19
Analysis of Seidel’s Algorithm
Seidel(A)
Let Z = A · A
Let B be an n⇢x n 0-1 matrix,
1 if i 6= j and (aij = 1 or zij > 0)
where bij =
0 otherwise
IF 8i, j, i 6= j bij = 1 then
Return D = 2B-A
Let T = Seidel(B)
Let X = T · A
Return n x n matrix
D,
⇢
2tij
if xij tij · degree(j)
where dij =
2tij 1 if xij < tij · degree(j)
Chandler Burfield
APSP with Matrix Multiplication
Runtime
O(M(n)logn)
where M(n)
denotes the
time necessary
to multiply two
n x n integer
matricies
March 15, 2013
6 / 19
In the following, we explain how Algorithm 2 works using the exam
1 computes2the square graph.
3
Example for Seidel’s
Algorithm
At the
beginning, the algorithm
Following
resentation in
Memory
graph amounts
to following a path of length one or two in the origin
obtain the adjacency matrix of the square graph, first, the matrix A i
multiplication) and stored into
6 Z in line 3.
5 The graph4and the matrix
es to represent a graph G = (V, E) in memory. Let the se
with edges E 1.VAdjacency
V , and
let m := |E|.
matrix
1
2
3
1
The adjacency matrix of a graph G = (V, E) is a |V | |V | matrix
aij is equal to 1 if there exists an
= (vi , vj ) 42 E and 0
6 edge e 5
2 function w3: E R is given,
weight
then aij = w vi , vj .
The adjacency matrix of the graph depicted above is
6
5
4
Z = AA =
0
1
0
0
0
1
1
0
1
1
1
0
0
1
0
1
0
0
0
1
1
0
A
1
0
0
1
0
1
=
0
1
1
0
0
01
00
10
00
0 1
0
0
11
10
11
00
11
00
10
10
11
1 0
0
0
1
01
11
00
01
0
0
1
11
00
11
00
0 1
1
1 0
000
=
100
.
001
110
2
0
1
1
2
0
0
4
1
2
1
2
0 0 1 0
The value of zij indicates the number of distinct paths between i and
the matrix B (the adjacency matrix of the square graph) is calculated
Note
that, since the graph in the example is undirected, the res
in each entry if the nodes are either directly connected by an edge or
of a graph G = (V, E) is a |V | |V | matrix A, where each en
are and
no self 0loops,
each entry in In
the mai
re exists an edgemetric.
e =Also,
(vi ,since
vj )there
2 E
otherwise.
cas
2. Edge
R isChandler
given,
then
aijList
=
w vi , vMultiplication
j .
Burfield
APSP with Matrix
March 15, 2013
7 / 19
Example for Seidel’s Algorithm
Continued
6
5
4
Z = AA =
0
1
0
0
0
1
1
0
1
1
1
0
0
1
0
1
0
0
0
1
1
0
1
0
0
1
0
1
0
1
1
0
0
0
1
0
0
1
0
0
0
1
1
0
1
1
1
0
0
1
0
1
0
0
0
1
1
0
1
0
0
1
0
1
0
1
1
0
0
0
1
0
=
2
0
1
1
2
0
0
4
1
2
1
2
1
1
2
1
2
0
1
2
1
3
1
1
2
1
2
1
3
0
0
2
0
1
0
2
en these two nodes. For the example, B is:
The value of zij indicates the number of distinct paths between i and j of length 2. Secon
the matrix B (the
of the square graph) is calculated in line 4, having a
0 1 adjacency
1 1 1 matrix
1
in each entry if1 the
directly connected by an edge or if there is at least o
0 nodes
1 1 are
1 either
1
1 1 0 1 1 0
B=
1 1 1 0 1 1
1 1 1 1 0 1
8-6
1 1 0 1 1 0
the main diagonal, only b36 and b63 are 0. Note that there is indeed
two between nodes 3 and 6 of the graph.
s of B (except for the diagonal) are 1, then all pairs of nodes in the
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
8 / 19
1 1 1 1 0 1
1 1 the
0 resulting
1 1 0 value
and
will be 2. Otherwise, the value of A will be s
1 1 1 Continued
1 1 0
Example
for
Seidel’s
Algorithm
for
this
entry.
es of the main diagonal, only b36 and b63 are 0. Note that there is indeed
Otherwise,
the
algorithm
is called
matrixis,Bthe
in lin
all between
the entries
except
for the
mainrecursively
diagonal.onThat
m
2 are
eme,
or two
nodesof
3B
and
6 of1,
the
graph.
call,
the
algorithm
starts
with
the
square
graph,
computes
the
square
es a of
complete
graph
Kndiagonal)
:
ntries
B (except
for the
are 1, then all pairs of nodes in the
us get at
back
to 2.
ourItexample.
Wedistinguish
use subscript 2 for the nam
are connected by a etc.
pathLet
of length
most
remains to
the
recursive
call.
The
matrix
A
=
B
is
squared:
2
distance is 1 from pairs for which the distance is 2. B is multiplied by
1 has length22, then the 3entry in A 5is 04 3 4
d (line 6). This way, if the path
ue will be 2. Otherwise, the value of A will be subtracted, yielding
4 15 3 4
3 3 4 3
A29.
A2 In
= the
BB recursive
=
algorithm is called recursively 6
on matrix B5Zin2 =
line
4 4 3 5
4
tarts with the square graph, computes the square of the square graph,
4 4 3 4
to our example. We use subscript 2 for the names of the matrices
3 in3 4 3
he matrix A2 = B is squared:
8-7
Next, B2 is computed:
5 4 3 4 4 3
0 1 1 1 1 1
4 5 3 4 4 3
1 0 1 1 1 1
3 3 4 3 3 4
1 1 0 1 1 1
Z2 = A2 A2 = BB =
B2 =
4 4 3 5 4 3
1 1 1 0 1 1
4 4 3 4 5 3
1 1 1 1 0 1
3 3 4 3 3 4
1 1 1 1 1 0
d:
4
4
3
4
5
3
3
3
4
3
3
4
This time, all the entries of B2 are 1, except for the main diagonal.
describes
graph Kn :
0 1 1 a1 complete
1 1
Chandler Burfield
March 15, 2013
9 / 19
1 0 1 1 1 APSP
1 with Matrix Multiplication
d
tik  tij for all neighbors k of j, 1and1 0 1 1 2
Tsubtracted.
=
is multiplied
2 and A2 =one
B is
resulting
matrix
1 k 1of 1jThe
0 Continued
1 1
tik < tby
neighbor
ij for at least
Example
for
Seidel’s
Algorithm
all of the function and stored in T :
1 1 1 1 0 1
1 1 2 1 1 0
1 There
1 1 also
1 1exists a k such that d
dij = 2l, then tij 0= l.
2l 1.
kj
Note that, according
1 0 to
1 T1, the
1 1distance from node 3 to node 6 is 2, meaning that, since T
corresponds to the
1dkj1distance
0 1 1 1in the
2 square graph, the distance could be either 4 or also (like
l
T t=
kj
in this example)
1 23.1 It1remains
0 2 1 1to distinguish two cases: whether the actual distance from
node i to node j1 is12tij1 or1 2t0ij 1 1. In order to distinguish these two cases, the matrix T is
tij . The proof for an odd dij is analogous.
kj
multiplied
with 1A and
1 2stored
1 1 in0 X:
)Tbe
thedistance
set of neighbors
of 3node
i. 6 is 2, meaning that, since T
, the
from node
to node
nce in the square graph,
0 1 the
0 distance
0 0 1 could
0 be
1 either
1 1 41 or1 also (like
2 3 2 3 3 1
t
tij deg(i)
d is even
emains kjto distinguish
from
1 two
0 cases:
1 ij 1 1whether
0
1the0 actual
1 1 distance
1 1
1 4 1 2 2 2
k2 (i)
or 2tij 1. In order 0to distinguish
two
T 3is 3 2 2 4 2
1 0 1 0 these
0
1 1cases,
0 1the1 matrix
2
= tATdeg(i)
=
=
tX
<
tored in
kjX:
ij
0 1 1dij 0is odd
1 0
1 1 1 0 1 1
2 3 1 3 2 2
k2 (i)
0 1 0 1 0 1
1 1 1 1 0 1
2 3 2 2 3 1
1 0 0 0 1 0
1 1 2 1 1 0
1 5 2 4 2 2
1D 0for0this
0 example
1
0 is:
1 1 1 1 1
2 3 2 3 3 1
0Finally,
1 1 in1 line
0 11 the
1 0 1 1 1matrix
1 D is1returned,
4 1 2 containing,
2 2
for each of its entries, either
0 1 2 resulting
2 2 1
1the0original
1 0 0value 1or 1the0decrement
1 1 2 of it. 3The
3 following
2 2 4 four
2 propositions shall explain this
1 0 1 1 1 2
=
1formula.
1 0 1 0
1 1 1 0 1 1
2 3 1 3 2 2
2 1 0 1 2 3
1 0 1D =
0 1
1 1 1 1 0 1
2 3 2 2 3 1
2 1 1 0 1 2
0Proposition
0 0 1 0 8.1.1 If1the2 distance
1 1 0 dij is 1even,
5 2the4 resulting
2 2
distance is 2tij and 2tij
1
2 1 2 1 0 1
otherwise. 1 2 3 2 1 0
ulting matrix D is returned, containing, for each of its entries, either
The i.e.
following
propositions
shall
explain
Chandler
Burfield
with
Multiplication
15, 2013
10 / 19
Proof:
dijof
isit.even,
dij =isfour
2s
and
the
shortest
path
is call
v0this
, v1 , . . . , vMarch
the path
ddecrement
graph,Ifthe
maximum
distance
at APSP
most
n. Matrix
In
each
recursive
2s then
Correctness of Seidel’s Algorithm
Lemma 1:
If the distance dij is even then the resulting distance is 2tij and 2tij -1
otherwise.
Lemma 2:
Let i and j be a pair of distinct vertices in G. For any neighbor k of j, we
have dij -1  dik  dij + 1. More over, there exists a neighbor k of j with
dik = dij + 1
Lemma 3:
If dij is even then tik tij for all neighbors of k and j
If dij is odd then tik  tij for all neighbors of k of j,
tik < tij for at least one neighbor k of j
Lemma 4:
Let (j) be the set of neighbors
of node j, then
P
dij is even i↵P k2 (j) tik tij · degree(j)
dij is odd i↵ k2 (j) tik < tij · degree(j)
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
11 / 19
Square Matrix Multiplication
Naively square matrix multiplication takes O(n3 ) time.
Strassen’s algorithm can preform it in O(n2.807 )
The fastest algorithm works in O(n2.373 )
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
12 / 19
Naive Matrix Multiplication Algorithm
!
This can be done in O(n3 ) time.
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
13 / 19
Naive Matrix Multiplication for 2x2 Matricies
!
Analysis of Algorithm
Recurrence Relation: T (n) = 8T (n/2) + O(n2 )
Runtime: O(nlog 8/log 2 ) = O(n3 )
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
14 / 19
Strassen’s Algorithm for 2x2 Matrices
2 x 2 Matrices:
!
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
15 / 19
Strassen’s Algorithm
Strassen for n x n Matrices
We can think of an n x n matrix as a 2 x 2 matrix whose elements are
n
2 matrices and apply the 2x2 algorithm recursively.
n
2
x
Analysis of Algorithm
Recurrence Relation:
T (n) = 7T (n/2) + O(n2 )
Runtime:
O(nlog 7/log 2 ) = O(n2.807 )
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
16 / 19
Fast Matrix Multiplication and Seidel’s Algorithm
Recall that Seidel’s Algorithm runs in O(M(n)logn).
Using Strassen’s algorithm instead of the naive algorithm, we
decrease the runtime from O(n3 logn) to O(n2.81 logn)).
If we used the fastest algorithm available to date, we could achieve a
runtime of O(n2.373 logn).
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
17 / 19
Other APSP Algorithms Involving Matrix Multiplication
Consider integer edge weights on the set {0,1,.., M}.
APSP in weighted, undirected graphs: Shoshan-Zwick ’99, O(Mn2.38 )
APSP in weighted, directed graphs: Zwick ’98, O(M 0.68 n2.58 )
Chandler Burfield
APSP with Matrix Multiplication
March 15, 2013
18 / 19
APSP Algorithms with Integer Weights {0, 1, 2,..., M}
that Use Matrix Multiplication
Graph
Unweighted and Undirected
Weighted and Undirected
Weighted and Directed
Chandler Burfield
Algorithm
Seidel ’95
Shoshan-Zwick ’99
Zwick ’98
APSP with Matrix Multiplication
Runtime
n2.38
Mn2.38
M 0.68 n2.58
March 15, 2013
19 / 19