The PageRank of a graph
Fan Chung
University of California, San Diego
What is PageRank?
•Ranking the vertices of a graph
•Partially ordered sets + graph theory
A new graph invariant for dealing with:
-Applications --web search,ITdata sets,
xxxxxxxxxxxxxpartitioning algorithms,…
-Theory --- correlations among vertices
fan
Outline of the talk
• Motivation
• Define PageRank
• Local cuts and the Cheeger constant
• Four versions of the Cheeger inequality
using:  eigenvectors
 random walks
 PageRank
 heat kernel
• Four partitioning algorithms
• Green’s functions and hitting time
What is PageRank?
What is Rank?
What is PageRank?
PageRank is defined on
any graph.
An induced subgraph of the collaboration
graph with authors of Erdös number ≤ 2.
A subgraph of the Hollywood graph.
A subgraph of a BGP graph
The Octopus graph
Yahoo IM graph
Reid Andersen 2005
Graph Theory has 250 years of history.
Leonhard Euler
1707-1783
The Bridges of Königsburg
Is it possible to walk over every
bridge once and only once?
Geometric graphs
Topological graphs
Algebraic graphs
General graphs
Massive data
Massive graphs
• WWW-graphs
• Call graphs
• Acquaintance graphs
• Graphs from any data
a.base
protein interaction network
Jawoong Jeong
Big and bigger graphs
New directions
in graph theory
Many basic questions:
• Correlation among vertices?
• The `geometry’ of a network ?
distance, flow, cut, …
• Quantitative analysis?
eigenvalues, rapid mixing, …
• Local versus global?
Google’s answer:
The definition for PageRank?
A measure for the “importance” of a website
x1  R( x14  x79  x785 )
x2  R( x1002  x3225  x9883  x30027 )
  
The “importance” of a website is proportional
to the sum of the importance of all the sites
that link to it.
A solution for the “importance” of a website
x1   ( x14  x79  x785 )
x2   ( x1002  x3225  x9883  x30027 )
  
Solve
n
xi    aij x j
j 1
for
x  ( x1 , x2 , , xn )
A solution for the “importance” of a website
x1   ( x14  x79  x785 )
x2   ( x1002  x3225  x9883  x30027 )
  
Solve
n
xi    aij x j
for
x  ( x1 , x2 , , xn )
j 1
x=ρAx
A  aij 
nn
Adjacency matrix
A solution for the “importance” of a website
x1   ( x14  x79  x785 )
x2   ( x1002  x3225  x9883  x30027 )
  
Solve
n
xi    aij x j
j 1
x=ρAx
for
x  ( x1 , x2 , , xn )
Graph models
(undirected) graphs
directed graphs
weighted graphs
Graph models
(undirected) graphs
directed graphs
weighted graphs
Graph models
(undirected) graphs
directed graphs
weighted graphs
2.3
1.2
1.5
1.1
1
2
3.3 2.8
\
1.5
In a directed graph,
there are two types of “importance”:
authority
hub
Jon Kleinberg 1998
Two types of the “importance” of a website
Importance as Authorities :
Importance as Hubs :
Solve
and
x=rAy
T
x = rs A A x
T
y = rs A A y
x  ( x1 , x2 , , xn )
y  ( y1 , y2 , , ym )
T
y=sAx
Eigenvalue problem for n x n matrix:.
n ≈ 30 billion websites
Hard to compute eigenvalues
Even harder to compute eigenvectors
In the old days,
compute for a given (whole) graph.
In reality,
can only afford to compute “locally”.
A traditional algorithm
Input: a given graph on n vertices.
Efficient algorithm means polynomial algorithms
n3, n2, n log n, n
New algorithmic paradigm
Input: access to a (huge) graph
(e.g., for a vertex v, find its neighbors)
Bounded number of access.
A traditional algorithm
Input: a given graph on n vertices.
Efficient algorithm means polynomial algorithms
n3, n2, n log n, n
New algorithmic paradigm
Input: access to a (huge) graph
(e.g., for a vertex v, find its neighbors)
Bounded number of access.
The definition of PageRank given by
Brin and Page is based on
random walks.
Random walks in a graph.
G : a graph
P : transition probability matrix
1
 if u v, du := the degree of u.
P(u, v)   du
0otherwise.

A lazy walk:
IP
W
2
Original definition of PageRank
A (bored) surfer
• either surf a random webpage
with probability α
• or surf a linked webpage
with probability 1- α
α : the jumping constant
p   ( 1n , 1n ,...., 1n )  (1   ) pW
Definition of personalized PageRank
Two equivalent ways to define PageRank pr(α,s)
(1)
p   s  (1   ) pW
s:
the seed as a row vector
α : the jumping constant
s
Definition of PageRank
Two equivalent ways to define PageRank p=pr(α,s)
(1)
(2)
p   s  (1   ) pW

p    (1   )t ( sW t )
t 0
1 1
1
(
,
,....,
s= n n
n)
the (original) PageRank
s = some “seed”, e.g., (1, 0,...., 0)
personalized PageRank
How good is PageRank as a measure
of correlationship?
Depends on the applications?
How “good” is the cut?
Isoperimetric properties
Isoperimetric properties
“What is the shortest curve
enclosing a unit area?”
In a graph G and an integer m,
what is the minimum cut disconnecting
a subgraph of ≥ m vertices?
In a graph G, what is the minimum cut e(S,V-S)
so that e(S,V-S)
_____
Vol S
is the smallest?
How “good” is the cut?
Two types of cuts:
• Vertex
S
E(S,V-S)
cut
• edge cut
e(S,V-S)
_____
Vol S
e(S,V-S)
_____
|S|
Vol S = Σ deg(v)
vεS
|S| = Σ 1
vεS
V-S
S
The Cheeger constant for graphs
The Cheeger constant
e( S , S )
hGhG  min
S
min(vol S , vol S )
The volume of S is
vol ( S )   d x
xS
hGG and its variations are sometimes called
“conductance”, “isoperimetric number”, …
The Cheeger inequality
The Cheeger constant
e( S , S )
hG  min h( S )  min
S
S
min(vol S , vol S )
The Cheeger inequality
hG 2
2hG   
2
 : the first nontrivial eigenvalue of the
xx(normalized) Laplacian of a connected graph.
The spectrum of a graph
•Adjacency matrix
Many ways to define the spectrum of a graph.
How are the eigenvalues related to
properties of graphs?
The spectrum of a graph
•Adjacency matrix
•Combinatorial Laplacian
L  D A
adjacency matrix
diagonal degree matrix
Gustav Robert Kirchhoff
1824-1887
The spectrum of a graph
•Adjacency matrix
•Combinatorial Laplacian
adjacency matrix
L  D A
diagonal degree matrix
Matrix tree theorem
#spanning
. trees
  i
i 0
Gustav Robert Kirchhoff
1824-1887
The spectrum of a graph
•Adjacency matrix
•Combinatorial Laplacian
L  D A
adjacency matrix
diagonal degree matrix
•Normalized Laplacian
Random walks
Rate of convergence
Gustav Robert Kirchhoff
1824-1887
The spectrum of a graph
loopless, simple
Discrete Laplace operator
1
f ( x) 
dx
L ( x, y ) 
{
not symmetric in general
 ( f ( x)  f ( y))
y x
1 if x  y
1

if x  y and x
dx
•Normalized Laplacian
symmetric
normalized
L( x, y ) 
with eigenvalues
{
y
1 if x  y
1

if x  y and x
dxd y
0  0  1    n1  2
y
The spectrum of a graph
Discrete Laplace operator
1
f ( x) 
dx
L ( x, y ) 
{
not symmetric in general
 ( f ( x)  f ( y))
y x
1 if x  y
1

if x  y and x
dx
•Normalized Laplacian
symmetric
normalized
L( x, y ) 
with eigenvalues
{
y
1 if x  y
1

if x  y and x
dxd y
0  0  1    n1  2

y

dictates many properties
of a graph.
• expander
• diameter
• discrepancy
• subgraph containment
• ….
Spectral implications for
finding good cuts?
Finding a cut by a sweep
For f : V (G )  R,
order the vertices
f (vn )
f (v1 ) f (v2 )

  
.
d v1
d v2
d vn
Consider sets
S j  {v1 , , v j }, j  1,..., n,
f
and the Cheeger constant of S j .
f
Define
h( f )  min h( S j )
f
j
Finding a cut by a sweep
Using a sweep by the eigenvector,
can reduce the exponential number of
choices of subsets to a linear number.
Finding a cut by a sweep
Using a sweep by the eigenvector,
can reduce the exponential number of
choices of subsets to a linear number.
Still, there is a lower bound guarantee
by using the Cheeger inequality.
h2
2h   
2
Four types of Cheeger inequalities.
Four proofs using:
 eigenvectors
 random walks
 PageRank
 heat kernel
Leading to four different
one-sweep partitioning algorithms.
Four proofs of Cheeger inequalities
• graph spectral method
spectral partition algorithm
• random walks
• PageRank
• heat kernel
local partition algorithms
Graph partitioning
Local graph partitioning
What is a local graph partitioning algorithm?
A local graph partitioning algorithm finds a small
cut near the given seed(s) with running time
depending only on the size of the output.
Examples of local partitioning
Examples of local partitioning
Examples of local partitioning
Examples of local partitioning
Examples of local partitioning
h
h
h
Four proofs of Cheeger inequalities
• graph spectral method
spectral partition algorithm
• random walks
• PageRank
• heat kernel
local partition algorithms
Four proofs of Cheeger inequalities
• graph spectral method Cheeger 60’s, Fiedler ’73
• random walks
Alon 86’, Jerrum+Sinclair 89’
Lovasz, Simonovits, 90, 93
Spielman, Teng, 04
• PageRank
Andersen, Chung, Lang, 06
• heat kernel
Chung, PNAS , 08.
The Cheeger inequality
Partition algorithm
Using eigenvector f ,
the Cheeger inequality can be stated as
2
h2
2h   

2
2
where  is the first non-trivial eigenvalue
of the Laplacian and  is the minimum
Cheeger ratio in a sweep using the
eigenvector f .
Proof of the Cheeger inequality:
from definition
by Cauchy-Schwarz ineq.
from the definition.
summation by parts.
A Cheeger inequality using random walks
Lovász, Simonovits, 90, 93
vol ( S )   
W (u , S )   ( S ) 
1 

du 
8 
2
k
k
k
Leads to a Cheeger inequality:
G 2
h2
2h   

8
8
where G is the minimum Cheeger ratio over
sweeps by using a lazy walk of k steps from
every vertex for an appropriate range of k .
A Cheeger inequality using PageRank
Using the PageRank vector.
Recall the definition of PageRank p=pr(α,s):
(1)
(2)
p   s  (1   ) pW

p    (1   ) ( sW )
t
t
t 0
Organize the random walks by a scalar α.
Random walks
versus
How fast is the
convergence to the
stationary distribution?
For what k, can one have
f Wk 
?
PageRank
Choose α to satisfy
the required
property.
A Cheeger inequality using PageRank
Using the PageRank vector with seed as a subset S
and vol ( S )  vol (G ) / 4, a Cheeger inequality
can be obtained :
 S2
hS 2
hS  S 

8log s
8log s
where S is the Dirichlet eigenvalue of the
Laplacian, and  S is the minimum Cheeger
ratio over sweeps by using the appropriate
personalized PageRank with seeds S.
Dirichlet eigenvalues for a subset S  V
S  inf
2
(
f
(
u
)

f
(
v
))

u v
f

f ( w) 2 d w
w
V
S
over all f satisfying the Dirichlet boundary
condition:
f (v )  0
for all v  S.
Local Cheeger constant for a subset S  V
V
hS  min h(T )
T S
S
A Cheeger inequality using PageRank
Using the PageRank vector with seed as a subset S
and vol ( S )  vol (G ) / 4, a Cheeger inequality
can be obtained :
 S2
hS 2
hS  S 

8log s
8log s
where S is the Dirichlet eigenvalue of the
Laplacian, and  S is the minimum Cheeger
ratio over sweeps by using personalized
PageRank with seed S.
Algorithmic aspects of PageRank
• Fast approximation algorithm for
x
personalized PageRank
greedy type algorithm, almost linear
complexity
• Can use the jumping constant to approximate
PageRank with a support of the desired size.
• Errors can be effectively bounded.
A graph partition algorithm using PageRank
Given a set S with
1
hS   , vol ( S )  vol (G ),
2
randomly choose a vertex v in S.
1
With probability at least ,
2
the one-sweep algorithm using
f  pr ( , v)
has an initial segment with the Cheeger
constant at most
h f  O(  log | S |).
Graph partitioning using PageRank vector.
198,430 nodes and 1,133,512 edges
Kevin Lang 2007
Four proofs of Cheeger inequalities
• graph spectral method Fiedler ’73, Cheeger, 60’s
• random walks
Alon 86’
Lovasz, Simonovits, 90, 93
Spielman, Teng, 04
• PageRank
Andersen, Chung, Lang, 06
• heat kernel
Chung, PNAS , 08.
PageRank

versus
p , s    (1   ) k ( sW k )
k 0
Geometric sum
heat kernel

k
(
tW
)
t ,s  et  s
k!
k 0
Exponential sum
PageRank

versus
p , s    (1   ) k ( sW k )
k 0
Geometric sum
heat kernel

k
(
tW
)
t ,s  et  s
k!
k 0
Exponential sum
p    (1   ) pW

  (I  W )
t
recurrence
Heat equation
A Cheeger inequality using the heat kernel
Theorem:
vol (S ) tt2,u / 4
t , u ( S )   ( S ) 
e
du
where  t ,u is the minimum Cheeger ratio over
sweeps by using heat kernel pagerank over all
u in S.
Theorem: For vol (S )  vol (G)2/ 3 ,
etS
t , S ( S )   ( S ) 
.
2
Definition of heat kernel
2
k
t
t
H t  et ( I  tW  W 2  ...  W k  ...)
2
k!
 t ( I W )
e
e
 tL
k
t2 2
t
 I  tL  L  ...  (1)k Lk  ...
2
k!

H t  ( I  W ) H t
t
t , s  sH t
A Cheeger inequality using the heat kernel
Using the upper and lower bounds,
a Cheeger inequality can be obtained :
S 2
hS 2
hS  S 

8
8
where S is the Dirichlet eigenvalue of the
Laplacian, and  S is the minimum Cheeger
ratio over sweeps by using heat kernel with
seeds S for appropriate t.
Many applications of PageRank for
problems in Graph Theory
• Graph drawing using PageRank
• Graph embedding using PageRank
• Pebbing and routing using PageRank
• Covering and packing using PageRank
• Relating graph invariants of subgraphs
to the host graph using PageRank
• Your favorite old problem using PageRank?
New Directions in Graph Theory
for information networks
Topics:
• Random graphs with general degrees
• pageranks
• Algorithmic game theory, graphical
games
Using:
• Spectral methods
• Probabilistic methods
• Quasirandom …