Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Random Walks:
Basic Concepts and Applications
Laura Ricci
Dipartimento di Informatica
25 luglio 2012
PhD in Computer Science
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Outline
1
Basic Concepts
2
Natural Random Walk
3
Random Walks Characterization
4
Metropolis Hastings
5
Applications
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Random Walk: Basic Concepts
A Random Walk in synthesis:
given an indirected graph and a starting point, select a
neighbour at random
move to the selected neighbour and repeat the same
process till a termination condition is verified
the random sequence of points selected in this way is a
random walk of the graph
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Random Walk
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Random Walk
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Random Walk
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Random Walk
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Random Walk
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Random Walk
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Random Walk
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Random Walk
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
The Natural Random Walk
Natural Random Walk
Given an undirected graph G = (V, E), with n =| V | and
m =| E |, a natural random walk is a stochastic process that
starts from a given vertex, and then selects one of its neighbors
uniformly at random to visit.
The natural random walk is defined by the following transition
matrix P :
(
1
, y is a neighbour of x
P (x, y) = degree(x)
0,
otherwise
where x is the out degree of the node x
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Natural Random Walk
note that we assume undirected graph: i.e. if the walker
can go from i to j, it can also go from j to i
this does not imply that the probability of the transition ij
is the same of the transition ji
it depends on the degree distribution of the nodes
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Natural Random Walk: Stationary Distribution
Stationary Distribution
Given an irreducible and aperiodic graph with a set of nodes N
and a set E of edges, the probability of being at a particular
node v converges to the stationary distribution
π RW (v) =
deg(v)
2×|E|
= ∝ degree(v)
if we run the random walk for sufficiently long, then we get
arbitrarily close to achieving the distribution π RW (v)
put in a different way: the fraction of time spent at a node
is directly proportional to the degree of node....
the probability of sampling a node depends on its degree
the natural random walk is inherently biased towards node
with higher degree
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Natural Random Walk Stationary Distribution
Consider a random walk on the house graph below:
d1 , d2 , d3 , d4 , d5 ) = (2, 3, 3, 2, 2)
so stationary distribution is
2 3 3 2 2
π1 , π2 , π3 , π4 , π5 = ( 12
, 12 , 12 , 12 , 12 )
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Natural Random Walk Stationary Distribution
A king moves at random on an 8 × 8 chessboard. The
number of moves in various locations are as follows:
interior tiles: 8 moves
edge tiles: 5 moves
corner tiles: 3 moves
The number of all possible edges(moves on the chessboard)
is 420
8
3
5
Therefore, the stationary distribution is ( 420
, 420
, 420
)
This gives an idea of the time spent by the king on each
kind of tile during the random walk
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Random Walks and Markov Chains
Time Reversible Markov Chain: the probability of the
occurrence of a state sequence occurring is equal to the
probability of the reversal of that state sequence
running time backward does not affect the distribution at all
Consider a Markov Chain with state space {s1 , . . . , sk } and
probability transition matrix P . A stationary distribution
π on S is said to be time reversible for the chain if
∀i, j ∈ {1, . . . k}, we have:
π(i) × P (i, j) = π(j) × P (j, i)
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Time Reversible Markov Chain
Time Reversibility interpretation
think of π(i) × P (i, j) as the limiting long run fraction of
transitions made by the Markov chain that go from state i
to state j.
time reversibility requires that the long run fraction of i to
j transitions is the same as that of the j to i transitions,
∀i, j.
note that this is a more stringent requirement than
stationarity, which equates the long run fraction of
transitions that go out of state i to the long run fraction of
transitions that go into state i.
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Natural Random Walk and Markov Chains
the natural random walk on a graph is a time reversible
Markov chain with respect to its stationary distribution
as a matter of fact, for all vi and vj neighbours,
π(i) × P (i, j) =
di
1
1
×
=
d
di
d
π(j) × P (j, i) =
dj
1
1
×
=
d
dj
d
otherwise, if vi and vj are not neighbours
π(i) × P (i, j) = π(j) × P (j, i)
(1)
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Random Walks and Markov Chains
not all walks on a graph are reversible Markov Chains
consider the following walker: at each time step, the walker
moves one step clockwise with probability 43 and one step
counter clockwise with probability 14
π = ( 41 , 14 , 14 , 41 ) is the only stationary distribution
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Random Walks and Markov Chains
The transition graph is the following one:
is it sufficient to show that the stationary distribution
π = ( 41 , 14 , 14 , 41 ) is not reversible, to conclude that the chain
is not reversible
π(1) × P (1, 2) =
1 3
3
× =
4 4
16
π(2) × P (2, 1) =
1 1
1
× =
4 4
16
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Random Walk Metrics
Important measures of Random Walk
Access or Hitting Time, Hij : expected number of steps
before node j is visited, starting from node i.
Commute Time: expected number of steps in the random
walk starting at i, before node j is visited and then node i
is reached again.
Cover time expected number of steps to reach every node,
starting from a given initial distribution.
Graph Cover Time Maximum Cover Times over all Vertexes
Mixing Rate measures how fast the random walk converges
to the Stationary Distribution (Steady State).
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Computing Random Walk Metrics: A Warm Up
Example
the values of the metrics depends on
the graph topology
the probability fluxes on the graph
let us consider the (simple and not realistic) case of
complete graph with nodes {0, . . . , n − 1}
each pair of vertices is connected by an edge
consider a natural random walk on this graph and compute
the access time for a pair of nodes
the cover time of the random walk
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Warm Up Example: Hitting Time
each node has the same number of connections to other
nodes
so we can consider a generic pair of nodes, for instance
node 0 and node 1 and compute H(0, 1), without loss of
generality.
the probability that, staring from the node 0, we reach
node 1 in the t-h step is
t−1
n−2
1
× n−1
n−1
so the expected hitting time is:
t−1
P
n−2
H(0, 1) = ∞
t
×
×
t=1
n−1
1
n−1
=n−1
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Warm Up Example: Cover Time
The problem of the coverage of a complete graph is closely
related to the Coupon Collector Problem
since you are eager of cereals you often buy them
each box of cereal contains one of n different coupons
each coupon is chosen independently and uniformly at
random from n ones
you cannot collaborate with other people to collect the
coupons!
when you have collected one of every type of coupon, you
win a prize!
under these conditions, what is the expected number of box
of cereals you have to buy before you win the prize?
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Warm Up Example: Cover Time
Coupon collection problem is modelled through a geometric
distribution
Geometric Random Distribution:
a sequence of independent trials repeated until the first
success
each trial succeeds with probability p
P(X = n) = (1 − p)n−1 p
E[X] =
1
p
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Warm Example: Cover Time
Modelling the coverage problem as a coupon collection problem:
the coupons are the vertexes of the graph
collecting a coupon corresponds to visiting a new node
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Warm Up Example: Cover Time
Cover time may be modelled by a sequence of Geometric
Variables
Let us define a vertex as collected when it has been visited
at least a time by the random walk
X number of vertexes visited by the random walk before
collecting all the vertexes in the graph
Xi number of vertexes visited after having collected i − 1
vertexes and before collecting a new vertex.
P
X = ni=1 Xi
we are interested in computing the expected value of
X = E[X]
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Warm Up Example: Cover Time
Xi is a geometric random variable with success probability
pi = 1 −
E[Xi ] =
i−1
n
n
1
=
pi
n−i+1
by exploiting the linearity of expectations
Pn
Pn
Pn
Pn
n
E[X] = E[ i=1 Xi ] = i=1 E[Xi ] = i=1 n−i+1
= n i=1
Pn
the summation i 1i is the harmonic number ≈ logn
we can conclude that the cover time (time to collect all the
vertexes of the graph) is ≈ nlogn
1
i
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
A Warm Up Example: Cover Time
Note that
if we want to collect all the nodes through a single random
walk, we need obviously at least n steps
the factor log(n) introduces a reasonable delay due to the
randomness of the approach
this result has been obtained for a completely connected
(regular) graph
the cover time in general depends from the topology of the
graph and from the probability fluxes on it
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
The Lollipop Graph
in the ”lollipop graph”, the asymmetry in the number of
neighbours implies very different hitting time Huv and Hvu
every random walk starting at u has no option but to go in
the direction of u, Hvu = Θ(n2 )
a random walk starting at v has very little probability of
proceeding along the straight line, Huv = Θ(n3 )
the cover time of the graph is high (Θ(n3 )) due to a similar
reason.
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
The Metropolis Hasting Method
Markov Chains (and Random Walks) are a very useful and
general tool for simulations
suppose we want to simulate a random draw from some
distribution π on a finite set S.
for instance: generate a set of numbers chosen according to
a power law distribution, or to a gaussian distribution,. . .
we can exploit the basic theorem of Markov Chains
find an irreducible, aperiodic probability transition matrix
P such that its stationary distribution is π (πP = π)
run the corresponding Markov chain for a sufficiently long
time
now the problem is to find the matrix P such that the
stationary distribution is π
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
The Metropolis Hasting Method
start from a given connected undirected graph defined on a
set S of states. For instance:
a P2P network overlay
the social relationship graph
define the probability transition matrix P such that the
random walk on this graph converges to the stationary
distribution π
the definition of P depends on:
the graph topology
the distribution probability π we want to obtain
different graph topologies lead to different matrices P with
stationary distribution π.
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
The Metropolis Hastings Method
to obtain a stationary distribution where the probability to
visit each node is proportional to its degree, simply run a
natural random walk
in general, however, we want to obtain a different,
arbitrary distribution π, such that:
π(i) ∝ degree(i) × f (i)
or
π(i) ∝ πRW × f (i)
Our Goal
find a simple way to modify the random walk transition
probabilities so that the modified probability transition matrix
has stationary distribution π.
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
The Metropolis Hastings Method
To obtain the distribution π, such that:
π(i) ∝ degree(i) × f (i)
(2)
the Metropolis Hasting Methods defines the following transition
matrix
 1
× min{1, ff (j)

 degree(i)
(i) } if j ∈ N (i)
P
P (i, j) = 1 − k∈N (i) P (i, k)
if i = j


0
otherwise
(3)
Theorem
Let π the distribution defined by (2)and P (i, j) defined by (3),
then πP = π, i.e. π is a stationary distribution.
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
The Metropolis Hastings Method
A modified version of natural random walk
To run the natural random walk, at each time, choose a
random neighbor and go there.
To run Metropolis Hastings, suppose to be at node i,
generate next random transition as follows:
start out in the same way as natural random walk by
choosing a random neighbour j of i. j is a ”candidate”
state
then make the next probabilistic decision: ”accept the
candidate” and move to j, or ”reject it” and stay at i
the probability to accept the candidate is given by the extra
factor
min{1, ff (j)
(i) }
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
The Metropolis Hasting Method
Let us consider the ”correction factor”
min{1, ff (j)
(i) }
if f (j) ≥ f (i), the minimum is 1, the chain definitely moves
from i to j.
if f (j) < f (i), the minimum is
with probability
f (j)
f (i) ,
the chain moves to j
f (j)
f (i) .
if f (j) is much smaller than f (i), the desired distribution π
places much less probability on j than on i
the chain should make a transition from i to j much less
frequently than the random walk does
this is accomplished in the Metropolis Hastings chain by
usually rejecting the candidate j.
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Metropolis Hastings: Adjusting for Degree Bias
avoid the bias toward higher degree nodes
build an uniform distribution even for non regular graphs
define the transition matrix as follows:
 1
degree(i)
× min{1, degree(j)
} if j ∈ N (i)

 degree(i)
P
P (i, j) = 1 − k∈N (i) P (i, k)
if i = j


0
otherwise
the bias toward higher degree nodes is removed by reducing
the probability of transitioning to higher degree nodes at
each step
instantiate the general formula by
1
1
f (j) =
, f (i) =
degree(j)
degree(i)
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Metropolis Hastings: Adjusting for Degree Bias
The algorithm for selecting the next step from the node x
Select a neighbour y of x uniformly at random
Query y for a list of its neighbours, to determine its degree
Generate a random number p, uniformly at random
between 0 and 1
If p ≤
degree(x)
degree(y) ,
y is the next step.
Otherwise, remain at x as the next step
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Metropolis Hasting: An Example
Consider again the house graph:
on the left: Random Walk Stationary Distribution
on the right: the target distribution
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Metropolis Hastings: An Example
Consider again the house graph:
to define ∀i f (i), exploit the relation
πT ARGET (i) = f (i)πRW (i)
πT ARGET (i) = f (i)
for instance f (1) =
similarly, f (2) =
2
3
4
12
2
12
d(j)
2M
=2
= f (3) and f (4) = 1 = f (5).
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Metropolis Hastings: An Example
For instance, the transition probabilities are computed as
follows:
2
1
1
× min{1, 3 } =
2
2
6
1 1
2
P (1, 1) = 1 − − =
6 6
3
P (1, 2) =
and so on . . .
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Random Walks: Applications
Random Walk exploited to model different scenarios in
mathematics and physics
brownian motion of dust particle
statistical mechanics
Random Walks in Computer Science
epidemic diffusion of the information
generate random samples from a large set (for instance a set
of nodes from a complex networks)
computation of aggregate functions on complex sets . . .
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Random Walks in Computer Science
The graph
nodes: nodes of a peer to peer network, vertex of a social
graph
edges: P2P overlay connections, social relation in a social
network
since a random walk can be viewed as a Markov chain, it is
completely characterized by its stochastic matrix
we are interested in the probabilities to be assigned to the
edge of the graph in order to obtain a given probability
distribution
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Random Walks: Sampling a Complex Network
measuring the characteristics of complex networks:
P2P networks
On line Social Networks
World Wide Web
the complete dataset is not available due to
the huge size of the network
the privacy concern
sampling techniques are essential for practical estimation of
studying properties of the network
study the properties of the network based on a small but
representative sample.
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Random Walks: Sampling a Complex Network
Parameters which may be estimated by a random walk
topological characteristics
node degree distribution
clustering coefficient
network size
network diameter
node characteristics
link bandwidth
number of shared files
number of friends in a social network
A large amount of proposals in the last years.
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Random Walks: Sampling A Complex Network
Random Walk convergence detection mechanisms are required
to obtain useful sample:
a valid sample of a network may be derived from a random
walk only when the distribution is stationary
this is true only asymptotically
how many of the initial samples in each walk have to be
discarded to lose dependence from the starting
point(burn-in problem) ?
a further problem:
how many samples to collect before we having collected a
representative sample?
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Random Walks: Convergence Detection
A naive approach:
run the sampling process long enough and discard a number
of initial burn in samples pro actively
pro: simplicity
cons: from a practical point of view the length of the burn
in phase should be minimized because to reduce bandwidth
consumption and computational time.
A more refined approach: estimate convergence from as a
set of statistical properties of the walks as they are
collected
several techniques from the Markov chain diagnostics
literature
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Geweke’s Diagnostics
basic idea: take two non overlapping parts of the Random
Walk
compare the means of both parts, using a difference of
means test
if the two parts are separated by many iterations, the
correlation between them is low
see if the two parts of the chain belong to the same
distribution
the two parts should be identically distributed when the
walk converges
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Geweke’s Diagnostics
Let X be a single sequence of samples of the metric of
interest, obtained by a random walk. Consider:
a prefix Xa of X (usually 10% of X)
a longer suffix Xb of X (usually 50% of X)
compute the z statistics
E(Xa ) − E(Xb )
z=p
V ar(Xa ) + V ar(Xb )
(4)
if | z |> T where T is a threshold value (usually 1.96),
iterates from the prefix segment were not yet drawn from
the target distribution and should be discarded
otherwise declare convergence
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Convergence: Using Multiple Parallel Random Walks
a single walk may get trapped in a cluster while exploring
the network
the chain may stay long in some non-representative region
this may lead to erroneous diagnosis of convergence
to improve convergence, use multiple parallel random walks
Gelman Rubin Diagnostics:
uses parallel chains and discards initial values of each chain
check if all the chains converge to the approximatively the
same target distribution.
the test outputs a single value R that is a function of means
and variances of all chains
failure indicates the need to run a longer chain: burn-in yet
to be completed
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Applications: Random Membership Management
a Membership Service provides to a node a list of members
of a dynamic network
the overhead of maintaining the full list of members may
be too high for a complex network:
a node maintain a random subset of the nodes.
A random-walk based membership service: random sample
of size k is computed by node i as follows:
start k Metropolis Hastings random walks in parallel
the probability of visiting a node converges to the uniform
one
each node visited by the random walk sends its membership
information (for instance its IP address) to i which updates
its local membership set
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Applications: Load Balancing
Load imbalance in P2P networks may be caused by several
factors:
uneven distribution of data
heterogeneity in node capacities.
Load biased Random Walks:
sample node with probabilities proportional to their load
discover overloaded nodes more often
each node issues a random walker that persistently runs as
a node sampler
overloaded nodes exchange tasks with more light weighted
nodes
Basic Concepts Natural Random Walk Random Walks Characterization Metropolis Hastings Applications
Applications: Index Free Data Search
Index Free Search Method as alternatives to structured
DHT
flooding
random walks
Popularity-biased Random Walks
content popularity of a peer pi : number of queries satisfied
by pi divided by the total number of queries received by pi .
define a bias in the search towards more popular peers
Index Free Searching: each peer is probed with a probability
proportional to the square root of its query popularity