Mathematics for Computer Science
Applications of Random Walk
MIT 6.042J/18.062J
•Physics — Brownian motion
•Finance — stocks, options
•Algorithms — web search,
clustering
Random Walks
Albert R Meyer,
Lec 14W.1
May 12, 2010
Graph With Probable
Transitions
1/4
1/4
1/2
G
1/4
Lec 14W.4
May 12, 2010
Distribution Over Nodes
1/4
1/4
1/4
1/4
1/4
1/2
1
1/2
1/2
1/2
1
1/2
2/3
1/4
1/4
1/4
1
00
What are p’B, p’O, p’G after 1 step?
1
Albert R Meyer,
1/4
1/3
0
(pB, pO, pG)
Suppose you start at B: (1, 0, 0)
1/2
1/2
1/4
1/4
1/4
1/2
2/3
B
Lec 14W.2
May 12, 2010
Distribution Over Nodes
Outgoing-edge
probabilities
sum to 1
1/3
O
Albert R Meyer,
Albert R Meyer,
Distribution Over Nodes
1/3
0
1/3
1/4
2/3
1/4
1/4
1/4
1
Lec 14W.6
May 12, 2010
00
1/2
1/2
1/4
1/4
1
2/3
1/4
1 1 1 , , 2
4
4
Dist after 1 step: (p’B, p’G, p’O)
Dist after 1 step:
only get places from B,
so
Dist after 2 steps: (p’’B, p’’O, p’’G)
Albert R Meyer,
May 12, 2010
1 1 1 , , 2 4 4 Lec 14W.7
Albert R Meyer,
May 12, 2010
1
Distribution Over Nodes
Distribution Over Nodes
1/3
1/3
1/4
1/2
1/2
1/4
1/4
1
1/4
2/3
1/4
1 1 1 , , 2 4 4 Dist after 1 step:
p’’O = Pr{B to O|at B}•p’B
+ Pr{O to O|at O}•p’o
+ Pr{G to O|at G}•p’G
Albert R Meyer,
1/2
1/2
1/2
1/4
Albert R Meyer,
1/4
2/3
7/24
1/2
1
1 5
7 ,
,
2 24 24 1/2
1/4
1/4
2/3
pG
1/2
1
May 12, 2010
1/4
1/3
2/3
ptG
1
May 12, 2010
Finding Stationary Dist.
1/3
distribution (pB, pO, pG) is stationary
if next-step distribution is the same.
What is a stationary dist. here?
Albert R Meyer,
ptB
ptO
Albert R Meyer,
Stationary Distribution
pB
May 12, 2010
…and as t?
May 12, 2010
pO
= 5/24
distribution after t steps?
(p’’B, p’’O, p’’G)
1/4
1 1 1 , , 2 4 4 Distribution Over Nodes
distribution after 2 steps:
Albert R Meyer,
1/4
1/4
p’’O = Pr{B to
O|at B}•p’1/2
B
1/3
1/2
1
Dist after 1 step:
May 12, 2010
5/24
1/4
2/3
1/4
+ Pr{O to
1/3
O|at O}•p’
o
1/4
+ Pr{G to0O|at G}•p’
G
Distribution Over Nodes
1/4
1/4
Lec 14W.15
pB
pO
1/4
1/3
2/3
pG
1
pB = pB’ = (1/2)pB + 1pG
pO = pO’ = (1/4)pB + (1/3)pO
pG = pG’ = (1/4)pB + (2/3)pO
pB + pO + pG = 1
Albert R Meyer,
May 12, 2010
Lec 14W.16
2
Finding Stationary Dist.
pO
1/4
1/2
1/4
pB
Google Page Rank
1/3
View the entire web as a graph
• vertices are webpages
• edge (u,v) exists if link from page
u to page v
• Pr{go to v from u} = 1/outdeg(u)
Find stationary distribution {pu}
Rank u above v if pu > pv.
2/3
pG
8 3 4 ,
,
15 15 15 solving for
(pB, pO, pG):
Albert R Meyer,
May 12, 2010
Lec 14W.17
Questions on Stationary Dist
Albert R Meyer,
May 12, 2010
Further Questions
1/3
O
• Does a stationary dist exist?(if graphYes
finite)
• Is it unique?
Sometimes
• Does a random walk
Sometimes
approach it from any
starting distribution?
– How quickly?
Varies
Albert R Meyer,
May 12, 2010
Lec 14W.19
Lec 14W.18
1/4
B
1/2
2/3
1/4
G
1
• Pr{ever reach O | start at B}
• Pr{reach G before O | start at B}
• Average # steps for B to O
Albert R Meyer,
May 12, 2010
Lec 14W.20
Gambler’s Ruin
• Decide to place $1 bets until
either going broke or reaching
some target amount of money.
• What is Pr{reach target}?
Gambler’s Ruin
(let’s go to Vegas)
May 12, 2010
Lec 14W.21
Albert R Meyer,
May 12, 2010
Lec 14W.22
3
Gambling: Fair Case
Gambling: Fair Case
Suppose we’re playing a fair game:
• Pr{win bet} = 1/2.
What is Pr{reach $200} if we start
with $100?
1/2
In general, if we start with $n
Pr{reach $T} = n/T
What about an unfair game?
What about Pr{reach $600} if we
start with $500?
5/6
Albert R Meyer,
May 12, 2010
Lec 14W.23
Albert R Meyer,
Gambling: Slightly Unfair
May 12, 2010
Lec 14W.24
US Roulette
Betting black in
US roulette
What is Pr{reach $500+100} starting
with $500?
(5/6 when fair)
< 1 / 37,000
What is Pr{reach $1,000,100} starting
with $1,000,000? ( 1 when fair)
Image by MIT OpenCourseWare.
< 1 / 37,000
no matter how many $ at start!
Pr{win bet} = 18/38 = 9/19 < 1/2
Albert R Meyer,
May 12, 2010
Lec 14W.25
Albert R Meyer,
Gambler’s Ruin
T
May 12, 2010
Lec 14W.26
Gambler’s Ruin
• Play $1 bets until going broke or
make enough money.
Parameters
• p ::= Pr{win $1 bet}
• n ::= initial capital
• T ::= gambler’s target
Question: What is Pr{reach target}?
Albert R Meyer,
May 12, 2010
Lec 14W.27
”target"
$$$
n "initial capital"
# of bets
Albert R Meyer,
May 12, 2010
Lec 14W.28
4
Dow Jones Trend
Gambler’s Ruin
View as random walk on a line.
$0
$1
…
n-1
…
n+1
n
T
p
• p ::=Pr{win a bet}
k
• q ::= 1-p = Pr{lose a bet}
random steps with “up” bias?
T-1
k+1
k
k-1
q
What is Pr{reach T before 0}?
Albert R Meyer,
Lec 14W.29
May 12, 2010
Albert R Meyer,
Lec 14W.30
May 12, 2010
© Source unknown. All rights reserved.
This content is excluded from our Creative Commons license.
For more information, see http://ocw.mit.edu/fairuse
General Approach
p
…
n-1
n
n+1
General Approach
p
…
…
q
+ Pr{target
+
wn-1
| n-1} • Pr{lose
• q bet}
May 12, 2010
n+1
…
we have a linear recurrence!
Albert R Meyer,
Lec 14W.32
May 12, 2010
Linear Recurrence
wn =
rn 1
w1
r 1
for r ::= q/p 1.
wn+1 = (1/p)wn - (q/p)wn-1
w0 = 0
(Gambler is broke)
wT = 1
(Gambler is at target)
Twist: we
don’t know w1
But wT = 1, so can solve for w1. Then
Solve using generating functions
and get:
May 12, 2010
wn = pwn+1 + qwn-1, so
wn+1 = (1/p)wn - (q/p)wn-1
Lec 14W.31
Linear Recurrence
Albert R Meyer,
n
q
wn ::= Pr{hit target | start at n}
== Pr{target
wn+1
| n+1} • Pr{win
p
bet}
Albert R Meyer,
n-1
wn =
Lec 14W.34
Albert R Meyer,
rn 1
rT 1
May 12, 2010
Lec 14W.35
5
Winning when Biased Against
wn =
r 1
n
<
r 1
T
r
r
Profit $100 in US Roulette
n
T
p = 18/38
intended profit
Tn
1 = r q = 20/38
1/r = 9/10
Pr{Profit $100} < (9/10)100
< 1/37,648
Suppose p < q, so r ::= q/p > 1.
Albert R Meyer,
Lec 14W.36
May 12, 2010
Profit $200 in US Roulette
p = 18/38
q = 20/38
1/r = 9/10
Pr{Profit $200} < (9/10)200
< 1/70,000,000
Albert R Meyer,
What About the Fair Case?
wn =
rn 1
rT 1
May 12, 2010
Lec 14W.39
(r ::= q/p = 1)
•Uh oh, dividing by 0.
•Use l’Hôpital’s Rule
limr1
Albert R Meyer,
Lec 14W.38
May 12, 2010
d(rn-1)/dr nrn-1
=
=
d(rT-1)/dr TrT-1
Albert R Meyer,
May 12, 2010
n
T
Lec 14W.40
Team Problems
Problems
1&2
Albert R Meyer,
May 12, 2010
Lec 14W.41
6
MIT OpenCourseWare
http://ocw.mit.edu
6.042J / 18.062J Mathematics for Computer Science
Spring 2010
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.