I
DiscreteTime
part I
continuous
Time Markov chain
An example of discrete time Markov chain
Definition
Branching Process
Branching Process
discretetime Markov chain
are i.i.cl random variables withpmf
SM
9
is
and
加⼆
a
niiiiiiiii nslo.tn
知⼼ 9i i
nth generation
in
mi
Xuemeans thenumbers of individuals number
of children of ith individual
Fort n th generation 9_9means the
theextinctionprobability
Main question Howto compute
let 419 Ě Pk Ok o egg
Theorem
extinction
P 如0 forsome n
Then 0 let up
O 0 Egg
u is thesmallest rut of 419
extinctbythe nth generation P Mio
lintmybytthgy
Let an P
Un 4 Uni
To
and u
compute
u or un
find Xu
0
you
nz
can
and 4191
find thesmallest not 419
find Pk Ko 1，2
fu
Assignment
solve
un
4
question
l
2
undum iteratively withinitialcondition u
Exercise 1
Find u and Us
Pia
Rita
let
pmffft
䧇V
1 a 90 o
94a. i
Ok
ĒR
419
a
ag ou
solvedD 9
l
ag2
a
7ag2
O l1 a
Aia
灬
a win
兴 I
U 1
If
a
u
pl
U
u
1 2
1
兴
1
92
0
1
哥
extinct by the third generation
0
a
Un
a
f
4M 1 410
411 a
Uz flu
10U2
f
Um
以
u
以
x 1
呰
ocaci.IE
If
U3
吢
81
0 a
a
a 04 t a
a 1
1 a 4 ta
a12
2
1
11 a
9
ta
1 a 4zacl.ci
tll
a
d11
a3ll al4tza4taP4a taR ctaj
q
a 4
1
Time Markov Chain and Poisson Process
continuous Time MarkovChain
Definition
Markov chain is a stochastic process
A continuous time
1，2
index set To too and theState space Sc go
Dare I
continuous
SXHYuit .tk
that satisfies the Markov property
t.tn 1
U of t.octic.tc.nc.tn
in l XHnkin X Gm Kim
Plan
㗊
vi
i
Main Main
X 01 i P
蕊
燕 苎
蕊
in ES
i
An importantant
Before introduce
class of CTMC
Poisson Process
we need to know two types of distribution
Poisson Process
Poisson distribution
variable
A discrete random
Definition
X is said to follow
1，2
Kk
etf KEfo
with parameter µso if P
a
If XvPoisson W then Et Var X
Duperty
If Xv Poissonlui Y v PoissonWH Xi Y
then Xt Yu Poisson Mtn
a
Poissondistr.hn
了
lb
Exponential distribution
continuous random variable
Definition A
with rate ⼊ if
PT
f
Var ㄒ
六
ffkofmnremy.JP
the
distribution
of
does we depend on how
ii
has exponential distribution
eit.tt
t
Property O ET
T
Pts IT
watt
times
men
b
PT
t
to
an
has passed
d l
s
event
dusider a post office that
Exercise 2
MA
run by two
discovers that
enters theoffice and
or
Mr.B is beingservedby
Mr.A willbegin his service as soon as
If theamount of time that a clerkspends
clerkland Mv.B.by clerk2
Mr.B
clerks
Mr.Bz leaves
distributed with rate ⼊
with a customer is exponentially
that ofthethreecustomers Mr.A is the last
probability
the
Find
leave thepose office
想 辔
n_n
1
Let
let
let
2
if Bzleave first.tn
䃕
䎕
花
begin his service
A
that
t be the time
exponan.dk
T
service time
the
A's
T.be
cho are
of
customer
th
time
T be the remaining
still
being served at
the
27
⼀⼀
iiipi.tt
Luhy
to
the service tire
let TB 有 be
If 131 leave first leavefirst
P 估 S 1131
If 137 leave first
A Bi
132
P 在 234t ⽯ t
P 下32 75
P 历 175
P Tzss1132 leave
却傆
ÍP 佦
Lau of total prob i 所以
P 有275
左 有2
左 expo ⼋
ˋ
Poisson Process
Poisson Process
Definition
A
homogeneous
X
Poisson process
X
of rate ⼊ is
⼆0
f any
time points to
⼼
ctu
a
GMC thatsatisfies
theprocess increments
Xltn Xltm
are independent
XG.tl lto Xltz X
Poisson at
X Stt X s
so and t so
for any
time
events that occur by
number
of
Xt means the total
ti
Theimǜǚìtsfi
then To下 ⽯
are
then and
N
ii.cl and Tine exponential
its
t
event
Exercise3
⼋
2
13
n
Xlt is a Poisson process of rate ⼊
Pr 如 61 姙 8
E It T T ⼗万 Mi 2
If
E X14⼈ 如
K
find
2
3
8
B1 加 1 6M41
⼆
X141 81
Pr 加1 6
Tg
iii
不如
⼼
1
器
如
只炎
华
加 1 X141
P 4
ㄨ4
学
灐管
127
P
1
E To
⼗下 坧⼗万
砹繭
Th
1X
ˋ
㖄
⼊
年
for2
HEKine
more events
2
1
⼤ 2
1
Ì
2
2
Elytra1
El X14__ 加 7
凹
巧
Poisson 2 ⼊
2 ⼋
⼆
4
Exercise 4
A bus will arrive at
a train
The arrivaltime follows uniform1011
according to
Passengers arrive
Let X
a
station
hours
Tisson process
with rat 7perhour
thenumber ofpeople who geton the train
品品
台
Find
ylbusarnle
n la
Y be the time that the train
Then You U o 1
Xiu Poisson Tg
Given Ky
EM 的
7yvarlxlkgj
arrive
7
EKEEMYKEMYK7
EE I
Var K EN ar MY
EMYD
Var17Y
t Var
E17Y t
7 EY 1 49 Vary
49 ⾆
⼆下 之
台