Basic properties:
Probability and Expectation:
•
•
•
•
•
•
The conditional probability of A given B:
Expectation for discrete case:
Expectation for continuous case:
The law of total expectation:
Bayes’ rule:
The law of total probability:
P(A | B) = P(A∩B) / P(B)
E(h(X)) = ∑xh(x) * p(x)
E(h(X)) = ∫h(x) * f(x)dx
E(X ) = E(E(X |Y))
P(A|B)=P(B|A)*P(A) / P(B)
P(A) = ∑yP(A |Y = y) * P(Y = y)
P(A) = ∫P(A |Y = y ) * fy(y)dy
• Variance:
Var(X) = E(Var(X |Y )) + Var(E(X |Y))
Var(X |Y ) = E(X² |Y ) − (E(X |Y))
Exponential distribution:
• The probability density function (pdf) of an exponential distribution is:
• The cumulative distribution function of an exponential distribution is:
• Properties:
▶ 𝐸(𝑋 ) =
1
𝜆
1
▶ 𝑉𝑎𝑟(𝑋 ) = 𝜆2
𝜆
▶ 𝑀𝐺𝐹: 𝜑(𝑡) = 𝜆−t , 𝑡 < 𝜆
• Memoryless property: 𝑃(𝑋 > 𝑠 + 𝑡 |𝑋 > 𝑡) = 𝑃(𝑋 > 𝑠) 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑠 , 𝑡 ≥ 0.
• Hazard/failure rate is equal to 𝜆 (only distribution with constant hazard rate).
𝜆1
• 𝑃(𝑋1 < 𝑋2 ) = 𝜆1 + 𝜆2
• Let X1,...,Xn be iid exponential with common mean 1/λ. Then the random variable Y =∑i Xi has a gamma
distribution with parameters n and λ.
• min Xj has an exponential distribution with rate ∑j λj.
• The random variable mini Xi and the rank ordering of the Xi (i.e., Xi1 < Xi2 < ···< Xin ) are independent.
Poisson distribution:
• The probability mass function of a Poisson distribution is:
• Properties:
▶ 𝐸(𝑋 ) = 𝑉𝑎𝑟(𝑋) = 𝜆
▶ Als 𝑋𝑖 ~𝑃𝑜𝑖𝑠(𝜆𝑖 ) 𝑎𝑛𝑑 𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡, 𝑡ℎ𝑒𝑛 ∑𝑛𝑖=1 𝑋𝑖 ~𝑃𝑜𝑖𝑠(∑𝑛𝑖=1 𝜆𝑖 )
▶𝑀𝐺𝐹: 𝜑(𝑡) = 𝑒 𝜆(𝑒
𝑡 −1)
1
Discrete time Markov Chain:
Markov Chain definition:
• A stochastic process is a collection of (infinitely many) random variables:
- Discrete time Stochastic process is of the form {Xn ,n ≥0}
- Continuous time Stochastic process is of the form {X (t),t ≥0}
• A stochastic process {Xn ,n≥0} with state space S is called a discrete time Markov chain if for all states
i ,j ,s0,...,sn−1 ∈ S: P(Xn+1 = j |Xn = i ,Xn−1 = sn−1,...,X0 = s0) = P(Xn+1 = j |Xn = i) (Markov property).
• In time homogeneous Markov Chains we have: P(Xn+1 = j |Xn = i) = P(X1 = j |X0 = i) = Pij.
• A random walk is a Markov Chain where if the chain is in state I, it only can got to i+1 or i-1.
Recurrent and Transient States:
• Recall the notation Pijk: the probability that starting from state i, after k steps, arriving at state j. Then j is
called accessible from i if Pijk > 0 for some k ≥ 0.
• States i and j are said to communicate if they are accessible from each other. We denote this by i ↔ j.
• Communicating states form a class. If there is only 1 class, the MC is ‘irreducible’, otherwise it’s ‘reducible’.
• A state is recurrent if fi = 1, and transient if fi < 1. (fi is probability that if you start in i, that you ever come
back to i).
• ▶ State i is recurrent if ∑Piin is infinite.
▶ State i is transient if ∑Piin is finite.
• Recurrence and transience are class properties.
• In a MC not all states are transient, and in a irreducible MC all states are recurrent.
• Two types of recurrence (both are class properties): Denote Nj = min{ n > 0, Xn = j}
▶ Positive recurrent if the expected time until the process returns to the same state is finite:
E(Nj |X0 = j) < +∞, in a finite state MC, alle recurrent states are positive recurrent.
▶ Null recurrent if the expected time until the process returns to the same state is infinite:
E(Nj |X0 = j) = +∞
• Period d of state i is (class property): d = gcd{n > 0 : Piin > 0}, with ‘gcd’ greatest common divisor:
▶ A state is periodic if d > 1.
▶ A state is aperiodic if d = 1.
• An aperiodic, positive recurrent state is called ergodic.
Long run limit:
• For an irreducible ergodic Markov chain: lim 𝑃𝑖𝑗𝑛 exists and is independent of i.
𝑛→∞
• Denote π𝑗 = lim 𝑃𝑖𝑗𝑛 for j ∈ S and the limiting distribution π = (πj) for each j ∈ S.
𝑛→∞
• Denote the stationary distribution with w = (wj) j ∈ S which is the unique solution of the steady-state
𝑤𝑗 = ∑𝑖 𝑤𝑖 𝑃𝑖𝑗 , 𝑗 ∈ 𝑆
𝒘 = 𝑃𝑇 ∗ 𝒘
equations: {
or
{ ∑
∑𝑗 𝑤𝑗 = 1
𝑗 𝑤𝑗 = 1
• Once the MC starts from w, we will always have P(Xn = j) = wj.
• For an irreducible ergodic Markov chain, the limiting distribution π coincides with the stationary distribution
w.
• Let {Xn, n ≥ 1} be an irreducible Markov chain with stationary probabilities πj , j ≥ 0, with r(j) as reward of
being in state j. Then: ∑𝑗 𝑟(𝑗)π𝑗 is called the average reward per unit time.
2
Standard questions:
• Denote T = {1,…,t} as the transient states and {t+1,…,s} as the recurrent states.
𝑠11 ⋯ 𝑠1𝑡
𝑃11 ⋯ 𝑃1𝑡
⋱
⋮ )
⋱
⋮ )and S = ( ⋮
• Let Pt = ( ⋮
𝑠𝑡1 ⋯ 𝑠𝑡𝑡
𝑃𝑡1 ⋯ 𝑃𝑡𝑡
Notation:
• fi = probability that, starting in state i, the process will ever re-enter state i.
• sij = expected number of time periods the MC is in j, given that it started in i (mean time spent). Note: i and j
are transient states.
• fij = probability that, starting in state i, the process will ever enter state j. Note: i and j are transient states.
• miR = expected number of steps to enter recurrent class R, given that it started in i (mean time it takes to
entry R). Note: i is a transient state and R is the only recurrent class.
• fiR1 = probability that, starting in state i, the process will ever enter recurrent class R1. Note: i is transient and
there can be multiple recurrent classes.
Solutions:
1 + ∑𝑡𝑘=1 𝑃𝑖𝑘 𝑠𝑘𝑗 , 𝑎𝑙𝑠 𝑖 = 𝑗
• 𝑠𝑖𝑗 = {
or 𝑆 = (𝐼 − 𝑃𝑇 )−1
∑𝑡𝑘=1 𝑃𝑖𝑘 𝑠𝑘𝑗 , 𝑎𝑙𝑠 𝑖 ≠ 𝑗
(𝑠𝑖𝑗 −1)
• 𝑓𝑖𝑗 = 𝑃𝑖𝑗 + ∑𝑡𝑘=1,𝑘≠𝑗 𝑃𝑖𝑘 𝑓𝑘𝑗 or 𝑓𝑖𝑗 = {
𝑠𝑖𝑗
𝑠𝑖𝑗
𝑠𝑗𝑗
𝑎𝑙𝑠 𝑖 = 𝑗
𝑎𝑙𝑠 𝑖 ≠ 𝑗
• 𝑚𝑖𝑅 = 1 + ∑𝑡𝑗=1 𝑃𝑖𝑗 𝑚𝑗𝑅 or 𝒎 = S ∗ 𝟏
• 𝑓𝑖𝑅1 = 𝑃𝑖𝑅1 + ∑𝑡𝑗=1 𝑃𝑖𝑗 𝑓𝑗𝑅1 or 𝒇𝑅1 = S ∗ 𝑷𝑅1 with 𝑷𝑅1 = (
3
∑𝑗∈𝑅1 𝑃1𝑗
⋮
)
∑𝑗∈𝑅1 𝑃𝑡𝑗
Continuous time Markov Chain:
Counting process:
• A stochastic process {N(t),t ≥ 0} is a counting process whenever N(t) denotes the total number of events that
occur by time t. It should satisfy the following:
▶ N(t) ≥ 0.
▶ N(t) is integer valued.
▶ For s < t, N(s) ≤ N(t).
• For s < t: N(t) − N(s) represents the number of events that occur in the interval (s,t].
• A counting process has independent increments whenever the number of events that occur in one time
interval is independent of the number of events that occur in another (disjoint) time interval.
▶ That is, N(s) is independent of N(s + t) − N(s).
• A counting process has stationary increments whenever the number of events that occur in any interval
depends only on the length of the interval.
▶ That is, the number of events in the interval (s,s + t] has the same distribution for all s.
Poisson Process:
First definition:
• The counting process {N(t),t ≥ 0} is a Poisson process with rate λ, λ > 0 when:
1. N(0) = 0.
2. The process has independent increments.
3. The number of events in any interval of length t is Poisson distributed with mean λt. In other words, for all
s,t ≥ 0:
𝑃(𝑁(𝑡 + 𝑠) − 𝑁(𝑠) = 𝑛) = 𝑒 −𝜆𝑡
(𝜆𝑡)𝑛
𝑛!
, 𝑛 = 0, 1, …
• Note that the last condition implies that a Poisson process:
▶ has stationary increments.
▶ E(N(t)) = λt.
Second Definition:
• The counting process {N(t),t ≥ 0} is a Poisson process with rate λ, λ > 0 when:
1. N(0) = 0.
2. The process has stationary and independent increments.
3. 𝑃(𝑁(ℎ) = 1) = 𝜆ℎ + 𝑜(ℎ), 𝑎𝑠 ℎ → 0
4. 𝑃(𝑁(ℎ) ≥ 2) = 𝑜(ℎ), 𝑎𝑠 ℎ → 0
𝑔(ℎ)
ℎ→𝑜 ℎ
• A function g(·) is said to be o(h) if lim
= 0, so g(h) goes faster to zero than h.
Third Definition:
• For a Poisson process, let Tn, n > 1 be the nth interarrival time: the time elapsed between the (n − 1)th event
and the nth event. It follows that Ti is exponential with rate λ for every i.
• The arrival time of the nth event, Sn, is also called the waiting time until the nth event. Clearly,
𝑛
𝑆𝑛 = ∑ 𝑇𝑖 , 𝑛 ≥ 1.
𝑖=1
• Thus, Sn has a gamma distribution with parameters with n and λ yielding
(𝜆𝑡)𝑛−1
𝑛
𝑛
𝑓𝑆𝑛 (𝑡) = 𝜆𝑒 −𝜆𝑡 (𝑛 − 1)! , 𝑡 ≥ 0. 𝐸(𝑆𝑛) = λ . 𝑉𝑎𝑟(𝑆𝑛) = λ2
• Note that 𝑁(𝑡) ≥ 𝑛 ⇐⇒ 𝑆𝑛 ≤ 𝑡.
• If we denote N(t) by: 𝑁(𝑡) ≡ 𝑚𝑎𝑥{𝑛 ∶ 𝑆𝑛 ≤ 𝑡}, with 𝑆𝑛 = ∑𝑛𝑖=1 𝑇𝑖 and Ti i.i.d. exponential random
variables with rate λ. It then follows that {N(t), t ≥ 0} is a Poisson process with rate λ.
4
Merging two Poisson Process:
• Suppose that {N1(t),t ≥ 0} and {N2(t),t ≥ 0} are independent Poisson processes with respective rates λ1 and λ2,
where Ni(t) corresponds to type i arrivals. Let N(t) = N1(t) + N2(t), for t ≥ 0. Then the following holds:
▶ The merged process {N(t),t ≥ 0} is a Poisson process with rate λ = λ1 + λ2.
▶ The probability that an arrival in the merged process is of type i is 𝜆𝑖 /(𝜆1 + 𝜆2 ).
Decomposing a Poisson Process:
• Consider a Poisson process {N(t),t ≥ 0} with rate λ. Suppose that each event in this process is classified as
type I with probability p and type II with probability (1 − p) independently of all other events. Let N1(t) and
N2(t) respectively denote the type I and type II events occurring in time (0,t]. Then, the counting processes
{N1(t),t ≥ 0} and {N2(t),t ≥ 0} are two independent Poisson processes with respective rates λp and λ(1 − p).
Conditional arriving process:
• If Y1, … , Yn are i.i.d. with density f, then the joint density of the order statistics Y(1), . . . , Y(n) is:
𝑛
f(y1 , . . . , y𝑛 ) = n! ∏ f(y𝑖 ) ,
y1 ≤ · · · ≤ y𝑛 .
𝑖=1
• Given that N(t) = n, the n arrival times S1, ... , Sn have the same distribution as the order statistics
corresponding to n independent random variables uniformly distributed on the interval (0,t). So:
(S1 , S2 , . . . , S𝑛 ) =𝑑 (U(1) , U(2) , . . . , U(𝑛) )
With U(1) ≤ U(2) ≤. . . ≤ U(𝑛) order statistics of i.i.d. random variables from U(0,t).
• For any function f (with symmetric operation): ∑𝑛𝑖=1 𝑓(𝑆𝑖 ) =𝑑 ∑𝑛𝑖=1 𝑓(𝑈(𝑖) ) =𝑑 ∑𝑛𝑖=1 𝑓(𝑈)
Nonhomogeneous Poisson process:
• The counting process {N(t),t ≥ 0} is said to be a nonhomogeneous Poisson process with intensity function λ(t),
t ≥ 0, if
1. N(0) = 0.
2. {N(t),t ≥ 0} has independent increments.
3. P(N(t + h) − N(t) = 1) = λ(t)h + o(h).
4. P(N(t + h) − N(t) ≥ 2) = o(h)
• For a nonhomogeneous Poisson process N(t) with intensity function λ(t) holds that 𝑁(𝑠 + 𝑡) – 𝑁(𝑡) is a
Poisson random variable with mean 𝑚(𝑠 + 𝑡) − 𝑚(𝑡), with:
𝑡
𝑚(𝑡) = ∫ 𝜆(𝑦)𝑑𝑦
0
General CTMC:
• ▶ Let {X(t),t ≥ 0} be a continuous-time stochastic process taking values in {0, 1, 2, . . .}.
▶ Let {x(t),t ≥ 0} be any deterministic function taking values in {0, 1, 2, . . .}.
• The process {X(t),t ≥ 0} is called a continuous-time Markov chain if:
𝑃(𝑋(𝑡 + 𝑠) = 𝑗 | 𝑋(𝑠) = 𝑖, 𝑋(𝑢) = 𝑥(𝑢), 0 ≤ 𝑢 < 𝑠) = 𝑃(𝑋(𝑡 + 𝑠) = 𝑗 | 𝑋(𝑠) = 𝑖)
for all s, t ≥ 0, functions {x(t), t ≥ 0} and i, j = 0, 1, 2, …
• If a continuous-time Markov chain {X(t),t ≥ 0} satisfies:
𝑃(𝑋(𝑡 + 𝑠) = 𝑗 | 𝑋(𝑠) = 𝑖) = 𝑃(𝑋(𝑡) = 𝑗 | 𝑋(0) = 𝑖) for every s,t ≥ 0, then {X(t),t ≥ 0} is stationary or
time-homogeneous.
• Let Ti denote the time the process {X(t),t ≥ 0} spends in state i before making a transition into a different
state. Then by Markov property it must have exponential distribution with rate vi.
• Let Pij denote the probability of next entering state j, given that the current state is i, then:
𝑃𝑖𝑖 = 0 𝑎𝑛𝑑 ∑ 𝑃𝑖𝑗 = 1 𝑓𝑜𝑟 𝑒𝑣𝑒𝑟𝑦 𝑖 = 0,1,2, …
𝑗
5
Birth and death process:
•
•
•
•
•
If only the transition from i to i + 1 is allowed, then the process is called a pure birth process.
If only transitions from i to i − 1 or i + 1 are allowed, then the process is called a birth and death process.
A pure birth process starting at zero is a counting process.
Arrivals occur with rate λi . That is, the time until the next arrival is exponentially distributed with mean 1/λi .
Departures occur with rate µi . That is, the time until the next departure is exponentially distributed with
mean 1/µi .
• The Poisson process is a pure birth process with all λi equal to common arrival rate λ.
• 𝐸(𝑇𝑖 ) =
1
,
𝜆𝑖 + µ𝑖
𝑃𝑖,𝑖−1 = 𝜆
µ𝑖
, 𝑃𝑖,𝑖+1
𝑖 + µ𝑖
=𝜆
𝜆𝑖
𝑖 + µ𝑖
Transition probabilities:
• The transition probability function of the continuous-time Markov chain is given by
𝑃𝑖𝑗 (𝑡) = 𝑃(𝑋(𝑡 + 𝑠) = 𝑗 | 𝑋(𝑠) = 𝑖).
• The rate of transition from state i into state j is given by 𝑞𝑖𝑗 = 𝑣𝑖 𝑃𝑖𝑗 .
▶ The qij values are called the instantaneous transition rates.
▶ The vi values are the rates of the time until next transition given that you are currently in state i.
▶ Note that qii = 0 as a consequence of the fact that Pii = 0.
𝑞𝑖𝑗
• It follows that: 𝑣𝑖 = ∑𝑗≠𝑖 𝑞𝑖𝑗 and 𝑃𝑖𝑗 =
∑𝑗≠𝑖 𝑞𝑖𝑗
• It can be proven that: lim
ℎ→0
𝑃𝑖𝑗 (ℎ)
ℎ
= 𝑞𝑖𝑗 , which shows that the instantaneous transition rate qij is the derivative
Pij’(0) of the transition probability Pij(t) with respect to t, evaluated in t = 0.
• It can be proven that: lim
ℎ→0
1−𝑃𝑖𝑖 (ℎ)
ℎ
= 𝑣𝑖 , which shows −vi is the derivative Pii’(0) of the transition probability
Pii(t) with respect to t, evaluated in t = 0.
Kolmogorov equations:
• Chapman-Kolmogorov equations: for all 𝑠 ≥ 0, 𝑡 ≥ 0
∞
𝑃𝑖𝑗 (𝑡 + 𝑠) = ∑ 𝑃𝑖𝑘 (𝑡)𝑃𝑘𝑗 (𝑠)
𝑘=0
• Kolmogorov backward-equations:
∞
𝑃𝑖𝑗′ (𝑡)
= ∑ 𝑞𝑖𝑘 𝑃𝑘𝑗 (𝑡) − 𝑣𝑖 𝑃𝑖𝑗 (𝑡)
𝑘≠𝑖
• Kolmogorov forward-equations:
∞
𝑃𝑖𝑗′ (𝑡)
= ∑ 𝑞𝑘𝑗 𝑃𝑖𝑘 (𝑡) − 𝑣𝑖 𝑃𝑖𝑗 (𝑡)
𝑘≠𝑖
Limiting probabilities:
𝑣𝑗 𝑃𝑗 = ∑𝑘≠𝑗 𝑞𝑘𝑗 𝑃𝑘 𝑣𝑜𝑜𝑟 𝑎𝑙𝑙𝑒 𝑗
• Balance equations: {
∑𝑗 𝑃𝑗 = 1
• Limiting probabilities exist if and only if all states of MC communicate and the MC is positive recurrent, i.e.
the MC is ergodic. Like in discrete-time, Pj are also called stationary probabilities.
• For a birth and death process follows that the balance equations are:
𝜆0 𝑃0 = 𝜇1 𝑃1 𝑣𝑜𝑜𝑟 𝑖 = 0
{
(𝜆𝑖 + 𝜇𝑖 )𝑃𝑖 = 𝜇𝑖+1 𝑃𝑖+1 + 𝜆𝑖−1 𝑃𝑖−1 𝑣𝑜𝑜𝑟 𝑖 > 0
• It follows: 𝑃0 = 1/(1 + ∑∞
𝑖=1
𝜆0 …𝜆𝑖−1
)
𝜇1 …𝜇𝑖
6
Queuing theory:
• Kendall’s notation: Queueing systems are often indicated via two letters followed by one or two numbers:
M/M/1, M/M/2/5.
▶ The first letter indicates the arrival process:
D: Deterministic: clients arrive at equidistant time points.
M: Markovian: clients arrive according to a Poisson process.
G: General: clients arrive according to a general arrival process.
▶ The second letter indicates the type of service times:
D: Deterministic: service times are fixed.
M: Markovian: service times S1, S2, . . . are independent exponential random variables with common rate.
G: General: service times S1, S2, . . . are independent and identically distributed (i.i.d.) random variables.
They may have any distribution.
▶ The first number indicates the number of servers.
▶ The second number indicates the capacity of the system, that is, the maximum number of clients in the
system. The capacity is equal to the number of servers plus the maximum number of waiting clients.
• M/M/1 queue:
▶ Customers arrive at the server according to a Poisson process with rate λ.
▶ The service needs some time to complete.
▶ The successive service times are independent exponential random variables with mean 1/µ
▶ The number of clients in the system is a birth and death process with common arrival rate λ and common
departure rate µ.
𝜆 𝑖
𝜇
𝜆
𝜇
𝜆
𝜇
▶ Limiting probabilities are 𝑃𝑖 = ( ) (1 − ) 𝑝𝑟𝑜𝑣𝑖𝑑𝑒𝑑 𝑡ℎ𝑎𝑡 < 1.
Little’s law:
• N(t) is number of arrivals up to time t.
• Overall arrival rate into the system: 𝜆 = lim
𝑡→∞
𝑁(𝑡)
.
𝑡
• Let Vn denote sojourn time of client n, that is, the time client n spends in the system.
• Then average sojourn time W (the average time client n spends in the system) is 𝑊 = lim
1
𝑡
1
𝑛→∞ 𝑛
∑𝑛𝑗=1 𝑉𝑛
• Let X(t) denote the number of clients in the system at time t. Then, 𝐿 = lim 𝑡 ∫0 𝑋(𝑠) 𝑑𝑠 is the average
𝑡→∞
number of clients in the system (over time). (sometimes 𝐿 = ∑𝑠𝑗=0 𝑗𝑃𝑗 if there are s+1 states)
• Little’s law: 𝐿 = 𝜆𝑊
𝜆
1
• For M/M/1 queue: 𝐿 = 𝜇−𝜆, 𝑊 = 𝜇−𝜆
PASTA Principle:
• ▶ Define the long-run or steady-state probability of exactly n clients in the system by:
𝑃𝑛 = lim 𝑃(𝑋(𝑡) = 𝑛), Pn is often also the long-run proportion the system contains exactly n clients.
𝑡→∞
▶ an long-run proportion of clients that find n in the system upon arrival.
▶ dn long-run proportion of clients that leave n in the system upon departure.
• In systems in which clients arrive and depart one at a time, the two probabilities an and dn coincide.
• The PASTA property: Poisson arrivals see time averages.
If the arrival process is a Poisson process:
▶ The arrivals occur homogeneously over time
▶ The averages over time and over clients are the same
Then it holds that 𝑷𝒏 = 𝒂𝒏
7
Gaussian Processes:
1
• A random variable R which satisfies P(𝑅 = −1) = 𝑃(𝑅 = +1) = 2 is called a Rademacher random
variable. Properties: 𝐸(𝑅) = 0, 𝑉𝑎𝑟(𝑅) = 1.
Brownian Motion:
First definition:
• 1. Brownian motion starts at zero: W (0) = 0.
2. Brownian motion has stationary and independent increments.
3. Brownian motion evaluated at a fixed time t1 is a normal random variable with mean zero and variance t1.
Second definition:
• 1.Brownian motion starts at zero: W (0) = 0.
2. For t1 ≤ t2, W (t1) and W (t2) have a bivariate normal distribution with mean zero and covariance t1.
Properties:
• W(0) = 0.
• Cov(W(t1),W(t2)) = min(t1,t2).
• For t1 ≤ t2: Cov(W (t1), W(t2) − W(t1)) = 0.
1
𝑛
√
𝑛→∞
[𝑛𝑡]
• Brownian motion is the limit of aa random walk: 𝑊(𝑡) = lim 𝑊𝑛 (𝑡) = lim ( ) ∑𝑖=1 𝑅𝑖
𝑛→∞
Reflection Principle:
• General case if Sn is the sum of n Rademacher variables (n and k different parity):
𝑃(𝑚𝑎𝑥𝑗=1,…,𝑛 𝑆𝑗 ≥ 𝑘) = 2𝑃(𝑆𝑛 ≥ 𝑘)
• Brownian motion case:
𝑃(𝑠𝑢𝑝0≤𝑡≤𝑏 𝑊(𝑡) > y) = 2𝑃(W(b) > y)
• Let Ta and Tb be two hitting times (first time BM hits the level y) with a, b > 0. The probability that
𝑃(𝑇𝑎 < 𝑇𝑏 ) is:
▶ 0 𝑖𝑓 𝑎 > 𝑏 > 0
▶ 1 𝑖𝑓 𝑏 > 𝑎 > 0
▶ −𝑏/(𝑎 − 𝑏) 𝑖𝑓 𝑎 > 0 > 𝑏
• Boundary crossing from both sides:
∞
𝑃(𝑠𝑢𝑝0≤𝑡≤𝑏 |𝑊(𝑡)| > y) = 2 ∑(−1)𝑗+1 𝑃(𝑠𝑢𝑝0≤𝑡≤𝑏 𝑊(𝑡) > (2j − 1)y)
𝑗=1
• Butler test: use to test if sample Y1,…,Yn is symmetric around 0.
▶ Rearrange the sample so as to satisfy |𝑌(1)| ≤ |𝑌(2)| ≤ . . . ≤ |𝑌(𝑛) |.
1 𝑖𝑓 𝑌(𝑖) > 0
▶ Define random variables R1, R2, . . . , Rn by 𝑅𝑖 = {
.
−1 𝑖𝑓 𝑌(𝑖) < 0
1
[𝑛𝑡]
▶ Butler’s test statistic 𝑇𝑛 = 𝑠𝑢𝑝0≤𝑡≤1 |( 𝑛) ∑𝑖=1 𝑅𝑖 | .
√
▶ Under null hypothesis, Tn converges in distribution to the absolute supremum of the Brownian motion on
the unit interval. Use critical values to perform test and reject null for high values of Tn.
𝑊(𝑡)
2
• Linear boundaries (Doob): 𝑃(𝑠𝑢𝑝𝑡≥0 1+𝑎𝑡 > 𝑦) = 𝑒 −2𝑎𝑦 .
▶ Can be used for Brownian motion with drift: 𝑋(𝑡) = 𝑊(𝑡) + 𝜇𝑡. It follows that 𝑃(𝑠𝑢𝑝𝑡≥0 𝑋(𝑡) > 𝑦) =
𝑃(𝑠𝑢𝑝𝑡≥0
𝑊(𝑡)
𝜇
𝑦
1− 𝑡
> 𝑦) = 𝑒 2𝜇𝑦
8
Brownian Bridge:
Empirical Process:
1
𝑛
• Empirical distribution function: 𝐹𝑛 (𝑥) = ∑𝑛𝑖=1 𝐼{𝑋𝑖≤𝑥} .
• Empirical process: √𝑛(𝐹𝑛 (𝑥) − 𝐹(𝑥)).
• CLT gives as n goes to infinity: √𝑛(𝐹𝑛 (𝑥0 ) − 𝐹(𝑥0 )) 𝑑→ 𝑁 (0, 𝐹(𝑥0 )(1 − 𝐹(𝑥0 ))) .
1
• Uniform empirical process: 𝐵𝑛 (𝑢) = √𝑛(𝑛 ∑𝑛𝑖=1 𝐼{𝑈𝑖≤𝑢} − 𝑢) 𝑣𝑜𝑜𝑟 0 ≤ 𝑢 ≤ 1, met U1,…,Un i.i.d. U[0,1].
• If random variable X has CDF F(x), then 𝐹(𝑋)~𝑈[0,1]. Because of this follows:
√𝑛(𝐹𝑛 (𝑥) − 𝐹(𝑥)) 𝑑= 𝐵𝑛 (𝐹(𝑥))
• Property: the uniform empirical process at two points s and t converges to a bivariate normal distribution
with mean zero and: 𝐶𝑜𝑣(𝐵𝑛 (𝑡), 𝐵𝑛 (𝑠)) = min(𝑠, 𝑡) − 𝑠𝑡
Definition:
• A Brownian Bridge is the limiting process of {𝐵𝑛 (𝑢), 0 ≤ 𝑢 ≤ 1}, and is denoted by {𝐵(𝑢), 0 ≤ 𝑢 ≤ 1}
• Definition:
1. 𝐹𝑜𝑟 𝑒𝑣𝑒𝑟𝑦 0 ≤ 𝑢 ≤ 1, 𝐵(𝑢) 𝑖𝑠 𝑎 𝑛𝑜𝑟𝑚𝑎𝑙 𝑟𝑎𝑛𝑑𝑜𝑚 𝑣𝑎𝑟𝑖𝑎𝑏𝑙𝑒 𝑤𝑖𝑡ℎ 𝑚𝑒𝑎𝑛 𝑧𝑒𝑟𝑜 𝑎𝑛𝑑 𝑣𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝑢(1 − 𝑢)
2. 𝐹𝑜𝑟 𝑒𝑣𝑒𝑟𝑦 0 ≤ 𝑢1 , 𝑢2 ≤ 1, (𝐵(𝑢1 ), 𝐵(𝑢2 )) 𝑖𝑠 𝑎 𝑏𝑖𝑣𝑎𝑟𝑖𝑎𝑡𝑒 𝑟𝑎𝑛𝑑𝑜𝑚 𝑣𝑒𝑐𝑡𝑜𝑟 𝑤𝑖𝑡ℎ
𝐶𝑜𝑣(𝐵(𝑢1 ), 𝐵(𝑢2 )) = 𝑚𝑖𝑛 (𝑢1 , 𝑢2 ) − 𝑢1 𝑢2
Asymptotic statistics:
• As the sample size n → ∞, the general empirical process {√𝑛(𝐹𝑛 (𝑥) − 𝐹(𝑥)), 𝑥 ∈ 𝑅} converges to a limiting
process {𝐵(𝐹(𝑥)), 𝑥 ∈ 𝑅}.
• A more rigorous way to formulate the convergence is as follows: 𝑠𝑢𝑝𝑥∈𝑅 |√𝑛(𝐹𝑛 (𝑥) − 𝐹(𝑥)) − 𝐵(𝐹(𝑥))| 𝑃→ 0
• Delta method(univariate):
𝐼𝑓 √𝑛(𝜃𝑛 − 𝜃) 𝑑→ 𝑁(0, 𝜎 2 ) 𝑡ℎ𝑒𝑛 √𝑛(𝑔(𝜃𝑛 ) − 𝑔(𝜃)) 𝑑→ 𝑔′ (𝜃)𝑁(0, 𝜎 2 ) 𝑓𝑜𝑟 𝑒𝑣𝑒𝑟𝑦 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡𝑖𝑎𝑏𝑙𝑒 𝑔(𝜃)
• Delta method (bivariate):
Δ𝑔
Δ𝑔
𝐿
𝜃
𝜃
𝐼𝑓 √𝑛 (( 𝑛 ) − ( )) 𝑑→ ( 1 ) 𝑡ℎ𝑒𝑛 √𝑛(𝑔(𝜃𝑛 , 𝜇𝑛 ) − 𝑔(𝜃, 𝜇)) 𝑑→
(𝜃, 𝜇)𝐿1 +
(𝜃, 𝜇)𝐿2 𝑓𝑜𝑟 𝑔(𝑥, 𝑦)
𝜇𝑛
𝜇
𝐿2
Δ𝑥
Δ𝑦
Brownian Motion to Brownian Bridge:
• Let {𝑊 (𝑡), 0 ≤ 𝑡 ≤ 1} be a Brownian motion. The process {𝑋(𝑡), 0 ≤ 𝑡 ≤ 1} defined by 𝑋(𝑡) =
𝑊 (𝑡) − 𝑡𝑊 (1) for 0 ≤ t ≤ 1 is a Brownian bridge on the unit interval.
• By conditioning on the event {𝑊 (1) = 0}, the Brownian motion {𝑊 (𝑡), 𝑡 ≥ 0} becomes a Brownian
bridge on the unit interval.
Brownian Bridge to Brownian Motion:
• Let Z be a standard normal random variable, independent of the Brownian bridge {𝐵(𝑡), 𝑡 ≥ 0}. Then, the
process {𝑋(𝑡), 𝑡 ≥ 0} defined by 𝑋(𝑡) = 𝐵(𝑡) + 𝑡𝑍, 0 ≤ 𝑡 ≤ 1 is a Brownian motion on the unit interval.
Kolmogorov-Smirnov test:
• Suppose we have a random sample Y1, Y2, . . . , Yn drawn from an unknown distribution, with the aim of
testing the null hypothesis that the unknown distribution has some given CDF F0(y). The Kolmogorov statistic
𝐾𝑛 = √𝑛 𝑠𝑢𝑝𝑦∈𝑅 |𝐹𝑛 (𝑦) − 𝐹0 (𝑦)| can be used.
• Under null: √𝑛 𝑠𝑢𝑝𝑦∈𝑅 |𝐹𝑛 (𝑦) − 𝐹0 (𝑦)| 𝑑→ 𝑠𝑢𝑝𝑦∈𝑅 |𝐵(𝐹0 (𝑦))| = 𝑠𝑢𝑝0≤𝑢≤1 |𝐵(𝑢)|
• For Brownian Bridge:
∞
𝑃(𝑠𝑢𝑝0≤𝑢≤1 𝐵(𝑢) > 𝑦) = 𝑒
−2𝑦 2
𝑎𝑛𝑑 𝑃(𝑠𝑢𝑝0≤𝑢≤1 |𝐵(𝑢)| > 𝑦) = 2 ∑(−1)𝑗+1 𝑒 −2𝑗
𝑗=1
9
2𝑦2
• Now, suppose we have drawn the sample. One way to perform the Kolmogorov-Smirnov test is to draw Fn(y)
first. Determine a maximum allowed distance by multiplying critical value kα by
𝐹𝑛 (𝑦) ±
𝑘α
√𝑛
1
.
√𝑛
Draw the two lines
red:
▶ If F0(y) falls completely between the red lines, do not reject the null hypothesis.
▶ If F0(y) exceeds one of the red lines, reject the null hypothesis.
Other CTCS Processes:
Ornstein Uhlenbeck:
• Let {𝑊 (𝑡), 𝑡 ≥ 0} be Brownian motion on the interval [0, ∞), and let α ≥ 0. The stochastic process
𝑎𝑡
{𝑋(𝑡), 𝑡 ≥ 0} defined by 𝑋(𝑡) = 𝑒 − 2 𝑊(𝑒 𝑎𝑡 ) is called the Ornstein-Uhlenbeck process.
• The Ornstein-Uhlenbeck process {𝑋(𝑡), 𝑡 ≥ 0} is a Gaussian process with zero mean function and
covariance function: 𝐶𝑜𝑣(𝑋(𝑡1), 𝑋(𝑡2)) = 𝑒𝑥𝑝{−𝛼|𝑡1 − 𝑡2|/2}.
• Stationarity (a process is stationary if at any given time point the distribution does not depend on the time,
i.e. is the same for every t):
▶ Brownian motion is NOT a stationary process
▶ Ornstein-Uhlenbeck process is a stationary process
• Increment:
▶ Brownian motion a process with independent (and stationary) increments
▶ Ornstein-Uhlenbeck process does not have independent increments, but it has stationary increments
Geometric Brownian Motion:
• Let {𝑊 (𝑡), 𝑡 ≥ 0} be Brownian motion on the interval [0, ∞). The stochastic process {𝑋(𝑡), 𝑡 ≥ 0} defined
by 𝑋(𝑡) = 𝑒 µ𝑡+𝜎𝑊(𝑡) is called the geometric Brownian motion with drift coefficient µ and variance
parameter 𝜎 2 .
• If {𝑋(𝑡), 𝑡 ≥ 0} is a geometric Brownian motion with drift coefficient µ and variance parameter 𝜎 2 , then
µ
{𝜎 −1 𝑙𝑛 𝑋(𝑡), 𝑡 ≥ 0} is a Brownian motion with drift coefficient 𝜎.
• Note that the geometric Brownian motion is not a Gaussian process!
▶ For fixed t, the random variable has a log-normal distribution with parameters µ𝑡 𝑎𝑛𝑑 𝜎 2
10
Tables:
Absolute supremum (Butler Test):
Kolgomorov-Smirnov test:
Standard Normal:
11
12