Stochastic Processes
Chapter 1 Introduction to Probability Theory
0. Reading:
•
Ross Chap 1 (main reading)
•
Rosenthal Chap 1 (a discussion on why we need measure theory, and the existence
of a nonmeasurable set)
•
Shreve §1.1 (a deeper discussion on σ-algebra and countable additivity)
1. Formal definition of probability space (Ω, F, P)
•
Definition:
The set of all possible outcomes of an experiment is called the sample space and
is denoted by Ω. An outcome (sample point) is denoted by ω ∈ Ω.
•
Definition:
A collection F of subsets of Ω is called a σ-filed (σ-algebra, filtration) if it satisfies
the following conditions:
(a) ∅ ∈ F;
(b) if A1 , A2 , · · · ∈ F, then
S∞
i=1
Ai ∈ F;
(c) if A ∈ F, then Ac ∈ F.
•
Definition:
A probability measure P on (Ω, F) is a function P : F → [0, 1] satisfying
(a) 0 ≤ P(A) ≤ 1;
(b) P(Ω) = 1;
(c) if A1 , A2 , · · · is a collection of disjoint (mutually exclusive) members of F, in
that Ai ∩ Aj = ∅ for all pairs i, j satisfying i 6= j, then
P(
∞
[
Ai ) =
i=1
•
∞
X
P(Ai ).
i=1
Discussion:
(a) Why not “finitely additive” but “countably additive”?
(b) What’s wrong with “uncountably additive”?
(c) What is a “measure”? What is an event? What does “measurable” mean?
1
(d) How do you define a “random variable”? (usually denoted as X(ω))
2. Unions and intersections
•
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
if A and B are mutually exclusive, this becomes P(A ∪ B) = P(A) + P(B)
•
P(A ∩ B) = P(A)P(B|A)
if A and B are independent, this becomes P(A ∩ B) = P(A)P(B)
•
conditional probability: P(B|A) =
P(A∩B)
P(A)
if A and B are independent, then P(B|A) = P(B)
•
3-event version:
P(A ∪B ∪ C) = P(A)+P(B)+P(C)−P(A∩B)−P(B∩C)−P(A∩C)+P(A∩B∩C)
P(A ∩ B ∩ C) = P(A)P(B|A) P(C|A ∩ B)
|
{z
}
P(A∩B)
•
general version:
P(A1 ∪ A2 ∪ · · · ∪ An ) =⇒ inclusion-exclusion identity, see p.6, (1.4)
P
if mutually exclusive, then P(A1 ∪ A2 ∪ · · · ∪ An ) = ni=1 P(Ai ) (axiom 3)
P(A1 ∩ A2 ∩ · · · ∩ An ) =⇒ see p.17, Exercise 23
Q
if independent, then P(A1 ∩ A2 ∩ · · · ∩ An ) = ni=1 P(Ai ) (see p.11, first formula)
3. Bayes’ Formula
S
• Partition Ω = ni=1 Bi (B1 , · · · , Bn are mutually exclusive events)
S
• A = ni=1 (A ∩ Bi )
P
P
• P(A) = ni=1 P(A ∩ Bi ) = ni=1 P(Bi )P(A|Bi )
•
P(Bj |A) =
P(A ∩ Bj )
P(Bi )P(A|Bj )
= Pn
P(A)
i=1 P(Bi )P(A|Bi )
2