AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 1 (10.17). Let X0 , X1 , X2 , . . . be independent random variables with P (Xn = 1) =
P (Xn = −1) = 21 for all n. Let Zn = Πni=0 Xi . Show that Z1 , Z2 , Z3 , . . . are independent.
1
Solution. P (Z0 = i) = P (X0 = i) = 12 for i ∈ {−1, 1}. Let for P (Zk = ik , k = 0, . . . , n) = 2n+1
for an arbitrary n = 0, 1, . . . and an arbitrary sequence i0 , . . . , in with ik ∈ {−1, 1}, k = 0, . . . , n.
1
Then P (Zk = ik , k = 0, . . . , n + 1) = P (Zk = ik , k = 0, . . . , n) ∗ 21 = 2n+2
for an arbitrary sequence
i0 , . . . , in+1 with ik ∈ {−1, 1}, k = 0, . . . , n + 1. The inductive argument and Theorem 10.4 imply
that Z0 , Z1 , Z2 , . . . are independent.
1
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 2 (11.11). Let X be positive with a density f . Let Y =
Solution.
Let
(
1/(x + 1),
g(x) :=
0,
1
X+1 .
Find the density for Y .
if x > 0,
if x ≤ 0.
Then Y = g(X). Since X is positive, it follows that FY (y) = 1 if y ≥ 1 and FY (y) = 0 if y ≤ 0.
Also, since g −1 (y) = (1/y) − 1, for y ∈ (0, 1)
FY (y) = P (Y ≤ y) = P (X ≥ (1/y) − 1) = 1 − FX ((1/y) − 1).
So, since (g −1 )0 (y) = −1/y 2 , for y ∈ (0, 1)
fY (y) = −fX ((1/y) − 1) · (−1/y 2 ) = f ((1/y) − 1)/y 2 .
Hence
(
f ((1/y) − 1)/y 2 ,
fY (y) =
0,
2
if 0 < y < 1,
otherwise.
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 3 (14.11).
β = 1:
Let X have the double exponential (or Laplace) distribution with α = 0,
1
fX (x) = e−|x| ,
2
−∞ < x < ∞
1
.
1+u2
Show that ϕX (u) =
Note that for u ∈ R,
Z ∞
Z
Z
1 ∞ iux x
1 ∞ iux −x
iux 1 −|x|
e · e
ϕX (u) =
e e 1[0,∞) (x)dx +
e e 1(−∞,0] (x)dx.
dx =
2
2 −∞
2 −∞
−∞
Solution.
(1)
Consider a r.v. Y that is exponential with parameter 1. Then fY (x) = e−x 1[0,∞) (x) for x ∈ R, so
Z
∞
eiux e−x 1[0,∞) dx = ϕY (u) =
−∞
1
,
1 − iu
u ∈ R.
(2)
In addition, f−Y (x) = fY (−x) = ex 1(−∞,0] (x), so
Z
∞
eiux ex 1(−∞,0] dx = ϕ−Y (u) = ϕY (−u) =
−∞
1
,
1 + iu
u ∈ R.
Combining (1), (2), and (3), we conclude that
1
1
1
1 1 + iu + 1 − iu
1
ϕX (u) =
+
= ·
=
.
2 1 − iu 1 + iu
2 (1 − iu)(1 + iu)
1 + u2
3
(3)
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 4 (15.11). Let X, Y be i.i.d. Suppose further that X + Y and X − Y are independent.
Show that ϕX (2u) = (ϕX (u))3 ϕX (−u).
Solution.
Suppose X and Y are Rn -valued. Since X + Y and X − Y are independent,
ϕX (2u) = ϕ2X (u) = ϕ(X+Y )+(X−Y ) (u) = ϕX+Y (u)ϕX−Y (u),
u ∈ Rn .
But since X and Y are i.i.d., the Tonelli-Fubini Theorem implies that, for u ∈ R,
Z Z
ϕX+Y (u) =
eihu,x+yi P X (dx)P X (dy) = ϕX (u)ϕX (u)
and
Z Z
ϕX−Y (u) =
eihu,x−yi P X (dx)P X (dy) = ϕX (u)ϕX (−u).
Combining (4), (5), and (6), we conclude that ϕX (2u) = (ϕX (u))3 ϕX (−u) for u ∈ R.
4
(4)
(5)
(6)
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 5 (16.3). Let X be N (0, 1) and let Z be independent of X with P (Z = 1) = P (Z =
−1) = 21 . Let Y = ZX. Show that L(Y ) = N (0, 1), but that (X, Y ) is not Gaussian (i.e., not
Multivariate normal).
Solution.
Since X and Z are independent, for y ∈ R
P (Y ≤ y) = P (Y ≤ y, Z = 1) + P (Y ≤ y, Z = −1)
1
= P (X ≤ y)P (Z = 1) + P (−X ≤ y)P (Z = −1) = [FX (y) + 1 − FX (−y)].
2
(7)
Since L(X) = N (0, 1), 1 − FX (−y) = FX (y) for y ∈ R, so (7) implies that P (Y ≤ y) = FX (y).
Therefore L(Y ) = N (0, 1).
To show that (X, Y ) is not Gaussian, note that P (X + Y = 0) = P (Z = 1) = 1/2. Since the
measure of a singleton according to any Normal distribution is zero, it follows that X + Y is not
Normally distributed. Since (X, Y ) is Gaussian if and only if every linear combination of X and Y
is Normally distributed, we conclude that (X, Y ) is not Gaussian.
5
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
2 <∞
Problem 6 (17.9). Let (Xn )n≥1 have finite variances and zero means (i.e., Var(Xn ) = σX
n
2
2
and E{Xn } = 0 for all n). Suppose limn→∞ σXn = 0. Show Xn converges to 0 in L and in
probability.
Solution.
Since E{Xn } = 0 for all n,
2
lim E{(Xn − 0)2 } = lim σX
= 0,
n
n→∞
n→∞
L2
which means Xn −→ 0. Since convergence in Lp implies convergence in probability, it follows that
P
→ 0.
Xn −
6
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 7 (18.12). Let µα denote the Geometric distribution of parameter α. Let αn → α > 0,
and show that µαn tends weakly to µα .
Solution.
Since Geometric random variables have a countable state space, and
lim µαn ({k}) = lim (1 − αn )k−1 αn = (1 − α)k−1 α = µα ({k})
n→∞
n→∞
D
→ µα .
it follows that µαn −
7
for k = 1, 2, . . . ,
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 8 (18.19). Let f be uniformly continuous and X, Y two R-valued random variables.
Suppose that |f (x) − f (y)| < whenever |x − y| < δ. Show that
|E{f (X)} − E{f (X + Y )}| ≤ + 2 sup |f (x)|P {|Y | ≥ δ}.
x
Solution.
|E{f (X)} − E{f (X + Y )}| = |E{f (X) − f (X + Y )}|
≤ E{|f (X) − f (X + Y )|}
= E{|f (X) − f (X + Y )|1|Y |<δ } + E{|f (X) − f (X + Y )|1|Y |≥δ }
≤ + 2 sup |f (x)|E{1|Y |≥δ }
x
= + 2 sup |f (x)|P (|Y | ≥ δ).
x
8
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 9 (20.5). Let (Xj )j≥1 be i.i.d. with Xj in L1 and suppose
in distribution to a random variable Z. Show that
√1
n
Pn
j=1 (Xj
− ν) converges
n
1X
lim
Xj = ν
n→∞ n
a.s.
j=1
√
We first show that Zn / n converges in distribution to the r.v. that is identically equal
√ D
D
→ Z, it follows that ϕZn converges to ϕZ
to zero, which is written here as Zn / n −
→ 0. Since Zn −
uniformly on compact subsets of R. In particular, fixing m > 0, ϕZn converges uniformly to ϕZ on
[−m, m]. For an arbitrary > 0, this means that
Solution.
sup
|ϕZn (x) − ϕZ (x)| < /2
for all sufficiently large n.
(8)
x∈[−m,m]
Now consider any u ∈ R, and note that
√
√
u/ n ∈ [−m, m] and |ϕZ (u/ n) − ϕZ (0)| < /2
for all sufficiently large n.
(9)
√
(the latter is possible because ϕZ is continuous). Since ϕZn (u/ n) = ϕZn /√n (u) and ϕZ (0) = 1,
(8) and (9) imply that for all sufficiently large n,
|ϕZn /√n (u) − 1| <
+ = .
2 2
Since u ∈ R was arbitrary, it follows that ϕZn /√n (u) → 1 for all u ∈ R. According to Lévy’s
√
Continuity Theorem, this means that the law of Zn / n converges weakly to a probability measure
whose Fourier transform is indentically equal to 1. But since the Fourier transform of the Dirac
mass at the point 0 is identically equal to 1, and the Dirac mass at 0 is the law of the r.v. identically
√ D
equal to zero, it follows from the Uniqueness Theorem for characteristic functions that Zn / n −
→ 0.
√ D
a.s.
1 Pn
We’ll now use the fact that Zn / n −
→ 0 to show that n j=1 Xj −−→ ν; note that for the
√ D
latter to hold, the Xj ’s need to be defined on the same probability space. But Zn / n −
→ 0 implies
√ P
√
√
→ 0, which means that there is a subsequence (Znk / nk )k≥1 of (Zn / n)n≥1 that
that Zn / n −
converges almost surely to 0. But according to Kolmogorov’s Strong Law of Large Numbers,
n
1X
Z
a.s.
√n =
Xj − ν −−→ E{X1 } − ν.
n
n
j=1
√
This means that (Znk / nk )k≥1 must also converge almost surely to E{X1 } − ν, which implies that
E{X1 } = ν. So, by Kolmogorov’s Strong Law of Large Numbers,
n
1X
lim
Xj = E{X1 } = ν
n→∞ n
j=1
9
almost surely.
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 10 (20.8). Let (Xj )j≥1 be i.i.d. with mean µ and variance σ 2 . Show that
n
1X
(Xi − µ)2 = σ 2
n→∞ n
lim
a.s.
i=1
Solution. For each i, E{(Xi − µ)2 } = σ 2 < ∞, which implies that (Xi − µ)2 ∈ L1 . The result
then follows from Kolmogorov’s Strong Law of Large Numbers.
10
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem
i.i.d Poisson random variables with parameter λ = 1. Let
P 11 (21.5). Let (Xj )j≥1 nbe
−n
Sn = nj=1 Xj . Show that limn→∞ S√
= Z, where L(Z) = N (0, 1).
n
Solution.
Since µ = σ 2 = 1 < ∞, the result follows from the Central Limit Theorem.
11
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 12 (21.8).
Show that
S√n
σ n
2 = σ 2 < ∞. Let S =
Let (Xj )j≥1 be i.i.d with E{Xj } = 0 and σX
n
j
Pn
i=1 Xi .
does not converge in probability.
Solution. By the Central Limit Theorem, the sequence σS√nn converges in distribution to Z ∼
N (0, 1). Since convergence in probability implies convergence in distribution, it is sufficient to show
that the sequence does not converges to Z in probability.
(i)
(i)
Observe that, if there are two sequences of random variables {Zn }n=1,2,... , i = 1, 2, and Zn
(1)
(2)
converge in probability to a random variable Z (i) , then Zn + Zn converge in probability to
Z (1) + Z (2) . Indeed, for each > 0
P (|(Xn(1) + Xn(2) ) − (X (1) + X (2) ))| ≥ ) ≤ P (|Xn(1) − X (1) | + |Xn(2) − X (2) | ≥ )
≤ max{P (|Xn(1) − X (1) | ≥ ), P (|Xn(2) − X (2) | ≥ )} → 0.
2
2
Let σS√nn converge to Z in probability. This implies that σS√2n2n converge in probability to Z.
Therefore
S2n
Sn
S2n − Sn
Sn
1
√
Yn := √ − √ =
+ √ ( √ − 1)
σ n 2
σ 2n σ n
σ 2n
converge to 0 in probability.
−Sn
√
Observe that S2n
and σS√nn are iid and, as follows from the Central Limit Theorem, each
σ n
of these sequences converges in distribution to a standard normal random variable. Therefore Yn
converge in distribution to a difference of two independent normal random variables, which is a
normal random variable (it can be easily computed, but this step is not needed). This contradicts
the convergence of Yn in probability to 0.
12
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 13 (22.4). Let L be a linear subspace of H and Π projection onto L. Show that Π y
is the unique element of L such that hΠ y, zi = hy, zi for all z ∈ L.
Solution. By definition, Π y ∈ L. Also, y − Π y is orthogonal to L, so hy − Π y, zi = 0 for all
z ∈ L. Since the inner product is linear in each component, this means hΠ y, zi = hy, zi for all
z ∈ L. On the other hand, if x ∈ L satisfies hx, zi = hy, zi for all z ∈ L, then hy − x, zi = 0 for
all z ∈ L, i.e. y − x is orthogonal to L. Hence y = x + (y − x) is a representation of y as the sum
of a vector in L and one in L⊥ . Since Π y is the unique element of L such that y − Π y ∈ L⊥ and
y = Π y + (y − Π y), this implies that x = Π y.
13
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 14 (23.6). If Y is positive, show that {E{Y |G} = 0} ⊂ {Y = 0} and {Y = +∞} ⊂
{E{Y |G} = +∞} almost surely.
Solution. Suppose E{Y |G} = 0. Then E{Y } = E{E{Y |G} = 0 which, since Y is positive,
implies that Y = 0 almost surely.
∞
On the other hand, note that {E{Y |G} = +∞}c = ∪∞
m=0 {E{Y |G} ≤ m} ⊂ ∪m=0 {Y ≤ m} =
c
{Y = +∞} , so {E{Y |G} = +∞} ⊃ {Y = +∞}.
14
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 15 (23.12). Let X ∈ L2 . Show that
E{(X − E{X|G})2 } ≤ E{(X − E{X})2 }.
Solution.
Since E{X|G} is the projection of X ∈ L2 on the subspace L2 (Ω, G, P ),
E{(X − E{X|G})2 }1/2 = kX − E{X|G}k = d(X, L2 (Ω, G, P )).
Since E{X} ∈ L2 (Ω, G, P ), it follows from (10) that
E{(X − E{X|G})2 }1/2 ≤ kX − E{X}k = E{(X − E{X})2 }1/2 ,
so E{(X − E{X|G})2 } ≤ E{(X − E{X})2 }.
15
(10)
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 16 (24.13). Let ϕ be convex and let M = (Mn )n≥0 be a martingale. Show that
n → E{ϕ(Mn )} is a nondecreasing function.
Solution. Consider any two nonnegative integers m, n where m ≤ n. Since (Mn )n≥0 is a martingale, Xm = E{Xn |Fm }, so Jensen’s inequality implies that
ϕ(Xm ) = ϕ(E{Xn |Fm }) ≤ E{ϕ(Xn )|Fm }.
Taking expectations on both sides of (11), we conclude that
E{ϕ(Xm )} ≤ E{E{ϕ(Xn )|Fm }} = E{ϕ(Xn )}.
16
(11)
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 17 (25.6). Let X = (Xn )n≥0 be a submartingale. Show that if ϕ is convex and
nondecreasing in R and if ϕ(Xn ) is integrable for each n , then Yn = ϕ(Xn ) is also a submatringale.
Solution. For each n, the integrability of ϕ(Xn ) means that E{|ϕ(Xn )|} < ∞. Also, since ϕ is
Borel-measurable (because for each λ ∈ R the set {x ∈ R : ϕ(x) > λ} is an interval and therefore
a Borel set), and Xn is Fn -measurable for each n, it follows that ϕ(Xn ) is also Fn -measurable
for each n. Finally, fix 0 ≤ m ≤ n. Since (Xn )n≥0 is a submartingale, E{Xn |Fm } ≥ Xm almost
surely; because ϕ is nondecreasing, this means ϕ(E{Xn |Fm }) ≥ ϕ(Xm ) almost surely. According
to Jensen’s inequality, this means
E{ϕ(Xn )|Fm } ≥ ϕ(E{Xn |Fm }) ≥ ϕ(Xm ).
Hence (ϕ(Xn ))n≥0 is a submartingale.
17
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 18 (26.1). Let Yn ∈ L2 and suppose limn→∞ E{Yn2 } = 0. Let (Fk )k≥0 be an increasing
sequence of σ-algebras and let Xkn = E{Yn |Fk }. Show that limn→∞ E{supk (Xkn )2 } = 0.
Solution. Fix n, and for k = 0, 1, . . . let Zkn := E{Yn2 |Fk } and Z̄kn := supj≤k Zjn . Note that
(Zkn )k≥0 is a martingale with respect to (Fk )k≥0 , and that Z̄kn ↑ supk Zkn as k → ∞. Also, by
Jensen’s Inequality
(Xkn )2 = (E{Yn |Fk })2 ≤ E{Yn2 |Fk } = Zkn
for all k ≥ 0.
(12)
According to Jensen’s Inequality and Doob’s Lp Martingale Inequality, there is a c ∈ R satisfying
(E{Z̄kn })2 ≤ E{(Z̄kn })2 } ≤ cE{Zkn } = cE{Yn2 },
which implies
E{Z̄kn } ≤
p
cE{Yn2 } for all k ≥ 0.
(13)
But since the Z̄kn ’s are positive and Z̄kn ↑ supk Zkn , the Monotone Convergence Theorem and (13)
imply that
p
E{supk Zkn } ≤ cE{Yn2 }.
(14)
Combining (14) with (12), it follows that
0 ≤ E{supk (Xkn )2 } ≤ E{supk Zkn } ≤
p
cE{Yn2 }
which, since limn→∞ E{Yn2 } = 0, implies that limn→∞ E{supk (Xkn )2 } = 0.
18
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 19 (27.5). Let (Xn )n≥1 be P
i.i.d. with P (Xn = 1) = P (Xn =P
−1) = 21 . Let (αn )n≥1 be
∞
2
a sequence of real numbers. Show that n=1 αn Xn is a.s. convergent if ∞
n=1 αn < ∞.
P
Solution. For m ≥ 1, let Mm := m
n=1 αn Xn and Fm := σ{Xk : k ≤ m}. We now show that
(Mm )m≥0 is martingale with respect to (Fm )m≥1 . First, since E{|Xn |} = 1 for all n, it follows that
for all m
( m
)
m
m
X
X
X
E{|Mm |} = E
αn Xn ≤
|αn |E{|Xn |} =
|αn | < ∞.
(15)
n=1
n=1
n=1
Also, for each m the definition of Fm implies that Mm is Fm -measurable. Finally, if k ≤ `, then
( `
)
X
E{M` |Fk } = E{Mk + (M` − Mk )|Fk } = E{Mk |Fk } + E
αn Xn Fk
n=k+1
= Mk +
`
X
αn E{Xn |Fk }
(16)
αn E{Xn }
(17)
α n · 0 = Mk ,
(18)
n=k+1
= Mk +
`
X
n=k+1
= Mk +
`
X
n=k+1
where (16) holds because Mk is Fk -measurable, (17) holds because the Xn ’s are independent, and
(18) holds because E{Xn } = 0 for all n. Therefore (Mm )m≥1 is a martingale with respect to
(Fm )m≥1 .
Next, note that (Mm )m≥1 is uniformly integrable, because for each m the fact that the Xn ’s
are iid with E{X12 } = 1 implies that
2
E{Mm
}=
X
αk αm E{Xk X` } =
m
X
αn2 E{Xn2 } =
n=1
1≤k,`≤m
m
X
αn2 ≤
n=1
Therefore the Martingale Convergence Theorem implies that
∞
X
n=1
αn Xn = lim Mm
m→∞
19
exists almost surely.
∞
X
n=1
αn2 < ∞.
AMS 569: Probability Theory I | Spring 2022
Sample Final Exam Solutions
Problem 20 (28.4). Let P and Q be two probabilities and let R =
P +Q
2 .
Show that P R.
Solution. Suppose ∧ ∈ F satisfies R(∧) = 0. Then [P (∧) + Q(∧)]/2 = 0; since P (∧) and Q(∧)
are both nonnegative, this implies P (∧) = 0.
20