AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 1 (10.17). Let X0 , X1 , X2 , . . . be independent random variables with P (Xn = 1) = P (Xn = −1) = 21 for all n. Let Zn = Πni=0 Xi . Show that Z1 , Z2 , Z3 , . . . are independent. 1 Solution. P (Z0 = i) = P (X0 = i) = 12 for i ∈ {−1, 1}. Let for P (Zk = ik , k = 0, . . . , n) = 2n+1 for an arbitrary n = 0, 1, . . . and an arbitrary sequence i0 , . . . , in with ik ∈ {−1, 1}, k = 0, . . . , n. 1 Then P (Zk = ik , k = 0, . . . , n + 1) = P (Zk = ik , k = 0, . . . , n) ∗ 21 = 2n+2 for an arbitrary sequence i0 , . . . , in+1 with ik ∈ {−1, 1}, k = 0, . . . , n + 1. The inductive argument and Theorem 10.4 imply that Z0 , Z1 , Z2 , . . . are independent. 1 AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 2 (11.11). Let X be positive with a density f . Let Y = Solution. Let ( 1/(x + 1), g(x) := 0, 1 X+1 . Find the density for Y . if x > 0, if x ≤ 0. Then Y = g(X). Since X is positive, it follows that FY (y) = 1 if y ≥ 1 and FY (y) = 0 if y ≤ 0. Also, since g −1 (y) = (1/y) − 1, for y ∈ (0, 1) FY (y) = P (Y ≤ y) = P (X ≥ (1/y) − 1) = 1 − FX ((1/y) − 1). So, since (g −1 )0 (y) = −1/y 2 , for y ∈ (0, 1) fY (y) = −fX ((1/y) − 1) · (−1/y 2 ) = f ((1/y) − 1)/y 2 . Hence ( f ((1/y) − 1)/y 2 , fY (y) = 0, 2 if 0 < y < 1, otherwise. AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 3 (14.11). β = 1: Let X have the double exponential (or Laplace) distribution with α = 0, 1 fX (x) = e−|x| , 2 −∞ < x < ∞ 1 . 1+u2 Show that ϕX (u) = Note that for u ∈ R, Z ∞ Z Z 1 ∞ iux x 1 ∞ iux −x iux 1 −|x| e · e ϕX (u) = e e 1[0,∞) (x)dx + e e 1(−∞,0] (x)dx. dx = 2 2 −∞ 2 −∞ −∞ Solution. (1) Consider a r.v. Y that is exponential with parameter 1. Then fY (x) = e−x 1[0,∞) (x) for x ∈ R, so Z ∞ eiux e−x 1[0,∞) dx = ϕY (u) = −∞ 1 , 1 − iu u ∈ R. (2) In addition, f−Y (x) = fY (−x) = ex 1(−∞,0] (x), so Z ∞ eiux ex 1(−∞,0] dx = ϕ−Y (u) = ϕY (−u) = −∞ 1 , 1 + iu u ∈ R. Combining (1), (2), and (3), we conclude that 1 1 1 1 1 + iu + 1 − iu 1 ϕX (u) = + = · = . 2 1 − iu 1 + iu 2 (1 − iu)(1 + iu) 1 + u2 3 (3) AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 4 (15.11). Let X, Y be i.i.d. Suppose further that X + Y and X − Y are independent. Show that ϕX (2u) = (ϕX (u))3 ϕX (−u). Solution. Suppose X and Y are Rn -valued. Since X + Y and X − Y are independent, ϕX (2u) = ϕ2X (u) = ϕ(X+Y )+(X−Y ) (u) = ϕX+Y (u)ϕX−Y (u), u ∈ Rn . But since X and Y are i.i.d., the Tonelli-Fubini Theorem implies that, for u ∈ R, Z Z ϕX+Y (u) = eihu,x+yi P X (dx)P X (dy) = ϕX (u)ϕX (u) and Z Z ϕX−Y (u) = eihu,x−yi P X (dx)P X (dy) = ϕX (u)ϕX (−u). Combining (4), (5), and (6), we conclude that ϕX (2u) = (ϕX (u))3 ϕX (−u) for u ∈ R. 4 (4) (5) (6) AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 5 (16.3). Let X be N (0, 1) and let Z be independent of X with P (Z = 1) = P (Z = −1) = 21 . Let Y = ZX. Show that L(Y ) = N (0, 1), but that (X, Y ) is not Gaussian (i.e., not Multivariate normal). Solution. Since X and Z are independent, for y ∈ R P (Y ≤ y) = P (Y ≤ y, Z = 1) + P (Y ≤ y, Z = −1) 1 = P (X ≤ y)P (Z = 1) + P (−X ≤ y)P (Z = −1) = [FX (y) + 1 − FX (−y)]. 2 (7) Since L(X) = N (0, 1), 1 − FX (−y) = FX (y) for y ∈ R, so (7) implies that P (Y ≤ y) = FX (y). Therefore L(Y ) = N (0, 1). To show that (X, Y ) is not Gaussian, note that P (X + Y = 0) = P (Z = 1) = 1/2. Since the measure of a singleton according to any Normal distribution is zero, it follows that X + Y is not Normally distributed. Since (X, Y ) is Gaussian if and only if every linear combination of X and Y is Normally distributed, we conclude that (X, Y ) is not Gaussian. 5 AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions 2 <∞ Problem 6 (17.9). Let (Xn )n≥1 have finite variances and zero means (i.e., Var(Xn ) = σX n 2 2 and E{Xn } = 0 for all n). Suppose limn→∞ σXn = 0. Show Xn converges to 0 in L and in probability. Solution. Since E{Xn } = 0 for all n, 2 lim E{(Xn − 0)2 } = lim σX = 0, n n→∞ n→∞ L2 which means Xn −→ 0. Since convergence in Lp implies convergence in probability, it follows that P → 0. Xn − 6 AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 7 (18.12). Let µα denote the Geometric distribution of parameter α. Let αn → α > 0, and show that µαn tends weakly to µα . Solution. Since Geometric random variables have a countable state space, and lim µαn ({k}) = lim (1 − αn )k−1 αn = (1 − α)k−1 α = µα ({k}) n→∞ n→∞ D → µα . it follows that µαn − 7 for k = 1, 2, . . . , AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 8 (18.19). Let f be uniformly continuous and X, Y two R-valued random variables. Suppose that |f (x) − f (y)| < whenever |x − y| < δ. Show that |E{f (X)} − E{f (X + Y )}| ≤ + 2 sup |f (x)|P {|Y | ≥ δ}. x Solution. |E{f (X)} − E{f (X + Y )}| = |E{f (X) − f (X + Y )}| ≤ E{|f (X) − f (X + Y )|} = E{|f (X) − f (X + Y )|1|Y |<δ } + E{|f (X) − f (X + Y )|1|Y |≥δ } ≤ + 2 sup |f (x)|E{1|Y |≥δ } x = + 2 sup |f (x)|P (|Y | ≥ δ). x 8 AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 9 (20.5). Let (Xj )j≥1 be i.i.d. with Xj in L1 and suppose in distribution to a random variable Z. Show that √1 n Pn j=1 (Xj − ν) converges n 1X lim Xj = ν n→∞ n a.s. j=1 √ We first show that Zn / n converges in distribution to the r.v. that is identically equal √ D D → Z, it follows that ϕZn converges to ϕZ to zero, which is written here as Zn / n − → 0. Since Zn − uniformly on compact subsets of R. In particular, fixing m > 0, ϕZn converges uniformly to ϕZ on [−m, m]. For an arbitrary > 0, this means that Solution. sup |ϕZn (x) − ϕZ (x)| < /2 for all sufficiently large n. (8) x∈[−m,m] Now consider any u ∈ R, and note that √ √ u/ n ∈ [−m, m] and |ϕZ (u/ n) − ϕZ (0)| < /2 for all sufficiently large n. (9) √ (the latter is possible because ϕZ is continuous). Since ϕZn (u/ n) = ϕZn /√n (u) and ϕZ (0) = 1, (8) and (9) imply that for all sufficiently large n, |ϕZn /√n (u) − 1| < + = . 2 2 Since u ∈ R was arbitrary, it follows that ϕZn /√n (u) → 1 for all u ∈ R. According to Lévy’s √ Continuity Theorem, this means that the law of Zn / n converges weakly to a probability measure whose Fourier transform is indentically equal to 1. But since the Fourier transform of the Dirac mass at the point 0 is identically equal to 1, and the Dirac mass at 0 is the law of the r.v. identically √ D equal to zero, it follows from the Uniqueness Theorem for characteristic functions that Zn / n − → 0. √ D a.s. 1 Pn We’ll now use the fact that Zn / n − → 0 to show that n j=1 Xj −−→ ν; note that for the √ D latter to hold, the Xj ’s need to be defined on the same probability space. But Zn / n − → 0 implies √ P √ √ → 0, which means that there is a subsequence (Znk / nk )k≥1 of (Zn / n)n≥1 that that Zn / n − converges almost surely to 0. But according to Kolmogorov’s Strong Law of Large Numbers, n 1X Z a.s. √n = Xj − ν −−→ E{X1 } − ν. n n j=1 √ This means that (Znk / nk )k≥1 must also converge almost surely to E{X1 } − ν, which implies that E{X1 } = ν. So, by Kolmogorov’s Strong Law of Large Numbers, n 1X lim Xj = E{X1 } = ν n→∞ n j=1 9 almost surely. AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 10 (20.8). Let (Xj )j≥1 be i.i.d. with mean µ and variance σ 2 . Show that n 1X (Xi − µ)2 = σ 2 n→∞ n lim a.s. i=1 Solution. For each i, E{(Xi − µ)2 } = σ 2 < ∞, which implies that (Xi − µ)2 ∈ L1 . The result then follows from Kolmogorov’s Strong Law of Large Numbers. 10 AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem i.i.d Poisson random variables with parameter λ = 1. Let P 11 (21.5). Let (Xj )j≥1 nbe −n Sn = nj=1 Xj . Show that limn→∞ S√ = Z, where L(Z) = N (0, 1). n Solution. Since µ = σ 2 = 1 < ∞, the result follows from the Central Limit Theorem. 11 AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 12 (21.8). Show that S√n σ n 2 = σ 2 < ∞. Let S = Let (Xj )j≥1 be i.i.d with E{Xj } = 0 and σX n j Pn i=1 Xi . does not converge in probability. Solution. By the Central Limit Theorem, the sequence σS√nn converges in distribution to Z ∼ N (0, 1). Since convergence in probability implies convergence in distribution, it is sufficient to show that the sequence does not converges to Z in probability. (i) (i) Observe that, if there are two sequences of random variables {Zn }n=1,2,... , i = 1, 2, and Zn (1) (2) converge in probability to a random variable Z (i) , then Zn + Zn converge in probability to Z (1) + Z (2) . Indeed, for each > 0 P (|(Xn(1) + Xn(2) ) − (X (1) + X (2) ))| ≥ ) ≤ P (|Xn(1) − X (1) | + |Xn(2) − X (2) | ≥ ) ≤ max{P (|Xn(1) − X (1) | ≥ ), P (|Xn(2) − X (2) | ≥ )} → 0. 2 2 Let σS√nn converge to Z in probability. This implies that σS√2n2n converge in probability to Z. Therefore S2n Sn S2n − Sn Sn 1 √ Yn := √ − √ = + √ ( √ − 1) σ n 2 σ 2n σ n σ 2n converge to 0 in probability. −Sn √ Observe that S2n and σS√nn are iid and, as follows from the Central Limit Theorem, each σ n of these sequences converges in distribution to a standard normal random variable. Therefore Yn converge in distribution to a difference of two independent normal random variables, which is a normal random variable (it can be easily computed, but this step is not needed). This contradicts the convergence of Yn in probability to 0. 12 AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 13 (22.4). Let L be a linear subspace of H and Π projection onto L. Show that Π y is the unique element of L such that hΠ y, zi = hy, zi for all z ∈ L. Solution. By definition, Π y ∈ L. Also, y − Π y is orthogonal to L, so hy − Π y, zi = 0 for all z ∈ L. Since the inner product is linear in each component, this means hΠ y, zi = hy, zi for all z ∈ L. On the other hand, if x ∈ L satisfies hx, zi = hy, zi for all z ∈ L, then hy − x, zi = 0 for all z ∈ L, i.e. y − x is orthogonal to L. Hence y = x + (y − x) is a representation of y as the sum of a vector in L and one in L⊥ . Since Π y is the unique element of L such that y − Π y ∈ L⊥ and y = Π y + (y − Π y), this implies that x = Π y. 13 AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 14 (23.6). If Y is positive, show that {E{Y |G} = 0} ⊂ {Y = 0} and {Y = +∞} ⊂ {E{Y |G} = +∞} almost surely. Solution. Suppose E{Y |G} = 0. Then E{Y } = E{E{Y |G} = 0 which, since Y is positive, implies that Y = 0 almost surely. ∞ On the other hand, note that {E{Y |G} = +∞}c = ∪∞ m=0 {E{Y |G} ≤ m} ⊂ ∪m=0 {Y ≤ m} = c {Y = +∞} , so {E{Y |G} = +∞} ⊃ {Y = +∞}. 14 AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 15 (23.12). Let X ∈ L2 . Show that E{(X − E{X|G})2 } ≤ E{(X − E{X})2 }. Solution. Since E{X|G} is the projection of X ∈ L2 on the subspace L2 (Ω, G, P ), E{(X − E{X|G})2 }1/2 = kX − E{X|G}k = d(X, L2 (Ω, G, P )). Since E{X} ∈ L2 (Ω, G, P ), it follows from (10) that E{(X − E{X|G})2 }1/2 ≤ kX − E{X}k = E{(X − E{X})2 }1/2 , so E{(X − E{X|G})2 } ≤ E{(X − E{X})2 }. 15 (10) AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 16 (24.13). Let ϕ be convex and let M = (Mn )n≥0 be a martingale. Show that n → E{ϕ(Mn )} is a nondecreasing function. Solution. Consider any two nonnegative integers m, n where m ≤ n. Since (Mn )n≥0 is a martingale, Xm = E{Xn |Fm }, so Jensen’s inequality implies that ϕ(Xm ) = ϕ(E{Xn |Fm }) ≤ E{ϕ(Xn )|Fm }. Taking expectations on both sides of (11), we conclude that E{ϕ(Xm )} ≤ E{E{ϕ(Xn )|Fm }} = E{ϕ(Xn )}. 16 (11) AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 17 (25.6). Let X = (Xn )n≥0 be a submartingale. Show that if ϕ is convex and nondecreasing in R and if ϕ(Xn ) is integrable for each n , then Yn = ϕ(Xn ) is also a submatringale. Solution. For each n, the integrability of ϕ(Xn ) means that E{|ϕ(Xn )|} < ∞. Also, since ϕ is Borel-measurable (because for each λ ∈ R the set {x ∈ R : ϕ(x) > λ} is an interval and therefore a Borel set), and Xn is Fn -measurable for each n, it follows that ϕ(Xn ) is also Fn -measurable for each n. Finally, fix 0 ≤ m ≤ n. Since (Xn )n≥0 is a submartingale, E{Xn |Fm } ≥ Xm almost surely; because ϕ is nondecreasing, this means ϕ(E{Xn |Fm }) ≥ ϕ(Xm ) almost surely. According to Jensen’s inequality, this means E{ϕ(Xn )|Fm } ≥ ϕ(E{Xn |Fm }) ≥ ϕ(Xm ). Hence (ϕ(Xn ))n≥0 is a submartingale. 17 AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 18 (26.1). Let Yn ∈ L2 and suppose limn→∞ E{Yn2 } = 0. Let (Fk )k≥0 be an increasing sequence of σ-algebras and let Xkn = E{Yn |Fk }. Show that limn→∞ E{supk (Xkn )2 } = 0. Solution. Fix n, and for k = 0, 1, . . . let Zkn := E{Yn2 |Fk } and Z̄kn := supj≤k Zjn . Note that (Zkn )k≥0 is a martingale with respect to (Fk )k≥0 , and that Z̄kn ↑ supk Zkn as k → ∞. Also, by Jensen’s Inequality (Xkn )2 = (E{Yn |Fk })2 ≤ E{Yn2 |Fk } = Zkn for all k ≥ 0. (12) According to Jensen’s Inequality and Doob’s Lp Martingale Inequality, there is a c ∈ R satisfying (E{Z̄kn })2 ≤ E{(Z̄kn })2 } ≤ cE{Zkn } = cE{Yn2 }, which implies E{Z̄kn } ≤ p cE{Yn2 } for all k ≥ 0. (13) But since the Z̄kn ’s are positive and Z̄kn ↑ supk Zkn , the Monotone Convergence Theorem and (13) imply that p E{supk Zkn } ≤ cE{Yn2 }. (14) Combining (14) with (12), it follows that 0 ≤ E{supk (Xkn )2 } ≤ E{supk Zkn } ≤ p cE{Yn2 } which, since limn→∞ E{Yn2 } = 0, implies that limn→∞ E{supk (Xkn )2 } = 0. 18 AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 19 (27.5). Let (Xn )n≥1 be P i.i.d. with P (Xn = 1) = P (Xn =P −1) = 21 . Let (αn )n≥1 be ∞ 2 a sequence of real numbers. Show that n=1 αn Xn is a.s. convergent if ∞ n=1 αn < ∞. P Solution. For m ≥ 1, let Mm := m n=1 αn Xn and Fm := σ{Xk : k ≤ m}. We now show that (Mm )m≥0 is martingale with respect to (Fm )m≥1 . First, since E{|Xn |} = 1 for all n, it follows that for all m ( m ) m m X X X E{|Mm |} = E αn Xn ≤ |αn |E{|Xn |} = |αn | < ∞. (15) n=1 n=1 n=1 Also, for each m the definition of Fm implies that Mm is Fm -measurable. Finally, if k ≤ `, then ( ` ) X E{M` |Fk } = E{Mk + (M` − Mk )|Fk } = E{Mk |Fk } + E αn Xn Fk n=k+1 = Mk + ` X αn E{Xn |Fk } (16) αn E{Xn } (17) α n · 0 = Mk , (18) n=k+1 = Mk + ` X n=k+1 = Mk + ` X n=k+1 where (16) holds because Mk is Fk -measurable, (17) holds because the Xn ’s are independent, and (18) holds because E{Xn } = 0 for all n. Therefore (Mm )m≥1 is a martingale with respect to (Fm )m≥1 . Next, note that (Mm )m≥1 is uniformly integrable, because for each m the fact that the Xn ’s are iid with E{X12 } = 1 implies that 2 E{Mm }= X αk αm E{Xk X` } = m X αn2 E{Xn2 } = n=1 1≤k,`≤m m X αn2 ≤ n=1 Therefore the Martingale Convergence Theorem implies that ∞ X n=1 αn Xn = lim Mm m→∞ 19 exists almost surely. ∞ X n=1 αn2 < ∞. AMS 569: Probability Theory I | Spring 2022 Sample Final Exam Solutions Problem 20 (28.4). Let P and Q be two probabilities and let R = P +Q 2 . Show that P R. Solution. Suppose ∧ ∈ F satisfies R(∧) = 0. Then [P (∧) + Q(∧)]/2 = 0; since P (∧) and Q(∧) are both nonnegative, this implies P (∧) = 0. 20