Distributions Derived from Normal Random Variables
Distributions Derived From
the Normal Distribution
MIT 18.443
Dr. Kempthorne
Spring 2015
MIT 18.443
Distributions Derived From the Normal Distribution
1
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
Outline
1
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
MIT 18.443
Distributions Derived From the Normal Distribution
2
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
Normal Distribution
Definition. A Normal / Gaussian random variable X ∼ N(µ, σ 2 )
has density function:
1
2
2
f (x) = √ e −(x−µ) /2σ , −∞ < x < +∞.
σ 2π
with mean and variance parameters: J
+∞
µ = E [X ]
= −∞ xf (x)dx
J
+∞
σ 2 = E [(X − µ)2 ] = −∞ (x − µ)2 f (x)dx
Note: −∞ < µ < +∞, and σ 2 > 0.
Properties:
Density function is symmetric about x = µ.
f (µ + x ∗ ) = f (µ − x ∗ ).
f (x) is a maximum at x = µ.
f "" (x) = 0 at x = µ + σ and x = µ − σ
(inflection points of bell curve)
Moment generating function:
2 2
MX (t) = E [e tX ] = e µt+σ t /2
MIT 18.443
Distributions Derived From the Normal Distribution
3
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
Chi-Square Distributions
Definition. If Z ∼ N(0, 1) (Standard Normal r.v.) then
U = Z 2 ∼ χ21 ,
has a Chi-Squared distribution with 1 degree of freedom.
Properties:
The density function of U is:
−1/2
fU (u) = u√2π e −u/2 , 0 < u < ∞
Recall the density of a Gamma(α, λ) distribution:
λα α−1 −λx
g (x) = Γ(α)
x
e
, x > 0,
So U is Gamma(α, λ) with α = 1/2 and λ = 1/2.
Moment generating function
MU (t) = E [e tU ] = [1 − t/λ]−α = (1 − 2t)−1/2
MIT 18.443
Distributions Derived From the Normal Distribution
4
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
Chi-Square Distributions
Definition. If Z1 , Z2 , . . . , Zn are i.i.d. N(0, 1) random variables
V = Z12 + Z22 + . . . Zn2
has a Chi-Squared distribution with n degrees of freedom.
Properties (continued)
The Chi-Square r.v. V can be expressed as:
V = U1 + U2 + · · · + Un
where U1 , . . . , Un are i.i.d χ21 r.v.
Moment generating function
MV (t) = E [e tV ] = E [e
t(U1 +U2 +···+Un ) ]
= E [e tU1 ]
· · · E [e tUn ] = (1 − 2t)−n/2
Because Ui are i.i.d. Gamma(α = 1/2, λ = 1/2) r.v.,s
V ∼ Gamma(α = n/2, λ = 1/2).
1
Density function: f (v ) = n/2
v (n/2)−1 e −v /2 , v > 0.
2 Γ(n/2)
(α is the shape parameter and λ is the scale parameter)
MIT 18.443
Distributions Derived From the Normal Distribution
5
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
Student’s t Distribution
Definition. For independent r.v.’s Z and U where
Z ∼ N(0, 1)
U ∼ χ2r
the distribution of T = Z / U/r is the
t distribution with r degrees of freedom.
Properties
The density function of T is
−(r +1)/2
Γ[(r + 1)/2]
t2
f (t) = √
1+
, −∞ < t < +∞
r
r πΓ(r /2)
For what powers k does E [T k ] converge/diverge?
Does the moment generating function for T exist?
MIT 18.443
Distributions Derived From the Normal Distribution
6
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
F Distribution
Definition. For independent r.v.’s U and V where
U ∼ χ2m
V ∼ χ2n
U/m
is the
V /n
F distribution with m and n degrees of freedom.
(notation F ∼ Fm,n )
Properties
the distribution of F =
The density function of F is
m
Γ[(m + n)/2] m n/2 m/2−1
f (w ) =
w
1+ w
Γ(m/2)Γ(n/2) n
n
with domain w > 0.
E [F ] = E [U/m] × E [n/V ] = 1 × n ×
1
n−2
=
n
n−2
−(m+n)/2
,
(for n > 2).
If T ∼ tr , then T 2 ∼ F1,r .
MIT 18.443
Distributions Derived From the Normal Distribution
7
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
Outline
1
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
MIT 18.443
Distributions Derived From the Normal Distribution
8
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
Statistics from Normal Samples
Sample of size n from a Normal Distribution
X1 , . . . , Xn iid N(µ, σ 2 ).
n
1n
Sample Mean: X =
Xi
n
i=1
n
Sample Variance: S 2 =
1 n
(Xi − X )2
n−1
i=1
Properties of X
The moment generating function of X is
n
t
MX (t) = E [e tX ] = E [e n 1 Xi ]
=
= e
i.e., X ∼
n
1 MXi (t/n)
σ 2 /n
µt+ 2
t 2
=
2
n
µ(t/n)+ σ2 (t/n)2
]
i=1 [e
N(µ, σ 2 /n).
MIT 18.443
Distributions Derived From the Normal Distribution
9
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
Independence of X and S 2 .
Theorem 6.3.A The random variable X and the vector of random
variables
(X1 − X , X2 − X , . . . , Xn − X ) are independent.
Proof:
Note that X and each of Xi − X are linear combinations of
X1 , . . . , Xn , i.e.,
Ln
X
=
ai Xi = aT X = U and
L1n (k)
(k) T
Xk − X =
1 bi Xi = (b ) X = Vk
where X
= (X1 , . . . , Xn )
a
= (a1 , . . . , an ) = (1/n, . . . , 1/n)
(k)
(k)
(k)
b
= (b1 , . . . , bn )
.
1 − n1 , for i = k
(k)
with bi =
− n1 , for i = k
U and V1 , . . . , Vn are jointly normal r.v.s
U is uncorrelated (independent) of each Vk
MIT 18.443
Distributions Derived From the Normal Distribution
10
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
Independence of U = aT X and V = bT X
where X = (X1 , . . . , Xn )
a = (a1 , . . . , an ) = (1/n, . . . , 1/n)
b = (b1 , . . . , bn )
.
1 − n1 , for i = k
(k)
with bi = bi =
(so V = Vk )
6 k
− n1 , for i =
Proof: Joint MGF of (U, V ) factors into MU (s) × MV (t)
T
T
MU,V (s, t) = E [e sU+tV
[e sa X+tb X ] P
P ] = EP
i ai Xi ]+t[ i bi Xi ] ] = E [e
i [(sai +tbi )Xi ] ]
= E [e s[
P ∗
[t
X
]
∗
= E [e i i i ] with ti = (sai + tbi )
n
n
Qn
[ti∗Xi ]
=
E
[e
]
=
MXi (ti∗ )
i=1
i=1
P
2
2 Pn
∗ 2
ti∗ µ+ σ2 (ti∗ )2
µ( ni=1 ti∗ )+ σ2
i=1 [(ti ) ]
e
=
e
i=1
P
n
2 2
2
2 2 +2sta b ]
i i
e µ(s×1+t×0)+σ /2[ i=1 [s ai +t bi P
n
2 /n)s 2 /2
2 σ 2 /2
2
sµ+(σ
t×0+t
b
i =1 i ]
[e
] × [e
Qn
=
=
=
L
= [mgf of N(µ, σ 2 /n)] × [mgf of N(0, σ 2 · ni=1 bi2 ]
MIT 18.443
Distributions Derived From the Normal Distribution
11
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
Independence of X and S 2
Proof (continued):
Independence of U = L
X and each Vk = Xk − X gives
X and S 2 = n1 Vi2 are independent.
Random variables/vectors are independent if their joint
moment generating function is the product of their individual
moment generating functions:
MU,V1 ,...,Vn (s, t1 , . . . , tn ) = E [e sU+t1 V1 +···tn Vn ]
= MU (s) × M(t1 , . . . , tn ).
Theorem 6.3.B The distribution of (n − 1)S 2 /σ 2 is the
Chi-Square distribution with (n − 1) degrees of freedom.
Proof:
Ln Xi −µ 2 Ln
1 Ln
2 =
(X
−
µ)
= i=1 Zi2 ∼ χn2 .
k
1
i=1
σ
σ2
�
�2
1 Ln
1 Ln
2
1 (Xk − µ) = σ 2
i=1 Xi − X + X − µ
σ2
�
�2
Ln
= σ12 i=1
(Xi − X )2 + σn2 X − µ
MIT 18.443
Distributions Derived From the Normal Distribution
12
Distributions Derived from Normal Random Variables
χ2 , t, and F Distributions
Statistics from Normal Samples
Proof (continued):
Proof:
1 Ln
1 Ln
2
2
1 (Xk − µ) = σ 2
i=1 (Xi − X )
σ2
χn2 = [distribution of (nS 2 /σ 2 )] + χ21
+
n
σ2
�
�2
X −µ
By independence:
mgf of χ2n = mgf of [distribution of (nS 2 /σ 2 )] × mgf of χ2
1
that is
(1 − 2t)−n/2 = MnS 2 /σ2 (t) × (1 − 2t)−1/2
So
MnS 2 /σ2 (t) = (1 − 2t)−(n−1)/2 ,
=⇒ nS 2 /σ 2 ∼ χ2n−1 .
Corollary 6.3.B For a X and S 2 from a Normal sample of size n,
X −µ
√ ∼ tn−1
T =
S/ n
MIT 18.443
Distributions Derived From the Normal Distribution
13
MIT OpenCourseWare
http://ocw.mit.edu
18.443 Statistics for Applications
Spring 2015
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.