Characterization of the Multivariate Normal
Distribution
• Cramer (1946) showed that the following characterizes a
multivariate normal distribution:
– X ∼ Np(µ, Σ) if and only if a0X ∼ N (0, σ 2) for every pvariate real vector a.
• The only if part of the proof is straightforward.
• The if part of the proof requires a combination of results
that we have seen in the past.
152
ML Estimation of Parameters
• Henceforth, we will work with a sample X1, X2..., Xn ∼ Np(µ, Σ).
1
• The MLEs for µ and Σ are given by µ̂p×1 = X̄ = n
P
1
and Σ̂ = n i(Xi − X̄)(Xi − X̄)0 = (n−1)
n S.
Pn
i=1 Xi
– The derivation uses properties of p.d. matrices and:
The identity matrix Ip is the sole among all p×p-dimensional
positive definite matrices B that maximizes
n
n/2
f (B) = |nB|
exp − trace(B) .
2
(1)
– We could also use vector and matrix derivatives.
153
Sampling distribution of MLE’s
• Suppose that X1, X2, ..., Xn ∼ Np(µ, Σ). Maximum likelihood
1 Pn
estimates are µ̂p×1 = X̄ = n
i=1 Xi and
1X
(n − 1)
Σ̂ =
(Xi − X̄)(Xi − X̄)0 =
S.
n i
n
• X̄ and S are independent
1 Σ).
• The sampling distribution of X̄ is Np(µ, n
• (n − 1)S is distributed as a Wishart random matrix with n − 1
df and parameter Σ.
154
Sampling distribution of S
• First consider the univariate case and recall that
(n − 1)s2 =
(xi − x̄)2 ∼ σ 2χ2
n−1 .
X
i
• Since χ2
n−1 is also the distribution of the sum of n − 1
independent standard normal random variables, then
2 )
(n − 1)s2 ∼ σ 2(Z12 + ... + Zn−1
= (σZ1)2 + ... + (σZn−1)2.
155
Sampling distribution of S
• Definition: The Wishart distribution with m
degrees of freedom and parameter Σ, denoted by Wm(Σ),
is the distribution of
m
X
ZiZi0,
i=1
where Zi ∼ Np(0, Σ) are independent random vectors.
• If Ap×p is a positive definite random matrix and A ∼ Wn−1(Σ),
the density function is
wn−1(A|Σ) =
|A|(n−p−2)/2 exp(−tr(AΣ−1)/2)
Qp
2p(n−2)/2π p(p−1)/4|Σ|(n−1)/2 j=1 Γ(0.5(n − j))
where Γ(.) is the gamma function.
156
,
Sampling distribution of S (cont’d)
• If A1 ∼ Wm1 (Σ) is independent of A2 ∼ Wm2 (Σ), then
A1 + A2 ∼ Wm1+m2 (Σ).
• If A ∼ Wm(Σ), then CAC 0 ∼ Wm(CΣC 0).
• If A ∼ Wm(Σ), then E(A) = mΣ.
• Using results above it can be shown that
(n − 1)S =
(Xi − X̄)(Xi − X̄)0 ∼ Wn−1(Σ)
X
i
with expectation (n − 1)Σ.
157