Exponential Family Facts
Stat 543 Spring 2016
Many standard families of distributions can be treated with a single set of analyses by recognizing them
to be of a common “exponential family” form. The following is a list of facts about such families. (See, for
example, Schervish Sections 2.2.1 and 2.2.2, or scattered results in Shao or the old books by Lehmann (TSH
and TPE ) for details.)
De…nition 1 Suppose that P = fP g is de…ned by pdf ’s or pmf ’s f (x) for X. P is called an exponential
family if for some h(x) 0,
!
k
X
: dP
f (x) =
(x) = exp a( ) +
.
i ( )Ti (x) h(x) 8
d
i=1
R
P
Claim 2 Let = ( 1 ; 2 ; :::; k ); E = f 2 Rk j h(x) exp (
i Ti (x)) d (x) < 1g, and consider also the
family of distributions with pdf ’s or pmf ’s of the form
!
k
X
f (x) = C( ) exp
,
2E .
i Ti (x) h(x)
i=1
Call this family P . This set of distributions for X is at least as large as P (and this second parameterization
is mathematically nicer than the …rst). E is called the natural parameter space for P . It is a convex subset
of Rk . (TSH, page 57.) If E lies in a subspace of dimension less than k, then f (x) (and therefore f (x))
can be written in a form involving fewer than k statistics Ti . We will henceforth assume E to be fully
k-dimensional. Note that depending upon the nature of the functions i ( ) and the parameter space , P
may be a proper subset of P . That is, de…ning E = f( 1 ( ); 2 ( ); :::; k ( )) 2 Rk j 2 g; E can be a
proper subset of E.
Claim 3 The "support" of P , de…ned as fxjf (x) > 0g, is clearly fxjh(x) > 0g, which is independent of .
Claim 4 From the Factorization Theorem, the statistic T = (T1 ; T2 ; :::; Tk ) is su¢ cient for P.
Claim 5 T has distributions derived from the distributions for X, say fP T j 2
exponential family.
Claim 6 If E
g, which also form an
contains an open rectangle in Rk , then T is complete for P. (See pages 142-143 of TSH.)
Claim 7 If E contains an open rectangle in Rk (and actually under the much weaker assumptions given
on page 44 of TPE) T is minimal su¢ cient for P.
Claim 8 If g is any real-valued function such that E jg(X)j < 1, then
Z
E g(X) = g(x)f (x)d (x)
is continuous on E and has continuous partial derivatives of all orders on the interior of E. These can be
calculated as
Z
@ 1+ 2+ + k
@ 1+ 2+ + k
E
g(X)
=
g(x)
f (x)d (x) .
@ 11@ 22
@ kk
@ 11@ 22
@ kk
(See page 59 of TSH.)
1
Claim 9 If for u = (u1 ; u2 ; :::; uk ), both
0
In particular, E 0 Tj (X) =
@2
j@
l
0
+ u belong to E
o
+ uk Tk (X) =
C( 0 )
:
C( 0 + u)
is in the interior of E, then
E 0 (T1 1 (X)T2 2 (X)
@
and
n
exp
u1 T1 (X) + u2 T2 (X) +
0
E
Further, if
0
( ln C( ))
=
@
@ j
Tk k (X)) = C(
( ln C( ))
=
0)
1+
@
@
1
1
@
, Var 0 Tj (X) =
0
2+
2
2
@2
@ 2j
+
@
k
k
k
1
C( )
( ln C( ))
=
:
=
0
, and Cov
0
(Tj (X); Tl (X)) =
0
.
0
Claim 10 If X = (X1 ; X2 ; :::; Xn ) has iid components,
Pn with each Xi P , then X generates a k-dimensional
exponential family. The k-dimensional statistic i=1 T (Xi ) is su¢ cient for this family. Under the condition that E contains an open rectangle, this statistic is also complete and minimal su¢ cient.
2