Information geometry of the power
inverse Gaussian distribution
Zhenning Zhang, Huafei Sun and Fengwei Zhong
Abstract. The power inverse Gaussian distribution is a common distribution in reliability analysis and lifetime models. In this paper, we give
the geometric structures of the power inverse Gaussian manifold from the
viewpoint of information geometry. Firstly, we obtain the Fisher information matrix, Riemannian connections, Gaussian curvature of the power
inverse Gaussian manifold. Then we consider the dual structure of this
manifold and investigate the KullBack divergence. At last we give an
immersion from the power inverse Gaussian manifold into an affine space.
M.S.C. 2000: 53C15, 62B10.
Key words: power inverse Gaussian distribution, information geometry, Kullback
divergence, geometric structure.
§1. Introduction
It is well known that information geometry has been successfully applied into
various fields, such as image processing, statistical inference and control theory. Recently, some scholars studied the probability density function from the viewpoint of
information geometry, and use the geometric metrics to give a new description to
the statistical distribution (see [4],[6],[7]). Here, the parameters of the probability
density function play an important role in statistical manifold and can be regard as
the coordinate system of the manifold.
The power inverse Gaussian distribution parameterized by an arbitrarily fixed real
number λ 6= 0 has the probability density function
(1.1)
f (x) = √
n
1 ³ x ´−(2+λ/2)
1 ³³ x ´λ/2 ³ x ´−(λ/2) ´2 o
exp − 2 2
−
,
2λ σ
µ
µ
2πµσ µ
where 0 < x < ∞, 0 < µ < ∞, 0 < σ < ∞. In particular, for λ = 1 or λ = −1, the
distribution becomes the inverse Gaussian distributions and the reciprocal inverse
Gaussian distributions, respectively. Also when λ → 0, (1.1) reduces to a log-normal
distribution.
In the present paper, we consider the geometric structure of the power inverse
Gaussian manifold. Firstly, we give the Fisher information matrix, the Riemannian
Applied Sciences, Vol.9, 2007, pp.
194-203.
c Balkan Society of Geometers, Geometry Balkan Press 2007.
°
Information geometry
195
connections, the Gaussian curvature under the coordinate system (µ, σ). Whilst we
can see that the power inverse Gaussian distribution is an exponential family distribution, and the corresponding manifold is ±1-flat, so we construct the dual structure
of the power inverse Gaussian manifold. Secondly, we give the Kullback divergence
in this manifold, and it is a distance like measure not satisfying the distance axiom.
Furthermore we give the relation of Kullback divergence and arc length. At last, we
give an immersion from the power inverse Gaussian manifold into the affine space.
§2. The geometric structures of the power inverse Gaussian
manifold
Definition 2.1. The set
(2.1)
n
o
S = f (x; ξ)|ξ = (ξ 1 , ξ 2 ) = (µ, σ) ∈ R+ × R+ ,
is called the power inverse Gaussian manifold, where f (x; ξ) is in the form of (1.1),
and ξ = (ξ 1 , ξ 2 ) = (µ, σ) plays the role of the coordinate system.
By a straightforward calculation, we get
(2.2)
h³ x ´λ
³ x ´−λ i
= −λ2 σ 2 ,
µ
µ
h³ x ´λ ³ x ´−λ i
Ef
+
= 2 + λ2 σ 2 ,
µ
µ
h³³ x ´λ/2 ³ x ´−λ/2 ´2 i
= λ2 σ 2 .
Ef
µ
µ
Ef
−
Using
Z
(2.3)
gij =
∂ log f (x) ∂ log f (x)
f (x)dx,
∂ξ i
∂ξ j
and (2.2), we get the Fisher information matrix with respect to the coordinate system
(µ, σ),
Ã
(2.4)
(gij (ξ)) =
2+λ2 σ 2
2µ2 σ 2
λ
− µσ
λ
− µσ
!
2
σ2
,
immediately the square of the element of the arc length is given by
(2.5)
ds2 =
2 + λ2 σ 2 2 2λ
2
dµ −
dµdσ + 2 dσ 2 .
2µ2 σ 2
µσ
σ
From (2.4), we can get the inverse of the Fisher information matrix
Ã
(2.6)
ij
(g (ξ)) =
µ2 σ 2
λµσ 3
2
λµσ 3
2
2
σ (2+λ2 σ 2 )
4
!
,
196
Zhenning Zhang, Huafei Sun and Fengwei Zhong
Combining
Γijk =
1
(∂1 gjk + ∂j gki − ∂k gij )
2
with (2.4), we obtain the Riemannian connections
2 + λ2 σ 2
λσ 2 + 1
, Γ112 =
,
2
3
2σ µ
µ2 σ 3
1
= − 2 3 , Γ122 = 0,
µ σ
λ
2
=
, Γ222 = − 3 ,
µσ 2
σ
Γ111 = −
(2.7)
Γ121
Γ221
and
λ−2
λ2 σ 2 + 2
, Γ211 =
,
2µ
4µ2 σ
λ
1
= − , Γ212 = − ,
σ
2µ
1
= 0, Γ222 = − .
σ
Γ111 =
(2.8)
Γ112
Γ122
Combining
Rijkl = (∂j Γsik − ∂i Γsjk )gsl + (Γjtl Γtik − Γitl Γtjk ),
(2.4) with (2.8), by a calculation, we get the nonzero component of the Riemannian
curvature tensor
(2.9)
R1212 = −
1
µ2 σ 4
.
Theorem 2.1. The Gaussian curvature of the power inverse Gaussian manifold
is given by
1
K=− .
2
(2.10)
Proof. Since the determinant of the Fisher information matrix is
det(gij (ξ)) =
2
,
µ2 σ 4
and the Gaussian curvature is defined by
K=
from (2.9), we obtain Theorem 2.1.
R1212
,
det(gij )
¤
Information geometry
197
§3. The dual structures of the power inverse Gaussian manifold
Proposition 3.1. The power inverse Gaussian distribution is an exponential
family distribution.
Proof. The power inverse Gaussian probability density function (1.1) can be
rewritten as
n
1
µλ −λ
λ
λ
x
x − (1 + ) ln x
−
2
2
λ
2
2
2λ σ µ
2λ σ
2
³
´o
1
1
λ
− ln σ − 2 2 − ln µ + ln 2π .
λ σ
2
2
f (x) = exp
(3.1)
−
Set
xλ
1
,
θ1 = 2 λ ,
2
2λ
σ µ
x−λ
µλ
y2 = − 2 ,
θ2 = 2 ,
2λ
σ
1
1
y1
M (y) = −( +
) ln( ),
4 2λ
y2
y1 = −
(3.2)
then the potential function ψ(θ) can be written as
√
θ1 θ2
1
1
(3.3)
ψ(θ) = −
− ln θ2 + ln 2π.
2
λ
2
2
That is, the power inverse Gaussian probability density function can be denoted as
f (y) = exp{y1 θ1 + y2 θ2 − ψ(θ) + M (y)},
so it is an exponential family distribution([1]).
¤
Remark: The power inverse Gaussian manifold is ±1-flat.
Proposition 3.2.
³
´
λ
i) θ = (θ1 , θ2 ) = σ21µλ , µσ2 , is the natural coordinate system of the power inverse Gaussian manifold.
√
ii) ψ(θ) = − θλ12θ2 − 12 ln θ2 + 12 ln 2π, is the potential function with respect to
natural coordinate system θ.
³
´
µλ
1+λ2 σ 2
iii) η = (η 1 , η 2 ) = − 2λ
, is the dual coordinate system, and is also
2 , − 2λ2 µλ
called expectation coordinate system.
iv) φ(η) = − 12 ln(4λ2 η 1 η 2 − 1) + 21 ln(−η 1 ) + 2 ln λ − 12 ln πe, is the dual potential
function with respect to the expectation coordinate system.
Proof. The proof of the (i) and (ii) can be seen in the proof of Proposition 3.1.
Now, let us give the proof of (iii) and (iv).
198
Zhenning Zhang, Huafei Sun and Fengwei Zhong
From
ηi =
∂ψ(θ)
,
∂θi
we get the expectation coordinate system of the power inverse Gaussian manifold
(η 1 , η 2 ) =
³
−
µλ
1 + λ2 σ 2 ´
,
−
.
2λ2
2λ2 µλ
The dual potential function with respect to the expectation coordinate system is
1
1
1
φ(η) = Σθi η i − ψ(θ) = − ln(4λ2 η 1 η 2 − 1) + ln(−η 1 ) + 2 ln λ − ln πe.
2
2
2
Under natural coordinate system, the related geometric metrics can be given by
(3.4)
gij (θ) = ∂i ∂j ψ(θ),
Tijk (θ) = ∂i ∂j ∂k ψ(θ) = ∂k (gij ),
1−α
(α)
Γijk (θ) =
Tijk (θ),
2
1 − α2
(α)
(Tkmi Tjln − Tkmj Tiln )g mn .
Rijkl =
4
Then, by a direct calculation, we get the following geometric metrics of the power
inverse Gaussian manifold with respect to the coordinate system (θ1 , θ2 ),
Ã
(3.5)
(gij (θ)) =
Ã
(3.6)
ij
(g (θ)) =
T111 = −
σ 2 µ2λ
4λ22
σ
− 4λ
2
2
σ
− 4λ
2
T122
T222
,
2λ2 σ 4 +σ 2
4λ2 µ2λ
4λ2 σ 2 +2
σ 4 µ2λ
2
σ4
2
σ4
2µ2λ
σ4
!
3µ3λ σ 4
,
8λ2
µλ σ 4
,
8λ2
σ4
= T212 = T221 = 2 λ ,
8λ µ
2 6
4
8λ σ + 3σ
=−
.
8λ2 µ3λ
T112 = T121 = T211 =
(3.7)
!
,
¤
Information geometry
199
and
(α)
1 − α 3µ3λ σ 4
,
2
8λ2
1 − α µλ σ 4
(α)
(α)
= Γ121 = Γ211 =
,
2 8λ2
1 − α σ4
(α)
(α)
= Γ212 = Γ221 =
,
2 8λ2 µλ
1 − α 8λ2 σ 6 + 3σ 4
=−
.
2
8λ2 µ3λ
Γ111 = −
(α)
Γ112
(3.8)
(α)
Γ122
(α)
Γ222
From the above, we obtain the nonzero component of the curvature tensor with
respect to the natural coordinate system,
(α)
(3.9)
R1212 =
(1 − α2 )σ 6
,
16λ2
and the α-Gaussian curvature is
K (α) = −
(3.10)
1 − α2
.
2
When α = 0, it is the Riemannian case, and K = − 21 , it is the same as (2.10).
§4. The Kullback divergence
Now, let us give a new distance like measure which is different from the arc length,
Kullback divergence. It plays an important role in statistical manifold.
Proposition 4.1. Let P and Q be two points in the power inverse Gaussian
manifold with coordinates (µP , σP ) and (µQ , σQ ), respectively. Then the Kullback
divergence between P and Q is given by
(4.1)
D(P, Q) =
2
λ µQ
σP
1 ³ µλQ
µλP ´ 1 σQ
µλP
1
1
ln
+ ln
+ 2 2
+
+
− 2 2 − .
2 µP
σQ
2λ σP µλP
2 σP2 µλQ
λ σP
2
µλQ
Proof. From the dual structure we construct above, we can get the Kullback
divergence between P and Q
(4.2)
D(P, Q) = ψ(θP ) + φ(ηQ ) − θP · ηQ .
The natural coordinate system θP , the potential function ψ(θP ) of the point P , the
expectation coordinate system ηQ and the dual potential function φ(ηQ ) of the point
Q are given by
200
Zhenning Zhang, Huafei Sun and Fengwei Zhong
(4.3)
³ 1
µλP ´
θP = (θ1P , θ2P ) =
,
,
σP2 µλP σP2
√
θ1P θ2P
1
1
ψ(θP ) = −
− ln θ2P + ln 2π,
λ2
2
2
2 2 ´
³ µλ
1
+
λ
σQ
Q
1
2
,
ηQ = (ηQ
, ηQ
) = − 2,−
λ
2
2λ
2λ µQ
1
1
1
1 2
1
φ(ηQ ) = − ln(4λ2 ηQ
ηQ − 1) + ln(−ηQ
) + 2 ln λ − ln πe,
2
2
2
respectively.
Combining (4.2) with (4.3), by a computation, we get Proposition 3.3.
¤
Now, let us consider some special cases:
i) When the parameter θ1 = µ is fixed, i.e., µP = µQ , we have
D(P, Q) = ln
(4.4)
2
σQ
σP
1
+ 2 − .
σQ
2σP
2
From (4.4), we see that when µ is fixed, the Kullback divergence doesn’t depend on
µ.
ii) When the parameter θ2 = σ is fixed, i.e., σP = σQ , we have
(4.5)
D(P, Q) =
1 ³ µλQ
µλP ´ 1 µλP
1
1
λ µQ
ln
+ 2 2
+
+
− 2 2 − .
λ
λ
λ
2 µP
2λ σP µP
2 µQ
λ σP
2
µQ
Next, using (2.5), we obtain the arc length in the 1-dimensional parameter space
from the point P (µP , σP ) to Q(µQ , σQ ) as follows:
i) When ξ 1 = µ is fixed, and ξ 2 = σ is free, we get
Z
ξQ
√
Sµ =
(4.6)
ξP
¯
¯
√ ¯ σQ ¯
2
¯
¯.
dσ = 2 ¯ln
σ
σP ¯
ii) When ξ 2 = σ is fixed, and ξ 1 = µ is free, we get
Z
(4.7)
ξQ
Sσ =
ξP
r
2 + λ2 σ 2 dµ
=
2σ 2
µ
r
2 + λ2 σ 2
2σ 2
¯
¯
¯ µQ ¯
¯ln
¯
¯ µP ¯ .
Then from the above results, we get
Proposition 4.2. Let P and Q be two points in the power inverse Gaussian
manifold with coordinates (µP , σP ) and (µQ , σQ ) respectively. Then the Kullback divergence D and the arc length S are connected in the following ways
i) when µ is fixed,
√
(4.8)
Dµ =
1 √
1
+ e 2Sµ − .
Sµ
2
2
2
Information geometry
201
The relation between Sµ and Dµ can be seen in Figure 1 and 2.
ii) when σ is fixed,
(4.9)
´ 1 − √ 2λσSσ
− √ 2λσSσ
λσ
1
1 ³ √ 2λσSσ
1
Dσ = √
Sσ + 2 2 e 2+λ2 σ2 + e 2+λ2 σ2 + e 2+λ2 σ2 − 2 2 − .
2
2
2λ
σ
2
λ
σ
2
4 + 2λ σ
√
√
√
The relation between Sσ and Dσ can be seen in Figure 3.
§5. Affine immersions
Let M be an m-dimensional manifold, f be an immersion from M to Rm+1 , and ξ
be a transversal vector field along f . We can identify Tx Rm+1 ≡ Rm+1 for ∀x ∈ Rm+1 .
The pair {f, ξ} is said to be an affine immersion from M to Rm+1 , if for each point
P ∈ M , the following formula holds
Tf (P ) Rm+1 = f∗ (TP M ) ⊕ spanξP .
(5.1)
We denote the standard flat affine connection of Rm+1 with D. Identifying the covariant derivative along f with D, we have the following decompositions
DX f∗ Y = f∗ (∇X Y ) + h(X, Y )ξ,
DX ξ = −f∗ (Sh(X)) + τ (X)ξ.
(5.2)
The induced objects ∇, h, Sh and τ are the induced connection, the affine fundamental form, the affine shape operator and the transversal connection form, respectively.
From the fundamental concepts of the affine immersions, we have the following
Proposition of the power inverse Gaussian manifold.
Proposition 5.1. Since (S, h, ∇, ∇∗ ) is the inverse Guassian manifold, it is
dually flat space with a global coordinate system. θ is an affine coordinate system
of ∇, and ψ is a θ-potential function. Then the power inverse Gaussian manifold
(S, h, ∇) can be immersed into R3 by the following way
(5.3)
f :S → R3
"
#
1
σ 2 µλ
µλ
σ2


→
√
− θλ12θ2
−
1
σ 2 µλ
µλ
σ2
1
2 ln θ2


,
+
1
2
ln 2π
which is called a graph immersion from S into R3 . And the transversal vector ξ =
0
(0, 0, 1) .
6
Figures.
By choosing suitable constants, we draw the pictures below.
202
Zhenning Zhang, Huafei Sun and Fengwei Zhong
250
Dµ
200
150
100
50
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
S
µ
√
2
Sµ
Fig 1. Dµ =
+ 12 e
√
2Sµ
− 12 , 0 < Sµ < 1.
80
D
µ
70
60
50
40
30
20
10
0
1
1.5
2
2.5
3
3.5
4
4.5
5
Sµ
Fig 2. Dµ =
√
2
Sµ
+ 12 e
√
2Sµ
− 12 , Sµ > 1.
40
D
σ
35
30
25
20
15
10
5
0
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
S
σ
√
Fig 3. Setting λ = 1, and σ = 2, (4.9) becomes to Dσ = S2σ + 14 eSσ + 34 e−Sσ −1, Sσ > 0.
References
[1] S. Amari, Differential geometrical methods in statistics, Springer Lecture Notes
in Statistics 28, Springer Verlag 1985.
Information geometry
203
[2] S. Amari, Methods of Information Geometry, Oxford University Press, 2000.
[3] S. Amari, Differential geometry of a parametric family of invertible linear systems Riemannian metric, dual affine connections and divergence, Mathematical
Systems Theory 20 (1987), 53-82.
[4] C. T. Dodson and H. Matsuzoe, An affine embedding of the gamma manifold,
Appl. Sci. 5 (2003), 1-6.
[5] Toshihiko Kawamuna and Kōsei Iwase, Characterizations of the distributions of
power inverse Gaussian and others based on the entropy maximization principle,
J. Japan Statist. Soc 33 (2003), 95-104.
[6] Z. Zhang, H. Sun and F. Zhong, The geometric structure of the generalized
Gamma manifold and its submanifolds, preprint.
[7] F. Zhong, H. Sun and Z. Zhang, Information geometry of Beta distribution,
preprint.
Authors’ addresses:
Zhenning Zhang, Huafei Sun and Fengwei Zhong
Department of Mathematics, Beijing Institute of Technology,
Beijing, 100081 China
e-mail: ningning0327@163.com, sunhuafei@263.net, killowind@bit.edu.cn