LIBRARY
OF THE
MASSACHUSETTS INSTITUTE
OF TECHNOLOGY
ALFRED
P.
SLOAN SCHOOL OF MANAGEMEN'
Multinormal Bayesian Analysis
;
Two Examples
by
J.
J.
Martin
mass.
i;>
May, 1964
85-64
MASSACHUSETTS
50
MEMORIAL DRIVE
VMBRIDGE, MASSACHUSETTS
021
Multinormal Bayesian Analysis
:
Two Examples
This research was done under the direction
of
G.M. Kaufman, Assistant Professor of Industrial
Manage'
ment, Alfred P. Sloan School of Management,
as part
of the requirements for the course
15.951, Special
Studies in Management, during the second term.
'
1963-1964.
NOV
I
I
IV,
ls\
Multinormal Bayesian Analysis
J.
1.
;
Two Examples
J. Martin
Introduction
The Bayesian analysis of the independent multinormal process in which neither the
mean nor precision are known with certainty is carried out by Ando and Kaufman in [1],
These same authors extend this analysis in [2] to
the reduced form data generating process.
a
generalized regression model, termed
In order to illustrate these results and to
suggest the manner in which multinormal Bayesian analysis can be applied to some vital
areas of operations research, two examples are presented here.
Particular attention is
given, in both cases, to the problem of assigning a prior distribution to the parameters
of the process.
in the sequel.
In section 2 the moments of certain distributions are derived for use
In section 3 a radar-tracking model is presented, in which the error
in the target velocity has a bivariate normal distribution.
The Bayesian analysis
of a system of simultaneous linear equations is illustrated in section 4, using
Leontief's static inter-industry model.
The mean and variance of the standardized and
non-standardized inverted generalized Beta distributions can be obtained using the
theorems of section 2.
These results are presented in Appendix A.
1751891
-
2
Distribution Theory
2.
The moments of certain distributions will be required in order to fit prior
These moments are derived in this section.
distributions.
The notation used here follows^ in general, that of Raiffa and Schlaifer [3j.
In particular, random variables are identified by means of the tilde; e.g., a, a,
and h are, respectively, a random variable, a random vector, and a random matrix.
The matrix with generic element h.
y.^
written
is
j^.
.
is denoted by h; a vector with generic element
All vectors are column vectors unless otherwise noted; column
vectors will, on occasion, be written {^,,...,^1
is h
.
1/h^..
The elements of h
The symbol
E^^i
[z]
}.
The transpose of the matrix h
are written hT., and are not to be confused with
denotes the expected value of
z
with respect to the
conditional distribution of x given y=y; E [z] is the expected value of
z
relative
to the unconditional distribution of x.
Theorem
1
Let the r x
.
1
vector ^ and the
r
x r positive definite symmetric (PDS)
matrix h have the Normal-Wishart distribution,
f$i^(ti.
Let h={hj^,.
.
.,Kj^3
h|m, y, n, V) = fJ'^^tLllIl.^h)
f^^^^hjv, v)
.
be a random vector consisting of the R=g-r(r+l) distinct elements
of H^
(1)
and let
h=(K^,...,hj^3 = (v+r-i)
(v;J,...,v^J,v 22^
(2)
Then
(3)
and the variance-covariance matrix of (^, Kj is, provided v
>
2.
-
~
3 -
-
-
4
But
E[(ll-m)(h-h)'^] =
E[Q-m)(^-m)'^] = E
and
_
_
E
Ej^
(^-m) (^-m)
=
^(^.2) I
= Ej^[(h-h)(h-h)'']
=
y
_
]
'^
^
>
(H)
"^
>
(12)
.
Q.E.D.
If the r x r PDS matrix h has the Wishart distribution with parameters
1.
V and V,
'
_
The last equation follows from [4], p. 161.
Theorem
(10)
,
]
[
^
E[(K-iL)(H-h)
=
u [(H-EXh-h)*^]
then
ll =
E[h"^]
-To
v-2 y
V
,
>
2
(13)
.
Let
h^
^^^
—1
h
=
[^,^,...,\\,^^^,...,^^\]
(14)
^-1
= E[h ^]
.
Then, provided v > 4,
v[ri]
where, if
=h
h.
and h. =h
„
W. .(V)
If i=i
E[(h-i-r^(r^-rS'^]
=
=
c
ds)
,
.
covirl, rlj
=
[21|ivl V„pV^^«^^Vp,+v^,Vp^l
3^,.,;^^.,)
we obtain the variance of h
,
w(v)
=
(16)
.
„
aP'
Proof:
(^,
1,
""trj]
=
"ll(«
=
[^
3(„^)\„4)
vj
+ V^Vppl
(17)
.
Let Q^, K) have the Normal-Wishart distribution with parameters
I,
v)
Consider, for any real
.
of the marginal distribution of
t~
1
•
Expressing
e
U
—
h
vector u, the characteristic function
1
j^,
t-
E[e^ii^] =EE,, [e^Hii]
ti
r x
=
y_^
e.E
h
,
„ [e^
ii ^i]
(18)
.
til
t~
M
=^
as a power series
- ^1
^
,,>k
,t~
Vhk'^'
=
V>io
,
(Ji
^ V<Jl
,.,k
^
Va>
J%io
Va>'l
<^'>
5 -
-
The interchange of expectation operator and summation operator is justified by
the fact that the power series converges for all real values of u and ^.
since^ for fixed h^
i
V^lht^
^ has the Normal distribution with parameters
u'^UT
r
r
^
-^^ =^h[-P(-yipil
,
,
=
1
(0,
Similarly^
h)
;..!
VpV^
klo-4f V^clipii vp^;J)'J
•
Since (19) and (20) are both power series expansions of the same function,
coefficients of like powers of the variables u
must be equal.
We thus obtain
the following identity, valid for all real u:
^k^ ^hKJi
In particular,
Ji
pil
Vp^aP>'j
=
izfer
Veil Va)'"^
(21)
•
for k=l,
Vp^ht^^p^
pii
=
ii
<22>
pii Vp'i,[^c/p^
and for k=2,
VpV6'h[^;p^;6] =ia.P?7,6 VpV5V^a^P^7i^6^
a,pf7,6
•
('^>
The marginal distribution of ^ is multivariate Student with parameters
(0, vY
of
j^
,
when
v).
y^
By noting that the Student distribution is the marginal distribution
and the univariate random variable H have the Normal-Gamma distribution,
it is seen that,
provided v > 4,
V^^aWsl
=f^a^?^y^b
^s'^^iilo^
^/J^a^P^^^S
^f^
v)d^
^"
4'^(ii|0' ^«H'') f,2(«|l^
72'
^)^ii
R
(24)
(%Wp5%6"P7>/-;i7
^aP^6-^^a7^p5+Va5^py
(v-2)(v-4)
*[3], p. 256.
^
^2(«|1^ v)dH
-
6
Since (22) and (23) hold for all real u
^
-1
^h^^aP^ = \^^c^?'^ = v-2
V
>
2
(25)
,
which yields (13), and
^ht^aP
Sl^
=
(26)
I \^ W7^5^
which yields
.^-1
^°^[%'
^-1.
M^
-
3(v-2)(v-4)
iilivi
^^^32^ VaP^5+Va,Vp6+VVp,]
•
(^7)
If a=7, P=6, (27) reduces to
^"'^t^ij^ = :^(v-l)(v-4)
Corollary
(m,
y,
1
.
n, v)
If (^, h)
(v-2) ^J
and
h-^
Q.E.D.
has the Normal-Wishart distribution with parameters
then
where
ffS vj+v^vpp]
(28)
-
and the variance-covariance matrix of
7
£
-
is
(33)
where
Proof
Q.
;
denotes the Kronecker product.
The non-degenerate generalized multivariate Student distribution is
obtained in [2] as the marginal density function of
jt
when £ and the m x m matrix
h have the joint density function.
DU,
h|p, Y, I, V) = t^^^'hnl^, b « V) f^'^^hli, V)
.
Thus
(34)
(35)
Since the marginal distribution of h is Wishart with parameters (e, v)
Theorem
2
yields
(36)
provided v >
Corollary
2
.
2.
Q.E.D.
If (£, h) has the Normal-Wishart distribution^
fr^iil£^ h«y)
f^^'^^hli, V)
then
V>
2
,
(37)
where
(38)
^e
(v-2)
«V-^
V
W(e)
> 4
.
(39)
-
8
Radar-Tracking Model
3.
In order to illustrate preposterior and prior-posterior analysis for the
independent multivariate Normal process when uncertainty exists about the mean
and the precision, we consider in this section a simple radar-tracking model.
The Bayesian analysis follows [1].
Let a single aircraft, on a fixed course and speed at a fixed altitude,
be tracked by a single radar.
Let
7
1
be the unknown velocity of the aircraft, with east-west component
v^
and north-
south component v„
If
(k)
(k)
k=l,2,..,
,(k)
is the position of the aircraft at the time of the kth radar scan and if
~(k)
^^l
'(k)
k=l,2,.,
~(k)
^2
is
the radar error in measuring aircraft position on the kth scan (a random
variable), then the radar-derived velocity on the kth scan is, letting
~(k)
1
=
~(k) ~(k-l)
»i
'k
>
'(k)
since
(^('^)^-(^))
,W
.
(^(lc-l)^^(k-l)^
.
~(k)
(40)
f(k-l)
.(k)
If the radar errors,
^
k=l,2,..,, are assumed to be independent, identically
,
distributed random variables, each having the bivariate Normal distribution with mean
0,
~fk)
then the radar-derived velocities, x
are independent, identically distributed
,
random variables, each with the bivariate Normal distribution with mean v and precision
h, neither of which is known with certainty.
While it is true that the variance-
covariance matrix for radar error can be derived from the theory of radar signals,
uncertainty about h is introduced by numerous factors in the tracking system, including
weather conditions, physical condition of the radar equipment, and random delays in
processing radar data.
Let us initially assume that prior information is negligible.
Inspection of
equation (7) of [1] shows that, if the parameters of the posterior distribution are
to be
determined only by sample information, then the following values
niist
be assigned
to the prior parameters:
n'=0
Then, if n >
2
,
v'=0
,
m'=0
V'=0_
,
(41)
.
independent radar observations of target velocity are
the statistics n, v, m, and V,
-s^t^
,
yielding
the parameters of tne posterior Normal-Wishart
distribution are, since c'=0, 5=1, and J=0,
n"=n
,
m"=m
,
v"=n-l
'f=i
,
''42)
.
Now suppose the following five radar observations of target velocity are obtained.
.(1)
.^2)
539
_
r5i4i
309
.(3)
528
300
250
(43)
.(4)
521
294
/3)
524
310
-
10 -
The data upon which this sample is based is presented in Appendix B.
Upon calculating
the sample statistics, v, n, m and Y, and substituting them into (42), the posterior
distribution is found to be nondegenerate' Normal-Wishart, with parameters
n"=5
v"=4
,
r525.2l
V"= V =
,
r342.6
249.31
|_249.3
315.4]
(44)
[300.6]
The actual mean vector, w, and variance matrix, S, used in constructing the sample
(43) were
w
[520
s
,
1
Ino
=
30oJ
|_
70
70l
,
9qJ
equivalent to a target on course 060° at a speed of 600 knots, with a standard
deviation of 10.95 knots in the
east-west
standard deviation of 9.48 knots in the
coefficient between
x,
and
component of radar-derived velocity and a
north-soutfa
component.
The correlation
is 0.673.
^.j
With negligible prior information and the sample (43), the posterior expectation
~
~-l
of V and h
is, using Corollary 1,
El?l =
(45,
300.
fF^i
=
v'^Vi
r
=
ir
=ri7i.3
[_124.6
i24.6"|
(46)
.
157.
7J
Let us now construct a prior distribution which is "tight" on h, but "loose" on
V.
This procedure might be desirable, for example, when the variance-covariance matrix
of the rectangular components of error, e, and e^, is virtually independent of target
velocity.
In assigning a prior distribution to (v, h)
N = r +
^|tli +
1
+
1
degrees of freedom in the prior parameters m'
Theorems
1
,
there are
= l.(r2+3r+4)
,
V', n', v'.
Using the results of
and 2 and Corollary 1, the following systematic procedure for assigning prior
parameters is suggested.
a.
The decision-maker states the mean^ h
Using Theorem 1,
V'
is
obtained in terms of h
,
of his marginal prior on h
and the, as yet, unspecified
parameter v'
y'
=
(v'-2)h"
(47)
.
If the decision-maker desires, he may, alternatively, state the mean, h, of
his marginal prior on h; then, using (2),
r'
=
T74nT5
(48)
•
Most individuals, however, find it easier to make probability statements about the
variance-covariance matrix, h
b.
rather than its inverse, and we shall use (47).
The decision-maker states the mean, v, of his marginal prior on v.
prior parameter
m'
m'
c.
,
The
is
= V
(49)
Having assigned values to m' and
v'
in such a way that the mean of the
Normal-Wishart prior on (v, h) is fixed, there remain two degrees of freedom by
which to further characterize the prior.
r
Ideally, one would desire to have at least
+ |-r(r+l) degrees of freedom remaining, allowing the decision-maker to incorporate
into the prior on (v, h) the variances of his marginal priors on v and h
However, the Normal-Wishart distribution is extremely parsimonious in this
respect.
Two possible alternatives suggest themselves.
The first is to obtain from
the decision-maker his marginal variance for one element, v., of v and his marginal
variance for one element, h.
variances are used to fix
1
.,
n'
^.1
of h
^
(or, equivalently, an element of h)
and v', implying variances and covariances
for the remaining elements of v and h.
If these implied variances and
covariances are in tolerable agreement with the decision-maker's
prior knowledge, then n' and v' are used as parameters for the prior
.
These
distribution of (v, y)
Precisely what is meant by "tolerable" must be defined in
.
terms of the terminal cost structure of the problem.
Another method of procedure is to choose n' and
v'
so as to minimize either the
maximum absolute deviation or the sum of squared deviations between the variances and
covariances of v and h implied by n' and v' and those of the decision-maker.
This
procedure may or may not be justified, depending upon the terminal cost structure of
The first method is developed here, resulting in equations (51
the problem.
(54
)
and
These equations can also be used to develop the least-squares or minimum
).
absolute deviation selection of
~-
and v'.
n'
~in the decision-maker's marginal prior on h
If the variance of h
is o.
.
then, by Theorem 2,
2v;,.^
ii -
,
_.l 2
V-4
"
(v'-2)2(v'-4)
(h,!)
(50)
ii'
which yields
v-
= 4
+ -2_ (H-J)2
(51)
.
ii
If
(48) were used to fix V'
""'
where a
.
=^:
^^ij -"^ii
in terms of v', then it follows from (6)
Nj^
is the variance of K.
.
-
r
+
1
that
(52)
,
in the decision-maker's marginal prior on h.
Since v has the multivariate Student distribution with parameters
(m'
,
n'v'V*'
,
v'), it follows that the marginal variance-covariance matrix of v is
^[?1 =
If s.
is
n^:2T
^'
the variance of v.
n'
=
-ii
=
b
^''
<^^>
'
in the decision-maker's marginal prior on v, then
(54)
.
*i
An alternative method of fixing
n'
and v' uses the squared coefficients of
variation of the dec is ion- makers marginal priors
•
Let C
.
.
'ii
be the squared coefficient
13
of variation of K"
and let C^ be the squared coefficient of variation of v..
2v:
^^ ~
(v'-2f(v'-4)
(V-2)^
Then
(55)
v'-4
(56)
yielding
(57)
(58)
Of course, using the squared coefficient of variation does not eliminate the inherent
difficulty of fitting two parameters to r+g-r(r+l) prior variance^but use of (57) and {56)
may simplify the procedure of fitting parameters.
--
To illustrate the use of the above procedure for assigning values to the prior
parameters, consider the following example.
Let us a priori assume the target will
have that velocity which is of the greatest threat- -
i. e.
,
assume the target is on a
course for the task force center and is travelling at maximum speed.
the present target position, this velocity is 650 knots on course 045
Suppose, for
(cf.
Figure 1).
-
14
-
Let the expected variance-cevariance matrix be
:-l
110
55
55
110
which implies a standard deviation of 10.49 knots in both
coefficient of 0.5.
v.
and V2, With a correlation
To be tight on h, let the prior squared coefficient of variation
of both hT, and h22 be 0.05.
To be loose on v, let the prior standard deviation in
Then, using (47), (49), (54), and (57), the prior
parameters are
-
15
379, 998. 4
790,522.8~|
f, 790,522.8
452,845. 7J
-
(63)
2445. Ol
[5172.3
4924.
2445.0
4J
~
~-l
tected values
va!
of v and h
are
The posterior expected
E[v] = m" =
r
523.11
300.8J
I
E[ri]
=
.0
104.)
The actual mean vector and covarlance matrix of the population from which the sample
(43) was drawn is
"520
,
300
Note that
I
=
ri20
70
_ 70
90
:]
being diffuse on v through an extremely small n', effectively negated the
prior parameter m'
,
so that m" essentially reflects the sample mean.
On the other hand^
being tight on h through a large v', the sample had little effect on E[h
We next illustrate the preposterior analysis of the multivariate Normal process.
There is a cost associated with the collection of radar observations, since, with
each additional observation, the target moves closer to the force center and therefore
poses an increased threat.
Assume, for simplicity, that the cost, c (n)
n radar observations on the target before firing,
Cg(n) = an
,
n=0,l,2,...
is
,
of making
linear:
.
(64)
On the other hand, the more observations collected before firing, the greater the
probability of destroying the target when it is fired upon.
Assume that, if v is the
estimated velocity set into the fire control mechanism when the target is fired upon.
the cost of firing after n observations is quadratic;
c^(n) = K[(v--0,)^ + (v,-0,)^]
(65)
16
-
-
Here v is the target's actual velocity and c (n) is the expected target threat after
K is a constant.
the weapon is fired.
Regarding v as a random variable^ it is
shown on p. 188 of [3] that, if the estimate v is chosBn
so as to minimize
E^[Cg(n)], then
V = E[v]
(66)
and the expected cost of firing is then
Ey[Cg(n)] = K(var[v^] + var[v2]
(67)
.
After a sample of size n is drawn, the posterior marginal distribution of v
is
multivariate Student with parameters (m", v"n"Y""
^tB
-~v^v^r= ^F{7^ r
Applying equation (5-28) of [3], if E
[•]
= v- -
V^^y[m']
v") and covariance matrix
(68)
.
denotes the expectation operator relative
to the distribution of the sample statistic (m, y)
\y[>]
,
,
then, before the sample is drawn,
(69)
.
Here v" is the posterior variance-covariance matrix of v and is a random variable
before the sample is drawnj
V
,,[
m, V
•
1
y'
is the prior variance-covariance matrix of v and
denotes the variance-covariance matrix of a random vector relative to the
distribution of (m, V)
.
Since, before the sample is drawn, m
*
Student distribution with parameters (m'
2
,
(n'7n) H
,,
has the multivariate
v'), where
(69) can be written as
f>l
\,V^^ J
E
-
-^
v'-2
of (m, y),
*[1], pp. 18 and 19.
-±-
V'
v'n' ^
-
-^
v'-2
(^ \^
V-'
""
1
y. _
y.
v'nn' ^ " n"(v'-2) ^
(70)
^
^
17
Elc^(n)]
^V^(7^
(^il
+ ^22>
(71)
•
If n observations are taken before firing,
let the total expected cost be C(n).
Then
an +
C(n)
(n'+n)(v'-2)
(72)
^"ll
Differentiating (72) with respect to n and setting the result equal to zero gives the
following equation for the optimal number of observations, n*:
/
i
If n*
<
^(^il+^22)
a(v'-2)
.
-
"
(73)
•
0, no observations are to be taken.
If n* is not an integer,
choose the
integer neatest n* whic.
For a Normal-Wishart prior distribution on (v, h) with parameters given by (59),
the optimal sample size is
(4620)
/2(4620
42a
0.00489
'4
14.83
If K=8, a=2,
-
-
0.00489
.
then
n* = 29.66 = 30
(74)
We now consider a prior distribution on (v, h) which is tight on v and loose on
h.
This situation is unlikely in the radar-tracking model, since it amounts to having
a small
prior variance on each of the elements of v, the target velocity, while having
a large
variance on each of the elements of h, which reflects the radar error.
theless, some instructive results follow, so we shall illustrate this case.
Again using
the
expected target velocity and error variance,
459.6
r-l
110
55J
459.6
55
lioj
Never-
-
18
-
we now assume the prior squared coefficient of variation of both
while the prior standard deviation of both
v^
(54), and (57), we obtain the prior parameter
459.6
V
= 6
and v„ is
3
knots.
^_1
h^
.
and h
I
to be 0.5,
Applying (47), (49),
19
["3,941,443.5
3,403,786.8
3,403,786.81
2,939,472.4
(79)
pis, 766.
-36,799.5~|'
90,337.2j
|-36,799.5
7^-1
The posterior expected values of v and h
are
478.7
E[v] = m"
413.4
and
:[h--^]
1433.3
-3345.4
3345.4
8212.5
V"
11
The actual mean and covariance of the process from which the sample
(43) was
drawn is
H
'
120
70
_70
90_
It is clear that being tight on v has prevented the sample from being as
influential on the posterior parameter m" as in the case of the prior with
n'
=0.00489.
Of greater significance is the fact that being jointly tight on v and
loose on h has produced a striking example of the effect discussed in [1], p. 8.
By choosing n'=12,2 and
459.6)
m' = (459.6,
,
certain characteristics were imputed
to the marginal prior on h which were not reflected in our choice of V'
and v'.
These characteristics did not become evident until the sample statistics were
combined with the prior distribution.
that the prior variance of both
h,
,
In this connection it is of interest to note
and h„„ is 6.05 x 10
,
while the posterior
variances are
var[h^j3= 4.57 x 10^
,
var[K22] = 1.50 x 10^.
The sample mean and unbiassed sample variance of (43) are
525.2
85.6
300.6
62.3
::]
-
20
-
Some comments about the radar-tracking model itself are perhaps in order.
The
model was termed a simple one because it concerned itself only with target velocity
and did not permit the target to maneuver.
These restrictions can probably be
removed by an extension of the analysis developed here.
of the model is the assumption that the radar error, ^^
A more serious defect
,
when expressed in
rectangular coordinates, is characterized by a stationary distribution.
The theory
of radar signals indicates that it is reasonable to assume the distribution of the
radar error, when expressed in polar coordinates,
~(k)
p.
and
~(k)
,
\i.\
is stationary.
But, expressing the rectangular components of error in terms of the polar components.
(80)
^f
where 6
(k)
is
>
.
/(^<^>)^ . Gl'h'
3i„(»<^> +
a<«)
,
the target's angular position on the kth scan and
(k)
it is seen that the joint distribution of
the target
s
position.
*
^
{|Ij
•'
,
^^
')
will, in general, vary with
In a computer-controlled tracking system, working space
in memory- and computing time are conserved by using a rectangular coordinate
system, so it is desirable to remove this restriction.
Unfortunately, the present
multivariate Bayesian theory does not extend to the case of non-stationary data
generating processes.
Note that, in the case of targets flying directly toward force center,
which are the most threatening targets, e(k) is virtually independent of k and
the assumption of a stationary distribution is valid.
-
4.
-
21
Leontief s Static Inter-Industry Model
The Leontief static inter-industry model is a system of simultaneous equations
in which the technological coefficients may be regarded as unknown, but fixed,
parameters which must be determined from observations of consumer demand and industrial
output.
We shall use the prior-posterior analysis developed by Ando and Kaufman [2]
for the reduced form data generating process.
To simplify numerical computation, a
two-industry economy is considered.
Let
X.
=
annual output of industry
b.
=
amount per year of the product of industry i demanded
by all sectors of the economy other than industries
1 and 2 (e.g., government demand, consumer demand)
(i=l,2)
)
number of units of product i required to produce one
unit of product j (technological coefficients).
(i,j=l,2)
Hj
Then the annual demand for product
X.
(i=l,2
i
=a..x. +a..x. +b.
11 1
a.
ij
J
1
1
is
i,i=:l,2
'J
,
'
>
which yields the system of simultaneous equations linear in the
x.
(81)
^21
+<l-^22^^2
""l
11
'2
•
^1
^21
n = 1-4
.
Then the equations (81) may be written
n X = b
(82)
.
Equation (82) can be used to determine the annual output of industries
and 2
1
if b is a forecast of consumer demand and the technological matrix A is known.
Alternatively, (82) can be used to estimate
x
and b
,
IT
(and, therefore, A) from observations
of annual industrial output and consumer demand.
Since x^
,
the output
22
-
in the kth year, is based on a forecast of b^
'
,
(82) will not be satisfied exactly by
Let the anticipated demand for the kth year be £
^(k)
p(k)
__
.
(k)
^(k)
(83)
^
'(k)
where b
is the actual demand in the kth year
(a random variable prior to its
~(k)
observation) and v
is an error vector, assumed to have a multinormal distribution
with mean
and precision h, where h is not known with certainty.
assumed mutually independent.
Once
(k)
^
(k)
is fixed, x^
~(k)
Successive v
are
is determined and the actual
demand, before it is observed, will satisfy
b^'^)
= 2 x(^)
-H
v^*^)
(84)
.
It may be argued that technological considerations should suffice to determine the
matrix A.
This, however, may not be possible for a complex industry or, if possible,
such an analysis may be more expensive to carry out than the Bayesian analysis
proposed here.
Moreover, the Bayesian analysis takes forecasting errors explicitly
into account, provided they satisfy the assumption of normality.
It seems reasonable
to ask that the estimated technological coefficients include factors to reflect the
uncertainty in the actual demand.
To illustrate the Bayesian analysis, we shall place a Nqrmal-Wishart prior
distribution on
~
(II,
~
h)
(k)
,
observe values of x^
~(k)
and b^ ' over a period of four years,
and combine the sample information with the prior distribution to find the posterior
distribution of (H, h)
,
following the methods of [2].
Let
ni
-
f^'^^1.
^l£'.
r>
then it is shown in [2] that
V)
I'.
^
23
-
= f^'^^illE',
h a
r) 4^\h\g', V),
has the non-degenerate generalized multivariate
Student distribution,
fsG^(i|p'^ I'' i'' V')
.
This distribution is defined in [2]j its mean and variance-covariance matrix were
derived in Theorem 3.
Using the results of Theorems
prior parameters as follows.
2 arid 3
and Corollary 2, we can assign values to the
~-l
Assume the mean of the marginal prior on h
0.90
.2ol
0.20
.30]
is
(86)
~
~-l
~-l
is 0.01,
and that the variance of h22 in the marginal prior on h
Then, using Theorem 2, the following equations are obtained:
!'
=
(v'-2)
0.90
0.20
0,20
0.30
(87)
2(0.3)-
0.01
v'-4
which yield
e'
22
18.00
4.00
4.00
6.00
(88)
.
If the mean of the prior distribution on A is
A
To. 10
10.90
then, since
E[it]
1.70l
0.2oJ
= u - E[a],
equation (32) yields
(89)
24
1
25
which yields
r.-l
0.00278
0.003501
[o. 00350
O.O3333J
and
V'
r415.0
=
-43.6
(92)
[_-
34
A3.
4
Suppose now that the following data is collected for the years 1955-1958
(details of the way in which this sample was constructed are presented in
Appendix B)
.
The observed industrial output is
ri9.44l
x^^^
,
Tie. 88]
[l0.98j
,
x^^^ =
L^-^^J
.(A)
[23.76]
6.67
L13.90J
(93)
and the observed consumer demand is
(1)
[4.09]
b(2> =
,
[2.99J
12.72
,
[4.25-]
,
b<3) .
r6.49l
[3.54J
[0.89J
,
b(^) =
[4.42]
(94)
[0.O9J
This data yields the following sample statistics:
V=2
6=1
.
m=2
<I)=0
.
4.09
4.25
6.49
4.42
2.99
0.89
3.54
0.09
19.44
16.88
23.76
12.72]
10.98
9.63
13.90
6.67]
(95)
(96)
(97)
k=l
-
-
p" = (X X^)"-^ X
0.60
b''
[1389.1
791. lol
791.1
451. OOJ
= V'-^ X
B*^
(98)
ro.60
-I.I3I
[-0.98
2.O1J
-0.98
(99)
[-1.13
2.O1J
26
and
e =
,l
e^'^)
eW^
e(^>
68.21
29.27
29.27
15.84]
(100)
= b<'^)
-
P X
(k)
Using the prior-posterior analysis of [2], the following posterior parameters
are obtained.
6"=1
(101)
V" = v'-K'+m-+S'+6-5"-<I>-l = 26
(102)
V" = V'+V
1804.18
747 .301
747.50
485 .60]
(103)
P" = [P'V' + P v]y
1
0.93
-1,
0.78
1,
(104)
["86.87
36.4o|
|_36.40
33.4oJ
(105)
After the sample (93) and (94) has been drawn, the posterior expected
values of H, A, and h
are
E[n] = g" =
0.93
-0.78
E[ J]
=
1-5" =
.~-l
(106)
I.37J
[0.07
1.57]
[o.78
-O.37J
(107)
1
13.62
1.52!
1.52
1.39)
•
(108)
The actual parameters underlying the sample (93) and (94) are the technological
matrix
A*
'0.05
1.20
0.40
0.10
^^^^j
-
27
-
and the variance-covariance matrix of the noise vector,
h*"^ =
11.20
0.08
,08
0.50
(110)
The prior expected values of A and K"
are given by (89) and (86).
The posterior mean of A poses a problem if used as an estimate, because a
negative technological coefficient has no physical meaning.
Since the posterior
expectation of A reflects, in part, the forecasting errors, perhaps the negative
element of equation (107) is an indication that the forecasting errors in our
example are excessively large and would not be encountered in a real economy
in the steady state.
It should be noted that the classical least squares
estimate of ^ encounters this same difficulty.
This estimate is, using the
solution (99) to the normal equations implied by (93) and (94),
0.40
0.98
1.13
-1.01
(Ill)
.
Comparison of equations (86), (108), and (110) shows that the posterior
expectation of the variance-covariance matrix of the noise differs significantly
from both the prior expectation and the actual variance matrix.
This, it is
believed, is a reflection of the coupling between the prior parameters on
(n,
b)
described in [1], p. 8, despite the fact that v'=22.
variances of the diagonal elements of h
var[h"}] = 1.19
var[h22] = ^'^^
whereas the prior variances are
var[hj^}] = 0.09
var[h22] = 0.01
.
The posterior
are, using equation (17),
-
28
If the prior parameter v'=52 had been used,
e'
while £* and
V'
45.00
10.00
10.00
15.00
then
(112)
remain as given by equations (90) and (92).
The posterior
parameters are unchanged except that
V"=56
(113)
and
113.87
42.40
42.40
42.40
(llA)
In this case the posterior expectation of h
:[r^]
=
i:
54
?'
2.11
0.79
0.79
0.79
is
(115)
-1
while the posterior variances of the diagonal elements of h
are
var[h^J] = 0.17
varfh^^] = 0.024
.
The coupling effect, while still present, is diminished.
One final point concerning the model of this section is worth mentioning.
The determination of the actual consumer demand may pose a problem, since the
vector
b*-
'
must include demands which were not satisfied, as well as demands which
could be satisfied.
In fact, shortages in the output x
(k)
will affect demand
in that, if shortages are known to exist, some potential demands may never be
presented.
On the other hand, an excess of demand over that forecast will
stimulate further production.
model.
These factors are not included in the Leontief static
APPENDIX A
MEAN AND VARIANCE OF THE INVERTED GENERALIZED BETA DENSITY FUNCTIONS
Non-Standardized Inverted Generalized Beta Distribution
1.
The
X r random matrix V is said to have the non-standardized inverted
r
generalized Beta distribution with parameters (a, b, C) if V has the density
function
nB^sia,
b, 5, =
j--^ .^-^
(AD
where
i(r+l)
p =
a
> i(r-l)
b
> i(r-l)
C is PDS
and the range of V is {y|v is PDS
r^(x) =
„'^('^-i)/'^
j
.
If we define the generalized Gamma function,
r(x)r(x-i)...r(x-^)
where r(x) is the Gamma function, then
r (a)r (b)
.(a,b)
for a
>
|-(r-l)
Theorem Al
;
rj.(a4b)
and b > ^(r-1).
If V is an r x r PDS matrix which has the non-standardized inverted
generalized Beta distribution with parameters (a, b, C)
b%C
E[Y] =
,
b
>
,
then
(A2)
p
and
^°^f^j'\/J
=
a(a+l)
3(b-p);b-p-i)
(^ik^ji^i/^jk)
(A3)
^ a[(l-2a)(b-pH3a]
3(b-p)2(b-p-l)
If i=k and ]=£,
(A3)
*[1], p. 10.
reduces to
^
"J
"^^
,^^^^
-
^^;^^) ,,
,,.
3(b-p)(b-p-l)
var[V.ijJ
.] =
Proof
;
In paragraph 2.5 of [1]
-
30
C. .C,,
11 jj
+
a[ (2-a) (b-p)4-3a]
2
it is shown that,
^
ij'
3(b.p)2(b-p-i)
> p +
1
.
if (V, h) has the joint density
function
D(V, h|a, b, C) = f^^^^Ylh, 2a-r+l) f^'^^hlg, 2b-r+l)
then the marginal density function of V is f^n (Y|a, b, C)
Theorems
,
Therefore, using
.
and 1,
1
m
=
Yvl^^SJ
from which (A2) follows.
E[^ij\/J
=
= 2a
E^[rl]
=
.^^^^
Z,
2b
> r+1
(A5)
Moreover, if 2b > r+3, by Theorem 1, we have
KS^^p^hl^
= '^ ^ht^ith.J+^:Jh-;;+2ah:Jh;;,^]
.
(A6)
Thus, from equations (24) and (26),
^f^ij\/^
=
3(b-p)(b-p-l)
(CijS/-«^ikCj/-^i/Cjk)
from which (A3) and (A4) are obtained.
,
(A7)
Q.E.D.
Standardized Inverted Generalized Beta Distribution
2.
The standardized inverted generalized Beta density function with parameters
where
a
> i(r-l)
b
> i(r-l)
and the range of the r x r random matrix X is (x|X is PDSj, is derived from (Al)
by the transformation
X
= t'^
Vd'^*"
*[1], pp. 10 and 11.
.
(A9)
(A4)
-
Theorem A2
-
31
If the r x r PDS matrix X has the standardized inverted generalized
:
Beta distribution with parameters (a, b), then
b%
E[i] =
and, for
o.,
>
b
.
i
(AlO)
p
P=l,...,r,
var[X^p]
=
var[2
=
^
]
j(b-p)(b-p-l)
^<^-^-P^
(A12)
(b-p)^b-p-l)
'^
r;;
(All)
a^P
'
a[
T
(l-2a)(b-p)+3a]
.
,
,^,
(b-p-1)
3(b-p)
All other convariances are zero.
Proof
:
Equation (AlO) is an immediate consequence of (A9) and (A2)
^
E[|] = l"^ E[V](x'b'' =
By equation (A9)
X
aP
=
^
t'^ CCT'b"' =
I
,
since
.
(A14)
the generic element of X is
,
z
ij
t"
t"
V
ij
Pj
ai
'
therefore, using (A7),
^tW75^
=
ij?k,. T^i ^Pj
^
i
jfk,i
^t
^s/
^t^iA,^
t;} t-J t;J t-J (CijC,^4<:.^Cj^-«:,^c.,)
(a15)
where
K =
But, if 5
g
is
a(a+l)
3(b-p)(b-p-l)
the Kronecker delta,
if J ^ai ^pj ^ij = ^aP
for all a, P=l,...,r.
^t^aP^s]
=
Thus,
•^<^ap^6%,5p5^a6^p,)
= 3K
,
a=P=y=6
=
,
otherwise
=
"^
^
a=P, 7=6, a^7
(A16)
.
-
The case a=& , P=7,
a/^p
is
32
-
the same as a=7 , P=6, a^P, since X is PDS
"I^C^J ' (b-p)^(b-p-l)
Therefore,
(A18)
.
from which (All), (A12), and (Al3) follow.
.
Q.E.D.
APPENDIX B
SAMPLE DATA
1.
Radar-Tracking Model
Suppose the target is on course 060
with speed 600 kts. (cf. Figure Bl).
,
Then,
W^,
if w is the target velocity, rounding to the
nearest integer.
600 cos 30°
=
600 sin 30^
5
(Bl)
[3
Let the covariance matrix of the error in
Figure Bl
the radar-derived velocity be
.20
70
[-70
90_
(B2)
which implies a standard deviation of 10.95 in the horizontal component of
velocity and a standard deviation of 9.49 in the vertical conq)onent.
The
correlation coefficient is 0.673.
Then
^
"
'^1
'
has the bivariate Normal distribution with mean w and variance S.
(B3)
Using the
results of paragraph 8.2.1 of [3], the marginal distribution of X2 is Normal
X9=x^
'2=^2 is Normal with mean
^
=
520+2°
(X2-3OO) = 286.7
90 ^"2
+|x2
(B4)
and variance
^^1^=65.6.
(B3)
34
-
-
With the aid of a table of random numbers uniformly distributed on
a sample of 10 Normal
(0,
1)
random variables, y^
probability integral transformation.
five sample values of X2 were obtained.
:<«
1)
i.lOO y
i=l,...,5.
(B6)
Using these sample values of X2 and the
286.7
I x^^^+
x,
i=l,...,5.
This data is shown in Tables B-1, B-2, and B-3.
Sample
No!
was obtained, using the
samples, five sample values of
(5+i)
1]^
Using the transformation
x^^^= 9.487 y^^^ + 300.00
remaining Normal (0,
,
[0,
were obtained from
(B7)
i
-
n*
,(1)
I-A* =
= =
0.95
-1.20
3 6
-
37
-
The marginal distribution of v- is Normal with mean
and variance 0.50.
Thus, if 72 is N(0, 1),
V2 = 0.707 72
.
This transformation is effected in Table B-5, using sample values 1-4 from
Table B-4.
i
(B16)
38
The noise sample is then
,(1)
-1.21
.(2)
,(3).
•0.25
1.01
0.89
0.59
0.54
,W
0.32
(B20)
-0.81
This sample has mean
2
=
^''' -
Kli
-0.14
(B2I)
-0.10
and unbiassed sample variance
0.63
-O.27I
-0.27
O.91J
(B22)
Combining (B20) with the forecasts (B13) to obtain the actual consumer demand,
bC^) = ^^^^ + v(^>
the vectors b^
.(1)
^
(B23)
are
4.09
2.99
.(2)
4.25
(3).
0.89
B(24)
.W
|0.09j
-
39
-
REFERENCES
[1]
Albert Ando and G.M. Kaufman, "Bayesian Analysis of the Independent Multinormal Process--Neither Mean Nor Precision Known, Part I/' Working
Paper No. 41-63, Alfred P. Sloan School of Management, Massachusetts
Institute of Technology, Cambridge (Revised March, 1964).
[2]
G.M. Kaufman and Albert Ando, "Bayesian Analysis of the Reduced Form System,"
Working Paper No. 79-64, Alfred P. Sloan School of Management,
Massachusetts Institute of Technology, Cambridge (to be published).
[3]
H.
Raffia and R. Schlaifer, Applied Statistical Decision Theory
,
Boston,
(1961).
[4]
T.W. Anderson, An Introduction to Multivariate Statistical Analysis
York, (1958).
,
New
Date Due
OCT IB'TS:
Df C
1
5
46
3
3
TOaO DD3
TOflO
fibT
Elfl
immiimi
iOflO
003 TOO 511
3 ^DflO
003 TOD V?!
3
3
TOfiO
1751
0D3 TOO 203
DD3
flbT
-«
M57
|i|!|''!|:''Mi''''iif7fr'ii't'il|
^i-fev
ll!|f;IMi::lil'i'l'i'i^ill
3
TDSD DD3
fibT
SQ7
iiiiiiiiiiiiiiiililiiiiiiiiiiiiiii
3
TDflD
<?^-b''/
003 TOO SOD
(j^-foV
3
TDflD
003 TOO
^fi4
MIT LIBRARIES
'-''
lillllllllillllliiil
fibT
4bS
TOfiO
003 TOO
Mhfl
TOflO
003
Mfll
3
TOaO 003
3
3
q-l^&>tl
'f?-'M
3
TOflO
003
flbT
flbT
531
<)i;f-
-Ifc
cSni