i
DEWEY
HD28
.M414
^3
ALFRED
P.
WORKING PAPER
SLOAN SCHOOL OF MANAGEMENT
OPTIMAL CONSUMPTION AND PORTFOLIO RULES
WTTH DURABILITY AND HABIT FORMATION
Ayman Hindy
Huang
Hang Zhu
Chi-fu
Massachusetts Institute of Technology
WP#3616-93-EFA
June 1993
MASSACHUSETTS
INSTITUTE OF TECHNOLOGY
50 MEMORIAL DRIVE
CAMBRIDGE, MASSACHUSETTS 02139
OPTIMAL CONSUMPTION AND PORTFOLIO RULES
WITH DURABILITY AND HABIT FORMATION
Ayman Hindy
Huang
Hang Zhu
Chi-fu
Massachusetts Institute of Technology
WP#3616-93-EFA
June 1993
1
M.I.T.
LIBRARIES
OPTIMAL CONSUMPTION AND PORTFOLIO RULES
WITH DUR.\BILITY AND HABIT FORMATION
Ayman
Hindy, Chi-fu Huang, and Hang Zhu
*
June 1993
Abstract
We
in two different
and habit formation and the combined
effect of durability of consumption goods and habit formation over service flows from those
goods. In a third interpretation, the model captures the idea of a dual purpose commodity.
The optimal allocation problem is from the class of free boundary singular control problems. We
discuss, formally, necessary and sufficient conditions for a consumption and portfolio policy to be
optimal. We aJso introduce a numerical technique bjised on approximating the original program
by a sequence of discrete parameter Markov chain control problems. We provide convergence
results of the value function, the optimal investment policy, and the optimal consumption
study a model of consumption choice and portfolio allocation that captures,
interpretations, the combined eifect of local substitution
regions in the approximating discrete control problems to those in the original continuous time
dynamic program. We construct numerically the consumption boundary that divides the state
space into two regions
one of immediate consumption and the other of abstinence. We show
that both the wealth required to start consuming and the optimal fraction of wealth invested
in the risky asset are cyclical functions in both the stock of the durable good and the standard
of living. This is due to the interaction between the durability and habit formation effects. We
also study the effect of the cyclical investment behavior on the equilibrium risk premium in a
representative consumer economy.
—
'Hindy i« at the Graduate School of Business, Stanford University, Stanford, CA 94305. Huang is at the
02139. Zhu is with the
Sloan School of Management, Massachusetts Institute of Technology, Cambridge,
Derivatives Research Division at CitiBank. We are grateful for useful discussions with Darrell Duffie, Faruk Gul,
MA
John Heaton, Kenneth Judd, Kenneth Singleton, and Jiang Wang. We acknowledge financial support from the
NSF SES-9022937 and NSF SES-9022939.
National Science Foundation under granU
INTRODUCTION AND SUMMARY
1
Summary
Introduction and
1
We study the problem of optimal consumption
and portfolio
rules for
an agent whose preferences
over consumption are given as
E[J\-''ui2{t),y{t))dt],
(1)
10
where
(2)
zit)
=
z{0-)e-'^^
+
(3)
y{t)
=
y(0-)e-^'
+
f3
e-'^^'-'UC{s)
f
and
A /' e-^('-)dC(s).
Jo-
in this formulation, the consumption process C{t) denotes the total
time
t.
and
^
>
The
of consumption
till
processes z{t) and y{t) are derived from consumption using the weighting factors
A, respectively,
>
with
X.
Both ^(0") >
captures the impatience of the agent.
increasing. Furthermore, u
is
strictly
The
and y{0~) >
felicity
function u
are given constants and
is
continuous and strictly
concave and has the property that Ui2
the second cross partial derivative of u with respect to z and
We
amount
>
0,
where Ui2
is
y.
entertain three different economic ideas in three different interpretations of the model
specified in (1), (2),
of local substitution
and
(3). In
one interpretation, preferences given by (1) exhibit the notions
and habit formation. Agents with such preferences treat consumptions
at adjacent dates as close substitutes
and consumptions at distant dates as complements. In
a second interpretation, the model represents habit forming preferences over the service flows
from
irreversible purchases of
a durable good that decays over time. In the third interpretation,
the model represents preferences for consumption of a dual purpose
the agent with two sources of
utility.
The two components
commodity that provides
of such a composite good, however,
have different half-lives.
Local substitution in continuous time, analyzed in Hindy, Huang, and Kreps (1992), Hindy
and Huang (1992) and Heaton (1993),
marginal
utility at
is
the notion that consumption at one time reduces
nearby times. Habit formation, studied since Marshall (1920),
is
the notion
that agents develop tastes because of past consumption experience. Specifically, a high standard
of living in the past increases the appetite of the agent for current consumption. Habit formation
1
INTRODUCTION AND SUMMARY
many
has been studied by
2
authors. Ryder and Heal (1973) introduced the notion of- adjacent
versus distant complementariness.
Stigler
and Becker (1977) emphasize the importance ot
analyzing the endogenous development of preferences in the search for factors that explain
differences in tastes.
Abel (1990), Constantinides (1990), DeTemple and Zapatero (1991),
Heaton (1993), and Sundaresan (1989) study habit formation models.
we study
In this paper,
habit formation.
felicity
The
and
preferences that exhibit a combination of local substitution
distinguishing feature of the preference specification in (1)
is
that the
function u depends only on exponentially weighted averages of past consumption.
particular, the current consumption rate does not
feature, which
is
absent in almost
all
appear directly
in the felicity function.
non-time-additive preferences in the literature,
to representing the notion of local substitution. For the details,
we
is
In
This
the key
refer the reader to Hindy,
Huang, and Kreps (1992) and Hindy and Huang (1992).
Preferences of representative consumers have been used, in general equilibrium models, to
Eichenbaum
explain the behavior of the returns on financial assets. Recent empirical studies of
and Hansen (1990), Gallant and Tauchen (1989), and Heaton (1993) suggest that there
substitution over short periods and habit formation over long periods.
In particular,
is
Heaton
(1993) showed, after correcting for the problem of temporal aggregation, that habit formation
alone does not provide significant explanation power for asset pricing over the time-additive
models; while local substitution, or durability, does. In addition, given local substitution, or
durability, habit formation over long horizons
becomes more
significant in its explanatory power.
These results imply that a model which combines local substitution and habit formation may
better explain the behavior of security returns.
single
consumer
is
an important step
In the durable
a durable good
till
The
in that direction.
good version of the model, we
time
t.
analysis of the optimization problem for a
identify C{t) with the total purchases of
Purchases of the good are irreversible either because there
secondary market for the good, or because of prohibitively high selling costs.
We
new
z{t) of the
good at time
t.
no
invite the
reader to think of such durables as grocery, cloth, small appliances, and, even, vacations.
good provides a flow of services proportional to the stock
is
The
Absent any
purchases, the service flow declines over time because of the deterioration in the stock of the
1
INTRODUCTION AND SUMMARY
durable good.
The standard
We
experience.
take
it
3
of living of the agent
is
given by y{t) and reflects past consumption
that the standard of living deteriorates at a
stock of the durable. This assumption
is
much
reflected-in the restriction that
slower rate than the.
/?
>
A. Discrete
time
model have been studied by Dunn and Singleton (1986), Eichenbaum, Hansen
versions of this
and Singleton (1988) and Hotz, Kydland and Sedlacek (1988), among others.
we take C{t)
In the third interpretation of the model,
f,
of a composite commodity. This
to be the total purchases,
till
commodity provides the agent with two sources of
time
utility.
For example, consider food that provides "calories" and "vitamins", or cloth that provides
and
"shelter"
and
z{t)
"style".
The
y{t).
The two components
half-lives of the
of the dual purpose
two components are
different.
commodity
are captured by
Furthermore, the two sources
of satisfaction are complementary. For example, a higher level of energy increases the marginal
utility of vitamins.
of the paper
we
This feature
will use the
is
captured by the assumption that ui2
>
0.
In the balance
terminology "consumption" and "purchases of the durable good"
interchangeably.
The
specification of preferences in (1) leads to very interesting behavior as well as a tech-
nically challenging optimization problem.
study a model without habit formation,
From the
is
it
Hindy and Huang (1993), who
analysis in
known
that the optimization problem specified
here belongs to the class of free boundary singular control problems.
problem
is
utility of
that consumption occurs periodically.
wealth
is
feature of the
The agent consumes only when the marginal
equal to a linear combination of the marginal utility of the stock of the
durable good and the marginal
for a particular
The main
utility of the
standard of
This condition
living.
combination of wealth, stock of the durable, and standard of
ing for this combination, or free boundary,
is
is
satisfied only
living.
Search-
the essence of solving the utility maximization
problem.
We first
discuss, formciDy, necessary
policy to be optimal.
value function
a
Such
sufficient conditions for
we provide a
— the maximum attainable
differential inequality.
trol
In particular,
and
verification
utility starting
theorem that shows that the
from any
differential inequalities are the
a consumption-investment
initial position
—
satisfies
hallmark of free boundary con-
problems. The verification theorems we provide require that the value function be twice
INTRODUCTION AND SUMMARY
1
W and once continuously differentiable
continuously differentiable in
In general,
Huang
it is
4
(1993) such smoothness could be established because there
problem we study
In the
not currently available.
in this paper,
For this reason, our numerical analysis
theorems for smooth value functions
relies
for conveying the
can be
We
economic intuition of the optimal
utilized in the future
if
two reasons.
for
soliltion.
a closed-form solution
is
is
approximate the
.„
u,
.
Hindy and,
on a weaker notion
known
Zhu
is
for
as viscosity solution,
(1993).
We
record the
First, the discussion
is
useful
Second, the verification theorems
obtained.
The
idea
to approximate the controlled processes, which are continuous-
time diffusions, by appropriately chosen Markov chains on a
utility function in (1)
by one which
approximating Markov chains are chosen to
is,
In
describe a numerical approach for solving the utility maximization problem.
of the numerical approach
tion
-
a closed-form expression
is
described in details in a companion paper by Hindy, Huang, and
verification
y.
such a closed-form solution
the solution of the dynamic programming equation. Such notion,
is
and
to ascertain the smoothness of the value function.
difRciilt
for the value function.
in z
satisfy
is
finite state space. In addition,
appropriate for the
Markov
chain.
we
The
a "local consistency" condition. This condi-
roughly, that the conditional expected change in the
Markov chain and
its
conditional
variance locally match the drift and variance of the original controlled process.
We
approximate the decision problem we study here by a sequence of Markov chain control
problems. Each Markov chain control problem
We
in
the sequence
is
readily solvable numericallyi
then show that the value functions in the sequence of approximating control problems
converge to the value function of the original controlled diffusion problem.
existence of the value function and the feasibility of computing
do not guarantee existence of optimal
its
We
remark that
values with high accuracy
policies that achieve the value function.
Contrast this
with classical feedback control case. In that case, optimal controls can be expressed as explicit
functions of the value function and
its derivatives.
As a consequence, existence of a smooth
value function immediately implies existence of optimal controls.
boundary problem, there
controls, which,
if exist,
is
In our case, with a free-
no direct relationship between the value function and the optimal
are not feedback.
1
INTRODUCTION AND SUMMARY
5
We, hence, show that the sequence of optimal investment
regions in the approximating
originaJ problem,
Markov chain problems converge
the latter exist. In this paper,
if
policies
technique and the convergence results.
and optimal consumption
to the optimal solutions in the
we only sketch without proof the numerical
The companion paper, Hindy, Huang, and Zhu (1993)
provides the details and proofs.
We
when the
provide numerical solutions for the infinite horizon maximization problem
price of the risky assets follow geometric
Brownian Motion.
We
analyze a
u{z,y,t) of the multiplicative form e'^'z^'j/"^ for some constants S,ai, and 03.
convergence results,
the numerical solutions
all
we
discuss here can be
made
function
felicity
Given the
arbitrarily close to
the optimal solution of the original problem. In particulaj, the regions of optimal consumption
and the optimal investment
policies
policies in the continuous time
the solutions
we
we
report are very good approximations of the optimal
problem,
report here "optimal".
if
the latter exist. For ease of exposition,
The reader should remember that such
we
will call
solutions are
rather e-optimal in the sense that they are very close approximations to the optimal solution.
We
compute the optimal consumption boundary W*{z,y). An agent with wealth
of the durable good z and standard of living y such that
W,
stock
W < W'(z,y) optimally refrains from
consumption. Such an agent waits as wealth increases, on average, and both z and y decline
till
the
first
An
boundary.
amount
time that the trajectory of the "state variables" {W,
agent with state variables such that
minimum amount
The
optimally consumes an
and
y.
and then
is
that the consumption boundary
For a fixed standard of living
an agent to start consuming
it
Once on the consumption boundary, an agent consumes
required to keep the state variables from crossing the boundary.
striking feature of the solution
as a function of z
Instead,
> W'{z,y)
the consumption
of consumption that reduces wealth and increases z and y instantly to bring the state
variables to the consumption boundary.
the
W
2, y) hits
is
y,
W'{z,y)
is
cyclical
the critical wealth required for
not a monotone function in the stock of the durable good
2.
increases for a while with increases in z, then declines with further increases in z,
reverses
its
trend and increases again with increases in
cyclical function of the standard of living y for fixed z.
z.
Similarly,
W'{z,y)
is
a
1
INTRODUCTION AND SUMMARY
The optimal investment
assets
— follows a similar
6
policy A*(iy, z, y)
cyclical pattern as
— the proportion of wealth invested
both the stock of the durable good and the standard
^^
of living change. In particular, the partial derivatives
as z
and
change.
y, respectively,
The
in the risky
and
^^
change their sign periodically
cyclical behavior of the critical
wealth level and the
investment policy contrasts with the behavior of an agent in the presence of local substitution,
or durability, but without habit formation. In this case, studied in Hindy
the critical wealth required to begin consumption
Moreover, the proportion optimally invested
The
cyclical pattern in the
is
linear in the stock of the durable good.
in the risky asset
conflicting effects.
more of the durable good
is
is
constant.
consumption boundary and the investment policy
the interaction of the durability and habit formation effects.
good has two
and Huang (1993),
The
reduced.
first is
An
a direct satiating
The second
additional unit of the durable
effect.
The
agent's appetite for
an indirect stimulating
is
from
results
effect.
As the stock
of the durable good increases, the agent's appetite for a higher standard of living increases.
This
the effect of complementarity or habit formation.
is
consume more of the good to increase the standard of
effects also influence the
As a
result, the
The
living.
investment behavior of the agent.
When
agent would
like to
and stimulating
satiating
the satiating effect dominates,
the agent invests a high fraction of wealth in the risky assets since he can afford to tolerate
high losses in wealth.
On
the other hand,
when the stimulating
invests a smaller fraction of wealth in the risky asset.
risk averse
The
manner
dominant, the agent
in this case behaves, in
a more
to protect the standard of living.
interesting feature of the solution
is
that the relative strength of the satiating
stimulating effects alternates as z ajid y change.
effects of
The agent
effect is
consumption
in
a
series of
We
document the
conflict
between the two
graphs that display the variations of marginal
stock of the durable and the standard of living as the state variables vary.
and
We
utilities
of the
also discuss the
long term behavior of optimally invested wealth and optimally followed standard of living and
stocks of durable goods in a population of agents.
the consumption boundary,
agents
who
As a consequence of the
— the
initial differences in life style
ratios
z/W
cyclical nature of
and
y/W —
between
are identical in preferences persist indefinitely. In essence, agents in the population
are divided into distinct life-style classes.
Members
of the
same
class eventually
adopt the same
FORMULATION
2
style.
life
However, a member of one
behaving optimally, does not migrate to another
class,
This contrasts with the anailysis of Constantinides (1990) and Hindy and
class.
in
7
Huang (1993)
which the ratio of the stock of the durable to wealth converges to a steady state distribution
regardless of the initial endowments.
We
We
compute the
show that the
cyclical
model on the determination of the
also study the implications of our
premium
risk
in
an economy with constant stochastic returns to
interaction between the satiating
movement
in
and stimulating
premium changes
risk
change
cyclically
The
in cycles.
scale.
We
consumption leads to
effects of
the risk premium. In the stationary investment environment
risk
premium.
risk
we
study, the
because the attitudes of the representative consumer towards
culprit, of course,
is
the interaction between durability, or local
substitution, with habit formation.
Finally,
we remark
that our analysis
is
confined to the decision of an individual to purchase
a durable good. Constructing aggregate time
many
agents of the type we study
is
series of
purchases
in
an economy populated with
important for understanding the relationships between
aggregate consumption and asset returns. Such an interesting study, however,
is
beyond the
scope of this paper.
The
rest of the
tion problem.
The
paper
is
organized as follows. Section 2 describes the setup of the optimiza-
necessary and sufficient conditions for optimality are discussed in section
The numerical technique and
its
implementation are discussed
in sections
4 and
5, respectively.
Section 6 reports an example of the numerical solution of an infinite horizon program
price of the risky asset follows a geometric
on the determination of the
risk
3.
when
the
Brownian motion. The implications of the solution
premium on
risky assets in equilibrium
discussed in section
7.
world of uncertainty where there
is
is
Section 8 contains some concluding remarks.
Formulation
2
Consider an agent
who
lives
from time
<
= Otof = ooina
a single good available for consumption at any time.
in
a
frictionless securities
indexed by n
=
0, 1,2,...,
market with
A'^.
N+
I
The agent has the opportunity
to invest
long lived securities, continuously traded, and
Security n, where n
=
1,2,
.
.
.,
jV,
is
risky,
pays dividends
at rate
FORMULATION
2
and
Pn{t),
We
t.
assume that
column vector
S{t) to denote the
does not pay dividend, and
riskless,
is
5„(<) at time
sells for
we have used
8
sells for
[5i(t), S2{t),
where
B
/' p{S{s)) ds
Jo
=
one and that
+
5(0)
/' fiiS{s)) ds
Jo
M>N
an M-dimensionaJ,
is
ability space {Cl,^, P).^
We
t
.
.
.,
Security
^
5;v(0]
exp{/Q r(5(5),s)d5} at time
and
r(-)
is
Assume that
for each integer
m
>
0,
,
+
where r{S)
t,
Borel measurable.
/' a{S{s)) dBis)
Jo
Vt €
[0,
where Tt
We
this diffusion process
We
measurable.
assume that
.Fj
oo)
is
strictly positive
a.s..
with probability
there exists a constant c^ so that f^[|5(<)|^'"]
denote this information structure by
the smallest sub-sigma-field of
is
..
.
standard Brownian motion defined on a complete prob-
assume that the agent has only access to the information contained
prices of the risky securities.
locally
is
price process for the risky securities follows a diffusion process given by:
5(0 +
(4)
=
B{t)
the instantaneous riskless interest rate at time
The
can be written as pnC'S'CO)) where
/>„(<)
^
<
with respect to which {5(a);
/",
e*^"*'.^
in the historical
F = {^ui
contains aU the probability zero sets of
<
or
F
S [0,oo)},
<
a
<} is
complete.
is
All processes to be discussed will be adapted to F.*
The agent can consume
finite rates
over intervals.
the single good at "gulps" at any
The agent can
moment, and can consume
from consumption altogether
also refrain
time. Moreover, the sample path of cumulative consumption at any time
component, that
almost
Let
is
some
can have a singular
a continuous nontrivial increasing function whos6^ derivative
is
zero for
all t.
X+-be
the space of
all
processes x whose sample paths are positive^, increasing and
Recall that an increasing function x{u),
right-continuous.
'The superscript ^ denotes transpose.
process V is a mapping Y.ii x [0, oo) —•
^A
?J
that
is
sigma-field generated by !F and the Borel sigma-field of
path and for each
t
£
[0,
oo), Y{.,t):U
—
St is
a
has a
.)
finite left -limit at
any
t
G
measurable with respect to Z"® B([0, oo)), the product
oo). For each u £ Q, Y{iji, .): [0, oo) — R is a sample
[0,
random
variable.
We
will
use the following notation:
If
;x
is
a
be the Euclidean norm of /x. In addition, if <t is a matrix, let \<j\^ denote tr((T<r ), where
the trace of a square matrix. The notation a.s. denotes statements which are true with probability one.
vector in St",
tr is
t
for
at
let
For brevity, we
|/i|
will
sometimes use
/i(t),
p{t), ff(<),
and
r(t) to
denote
/i(5(t)), p{S{t)), <t(5(<)),
and r{S{t)),
respectively.
'The latter can be ensured by a growth condition on [ft — p) and on <r; see Friedman (1975, theorem 5.2.3).
*The process Y is said to be adapted to F if for each t £ [0, oo), Y{t) is J'l-measurable. This is a natural
information constraint: the value of the process at time
^We
t
cannot depend on information yet to be revealed.
use weak relations. Positive means nonnegative and increasing
means nondecreasing.
FORMULATION
2
9
(0,00) denoted by i(a;,t~).
exist for the
The
We
sample paths of any x € X+, a jump of i(u;,
stochastic process
C
£
X+
is
C{u),
t)
at rates C'{u,t),
Finally, C{u},t)
left limits
Ai(w,r) = x{lj,t)-x{u>,t~).
is
are the
to time
in state
t
For any
w.
moments when the agent consumes a
G
a;
fi,
the
"gulp". Moreover,
may have
where C'{u;,t) denotes the consumption rate at time
t
in state
is
lj.
a singular part.
investment strategy
is
an A'^-dimensional process
A =
{A{t)
=
{Ai{t),.
.
.,
A!^{t));t £
00)}, where An{t) denotes the proportion of wealth invested in the n-th risky security at time
[0,
t
at r
Since
has an absolutely continuous component over the intervals during which the agent
consuming
An
t)
.)
a. 5.
a consumption pattern available to the agent with C{u,t)
denoting the cumulative consumption from time
points of discontinuity of C(a',
=
use the convention that x{u,0~)
will
before consumption and trading.
where
1 is
strategy
A
a vector of
I's.
consumption plan
C
invested in the riskless security
G
X^
is
where W{t)
is
wealth at time
t
Wt €
by an investment
left-continuous sample paths and that W{t'^)
The agent
given by (1).
standard of
derives satisfaction
The
felicity
a.s.
00)
[0,
before consumption and Is-\{t)
with the n-th element on the diagonal equal to S{t)~^.
z(0
— A(i)^l,
lo{ms)r{s) + W{s)A^{3)Is-r{s){ti{s) - r(.)5(.))) ds - C{t-)
+ /(J W{s)A'^{s)Is-i {s)(t{s) dB{s),
(6)
said to be financed
is 1
if
mt) = mO) +
(5^
A
The proportion
is
an
X
N
diagonal matrix
Note that the wealth process has
= W{t) — AC{t).
from past consumption and from
function at time
N
t is
final
wealth and
utility is
defined over the stock of durable, z, and the
living, y, defined, respectively, as
=
ziO-)e-'^^
+
e-^^'-'UCis)
a.s.,
A /' e-^('-')dC(3)
a.s.,
(3
f*
Jo-
y(0
(7)
=
y(0-)e-^'
+
No-
where 2(0") >
and ^(O") >
are constants,
/?
and
A,
with
/?
>
A, are
weighting factors, and
the integrals in (6) and (7) are defined path by path in the Lebesgue-Stieltjes sense. Note that
the lower limit of the integrals in (6) and (7)
t
=
0.
is
0~, to account for the possible
Also note that z and y are right continuous processes which
have singular component whenever
C
does.
jump
jump whenever
C
of
C
at
does and
3
NECESSARY AND SUFFICIENT CONDITIONS FOR OPTIMALITY
A
C
consumption plan
sible if (5)
C, and
well-defined^,
is
for all integers
m
A
and the investment strategy
E[J^ e~^*\u{z{t),y{t))\dt] <
>
T
and
€
3*?+,
there
oo,
k^
it
are said to be a,dmis-
where z and y are associated with
so that, for
all
t
< T,
'
.
-
.
E[\Cit)\'^"']
<
e*=-'
Denote by C and
A
the space of admissible consumption plans and trading strategies, respec-
(8)
and
E[|M^(01""]
e*-'.
Formally, the agent manages wealth dynamically to solve the following program:
tively.
E[j^e-''nizit),yit))dt\
supcec
(9)
C
s.t.
is
financed hy
A
£
A
with
and W{t) - AC{t) >
where
<
ejcists
that finances
10
Wq
is
the
initial
W{0) = Wq,
Vt €
[0,
oo),
wealth of the agent and u
the second constraint of (9)
is
is
the felicity function at time
a positive wealth constraint
t.
Note that
— wealth after consumption at any
time must be positive.
3
Necessary and Sufficient Conditions for Optimality
In this section
we
discuss, formally, necessary
and
sufficient conditions for
a consumption-
investment policy to be optimal. In particular, we provide a verification theorem that shows
that the value function satisfies a differential inequality.
Such
differential inequalities are the
hallmark of free boundary control problems. The verification theorem requires that the value
function be twice continuously differentiable in
W
ajid
once continuously differentiable
in z
and
y-
In general,
it is
difficult to ascertain
Huang (1993) such smoothness
for the value function.
In the
the smoothness of the value function.
could be established because there
problem we study
in this
*For this we
mean both
relies
Zhu (1993).
a closed-form expression
on the notion of
We
W to the stochastic
differential
equation (5) for every pair of controls.
is
viscosity
record the verification
the Lebesgue integral and the Ito integral are well-defined.
control depending on (W{t), z{t~),S(i),i) and C(t) depends on the history of
solution
Hindy and
paper, such a closed-form solution
not currently available. For this reason, the numerical analysis
solution as described in details in Hindy, Huang, and
is
In
When
A(t)
is
a feedback
exists a
(W,S), we mean there
NECESSARY AND SUFFICIENT CONDITIONS FOR OPTIMALITY
3
theorems for smooth value functions
for
two reasons.
Firsts the discussion
is
-
11
useful for conveying
the economic intuition of the optimal solution. Second, the verification theorems can be-utilized
in
the future
if
a closed-form solution
obtained.
is
-
.
..
Necessary Conditions
3.1
We
use Bellman's optimality principle to derive necessary conditions about
continuously differentiable, twice in
W,
and once
it
is
is
that the usual Bellman equation in dynamic
in
y and
programming
is
z.
One
derived
J assuming
difficiilty
when
that
that arises
the admissible
consumption plans are purely at rates and are feedback controls. Here, cumulative consumption
can be
in gulps
components,
it
and may have singular
parts. Moreover,
therein for recent
We
work on the
refer the
reader to Zhu (1991) and the references
class of singidar control
problems.
we observe that
First,
if
cumulative consumption has singular
cannot be expressed in feedback form. Thus, our derivation of the necessary
conditions wiU be heuristic in nature.
(10)
if
JiW,
z, y,
5)
=J{W-A,z + /?A, y + AA, 5).
a consumption gulp of size
A
is
prescribed at the state {W,z,y,S). This
quantities are equal to E[ji^ e~**u(z(t), y{t))dt], where {z{t)
,
y{t); s
€
[0,
is
so because both
oo)} are defined along
the optimal path on [0,oo). Moreover, the size of the gulp should be chosen to maximize -J.
Thus, we must have
JwiW-A,z + (3A,y + XA,S)
^''>
=
f3J,{W-A,z-\-PA,y + XA,S) + XJy{W-A,z + PA,y + XA,S),
where JwiJz
a-nd
Jj,
and third arguments,
We now show that
denote the
A
is
partial derivatives of
J with
respect to
its first,
(10) and (11) imply that
is
Jw must
be equal to PJ^ + XJy at any {W,z,y,S)
prescribed. Differentiating (10) with respect to
an implicit function of
W,
y,
and
z defined
W and noticing
through (11) gives
Jw{W,z,y,S)
=
second,
respectively.
where a gulp of consumption
that
first
[-Jw{W -A,z-\-PA,y + AA, S) +
0MW -A,z + 0A,y + AA, S)
NECESSARY AND SUFFICIENT CONDITIONS FOR OPTIMALITY
3
+XJy{W -A,z + pA,y + AA, 5)]— + Jw{W -
=
7w(Vy-A,z +
/?A,y
equality.
Similarly,
A,z + (3A,y + AA, 5) and Jy{W, z, y, 5) = Jy^W - A, z +
is
Jw{^, Zi y, S) =
^Ji(Vr,
z, y,
+
/JA, 5)
+ AA,5),
where we used (11) to get the second
have
A, z
12
S)
we have JziW,z,y,S) = Jzi^W —
/3A, y
+ AA, 5)
.
+ \Jy{W, z, y, 5) at {W, z, y, S) where
Given (11), we then
a consumption gulp
prescribed.
Second, assume that J
any time
t,
is
smooth
sufficiently
for the generalized Ito's
lemma
to applyJ For
the principle of optimality in dynamic programming and the generalized Ito's
lemma
imply that
=
Et [l/^^' e-^'u{z{s), y{s))ds + //+^* e-^'[D^J{s) - SJ^s)] ds
//+^' e-''[0Us) + XJyis) - Jw{s)] dC{s) + ^i[J{r^) " J{ri)\ - E.[^H'(r.)AVr(r.)
max^cA
+
(12)
+J,(r,)Az(r,)
where
dC
Jy(r,)Ay(r,)]
a.s.,
denotes the consumption plan on the time interval
prescribed by
Huang
+
dC on
[f,
i+ A<),
V^J
is
(1993, footnote 8), J{s) and
and we have assumed, without
[f,f
+ Ai),
is
the i-th
the differential operator associated with
its
A
jump
(see
point
Hindy and
derivatives are evaluated at {W{s),z{s~),y{s~),S{s)),
loss of generality, that Ito integral
a martingale and thus vanishes when the conditional expectation
Since no consumption
r, is
always feasible, putting
dC =
appearing in
is
in (12)
Ito's
lemma
is
taken.
and letting At decrease to
zero gives
0>
(13)
u(z,y)
+
max[P'*J] -<5J.
A
This relation must hold for
Suppose
t
is
all
values of
W,
y,
z,and S.
continuity of C, there will not be any gulps on
with no discontinuities on
At,
it
Then
not a point for consumption gulps.
[t^t
+
[t,t
+
enough A, by right-
At). For a nontrivial consumption dC,
Af), to maximize the right-hand side of (12) for small enough
must be the case that Jw{t) = PJzit)
+
AJy(f).
This, together with earlier discussion
^The generalized Ito lemma we will use throughout can be found
Meyer (1982, VIII.27).
Dellacherie and
for small
in
Krylov (1980, theorem 2.10.1) and
NECESSARY AND SUFFICIENT CONDITIONS FOR OPTIMALITY
3
when
there
is
a consumption gulp at
form, can only occur at time
t
t,
implies that nontrivial consumption, independent of
xjy{W{t),
in
z(r ), y(r), S{t))
Jw{W{t),z{r),y{r),S{t)).
Moreover, when optimal cumulative consumption
ments
its
when
0MW{t), z{t-), y(r ), S{t)) +
=
13
dynamic programming show that
is
continuous on
(13) holds as
an equality
+
[t,t
at
t
At), standard argu-
when At decreases
to
zero.
Next suppose that the optimal consumption plan prescribes zero consumption on
For zero
dC
to maximize the right-hand side of (12) for small enough At,
it
[t,t
+ At).
must be the case
that
PMWit), z{t-), y(r ), 5(0) +
xpj.iw{t), z{t-), y(r), s{t))
<Jw{W{t),z{t-),y{n,S{t)).
dC =
Moreover, given that
In
•
consumption gulp
+
at
XJy{t)
+
\Jy{t)
+
following necessary conditions for optimality:
- Jwit) =
maxfP-* J(0] - SJ{t) <
A
u{z{t), y{t))
+
u{z{t), y{t))
+ max[X>^7(0] -
6J{t)
=
+ max[P^ J(0] -
SJ{t)
=
t:
- Jw{t) =
at
t:
XJy{t)
- Jw{t) <
ri{z{t), y{t))
A
These conditions can be written compactly as a
(14)
(13) holds as an equality.
A
no consumption
f3J,{t)
t,
i:
continuous consumption at
(3J,{t)
•
the optimal consumption plan at
summary, we have derived the
l3J,{t)
•
is
max|max[u(r,y)-|-P^J-^J],
flJ^
differential inequality:
+ XJy-Jw\ =
0,
.
NECESSARY AND SUFFICIENT CONDITIONS FOR OPTIMALITY
3
plus the condition that nontrivial consumption occurs only at times
t
where
14
(iJz(t)
+
AJj,(<)
=
Jw{t).
above necessary conditions,
In addition to the
it is
clear that
J must
satisfy the following
boundary coaditioa:
(15)
H
^mJ{W,z,y,S)=
vvio
Condition (15)
is
zero, the only feasible policy afterwards
is
we provide a verification theorem
{W,
z, y, S).
feasible investment
to check the optimality of investment-consumption
We
satisfies
some
regularity conditions, then
then give conditions so that J{W,
and consumption
policy.
It
Proposition
1
Let J
:
If^"*"^
uously differentiahle over
over Ht^"^^ in
its first
N
J{W,z,y,S)=
In addition,
(16)
J
Hir^'^'^
S)
is
in all of its arguments,
(14) with the boundy,
S)
> J{W, z, y, S)
attained by a candidate
z, y,
S)
= J{W, z, y, S)
the optimal policy.
its first
three arguments, contin-
and twice continuously
arguments, except possibly on a smooth manifold
\-'i
differentiahle
M.^
satisfying
boundary condition
f°° e-^^u{ze-^\ye-^')dt
satisfies the
is
—* 5J+ be positive, concave in
the differential inequality of (I4) with the
lim
z, y,
J{W, 2,
then follows that J{W,
and the candidate investment and consumption policy
Kj >
no consumption.
is
We show that if there exists a solution J to the differential inequality
ary condition (15), and J
for all
negative.
Sufficient Conditions
In this section
plans.
become
implied by the constraint that wealth at any time cannot
Thus, whenever wealth
3.2
e-'U{ze-P\ye-'^')dt
Jo
<
00.
growth condition: for every
T
KJ >
G 5J+, there exists
and
so that
\JiX)\<KUl + \X\f^ WX = iW,z,y,S)eR':^^\
Assume furthermore
that there exists a
the associated state variables
consumption policy
W',y*, and
z'
,
such
'For the definition of smooth manifolds see Hindy and
that,
Huang
C £ C financed by A" G A,
putting g
=
inf{<
(1993, footnote 8).
>
:
W{t) =
with
0},
,
NECESSARY AND SUFFICIENT CONDITIONS FOR OPTIMALITY
3
Vt e
(0,^],
P-a.5.
u{z',y')
=
f\jwiW\z-,y',S)-pj\{W',z-,y',S)-XJ,iW',z',y',S)]dC'{s)
=
]imE[e-''J{W'{t),z'{t-),y'{r),S-it))]
=
-SJ + V^'J +
(17)
(18)
15
(19)
(—oo
0,
and, almost surely,
J{W{Ulz{t-),y{t-),Sit,))
(20)
= JiWiU) where
AC'it,), z{t-)
+ PAC'iU), yit-) + XAC'iU), 5(1,))
C
are the times of gulps prescribed by
ti's
on
[0,^).
Then J = J and {C',A')
is
an
optimal consumption and investment policy.
The proof
Proof.
is
similar to that of Proposition 1
and Theorem
Hindy and Huang
I
(1993)
Note that Proposition
requires
1
two conditions.
First,
J
is
positive. Second, the expected
value of J{t) along the optimal path must converge to zero as
latter condition ensures that the agent exhibits
indefinitely without
consumption
convenience and we can replace
is
it
just for the optimal policy but for
As a
his
wealth
not optimzd.
We imposed
is
is
satisfied.
equal to the
all
of
(/?
is
calls for
function immediately before
the former condition for technical
1
outlines general principles that the agent
strictly positive. In particular, the
z, y) in
theorem instructs the
the region where the differential
may consume
only
when marginal
utility of
times) the marginal utility of the service flow and (A times)
the marginal utility of the standard of living.
consumption policy
The
feasible plans.
In addition, the agent
sum
increases to infinity.
by a stronger condition which requires that (19) holds not
agent to use a control policy that keeps the triple {W,
equation (17)
t
enough impatience so that accumulating wealth
recipe for decision making. Proposition
should follow as long as
wecdth
1 in
Finally, in the contingency that the
optimal
a "gulp", the size of the gulp should be chosen such that the value
is
equal to the value function immediately after the gulp.
4
NUMERICAL APPROACH
16
The optimal consumption and
necessary conditions of Section 3.1. Applying Proposition
satisfies all the
know
a priori that the value function
we can use a weaker notion
Zhu
(1993),
requires that one
1
twice continuously differentiable. Absent this knowledge,
is
of solution to the differential inequality (14). In Hindy, Huang, and
we use the notion
such a solution of (14)
4
portfolio policy characterized in Proposition 1 of course
of viscosity solutions, Crandall
and Lions (1982), and show that
indeed the value function of the optimal consumption problem.
is
Numerical Approach
In this section
we
describe the numerical approach utilized to solve the infinite horizon utility
maximization problem.
We
replace the continuous time processes
{W,
z, y)
by a sequence of
approximating discrete parameter Markov chains. The original optimization problem
approximated by a sequence of discrete parameter control problems.
function of each
Markov
We
is
then
show that the value
chain control problem satisfies an iterative, and hence easily pro-
grammable, discrete BeUman equation. Bellman's equation of the discrete control problem
a natural finite-difference approximation to the continuous Bellman's 'nordinear' partial
ential equation.
We
consistent, stable,
also
is
differ-
show that the method of Markov chain approximation produces a
and convergent numerical scheme. Furthermore, the
special monotonicity
and concavity features of the problem allow us to prove convergence npt ordy of the
approximations of the value function J{W^
y, z),
discrete
but also of the corresponding discrete approx-
imations of the optimal policies, when the latter exist.
For numerical implementation, we need to
variables to a finite cubic region
reflection processes Lt, Rt
[0,
Mw]
x
[0,
restrict the
My] X
[0,
domain of
definition of the state
M^]. For this purpose, we introduce the
and Dt at the boundaries of the cubic region.
We
hence consider the
following modified state variables:
XdCt — dLt
dyt
=
—Xytdt
-\-
dzt
=
-fiztdt
-I-
dWt
=
Wt{r +
Atifi
where
Lf, Rt
lidCt
-
,
- dRt
r))dt
,
and
- dCt + WtAtadBt - dDt
,
and Dt are nondecreasing and increase only when the state processes (W,z,y)
hit
4
NUMERICAL APPROACH
W = Mw, z = M^, and y = My, respectively.
the cutoff boundaries
We
whose
will
assume that the
price process
17
riskless rate r
is
-
constant and that there
is
a single risky asset
a geometric Brownian Motion given as
is
S{t)+ f p{s)ds = 5(0)+ I fMS{s)ds+ I aS{s)dB{s),
Jo
where n >
Jo
and a are positive constants and where B{t)
r
Since the investment environment
of wealth
W,
is
z, y, t)
=
We
a constant impatience parameter.
We
it
henceforth by J{W,
will solve
a standard Brownian Motion.
J depends only on the
the standard of living y, and time
since the felicity function
will focus
is
t.
It is
level
also
time separable with
our analysis on the function J{W,
z, y, 0)
and
z, y).
the following infinite horizon program on the restricted domain:
E
Max
(21)
z,
J{W, z, y, 0)
e~
is
stationary, the value function
the stock of the durable good
easy to see that J{W,
denote
Jo
e-^'uiyt,zt)dt
= J^{W,y,z)
Jo
subject to the dynamics of z and y and the dynamic budget constraint. Bellman's equation for
dynamic program takes the form:
this
(22)
max{-(5 J^ + max{£^ J^} + u(y,
on
Afvv]
[0,
operator
We
X
L^
[0,A/y]
is
x
[0, A/^],
given by
£^ = \W^A'^ct^^ + W[r +
—
A{fi
- r)]^ - Xy^ - /3z^.
—
QjM
=
0,
and
respectively, at the cutoff boundaries
reflect the
0,
and add to (22) the boundary constraints
QjM
0,
XJ^ + pj^ - J^} =
together with the appropriate boundary conditions, where the
follow the lead of Soner (1986)
QjM
__
=
z),
W
=
= Mw-,
0,
y
= My, and
z
= Mj. These
constraints
asymptotic behavior of the value function on the original unbounded domain.
worthwhile to remark that
the value functions
J^
in the theoretical analysis of
It is
convergence of the numerical scheme,
converge pointwise to 7 as the sequence of restricted domains Q\f
increases to Q, regardless o/ the specification of the behavior of
boundary conditions we chose, however,
7^
at the boundaries.
affect the quality of the actual
The
numerical solution.
NUMERICAL APPROACH
4
The Markov Chain Scheme
4.1
We
18
introduce a sequence of grids
Gh = {ik,iJ):W = kxh,y = ixh,z =
where h
Each
z
=
is
the step
grid point
(fc,
size,
i,j)
and where Ny
€
j X h. For simplicity,
we introduce the space
jxh;0<k< Nw,0
= My/h, N^ = M^/h, and
corresponds to a state {W,
Gh
we take the same
step size in
z) with
j/,
all
< i< Ny,0 <
Nw
=
Mw /h
j
< N^}
,
are integers.
W = kxh, y = ixh, and
three directions. For a fixed grid,
of discrete strategies
ACn = {(A,C);A = /x A,AC = OorA;0 </ < Na}
A
where
artificial
We
is
the step size of control,
bound to be relaxed
N^ = A/ A
in the limit as
A
i
is
and
approximate the continuous time process {W,
Markov
chains {{Wl^,y^,z^);n
=
either
is
Na
z, y)
x
A
of control steps with
an
t oo-
by a sequence of discrete parameter
h, the chain
and
has the property that, at
a choice between investment and consumption. At any time, a chain can
make an instantaneous jump, (AC =
h), or follow
a "random walk",
neighboring states on the grid. At the cutoff boundaries the chain
in
A
1,2, •••} with h denoting the granularity of the grid
n denoting the steps of the Markov chain. For every
each step n, there
number
total
a manner consistent with
its
dynamics
in the interior of the
is
(AC =
0), to the
reflected to the interior
domain. Fix a chain and
its
corresponding grid size h. The transition probabilities for this Markov chain are specified as
follows:
1.
The case of no consumption
The
chain can possibly
states: {k
—(AC = 0):
move from
+ l,i,j),{k-l,i,j),
the current state {k, i,j) only to one of the four neighboring
{k,i- l,j),and {k,i,j -I). For any investment policy
the transition probabilities in this case are defined as
pak,i.JUk-ui,j)] =
^^^-^
A =
/x A,
4
NUMERICAL APPROACH
19
P;^[ik,iJ),{k,i-hj)]
=
Pi^[{k,i,j),{k,i,j-l)]
=
Q^^y
Pl^[{k,i,j),{k,i,j)]
=
l-Pi^[{k,iJ),{k+l,i,j)]-PI^[ik,iJ),ik-\,i,j)]
QHk,ijy
and
-P(^[{k,i,j),ik,i-l,j)]-P^[(k,i,j),ik,i,j-l)]
where the normalization factor Q^{k,i,j)
Q''{k,i,j)
The
= max [k^h\l
X
is
given by
A)V + kh'^[r + lxA{n + r)] + (At + f3j)h^].
recipe for these transition probabilities
is
a
slightly modified version of those suggested
by
Kushner (1977).
Furthermore, we define the incremental difference
AW^ =
Ty^^.1
— W^. At
Ay^ =
y^^i
-
y^,
Az^ =
—
z^^^
2^,
and
step n of the chain, with the previously defined time scale A„f'', one
can verify that
=
=
=
=
i:„^[Ay^]
^n[^^n\
,23N
^
^/^[AH^n']
i;^[AH^„^
-
£;^[AVy„^]]2
-AyAt^-AL^
-PzAt),-ARi,
M^[r + A(/i-r)]At^-AZ)^ and
V^2^V2At^ + C»(A<i),
where E^ denotes expectation conditional on the nth-time state
reflecting processes
W^
AX^, Ai2j, and AjD^
(H^,J, y^, 2^),
equal to the positive value h only
reach their respective boundaries. This implies that the
first
2.
The case of consumption
The
(fc
Markov
-
chain
1,
i
+
+
/3).
direction vector
(
— l,A,/3)
and
We
call this
property
chain.
direction
(— l,A,/3) from the current state {k,i,j) to the state
However, the later
except in the trivial case A
y^, 2^,
— (AC = h):
jumps along the
A, J
when
and second moments of the
Markov chain approximate those of the continuous process {Wt,y,,Zi).
"local" consistency of the
and where the
=
1
=
/?.
state, {k
—
1,
t
+
A, j
+
/?), is
usually not on the grid
For ease of programming, we take the intersection of the
with the corresponding surface boundary and randomize between
three grid points adjacent to the intersection point.
We
randomize
in such
a way that the
4
NUMERICAL APPROACH
expected random increment
will
20
be along the direction
(
— l,A,/3) and
of length^equal to the
distance from the starting state to the point of intersection.
Without
we assume
loss of generality,
hereafter that A
<
1
<
In this case, the intersection
/3.
occurs inside a triangle spanned by the three grid points {k,i,j
{k
-
l,i
+
l,j
+
I).
The
(24)
=
Pfp,z,j),(fc-l,t,j+l)]
=
^,
^,
Pf[(fc,i,j),(A:-l,i+l,i+l)]
=
^,
AC =
changes
in
3
is
4.2
—
l,i,j
+
1),
and
and
verify the property of "local" consistency:
£„^[Az^]
=
and
/3AC,
E^^lAW!:]
= - AC
the continuous time process for a consumption increment of the same magnitude
of the
Markov chain are depicted
Figure
1.
Markov property
in
in
The Markov— Chain Decision Problem
policy
{A,C)
is
admissible
if it
conditional on
where
(k
the increment of consumption. These quantities correspond to the respective
AC. The movements
A
we can
£^[Ay^] = AAC,
where
1),
transition probabilities can be defined as:
P^[{k,i,j),{k,iJ+l)]
In this case, also,
+
W = kh and W =
preserves the
!;V^.'^J"/<'.
k'h, y
=
ih
and
for the discrete
parameter Markov chain
(25)
=
7''(fc,t-,i)
inax Et..
is
y'
=
i'h, z
/
=
j7i
\
and
that
Pk[(f^^i^J)dk\i',j%
z'
=
j'h.
The
if
AC =
control problem
then to solve the program:
f] ^"""«(3/n,4)^'n,
n=0
where
f^
= IZo<Kn ^^'' and
sense that the
sum
in (25)
Af''(Jb, t,j)
=
h} IQ^{k,i,j). Note, this
approximates the expected integral
is
in (21).
analogous to (21)
in the
h
,
4
NUMERICAL APPROACH
The
discrete
21
dynamic programming equation now takes the
J>^{k,iJ) .=
^
•
m^^{Zk;,',:'P^[{k,i,j),{k\i',j')]JHk',i',j'),
maxo<,<N.
(26)
form
iterative
{e-*^'''^*''-^) Eit',-,/
P^^Kk,
.
r-:
.
i,j), {k', i',j')]J'{k', i\j')]
+ UiiJ)At\k,i,j)]
for {k,i,j)
6 Gn, with iterative reflection at the
artificial
boundaries, where
P^[*,*] and P^[*y*] are the transition probabilities of the chain. Now,
D-
Klc
i\
i
,•>.
n+r/L
DJJik,
r,2
^k
Using
•
-x
j)
J,
_ J{k,i+l,j)-J{k,i,j)
=
x
jri.
•/(,«,»,;)
=
^
^
J(k
+
_ Jik,i,j+l)-J(k,i,j)
,
+
Dj J{k,i,j) -
,
D^
l,i,j)-J{k,i,j)
+
us denote
l,i,j)-2J{k,i,j)
.
J{k,
+ J{k -
t,
j)
,
= J{k,i,j)-J{k-l,i,j) and
,
l,i,j)
•
j^
this notation,
"
J{k
recall that
_ J{k,i,j)-J{k,i,j-l)
- Ak,iJ)-Jik,i-l,j)
j(L.
n+ J{k,t,j)=
V,•
let
we
we can
express the discrete Bellman's equation (26) in the form:
_g-5A«''(fc,.J)^
J'
+ max{CtJ\k,i,j)} + UiiJ) <
0,
<
0,
At''(fc,i,j)
AZ?t7''(fc-l,i,j+l)
£^ = hv'^A^o'^Dl +
+
/3Z)t7''(fc,i,j)-D^j''(fc,i,j+l)
W^(r
+ AyL)Dt - WAtDI - XyD; -
with
fizDj.
Furthermore, one of these two inequalities must hold as an equality at each (k,i,j) € G^.
Clearly, (1
— e~^^' )/^^^ approximates
the discount factor A as
/i
—
0.
As a
result,
the above discrete differential inequalities are a finite-diflference approximation of Bellman's
equation (22) for the majcimization problem on the
At the cutoff boundaries, we specify a
refer the reader to Hindy,
4.3
finite
domain [0,Afw] x
slightly different finite-difference
Huang, and Zhu (1993)
[0,Afy]
X [0,Mj].
approximation.
We
for the details.
Convergence of the Markov chain Approximation
In this section,
we
state,
without proofs, the convergence results of the Markov chain approxi-
mations. Specifically, we state that the value functions in the sequence of
Markov chain
control
NUMERICAL IMPLEMENTATION
5
22
problems converge to the value function of the continuous time problem on the bounded dom£un. Furthermore, the optimal portfolio policies and the optimal consumption regions in the
sequence of the Markov chain decision problems converge to those of the continuous time problem,
technical details
Theorem
difference
size
h
For an extensive discussion of the sense of convergence and for the
the latter exist.
if
and proofs, we
The
1
discrete
hi, hj)
Bellman's equation converges
h
Furthermore,
[ 0.
—
{W,
(22).
z, y).
to the value
there exists
if
Hindy, Huang, and Zhu (1993).
Markov chain approximation
scheme for Bellman's equation
such that (hk,
I
refer the reader to
yields a consistent
and
stable finite-
Consider a sequence of grids and
The sequence of solutions J^{k,
let the
grid
i,j) of the discrete
function J{W, y, z) of continuous-time problem as
an optimal policy {A',C') for the original problem, then
as h I 0, the sequence of approximate policies A^{k,i,j) converges to the optimal investment
policy
A'{W,z,y).
Finally, if the point
{W,z,y)
is
in the optimal
consumption (abstinence)
region in the original problem, then almost all grid points that converge to {W,
the
5
z, y) will be in
consumption (abstinence) regions of the corresponding Markov-chain problems.
Numerical Implementation
we
In this section,
the discrete
describe briefly the implementation of the numerical scheme for solving
Markov chain program. We designed the numerical scheme
to reflect the iterative
nature of the discrete Bellman's equation:
J'ik,ij)
At each
(AC =
state,
0). If
we
=
mB^{^,.^i,j,P^[{k,i,JUk',i'J')]j\k',i',j')
+
Tn^xo<i<NA^-^^''j:k',,-,j'PH[{k,iJ),ik',i'J')]j'^{k,i,j)}+Uii,j)At}.
first
consumption
J\k,ij)= J^
Otherwise,
need to choose between consumption
is
chosen,
(AC =
h) and no consumption
we use the consumption scheme
P^[{k,i,jUk',i',j')]j\k',i'j').
we use the investment scheme
J''(fc,t,j)=
max
e-^^'
^
P;^[{k,iJUk',i',j')]j\k,i,j)^U{i,j)At
-
NUMERICAL IMPLEMENTATION
5
where /'^[,
and
*]
PJp[*,
*]
The numerical algorithm
space and approximation
.
—
are given in section 4.
is
23
a combination of two iterative schemes: approximation in policy
in value space.
in the space of control policies, while the
The
method can be viewed
first
as
a "descent" method
second method calculates the (n+l)-step value function
with the updated policies from the previous iteration.
Specifically,
1- Guess an
=
Ao(fc,t,j)
is
we implement
initial
=
the following steps.
value function Jo{k,i,j)
ACo(fc,t,i) for {k,i,j) 6
K
=
Gh where
x U{i,j) and
simply the utility function.
compute
a constant, and U{i,j)
is
=
as
u{y,z)
''
2- Given the n-step value function {J^{k, i,j)
ACJJ(A;,t, J )},
>
/if
set the initial policies
at each state (fc,t,j)
At each {k,i,j) 6 Gh, compute the
:
{k, i,j)
G Gh} and the n-step
€ Gh the updated
maximum
policies {A^(fc,
i,
j),
policies {A!|^^^{k,i,j),AC^^^{k,i,j)}.
attainable utility from investment as
max^e-^^'{p,^[(fc,z\j),(fc+l,i\j)]JiJ(fc+l,t\i)+P,^[(fc,i,i),(/:-l,i,j)]J„^(fc-l,f,j)
+Pl^[{k,i,JUk,i-l,j)]Jil{k,i-l,j)
+
P^[{k,i,j),ik,i,j-l)]j:i{k,i,j-l)
+P^[{k,i,j),{k,i,j)]j!lik,iJ)}+Uii,j)At\
where
Af''
is
the interpolation time and P'*[*,
Meanwhile, we also compute the
utility for
consumption by
^jl^{k,ij+i) + ^j![{k-i,i,j +
which equates the marginal
utility of
sumption and the marginal
utility of living
are the transition probabilities of the chain.
*]'s
i)
+ ^j!^{k-i,i+i,j +
i)
we3dth to a combination of the marginal
standard. Then, update the (n
+
utility of con-
l)-step policies
by choosing
A>:,^,{k,i,j)
and AC^^i(fc,i, j)
reverse
is
true,
€ argmax^=;,^,o</<;.^{e-'^'''
=
if
the utility from investing
E
is
P^[ik,iJ),{k',i'J')]J::{k',i'J')},
higher than that from consuming.
we choose AC^^i(fc,i,j) = h and An^i(k,i,j)
is set
3- With the updated policy (>l^+i, AC^+j), we now evjJuate the
(n
+
l)-step value function J^^i- In this step,
first
to
If
the
some arbitrary number.
utility functional to
obtain the
update the transition probabilities P-^[*,*]
5
NUMERICAL IMPLEMENTATION
according to the
new
Then
policies.-
24
at each {k,i,j)
6 G°, compute Jn+ii^^hj) by solving a
linear system:
+P/f[(fc,i,j),(/:-l,i,i)]J„\i(*-l,t,i)
+PI^[{k,iJ),ik,i- l,j)]J„Vi(fc,»- l,i)
+P,^p,»,i),(fc,t,j- l)]J„V(fc,t,i-
1)
+P/^[{k,i,j),ik,i,j)]j!^^,{k,i,j)}
+ U{i,j)At\
where
At'^ =^'/Q""^^(fc,i,j),
and Q^'^^{k,i,j)
is
the normalizing constant as in the previous
step but with the (n + l)-step updated policies An+i{k,i,j).
is
prescribed at {k,i,j) £
G°
(i.e.
AC^^i(fc,i,j)
=
h),
On
the other hand,
if
consumption
compute Jn+i{k,i,j) by solving the
following different linear system:
+ ^j!^+,{k-l,i+l,j +
l)
Finally, the evaluation at the cutoff-boundary can be obtained according to the reflection rules
specified in Hindy,
at
A;
=
is
Huang, and Zhu (1993,
§5.4).
Throughout the
chosen to satisfy the boundary condition at W^
=
entire procedure, the value
0.
4- Compute the roundoff error
1
If
ERR{n) <
€,
results taken
bls
increase n by
1
the desired precision
is
achieved and the program
is
terminated with the
the value j'^{k,i,j) a.nd the optimal policies {A'^{k,i,j),C'*{k,i,j)}. Otherwise,
and return to steps 2 and
3.
Here,
e
>
is
the tolerance error prescribed in the
program.
Computing the value function
equations.
We
use a relaxation
at every iteration requires solving a large linear
method
to solve this system rather than a
system of
method that
requires
6
NUMERICAL SOLUTION
25
computing the inverse of a matrix. The large
size of the involved three
The usage
renders the latter algorithm numerically very expensive.
is
dimensional matrices
of the relaxation technique
guaranteed to produce accurate results because of the fact that the matrices describing the
linear
system to be solved define a contraction mapping as explained
(1993, §6.1).
The numerical
in
Hindy, Huang, and Zhu
results are discussed in the following section.
Numerical Solution
6
we report the optimal consumption and
In this section,
program when the
utility function is
u{z,y,t)
=
e~*'z'*'y"^, with
discount factor 6 expresses the impatience of the agent.
an
portfolio rules in
The
<
a,-
<
infinite
1,
=
t
horizon
1,2.
The
differential inequality (14) simplifies
to
max
(27)
The boundary
(28)
condition
J(0,.,v)=
We
to the cube
function
is
is
[0, 1]
x
[0, 1]
X
well defined for
life
+
\Jy -
Jw\ =
0.
is
.
[0, 1].
restrict the
domain of the state variables {W,z,y)
Observe that the multiplicative specification of the
strictly positive values of z
all
increasing in both z and y.
ceteris paribus.
pj,
problem numerically and
Abel (1990). The
A
and
y.
Furthermore, the
similar multiplicative specification
utility function is
who
analyze the form u{z,y,t)
only well defined
when
z
>
y.
For 2
<
=
y,
e~*'(z
was introduced by
the consumption rate c{t) at time
time satisfaction of an agent
is
t.
The
Wq
some 7 <
for
1.
utility function.
when
/3
=
00 and hence
difference utility function implies that the
lower, the higher
specification requires that initial wealth
— yy,
we can take u{z,y,t) = —00.
Constantinides (1990) and Sundaresan (1989) study the special case
is
utility
This contrasts with the "difference specification" studied by Constantinides
This extension maintains the concavity of the
z{t)
utility
time satisfaction of the agent increases as the standard of living increases,
(1990) and Sundaresan (1989)
This
6 J,
W=
when
^^^^;^^^
solve this
function
+ max[P^J] -
Iz^'y"'
is his
standard of
living.
Furthermore,
be at least equcd to zo/r for the
life
time
life
this
utility
6
NUMERICAL SOLUTION
26
mjLximJzation problem to have a feasible policy that keeps -the consumption rate higher than
the standard of living at
all
An
future dates.
to maintain the initial standard of living.
an agent
initial
— oo. For
is
agent
As a
interest rates in the range
standard of living
who
result, the
2% — 20%, an
utility
and
(5), (6),
and
(7), respectively, are linear.
of degree qi
principle,
=
=
result, the value function
^)Vxj
condition
J
is
y,
given in
homogeneous
In
for all positive k.
number
of state
For example, we could divide by the stock level z and write
We
could then deal with the
reformulated problem with two state variables, however,
—
and
new
state variables xi
=
^
^ and rewrite Bellman's equation in terms of xi and X2-
is
not easier to analyze com-
is
Let the value function in the reformulated problem be z°'^'^°''V{xi,X2).
optimality condition
As a
As a
z,
the homogeneity of the value function to reduce the
z°'^'^°^J{W/z,l,y/z).
putationally.
{Px2
life
homogeneous of degree
J{kW,kz,ky) = k°^'^°''J{W,z,y)
In other words,
we could use
J{W,z,y)
A
qj.
from three to two.
variables
and X2
+
In addition, the
is
dynamics of the state variables W,
in 2
y.
50 times the
domain of the state space where the
that the felicity function u{z,y,i)
02
Qi
—
meaningful.
is
we proceed, we observe
Before
+
optimization problem
agent requires 5
wealth to avoid the extremely painful consequences of the
in financial
difference specification. This requirement restricts the
time
Wq < zq/t is certain to fail
maximum life time utility of such
starts with
—
/3{ai
Jw — 0Jz —
+ a2)V
^Jy in the original formulation corresponds to (1
in the
new
formulation. Note that in the'
new
The
+ Pxi)Vx^ +
formulation, this
expressed as a linear partial differential equation with zero-order term /?(ai -\-a2)V.
result, the
numerical solution of this equation
is
susceptible to numerical instabilities that
require very small grid size or the use of implicit techniques. For
merical schemes,
we
refer the reader to Celia
original formulation does not contain
and Gray (1992,
more on the
§4.2).
The same
any zero-order terms and hence
is
stability of nu-
condition in the
immune
to numerical
instability.
There
is,
hence, a tradeoff in choosing one of the two theoretically equivalent formulations
of the problem.
memory
free of
On
the one hand, the three dimensional formulation requires more computer
to process the three dimensional grid.
numerical instability.
On
The numerical algorithm, however,
the other hand, the two dimensional formulation
is
simple and
demands
less
.
NUMERICAL SOLUTION
6
27
storage because of the reduction in the
number of
state variables. This^'formulation,- however,
requires special techniques to handle numerical instability. Such techniques reduce the speed of
convergence of the algorithm.
We
elected to solve the
problem numerically
dimen-
in its three
sional formulation after our initial experiments suggested that the three dimensional numerical
procedure converges faster, for the same accuracy, than the corresponding two dimensional one.
We
the case in which the standard of living
is
which A
also tested the numerical procedure by solving the special case in
is
boundary that determines the region of consumption
W = k'z.
straight line
The
critical ratio k'
lem and ranges from 10 to 20
straight line free boundaries
for typical
0.
This
constant and does not change with the level of
consumption. Hindy and Huang (1993) report the closed form solution for
free
=
depends on
in the state
all
this
problem. The
space {W, z)
is
given by a
the parameters of the decision prob-
parameter values. The numerical procedure produces
whose slope agree with the analytically computed k' to an accu-
racy of 10"^. Such computations were performed on the original two dimensional formulation,
rather than on the equivalent one dimensional reformulation, of the decision problem.
Optimal Consumption Rules
6.1
We
present examples of the numerical solutions.
we chose
In these examples,
the following
parameter values
r
= 6%
/i
Given theorem
= 12%
1, all
CT
= 23.1%
A
=
0.4
/?
=
7.0
and
numerical solutions we discuss here can be
^
=
0.5
made
arbitrarily close to the
optimal solution of the original problem. In particular, the regions of optimal consumption and
the optimal investment policies
in the
we report
are very
good approximations of the optimal
continuous time problem. For ease of exposition, we
"optimal".
The
will call
the solutions
we
policies
report here
reader should remember that such solutions are rather e-optimal in the sense
that they are very close approximations to the optimal solution.
Figures 2 and 3 display the optimal free boundary in the case of a\
a\
=
free
0.2 ,Q2
=
0.8, respectively.
The
striking feature in
both figures
is
=
0.8
,
Q2
=
0.2
and
the cyclic shape of the
boundary. For a stock of the durable good z and a standard of living
y, let
W'{z^y) be
the corresponding critical level of wealth on the consumption boundary. If wealth
W
is
such
6
NUMERICAL SOLUTION
that
W
28
> M^*,the optimal behavior of the agent
is
to
consume immediately the amount of
wealth that brings the' state variables to the consumption boundary.
W < W, the optimal behavior
and y to decline
to
is
when the
until the first time
Subsequently, the optimal policy
is
consume nothing and to wait
to
W
2:
consumption boundary.
level
corresponding to the durable stock and the
living.
y,
the critical value of wealth
monotonically increasing nor monotonically decreasing
cyclical in z for a fixed y. Similarly, the figures
values of
wealth to grow and for
state variables reach the
For a fixed level of the standard of living
is
on the other hand,
consume the minimum amount required to keep the
of wealth from exceeding the critical value
standard of
for
If,
Hindy and Huang (1993). In that
consumption
of the stock
is
is,
show that W'(z, y)
is
Monotonicity of the
is
neither
W'(z,
y)
cyclical in y for fixed
=
0,
analyzed in
case, the critical wealth level that determines the states of
monotonically increjising
the higher
is
in 2. Instead, the critical value
This contrasts with the case of constant standard of living, A
z.
W
in the stock of the
durable good.
The
higher the level
the level of wealth required for the agent to start consumption.
critical level
W
in the case of
non-changing standard of living
is
a
consequence of the tradeoffs that the agent faces. In that case, a decision to consume would
increase the level of the durable
K the
marginal
utility of
the optimal decision
is
good and simultaneously reduce the
level of financial wealth.
wealth exceeds the marginal utility of the stock of the durable good,
to refrain
Otherwise, the optimal choice
is
equalize the marginal utilities of
from consumption to increase the expected
consume to increase the
to
W and
level of
level of wealth.
the durable good and
z.
Fix the level of wealth. By concavity of the indirect
utility function,
the higher the level
of the durable
good stock
relative to the
marginal value of one additional unit of wealth. In other words, the higher the
value of the stock
the durable stock.
levels of
Such
is,
is,
the lower
the more useful
As a
it is
the marginal value of an additional unit of the good
to increase expected wealth rather than the level of
result, higher levels of the stock of the
wealth before consuming
is
is
is
durable good require higher
optimal.
not the case with the introduction of the standard of living that increases with con-
sumption. In that case, consumption increases both the stock of the durable and the standard
NUMERICAL SOLUTION
6
of living
and reduces
29
is
determined by the quantity
is
equal to zero.
When Jw
Jw —
{PJz
exceeds the
sumption to increase the expected
+
The agent consumes only when
AJ^).
sum
of jJJ^
and XJy,
living
numerical results show that J^y
is
optimal to refrain from con-
it is
y.
The
and the stock of the durable good introduces
that the second partial derivatives u^y
effects. Recall
this quantity
wealth and reduce the levels of both z and
level of
complementarity between the standard of
new
whether the agent optimally consumes or not
financial wealth.- Recall that
also strictly positive.
is
strictly positive.
As a
Furthermore, our
result, as the level of the stock of
the durable good increases, so does the marginal value of an increase in the standard of living.
This complementarity
effect
the source of the cyclical form of the consumption boundary.
is
how
Fix the level of wealth and the standard of living and consider
the agent change as the stock of the durable good increases.
durable good
z, ceteris
two opposing
paribus, has
effects.
The
An
the tradeoffs facing
increase in the stock of the
a satiating direct
effect.
a stimulating indirect
effect.
first is
An
increase in z reduces the marginal utility J^.
An
increase in z increases the marginal utility Jy because of the complementJirity
formation
effects.
The
an increase
total effect of
durable good relative to
W and
The second
in z
is
depends on the
size of the stock of the
W,
Sometimes, the relative values of
y.
z,
and y are such that
sum
the satiating effect dominates and an increase in z leads to a reduction in the
relative to Jyy.
As a consequence,
times, the relative values of
W,
refraining from
z,
and y
cire
consumption
is
Jw-
In this case,
/SJ^
the optimal choice.
-|-
XJy
Other
such that the stimulating, complementarity-
induced, effect dominates and an increase in z leads to an increase in the
relative to
and habit
sum PJz + XJy
consuming the amount required to equalize J\y with pjz
+
XJy
is
the optimal choice.
The important
feature of the interaction between the stimulating and the satiating effects
of increasing the stock of the durable good
is
that relative
as z increases. For low levels of z, the satiating effect
the
sum
/37j
+
XJy.
and further increases
effect regains
For higher values of
in z increase the
dominance and increases
the alternating behavior of the
sum
z, ceteris
sum pjz +
in z
fij^
+
dominates and an increase
in
in z decreases
paribus, the stimulating effect dominates
XJy. For
reduce the
XJy
dominance alternates between them
still
sum
higher values of
(3Jz
+
Figures 4 and 5.
XJy.
We
z,
the satiating
present samples of
NUMERICAL SOLUTION
6
'
-An
30
interesting consequence of the cyclical form of the consumption -boundary
section of agents with identical levels of wealth
and standard of
is
that a cross
living will exliibit alternating
patterns in their consumption behavior. Agents with relatively low levels of the stock of the
durable good
will
consume because a
unit of
consumption
of the durable good. In their case the value of J^
Agents with relatively high
different reason. In their case, the
In essence, a unit of consumption
durable good
marginal value of Jy
is
a vaJuable addition to their stock
high enough to encourage consumption.
is
levels of the stock of the
is
is
will also
consume but
for a
high enough to induce consumption.
a value increment to their standard of living. Agents with
intermediate levels of the stock of the durable good have relatively low levels of both J^ and Jy.
Their optimal policy
'
wealth
is
to refrain from consumption on the grounds that increasing financial
is
more valuable than
Since agents
increasing either the stock of durable or the standard of living.
consume because of different
with identical levels of
W and
y,
reasons,
impossible to infer which of two agents,
it is
has a higher level of
z.
Figures 2, 3, 4, and 5 reveal, and a detailed examination of the numerical results confirms,
sum ^J^
XJy
is
cyclical in the value of the standaird of living y for fixed
The economic reasoning
is
the same.
that the
has two opposing
-\-
effects.
stimulating indirect eifect
The
is
An
W and
z.
increase in the standard of living y, ceteris paribus,
satiating direct effect
is
due to the natural decline
due to the habit formation
in Jy.
effect that increases Jj, the
The
marginal
utility of the
durable good, as the standard of living increases. The reFative strength of these
two opposing
effects alternates as y increases
as the standard of living increases.
and hence the sum
The consequences
cyclical effect of z. In particular, a cross section of agents
durable good wiU reflect this cyclical pattern in
its
(3Jz
are also the
+
XJy changes
same
in cycles
as those from the
with identical wealth and stock of the
consumption behavior.
Specifically, agents
with relatively high and low standards of living consume, albeit for different reasons, whereas
agents of intermediate standards of living optimally invest in the financial asset and are content
to derive utility from the current stock of the durable and the current standard of living.
It is
easy to see from the figures, and the numerical results substantiate, that the same
cyclical effect
Xi
=
is
^ and X2
present in the equivalent two state variable formulation of the problem. Define
=
^. Define the
consumption boundary corresponding to xi, ijCxi), as
follows:
NUMERICAL SOLUTION
6
if
the state variables
amount required
the
31
and 12 are such that xj > ^2(^1), the optimal policy
x-i
to bring xi and 12 to the consumption boundary. Otherwise,
not to consume and wait
till
to
consume
it is
optimal
the trajectory of (ii,X2) reaches the consumption boundary.-*The
numerical results show that the consumption boundary ij
's
a cyclic function of ij.
also interesting to consider the long run behavior of the optimally controlled state
It is
variables in a population of agents. In the case of constant standard of living, A
Huang
is
(1993, §7)
show that the
ratio of the
optimaUy controlled wecdth,
=
0,
Hindy and
W, and stock of the
durable good, z', reaches a steady state distribution. After long enough time, the distribution
of the ratio
W
enough time,
/z*
all
is
independent of the
initial
conditions Wq/zq. In other words, after long
agents would have probabilistically the same level of the durable good, relative
to wealth, regardless of the discrepancies in their initial starting condition.
This result of eventual similarity of the optimal ratio of the stock of durable to the level of
wealth does not obtain
in the
presence of a standard of living that changes with consumption.
Refer again to figures 2 and 3 and observe that the state space below the consumption boundary
W'{z,
y)
is
divided into
diflferent distinct regions. In
there are three distinct regions which
regions where
it is
for
label as regions
optimal not to consume.
the optimal policy
and
we
is
the restricted state space
If
I, II,
and
III.
[0, 1]
X [0,
1]
X [0,
1],
Recall that these are the
an agent starts inside region
I,
example, then
for
to abstain from consumption and wait for wealth to grow, on average,
both z and y to decline
till
the trajectory of the state variables {W,z,y) reaches the
consumption boundary. At that time, the agent consumes the amount required to keep the
state variables
From
{W,
z, y)
from moving across the consumption boundary.
the geometry of the consumption regions,
state variables,
it is
clear that
if
I, II,
and
III,
and from the dynamics of the
an agent starts inside one of those regions, the trajectory of the
state variables will remain inside that region during the period of
no consumption. Furthermore,
once the trajectory of (W,z,y) reaches the boundary, consumption
to the inside of the region from which
it
originated.
As a
result,
one of the regions, the optimal behavior confines the values of
all
will reflect
that trajectory
once the agent starts inside
future state variables to be
inside that particular region.
The
cyclical
shape of the consumption boundary divides the state space into distinct regions.
6
NUMERICAL SOLUTION
As a
result, the
levels of
W,
z,
32
population of agents
and
y.
Once
is
divided into classes that depend on the initial -starting
a particular
in
an agent can not migrate to another
class,
because of the shape of the consumption boundary.
An
class
agent that starts in the consumption
endowment
region with wealth higher than the critical level corresponding to his
of z and y
takes an immediate gulp of consumption and moves to a point on the consumption boundary.
The entry
he
point into the region of no consumption determines the class of the agent to which
be confined forever.
will
Although agents are identical
in their preferences, initial variations in
stock of the durable good, and standard of living persist indefinitely.
into distinct classes.
The members
tion of stocks of the durable
may become
of each class
good and standard of
more, a member of one
It is
born
class
worth mentioning that
is
may
The population
and
y/W
vary
among
classes.
diffusion.
We
this eternal
if
However,
Further-
confinement of an agent to the class in which he was
The
price of the
amounts over short periods of time because we assumed
conjecture that
divided
is
not emigrate to another class even after a very long time.
a result of our assumptions on the available investment opportunity.
risky asset changes in small
of wealth,
eventually similar in the distribu-
living relative to financial wealth.
z/W
the eventual distributions of the optimal ratios
endowments
the investment opportunity admits large
an agent has a chance to migrate to another
class.
jumps
it
to be a
in wealth,
then
For tractability, we limited our analysis to
asset prices of the diffusion family. Introducing a price process of the jump-diffusion family
an interesting further direction of
6.2
is
this research.
Optimal Investment Rules
In this section,
we
present the optimal proportion of wealth
A' invested
in
the risky asset.
From
Bellman's equation, direct computation shows that
(29)
A-(Vi',z,v)
=
-^!^.
In the case of constant standard of living studied in
fraction of wealth invested in the risky asset
is
Hindy and Huang (1993), the optimal
a constant that does not vary with the level of
wealth or the stock of the durable good. The constant fraction invested in the risky asset
higher for higher durability of the good (lower
/?),
for lower interest rate r,
and
is
for lower rate
6
NUMERICAL SOLUTION
33
of discount ^.'Furthermore, as the durability effect weakens
(/3
the optimal fraction of
f oo)
wealth invested in the risky asset converges to the constant fraction used by an investor with
time additive
utility
In the current
over consumption rates as given in Merton (1971).
model with the standard of living that changes with the
the fraction invested in the risky asset
not a constant.
is
level of
consumption,
Furthermore, the optimal fraction
of wealth invested in the risky asset exhibits cyclical behavior as the value of the stock of the
durable and the standard of living change. Specifically, the partial derivatives
cyclical functions in z
durable good
z,
and
y, respectively.
Fixing the level of weaJth
^^
^^
are
W and the stock of the
the numerical solution reveals that, for Small values of y relative to
and hence the optimal fraction A' declines
and
as y increases. In this low range of y,
W, ^^ <
an agent with
a higher standard of living invests more defensively, ceteris paribus, than another agent with a
lower standard of living. In other words, the agent with the higher standard of living behaves
more
in a
risk averse
manner
to protect his standard of living.
^^
In a higher range of y,
>
and hence as the standard of
investment proportion A' increases, ceteris paribus.
in this range, the agent with the higher
and
invests
more
living standard.
^^
<
The same
For two agents with standard of living
standard of living acts in a less risk averse manner
aggressively in the risky asset.
behavior changes and
For stiU higher values of
cyclical behavior of the
optimal investment policy
A'
^^ >
A' with wealth, however, does not
variation of
the investment
declines with further increases in z.
W increases, ceteris paribus. A'
declines.
is
exhibited as
initially increases as z increases, ceteris paribus.
z reaches a critical level,
The
y,
reversing the trend for investment in the preceeding range of
the stock of the durable good varies. A'
0.
living increases, the optimal
A
For
still
higher levels of
exhibit this cyclical behavior.
richer individual
As
z,
As
would invest a lower fraction of
wealth in the risky asset than, an otherwise identical, poorer individual.
Note that, over the long run, each individual goes through cycles of investment behavior.
During the periods of no consumption, wealth increases, on average, while z and y
The investment
The
result
is
policy of the agent adjusts continuously as the state variables
frequent upward and
downward
{W,
z, y)
decline.
change.
revision of the fraction of wealth invested in
the risky asset. This cyclical investment behavior occurs, even in the stationary environment
6
NUMERICAL SOLUTION
we
analyze, due to-the interaction of habit formation
34
and
durability effects.' This behavior
contrasts sharply with the- constant fraction policies analyzed by
time additive
utility,
by Grossman and Laroque (1990)
Hindy and Huang (1993)
in
Merton (1971)
in the case of
in the case of transaction costSj
the case of durable goods. This behavior
is
also different
monotone behavior reported by Constantinides (1990) and Sundaresan (1989).
and by
from the
In both studies
the fraction of wealth invested in the risky asset declines as the standard of living increases
relative to wealth.
To understand
where A'(Ty,
z, y) is
given in (29).
JwJwWy]- However, from
Jw >
and
0.
—JwwJwy
'^'^'
the sources of the cyclical investment behavior, consider the sign of
The
^-^^^
sign of
is
the same as the sign of
JwJwWy
'Furthermore, the term
Hence, the sign of
^^'
dy'
is
numerically
much
,
[—JwwJwy +
we know that
the properties of the value function J,
^^
Jww <
0,
smaller than the term
approximately, the same as the sign of
Jwy
Figures 6 and 7 display typical behavior of the marginal value of wealth as the state variables
{W,
z,
y) vary.
Consider the variations of the marginal value of wealth J\y as the standard of living y
changes.
Figures 6 and 7 show that
habit formation effect.
An
Due
for further
is
a cyclical function of
y.
increase in the standard of living y has
The
the marginal utility of wealth.
wealth.
Jw
This
two
is
a result of the
conflicting effects
direct satiating effect tends to reduce the
on
marginal value of
to this effect, increasing the standard of living reduces the appetite of the agent
improvements
of wealth. There
is
in
the living standard.
Hence, the agent discounts additional units
also an indirect stimulating effect.
agent's appetite for the durable good increases. This
is
As the standard
of living increases, the
the habit formation effect.
Due
to that
increased appetite, the agent places high value on additional units of wealth.
The
relative strength of the direct satiating effect
As a
result,
^^
<
and
^^
<
in that range.
variables, the indirect stimulating effect dominates. Hence,
The
relative
relative to
dominance of the
W and
z,
changes.
indirect stimulating effect change
At some values of the state variables, the direct satiating
as the state variables (W,z,y).
dominates.
and the
direct
and
The same
effect
For other values of the state
-^^ >
and -g— >
in that range.
indirect effect alternates as the standard of living,
intuition explains the variations of
A' with
z.
The
7
EQUILIBRIUM RISK PREMIUM
cyclical variation of J\y with z
we
In this section,
displayed in figures 6 and
is
7.
'
-
We
use the representative
agent framework of Cox, Ingersoll, and Rxjss (1985) and presume that there
technology whose rates of returns are given by a
conclude that the equilibrium riskless rate,
=n
where J
is
(7)-Brownian Motion with ^ and a
(/x,
r, is
given by:
the indirect utility function for an agent
all
investment
is
in the risky
the parameter values n
equilibrium risk
risk
whose preferences were analyzed
utility function u{z,y,t)
=
premium
23.1%, A
in Figures 8, 9,
premium
=
and
figures show, the equilibrium risk
premium
is
under the
—
an economy with a repre-
r in
we studied
the case
0.4
10.
=
and
Table
1
7.0.
We
compares
utility as in
report samples of the
tlfe results
with those in
Merton (1973) and with
and Huang (1993).
with the case without the habit formation
the risk
/x
in section 6. Specifically,
economies with a representative agent with time additive
durability effects as in Hindy
life-time utility
Furthermore, we chose, somewhat arbitrarily,
e~°'^'2°"'y°®.
= 12%, cr =
who maximizes
production technology.
computed numerically the equilibrium
As the
strictly
(j%
sentative agent
with
a risky production
Jw
constraint that
We
is
Using arguments similar to those of Cox, Ingersoll, and Ross (1985), we
positive scalars.^
r
->,'
-•.
consumption and investment behavior we
discuss the implications of the
studied in this paper on the determination of the risk premium.
(30)
•
Premium
Equilibrium Risk
7
35
premium
effect
is
not constant.
analyzed by Hindy and
This result contrasts
Huang (1993)
in
which
a constant. Furthermore, in that case, the risk premium increases as the
durability effect, as captured by ^, decreases. Durability
lead to cyclical behavior of the risk premium.
and habit formation
As our discussion
effects
combined
in section 6.2 reveals, the
attitudes of the investor towards risk change with changes in the level of wealth relative to both
'More
specifically,
the level of risky capital, V{t), at time
evolves according to the equation: dV(i)
motion.
=
tiV(t)
t
starting from one unit continuously reinvested
dt+aV^t) dB{i), where
B
is
one dimensional standard Brownian
7
EQUILIBRIUM RISK PREMIUM
Model
36
8
CONCLUDING REMARKS
Concluding Remarks
8
In this paper
we provide a complete
portfolio choice for
In
37
an agent with
analysis of the
problem of optimal consumption and
utility functions that
admit three
different interpretations.
one interpretation, the preferences of the agent exhibit both local substitution and habit
formation. In a second interpretations, the model represents habit formation over the service
flows from irreversible purchases of a durable good.
preferences for consumption of a dual purpose
We
model represents
commodity that provides two sources of
provide numerical techniques for solving this optimization program which
class of free
boundary singular control problems. Our technique
control problem by a sequence of controlled
and Zhu (1993), provides
many
In a third version, the
all
chains.
A
from the
based on approximating the
companion paper, Hindy, Huang,
the technical details. Free boundary control problems appear in
We
areas in economics.
Markov
is
is
utility.
hope that the technique we presented
finds use in
many
other
applications.
References
9
1.
A. Abel, Asset Prices Under Habit Formation and Catching
ican
2.
M.
Up
with the Joneses, Amer-
Economic Review Papers and Proceedings 80, pp. 38-42, 1990.
Celia and
W.
Gray, Numericsd Methods for Differential Equations, Prentice-Hall,
Inc.,
1992.
3.
G. Constantinides, Habit Formation:
of Political
4.
J.
Cox,
Prices,
5.
Economy
J. Ingersoll,
A
Ame. Math.
Premium
Puzzle, Journal
98, pp. 519-543, 1990.
and
S.
Ross,
An
Intertemporal General Equilibrium Model of Asset
Econometrica 53, pp. 363-384
M. Crandall and
Resolution of the Equity
,
1985.
P.L. Lions, Viscosity solutions of Hamilton-Jacobi equations.
Soc. 277, pp. 1-42, 1982.
Trans.
9
REFERENCES
6.
J.
Detemple and
tion,
7.
38
Econometrica. 59, pp. 1633-1658
,
with Habit Forma-
1991.
-
C. Dellacherie and P. Meyer, Probabilities and Potential B: Theory of Mertinagles
Holland Publishing Company,
8.
An Exchange Economy
F. Zapatero, Asset Prices in
K.
Dunn and K.
New
,
-
North-
York, 1982.
Singleton, Modeling the
Term Structure
separability of Preferences and Durability of
of Interest Rates
Under Non-
Goods, Journal of Financial Economics 17,
pp. 27-55, 1986.
9.
-:
M. Eichenbaum and
L.
Hansen, Estimating Models with Intertemporal Substitution using
Aggregate Time-Series Data, Journal of Business and Economic Statistics
8, pp. 53-69,
1990.
10.
M. Eichenbaum,
L.
Hansen and K. Singleton,
A Time
Series Analysis of Representative
Agent Models of Consumption and Leisure, Quarterly Journal of Economics CIII, pp.
51-78, 1988.
11.
A. Gallant and G. Tauchen, Seminonparametric Estimation of Conditionally Constrained
Heterogeneous Processes: Asset Pricing Applications, Econometrica 57, pp. 1091-1120,
1989.
12. J.
Heaton, The Interaction between Time-Nonseparable Preferences and Time Aggrega-
tion,
13.
Econometrica 61, pp. 325-352, 1993.
A. Hindy and C. Huang, Optimal Consumption and Portfolio Rules with Durability and
Local Substitution, Econometrica 61, pp. 85-121, 1993.
14.
A. Hindy and C. Huang,
On
Intertemporal Preferences for Uncertain Consumption:
A
Continuous Time Approach, Econometrica 60, pp. 781-801, 1992.
15. A.
Hindy, C.
The Case
Huang and D. Kreps, On Intertemporal
Preferences in Continuous
Time
of Certainty, Journal of Mathematical Economics 21, pp. 401-440, 1992.
I:
16. A. Hindy, C.
Problem
17. J.
in
Huang, and H. Zhu,- Numerical Analysis of a Free Boundary Singular Control
Financial Economics, Working Paper, 1993.
.
.
Hotz, F. Kydland and G. Sedlacek, Intertemporal Preferences and Labor Supply, Econo-
metrica 56, pp. 335-360, 1988.
18.
N. Krylov, Controlled Diffusion Processes, Springer- Verlag,
Kushner, Proba.bility Methods
19. H. J.
Elliptic Equations,
Kushner and
20. H. J.
lems,-
21. H. J.
SIAM
F. Martins, Numerical
Kushner and
,
P.
Stochastic Control and for
Methods
6,
for Stochastic Singular Control Prob-
pp. 1443-1475, 1991.
Dupuis, Numerical Methods for Stochastic Control Problems in
Springer- Verlag,
22. A. Marshall, Principles of
23. R.
ia
York, 1980.
Academic Press, New York, 1977.
Journai on Control 29, No.
Continuous Time
Approximations
(or
New
New
York, 1992.
Economics, Eighth Edition, London, 1920.
Merton, Optimum Consumption and Portfolio Rvdes
in
a Continuous
Time Model,
J.
Econ. Theory 3, pp. 373-413, 1971.
24. H.
Ryder and G. Heal, Optimal Growth with IntertemporaDy Dependent Preferences,
Review of Economic Studies 40, pp. 1-31, 1973.
25. G. Stigler
and G. Becker, De Gustibus Non Est Disputandum, American Economic Review
67, 1977.
26. S.
Sundaresan, Intertemporally Dependent Preferences and the Volatility of Consumption
and Wealth, Review of Financial Economics
27. T. Zariphopoulou,
Chain Parameters,
Investment-Consumption Models with Transaction Fees and Markov
SIAM
J.
Control Optim., 30, pp. 613-636, 1992.
28. H. Zhu, Variational Inequalities
trol,
2, pp. 73-89, 1989.
and Dynamic Programming
for Singular Stochastic
Con-
Doctoral Dissertation, Division of Applied Mathematics, Brown University, 1991.
39
(k+l.i,j)
Y+
(k
h
(k,i,j)
^
.'T
-
h
(k',i',j')
(k.i-i.j:
W-h
^^^^ij+ij+ry
(k-l.i.j)
(k-l,i,j+l)
A- The Case of No Consumption
•
Czise
A
At state
{k,i,j),
when
the four neighboring states: {k
+
B- The Case of Consumption
AC =
0,
the
l,i,j), {k
—
Markov chain moves to one
- l,j), {k,i,j —
l,i,j), {k,i
of
1)
with the appropriate probabilities.
• Czise
B
At state
{k,i,j),
neighboring states:
Figure
1:
when
AC
=
h, the
Markov chain jumps
to the
implemented by randomizing among the three
{k,i,j + 1), {k - l,i,j + 1) and {k - l,i + l,j + 1).
intersection (k',i',j'). This
is
Local Transitions of
The Controlled Markov Chain
40
1.0 -^
Z
-
Consumption
Y
0.5
-
Living Standard
0.0
Figure
2:
The optimal Free Boundary
41
for u{z,y)
_
=
,0.8 ,,0.2
2
y
Z
-
Consumption
Y
0.5
-
Living Standard
0.0
Figure
3:
The optimal FVee Boundary
42
for u(z,y)
=
z°'^y°-^
Marginal
lambda
J
Utility at
W = 0.10
+ beta J
Z-Cons
tandard
0.0
Figure
4:
Samples of The Alternating Behavior of The
43
Sum
[ij^
+ XJy
—
I.
Marginal
Utility at
W = 0.30
0.85
0.80
0.75
Z
-
Consumption
Y
-
Living Standard
0.0
Figure
5:
Samples of The Alternating Behavior of The
I
44
Sum
[iJ^
+
\Jy
—
II.
Marginal
Z
-
Utility
—
J
at
W
= 0.100
Consum
Standard
0.0
Figure
6:
Samples of The Variations
45
in
Marginal Value of Wealth
—
I.
Marginal
Z
-
Utility
—
J
at
W = 0.300
Consumption
Y
-
Living Standard
0.0
Figure
7:
Samples of The Variations
46
in
Marginal Value of Wealth
—
II.
Equilibrium Risk
Premium
W = 0.30
Z
-
Cons
Standard
0.0
Figure
8:
Samples of The Equilibrium Risk
47
Premium— I.
Equilibrium Risk
Premium
W = 0.40
Z
-
Consum
Standard
0.0
Figure
9:
Samples of The Equilibrium Risk Premium
48
—
II.
Equilibrium Risk
Premium
W = 0.50
Z
-
Cons
Standard
0.0
Figure 10:
Samples of The Equilibrium Risk Premium
49
—
III.
293
'66
Mil
3
TOflO
LIBRARirS
DOflSbMflM
fl
Date Due