Chapter 7 Applications to marketing
State Equation:
Sale expressed in terms of advertising (which is a
control variable)
Objective:
Profit maximization
Constraints:
Advertising levels to be nonnegative
The Nerlove-Arrow Advertising Model:
Let G(t)  0 denote the stock of goodwill at time t .
where
is the advertising effort at time t
measured in dollars per unit time. Sale S is given by
Assuming the rate of total production costs is c(S), we
can write the total revenue net of production costs as
the revenue net of advertising expenditure is therefore
.
The firm wants to maximize the present value of net
revenue streams discounted at a fixed rate  , i.e.,
subject to (7.1).
Since the only place that p occurs is in the integrand,
we can maximize J by first maximizing R with to price
p holding G fixed, and then maximize the result with
respect to u. Thus,
which implicitly gives the optimal price
Defining
as the elasticity of demand
with respect to price, we can rewrite condition (7.5) as
which is the usual price formula for a monopolist,
known sometimes as the Amoroso-Robinson relation.
In words, the formula means that the marginal revenue
must equal the marginal cost
. See,
e.g., Cohen and Cyert (1965, p.189).
Defining
, the objective function in
(7.4) can be rewritten as
For convenience, we assume Z to be a given constant
and restate the optimal problem which we have just
formulated:
Solution by the Maximum Principle
The adjoint variable (t) is the shadow price
associated with the goodwill at time t . Thus, the
Hamiltonian in (7.8) can be interpreted as the dynamic
profit rate which consist of two terms: (i) the current
net profit rate
and (ii) the value
of the new goodwill created by advertising at rate u.
Equation (7.9) corresponding to the usual
equilibrium relation for investment in capital goods:
see Arrow and Kurz (1970) and Jacquemin (1973). It
states that the marginal opportunity cost
of
investment in goodwill should equal the sum of the
marginal profit
from increased goodwill and
the capital gain:
Defining
as the elasticity of demand
with respect to goodwill and using (7.3), (7.5), and
(7.9), we can derive ( see exercise 7.3)
We use (3.74) to obtain the optimal long-run stationary
equilibrium or turnpike
. That is, we obtain
from (7.8) by using
. We then set
and
in (7.9). Finally, from (7.11) and
(7.9), or also the singular level
can be obtained
as
The property of is that the optimal policy is to go to
as fast as possible. If
, it is optimal to jump
instantaneously to
by applying an appropriate
impulse at t =0 and then set
for t >0. If
, the optimal control u*(t)=0 until the stock of
goodwill depreciates to the level , at which time
the control switches to
and stays at this level
to maintain the level of goodwill. See Figure 7.1.
Figure 7.1: Optimal Policies in the Nerlove-Arrow
Model
For a time-dependent Z, however,
will be a
function of time. To maintain this level of
, the
required control is
. If
is decreasing
sufficiently fast, then
may become negative and
thus infeasible.If
for all t, then the optimal policy
is as before. However, suppose
is infeasible in the
interval [t1,t2] shown in Figure 7.2. In such a case, it is
feasible to set
for t < t1 ; at t = t1 ( which is
point A in figure 7.2) we can no longer stay on the
turnpike and must set u(t)=0 until we hit the turnpike
again (at point B in figure 7.2). However, such a policy
is not necessarily optimal.
Figure 7.2: A Case of a Time-Dependent Turnpike
and the Nature of Optimal Control
For instance, suppose we leave the turnpike at point C
anticipating the infeasibility at point A. The new path
CDEB may be better than the old path CAB. Roughly
the reason this may happen is that path CDEB is
“closer” to the turnpike than CAB. The picture in Figure
7.2 illustrates such a case. The optimal policy is the
one that is “closer” to the turnpike. This discussion will
become clearer in Section 7.2.2, when a similar
situation arises in connection with the Vidale-Wolfe
model. For further details, see Sethi (1977b) and
Breakwell (1968).
A Nonlinear Extension
Since
, we can invert
a function of . Thus,
to solve (7.16) for u as
We note that
which implies
Figure 7.3: Phase Diagram of System (7.18) for
Problem (7.13)
Because of these conditions it is clear that for a given
G0 , a choice of 0 such that (0 ,G0 ) is in Regions II
and III, will not lead to a path converging to the
turnpike point
. On the other hand, the choice of
(0 ,G0 ) in Region I when
or (0 ,G0 ) in Region
IV when
, can give a path that converges to
From a result in Coddington and Levinson(1955), it
can be shown that at least in the neighborhood of
,
there exists a locus of optimum starting points
.
Given
, we choose 0 on the saddle point path in
Region I of figure 7.3. Clearly, the initial control
u*(0)=f1(0). Furthermore, (t) is increasing and by
(7.17), u(t) is increasing, so that in this case the
optimal policy is to advertise at a low rate initially and
and gradually increase advertising to the turnpike level
. If
, it can be shown similarly that
the optimal policy is to advertise most heavily in the
beginning and gradually decrease it to the turnpike
level
as G approaches
.
Note that the approach to the equilibrium is no
longer via the bang-bang control as in the NerloveArrow model. This, of course, is what one would
expect when a model is made nonlinear with respect
to the control variable u .
The Vidale-Wolfe Advertising Model
Now we can rewrite (7.19) as
The optimal control problem can be stated as
Solution Using Green’s Theorem when Q is Large
To make use of Green’s theorem, it is convenient to
consider times and , where
, and the
problem:
subject to
To change the objective function in (7.24) into a line
integral along any feasible arc
from
to
in
(t,x)-space as shown in figure 7.4, we multiply by dt
and obtain the formal relation:
which we substitute into the objective function (7.24).
Thus,
Consider another feasible arc from
to
lying
above as shown in figure 7.4. Let
,where
is a simple closed curve traversed in the counterclockwise direction. That is, goes along
in the
direction of its arrow and along
in the direction
opposite its arrow. We now have
Figure 7.4: Feasible Arcs in (t,x) - Space
Since
is a simple closed curve, we can use Green’s
theorem to express
as an area integral over the
region R enclosed by . Thus, treating x and t as
independent variables, we can write
Denote the term in brackets of the integrand of (7.28)
by
Note that the sign of the integrand is the same as the
sign of I(x).
Lemma 7.1(Comparison Lemma). Let
and
be
the lower and upper feasible arcs as shown in figure
7.4. If I(x)  0 for all (x,t)R, then the lower arc
is at
least as profitable as the upper arc . Analogously, if
I(x) 0 for all (x,t)R, then
is at least as profitable
as .
Proof. If I(x)  0 for all (x,t)R, then
 0 from (7.28)
and (7.29). Hence from (7.27),
. The proof of
the other statement is similar.
To make use of this lemma to find the optimal control
for the problem stated in (7.23), we need to find
regions where I(x) is positive and where it is negative.
For this, note first that I(x) is an increasing function of x
in [0,1]. Solving I(x)=0 will give that value of x, above
which I(x) is positive and below which I(x) is negative.
Since I(x) is quadratic in 1/(1-x), we can use the
quadratic formula ( See Exercise 7.15) to get
To keep x in the interval [0,1], we must choose the
positive sign before the radical.
The optimal x must be nonnegative so we have
where the superscript s is used because this will turn
out to be a singular trajectory.Since
is nonnegative,
the control
Note that
and
if, and only if,
Furthermore, the firm is better of with larger  and r ,
and smaller  and  . Thus  r/( +) represents a
measure of favorable circumstances.
Figure 7.5: Optimal Trajectory for Case 1: x0<xs
and xs  xT
Figure 7.6: Optimal Trajectory for Case 2: x0 xs
and xs  xT
Figure 7.7: Optimal Trajectory for Case 3: x0 xs
and xs  xT
Figure 7.8: Optimal Trajectory for Case 4: x0 xs
and xs  xT
Figure 7.9: Optimal Trajectory (Solid Lines)
Theorem 7.1 Let T be large and let xT be reachable
From x0. For the Cases 1-4 of inequalities relating x0
and xT to xs , the optimal trajectories are given in
figures 7.5-7.8, respectively.
Proof. We give details for Case 1 only. The proofs for
the other cases are similar. Figure 7.9 shows the
optimal trajectory for figure 7.5 together with an
arbitrarily chosen feasible trajectory, shown dotted. It
should be clear that the dotted trajectory cannot cross
the arc x0 to C ,since u=Q on that arc. Similarly the
dotted trajectory cannot cross the arc G to xT,because
u=0 on that arc.
We subdivide the interval [0,T ] into subintervals over
which the dotted arc is either above, below, or identical
To the solid arc. In figure 7.9 these sub-intervals are
[0,d],[d,e],[e,f], and [f,T]. Because I(x) is positive for
and I(x) is negative for
, the regions enclosed
by the two trajectories have been marked with + or –
sign depending on whether I(x) is positive or negative
on the regions, respectively. By Lemma 7.1, the solid
arc is better than the dotted arc in the subintervals
[0,d],[d,e],and [f,T]; in interval [e,f], they have identical
values. Hence the dotted trajectory is inferior to the
solid trajectory. This proof can be extended to any
(countable) number of crossing of the trajectories; see
Sethi(1977b).
Theorem 7.2 Let T be small, i.e., T < t1+t2 , and let xT
be reachable from x0 . For the two possible Case1 and
2 of inequalities relating x0 to xT and xs , the optimal
trajectories are given in figure 7.10 and 7.11,
respectively.
Proof. The requirement of feasibility when T is small
rules out cases where x0 and xT are on opposite
sides of or equal to xs . The proofs of optimality of the
trajectories shown in figures 7.10 and 7.11 are similar
to proofs of the parts of theorem 7.1, and are left as
exercise 7.23. In figures 7.10 and 7.11, it is possible to
have either t1  T or t2  T. Try sketching some of
these special cases.
Figure 7.10: Optimal Trajectory When T is Small in
Case 1: x0< xs and xT > xs
Figure 7.11: Optimal Trajectory When T is Small in
Case 2: x0> xs and xT < xs
Figure 7.12: Optimal Trajectory for Case 2 of
Theorem 7.1 for Q = 
Solution When Q is Small
where (T) is a constant, as in Row 2 of Table 3.1,
that must be determined. Furthermore, the Lagrange
multiplier u in (7.34) must satisfy
From (7.33) we notice that the Hamiltonian is linear in
the control. So the optimal control is
where
Solution When T is Infinite
We now formulate the infinite horizon version of (7.23)
When Q is small, i.e., Q< us , it is not possible to follow
the turnpike x = xs , because that would require u = us ,
which is not a feasible control. Intuitively, it seems
clear that the “closest” stationary path which we can
follow is the path obtained by setting
and u=Q,
the largest possible control, in the state equation of
(7.39) .
This gives
by setting u = Q, and
in (7.35) and solving for  .
More specifically, we state the following theorems
which give the turnpike and optimal control when Q is
small. To prove these theorems we need to define two
more quantities, namely,
Theorem 7.3 when Q is small, the following quantities
form a turnpike.
Proof. We show that the conditions in (3.73) hold for
(7.44). The first two are obvious. By exercise 7.28 we
know
, which, from definitions (7.42) and (7.43),
implies
. Furthermore
, so (7.36) holds
and the third condition of (3.73) also holds. Finally
because
from (7.38) and (7.43), it follows that
W  0 , so the Hamiltonian maximizing condition of
(3.73) holds with
, and the proof is complete.
Figure 7.13: Optimal Trajectory x0<
Figure 7.14: Optimal Trajectory x0<
Theorem 7.4 When Q is small, the optimal control at
time is given by:
Proof. (a) we set
for all
and note that 
satisfies the adjoint equation (7.35) and the
transversality condition (3.70).
By Exercise 7.28 and the assumption that
, we
know that
for all t . the proof that (7.36) and
(7.37) hold for all
relies on the fact that
and on an argument similar to the proof of the previous
theorem.
Figure 7.13 shows the optimal trajectories for
and two different starting values x(0), one above and
the other below . Note that in this figure we are
always in Case (a) since
for all
.
(b) Assume
. For this case we will show that
the optimal trajectory is as shown in figure 7.14,
which is obtained by applying u=0 until
and
thereafter. Using this policy we can find the
time t1 at which
, by solving the state equation
in (7.39) with u = 0. This gives
Clearly for t  t1 , the policy u=Q is optimal because
Case (a) applies. We now consider the interval [0, t1).
Let
be any time in this interval as shown in figure
7.14, and let
be the corresponding value of the
sate variable. Consider the following two-point
boundary value problem in the interval
In Exercise 7.31 you are asked to show that the
switching function W(t) defined in (7.38) is negative in
the interval
and W(t1)=0.
Therefore by (7.37), the policy u=0 used in deriving
(7.46) satisfies the maximum principle. This policy
“joins” the optimal policy after t1 because
In this book the sufficiency of the transversality
condition (3.70) was stated under the hypothesis that
the derived Hamiltonian was concave; see Theorem
2.1. In the present example, this hypothesis does not
hold. However, as mentioned in Section 7.2.3, for this
simple bilinear problem, it can be shown that (3.70) is
sufficient for optimality. Because of the technical
nature of this issue we omit the details.