The Maximum Principle of Optimal Control:
A History of Ingenious Idea
and Missed Opportunities
Hans Josef Pesch 1, Michael Plail 2
1 University
of Bayreuth, Germany
2 Steinebach, Wörthsee, Germany
Optimization Day, University of Southern Australia,
Adelaide, Australia,
January 29, 2011
Outline
• Carathéodory‘s Royal Road of the Calculus of Variations
• Hestenes‘ secret report and first formulation
• Bellman‘s and Isaacs‘ regrets
Missed Opportunities
to the Maximum Principle of Optimal Control
• Pontryagin and his students: adoration and embitterment
Hans Josef Pesch, Roland Bulirsch:
The Maximum Principle, Bellman‘s Equation, and Carathéodory‘s Work
J. of Optimization Theory and Applications, Vol. 80, No. 2, Feb. 1994
Hans Josef Pesch, Michael Plail:
The Maximum Principle of Optimal Control: A History of Ingenious Idea
and Missed Opportunities
Control and Cybernetics, Vol. 38, No. 4A, 973-995, 2009.
Carathéodory‘s Royal Road in the Calculus of Variations
Relationship between
Hilbert‘s Independence Theorem
and
Hamilton-Jacobi Equations
allows the reduction of
Problems of the Calculus of Variations
to
Problems of Finite Optimization
Carathéodory‘s Royal Road in the Calculus of Variations
Lagrangian problems: precursors of optimal control
Search for
-curves
that extremize
subject to implicit differential equations
DOF: n - p
controls
for line elements
with
of curves
Carathéodory‘s Royal Road in the Calculus of Variations
Stage 1: Definition: extremal (minimal or maximal)
Different from today‘s terminology: weak extremum / minimum / maximum
closer neighborhood
Stage 2: Proof of necessary Legendre-Clebsch condition
or
in today‘s terminology for minimization
has a positive definit Hessian
for fixed
Carathéodory‘s Royal Road in the Calculus of Variations
Stage 3: Caratheodory‘s equivalent variational problems
Let
then
independent of
Carathéodory‘s Royal Road in the Calculus of Variations
Let
adding a null Lagrangian
Then: integration along two curves yields
Thus
and therefore any line element
where
can be passed by one and only one minimal curve
Carathéodory‘s Royal Road in the Calculus of Variations
Stage 4: Caratheodory‘s existence result for a special problem
If there exists
with
for all
and all
then the solutions of
with
are extremals of
Carathéodory‘s Royal Road in the Calculus of Variations
Stage 5: Caratheodory‘s sufficient condition
If there exists
for which there hold
and
for sufficiently small
yield
, then the solutions of
Carathéodory‘s Royal Road in the Calculus of Variations
Hence we have to determine the functions
such that
or
(as function of
) possesses a minimum for
with
value
C‘s fundamental
equations:
(Carathéodory, 1935)
No imbedding
or extremal
That
is the so-called
Bellmanfields
Equation
on Carathéodory‘s Royal Road
Carathéodory‘s Royal Road in the Calculus of Variations
Stage 6: Caratheodory‘s formulation of Weierstraß‘ condition
Substituting the fundamental equations and replacing
by
yields
Hence we obtain the necessary condition of Weierstraß
Carathéodory‘s Royal Road in the Calculus of Variations
Stage 7: Lagrangian variational problems
Similarly
Defining
the canonical
Hamiltonian
in
canonical coordinates
Introducing
the
Lagrange
function
variables
the fundamental equations take the form
with
and yields
and solving these equation for
the Weierstraß necessary condition takes the form
(Carathéodory: 1926)
Exit to the Maximum Principle?
Exit to the Maximum Principle from C‘s Royal Road
Recall Caratheodory‘s Hamiltonian
Carathéodory‘s closed approach to optimal control (from 1935)
degree of freedom:
control?
control
Today‘s Hamiltonian
call them controls
Exit to the Maximum Principle from C‘s Royal Road
With the maximizing Hamiltonian for
and the costate
we obtain
as long as
By means of the Euler-Lagrange equation
canonical
equations
and because of
Exit to the Maximum Principle from C‘s Royal Road
Furthermore
Hence,
with respect to
must have a maximum
along a curve
From here it is still a big step to
Missed Carathéodory the exit?
Constantin Carathéodory (1873 - 1950)
1932
1904
Constantin Carathéodory (1873 - 1950)
• Born in Berlin to Greek parents, grew up in Brussels
(father was the Ottoman ambassador) to Belgium
• The Carathéodory family was well-respected in Constantinople
(many important governmental positions)
• Formal schooling at a private school in Vanderstock (1881-83);
travelling with is father to Berlin, Italian Riviera;
grammar school in Brussels (1985);
high school Athénée Royal d'Ixelles, graduation in 1891
• Twice winning of a prize as the best mathematics student in Belgium
• Trelingual (Greek, French, German),
later: English, Italian, Turkish, and the ancient languages
• École Militaire de Belgique (1891-95),
École d'Application (1893-1896): military engineer
• War between Turkey and Greece (break out 1897);
British colonial service: construction of the Assiut dam (until 1900);
Studied mathematics: Jordan's Cours d'Analyse a.o.;
Measurements of Cheops pyramid (published in 1901)
Constantin Carathéodory (1873 - 1950)
• Graduate studies at the University of Göttingen (1902-04)
(supervision of Hermann Minkowski: dissertation in 1904 (Oct.) on
Diskontinuierliche Lösungen der Variationsrechnung
• In March 1905: venia legendi (Felix Klein)
• Various lecturing positions in Hannover, Breslau, Göttingen and Berlin
(1909-20)
• Prussian Academy of Sciences (1919, together with Albert Einstein)
• Plan for the creation of a new University in Greece (Ionian University)
(1919, not realized due to the War in Asia Minor in 1922);
the present day University of the Aegean claims to be the continuation
• University of Smyrna (Izmir), invited by the Greek Prime Minister (1920);
(major part in establishing the institution, ends in 1922 due to war
• University on Athens (until 1924)
• University of Munich (1924-38/50); Bavarian Academy of Sciences (1925)
• C. played a remarkable opposing role together with the Munich
„Dreigestirn“ (triumvirate) (Perron, Tietze) within
the Bavarian Academy of Science during the Nazi terror in Germany
The Maximum Principle (first formulation, controls, 1950)
Thus,
has a maximum value
with respect to
along
a minimizing curve
.
Research Memorandum RM-100,
Rand Corporation, 1950
I became interested in control theory in 1948.
At that time I formulated the general control
problem of Bolza …, and observed the maximum
principle … is equivalent to the conditions of
Euler-Lagrange and Weierstrass …
Magnus Rudolph Hestenes
(1906 – May 31, 1991)
It turns out that I had formulated what is now
known as the general optimal control problem.
Missed opportunity
The Maximum Principle (Bellman‘s & Isaacs‘ Equation, 1951+)
Richard Ernest Bellman
(Aug. 26, 1920 – March 19, 1984)
Rufus Philip Isaacs
(1914 – 1981)
Isaacs in 1973 about his Tenet of Transition of 1951
Once I felt that here was the heart of the subject …..
Later I felt that it … was a mere truism.
Thus in (my book) Differential Games
it is mentioned only by title. This I regret.
I had no idea, that Pontryagin‘s principle
and Bellman‘s maximal principle
(a special case of the tenet, appearing a little later
in the Rand seminars) would enjoy such
a widespread citation.
Missed opportunities
The Maximum Principle (1956)
This fact is a special case
of the following general principle
which we call maximum principle
Doklady Akademii Nauk SSSR,
Vol. 10, 1956
Lev Semenovich Pontryagin
(Лев Семёнович Понтрягин)
(Sept. 3, 1908 – May 3. 1988)
The Maximum Principle (1956)
Vladimir G. Boltyanski
Revaz V. Gamkrelidze
proved the Maximum Principle
Boltyanski in 1991 about the Maximum Principle of 1956
By the way, the first statement of the maximum principle was given
by Gamkrelidze, who has established (generalizing the famous
Legendre Theorem) a sufficient condition for a sort of weak
optimality problem. Then, Pontryagin proposed to name
Gamkrelidze‘s condition Maximum Principle. … Finally, I understood
that the maximum principle is not a sufficient, but only a necessary
condition of optimality.
Pontryagin was the Chairman of our department at the Steklov
Mathematical Institute, and he could insist on his interests.
So, I had to use the title Pontryagin‘s Maximum Principle
in my paper. This is why all investigators in region of mathematics
and engineering know the main optimization criterium as the
Pontryagin‘s Maximum Principle.
Gamkrelidze in 2008 about Pontryagin
My life was a series of missed opportunities,
but one opportunity, I have not missed,
to have met Pontryagin.*
* at the Banach Center Conference
on 50 Years of Optimal Control in
Bedlewo, Poland, on September 15, 2008
Plail, M.: Die Entwicklung der optimalen Steuerungen.
Vandenhoeck & Ruprecht, Göttingen, Germany, 1998
Carathéodory‘s words:
I will be glad if I have succeeded in impressing
the idea that it is not only pleasant and entertaining
to read at times the works of the old mathematicial
authors, but that this may occasionally be of use
for the actual advancement of science.
Besides this there is a great lesson we can derive
from the facts which I have just referred to. We have
seen that even under conditions which seem most
favorable very important results can be discarded
for a long time and whirled away from the main stream
which is carrying the vessel science. …
If their ideas are too far in advance of their time, and
if the general public is not prepared to accept them,
these ideas may sleep for centuries on the shelves
of our libraries … awaiting the arrival of the prince
charming who will take them home.
(C.C. 1937)
Constantin Carathéodory (Κωνσταντίνος Καραθεοδωρή)
* Sept. 13, 1873 in Berlin; † Feb. 2, 1950, Munich
Thank you for your attention!
Both papers and a third forthcoming one
can be downloaded from
www.ingmath.uni-bayreuth.de/
Email: hans-josef.pesch@uni-bayreuth.de