Seeing maximum entropy from the principle of virtual work
Qiuping A. Wang
Institut Supérieur des Matériaux et Mécaniques Avancées du Mans,
44 Av. Bartholdi, 72000 Le Mans, France
Abstract
We propose an application of the principle of virtual work of mechanics to random
dynamics of mechanical systems. The total virtual work of the interacting forces and inertial
forces on every part of the system is calculated by considering the motion of each particle.
This calculation leads to a known expression of thermodynamic work in statistical mechanics:
the work of a infinitesimal process is the average of the infinitesimal energy variation over all
microstates of the system. Then according to the Lagrange-D’Alembert principle, the
vanishing virtual work yields stable thermodynamic equilibrium with the maximization of
thermodynamic entropy for canonical ensemble. This application reveals a close relationship
between the maximum entropy approach for statistical mechanics and a fundamental principle
of mechanics. This work is an attempt to give the maximum entropy approach, considered by
many as an inference principle based on the subjectivity of probability and entropy, the status
of a fundamental physics law.
PACS numbers : 02.30.Xx, 05.40.-a, 05.20.-y, 02.50.-r
1
1) Introduction
The principle of maximum entropy (maxent) is widely used in the statistical sciences and
engineering as a powerful tool and fundamental rule. The maxent approach in statistical
mechanics can be traced back to the works of Boltzmann and Gibbs[3] and finally be given
the status of principle thanks to the work of Jaynes[4] who used it with Boltzmann-GibbsShannon entropy (see below) to derive the canonical probability distribution for statistical
mechanics in a simple manner. However, in spite of its success and popularity, maxent has
always been at the center of scientific and philosophical discussions and has raised many
questions and controversies in physics[4][5][6]. A central question is why a thermodynamic
system chooses the equilibrium microstates such that the entropy of the second law gets to
maximum. As the basic assumption of a scientific theory, maxent is not directly or indirectly
related to observation and assumed from undoubted facts. In the literature of statistical
mechanics, maxent is postulated as such and often justified either a priori by the second laws
with additional hypothesis such as the entropy functional (Boltzmann or Shannon entropy)[6],
or a posteriori by the correctness of the probability distributions derived from it[4]. In
statistical inference theory, it was often justified by intuitive arguments based on the
subjectivity of probability[4] or by relating it to other principles such as the consistency
requirement and the principle of insufficient reason of Laplace, which have been the object of
considerable criticisms[5].
Another important question about maxent is whether or not the entropy of BoltzmannGibbs-Shannon is unique as the measure of uncertainty or disorder that can be maximized in
order to determine probability distributions. This was already an question raised 40 years ago
by the scientists who tried to generalize the Shannon entropy by mathematical considerations
[9][10]. Nowadays, the answer to this question becomes much more urgent and waited due to
the controversy and debate surrounding the development of the statistical theories using
maxent with different entropy functionals [11].
In the present work, we try to contribute to the debate around maxent by an attempt to
derive maxent from a well known fundamental principle of classical mechanics, the virtual
work principle (viwop) [1][2] without additional hypotheses to viwop and about entropy
property. Viwop is widely used in physical sciences as well as in mechanical engineering. It is
a basic principle capable of yielding all the basic laws of statics and of dynamics of
mechanical systems. It is in addition a simple, clearly defined, easily understandable and
2
palpable law of physics. It is hoped that this derivation will be scientifically and
pedagogically beneficial for the understanding of maxent as a rule of physics as well as of the
relevant questions and controversies around it. In what follows, we use the term entropy,
denoted by S, in the general sense as a measure of uncertainty or randomness of random
dynamics. In the case of equilibrium thermodynamics, S denotes the entropy of the second
law of thermodynamics.
The paper is organized as follows. In the first section, we recall the principle of virtual
work before applying it to equilibrium thermodynamic system to derive maxent for the
thermodynamic entropy of equilibrium state. Then we will briefly mention a previous result in
order to show that, according to this reasoning from viwop, the entropy functionals that can be
maximized are not unique in general.
2) Principle of virtual work
The variational calculus in mechanics has a long history which may be traced back to
Galilei and other physicists of his time who studied the equilibrium problem of statics with
the principle of virtual work (or virtual displacement1). Viwop gots unified and concise
mathematical forms thanks to Lagrange[1] and D’Alembert[2]. This principle is considered as
a most basic one of mechanics from which all the fundamental laws of statics and dynamics
can be understood thoroughly.
Viwop says that the total work done by all forces acting on a system in static equilibrium
is zero on all possible virtual displacements which are consistent with the constraints of the
system. Let us suppose a simple case of a system of N points of mass in equilibrium under the
action of N forces Fi (i=1,2,…N) with Fi on the point i, and imagine virtual displacement of

each point ri for the point i. According to viwop, the virtual work W of all the forces Fi on

all ri is given by
N


W   F i  ri  0
(1)
i 1
1
In mechanics, the virtual displacement of a system is a kind of imaginary infinitesimal displacement with no
time passage and no influence on the forces. It should be perpendicular to the constraint forces.
3
This principle for statics was extended to dynamics by D’Alembert[2] in the Lagrange
d'Alembert principle by adding the initial force  mi ai on each point given by

N


(2)
W   ( F i  mi ai )  ri  0
i 1

where mi is the mass of the poin i and ai its acceleration. From this principle, we can not only
derive Newtonian equation of dynamics, but also other fundamental principles such as the
least action principle .
3) Why maximum thermodynamic entropy ?
Now let us consider an equilibrium mechanical system composed of N particles in random
free motion with vi the velocity of the particle i. Without any loss of generality, let us
N
suppose a system without macroscopic motion, i.e.,  vi  0 .
i 1
We imagine that the system is in a canonical ensemble and leaves the equilibrium state by

a quasi-equilibrium infinitesimal virtual process at some moment. Let F i be the force on a

particle i of the system at that moment. F i includes all the interacting forces particlesparticles and particles-walls of the container at the moment. During the virtual process, each

particle with acceleration ri has a virtual displacement  ri . The total dynamical virtual work
on this displacement is given by
N



(3)
W   ( F i  mri )  ri
i 1
N

Although the sum of the accelerations of all the particles vanishes, i.e.,  mri  0 , the
i 1
N

 
acceleration ri on each particle can be nonzero. So in general  mri  ri  0 . As a matter of
i 1
1 
 
 
fact, we have mri  ri  mri  ri   ( mri 2 )  eki where eki is the kinetic energy of the
2
particle. On the other hand, if epi is the potential energy of the particle, we can write
N 
N
 N

 F i  ri   iei  ri     e pi
i 1
i 1
(4)
i 1
So finally it follows
4
N
N
i 1
i 1
(5)
W    (e pi  eki )    ei .
where ei is a virtual variation of the total energy ei  e pi  eki of the particle i. Now let us
consider the microstates of the total system. Suppose p j  p( E j ) is the probability that the
system is at the microstate j with total energy Ej, and there are w microstates, the above sum
of energy variation over each particle can be written as an average (over different microstates)
of the energy change of the total system at the time of the virtual displacements, i.e.
(6)
w
W    p j E j  E .
j 1
This is a well known relationship in statistical mechanics. Here we have derived it from the
microscopic consideration of virtual work on each particle of the system. A simple calculation
w
w
w
j 1
j 1
j 1
shows that   p j E j   E jp j    p j E j which means
E  Q  W  Q  E .
(7)
w
w
j 1
j 1
where E   p j E j is the total average energy with a virtual change E and Q   E jp j is
a virtual heat transfer. This is the expression of the first law of thermodynamics in the virtual
sense. If we consider a quasi-equilibrium virtual process, we can use Q 
S
where S is

the infinitesimal virtual change of the equilibrium thermodynamic entropy and  the inverse
absolute temperature according to the second law of thermodynamics. From the LagrangeD’Alembert principle in Eq.(2), it follows that
W 
S
 E  0 .

(8)
w
Then we should add the constraint due to the normalization  p j  1 into the variational
j 0
expression with a Lagrange multiplier  , the viwop in Eq.(8) becomes
w
w
j 0
j 0
 (S    p j    p j E j )  0
(9)
5
which is the variational calculus of maxent applied to thermodynamic entropy for canonical
ensemble. Note that at this stage the entropy functional S(pj) is not yet determined. In the
following section, this point will be discussed by considering S as a general measure of
uncertainty which can be maximized to given some known probability distribution.
The conclusion of this section is that, at thermodynamic equilibrium, the maxent under the
constraints of average energy is a consequence of the equilibrium condition of mechanical
systems subject to random motion. From the above discussion, one notices that maxent can be
written in a concise form such as
E  0 .
(10)
4) Maximizable entropy functionals
In general, the entropy functionals are given either as a first hypothesis or from physical
or mathematical considerations about the entropy property. Then the standard approach is to
use maxent for given entropy in order to derive the probability distribution. Here we inverse
the reasoning just in order to see what are the possible entropy forms which can be maximized
according to maxent in Eq.(10) to yield known probability distributions. This approach is
already considered by several authors[12][13] to see the possible maximizable entropy forms
for stretched exponential distribution.
Before using maxent in Eq.(10), we can write Eq.(7) as follows
Q   x  x .
(11)
by replacing energy E with certain random variable x. This equation can be considered as the
definition of an uncertain measure Q which is not necessarily the heat in thermodynamics.
Obviously, for a thermodynamic system in equilibrium, x is the energy and Q the heat. In
mimicking the second law , another general measure S of uncertainty can be defined by
S=Q where  is a constant. It is easy to calculate
w
S    x j p j
(12)
j 1
This equation makes it possible to calculate the entropy functional directly from known
probability distribution. The definition of entropy by Eq.(11) ensures that the so obtained
entropy can be maximized by Eq.(10) to give the original distribution.
6
It is well known that for exponential distribution of canonical ensemble p j  e x j with
x j  E j and =, the Gibbs-Shannon entropy of equilibrium thermodynamics follows from
Eq.(12) (with Boltzmann constant kB=1):
(13)
w
S    p j ln p j
j 1
For different probabilities, different maximizable entropy forms are possible. For example, it
was proved[8] that, for a distribution such as p j 

1
1  ax j
Za

1
a
for finite system in
equilibrium with finite heat bath[7], the entropy defined by Eq.(11) is the Tsallis one
Sq  
1 a
 p j 1
j
a
pj  pj
q
 
j
1 q
where q=1+a and  p j  1; and for p j 
j
11 
large xj Lévy flight), one gets S 
 pj
1 
x j (Zipf law or
Z
1
j
11 
. This result shows that the entropy as
maximizable uncertainty measure may take different forms, each can be maximized to yield
the corresponding probability.
This conclusion does not exclude the possibility to use an given entropy functional for
measuring the uncertainty of any probability distribution. Nevertheless, such an uncertainty
measure is obviously not maximizable for any probability distribution.
5) Concluding remarks
It is argued that the maximum entropy principle has a close connection with a
fundamental principle of classical mechanics, i.e., the principle of virtual work. In other
words, for a mechanical system in thermodynamics equilibrium, the thermodynamic entropy
gets to maximum when the total virtual work of all random forces on all the elements
(particles) of the system vanishes. Indeed, if one admits that thermodynamic entropy is a
measure of dynamical disorder and randomness, it is natural to say that this disorder must get
to maximum in order that all the random forces act on each degree of freedom of the motion
in such a way that over any possible (virtual) displacement, the work of all the forces is zero.
7
In other words, this vanishing work can be obtained if and only if the randomness of the
forces is at maximum over all degree of freedom allowed by the constraints to get stable
equilibrium state. This is why if we take off the constraint associated with average energy for
microcanonical ensemble, all microstates (combination of all degrees of freedom) becomes
equally probable. Equiprobability is a distribution of disorder in which no degree of freedom
is privileged.
To our opinion, the present result is helpful not only for the understanding of maxent
derived from a more basic and well understood mechanical principle, it also shows that
entropy in physics is not necessarily a subjective quantity reaching maximum for correct
inference, and that maximum entropy is a law of physics but not merely an inference
principle.
After finishing this paper, the author became aware of the works of Plastino and
Curado[14] on the equivalence between the particular thermodynamic relation S  E and
maxent in the derivation of probability distribution. They consider the particular
thermodynamic process affecting only the microstate population in order to find a different
way from maxent to derive probability. The work part is not considered in their work. Their
analysis is pertinent and consequential. The present work provides a substantial support of
their reasoning from a basic principle of mechanics. Our conclusion is however a little
different: S  E is not only equivalent to maxent, it is maxent by virtue of a much more
fundamental principle of physics.
8
References
[2]
J.L. Lagrange, Mécanique analytique, Blanchard, reprint , Paris (1965) (Also:
Oeuvres, Vol. 11.)
J. D’Alembert, Traité de dynamique, Editions Jacques Cabay , Sceaux (1990)
[3]
J. Willard Gibbs, Principes élémentaires de mécanique statistique (Paris, Hermann,
[1]
1998)
[4]
E.T. Jaynes, The evolution of Carnot's principle, The opening talk at the EMBO
Workshop on Maximum Entropy Methods in x-ray crystallographic and biological
macromolecule structure determination, Orsay, France, April 24-28, 1984; Gibbs
vs Boltzmann entropies, American Journal of Physics, 33,391(1965) ; Where do
we go from here? in Maximum entropy and Bayesian methods in inverse problems,
pp.21-58, eddited by C. Ray Smith and W.T. Grandy Jr., D. Reidel, Publishing
Company (1985)
[5]
Jos Uffink, Can the maximum entropy principle be explained as a consistency
requirement, Studies in History and Philosophy of Modern Physics, 26B (1995):
223-261
[6]
L.M. Martyushev and V.D. Seleznev, Maximum entropy production principle in
physics, chemistry and biology, Physics Reports, 426, 1-45 (2006)
[7]
Y.P. Terletskii, Statistical physics, North-Holland Publishing Company,
Amsterdam, 1971
[8]
Q.A. Wang, Some invariant probability and entropy as a maximizable measure of
uncertainty, cond-mat/0612076
[9]
A. Rényi, Calcul de probabilité, Paris, Dunod, 1966, P522
A. Wehrl, Rev. Mod. Phys., 50(1978)221
[10] M.D. Esteban, Kybernetika, 31(1995)337
[11] R.
Balian
and
M.
Nauenberg,
Europhysics
news,
37/2
(2006)9;
R. Luzzy, A.R. Vasconcellos and J.G. Ramos, Europhysics news, 37/2 (2006)11;
M. Nauenberg, Phys. Rev. E, 67 (2003) 036114; Phys. Rev. E, 69 (2004) 038102;
cond-mat/0305365v1; See also the special issue of Comptes Rendus Physique, 7,
(2006) and the special issue of Europhysics news, 36/6 (2005)
9
C Anteneodo and A R Plastino, Maximum entropy approach to stretched
exponential probability distributions, J. Phys. A, 32, 1089(1999)
S. Abe, Generalized Entropy Optimized by an Arbitrary Distribution, J. Phys. A,
[13]
33, 8733(2003)
[12]
[14] A. Plastino and E.F.M. Curado, Phys. Rev. E, 72, 047103(2005);
E.F.M. Curado and A. Plastino, arXiv:cond-mat/0601076; also cond-mat/0509070
10