Statistical Mechanics
Z. Suo
Free Energy
For a system in thermal contact with the rest of the world, we have
described three quantities: entropy, energy, and temperature. We have also
described the idea of a constraint internal to the system, and associated this
constraint to an internal variable.
The system can be isolated at a particular value of energy. For such an
isolated system, of all values of the internal variable, the most probable value
maximizes entropy. We will paraphrase this statement under two different
conditions, either when the entropy is fixed, or when the temperature is fixed.
Under these conditions, the system is no longer isolated. Consequently, we need
to maximize or minimize quantities other than entropy.
To illustrate thermodynamic modeling, we study two types of phase
transition. We use experimental observations to motivate the construction of the
models.
An isolated system. An isolated system has a set of quantum states.
The number of the quantum states is denoted by Ω . The isolated system flips
ceaselessly from one quantum state to another. The fundamental postulate states
that a system isolated for a long time is equally probable to be in any one of its
quantum states (http://imechanica.org/node/290). We call the quantity log Ω
the entropy, and give it a symbol S, namely,
S = log Ω .
A system with variable energy. We now open a system in a particular
manner: the system can vary its energy U by thermal contact with the rest of the
world. We block all other modes of interaction between the system and the rest of
the world, so that the energy of the system is the only independent variable.
When the energy of the system is fixed at a particular value U, the system
becomes isolated, and has a particular set of quantum states. The number of
quantum states of the system is a function of the energy, Ω(U ) .
A system with variable energy is characterized by the function Ω(U ) or,
equivalently, by the function S (U ) as defined by
S (U ) = log Ω(U ) .
For a simple system, the function may be determined by using quantum
mechanics.
For a complex system, the function Ω(U ) is determined
experimentally. See notes on Temperature (http://imechanica.org/node/291).
S
U
Consider a plane spanned by the coordinates S and U. On this plane, a
system with variable energy is represented by a curve S (U ) . A point on the curve
February 23, 2009
Free Energy - 1
http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
represents the system isolated at energy U, flipping among exp(S ) number of
quantum states.
Consider leading characteristics of this curve S (U ) . The horizontal
position of the curve has no empirical significance, because energy is meaningful
up to an additive constant. By contrast, the vertical position of the curve does
have empirical significance. We know Ω ≈ 1 or S ≈ 0 at the ground state of the
system.
We have agreed to define temperature T by the relation
1 ∂S (U )
=
.
T
∂U
Given a system of variable energy, the function S (U ) is known, so that the above
equation expresses the temperature of the system as a function of energy, T (U ) .
On the (S,U ) plane, the temperature is the inverse of the slope of the curve. Note
that S → 0 as T → 0 .
S
1
T
U
The function S (U ) monotonically increases. The more energy the
system has, the more quantum states the system has. That is, the function S (U )
is a monotonically increasing function. The temperature defined above is positive.
Any monotonic function is 1-to-1, and can be inverted. Consequently, the
function S (U ) and its inverse U (S ) characterize the same family of isolated
systems. The two functions correspond to the same curve on the (S ,U ) plane. An
equivalent definition of temperature is
∂U (S )
T=
.
∂S
Given a system of variable energy, the function U (S ) is known, so that the above
equation expresses the temperature of the system as a function of entropy, T (S ) .
The function S (U ) is convex. Consider a system with variable energy,
characterized by a nonconvex function S (U ) . Put two copies of the system
together as two parts of a composite. The composite has a fixed amount of total
energy 2U , but the two parts can exchange energy. Let the energy be U − Q in
one part, and be U + Q in the other part. Consequently, the composite is an
isolated system with an internal variable Q. When a part has a fixed among of
energy, the part itself is an isolated system, and has its own set of quantum states.
February 23, 2009
Free Energy - 2
http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
A quantum state of one part and a quantum state of the other part in
combination constitute a quantum state of the composite. The set of quantum
states of the composite with internal variable Q constitute a macrostate of the
composite. The entropy of this macrostate is S (U − Q ) + S (U + Q ) . For another
macrostate of the composite, each of the two parts has energy U, so that the
entropy of the this macrostate of the composite is 2 S (U ) .
S
S (U + Q )
S (U − Q )
S (U )
U
U −Q
U
U +Q
At a value of U where the function S (U ) is concave, we can find many
values of Q to satisfy the following inequality:
S (U − Q ) + S (U + Q ) > 2 S (U ) .
Consequently, the composite is more likely to be in a macrostate where the two
parts have unequal amounts of energy. The convex portion of the function S (U )
is never realized in the composite.
That we have used two copies of the system is not as artificial as it may
appear. Most systems can redistribute energy internally, and effectively behave
like a composite of multiple copies of a smaller system.
A system with variable energy and an internal variable.
Entropy
S (U , Y )
.
In
notes
on
Isolated
Systems
(http://imechanica.org/node/290), we have described the idea of a constraint
internal to an isolated system, and associated the constraint to an internal
variable. We have then used the internal variable to dissect the whole set of
quantum states of the isolated system into subsets. In each subset the quantum
states have the same value of the internal variable. The subset of quantum states
constitutes a macrostate of the isolated system. This macrostate may be labeled
by the value of the internal variable. The fundamental postulate implies that, of
all values of the internal variable, the most probable value has the largest number
of quantum states.
We now wish to describe the idea of a constraint internal to a system with
variable energy. As an example, once again consider the half glass of wine, now in
thermal contact with the rest of the world. We are interested in two variables:
the energy in the glass, and the number of molecules in the gas phase. The glass
is sealed to prevent molecules from escaping the glass. If we regard the half glass
of wine as a system, because it exchanges energy with the rest of the world, the
February 23, 2009
Free Energy - 3
http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
system is not isolated. Consequently, the number of molecules in the gas phase is
an internal variable in the system with variable energy.
In generic terms, a system is in thermal contact with the rest of the world,
and we are interested in two variables: the energy of the system U, and an
internal variable of the system Y. When the energy U of the system is fixed, and Y
is allowed to vary, the system becomes an isolated system with an internal
variable, and has a set of quantum states. When both U and Y are fixed at specific
values, the system can only flip among a subset of the quantum states that have
value Y. Denote the number of the quantum states in this subset by Ω(U , Y ) , and
write
S (U , Y ) = log Ω(U , Y ) .
The fundamental postulate implies the following statement:
At a constant U, of all possible value of Y, the most probable value of Y
maximizes the function S (U , Y ) .
Energy U (S , Y ) . For a rectangular block in a field of gravity, the number
of quantum states of the block is independent of the orientation of the block, but
the energy of the block does vary with the orientation. In this case, the
orientation of the block may be regarded as an internal variable. We wish to
express the energy as a function of the orientation.
In generic terms, when Y is fixed, Ω is an increasing function of U. That
is, the more energy the system has, the more quantum states the system has.
Consequently, we can invert the function Ω(U , Y ) , and obtain the function
U (Ω, Y ) . Of course, the two functions, Ω(U , Y ) and U (Ω, Y ) , characterize the
same system. The previous statement of maximal number of quantum states is
equivalent to the following statement:
At a constant Ω , of all possible value of Y, the most probable value of Y
minimizes the function U (Ω, Y ) .
Recall the definition S = log Ω . The above the statement is correct if we
replace the function U (Ω, Y ) with the function U (S , Y ) .
Free energy F (T , Y ) . A system can be held at a constant temperature
when a device adds energy to, or extracts energy from, the system. Such a device
is called a thermostat. For example, a mixture of ice and water is a thermostat
that holds the temperature at 0C. When a system is in thermal contact with the
mixture of ice and water, the amount of ice increases if the system draws energy
February 23, 2009
Free Energy - 4
http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
from the mixture, and the amount of ice decreases if the system gives heat to the
mixture. One can construct a thermostat to hold any other temperature.
We regard the system and the thermostat together as a composite. The
composite is an isolated system with two internal variables: the energy U in the
system, and the internal variable Y. A pair of values (U , Y ) specifies a macrostate
of the composite. The entropy of the system relates to the two internal variables
by the known function S (U , Y ) . Of course, we can invert the function to obtain
U (S , Y ) .
System
Y
U
energy
Thermostat
TR
U tot − U
Isolated system
Let TR be the temperature fixed by the thermostat. When the system
draws energy U from the thermostat, the entropy of the thermostat reduces by
U / TR . Consequently, the entropy of the macrostate (U , Y ) of the composite is
U
S (U , Y ) −
.
TR
According to the fundamental postulate, of all possible values of the
internal variables, the most probable values maximize the above entropy of the
macrostate. The above maximization is equivalent to the following minimization.
Of all possible values of the internal variables, the most probable values minimize
the following function:
U (S , Y ) − TR S .
This is a function of S and Y. Setting the variation with respect to S to
zero, we obtain that
∂U (S , Y )
= TR .
∂S
This equation recovers the statement that the system is in thermal equilibrium
with the thermostat. The equation also solves the entropy as a function of the
temperature and the internal variable, S (TR , Y ) . Consequently, we can rewrite
the above function as
F (T , Y ) = U − TS ,
where T is the temperature of the system, specified by T = ∂U (S , Y )/ ∂S . The
function F (T , Y ) now contains quantities of the system alone, and is known as
the Helmholtz free energy of the system.
When the system is held at a fixed temperature (i.e., in thermal
equilibrium with a thermostat), of all values of the internal variable Y, the most
February 23, 2009
Free Energy - 5
http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
probable value minimizes the free energy F (T , Y ) . In this minimization, the
temperature is not a variable, but is fixed by the thermostat.
Taking differential of the free energy F = U − TS , we obtain that
dF = dU − TdS − SdT
When Y is held constant, dU = TdS , the above equation becomes that
dF = − SdT .
Consequently, the entropy relates to a partial differential coefficient of the free
energy:
∂F (T , Y )
S=−
.
∂T
For a constant value of Y, the function F (T , Y ) looks like this:
F
T
An alternative way to introduce of the free energy. The above way
of introducing the free energy comes directly from the fundamental postulate, but
does leave it vague as to what the free energy really is. In the following alternative
introduction, we characterize the system by the function U (S , Y ) , using entropy
as an independent variable, rather than energy. We regard the system and the
thermostat together as a composite. The composite is now a system with variable
energy. We adjust the of energy of the composite such that the entropy of the
composite, Stot , is fixed.
When the system gains entropy S, the thermostat loses entropy by the
same amount. Consequently, this transfer of entropy causes the thermostat to
lose energy by TR S . The sum of the energy of the system and that of the
thermostat gives the energy of the composite:
U (S , Y ) − TR S .
The composite has two internal variables: the entropy of the system, S, and the
internal variable of the system, Y. Recall that the composite is a system with fixed
entropy. Consequently, of all values of the internal variables of the composite, S
and Y, the most probable ones minimize the energy of the composite. The
remainder of the development will be the same as the previous way of introducing
free energy.
This alternative introduction makes it clear that the free energy is the total
energy of the composite of the system and the thermostat in thermal equilibrium.
February 23, 2009
Free Energy - 6
http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
System
Y
S
entropy
Thermostat
TR
Stot − S
System with variable energy
Co-existent phases of a substance. To illustrate thermodynamic
modeling, consider a mixture of ice and water at the melting temperature. When
energy is added to the mixture, the amount of water increases at the expense of
the amount of ice, but the temperature remains constant. We would like to trace
this empirical observation back to the fundamental postulate, using a model.
U
S
water
ice
water
ice
T
Tc
Tc
T
A substance is an aggregate of a large number of a single species of
molecules. The entropy and the energy of the piece of the substance, S and U, are
proportional to the number of molecules in the piece, N. The energy and the
entropy per molecule of the substance are defined as
S
U
s=
u=
,
.
N
N
The function s (u ) is specific to the substance, and is independent of the size and
shape of the piece. The temperature is given by
1 ∂s (u )
=
.
T
∂u
The substance can be in two phases, A′ and A′′ . We may regard the two
phases as two systems, one characterized by function s′(u′) , and the other by
s′′(u′′) . Let N ′ be the number of molecule in one phase, and N ′′ be the number
of molecules in the other phase. When the two phases coexist, molecules can
detach from one phase and attach to the other. The total number of molecules in
the mixture, N, remains unchanged:
February 23, 2009
Free Energy - 7
http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
N = N ′ + N ′′ .
We neglect energy associated with the phase boundaries, so that the energy of the
two-phase mixture is the sum of the energies of the two phases:
U = N ′u′ + N ′′u′′ .
Similarly, the entropy of the two-phase mixture is the sum of the entropies of the
two phases:
S = N ′s′ + N ′′s′′ .
The above equations are known as the rules of mixture.
When N and U are fixed, the mixture is an isolated system with internal
variables: N ′ , N ′′ , u′ , u′′ . Of all values of the internal variables, the most
probable ones maximize the entropy of the mixture, subject to the constraint of
the fixed N and U .
This problem of can be readily solved in a number of ways. For example,
consider the function
f ( N ′, N ′′, u′, u′′,α , β ) = N ′s′(u′) + N ′′s′′(u′′) + α ( N − N ′ − N ′′) + β (U − N ′u′ − N ′′u′′) ,
where α and β are Lagrange multipliers. The two functions, s′(u′) and s′′(u′′) ,
are taken to be given. The six variables listed for the function are regarded as
independent, and we require that the partial derivative of f with respect to each of
the variable vanish, resulting in six algebraic equations that determine the six
variables.
Thus, ∂f / ∂α = 0 recovers that constraint that the number of molecules in
the mixture is conserved, and ∂f / ∂β = 0 recovers that constraint that the energy
in the mixture is conserved. Setting ∂f / ∂N ′ = ∂f / ∂N ′′ = 0 , we obtain that
s′(u′) − βu′ = s′′(u′′) − βu′′ = α .
We can solve for β , giving
s′ − s′′
=β .
u′ − u′′
Setting ∂f / ∂U ′ = ∂f / ∂U ′′ = 0 , we obtain that
∂s′(u′) ∂s′′(u′′)
=β .
=
∂u′′
∂u′
Thus, co-existent phases equalize their temperature. The Lagrange multiplier β
is the inverse of the phase-transition temperature, β = 1 / Tc . The latent heat is
given by
L = u′′ − u′
The difference in entropies of the two co-existent phases is
u′′ − u′
s′′ − s′ =
.
Tc
Graphical representation of the condition for coexistent phases.
We can also read the above equations off a graph. Given a function s′(u′) , the set
of points ( Nu′, Ns′) is a curve, representing the substance when all molecules are
February 23, 2009
Free Energy - 8
http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
in phase A′ . Similarly, given a function s′′(u′′) , the set of points ( Nu′′, Ns′′) is a
curve, representing the substance when all molecules are in phase A′′ .
Now pick one particular point ( Nu′, Ns′) on one curve, and pick another
particular point ( Nu′′, Ns′′) on the other curve. Through the two points draw a
straight line. Recall the rules of mixture:
N = N ′ + N ′′ ,
U = N ′u′ + N ′′u′′ ,
S = N ′s′ + N ′′s′′ .
The energy and the entropy of the mixture, U and S, are linearly proportional to
the number of molecules in the two phases. Consequently, (U , S ) is a point on
the straight line.
S
A′′
Ns′′
N ′s′ + N ′′s′′
Ns′
A′
Nu′
N ′u′ + N ′′u′′
Nu′′
U
When the energy of the mixture is fixed, the entropy of the mixture is
maximized when the straight line is tangent to both curves. Reading off the graph,
this condition corresponds to
s′ − s′′ ∂s′(u′) ∂s′′(u′′)
=
=
.
u′ − u′′
∂u′′
∂u′
For a substance with a nonconvex function S (U ) , the concave part of the
function will not be realized, and will be replaced by a common tangent line. The
two tangent points represent coexistent phases.
February 23, 2009
Free Energy - 9
http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
S
U
Examine co-existent phases using the free energy. Consider a
mixture of two phases held at a temperature T. Let the free energy of the two
phases be f ′(T ) and f ′′(T ) . The free energy of the mixture is.
F = N ′f ′(T ) + N ′′f ′′(T ) .
The number of molecules in one phase is the internal variable, to be selected to
minimize the total free energy. The equation
f ′(T ) = f ′′(T )
determines the phase-transition temperature Tc . When T < Tc , all molecules are
in one phase. When T > Tc all molecules are in the other phase.
F
A′
A′′
Tc
T
Phase transition of the second kind. As another illustration of
thermodynamic modeling, consider the following experimental observation. A
crystal has a rectangular symmetry at a high temperature. When the temperature
drops below a critical value, Tc , the crystal spontaneously distorts by an angle α .
Because of the symmetry, the distortion can go in both directions. Below the
critical temperature, the angle of distortion is observed to vary with the
temperature.
We regard the angle of distortion α as an internal variable, also known as
an order parameter. The thermodynamics of the crystal is characterized by the
free-energy function F (T ,α ) . To model the experimental observation, we assume
that the free-energy function has the following form:
February 23, 2009
Free Energy - 10 http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
1
1
A(T − Tc )α 2 + Bα 4 ,
2
4
where A and B are positive constants. This form is a Taylor expansion in the
powers of α . Due to symmetry, the crystal is equally likely to distort in two
directions, so that we keep the even powers of α . Let us check if this function
does reproduce the observed phase transition.
F (T ,α ) = C (T ) +
α
T
Tc
α
T < Tc
T > Tc
When T > Tc , the coefficient of the α 2 term is positive, so that the the
equilibrium state is α = 0 . In this case, the α 4 term is unnecessary to describe
the behavior of the crystal.
When T < Tc , the coefficient of the α 2 term is negative, and the energy is
no longer minimal at α = 0 . Instead, the energy is minimal at two nonezero
angles, denoted as the spontaneous angles, ± α s . In this case, the α 4 term will
ensure that energy goes up again when α is large. Note that
∂F (T ,α )
= A(T − Tc )α + Bα 3 .
∂α
Setting ∂F (T ,α )/ ∂α = 0 , we find the spontaneous values of the angle:
α s = ± A(Tc − T )/ B .
A significant prediction of this model is
1/2
α s ~ (Tc − T ) .
This prediction should be compared with the angle of distortion observed in the
experiment.
February 23, 2009
Free Energy - 11 http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
F
T > Tc
T < Tc
α
February 23, 2009
Free Energy - 12 http://imechanica.org/node/4878