Statistical Mechanics
http://imechanica.org/node/4878
Z. Suo
Free Energy
In modeling a system in thermal contact with the rest of the world, we
have invoked three variables: entropy, energy, and temperature. We have also
described the idea of a constraint internal to the system, and associated this
constraint to an internal variable.
The system can be isolated at a particular value of energy. For such an
isolated system, of all values of the internal variable, the most probable value
maximizes entropy. We will paraphrase this statement under two other
conditions, either when the entropy is fixed, or when the temperature is fixed.
Under these conditions, the system is no longer isolated. Consequently, we need
to maximize or minimize quantities other than entropy.
To illustrate the process of thermodynamic modeling, we study two types
of phase transition. We use experimental observations to motivate the
construction of the models.
An isolated system. An isolated system has a set of quantum states.
The number of the quantum states is denoted by Ω . The isolated system flips
ceaselessly from one quantum state to another. The fundamental postulate states
that a system isolated for a long time is equally probable to be in any one of its
quantum states (http://imechanica.org/node/290). We call the quantity log Ω
the entropy, and give it a symbol S, namely,
S = log Ω .
A system with variable energy. We now open a system in a particular
manner: the system can exchange energy with the rest of the world. We block all
other modes of interaction between the system and the rest of the world, so that
the energy of the system is the only independent variable. When the energy of the
system is fixed at a particular value U, the system becomes isolated, and has a
particular set of quantum states. The number of quantum states of the system is
a function of the energy of the system,
Ω = Ω(U ) .
When the system is isolated at energy U, the entropy is S (U ) , defined by
S (U ) = log Ω(U ) .
For a simple system, the function may be determined by using quantum
mechanics.
For a complex system, the function Ω(U ) is determined
experimentally. See notes on Temperature (http://imechanica.org/node/291).
We have agreed to define temperature T by the relation
1 ∂S (U )
=
.
T
∂U
Given a system of variable energy, the function S (U ) is known, so that the above
equation expresses the temperature of the system as a function of energy, T (U ) .
The heat capacitance C is defined by
1 ∂T (U )
=
.
C
∂U
March 7, 2009
Free Energy - 1
Statistical Mechanics
http://imechanica.org/node/4878
Z. Suo
Once the function S (U ) is known, the above equation defines the function C (U ) .
Graphic representation of functions. Consider a plane with two
coordinates S and U. On this plane, a system with variable energy is represented
by a curve S (U ) . A point on the curve represents the system isolated at energy U,
flipping among exp(S ) number of quantum states. The slope of the curve S (U )
gives the inverse of T. We can also plot the function T (U ) on the plane with
coordinates U and T.
The horizontal positions of both curves
S
have no empirical significance, because energy
1
is meaningful up to an additive constant. By
contrast, the vertical positions of the curves do
T
have empirical significance. We know Ω ≈ 1 or
S ≈ 0 at the ground state of the system T → 0 .
U
The more energy the system has, the
more quantum states the system has. That is,
the function S (U ) is a monotonically increasing
function. The temperature is positive.
T
If the function S (U ) is convex, the
1
function T (U ) is a monotonically increasing
function, and C >0. That is, the system must
C
receive energy to increase its temperature.
U
The function S (U ) is usually
convex. Consider a system with variable energy, characterized by a nonconvex
function S (U ) . Put two copies of the system together as two parts of a composite.
The composite has a fixed amount of total energy 2U , but the two parts can
exchange energy. Let the energy be U − Q in one part, and be U + Q in the other
part. Consequently, the composite is an isolated system with an internal variable
Q. When a part has a fixed among of energy, the part itself is an isolated system,
and has its own set of quantum states.
A quantum state of one part and a quantum state of the other part in
combination constitute a quantum state of the composite. The set of quantum
states of the composite with internal variable Q constitute a macrostate of the
composite. The entropy of this macrostate is S (U − Q ) + S (U + Q ) . For another
macrostate of the composite, each of the two parts has energy U, so that the
entropy of the this macrostate of the composite is 2 S (U ) .
At a value of U where the function S (U ) is concave, we can find many
values of Q to satisfy the following inequality:
S (U − Q ) + S (U + Q ) > 2 S (U ) .
March 7, 2009
Free Energy - 2
Statistical Mechanics
http://imechanica.org/node/4878
Z. Suo
Consequently, the composite is more likely to be in a macrostate where the two
parts have unequal amounts of energy. The convex portion of the function S (U )
is never realized in the composite.
That we have used two copies of the system is not as artificial as it may
appear. Most systems can redistribute energy internally, and effectively behave
like a composite of multiple copies of a smaller system.
S
S (U + Q )
S (U )
S (U − Q )
U −Q
U
U
U +Q
A system with variable energy and an internal variable. We have
described the idea of a constraint internal to an isolated system, and associated
the constraint to an internal variable (http://imechanica.org/node/290). The
isolated system flips among a set of quantum states. When the internal variable is
at a fixed value, the isolated system flips among a subset of the quantum states.
The subset of quantum states constitutes a macrostate of the isolated system.
This macrostate may be labeled by the value of the internal variable. The
fundamental postulate implies that, of all values of the internal variable, the most
probable value has the largest number of quantum states.
We now wish to describe the idea of a constraint internal to a system with
variable energy. As an example, once again consider the half glass of wine, now in
thermal contact with the rest of the world. We are interested in two variables:
the energy in the glass, and the number of molecules in the gas phase. The glass
is sealed to prevent molecules from escaping the glass. If we regard the half glass
of wine as a system, because it exchanges energy with the rest of the world, the
system is not isolated.
Consequently, the S
number of molecules in the gas phase is an
Y1
Y2
internal variable in the system with variable
energy.
S
Entropy S (U , Y ) . In generic terms, a
system is in thermal contact with the rest of the
world, and we are interested in two variables:
the energy of the system U, and an internal
variable of the system Y. When the energy U of
the system is fixed, and Y is allowed to vary, the
U
U
system becomes an isolated system with an
internal variable, and flips among a set of quantum states. When both U and Y
March 7, 2009
Free Energy - 3
Statistical Mechanics
http://imechanica.org/node/4878
Z. Suo
are fixed at specific values, the system flips among a subset of the quantum states
that have value Y. Denote the number of the quantum states in this subset by
Ω(U ,Y ) , and write
S (U , Y ) = log Ω(U , Y ) .
The fundamental postulate implies the following statement:
At a constant U, of all possible value of Y, the most probable value of Y
maximizes the function S (U , Y ) .
This statement is graphed for the case that the internal variable can take
only two values, Y1 and Y2 . For the curves graphed, value Y1 is more probable
than value Y2 .
Use entropy as independent variable. Any monotonic function is 1to-1, and can be inverted. Consequently, we can
U
invert the function S (U ) , and obtain the function
U = U (S )
T
The two functions S (U ) and U (S ) are alternative
ways to describe the same thermodynamic model of
the same isolated system. The two functions
1
correspond to the same curve on the (S,U ) plane.
S
(
)
Once the function U S is known, the
T
temperature is
∂U (S )
.
T=
∂S
This equation expresses the temperature as a
U
function of the entropy, T (S ) . The temperature is
S
the slope of the U (S ) curve, while the energy is the
area under the curve T (S ) .
When the function S (U ) is convex, the function T (S ) monotonically
increases. We will soon see an example when the function S (U ) is nonconvex.
Energy U (S , Y ) . For a rectangular block in a field of gravity, the number
of quantum states of the block is independent of the orientation of the block, but
March 7, 2009
Free Energy - 4
Statistical Mechanics
http://imechanica.org/node/4878
Z. Suo
the energy of the block does vary with the orientation. In this case, the
orientation of the block may be regarded as an internal variable. We wish to
express the energy as a function of the orientation.
In generic terms, when Y is fixed, S is an increasing function of U. That
is, for an isolated system with an internal constraint, the more energy the system
has, the more quantum states the system has. Consequently, we can invert the
function S (U , Y ) , and obtain the function U (S , Y ) . Of course, the two functions
characterize the same system. The previous
statement of maximal number of quantum S
Y1
Y2
states is equivalent to the following statement:
At a constant S , of all possible value of S
Y, the most probable value of Y minimizes the
function U (S , Y ) . This statement is graphed for
the case that the internal variable can take only
two values, Y1 and Y2 . For the curves graphed,
value Y1 is more probable than value Y2 .
U
U
On the choices of the independent
variable. In the above, we have modeled a system by a function of a single
variable,
S = S (U ) .
This function is specific to the system, and can be determined by experimental
measurements. Once the function S (U ) is known, we can obtain another
function T (U ) by taking derivative, namely,
1
∂S (U )
=
.
(
)
TU
∂U
Thus, the system has three variables: S, U, and T. Any one of them can be chosen
to be an independent variable, while the other two can be expressed in terms of
the choice by using the above two equations. This procedure involves inverting
functions, so we should be mindful if a function is 1-to-1.
We can introduce other variables, such as heat capacitance C . However,
we usually do not use C as an independent variable, partly because the function
C (T ) is usually not 1-to-1.
One difficulty in learning thermodynamics is to learn these alternative
choices of the independent variable. So far we have been dealing systems with a
single independent variable. The number of choices will proliferate when we look
at systems with more independent variables.
Use temperature as independent variable. We can rewrite the
above equation as
dU = C (T )dT .
We can also write a similar expression for entropy,
March 7, 2009
Free Energy - 5
Statistical Mechanics
http://imechanica.org/node/4878
Z. Suo
C (T )
dT .
T
Once the function C (T ) , is known, the pair of equations above can be integrated
to obtain U (T ) and S (T ) . That is, we have used the temperature as the
independent variable.
Free energy F (T , Y ) . A system can be held at a constant temperature
when a device adds energy to, or extracts energy from, the system. Such a device
is called a thermostat. For example, a mixture of ice and water is a thermostat
that holds the temperature at 0C. When a system is in thermal contact with the
mixture of ice and water, the amount of ice increases if the system draws energy
from the mixture, and the amount of ice decreases if the system gives heat to the
mixture. One can construct a thermostat to hold any other temperature.
We regard the system and the thermostat together as a composite. The
composite is an isolated system with two internal variables: the energy U in the
system, and the internal variable Y. A pair of values (U , Y ) specifies a macrostate
of the composite. The entropy of the system relates to the two internal variables
by the known function S (U , Y ) . Of course, we can invert the function to obtain
U (S , Y ) .
dS =
System
Y
U
energy
Thermostat
TR
U tot − U
Isolated system
Let TR be the temperature fixed by the thermostat. When the system
draws energy U from the thermostat, the entropy of the thermostat reduces by
U / TR . Consequently, the entropy of the macrostate (U , Y ) of the composite is
U
S (U , Y ) −
.
TR
According to the fundamental postulate, of all possible values of the
internal variables, the most probable values maximize the above entropy of the
macrostate. The above maximization is equivalent to the following minimization.
Of all possible values of the internal variables, the most probable values minimize
the following function:
U (S , Y ) − TR S .
This is a function of S and Y. Setting the variation with respect to S to
zero, we obtain that
∂U (S , Y )
TR =
.
∂S
March 7, 2009
Free Energy - 6
Statistical Mechanics
http://imechanica.org/node/4878
Z. Suo
This equation recovers a fact we already know: When the system is in thermal
equilibrium with the thermostat, the temperature of the system equals that of the
thermostat.
Assuming the function U (S , Y ) is known, the above equation defines the
function T (S , Y ) . Here we have dropped the subscript R. Further assuming that
the temperature and the entropy is 1-to-1, we can invert the function T (S , Y ) , and
obtain the function S (T , Y ) . Consequently, we can rewrite the above function as
F (T , Y ) = U − TS ,
The function F (T , Y ) now contains quantities of the system alone, and is known
as the Helmholtz free energy of the system.
When the system is held at a fixed temperature (i.e., in thermal
equilibrium with a thermostat), of all values of the internal variable Y, the most
probable value minimizes the free energy F (T , Y ) . In this minimization, the
temperature is not a variable, but is fixed by the thermostat. Both the energy and
the entropy vary.
Taking differential of the free energy F = U − TS , we obtain that
dF = dU − TdS − SdT
F
When Y is held constant, dU = TdS , the above
equation becomes that
dF = − SdT .
Consequently, the entropy relates to a partial
differential coefficient of the free energy:
∂F (T , Y )
.
S=−
∂T
For a constant value of Y, the function F (T , Y ) is
T
a monotonically decreasing function of the
temperature.
An alternative way to introduce of the free energy. The above way
of introducing the free energy comes directly from the fundamental postulate, but
does leave it vague as to what the free energy really is. In the following alternative
introduction, we characterize the system by the function U (S , Y ) , using entropy
as an independent variable, rather than energy. We regard the system and the
thermostat together as a composite. The composite is now a system with variable
energy. We adjust the of energy of the composite such that the entropy of the
composite, Stot , is fixed.
When the system gains entropy S, the thermostat loses entropy by the
same amount. Consequently, this transfer of entropy causes the thermostat to
lose energy by TR S . The sum of the energy of the system and that of the
thermostat gives the energy of the composite:
U (S , Y ) − TR S .
The composite has two internal variables: the entropy of the system, S, and the
internal variable of the system, Y. Recall that the composite is a system with fixed
March 7, 2009
Free Energy - 7
http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
entropy. Consequently, of all values of the internal variables of the composite, S
and Y, the most probable ones minimize the energy of the composite. The
remainder of the development will be the same as the previous way of introducing
free energy.
System
Y
S
entropy
Thermostat
TR
Stot − S
System with variable energy
This alternative introduction makes it clear that the free energy is the total
energy of the composite of the system and the thermostat in thermal equilibrium.
Co-existent phases of a substance. To illustrate the process of
thermodynamic modeling, consider a mixture of ice and water, coexistent at the
melting temperature Tm . When energy is added to the mixture, the amount of
water increases at the expense of the amount of ice, but the temperature remains
constant.
We would like to trace this empirical observation back to the
fundamental postulate, using a thermodynamic model.
U
S
water
ice
water
ice
T
Tm
Tm
T
A substance is an aggregate of a large number of a single species of
molecules. The entropy and the energy of the piece of the substance, S and U, are
proportional to the number of molecules in the piece, N. The entropy per
molecule of the substance is
s=
S
,
N
u=
U
.
N
and the energy per molecule is
March 7, 2009
Free Energy - 8
Statistical Mechanics
http://imechanica.org/node/4878
Z. Suo
The function
s = s (u )
is specific to the substance, and is independent of the size and shape of the piece.
The temperature is given by
1 ∂s (u )
=
.
T
∂u
The substance can be in two phases, A′ and A′′ . We may regard the two
phases as distinct systems, one characterized by function s′(u′) , and the other by
s′′(u′′) . Let N ′ be the number of molecule in one phase, and N ′′ be the number
of molecules in the other phase. When the two phases coexist, molecules can
detach from one phase and attach to the other. The total number of molecules in
the mixture, N, is the sum of the numbers of molecules in the two phases:
N = N ′ + N ′′ .
We neglect energy associated with the phase boundaries, so that the energy of the
two-phase mixture is the sum of the energies of the two phases:
U = N ′u′ + N ′′u′′ .
Similarly, the entropy of the two-phase mixture is the sum of the entropies of the
two phases:
S = N ′s′ + N ′′s′′ .
The above equations are known as the rules of mixture.
When N and U are fixed, the mixture is an isolated system with internal
variables: N ′ , N ′′ , u′ , u′′ . According to the fundamental postulate, of all values
of the internal variables, the most probable ones maximize the entropy of the
mixture, subject to the constraint of the fixed N and U .
This problem can be readily solved in a number of ways. For example,
consider the function
f ( N ′, N ′′, u′, u′′,α , β )
= N ′s′(u′) + N ′′s′′(u′′) + α ( N − N ′ − N ′′) + β (U − N ′u′ − N ′′u′′)
where α and β are Lagrange multipliers. The two functions, s′(u′) and s′′(u′′) ,
are known. The six variables listed for the function are regarded as independent,
and we require that the partial derivative of f with respect to each of the variable
vanish. This requirement results in six algebraic equations that determine the six
variables.
Thus, ∂f / ∂α = 0 recovers that constraint that the number of molecules in
the mixture is conserved, and ∂f / ∂β = 0 recovers that constraint that the energy
in the mixture is conserved.
Setting ∂f / ∂N ′ = ∂f / ∂N ′′ = 0 , we obtain that
s′(u′) − βu′ = s′′(u′′) − βu′′ = α .
We can solve for β , giving
March 7, 2009
Free Energy - 9
http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
s′ − s′′
=β .
u′ − u′′
Setting ∂f / ∂U ′ = ∂f / ∂U ′′ = 0 , we obtain that
∂s′(u′) ∂s′′(u′′)
=β .
=
∂u′′
∂u′
Thus, co-existent phases equalize their temperature. The Lagrange multiplier β
is the inverse of the melting temperature, β = 1 / Tm . The latent heat is given by
L = u′′ − u′
The difference in entropies of the two co-existent phases is
u′′ − u′
.
s′′ − s′ =
Tm
S
A′′
Ns′′
N ′s′ + N ′′s′′
Ns′
A′
Nu′′
Nu′
N ′u′ + N ′′u′′
U
Graphical representation of the condition for coexistent phases.
We can also read the above equations off a graph. Let S and U be the entropy and
the energy of the mixture of two phases. In the plane (S,U ) , each point represent
a particular mixture of the two phases.
Given a function s′(u′) , the set of points ( Nu′, Ns′) is a curve on the (U , S )
plane, representing the substance when all molecules are in phase A′ . Similarly,
given a function s′′(u′′) , the set of points ( Nu′′, Ns′′) is a curve on the (S,U ) plane,
representing the substance when all molecules are in phase A′′ .
Now pick one particular point ( Nu′, Ns′) on one curve, and pick another
particular point ( Nu′′, Ns′′) on the other curve. Recall the rules of mixture:
N = N ′ + N ′′ ,
U = N ′u′ + N ′′u′′ ,
March 7, 2009
Free Energy - 10
http://imechanica.org/node/4878
Statistical Mechanics
Z. Suo
S = N ′s′ + N ′′s′′ .
Consequently, the energy and the entropy of the mixture (U , S ) correspond to a
point on the straight line connecting the two points ( Nu′, Ns′) and ( Nu′′, Ns′′) .
S
A′′
Ns′′
N ′s′ + N ′′s′′
Ns′
A′
U
T
A′′
Tm
A′
Nu′
N ′u′ + N ′′u′′
Nu′′
U
When the energy U of the mixture is fixed, the entropy S of the mixture is
maximized when the straight line is tangent to both curves. Reading off the graph,
this condition corresponds to
s′ − s′′ ∂s′(u′) ∂s′′(u′′)
=
=
.
∂u′
∂u′′
u′ − u′′
Metastable phase. The graphic representation also sheds insight into a
condition known as metastability. When all molecules aggregate in a single phase,
say A′′ , the thermodynamic state of the substance is represented by a point on
the curve s′′(u′′) . Assume that this curve is convex. At a high temperature, the
single phase is stable. Now we gradually lower the temperature of the system.
March 7, 2009
Free Energy - 11
Statistical Mechanics
http://imechanica.org/node/4878
Z. Suo
Below the temperature Tm , the single phase is unstable. However, to form a
mixture of two phases, the new phase has to nucleate within the old phase. This
nucleus can be difficult to form. Consequently, the substance main remain in the
old phase even when the temperature drops below the melting temperature. This
phase is known as a metastable phase.
Examine co-existent phases using the function u(s ) . For a
substance with a nonconvex function s (u ) , part of the function will not be
realized, and will be replaced by a common tangent line. The two tangent points
represent coexistent phases. We now use the function u(s ) to examine this
situation
When the function u(s ) is nonconvex, the function T (s ) is not a
monotonically increasing function. The temperature for the phase transition, Tt ,
is the slope of the common tangent, so that
u′ − u′′ = Tt (s′ − s′′) .
On the (s, T ) plane, the area under the curve
u
T (s ) represents the energy u(s ) . Inspecting the
u′′
graph, we can readily see that the above condition
means that Tt , is at the level such that the two
shaded areas are equal. This graphic interpretation
is known as Maxwell’s rule.
Also marked on the u(s ) curve are the two
u′
s
inflection points, where
∂ 2 u(s )
=0.
T
∂s 2
The part of the u(s ) curve between the inflection
points is concave, where a single phase is unstable.
Tt
Parts of the u(s ) curve are convex, but are between
s
the interval (s′, s′′) . These parts correspond to
s′
s′′
metastable phases.
Examine co-existent phases using the free energy. Consider a
mixture of two phases held at a temperature T. Let the free energy of the two
phases be f ′(T ) and f ′′(T ) . The free energy
of the mixture is.
F
F = N ′f ′(T ) + N ′′f ′′(T ) .
A′
The number of molecules in one phase is the
internal variable, to be selected to minimize
the total free energy. The equation
f ′(T ) = f ′′(T )
A′′
T
Tt
March 7, 2009
Free Energy - 12
Statistical Mechanics
http://imechanica.org/node/4878
Z. Suo
determines the phase-transition temperature Tc . When T < Tc , all molecules are
in one phase. When T > Tc all molecules are in the other phase.
Phase transition of the second kind. As another illustration of
thermodynamic modeling, consider the following experimental observation. A
crystal has a rectangular symmetry at a high temperature. When the temperature
drops below a critical value, Tc , the crystal spontaneously distorts by an angle α .
Because of the symmetry, the distortion can go in both directions. Below the
critical temperature, the angle of distortion is observed to vary with the
temperature.
α
T
Tc
α
T < Tc
T > Tc
We regard the angle of distortion α as an internal variable, also known as
an order parameter. The thermodynamics of the crystal is characterized by the
free-energy function F (T ,α ) . To model the experimental observation, we assume
that the free-energy function has the following form:
1
1
F (T ,α ) = C (T ) + A(T − Tc )α 2 + Bα 4 ,
2
4
where A and B are positive constants. This form is a Taylor expansion in the
powers of α . Due to symmetry, the crystal is equally likely to distort in two
directions, so that we keep the even powers of α . Let us check if this function
does reproduce the observed phase transition.
When T > Tc , the coefficient of the α 2 term is positive, so that the the
equilibrium state is α = 0 . In this case, the α 4 term is unnecessary to describe
the behavior of the crystal.
When T < Tc , the coefficient of the α 2 term is negative, and the energy is
no longer minimal at α = 0 . Instead, the energy is minimal at two nonezero
angles, denoted as the spontaneous angles, ± α s . In this case, the α 4 term will
ensure that energy goes up again when α is large. Note that
March 7, 2009
Free Energy - 13
Statistical Mechanics
http://imechanica.org/node/4878
Z. Suo
∂F (T ,α )
= A(T − Tc )α + Bα 3 .
∂α
Setting ∂F (T ,α )/ ∂α = 0 , we find the spontaneous values of the angle:
α s = ± A(Tc − T )/ B .
A significant prediction of this model is
1/2
α s ~ (Tc − T ) .
This prediction should be compared with the angle of distortion observed in the
experiment.
F
T > Tc
T < Tc
α
March 7, 2009
Free Energy - 14