The microcanonical ensemble
Finding the probability distribution
We consider an isolated system in the sense that
the energy is a constant of motion.
N 
E  H ( p, q)  E  
We are not able to derive  from first principles
Two typical alternative approaches
Postulate of Equal a Priori Probability
Construct entropy expression from it and
show that the result is consistent
with thermodynamics
Use (information) entropy as starting
concept
Derive  from maximum entropy
principle
Information entropy*
*Introduced
by Shannon. See textbook and E.T. Jaynes, Phys. Rev. 106, 620 (1957)
Idea: Find a constructive least biased criterion for setting up probability
distributions on the basis of only partial knowledge (missing information)
What do we mean by that?
Let’s start with an old mathematical problem
Consider a quantity x with discrete random values x1 ,
Assume we do not know the probabilities
x2 , ..., xn
p1 , p2 , ..., pn
We have some information however, namely
p
n
i 1
i
1
and we know the average of the function f(x) (we will also consider cases
where we know more than one average)
n
f ( x )   pi f ( xi )
i 1
With this information, can we calculate an average of the function g(x) ?
To do so we need all the
n
p
i 1
i
 1 and
n
p1, p2 , ..., pn but we have only the 2 equations
f ( x )   pi f ( xi )
i 1
we are lacking (n-2) additional equations
What can we do with this underdetermined problem?
There may be more than one probability distribution creating
We want the one which requires no further assumptions
We do not want to “prefer” any pi if there is no reason to do so
n
f ( x )   pi f ( xi )
i 1
Information theory tells us how to find this unbiased distribution
(we call the probabilities now  rather than p)
Shannon defined information entropy, Si:
Si  k  n lnn
n

n
or
Si  k  dpdq  ( p, q )ln  ( p, q )
for continuous distribution
with normalization
n
1
 dpdq  ( p, q)  1
We make plausible that:
- Si is a measure of our ignorance of the microstate of the system.
More quantitatively
- Si is a measure of the width of the distribution of the n.
Let’s consider an extreme case:
An experiment with N potential outcomes (such as rolling dice)
However:
Outcome 1 has the probability 1=1
Si  k  n lnn  0
Outcome 2,3,… ,N have
n=0
n
-Our ignorance regarding the outcome of the experiment is zero.
-We know precisely what will happen
- the probability distribution is sharp (a delta peak)
Let’s make the distribution broader:
Outcome 1 has the probability 1=1/2
Outcome 2 has the probability 2=1/2
Outcome 3,4, … ,N have
n=0
Let’s consider the more general case:
Outcome 1 has the probability 1=1/M
Outcome 2 has the probability 2=1/M
.
.
.
Outcome M has the probability M=1/M
Outcome M+1, … ,N have
n=0
1 1 1 1

Si  k  n lnn  k  ln  ln  0  ...  0 
2 2 2 2

n
1
1

  k  ln 2  ln 2   k ln 2
2
2

Si  k  n lnn
n
1
1
1
1
1
 1

 k  ln  ln  ...  ln  ...  0 
M M
M M M M

 k ln M
So far our considerations suggest:
Ignorance/information entropy increases with increasing width of the distribution
Which distribution brings information entropy to a maximum
For simplicity let’s start with an experiment with 2 outcomes
Binary distribution with 1, 2 and 1+ 2=1
Si  k  1ln1  2ln2 
1  2  1
with
Si  k  1ln1  1 1  ln 1 1 

dSi
1 
1
 k  ln1  1  ln 1  1   1  1 


k
ln

d i
1


1  1

1 
0.7
0.6
maximum
0.5
dSi

 k ln 1  0
d i
1  1
1
1
1  1
Si
0.4
1  1/ 2  2
uniform distribution
0.3
0.2
0.1
0.0
0.0
0.2
0.4
0.6
0.8
1
1.0
Lagrange multiplier technique
Once again
Si  k  1ln1  2ln2 
at maximum
with
1  2  1
constraint
F (1, 2 , )  k  1ln1  2ln2     1  2 1
F
  k  ln 1  1    0
1
F
  k  ln  2  1    0
 2
F
 1   2  1  0

From
http://en.wikipedia.org/wiki/File:LagrangeMultipliers2D.svg
Finding an extremum of f(x,y) under the constraint g(x,y)=c.
1  2
1  2  1
1  1/ 2  2
uniform distribution
Let’s use Lagrange multiplier technique to find distribution that maximizes
M
Si  k  n lnn
n 1
M

F ( 1 , 2 ,..., M ,  )  k  n lnn     n  1
n 1
 n1

M
F
 k  ln j  1    0
 j
with
M

n 1
Si
n
max
1
1  2  ...  M  e1 / k 
1   2  ...   M 
1
M
uniform distribution
maximizes entropy
 k ln M
In a microcanonical ensemble where each system has N particles, volume V
and fixed energy between E and E+ the entropy is at maximum in equilibrium.
Distribution function
- When identifying information entropy with thermodynamic entropy
1

const
.


Z (E)
 ( p, q)  
0 otherwise

if E  H ( p, q)  E  
Where Z(E) = # of microstate with energy in [E,E+ ]
called the partition function of the microcanonical ensemble
Information entropy and thermodynamic entropy
When identifying k=kB
S  kB  n lnn
n
has all the properties we expect from the thermodynamic entropy
(for details see textbook)
We show here S is additive
S 1 2  S 1  S 2
S1
 (1) n : probability distribution
for system 1
S2
 (2) m : probability distribution
for system 2
Statistically independence of system 1 and 2
probability of finding system 1 in state n and system 2 in state m
S (1 2)  k B   n(1)  m(2) ln  n(1)  m(2)  k B   n(1)  m(2)  ln  n(1)  ln  m(2) 
n,m
n,m
 k B  m(2)  n(1) ln n(1)  k B  n(1)  m(2) ln m(2)
m
n
n
m
 k B  n(1) ln n(1)  k B  m(2) ln m(2)  S (1)  S (2)
n
m
n(1) m(2)
Relation between entropy and the partition function Z(E)
S  kB  n lnn
n
1
1
 k B 
ln
Z (E)
n Z (E)
 kBlnZ  ( E )
n
1
Z (E)
1
S  kBlnZ (E)
Derivation of Thermodynamics in the microcanonical Ensemble
Where is the temperature ?
In the microcanonical ensemble the energy, E, is fixed
E (S ,V )  U ( S ,V )
with
dU  TdS  PdV
1  S 


T  U V
and
P  S 


T  V U
Let’s derive the ideal gas equation of state from the microcanonical ensemble
despite the fact that there are easier ways to do so
Major task: find Z(U) :=# states in energy shell [U,U+ ]
Z  (U ) 
p
U+ 
1
3N
3N
d
p
d
q
3N

N ! h U  H ( p ,q )U 
U
q
another leftover from qm: phase space quantization, makes Z a dimensionless #
“correct Boltzmann counting”
Solves Gibb’s paradox
requires qm origin of indistinguishability of atoms
We derive it when discussing the classical limit of qm gas
U
N

pi2
 2 m U 
i 1
d
3N
VN
pd q 
N ! h3 N
3N
U
N

pi2
 2 m U 
i 1
pi2
i 1
2m
H 
For a gas of N non-interacting particles we have
1
Z  (U ) 
N ! h3 N
N
d 3 p1 d 3 p2 ...d 3 pN
N
 pi2
3N dim. sphere
in momentum space
 2 mU
i 1
2 m (U  )
2mU
VN
Z  (U ) 
N ! h3 N
U
N

3
3
3
d p1 d p2 ...d pN
pi2
 2 m U 
V N C3 N

N ! h3 N

 2m(U  ) 
3N / 2
  2mU 
3N / 2

V2 dim   S 2dim dr   2 rdr   r 2
i 1
V3N ( pU   2m(U  )) V3N ( pU  2mU )
Remember:
4
V3 dim   S3dim dr   4 r 2 dr   r 3
3
...
V3 N dim   S3 N dim dr   r 3 N 1dr  C3 N r 3 N
3N / 2



U
3
N
/
2


N
 V  U   
1  
   const



 U   


S  kBlnZ (U )


  U 3 N / 2 
3
 kB  N ln V  N ln U     ln 1  
  ln const 




2
 U   



In the thermodynamic limit of
N 
V 
  U
lim a n  0
n 
N
 const
V
for 0  a  1
ln1=0
http://en.wikipedia.org/wiki/Exponentiation

3
S  k B  N ln V  N ln U    

2

  U 3 N / 2 
ln 1  

  U   



 ln const 


3


S  kB  N ln V  N ln U  ln const 
2


1  S 
with


T  U V
P  S 


T  V U
U
3
Nk BT
2
PV  NkBT