PHYSICS 210A : STATISTICAL PHYSICS
HW ASSIGNMENT #1
(1) Let p(x) = Pr[X = x] where X is a discrete random variable and both X and x are
taken from an ‘alphabet’ X . Let p(x, y) be a normalized joint probability distribution on
two
Prandom variables X ∈ X and Y ∈ Y. The entropy of the joint distribution is S(X, Y ) =
− x,y p(x, y) log p(x, y). The conditional
probability p(y|x) for y given x is defined as
P
p(y|x) = p(x, y)/p(x), where p(x) = y p(x, y).
(a) Show that the conditional entropy S(Y |X) = −
P
x,y
p(x, y) log p(y|x) satisfies
S(Y |X) = S(X, Y ) − S(X) .
Thus, conditioning reduces the entropy, and the entropy of a pair of random variables is the sum of the entropy of one plus the conditional entropy of the other.
(b) The mutual information I(X, Y ) is
I(X, Y ) =
X
x,y
p(x, y)
p(x, y) log
p(x) p(y)
.
Show that
I(X, Y ) = S(X) + S(Y ) − S(X, Y ) .
(c) Show that S(Y |X) ≤ S(Y ) and that the equality holds only when X and Y are independently distributed.
(2) Compute the information entropy in the Fall 2012 Physics 140A grade distribution. See
http://www-physics.ucsd.edu/students/courses/fall2012/physics140a/index.html.
(3) Study carefully Fall 2011 Physics 140A homework problem 2.2. Suppose I have three
bags. Initially, bag #1 contains a quarter, bag #2 contains a dime, and bag #3 contains
two nickels. At each time step, I choose two bags randomly and
Prandomly exchange one
coin from each bag. The time evolution satisfies Pi (t + 1) =
j Yij Pj (t), where Yij =
P (i , t + 1 | j , t) is the conditional probability that the system is in state i at time t + 1 given
that it was in state j at time t.
(a) How many configurations are there for this system?
P
(b) Construct the transition matrix Yij and verify that i Yij = 1.
(c) Find the eigenvalues of Y (you may want to use something like Mathematica).
(d) Find the equilibrium distribution Pieq .
1
(4) Study carefully Spring 2010 Physics 210A homework problem 1.4. Consider a modification of this problem, where the Hamiltonian describing two point particles is
Ĥ =
p2
P2
+ 12 KX 2 +
+ 1 kx2 .
2M
2m 2
The particles undergo elastic collisions with a wall at x = 0, and with each other as well.
The particle of mass m is constrained always to lie between the wall and the other particle,
i.e. 0 ≤ x ≤ X. When the particles coincide, they undergo an elastic
pcollision. To analyze
this system, choose an energy scale E0 , a momentum scale P0 = M E0 , a length scale
p
X0 = E0 /K, and define the dimensionless variables r = m/M and s = k/K. Then in
rescaled variables
p̄2
+ 1 sx̄2 .
Ĥ = 21 P̄ 2 + 21 X̄ 2 +
2r 2
It is notationally convenient to drop the bars.
(a) The microcanonical distribution is
̺(P, p, X, x) =
δ(E − Ĥ) Θ(x) Θ(X − x)
Tr δ(E − Ĥ) Θ(x) Θ(X − x)
Find the microcanonical average hXi. Compute the corresponding average if the
particles are allowed to pass through one another.
(b) Compute the microcanonical average rate γ at which the mass m particle undergoes
collisions with the wall.
(c) Write a computer program to simulate this system. Compare microcanonical and
numerical averages for several sets of (r, s) values. Plot the Poincaré section of P
versus X for those times that x = 0.
2