Information
and Thermodynamic
Entropy
John D. Norton
Department of History and Philosophy of Science
Center for Philosophy of Science
University of Pittsburgh
Pitt-Tsinghua Summer School for Philosophy of Science
Institute of Science, Technology and Society, Tsinghua University
Center for Philosophy of Science, University of Pittsburgh
At Tsinghua University, Beijing
June 27- July 1, 2011
1
Philosophy and Physics
Information
ideas and concepts
=
Entropy
heat, work,
thermodynamics
And why not?
Mass = Energy
Particles = Waves
Geometry = Gravity
….
Time = Money
2
This Talk
Background
Foreground
Maxwell’s demon and the
Failed proofs of Landauer’s Principle
molecular challenge to the second law
of thermodynamics.
Exorcism by principle
Szilard’s Principle,
Landauer’s principle
Thermalization, Compression of phase space
Information entropy, Indirect proof
The standard inventory of processes in
the thermodynamics of computation neglects
fluctuations.
3
Fluctuations
and Maxwell’s
demon
4
The original conception
Demon operates
door intelligently
Divided chamber
with a kinetic gas.
J. C. Maxwell in a letter to
P. G. Tait, 11th December
1867
“…the hot system has got
hotter and the cold system
colder and yet no work
has been done, only the
intelligence of a very
observant and neatfingered being has been
employed.”
“[T]he 2nd law of thermodynamics has the same degree of truth as
the statement that if you throw a tumblerful of water into the sea
you cannot get the same tumblerful of water out again.”
5
Maxwell’s demon lives
in the details of Brownian motion and other fluctuations
“…we see under out eyes now motion
transformed into heat by friction, now
heat changed inversely into motion,
and that without loss since the
movement lasts forever. That is the
contrary of the principle of Carnot.”
Poincaré, 1907
“One can almost see Maxwell’s demon
at work.”
Poincaré, 1905
Could these momentary, miniature
violations of the second law be accumulated
to large-scale violations?
Guoy (1888), Svedberg
(1907) designed minimachines with that purpose.
6
Szilard’s
One-Molecule
Engine
7
Simplest case of fluctuations
Many molecules
A few molecules
One molecule
Can a demon exploit
these fluctuations?
8
The One-Molecule Engine
Szilard 1929
A partition is inserted
to trap the molecule on
one side.
Initial state
The gas undergoes a
reversible, isothermal
expansion to its
original state.
Work kT ln 2
gained in raising the weight.
It comes from the
heat kT ln 2,
drawn from the heat bath.
Net effect of the completed cycle:
Heat kT ln 2 is drawn
from the heat bath and
fully converted to work.
The total entropy of
the universe
decreases by k ln 2.
The Second Law of
Thermodynamics
is violated.
The One-Molecule Engine
Szilard 1929
A partition is inserted
to trap the molecule on
one side.
Initial state
The gas undergoes a
reversible, isothermal
expansion to its
original state.
Work kT ln 2
gained in raising the weight.
It comes from the
heat kT ln 2,
drawn from the heat bath.
Net effect of the completed cycle:
Heat kT ln 2 is drawn
from the heat bath and
fully converted to work.
The total entropy of
the universe
decreases by k ln 2.
The Second Law of
Thermodynamics
is violated.
Exorcism
by principle
11
Szilard’s Principle
Von Neumann 1932
Brillouin 1951+…
Acquisition of one bit of
information creates k ln 2
of thermodynamic entropy.
Proof:
By “working backwards.”
By suggestive thought
experiments.
(e.g. Brillouin’s torch)
versus
Landauer’s Principle
Landauer 1961
Bennett 1987+…
Erasure of one bit of
information creates k ln 2 of
thermodynamic entropy.
Szilard’s principle is false.
Real entropy cost only taken when
naturalized demon erases the memory
of the position of the molecule
Proof: …???...
12
Failed proofs of
Landauer’s Principle
13
Direct Proofs that model the erasure processes in the
memory device directly.
or
1. Thermalization
An inefficiently designed erasure procedure creates
entropy.
No demonstration that all must.
2. Phase Volume Compression
aka “many to one argument”
3. Information-theoretic Entropy
“p ln p”
Erasure need not compress phase
volume but only rearrange it.
Wrong sort of entropy.
No connection to heat.
Associate entropy with our uncertainty
over which memory cell is occupied.
See: "Eaters of the Lotus: Landauer's Principle and the Return of Maxwell's
Demon." Studies in History and Philosophy of Modern Physics, 36 (2005), pp. 375411.
14
4. Indirect Proof: General Strategy
Process
known to
reduce entropy
coupled
to
Arbitrary
erasure process
Entropy
reduces.
Assume
second law of
thermodynamics
holds on average.
Entropy must
increase on
average.
15
Ladyman et al., “The
connection between logical
and thermodynamic
irreversibility,” 2007.
4. An Indirect Proof
OneMolecule
gas
insert
partition
or
isothermal
reversible
expansion
dissipationlessly
Reduces entropy of
heat bath by k ln 2.
detect gas state
OneMolecule
memory
Original proof given only in
terms of quantities of heat
passed among components.
or
shift cell
to match
perform any
erasure
Assume second law
of thermodynamics
holds on average.
Erasure must create
entropy k ln 2 on
average.
16
4. An Indirect Proof
Fails
Inventory of admissible processes allows:
Processes that erase dissipationlessly
Processes that violate the
second law of thermodynamics,
(without passing heat to surroundings) in
violation of Landauer’s principle.
even in its statistical form.
See: “Waiting for Landauer,” Studies in History and
Philosophy of Modern Physics, forthcoming.
17
Dissipationless Erasure
or
First method.
Second method.
1. Dissipationlessly detect
memory state.
1. Dissipationlessly detect memory state.
2. If R, shift to L.
2. If R, remove and reinsert partition and go to 1.
Else, halt.
18
The Importance of
Fluctuations
19
Exorcism of
Maxwell’s demon
by fluctuations.
Marian Smoluchowski, 1912
The best known of
many examples.
Trapdoor hinged so that fast molecules moving from
left to right swing it open and pass, but not vice versa.
BUT
AND
SO
The trapdoor must
be very light so a
molecule can
swing it open.
The trapdoor has its
own thermal energy
of kT/2 per degree of
freedom.
The trapdoor will flap
about wildly and let
molecules pass in both
directions.
The second law holds on average only over time.
Machines that try to accumulate fluctuations are
disrupted fatally by them.
20
Fluctuations disprupt
Reversible
Expansion and
Compression
21
The Intended Process
Infinitely slow expansion
converts heat to work in the
raising of the mass.
Mass M of piston continually
adjusted so its weight remains in
perfect balance with the mean gas
pressure P= kT/V.
Equilibrium height is
heq = kT/Mg
22
The massive piston…
….is very light since it must be
supported by collisions with a single
molecule. It has mean thermal energy
kT/2 and will fluctuate in position.
Probability density for the piston at
height h
p(h) = (Mg/kT) exp ( -Mgh/kT)
Mean
height = kT/Mg = heq
Standard
deviation = kT/Mg = heq
23
What Happens.
Fluctuations
obliterate the
infinitely slow
expansion
intended
This analysis is approximate.
The exact analysis replaces the
gravitational field with
piston
= 2kT ln (height)
energy
24
Fluctuations disrupt
Measurement and
Detection
25
Bennett’s Machine for Dissipationless Measurement…
FAILS
Measurement apparatus, designed by the author to fit
the Szilard engine, determines which half of the
cylinder the molecule is trapped in without doing
appreciable work. A slightly modified Szilard engine
sits near the top of the apparatus (1) within a boatshaped frame; a second pair of pistons has replaced
part of the cylinder wall. Below the frame is a key,
whose position on a locking pin indicates the state of
the machine's memory. At the start of the
measurement the memory is in a neutral state, and the
partition has been lowered so that the molecule is
trapped in one side of the apparatus. To begin the
measurement (2) the key is moved up so that it
disengages from the locking pin and engages a "keel"
at the bottom of the frame. Then the frame is pressed
down (3). The piston in the half of the cylinder
containing no molecule is able to desend completely,
but the piston in the other half cannot, because of the
pressure of the molecule. As a result the frame tilts
and the keel pushes the key to one side. The key, in its
new position. is moved down to engage the locking pin
(4), and the frame is allowed to move back up (5).
undoing any work that was done in compressing the
molecule when the frame was pressed down. The
key's position indicates which half of the cylinder the
molecule is in, but the work required for the operation
can be made negligible To reverse the operation one
would do the steps in reverse order.
…is fatally disrupted by fluctuations that leave
the keel rocking wildly.
Charles H. Bennett, “Demons, Engines and the
Second Law,” Scientific American 257(5):108-116
(November, 1987).
26
A Measurement Scheme Using Ferromagnets
Charles H. Bennett, “The Thermodynamics of
Computation—A Review,” In. J. Theor. Phys. 21, (1982),
pp. 905-40,
27
A Measurement Scheme Using Ferromagnets
Charles H. Bennett, “The Thermodynamics of
Computation—A Review,” In. J. Theor. Phys. 21, (1982),
pp. 905-40,
28
A General Model of Detection
First step: the detector is
coupled with the target system.
The process intended:
The process is isothermal,
thermodynamically reversible:
• It proceeds infinitely slowly.
• The driver is in equilibrium
with the detector.
The coupling is an
isothermal, reversible
compression of the
detector phase space.
29
A General
“No-Go”
Result
30
Fluctuation Disrupt All Reversible, Isothermal
Processes at Molecular Scales
Intended
process
Actual
process
l=l1
l
l=l2
l=l1
l
l=l2
31
Einstein-Tolman Analysis of Fluctuations
Total system of gas-piston
or target-detector-driver is
canonically distributed.
Different stages l
Probability density
that system is in stage l
Free energy of
stage l
Probability density
for fluctuation to
stage  :
p(x, p) = (1/Z) exp(-E(x,p)/kT)
Different subvolumes of the phase space.
p(l) proportional to Z(l)
Z(l) =
∫
l
exp(-E(x,p)/kT) dxdp
F(l) = - kT ln Z(l)
p(λ) proportional to exp(-F( )/kT)
p(l2)
p(l1)
= exp
(- F( kT)-F( ) )
2
1
32
Equilibrium implies uniform probability over l
Condition for
equilibrium
Probability
distribution over l
∂F/∂l = 0 F(l) = constant
p(l) = constant
p(l1) = p(l2)
since
p(l2)
p(l1)
= exp
(- F( kT)-F( ) )
2
1
Time evolution over phase space
Expected
Actual
33
One-Molecule Gas/Piston System
Overlap of subvolumes
corresponding to stages
h = 0.5H
h=0.75H
h=H
h=1.25H
Slice through phase space.
34
Fluctuations Obliterate Reversible Detection
What we expected:
What happens:
35
What it takes to overcome fluctuations
Enforcing a small
probability gradient…
…requires a
disequilibrium…
…which creates
entropy.
p(l2)
p(l1)
= exp
(-F( kT)-F( ) ) > exp(3) = 20
2
1
F( 1) > F( 2) + 3kT
S( 2)-S( 1) – (E( 2)-E( 1))/T = 3k
Exceeds the entropy k ln2 = 0.69k
tracked by Landauer’s Principle!
No problem for
macroscopic reversible
processes.
F( 1) - F( 2) = 25kT
= mean thermal energy of
ten Oxygen molecules
p( 2)/p( 1) = 7.2 x 1010
36
More Woes
37
Dissipationless Insertion of Partition?
No friction-based
device is allowed to
secure the partition.
With a conservative
Hamiltonian, the partition
will bounce back.
Arrest partition with a
spring-loaded pin?
The pin will
bounce back.
Feynman, ratchet and pawl.
38
In Sum… We are selectively ignoring fluctuations.
Dissipationless detection
disrupted by fluctuations.
Reversible, isothermal
expansion and contraction
does not complete due
thermal motions of piston.
Need to demonstrate that each of
these processes is admissible. None
is primitive.
Inserted partition bounces
off wall unless held by…
what?
Friction?? Spring loaded
pin??...
Inventory assembled inconsistently.
It concentrates on fluctuations when
convenient; it ignores them when not.
39
Conclusions
40
Why should we believe that…
…the reason for the supposed
failure of a Maxwell demon is
localizable into some single
information theoretic process?
(detection? Erasure?)
…the second law obtains
even statistically when we deal
with tiny systems in which
fluctuations dominate?
41
Conclusions
Is a Maxwell demon
possible?
The best analysis is the Smoluchowski fluctuation exorcism
of 1912. It is not a proof but a plausibility argument against
the demon.
Efforts to prove Landauer’s
Principle have failed.
…even those that presume a form of the second law. It
is still speculation and now looks dubious.
Thermodynamics of
computation has incoherent
foundations.
The standard inventory of processes admits composite
processes that violate the second law and erase without
dissipation.
Its inventory of processes is
assembled inconsistently.
It selectively considers and ignores fluctuation
phenomena according to the result sought.
42
http://www.pitt.edu/~jdnorton/lectures/Tsinghua/Tsinghua.html
43
Finis
44
Appendix
45
A dilemma
for information
theoretic exorcisms
46
Do information
theoretic ideas reveal why
the demon must fail?
EITHER
Total system =
gas + demon +
all surrounding.
Canonically thermal
= obeys your favorite
version of the second
law.
Demon’s failure assured by our
decision to consider only
system that it cannot breach.
the total system IS
canonically thermal.
(sound horn)
OR
the total system is NOT
canonically thermal.
Principles need independent
justifications which are not
delivered.
(profound horn)
Cannot
have
both!
Profound
“ …the real reason Maxwell’s demon
cannot violate the second law
…uncovered only recently… energy
requirements of computers.”
Bennett, 1987.
Earman and
Norton,
1998, 1999,
“Exorcist
XIV…”
(…and cannot? Zhang and Zhang
pressure demon.)
and
Sound
Deduce the principles (Szilard’s,
Landauer’s) from the second law
by working backwards.
47
48
1. Thermalization
Initial data
L or R
Reversible
isothermal
compression passes
heat kT ln 2 to heat
bath.
Irreversible
expansion
“thermalization”
!!!
Entropy created in this illadvised, dissipative step.
Data reset to L
Entropy k ln 2
created in heat
bath
!!!
Proof shows only that an
Mustn’t we thermalize so the
inefficiently designed erasure
procedure creates entropy.
No demonstration that all must.
procedure works with arbitrary data?
No demonstration that thermalization is
the only way to make procedure robust.
49
50
2. Phase Volume Compression
aka “many to one argument”
Boltzmann
statistical
mechanics
thermodynamic
entropy
= k ln (accessible phase volume)
“random” data
occupies twice the
phase volume of
reset data
Erasure halves
phase volume.
Erasure reduces
entropy of memory
by k ln 2.
Entropy k ln 2 must be
created in surroundings to
conserve phase volume.
51
2. Phase Volume Compression
FAILS
aka “many to one argument”
“random” data
DOES NOT occupy
twice the phase
volume of
reset data
It occupies the
same phase volume.
Confusion
with
thermalized
data
52
A Ruinous Sense of “Reversible”
Random data
and
insertion of
the partition
removal of
the partition
thermalized data
have the same entropy
because they are connected
by a reversible, adiabatic
process???
DS = 0
random data
thermalized data
No. Under this sense
of reversible,
entropy ceases to be
a state function.
DS = k ln 2
53
54
3. Information-theoretic Entropy “p ln p”
Information
entropy
S = - k Si
inf
Pi ln Pi
“random” data
PL = PR = 1/2
Sinf = k ln 2
reset data
PL = 1; PR = 0
Sinf = 0
Hence erasure reduces the entropy
of the memory by k ln 2, which
must appear in surroundings.
But…
in this
case,
Information
entropy
does
NOT
equal
Thermodynamic
entropy
Thermodynamic entropy is
attached to a probability only in
special cases. Not this one.
55
What it takes…
Information
entropy
“p ln p”
DOES
equal
Thermodynamic
entropy
Clausius dS = dQrev/T
IF…
A system is
distributed
canonically over
its phase space
AND
p(x) = exp( -E(x)/kT) / Z
Z normalizes
All regions of phase space of non-zero E(x)
are accessible to the system over time.
For details of the proof
and the importance of the
accessibility condition,
see Norton, “Eaters of
the Lotus,” 2005.
Accessibility condition FAILS for
“random data” since only half of
phase space is accessible.
56
57
4. An Indirect Proof
Fails
OneMolecule
gas
insert
partition
or
dissipationlessly
Reduces entropy of
heat bath by k ln 2.
detect gas state
OneMolecule
memory
isothermal
reversible
expansion
or
shift cell
to match
Net effect is a reduction of entropy of
heat bath. Second law violated even in
statistical form.
(Earman and Norton, 1999, “no-erasure” demon.)
Dissipationlessly
detect memory state.
If R, shift to L.
Final step is a dissipationless erasure
built out of processes routinely
admitted in this literature.
58
“…the same bit cannot be both the control and the
target of a controlled operation…”
The Most Beautiful Machine 2003
Trunk, prosthesis, compressor, pneumatic cylinder
13,4 x 35,4 x 35,2 in.
“…the observers are supposed to push the ON button. After a while the lid of
the trunk opens, a hand comes out and turns off the machine. The trunk
closes - that's it!”
http://www.kugelbahn.ch/sesam_e.htm
Every negative feedback
control device acts on its
own control bit.
(Thermostat, regulator.)
59
Marian Smoluchowski, 1912
The best known of
many examples.
Exorcism of Maxwell’s
demon by fluctations.
Trapdoor hinged so that fast molecules moving from
left to right swing it open and pass, but not vice versa.
BUT
AND
SO
The trapdoor must
be very light so a
molecule can
swing it open.
The trapdoor has its
own thermal energy
of kT/2 per degree of
freedom.
The trapdoor will flap
about wildly and let
molecules pass in both
directions.
The second law holds on average only over time.
Machines that try to accumulate fluctuations are disrupted fatally by them.
60
The standard
inventory of
processes
61
We may…
Exploit the fluctuations of single
molecule in a chamber at will.
Inventory read from
steps in Ladyman et
al. proofs.
Insert and remove
a partition
Perform reversible,
isothermal expansions
and contractions
62
We may…
Detect the location of
?
the molecule without
dissipation.
?
Shift between equal
entropy states without
dissipation.
Memory
Trigger new processes
according to the location
detected.
R
Gas
?
L
63