Information and
Thermodynamic
Entropy
or, Waiting for Landauer
John D. Norton
Department of History and Philosophy of Science
Center for Philosophy of Science
University of Pittsburgh
CARL FRIEDRICH VON WEIZSÄCKER LECTURES
UNIVERSITY OF HAMBURG
June 2010
1
Philosophy and Physics
Information
ideas and concepts
=
Entropy
heat, work,
thermodynamics
And why not?
Mass = Energy
Particles = Waves
Geometry = Gravity
….
Time = Money
2
This Talk
Background
Foreground
Maxwell’s demon and the
A dilemma for information theoretic exorcisms
molecular challenge to the second law
of thermodynamics.
Exorcism by fluctuations
Exorcism by principle
Szilard’s Principle,
Landauer’s principle
Failed proofs of Landauer’s Principle
Thermalization, Compression of phase
space
Information entropy, Indirect proof
The standard inventory of processes in
the thermodynmics of computation neglects
fluctuations.
3
Maxwell’s
demon
4
The original conception
Demon operates
door intelligently
Divided chamber
with a kinetic gas.
J. C. Maxwell in a letter to
P. G. Tait, 11th December
1867
“…the hot system has got
hotter and the cold system
colder and yet no work
has been done, only the
intelligence of a very
observant and neatfingered being has been
employed.”
“[T]he 2nd law of thermodynamics has the same degree of truth as
the statement that if you throw a tumblerful of water into the sea
you cannot get the same tumblerful of water out again.”
5
The Devil lives
in the details of Brownian motion and other fluctuations
“…we see under out eyes now motion
transformed into heat by friction, now
heat changed inversely into motion,
and that without loss since the
movement lasts forever. That is the
contrary of the principle of Carnot.”
Poincaré, 1907
Could these momentary, miniature
violations of the second law be accumulated
to large-scale violations?
Guoy (1888), Svedberg
(1907) designed minimachines with that purpose.
6
Exorcism
by Fluctuation
7
Marian Smoluchowski, 1912
The best known of
many examples.
Trapdoor hinged so that fast molecules moving from
left to right swing it open and pass, but not vice versa.
BUT
AND
SO
The trapdoor must
be very light so a
molecule can
swing it open.
The trapdoor has its
own thermal energy
of kT/2 per degree of
freedom.
The trapdoor will flap
about wildly and let
molecules pass in both
directions.
The second law holds on average only over time.
Machines that try to accumulate fluctuations are disrupted fatally by them.
8
Szilard’s
One-Molecule
Engine
9
Simplest case of fluctuations
Many molecules
A few molecules
One molecule
Can a demon exploit
these fluctuations?
10
The One-Molecule Engine
Szilard 1929
A partition is inserted
to trap the molecule on
one side.
Initial state
Heat kT ln 2 is drawn
from the heat bath and
converted fully to work.
Cycle is completed and
second law is violated?
The gas undergoes a
reversible, isothermal
expansion to its
original state.
…&*%#???!!!
Szilard’s proposal: Second Law is protected by the entropy cost of the operation of demon.
Proof by
“working
backwards”
The heat bath
entropy
decreases by
k ln 2.
There must be
an entropy
creation of of k
ln2 somewhere
else.
Entropy k ln 2 must be created
by the demon in the process
of measuring which side
holds the molecule.
11
Exorcism
by principle
12
Szilard’s Principle
Von Neumann 1932
Brillouin 1951+…
Acquisition of one bit of
information creates k ln 2
of thermodynamic entropy.
Proof:
By “working backwards.”
By suggestive thought
experiments.
(e.g. Brillouin’s torch)
versus
Landauer’s Principle
Landauer 1961
Bennett 1987+…
Erasure of one bit of
information creates k ln 2 of
thermodynamic entropy.
Szilard’s principle is false.
Real entropy cost only taken when
naturalized demon erases the memory
of the position of the molecule
Proof: …???...
13
Bennett’s Machine for Dissipationless Measurement…
FAILS
Measurement apparatus, designed by the author to fit
the Szilard engine, determines which half of the
cylinder the molecule is trapped in without doing
appreciable work. A slightly modified Szilard engine
sits near the top of the apparatus (1) within a boatshaped frame; a second pair of pistons has replaced
part of the cylinder wall. Below the frame is a key,
whose position on a locking pin indicates the state of
the machine's memory. At the start of the
measurement the memory is in a neutral state, and the
partition has been lowered so that the molecule is
trapped in one side of the apparatus. To begin the
measurement (2) the key is moved up so that it
disengages from the locking pin and engages a "keel"
at the bottom of the frame. Then the frame is pressed
down (3). The piston in the half of the cylinder
containing no molecule is able to desend completely,
but the piston in the other half cannot, because of the
pressure of the molecule. As a result the frame tilts
and the keel pushes the key to one side. The key, in its
new position. is moved down to engage the locking pin
(4), and the frame is allowed to move back up (5).
undoing any work that was done in compressing the
molecule when the frame was pressed down. The
key's position indicates which half of the cylinder the
molecule is in, but the work required for the operation
can be made negligible To reverse the operation one
would do the steps in reverse order.
…is fatally disrupted by fluctuations that leave
the keel rocking wildly.
Charles H. Bennett, “Demons, Engines and the
Second Law,” Scientific American 257(5):108-116
(November, 1987).
14
A dilemma
for information
theoretic exorcisms
15
Do information
theoretic ideas reveal why
the demon must fail?
EITHER
Total system =
gas + demon +
all surrounding.
Canonically thermal
= obeys your favorite
version of the second
law.
Demon’s failure assured by our
decision to consider only
system that it cannot breach.
the total system IS
canonically thermal.
(sound horn)
OR
the total system is NOT
canonically thermal.
Principles need independent
justifications which are not
delivered.
(profound horn)
Cannot
have
both!
Profound
“ …the real reason Maxwell’s demon
cannot violate the second law
…uncovered only recently… energy
requirements of computers.”
Bennett, 1987.
Earman and
Norton,
1998, 1999,
“Exorcist
XIV…”
(…and cannot? Zhang and Zhang
pressure demon.)
and
Sound
Deduce the principles (Szilard’s,
Landauer’s) from the second law
by working backwards.
16
Failed proofs of
Landauer’s Principle
17
18
1. Thermalization
Initial data
L or R
Reversible
isothermal
compression passes
heat kT ln 2 to heat
bath.
Irreversible
expansion
“thermalization”
!!!
Entropy created in this illadvised, dissipative step.
Data reset to L
Entropy k ln 2
created in heat
bath
!!!
Proof shows only that an
Mustn’t we thermalize so the
inefficiently designed erasure
procedure creates entropy.
No demonstration that all must.
procedure works with arbitrary data?
No demonstration that thermalization is
the only way to make procedure robust.
19
20
2. Phase Volume Compression
aka “many to one argument”
Boltzmann
statistical
mechanics
thermodynamic
entropy
= k ln (accessible phase volume)
“random” data
occupies twice the
phase volume of
reset data
Erasure halves
phase volume.
Erasure reduces
entropy of memory
by k ln 2.
Entropy k ln 2 must be
created in surroundings to
conserve phase volume.
21
2. Phase Volume Compression
FAILS
aka “many to one argument”
“random” data
DOES NOT occupy
twice the phase
volume of
reset data
It occupies the
same phase volume.
Confusion
with
thermalized
data
22
A Ruinous Sense of “Reversible”
Random data
and
insertion of
the partition
removal of
the partition
thermalized data
have the same entropy
because they are connected
by a reversible, adiabatic
process???
DS = 0
random data
thermalized data
No. Under this sense
of reversible,
entropy ceases to be
a state function.
DS = k ln 2
23
24
3. Information-theoretic Entropy “p ln p”
Information
entropy
S = - k Si
inf
Pi ln Pi
“random” data
PL = PR = 1/2
Sinf = k ln 2
reset data
PL = 1; PR = 0
Sinf = 0
Hence erasure reduces the entropy
of the memory by k ln 2, which
must appear in surroundings.
But…
in this
case,
Information
entropy
does
NOT
equal
Thermodynamic
entropy
Thermodynamic entropy is
attached to a probability only in
special cases. Not this one.
25
What it takes…
Information
entropy
“p ln p”
DOES
equal
Thermodynamic
entropy
Clausius dS = dQrev/T
IF…
A system is
distributed
canonically over
its phase space
AND
p(x) = exp( -E(x)/kT) / Z
Z normalizes
All regions of phase space of non-zero E(x)
are accessible to the system over time.
For details of the proof
and the importance of the
accessibility condition,
see Norton, “Eaters of
the Lotus,” 2005.
Accessibility condition FAILS for
“random data” since only half of
phase space is accessible.
26
27
Ladyman et al., “The
connection between logical
and thermodynamic
irreversibility,” 2007.
4. An Indirect Proof
OneMolecule
gas
insert
partition
or
isothermal
reversible
expansion
dissipationlessly
Reduces entropy of
heat bath by k ln 2.
detect gas state
OneMolecule
memory
Original proof given only in
terms of quantities of heat
passed among components.
or
shift cell
to match
perform any
erasure
Assume second law of
thermodynamics holds on
average.
Erasure must create
entropy k ln 2 on
average.
28
4. An Indirect Proof
Fails
OneMolecule
gas
insert
partition
or
dissipationlessly
Reduces entropy of
heat bath by k ln 2.
detect gas state
OneMolecule
memory
isothermal
reversible
expansion
or
shift cell
to match
Net effect is a reduction of entropy of
heat bath. Second law violated even in
statistical form.
(Earman and Norton, 1999, “no-erasure” demon.)
Dissipationlessly
detect memory state.
If R, shift to L.
Final step is a dissipationless erasure
built out of processes routinely
admitted in this literature.
29
The standard
inventory of
processes
30
We may…
Exploit the fluctuations of single
molecule in a chamber at will.
Inventory read from
steps in Ladyman et
al. proofs.
Insert and remove
a partition
Perform reversible,
isothermal expansions
and contractions
31
We may…
Detect the location of
?
the molecule without
dissipation.
?
Shift between equal
entropy states without
dissipation.
Memory
Trigger new processes
according to the location
detected.
R
Gas
?
L
32
We are selectively ignoring fluctuations.
Dissipationless detection
disrupted by fluctuations.
Reversible, isothermal
expansion and contraction
does not complete due
thermal motions of piston.
Need to demonstrate that each of
these processes is admissible. None
is primitive.
Inserted partition bounces
off wall unless held by…
what?
Friction?? Spring loaded
pin??...
Inventory assembled inconsistently.
It concentrates on fluctuations when
convenient; it ignores them when not.
33
Conclusion
34
Conclusions
Is a Maxwell demon
possible?
The best analysis is the Smoluchowski fluctuation exorcism
of 1912. It is not a proof but a plausibility argument.
Information principle
based exorcisms are
troubled.
They either presume the result or base the exorcism on
principles posited without good foundation.
Efforts to prove Landauer’s
Principle have failed.
…even those that presume a form of the second law. It
is still speculation and now looks dubious.
Thermodynamics of
computation has incoherent
foundations.
The standard inventory of processes admits composite
processes that violate the second law and erase without
dissipation.
Its inventory of processes is
assembled inconsistently.
It selectively considers and ignores fluctuation
phenomena according to the result sought.
35
Read
all about it.
36
37
38
Commercials
39
40
Finis
41