Misplaced Idealizations
Entropy, Information and
Maxwell's Demon
John D. Norton
Department of History and Philosophy of Science
Center for Philosophy of Science
University of Pittsburgh
4th Tuebingen Summer School
in History and Philosophy of Science, July 2015
1
This Lecture
The thermodynamics of computation presumes it is
possible to…
Chain molecular-scale computational
steps that are thermodynamically
reversible or nearly so.
steps
Compression and
expansion of
components spaces.
Detection of memory
device states.
Moving data from one
location to another.
Bad Idealization
…
This Lecture
No-Go
result
Thermal fluctuations (noise) prevent completion
of any individual, molecular scale step.
Thermodynamic entropy must be created to
complete each step.
Pfin 
Stot  k ln   k ln O fin
Pinit 
Minimum entropy creation not set by the
logical specification of the computation, but by the
number of steps chained.

Maxwell’s
Demon
4
The Maxwell Era
1867-1905
5
Theory of Heat,
1871, first ed.
Also
Letter to Tait, 1867;
Rayleigh 1871
6
Theory of Heat
Better scan from 1872, 2nd ed.
7
Maxwell’s Proposal
“He will thus,
without expenditure
of work,
air initially at
uniform temperature
raise the
temperature of B
and lower that of A,
in contradiction to
the second law of
thermodynamics.”
8
Maxwell’s Moral: The
Demon Wins
“This is only one of the instances in
which conclusions which we have
drawn from our experience of bodies
consisting of an immense number of
molecules may be found not to be
applicable to the more delicate
observations and experiments
which we may suppose made by
one who can perceive and handle
the individual molecules which we
deal with only in large masses.
In dealing with masses of matter, while we do
not perceive the individual molecules, we are
compelled to adopt what I have described as
the statistical method of calculation, and to
abandon the strict dynamical method, in
which we follow every motion by the
calculus.”
Theory of Heat.
The demon illustrates
that Second Law would fail if
we could manipulate
individual molecules.
No compulsion to
exorcise the demon to
protect the Second Law.
…. Nanotechnology has not yet
overturned the Second Law.
9
The Fluctuation Era
1905-1929
10
Einstein’s Brownian Motion Paper
"On the motion of small particles suspended in liquids at rest
required by the molecular-kinetic theory of heat.” Annalen der
Physik, 17(1905), pp. 549-560. (May 1905; received 11 May
1905)
11
“…no longer strictly valid…”
“If it is really possible to
observe the motion
discussed here …”
“… then classical thermodynamics can no
longer be viewed as strictly valid even for
microscopically distinguishable spaces....”
“… … and an exact determination of the
real size of atoms becomes possible.”
12
Maxwell’s demon lives
in the details of Brownian motion and other fluctuations
“[…] we see under our eyes now motion
transformed into heat by friction, now heat
changed inversely into motion, and that
without loss since the movement lasts
forever. This is the contrary of the principle
of Carnot.
If this be so, to see the world return
backward, we no longer have need of the
infinitely keen eye of Maxwell's demon;
our microscope suffices.”
Poincaré, 1904
Could these momentary, miniature
violations of the second law be accumulated
to large-scale violations? A real Maxwell’s
demon?
Guoy (1888), Svedberg
(1907) designed minimachines with that purpose.
13
Svedberg’s Proposal
Colloid
cools
Charged colloid
particles radiate
their thermal
energy.
Casing
heats
Tuned lead casing
absorbs the
radiation.
…plus many more layers, details
designed to prevent return of heat.
Svedberg, The. “Über die Bedeutung der Eigenbewegung der Teilchen in kolloidalen Lösungen für die Beurteilung
der Gültigkeitsgrenzen des zweiten Haupsatzes der Thermodynamik”.Annalen der Physik, 59 (1907) pp. 451–458.
14
Exorcism of
Maxwell’s demon
by fluctuations.
Marian Smoluchowski, 1912
Trapdoor hinged so that fast molecules moving from
left to right swing it open and pass, but not vice versa.
BUT
AND
SO
The trapdoor must
be very light so a
molecule can
swing it open.
The trapdoor has its
own thermal energy
of kT/2 per degree of
freedom.
The trapdoor will flap
about wildly and let
molecules pass in both
directions.
The second law holds on average only over time.
Machines that try to accumulate fluctuations are
disrupted fatally by them.
15
Marian Smoluchowski, 1912
Other examples of
defeated demons.
Later popularized by Feynman
The second law holds on average only over time.
Machines that try to accumulate fluctuations are
disrupted fatally by them.
16
The Information Era
1929- ????
17
Szilard 1929
“On the decrease of entropy in a
thermodynamic system by the
intervention of intelligent beings.”
Zeitschrift für Physik, 53 (1929),
pp. 840-856.
18
The One-Molecule Engine
Szilard 1929
1
4
2
3
A partition is
Initial
inserted to trap the
molecule on one
side.
state
Work kT ln 2
The gas undergoes a
reversible, isothermal
expansion to its
original state.
gained in raising the weight.
It comes from the
heat kT ln 2,
drawn from the heat bath.
Net effect of the completed cycle:
Heat kT ln 2 is drawn
from the heat bath and
fully converted to work.
The total entropy of
the universe
decreases by k ln 2.
The Second Law of
Thermodynamics
is violated.
Szilard’s Principle
Szilard 1929
Von Neumann 1932
Brillouin 1951+…
Acquisition
of one bit of information
by the demon creates k ln 2
of thermodynamic entropy.
versus
Landauer’s Principle
Landauer 1961
Bennett 1987+…
Szilard’s principle is false.
Erasure
of one bit of information by
the demon creates k ln 2 of
thermodynamic entropy.
Real entropy cost only taken when the
naturalized demon erases the memory
of the position of the molecule.
20
Landauer’s Principle
“Landauer’s principle, often regarded as
the basic principle of the thermodynamics
of information processing, holds that any
logically irreversible manipulation of
information, such as the erasure of a bit or
the merging of two computation paths,
must be accompanied by a corresponding
entropy increase in non-informationbearing degrees of freedom of the
information-processing apparatus or its
environment….”
“…Conversely, it is generally accepted
that any logically reversible
transformation of information can in
principle be accomplished by an
appropriate physical mechanism
operating in a thermodynamically
reversible fashion.”
Logically
irreversible
operation
Must
pass entropy to
environment
k ln 2
per bit
erased
(e.g. erasure)
Process is thermodynamically
reversible if data is “random”;
not if “known” data.
Logically
reversible
operation
Can be
thermodynamically
reversible
Bennett, Charles H. (2003). “Notes on Landauer’s
Principle, Reversible Computation, and Maxwell’s
Demon,” Studies in History and Philosophy of Modern
Physics, 34, pp. 501-10.
21
The Standard Erasure Procedure
Model of binary memory.
One molecule gas in a divided chamber.
Heat kT ln 2
Entropy k ln 2
passes to environment.
22
No-Go
Result
23
No-Go Result
NO molecular-scale process
that completes is
thermodynamically reversible.
Thermodynamic entropy must be created to
complete each step.
Pfin 
Stot  k ln   k ln O fin
Pinit 
S
S

24
No-Go
Result
Illustrated
25
Fluctuations disrupt
Reversible
Expansion and
Compression
26
The Intended Process
Very slow expansion converts
heat to work in the raising of the
mass.
Mass M of piston continually
adjusted so its weight remains in
near perfect balance with the
mean gas pressure P= kT/V.
Equilibrium height is
heq = kT/Mg
Heat kT ln 2 = 0.69kT
passed in tiny increments
from surroundings to gas.
27
The massive piston…
….is very light since it must be
supported by collisions with a single
molecule. It has mean thermal energy
kT/2 and will fluctuate in position.
Probability density for the piston at
height h
p(h) = (Mg/kT) exp ( -Mgh/kT)
Mean
height = kT/Mg = heq
Standard
deviation = kT/Mg = heq
28
What Happens.
Fluctuations
obliterate the
very slow
expansion
intended
Heat kT ln 2 = 0.69kT
passed in tiny increments
from surrounding to gas.
A better analysis (elsewhere) does
not need external adjustment of
weight during expansion. It
replaces the gravitational field
withpiston
= 2kT ln (height)
energy
Mean energy of gas 3kT/2
Standard deviation (3/2)1/2kT = 1.225kT
29
Fluctuations disrupt
Measurement and
Detection
30
Measurement is compression of detector phase space
First step: the detector is
coupled with the target
system.
The process intended:
The process is isothermal,
thermodynamically reversible:
• It proceeds very slowly.
• The driver is in equilibrium
with the detector.
The coupling is an
isothermal, reversible
compression of the
detector phase space.
31
Fluctuations Obliterate Reversible Detection
What we expected:
What happens:
32
Bennett’s Machine for Dissipationless Measurement…
FAILS
Measurement apparatus, designed by the author to fit
the Szilard engine, determines which half of the
cylinder the molecule is trapped in without doing
appreciable work. A slightly modified Szilard engine
sits near the top of the apparatus (1) within a boatshaped frame; a second pair of pistons has replaced
part of the cylinder wall. Below the frame is a key,
whose position on a locking pin indicates the state of
the machine's memory. At the start of the
measurement the memory is in a neutral state, and the
partition has been lowered so that the molecule is
trapped in one side of the apparatus. To begin the
measurement (2) the key is moved up so that it
disengages from the locking pin and engages a "keel"
at the bottom of the frame. Then the frame is pressed
down (3). The piston in the half of the cylinder
containing no molecule is able to desend completely,
but the piston in the other half cannot, because of the
pressure of the molecule. As a result the frame tilts
and the keel pushes the key to one side. The key, in its
new position. is moved down to engage the locking pin
(4), and the frame is allowed to move back up (5).
undoing any work that was done in compressing the
molecule when the frame was pressed down. The
key's position indicates which half of the cylinder the
molecule is in, but the work required for the operation
can be made negligible To reverse the operation one
would do the steps in reverse order.
…is fatally disrupted by fluctuations that leave
the keel rocking wildly.
Charles H. Bennett, “Demons, Engines and the
Second Law,” Scientific American 257(5):108-116
(November, 1987).
33
No-Go
Result
Preparatory notions
34
Thermodynamically Reversible Processes
For…
Two systems interacting isothermally
in thermal contact with constant
temperature surroundings at T:
Thermodynamically
reversible
process
Set of irreversible processes
that approach a perfect balance
of all thermodynamic forces in
the (unrealized) limit.
T
1
2
env
internal
energy
change
heat
transferred
generalized
generalized
force
displacement
dU = dq –X dx
X = -∂F/∂l
Condition approached
arbitrarily closely in the limit:
Total entropy
of universe is
constant.
Total generalized
forces vanish.
X1+X2=0
for process parameter l
Total free energy
F=U-TS is constant.
F1+F2=constant
35
Self-contained thermodynamically reversible processes
No interventions from
non-thermal or
far-from-equilibrium systems.
External hand
removes shot one
at a time to allow
piston to rise
slowly.
Slow compression by slowly
moving, very massive body.
Mass is far from thermal
equilibrium of a one-dimensional
Maxwell velocity distribution.
36

Computing Fluctuations
probability system at
point with energy E
probability P that
system is in nonequilibrium state with
 V
phase volume


Canonically distributed
system in heat bath at T.
 E 
exp  
 kT 
Z(V)
 E(x) 
Z(V )  V exp 
dx
 kT 
give equilibrium, macroscopic
description of non-equilibrium state
F = -kT ln Z(V)
F = -kT ln P + constant
P  exp(-F/kT)
37
No-Go
Result
It, at last.
38
Combine 1. and 2.
any
isothermal,
reversible
process
final
middle
initial
stages
l
1. Process is thermodynamically reversible
Finit = Fmid = Ffin
2. Fluctuations carry the
system from one stage to another
Pinit exp(-Finit/kT)
Pmid exp(-Fmid/kT)
Pfin exp(-Ffin/kT)
Pinit = Pmiddle = Pfin
No-Go
result
39
Fluctuation Disrupt All Reversible, Isothermal
Processes at Molecular Scales
Intended
process
Actual
process
l=l1
l
l=l2
l=l1
l
l=l2
40
Beating
Fluctuations
41
What it takes to overcome fluctuations
Downward
gradient in
free energy
final
initial
release
from
here
..but system can also be found in
undesired intermediate states.
Process moves from high
free energy state to low
free energy state.
Fsys
recapture in
most likely
state
Net creation of
thermodynamic entropy.
Stot = -Fsys/T
42
What it takes to overcome fluctuations
Least
dissipative
case
free
energy
final
initial
release
from here
High free energy mountain makes it unlikely
that system is in intermediate stage.
 Ffin  Finit 
Pfin
Stot 
 exp 

 exp 
 k 
Pinit
kT 

Pinit = probability that fluctuation throws
the system back to the initial state.
recapture in most
likely state
Pfin 
Stot  k ln   k ln O fin
Pinit 
odds of
final state
43
Doing the sums…
Molecular Scale
Odds of completion
Ofin = 20
Pfin = 0.95
Stot = k ln 20 = 3k
compare
Landauer’s principle
k ln2 = 0.69 k
Macroscopic Scale
Odds of completion
Ofin = 7.2x1010
Stot = k ln (7.2x1010) = 25k
25kT is the mean
thermal energy of ten
nitrogen molecules.
44
Bead on a
Wire
45
Each position is an
equilibrium position
Macroscopically…
Effect of
thermal
fluctuation
s
Slow motion of bead over wire is
a thermodynamically reversible
process. (Tilt wire minutely.)
Molecular scale…
For 5g bead and T=25C
For 100 amu mass (n-heptane molecule)
and T=25C
vrms = 9.071 x 10-10
m/s
vrms = 157 m/s
46
Overcome fluctuations by tilting wire
Macroscopically…
For Pfin = 0.999
stages
T=25C
length
1/10th
For 5g bead
q = 5.8x10-18 radians
Depress by ~10-7 Bohr
radius H atom per meter.
Molecular scale…
For 100 amu mass (n-heptane molecule),
turning the wire vertically has
negligible effect!
n-heptane is volatile!
47
Least dissipative case
48
More
complicated
cases
49
Electric field
moves a charge
through a channel.
Two state dipole
measures sign of
target charge.
Computed in “All Shook Up…”
50
Conclusion
51
Thermal Fluctuations
Cannot be Idealized Away
No-Go
result
Thermal fluctuations (noise) prevent completion
of any individual, molecular scale step.
Thermodynamic entropy must be created to
complete each step.
Pfin 
Stot  k ln   k ln O fin
Pinit 
Minimum entropy creation not set by the
logical specification of the computation, but by the
number of steps chained.

The
End
53
Appendices
54
A Measurement Scheme Using Ferromagnets
Charles H. Bennett, “The Thermodynamics of
Computation—A Review,” In. J. Theor. Phys. 21, (1982),
pp. 905-40,
55
A Measurement Scheme Using Ferromagnets
Charles H. Bennett, “The Thermodynamics of
Computation—A Review,” In. J. Theor. Phys. 21, (1982),
pp. 905-40,
56
Thermodynamically reversible processes are NOT…
…merely very
slow processes.
…merely processes
that can go easily in
either way.
capacitor
discharges very
slowly through
resistor
balloon
deflates slowly
through a
pinhole
one molecule
gas released
57
Computing Fluctuations
probability P that
system is in nonequilibrium state
with phase volume V

Isolated,
microcanonically
distributed system
phase volume
V
give equilibrium,
macroscopic description
of non-equilibrium state
S = k ln V
S = k ln P + constant
P exp(S/k)
58