No-Go Result
for the Thermodynamics
of Computation
John D. Norton
Department of History and Philosophy of Science
Center for Philosophy of Science
University of Pittsburgh
1
Warm Up
Computers
Generate Lots of
Heat
2
Cray 2 computer (1985)
From Cray sales brochure.
3
Cooling chips
fan
heat pipe
vanes
4
What is the
Minimum Heat
Generation?
5
To minimize heat generation…
Use the smallest
systems possible
=
molecular
scale devices
6
Landauer’s Principle
0
or
1
Erasure
RESTORE TO ONE
1
Logically irreversible
operations like erasure pass
heat/entropy to the environment.
Erase
one bit
Pass
kT ln 2 heat
k ln 2 entropy
Logically reversible
operations can be carried out
non-dissipatively.
Rolf Landauer (1961) "Irreversibility and heat generation in the
computing process," IBM Journal of Research and
Development, 5, pp. 183–191. (p.265)
7
The Standard Erasure Procedure
Model of binary memory.
One molecule gas in a divided chamber.
Plausibility of Landauer’s Principle
1
Credible erasure
procedures
generate kT ln 2
of heat.
2
Erasure compresses the
space by a factor of 2.
Corresponds to entropy
change of k ln 2.
Heat kT ln 2
Entropy k ln 2
passes to environment.
3
Information
theoretic entropy of
ln 2 is associated
with distribution
P(L) = P(H) = 1/2
Sketchy… but someone has worked out the details…..??
8
Bennett’s 2003 statement of Landauer’s Principle
Logically
irreversible
Must
operation
pass entropy to
environment
Logically
Can be
reversible
operation
thermodynamically
reversible
“Landauer’s principle, often regarded as the basic
principle of the thermodynamics of information
processing, holds that any logically irreversible
manipulation of information, such as the erasure of
a bit or the merging of two computation paths,
must be accompanied by a corresponding entropy
increase in non-information-bearing degrees of
freedom of the information-processing apparatus
or its environment….”
“…Conversely, it is generally accepted that any
logically reversible transformation of information
can in principle be accomplished by an
appropriate physical mechanism operating in a
thermodynamically reversible fashion.”
Bennett, Charles H. (2003). “Notes on Landauer’s Principle,
Reversible Computation, and Maxwell’s Demon,” Studies in History
and Philosophy of Modern Physics, 34, pp. 501-10.
9
Bennett’s 2003 statement of Landauer’s Principle
Why not…
Logically
irreversible
operation
Must
pass entropy to
environment
Thermodynamically
irreversible process
…create entropy in a
thermodynamically
irrreversible process?
Logically
reversible
operation
Can be
thermodynamically
reversible
Thermodynamically
reversible process
10
Erasing Random versus Nonrandom/Known Data
“When truly random data (e.g., a bit
equally likely to be 0 or 1) is erased, the
entropy increase of the surroundings is
compensated by an entropy decrease of the
data, so the operation as a whole is
thermodynamically reversible. … [I]n
computations, logically irreversible
operations are usually applied to
nonrandom data deterministically
generated by the computation. When
erasure is applied to such data, the entropy
increase of the environment is not
compensated by an entropy decrease of the
data, and the operation is
thermodynamically irreversible.”
Bennett, Charles H. (1988). “Notes on the History of
Reversible Computation,” IBM Journal of Research
and Development, 32 (No. 1), pp. 16-23
Random
data
erase
Known
data
NO
entropy
created
Entropy
created
Erasure procedure on known data…
…DOES use
what is known.
Erasure is just
rearrangement. No
entropy created.
…does NOT use
what is known.
No difference in
erasing known
and random data.
11
To Come
12
This Presentation
No-Go Result
Demonstrations of
Landauer’s Principle fail.
1. Thermalization
2. Compression of phase space
3. Information entropy
(4. New indirect proof)
The standard inventory of
processes in the thermodynamics
of computation neglects
fluctuations.
All process must create entropy
to overcome them and the
quantities created swamp those
tracked by Landauer’s principle.
13
Failed proofs of
Landauer’s Principle
14
15
1. Thermalization
Initial data
L or R
Reversible
isothermal
compression passes
heat kT ln 2 to heat
bath.
Irreversible
expansion
“thermalization”
!!!
Entropy created in this illadvised, dissipative step.
Data reset to L
Entropy k ln 2
created in heat
bath
!!!
Proof shows only that an
Mustn’t we thermalize so the
inefficiently designed erasure
procedure creates entropy.
No demonstration that all must.
procedure works with arbitrary data?
No demonstration that thermalization is
the only way to make procedure robust.
16
Dissipationless Erasure
or
First method.
Second method.
1. Dissipationlessly detect
memory state.
1. Dissipationlessly detect memory state.
2. If R, shift to L.
2. If R, remove and reinsert partition and go to 1.
Else, halt.
17
18
2. Phase Volume Compression
aka “many to one argument”
thermodynamic
entropy
= k ln (accessible phase volume)
…if entropy is to
connect to heat
via Clausius’
dS = dqrev/T
“random” data
occupies twice the
phase volume of
reset data
Erasure halves
phase volume.
Erasure reduces
entropy of memory
by k ln 2.
Entropy k ln 2 must be
created in surroundings to
conserve phase volume.
19
2. Phase Volume Compression
FAILS
aka “many to one argument”
“random” data
DOES NOT occupy
twice the phase
volume of
reset data
It occupies the
same phase volume.
Confusion
with
thermalized
data
20
A Ruinous Sense of “Reversible”
Hence confusion
over random and
known data.
Random data
and
insertion of
the partition
removal of
the partition
thermalized data
have the same entropy
because they are connected
by a reversible, adiabatic
process???
DS = 0
random data
thermalized data
No. Under this sense
of reversible,
entropy ceases to be
a state function.
DS = k ln 2
21
22
3. Information-theoretic Entropy “p ln p”
Information
entropy
S = - k Si
inf
Pi ln Pi
“random” data
PL = PR = 1/2
Sinf = k ln 2
reset data
PL = 1; PR = 0
Sinf = 0
Hence erasure reduces the entropy
of the memory by k ln 2, which
must appear in surroundings.
But…
in this
case,
Information
entropy
does
NOT
equal
Thermodynamic
entropy
Thermodynamic entropy is
attached to a probability only in
special cases. Not this one.
23
What it takes…
Information
entropy
“p ln p”
DOES
equal
Thermodynamic
entropy
Clausius dS = dQrev/T
IF…
A system is
distributed
canonically over
its phase space
AND
p(x) = exp( -E(x)/kT) / Z
Z normalizes
All regions of phase space of non-zero E(x)
are accessible to the system over time.
For details of the proof
and the importance of the
accessibility condition,
see Norton, “Eaters of
the Lotus,” 2005.
Accessibility condition FAILS for
“random data” since only half of
phase space is accessible.
24
25
4. Indirect Proof: General Strategy
Process
known to
reduce entropy
coupled
to
Arbitrary
erasure process
Ladyman et al., “The
connection between logical
and thermodynamic
irreversibility,” 2007.
Entropy
reduces.
Assume
second law of
thermodynamics
holds on average.
Entropy must
increase on
average.
26
The Debate is Ongoing
27
The Debate is Ongoing
My concerns:
The inventory of processes assumed as possible is…
…inconsistent with
the statistical form of
the second law.
…survives only with
artificial restrictions on
what is possible.
(“controlled operations”)
…selectively ignores
fluctuations present in all
molecular scale processes.
No-go result
Ladyman, James; Presnell, Stuart; Short, Anthony J. and Groisman, Berry (2007). “The connection
between logical and thermodynamic irreversibility,” Studies in the History and Philosophy of Modern
Physics, 38, pp. 58–79.
Ladyman, James; Presnell, Stuart and Short, Anthony J. (2008). ‘The Use of the Information-Theoretic
Entropy in Thermodynamics’, Studies in History and Philosophy of Modern Physics, 39, pp. 315-324.
Ladyman, James and Robertson, Katie (forthcoming). “Landauer Defended: Reply to Norton, Studies
in History and Philosophy of Modern Physics.
Norton, John D. (2011). “Waiting for Landauer.” Studies in History and Philosophy of Modern
Physics, 42, pp. 184–198.
Norton, John D. (2013). “Author's Reply to Landauer Defended” Studies in History and
Philosophy of Modern Physics. Available online, May 24, 2013
28
“…the same bit cannot be both the control and the
target of a controlled operation…”
The Most Beautiful Machine 2003
Trunk, prosthesis, compressor, pneumatic cylinder
13,4 x 35,4 x 35,2 in.
“…the observers are supposed to push the ON button. After a while the lid of
the trunk opens, a hand comes out and turns off the machine. The trunk
closes - that's it!”
http://www.kugelbahn.ch/sesam_e.htm
Every negative feedback
control device acts on its
own control bit.
(Thermostat, regulator.)
29
Failed proofs of
Converse
Landauer’s Principle
30
Brownian Computation is the Constructive Proof
“…Conversely, it is generally accepted that any
logically reversible transformation of information
can in principle be accomplished by an
appropriate physical mechanism operating in a
thermodynamically reversible fashion.”
(Bennett, 2003)
Closer analysis…
Brownian computation is
thermodynamically irreversible,
the thermodynamic analog of the
uncontrolled expansion of a one
molecule gas.
Norton, John D. (Manuscript on
website). “Brownian Computation is
Thermodynamically Irreversible.”
31
No-Go
Result
Illustrated
32
Fluctuations disrupt
Reversible
Expansion and
Compression
33
The Intended Process
Infinitely slow expansion
converts heat to work in the
raising of the mass.
Mass M of piston continually
adjusted so its weight remains in
perfect balance with the mean gas
pressure P= kT/V.
Equilibrium height is
heq = kT/Mg
Heat kT ln 2 = 0.69kT
passed in tiny increments
from surrounding to gas.
34
The massive piston…
….is very light since it must be
supported by collisions with a single
molecule. It has mean thermal energy
kT/2 and will fluctuate in position.
Probability density for the piston at
height h
p(h) = (Mg/kT) exp ( -Mgh/kT)
Mean
height = kT/Mg = heq
Standard
deviation = kT/Mg = heq
35
What Happens.
Fluctuations
obliterate the
infinitely slow
expansion
intended
Heat kT ln 2 = 0.69kT
passed in tiny increments
from surrounding to gas.
A better analysis does not need
external adjustment of weight
during expansion. It replaces the
gravitational field with
piston
= 2kT ln (height)
energy
Mean energy of gas 3kT/2
Standard deviation (3/2)1/2kT = 1.225kT
36
Fluctuations disrupt
Measurement and
Detection
37
Bennett’s Machine for Dissipationless Measurement…
FAILS
Measurement apparatus, designed by the author to fit
the Szilard engine, determines which half of the
cylinder the molecule is trapped in without doing
appreciable work. A slightly modified Szilard engine
sits near the top of the apparatus (1) within a boatshaped frame; a second pair of pistons has replaced
part of the cylinder wall. Below the frame is a key,
whose position on a locking pin indicates the state of
the machine's memory. At the start of the
measurement the memory is in a neutral state, and the
partition has been lowered so that the molecule is
trapped in one side of the apparatus. To begin the
measurement (2) the key is moved up so that it
disengages from the locking pin and engages a "keel"
at the bottom of the frame. Then the frame is pressed
down (3). The piston in the half of the cylinder
containing no molecule is able to desend completely,
but the piston in the other half cannot, because of the
pressure of the molecule. As a result the frame tilts
and the keel pushes the key to one side. The key, in its
new position. is moved down to engage the locking pin
(4), and the frame is allowed to move back up (5).
undoing any work that was done in compressing the
molecule when the frame was pressed down. The
key's position indicates which half of the cylinder the
molecule is in, but the work required for the operation
can be made negligible To reverse the operation one
would do the steps in reverse order.
…is fatally disrupted by fluctuations that leave
the keel rocking wildly.
Charles H. Bennett, “Demons, Engines and the
Second Law,” Scientific American 257(5):108-116
(November, 1987).
38
A Measurement Scheme Using Ferromagnets
Charles H. Bennett, “The Thermodynamics of
Computation—A Review,” In. J. Theor. Phys. 21, (1982),
pp. 905-40,
39
A Measurement Scheme Using Ferromagnets
Charles H. Bennett, “The Thermodynamics of
Computation—A Review,” In. J. Theor. Phys. 21, (1982),
pp. 905-40,
40
A General Model of Detection
First step: the detector is
coupled with the target system.
The process intended:
The process is isothermal,
thermodynamically reversible:
• It proceeds infinitely slowly.
• The driver is in equilibrium
with the detector.
The coupling is an
isothermal, reversible
compression of the
detector phase space.
41
Fluctuations Obliterate Reversible Detection
What we expected:
What happens:
42
No-Go
Result
43
Preparatory notions
Isothermal, thermodynamically reversible process
Self-contained isothermal, thermodynamically
reversible process.
Computing fluctuations: How to do it.
44
Thermodynamically Reversible Processes
For…
Two systems interacting isothermally
in thermal contact with constant
temperature surrounding at T
Condition for All thermodynamic forces
thermodynamic are in perfect balance (or
reversibility minutely removed from it).
Process is a sequence
of equilibrium states.
T
1
2
env
internal
energy
change
heat
transferred
generalized
generalized
force
displacement
dU = dq –X dx
X = -∂F/∂l
for process parameter l
Total entropy
of universe is
constant.
Total generalized
forces vanish.
X1+X2=0
Total free energy
F=U-TS is constant.
F1+F2=constant
45
Thermodynamically reversible processes are NOT…
…merely very
slow processes.
…merely processes
that can go easily in
either way.
capacitor
discharges very
slowly through
resistor
balloon
deflates slowly
through a
pinhole
one molecule
gas released
46
Self-contained thermodynamically reversible processes
No interventions from
non-thermal or farfrom-equilibrium
systems.
External hand
removes shot one
at a time to allow
piston to rise
slowly.
Slow compression by slowly
moving, very massive body.
Mass is far from thermal
equilibrium of a one-dimensional
Maxwell velocity distribution.
47
Computing Fluctuations
probability P that
system is in nonequilibrium state
with phase volume V
Isolated,
microcanonically
distributed system
phase volume
V
give equilibrium,
macroscopic description
of non-equilibrium state
S = k ln V
S = k ln P + constant
P exp(S/k)
48
Computing Fluctuations
probability system at
point with energy E
probability P that
system is in nonequilibrium state with
 V
phase volume
Canonically distributed
system in heat bath at T.
 E 
exp  
 kT 
Z(V)
 E(x) 
Z(V )  V exp 
dx
 kT 
give equilibrium, macroscopic
description of non-equilibrium state
F = -kT ln Z(V)
F = -kT ln P + constant
P
exp(-F/kT)
49
No-Go
Result
It, at last.
50
Combine 1. and 2.
any
isothermal,
reversible
process
final
middle
initial
stages
l
1. Process is thermodynamically reversible
Finit = Fmid = Ffin
2. Fluctuations carry the
system from one stage to another
Pinit exp(-Finit/kT)
Pmid exp(-Fmid/kT)
Pfin exp(-Ffin/kT)
Pinit = Pmiddle = Pfin
No-Go
result
51
Fluctuation Disrupt All Reversible, Isothermal
Processes at Molecular Scales
Intended
process
Actual
process
l=l1
l
l=l2
l=l1
l
l=l2
52
Beating
Fluctuations
53
What it takes to overcome fluctuations
Downward
gradient in
free energy
final
initial
release
from
here
..but system can also be found in
undesired intermediate states.
Process moves from high
free energy state to low
free energy state.
DFsys
recapture in
most likely
state
Net creation of
thermodynamic entropy.
DStot = -DFsys/T
54
What it takes to overcome fluctuations
Least
dissipative
case
free
energy
final
initial
release
from here
High free energy mountain makes it unlikely
that system is in intermediate stage.
 Ffin  Finit 
Pfin
 DStot 
 exp 

  exp 
 k 
Pinit
kT 

Pinit = probability that fluctuation throws
the system back to the initial state.
recapture in most
likely state
 Pfin 
DStot  k ln 
  k ln O fin
 Pinit 
odds of
final state
55
Doing the sums…
Molecular Scale
Odds of completion
Ofin = 20
Pfin = 0.95
DStot = k ln 20 = 3k
compare
Landauer’s principle
k ln2 = 0.69 k
Macroscopic Scale
Odds of completion
Ofin = 7.2x1010
DStot = k ln (7.2x1010) = 25k
25kT is the mean
thermal energy of ten
nitrogen molecules.
56
Bead on a
Wire
57
Each position is an
equilibrium position
Macroscopically…
Effect of
thermal
fluctuation
s
Slow motion of bead over wire is
a thermodynamically reversible
process. (Tilt wire minutely.)
Molecular scale…
For 5g bead and T=25C
For 100 amu mass (n-heptane molecule)
and T=25C
vrms = 9.071 x 10-10
m/s
vrms = 157 m/s
58
Overcome fluctuations by tilting wire
Macroscopically…
For Pfin = 0.999
stages
T=25C
length
1/10th
For 5g bead
q = 5.8x10-18 radians
Depress by ~10-7 Bohr
radius H atom per meter.
Molecular scale…
For 100 amu mass (n-heptane molecule),
turning the wire vertically has
negligible effect!
n-heptane is volatile!
59
Least dissipative case
60
More
complicated
cases
61
Electric field
moves a charge
through a channel.
Two state dipole
measures sign of
target charge.
Computed in “All Shook Up…”
62
Conclusion
63
Abstract and Concrete
Information
ideas and concepts
=
Entropy
heat, work,
thermodynamics
And why not?
Mass = Energy
Particles = Waves
Geometry = Gravity
….
Time = Money
64
The
End
65