Min- and Max-Relative Entropies
Nilanjana Datta
University of Cambridge,U.K.
„
Focus : on an important quantity which arises in
Quantum Mechanics & Quantum Information Theory
Quantum Entropy or von Neumann Entropy
„
and its parent quantity:
Quantum Relative Entropy
„
& on new quantum relative entropies :
Min- & Max-Relative Entropies
Quantum Entropy or von Neumann Entropy
„
In 1927 von Neumann introduced the notion of a mixed
state, represented by a density matrix:
k
ρ = ∑ pi ψ i ψ i ;
i =1
„
„
„
„
& defined its entropy:
ρ ≥ 0; Tr ρ = 1
ρ ↔ { pi , ψ i
}
S ( ρ ) := −Tr ( ρ log ρ )
Why? - To extend the classical theory of Statistical Mechanics
developed by Gibbs, Boltzmann et al to the quantum domain;
not for the development of Quantum Information Theory
In fact, this was well before Shannon laid the
foundations of Classical Information Theory (1948).
S ( ρ ) := −Tr ( ρ log ρ )
k
ρ = ∑ pi ψ i ψ i ;
i =1
S (ρ ) = 0
if and only if
ρ
ψ i ψ j ≠ δ ij
is a pure state:
ρ= Ψ Ψ
∴ S (ρ ) =
a measure of the “mixedness” of the state
ρ
Relation with Statistical Mechanics
„
The finite volume Gibbs state given by the density matrix:
e− β HΛ
=
− β HΛ
Tr e
ρ ΛGibbs
maximizes the von Neumann entropy, given the expected value
of the energy
i.e., the functional :
ρ
is maximized if and only if
S ( ρ Λ ) − β Tr ( ρ Λ H Λ )
ρΛ = ρ
Gibbs
Λ
log Z Λ = max ( S ( ρ Λ ) − β Tr ( ρ Λ H Λ ) )
ρΛ
„
In 1948 Shannon defined the entropy of a random variable
„
Let
„
Shannon entropy of
X ∼ p ( x);
x ∈ J;
X;
J = a finite alphabet
H ( X ) := −∑ p( x) log p ( x)
H ( X ) ≡ H ({ p ( x)} )
„
x∈J
Supposedly von Neumann asked Shannon to call this quantity
entropy, saying
“You should call it ‘entropy’ for two reasons; first, the function is
already in use in thermodynamics; second, and most importantly,
most people don’t know what entropy really is, & if you use
‘entropy’ in an argument, you will win every time.”
Relation between Shannon entropy & thermodynamic entropy
total # of
microstates
S := k log Ω
„
Suppose the
r th microstate occurs with prob. pr
……….
„
„
„
consider
replicas of the system
v
Then on average vr ≈ [vpr ] replicas are in the
Total # of
microstates
v!
Ω=
v1 !v2 !..vr !
Th.dyn.entropy of compound
system of v replicas
S = Sv / v = − k
∑p
r
r
v
state
v
v1 v2
1
2
v v ..vr
Sv = − kv
r
th
(by Stirling’s)
vr
∑p
r
log pr
r
log pr = k H ({ pr } )
Shannon
entropy
Relation between Shannon entropy & von Neumann entropy
S ( ρ ) := −Tr ( ρ log ρ ) ;
ρ ↔ { pi , ψ i } ;
„
BUT
ρ = ρ†,
ρ ≥ 0;
Tr ρ = 1
k
ρ = ∑ pi ψ i ψ i ;
i =1
ψ i ψ j ≠ δ ij
eigenvalues
n
Spectral decomposition
ρ = ∑ λi Π i ;
i =1
⇒ λi ≥ 0,
n
n
∑λ
i =1
i
= 1;
{λi }i =1 =
n
a probability
distribution
S ( ρ ) := −∑ λi log λi = H ({λi } )
i =1
„
Classical Information Theory [Shannon 1948]
= Theory of information-processing tasks
e.g. storage & transmission of information
„
Quantum Information Theory:
A study of how such tasks can be accomplished using
quantum-mechanical systems
„
not just a generalization of Classical Information Theory
to the quantum realm!
novel features!
The underlying
which have no
quantum mechanics
classical analogue!
In Quantum Information Theory, information is carried by
physical states of quantum-mechanical systems:
e.g. polarization states of photons, spin states of electrons etc.
Fundamental Units
„
„
„
„
Classical Information Theory: a bit ; takes values 0 and 1
Quantum Information Theory: a qubit
= state of a two-level quantum-mechanical system
Physical representation of a qubit
Any two-level system; e.g. spin states of an electron,
polarization states of a photon……
A multi-level system which has
2 states which can be effectively
decoupled from the rest;
E1
E0
Operational Significance of the Shannon Entropy
= optimal rate of data compression for a classical i.i.d.
(memoryless) information source
successive signals emitted by source : indept. of each other
Modelled by a sequence of i.i.d. random variables
U1 , U 2 ,..., U n
„
U i ∼ p (u )
signals emitted by the source =
u∈J
(u1 , u2 ,..., un )
Shannon entropy of the source:
H (U ) := −∑ p (u ) log p (u )
u∈J
Operational Significance of the Shannon Entropy
„
(Q) What is the optimal rate of data compression for
such a source?
[ min. # of bits needed to store the signals emitted
per use of the source] (for reliable data compression)
„
„
Optimal rate is evaluated in the asymptotic limit
n = number of uses of the source
One requires
„
n→∞
(n)
perror
→0 ; n→∞
(A) optimal rate of data compression =
H (U )
Shannon entropy of the source
Operational Significance of the von Neumann Entropy
= optimal rate of data compression for a
memoryless (i.i.d.) quantum information source
„
A quantum info source emits:
signals (pure states)
with probabilities
„
ψ 1 , ψ 2 ...., ψ k ∈
p1 , p2 ,..., pk
Then source characterized by:
{ρ , / }
/
Hilbert space
density matrix
k
ρ = ∑ pi ψ i ψ i
i =1
ψ i ψ j ≠ δ ij
To evaluate data compression limit :
Consider a sequence
{ρn , / n }
If the quantum info source is memoryless (i.i.d.)
/ n =/
⊗U
;
ρn = ρ
Optimal rate of data compression
„
ρ ∈/
⊗n
= S (ρ )
NOTE: Evaluated in the asymptotic limit
n = number of uses of the source
One requires
(n)
err
p
→0
n →∞
n→∞
„
„
e.g. A memoryless quantum info source emitting qubits
Characterized by
ρn = ρ
{ρn ,/ n } ;
Or simply by
{ρ ,/ }
„
Consider
„
Stored in
⊗n
;
/ n =/
n successive uses of the source ; n
mn qubits (data compression)
⊗U
;
qubits emitted
mn
rate of data compression =
n
Optimal rate of data compression
under the requirement that
mn
R∞ := lim
n →∞ n
(n)
perror
→0
n →∞
= S (ρ )
Quantum Relative Entropy
„
A fundamental quantity in Quantum Mechanics & Quantum
Information Theory is the Quantum Relative Entropy
of
ρ
w.r.t.
σ , ρ ≥ 0, Tr ρ = 0, σ ≥ 0:
S ( ρ || σ ) := Tr ρ log ρ − Tr ρ log σ
well-defined if
„
supp ρ ⊆ supp σ
It acts as a parent quantity for the von Neumann entropy:
S ( ρ ) := −Tr ρ log ρ = − S ( ρ || I )
(σ = I )
„
It also acts as a parent quantity for other entropies:
e.g. for a bipartite state
„
ρ AB
:
A
B
Conditional entropy
S ( A | B ) := S ( ρ AB ) − S ( ρ B ) = − S ( ρ AB || I A ⊗ ρ B )
„
Mutual information
ρ B = TrA ρ AB
I ( A : B ) := S ( ρ A ) + S ( ρ B ) − S ( ρ AB ) = S ( ρ AB || ρ A ⊗ ρ B )
Some Properties of
„
S ( ρ || σ ) ≥ 0
Klein’s inequality:
= 0 if & only if ρ = σ
“distance”
„
Joint convexity:
S (∑ p ρ ) || ∑ p σ ) ≤ ∑ pk S ( ρ k || σ k )
k
k
„
S ( ρ || σ )
k
k
k
k
k
Monotonicity under completely positive trace preserving
(CPTP) maps Λ :
S (Λ ( ρ ) || Λ (σ )) ≤ S ( ρ || σ )
„
What is a CPTP map?
For an isolated quantum system: time evolution is unitary
-- dynamics governed by the Schroedinger equation.
„
In Quantum Info. Theory one deals with open systems
„
„
unavoidable interactions between
system (A) & its environment (B)
Time evolution not unitary in general
A
B
Most general description of time evolution of an open system
given by a completely positive trace-preserving (CPTP) map
Λ: ρ →σ
density
operators
describes discrete state changes
resulting from any allowed physical
process :
a superoperator
Λ
Λ: ρ →σ
Properties satisfied by a superoperator
„
Linearity:
Λ ( ∑ pk ρ k ) = ∑ pk Λ ( ρ k )
k
k
σ = Λ( ρ ) ≥ 0
„
Positivity:
„
Trace-preserving: (TP)
„
Complete positivity:
(CP)
Trσ = Tr Λ ( ρ ) = Tr ρ = 1
/ A ⊗/
B
(Λ A ⊗ id B )(ρ AB ) ≥ 0
„
Kraus Representation
Theorem:
Λ ( ρ ) = ∑ Ak ρ Ak† ;
k
B
A
ρ AB
∑ Ak Ak = I
†
k
Monotonicity of Quantum Relative Entropy under CPTP map Λ :
v.powerful!
……….(1)
Many properties of other entropic quantities can be proved
using (1)
„
„
„
S (Λ ( ρ ) || Λ (σ )) ≤ S ( ρ || σ )
e.g. Strong subadditivity of the von Neumann entropy
Conjecture by Lanford, Robinson – proved by Lieb & Ruskai ‘73
S ( ρ ABC ) + S ( ρ B ) ≤ S ( ρ AB ) + S ( ρ BC )
B
A
C
Outline of the rest
„
Define 2 new relative entropy quantities
„
Discuss their properties and operational significance
„
Give a motivation for defining them
„
„
Define 2 entanglement monotones (I will explain what
that means )
Discuss their operational significance
Two new relative entropies
„
Definition 1 : The max- relative entropy of a density matrix
ρ & a positive operator
is
σ
S max ( ρ || σ ) := log ( min {λ : ρ ≤ λσ } )
(λσ − ρ ) ≥ 0
„
Definition 2: The min- relative entropy of a state
positive operator σ is
ρ
&a
S min ( ρ || σ ) := − log Tr (π ρ σ )
where
πρ
denotes the projector onto the support of
(supp ρ )
ρ
„
Remark: The min- relative entropy
Smin ( ρ || σ ) := − log Tr π ρ σ
(where π ρ
denotes the projector onto
supp ρ
)
is expressible in terms of: quantum relative Renyi entropy
of order
α
with
α ≠1
1
Sα ( ρ || σ ) :=
log Tr ρ α σ 1−α
α −1
as follows:
supp ρ ⊆ supp σ
S min ( ρ || σ ) = lim+ Sα ( ρ || σ )
α →0
„
Like
S (ρ
Min- and Max-Relative Entropies
|| σ ) we have
S* ( ρ || σ ) ≥ 0
for
S* (Λ ( ρ ) || Λ (σ )) ≤ S* ( ρ || σ )
„
„
Also
* = max, min
for any CPTP map
S* ( ρ || σ ) = S* (U ρU || U σ U )
†
†
Λ
for any unitary
operator
Most interestingly
S min ( ρ || σ ) ≤ S ( ρ || σ ) ≤ S max ( ρ || σ )
U
„
The min-relative entropy is jointly convex in its arguments:
n
For two mixtures of states
ρ = ∑ pi ρi
& σ =
i =1
as for
i =1
„
The max-relative entropy is quasiconvex:
S max ( ρ || σ ) ≤ max S max ( ρi || σ i )
1≤i ≤ n
∑ pσ
i =1
n
S min ( ρ || σ ) ≤ ∑ pi S min ( ρi || σ i )
n
i
i
S ( ρ || σ )
Min- and Max- entropies
H max ( ρ ) := − S min ( ρ || I )
H min ( ρ ) := − S max ( ρ || I )
= log rank(ρ )
= − log || ρ ||∞
analogous to:
„
S ( ρ ) = − S ( ρ || I )
For a bipartite state ρ AB
:
H min ( A | B) ρ := − S max ( ρ AB || I A ⊗ ρ B )
analogous to:
S ( A | B) = − S ( ρ AB || I A ⊗ ρ B )
H min ( A : B) ρ := S min ( ρ AB || ρ A ⊗ ρ B )
analogous to:
etc.
S ( A : B ) = S ( ρ AB || ρ A ⊗ ρ B )
etc.
Min- and Max- Relative Entropies satisfy the:
(1) Strong Subadditivity Property
B
A
C
S ( ρ ABC ) + S ( ρ B ) ≤ S ( ρ AB ) + S ( ρ BC )
just as
H min ( ρ ABC ) + H min ( ρ B ) ≤ H min ( ρ AB ) + H min ( ρ BC )
(2) Araki-Lieb inequality
just as
S ( ρ AB ) ≥ S ( ρ A ) − S ( ρ B )
H min ( ρ AB ) ≥ H min ( ρ A ) − H min ( ρ B )
A
B
(Q) What is the operational significance of the
min- and max- relative entropies?
In Quantum information theory, initially one evaluated:
„
optimal rates of info-processing tasks:
e.g. data compression,
transmission of information through a channel, etc.
under the following assumptions:
„
information sources & channels were memoryless
„
they were used an infinite number of times (asymptotic
limit)
n→∞
BUT: these assumptions are unrealistic!
In practice :
„
Each use of a source or channel need not be memoryless ;
in fact, correlations/memory effects unavoidable
(e.g. quantum info. source : a highly correlated electron system)
„
also sources and channels are used a finite number of times
„
„
Hence it is important to evaluate optimal rates for
finite number of uses (or even a single use)
of an arbitrary source or channel
Corresponding optimal rates:
optimal one-shot rates
Min- & Max relative entropies: S min ( ρ || σ ), S max ( ρ || σ )
act as parent quantities for one-shot rates of protocols
just as
Quantum relative entropy:
S ( ρ || σ )
act as a parent quantity for asymptotic rates of protocols
e.g. Quantum Data Compression
asymptotic rate:
one-shot rate:
S (ρ )
= − S ( ρ || I )
H max ( ρ ) = − S min ( ρ || I )
[Koenig & Renner]
Further Examples
S min ( ρ || σ ) :
„
parent quantity for the following:
[Wang & Renner] : one-shot classical capacity of a
quantum channel
„
[ND & Buscemi] : one-shot quantum capacity of a
quantum channel
„
Relative entropies
Entanglement
“spooky action at a distance”
ρ AB
A
time
B
Y
X
entangled!
A
„
measures here
non-local quantum
correlations
B
„
affected there
Bipartite Quantum States :
separable
Separable IF
/ A ⊗/ B ;
entangled
Ψ AB = ϑA ⊗ ξ B
else entangled
ρ AB = ∑ piω ⊗ σ
A
i
B
i
i
else entangled
A
B
Entanglement : strictly non-local quantum property
A fundamental feature of entanglement
it cannot be created or increased by :
(i) local operations (LO) & (ii) classical communications(CC)
LOCC
shared entangled state
X
Y
ρ AB
Local (LO)
operations
classical communication (CC)
Λ : LOCC
σ AB
ρ AB ⎯⎯⎯
→ σ AB ⇒ E(σ AB ) ≤ E(ρ AB )
LOCC
entanglement
0 ≡
1 ≡
Maximally entangled states
e.g. Pure state of 2 qubits A,B :
Ψ AB
where
„
„
Ψ = Ψ AB Ψ AB
1
=
(0 ⊗ 0 +1 ⊗1
2
)
Bell state
state of the 2 qubits known exactly ; since AB in pure state
Reduced state of qubit A
IA 1
1
ρ A = TrB Ψ = = 0 0 + 1 1
2 2
2
Ψ
completely mixed state
similarly
IB
ρ B = TrA Ψ =
2
A
B
Bell State
Ψ
valuable resource in Quantum Info Theory
„
Ψ
IF
A
B
X
„
They can perform tasks e.g. quantum teleportation
which are important in the classical realm!
Y
QuantumTeleportation
X
A
B
Ψ
Y
Classical communication
unknown quantum
state
„
„
intriguing example of how quantum entanglement can
assist practical tasks!
usefulness of Bell states!
Entanglement monotones
„
A measure of how entangled a state
i.e., the amount of entanglement
:“minimum distance” of
ρ
ρ
is ;
E(ρ )
ρ from the set :
in the state
ρ
of separable states.
+
E(ρ )
:
E ( ρ ) = min S ( ρ || σ )
σ ∈:
„
Relative
entropy of
entanglement
Properties of an Entanglement Monotone
E(ρ ) = 0
if
ρ
E(ρ )
is separable
E ( Λ LOCC ( ρ )) ≤ E ( ρ )
E(ρ )
is not changed by a local change of basis etc.
E ( ρ ) = min S ( ρ || σ )
σ ∈:
satisfies these properties
a valid entanglement monotone
Entanglement Monotones
E ( ρ ) = min S ( ρ || σ )
σ ∈:
„
relative entropy of
entanglement
We can define two quantities:
Emax ( ρ ) := min S max ( ρ || σ )
Max-relative entropy of
entanglement
Emin ( ρ ) := min S min ( ρ || σ
Min-relative entropy of
) entanglement
σ ∈:
σ ∈:
these can be proved to be entanglement monotones!
Emax ( ρ )
and
Emin ( ρ )
have interesting operational significances in
entanglement manipulation
„
What is entanglement manipulation ?
= Transformation of entanglement from one form to
another by LOCC :
ρ AB
Λ : LOCC
X
Ψ
ρ AB ∈) (2 A ⊗ 2 B )
Partially entangled state
Y
⊗m
Ψ ∈/ A ⊗/ B ;
Bell state
dim (2 A ⊗ 2 B ) > dim (/ A ⊗ / B )
One-Shot Entanglement Distillation
ρ AB
Λ : LOCC
X
„
Ψ
Y
⊗m
What is the maximum number of Bell states that
you can extract from the state ρ AB using LOCC?
i.e., what is the maximum value of
m
?
“one-shot distillable entanglement
ρ AB ”
„
Result 4 :
“one-shot distillable entanglement of
ρ AB”
= Emin ( ρ AB )
ND &F.Brandao
Min-relative entropy of entanglement
One-shot Entanglement Dilution
„
Bell states : resource for creating a desired target state
⊗m
Ψ
Λ:
ρ AB
„
LOCC
ρ AB
: partially entangled bipartite target state
What is the minimum number of Bell states needed to
ρ
create the state
AB using LOCC?
i.e., what is the minimum value of
m
?
“one-shot entanglement cost of
ρ AB”
„
Result 5 :
“one-shot entanglement cost of
ρ AB ”
= Emax ( ρ AB )
ND &F.Brandao
Max-relative entropy of entanglement
Summary
„
Introduced 2 new relative entropies
(1) Min-relative entropy & (2) Max-relative entropy
S min ( ρ || σ )
„
≤ S ( ρ || σ ) ≤ Smax ( ρ || σ )
Parent quantities for optimal one-shot rates for
„
„
(i) data compression for a quantum info source
(ii) transmission of (a) classical info & (b) quantum info
through a quantum channel
Entanglement monotones
„
„
„
Min-relative entropy of entanglement
Emin ( ρ AB )
Max-relative entropy of entanglement
Emax ( ρ AB )
Operational interpretations:
Emin ( ρ AB ) :
Emax ( ρ AB ) :
One-shot distillable entanglement of
One-shot entanglement cost of
ρ AB
ρ AB
References:
„
„
„
„
ND: Min- and Max- Relative Entropies; IEEE Trans. Inf. Thy,
vol. 55, p.2807, 2009.
ND: Max-Relative Entropy alias Log Robustness; Intl.
J.Q.Info. Thy, vol, 7, p.475, 2009.
F. Brandao & ND: One-shot entanglement manipulation
under non-entangling maps (to appear in IEEE Trans. Inf.
Thy, 2009)
F. Buscemi & ND: Quantum capacity of channels with
arbitrarily correlated noise (to appear in IEEE Trans. Inf.
Thy, 2009)