Uncertainty Principles
Jascha Sohl-Dickstein
9/8/06
In quantum mechanics
• For observables A and B
2


1
 A2  2B  
Aˆ , Bˆ 
2i

 
where:
 is the variance of A

Aˆ is the operator associated with A,
such that
A   | Aˆ     Aˆ  dx
2
A

*

(ditto for B)

Position / Momentum
A=x
B=p
xˆ  x
pˆ 
1
2 2
 x p  
2i

2

 2

d 
x,
   




 i dx  2 
 x p 


d
i dx
2
Time/Frequency Uncertainty
(Gabor, 1946)
Write signal as real part of analytic signal:
s(t)  Re (t)
(t)  s(t)  i (t)
Create complementary Fourier signal:

(t) 
2 ift
(
f
)e
df

( f ) 

(t)e2 iftdt
Time/Frequency Uncertainty
Calculate moments:
 * t dt
n 
t
  * dt
n    * f df
f
  * df
____
n
____
n
Rewrite the frequency moments as a
function
of
psi
(!)
:

d

*
dt


*
f
df



1
f n    *df  2i    dt* dt
n
____
n
n
n
Time/Frequency Uncertainty
But this matches the QM formalism! We can
treat our moments as observables, and the
sandwiched expressions as the
corresponding operators:
__ˆ __
n  tn
t
n


1
d
n   
2 i  dt n
__ˆ __
f
n
Time/Frequency Uncertainty
• Plugging this in to the QM uncertainty
formula:
1
 t f 
2
• Which also implies:


 t  f  2
Discrete Unordered
Uncertainty Principle
(Donoho, Stark, 1989)
Let
be a sequence of length N and let
be its discrete Fourier transform.
Suppose that is non-zero at
points and
that
is non-zero at
points. Then:
QuickT i me™ and a
T IFF (Uncompressed) decompressor
are needed t o see thi s pi cture.
QuickTi me™ and a
T IFF (Uncom pressed) decom pressor
are needed to see t his pict ure.
Quick Time™and a
TIF F (Uncompress ed)dec ompres sor
are needed to s ee th is pic tu re.
Quick Time™and a
TIFF(Uncompress ed)dec ompres sor
are needed to s ee th is pic tu re.
Quco
i ck
Ti e
m ss
e ™
TI F F ( Un
m pr
edan
) ddeca om p r es so r
ar e n ee de d t o s ee t h is pi ct u re .
Q ui ck Ti m e ™ an d a
T IF F ( Un co m p re ss ed ) d ec om pr e ss or
a re ne ed ed t o s ee h
t i s pi c tu r e.
Quic kTime™ and a
TIFF (Unc ompres sed) dec ompres sor
are needed to see this pic ture.
(proof - Nw restricted to < Nt consecutive zeros
by non-singularity of DFT matrix)
Uniqueness of Sparse
Representation
(Donoho, Huo, 2001)
Mutual incoherence:
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Under inequality:
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
(fourier transform achieves equality)
Uniqueness of Sparse
Representation
(Elad, Bruckstein, 2002)
Uncertainty principles:
1
0
 1 2
 2 0   
M 
2
1 0  2 0 
M
where

S  11  22
If two bases are mutually incoherent, then no
signal can have a highly sparse
representation in both bases simultaneously.
(proof - uses Lagrangians to maximize lower bound)
Miscellany
2-d Uncertainty Principles
(Daugman, 1985)
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
• Derives orientation uncertainty as a
function of aspect ratio lambda and
bandwidth in octaves delta-mu:
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Information Theory Based
Uncertainty Principle
(DeBrunner, Havlicek, Przebinda,
Ozaydin, 2005)
e 
H (x) H ( f )  N ln  
2 
where x is the signal, f its fourier
 transform, and N the number of
samples
Signal Recovery
(baby compressed sensing)
• Uncertainty principle guarantees/proves
there is sufficient redundancy across
representations
• P. 9 In SMM