Entropy, Information and
Compressive Sensing in the
Quantum Domain.
John Howell
Greg Howland
James Schneeloch
Daniel Lum
Sam Knarr
Clemente Cuevas(REU)
Matt Ware (REU)
Robert Boyd
Cliff Chan
Petros Zerom
Outline
• Introduction to compressive sensing
– Shannon entropy
– Nyquist sampling
– Lossy compression
– K-sparse sensing
– L1 norm reconstruction
– Advantages
• Applications
– Ghost imaging with entangled
photons
– Photon counting Lidar
– Depth Maps
– Object tracking
– High dimensional entanglement
characterization
Shannon Entropy
• Entropy
– Measure uncertainty of
random variable X with
distribution p(x).
H ( X )   p ( x) log 2 p ( x)
– Find number of symbols
and bits per symbol
(e.g., 0 or 1 binary)
– Compression removes
intersymbol correlations
•
•
Example (alphanumeric)
______
_______
•
____R_
_E_____
•
Q___R_
_E_T___
•
QSTART
MEET_NG
•
Shannon showed there are approximately 1
bit per symbol in English language
Information Theorist 
Marriage Therapist
“I see that your channel capacity is quite low. You
need to spend more time maximizing your mutual
information. Please increase the entropy of your
communication while minimizing the noise in your
classical channels.”
Shannon Entropy for Images:
Compression After Sensing
• Compression removes interpixel
correlations
• Decompose in decorrelated transform
basis
– k-sparse representation
– DCT, DFT, wavelets etc.
• Significant reduction in memory
requirements, reduced uncertainty
• Standard Sensing
Paradigm (e.g., jpeg)
– Sample (at least two
times above Nyquist)
– Transform to sparse
basis
– Preferentially attenuate
high frequency
components
– Round coefficients
– Inverse transform
Sensing Paradigms
• Typical Sensing: Compress after you sense
• Compressed sensing: Compress while you
sense
CS Literature of Interest
• Tutorials on CS
– R.G. Baraniuk, IEEE SIGNAL PROCESSING
MAGAZINE [118] JULY 2007
– E.J. Candes and M.J. Wakin, IEEE SIGNAL
PROCESSING MAGAZINE [21] MARCH
2008
• Single Pixel Camera
– Duarte et al, IEEE SIGNAL PROCESSING
MAGAZINE [83] MARCH 2008
Introduction to CS
Consider a 1 dimensional signal x of length N and a transform basis s.
x  s
Transform matrix of dimension NxN
We require a sensing matrix  which is not sparse when transformed
(incoherence or restricted isometry property).
y  x  s
Random, length N, sensing matrices satisfy this requirement.
Introduction to CS
M random measurements needed to recover signal
N
M  k log 2  N
k
Reconstruct image using l1 norm minimization


min  si 
 i 1 
N
y  s
We use Gradient Projection for Sparse Reconstruction algorithm
(noise tolerant BPDN)
Figueiredo et al, IEEE Selected Topics in Signal Processing, 2007
Compressive sensing
Reflected light
Scene
Photodiode
DMD
Compressive sensing
We take M << N measurements with different random DMD
patterns and then reconstruct x
Outside information is needed to solve our underdetermined linear
system to reconstruct our image!
Why CS?
• Sampling rate
– Above information rate
– NOT above Nyquist rate.
• Resource efficient
– Single pixel measurements
– Fewer measurements
– Automatically finds large k-sparse coefficients
Cool stuff recently done
• “Compressive Sensing Hyperspectral Image” T. Sun and K. Kelly
(COSI) (2009)
• “Sparsity-based single-shot sub-wavelength coherent diffractive
imagin” A. Szameit et al (M. Segev), Nature Materials 11, 455
(2012)
• “Compressive Depth Map Acquisition Using a Single PhotonCounting Detector” A. Colaco et al Proc. IEEE Conf. Computer
Vision and Pattern Recognition (2012)
• “Compressive Sensing for Background Subtraction” Volkan Cevher,
(Baraniuk)
• 3D COMPUTATIONAL IMAGING WITH SINGLE-PIXEL
DETECTORS (Padgett) Science 340, 844 (2013)
Some of our applications
•
•
•
•
•
Ghost imaging with entangled photons
Photon counting Lidar
Depth Maps
Object tracking
High dimensional entanglement
characterization
Entangled Photon Compressive
Imaging
Comparison to Raster Scanning
• Compressive Sensing
– 4500 measurements
– N=128x128 pixels
– 9 seconds/measurement
– SNR 8
– Acquisition time
• 4500 x 9s~ ½ day
• Raster Scanning
– For same SNR and resolution it would take almost
3 years to acquire image with same flux
LIDAR with JIGSAW
Pros
•32x32 APD detectors
•Time of Flight Measurements
•High Signal to Noise (no
amplification noise)
•Foliage Penetrating, Aerosol
•Low Light Level Detection
Cons
•Low Fill Factor
•Difficult to Scale
•Expensive
•Resource Heavy
•Visible Wavelengths
•Large Payload
MIT LL
JIGSAW
Experimental Setup
G. Howland, P.B. Dixon and J.C. Howell, Appl. Optics 50, 5917 (2011)
Results: 3D Imaging
Wall
U
R
Imaging Through Obscurants
Low Flux 3D Object Tracking
Light -Tight
Enclosure
Ellipt ical
Mirror
1440 Hz
DMD
Driver
Mechanical
Shut t er
DMD
From Target s
Im aging
Lens
NB
Filt er
DVI
Com put er
PMT
Module
USB
To Target s
TCSPC
Module
Pulse
Generat or
Pulsed
Laser Diode
Swinging Ball Trajectory
Frame by Frame
Depth Map of Natural Scene
2nd-Order Correlations
Slow Method
•
•
>7 bits mutual information in X and P
P. Ben Dixon et al PRL 108, 143603 (2012)
With CS
• Replace Raster with CS
• N log N scaling rather
than N3 to N4.
• 8 hours instead of a year
Efficient High-Dimensional Entanglement Imaging with a
Compressive-Sensing Double-Pixel Camera
Gregory A. Howland and John C. Howell
Phys. Rev. X 3, 011013 (2013).
32x32 Position Position
Correlations (3 raster)
Mutual Information in X and P
Violation of Continuous-Variable Einstein-Podolsky-Rosen Steering
with Discrete Measurements J. Schneeloch, P. Ben Dixon, G. A. Howland, C.
J. Broadbent, and J. C. Howell Phys. Rev. Lett. 110, 130407 (2013).
Background Subtraction Object Tracking
Compressive object tracking using entangled photons
Omar S. Magana-Loaiza, Gregory A. Howland, Mehul Malik, John C.
Howell, and Robert W. Boyd Appl. Phys. Lett. 102 231104 (2013).
t ions, dark and shot noise were int roduced by means of
poissonian dist ribut ions. T he amount of dark noise was
Ghost Object Tracking
#! "
!"
.90#
!#
! "#
$"#
%"#
./ 0#
"*+#
"*( #
"*&#
&"#
' "#
( "#
"*$#
! "# $"#%"# &"#' "# ( "#
"#
#! "
5 678"*"$"' #
"*$# .30#
.20#
5 678"*"$$, #
#! "
"*$# .10#
"#
"*! #
)"*$#
)"*! #
)"*&#
)"*%#
)"*( #
)"*' #
)"*+#
)"*, #
#! "
"*! #
"#
)"*! #
)"*$#
)"*%#
)"*&#
)"*' #
)"*( #
)"*, #
)"*+#
)"*- #
5 678"*"$! +#
"*%#
5 678"*"! - %#
#! "
"*%#
"*! #
.40#
)"*! #
)"*%#
)"*' #
)"*, #
5 678"*"$, &#
FIG. 2. Compressed ghost image of (a) t he background of
Novel Acquisition Paradigm
•
•
•
•
•
Quantum imaging
Entanglement mutual information
Low flux LIDAR
Precision measurements
Real-time video