OCPL, ECE Dept., University of Arizona, Tucson, AZ
Imaging with Object-Specific Feature
Measurements
Premchandra M Shankar and Mark A Neifeld
University of Arizona
Department of Electrical and Computer Engineering
Optical Sciences Center
Tucson, AZ 85719
David Brady
Fitzpatrick Center / ECE Department
Duke University
Durham, NC 27708
OUTLINE
I.
II.
III.
IV.
Introduction to Multiplexed Imaging
Additive Noise Dominated MUX Imaging
Quantization Error Dominated MUX Imaging
Extensions and Conclusions
OCPL, ECE Dept., University of Arizona, Tucson, AZ
Feature Extraction and Imaging
 Many tasks require feature extraction
- Object recognition
- Image enhancement
- Data compression
 Features are usually computed from image measurement
Feature
Extraction
Conventional
imager
fS
Direct Image
Object
 Linear features may be measured directly
MUX
imager
Object
 Implementation Options
- Diffractive optical elements
- Micromirror arrays
- Volume holographic techniques
fM
OCPL, ECE Dept., University of Arizona, Tucson, AZ
MUX Imaging Framework
 Spatially discrete object vector g (Nx1)
 Feature projection matrix P (MxN)
 Linear features vector f = Pg (Mx1)
 Local features:
g1  f1
g2  f2
g3  f3
Input Object
Sequence of object blocks
 Dual-Rail Optical Measurement: f = f+ - f where f+ = P+ g and
P+
f - = P- g
f+
f
g
Object
P-
f-
OCPL, ECE Dept., University of Arizona, Tucson, AZ
Principal Component Features
 PCA provides minimum MSE for image reconstruction
 Examine PCA features for noise-free imaging
- 112x92 pixels in original object
- N = 4x4 object blocks for MUX imaging
- Retain M = 5 features per block for MSE = 2.5
DIRECT
5 PCA features
 Demonstrates reduction from 10,304 detectors to 3,220
 We have ignored measurement noise.
OCPL, ECE Dept., University of Arizona, Tucson, AZ
The Photon Count Constraint
Photons
I
Object
N - Detector Array
Direct Image
Photons
P
Object
M - MUX Detector Array
MUX features
 SNR is determined by number of object photons
 A fair comparison requires equal number of photons
 Feature measurement must share photon budget
 Maximum column sum of projection matrix P = 1
OCPL, ECE Dept., University of Arizona, Tucson, AZ
The Photon Count Constraint
 Consider additive white Gaussian noise
 Normalized feature projectors ∑ p2ij = 1 , 1<i<M
j
 Direct measurement
x=g+n
fs = p·x
MSESTD = <(fT-fs)2> where fT = p·g
MSESTD = σ2
 MUX measurement
y = (1/C) p·g + n
where C = max ∑pij = maximum column sum
j
i
fM = Cy
MSEMUX = <(fT-fM)2>
MSEMUX = C2 σ2
 C increases with number of features
 More features produce lower fidelity per feature
OCPL, ECE Dept., University of Arizona, Tucson, AZ
AWGN Dominated Imaging
 Examine feature measurement fidelity
- MSEMUX < MSESTD for M<7
 Tradeoff between reconstruction MSE & M
 Two sources of error in reconstruction
- error from throwing away features
- additive noise from measurement
OCPL, ECE Dept., University of Arizona, Tucson, AZ
AWGN Dominated Imaging
 Minimum MSE vs noise
 MSEMUX < MSEDIRECT for σ2 > 100
 Sample reconstruction
Direct 112x92
MUX MSE = 100
MSE = 100
M = 10, N = 8x8
Direct MSE = 900
MUX MSE = 160, M=6
OCPL, ECE Dept., University of Arizona, Tucson, AZ
Quantization Error Limited Case
 Setup
Original
image
True
Features
S-bit
Quantizer
K-bit
Quantizer
DIRECT
image
MUX
features
STD
Features
MUX
reconstructed
image
 Photon count constraint is no longer relevant
 Consider the case when S = 4 and K = 8
 Local features from N=8x8 object blocks
 Feature fidelity
- MUX features are better than STD.
OCPL, ECE Dept., University of Arizona, Tucson, AZ
Quantization Error Limited Case
 Reconstruction
 32 features x 8 bits per feature = 256 bits MUX
 8x8 pixels x 4 bits per pixel = 256 bits
DIRECT
 Demonstrates
- Reduction in MSE from 18 to 15
- Reduction of 10,304 detectors to 5,152
 Example reconstruction
Direct, MSE = 18
MUX, MSE = 15
OCPL, ECE Dept., University of Arizona, Tucson, AZ
Extensions
Global Image Features
 Trained on large number of 32x32 images.
 MSEMUX < MSESTD for 25 or less features
 Example reconstruction
Direct MSE=1000
MUX MSE=576
- Reduction from 1,024 detectors to 51
- Reduction in MSE from 1000 to 576
OCPL, ECE Dept., University of Arizona, Tucson, AZ
Extensions
ICA features
 Features optimized in information theoretical sense
 Assumption of Gaussian object is not required
 Minimize mutual information among features
 Local features from N=8x8 object blocks
- AWGN variance of 100
 ICA feature measurements superior in the same way
OCPL, ECE Dept., University of Arizona, Tucson, AZ
Extensions
Water filling
 Some features are more important than others
 Photons distributed according to feature importance
Optimization
 Gradient search to find features optimized both in mean
square error and photon count constraint.
 Minimal improvement in feature MSE.
OCPL, ECE Dept., University of Arizona, Tucson, AZ
Conclusions
 Reduction in number of detectors
 Reduces noise when measurements are dominated by
-
AWGN: for σ2 > σ2min
- Quantization Error
 Valid for both local and global features
 Alternate features (e.g., ICA) can be measured similarly
 Continuing work on optimization