Multiscale Analysis for
Intensity and Density
Estimation
Rebecca Willett’s MS Defense
Thanks to Rob Nowak, Mike Orchard,
Don Johnson, and Rich Baraniuk
Eric Kolaczyk and Tycho Hoogland
Poisson and Multinomial
Processes
Why study Poisson
Processes?
Astrophysics
Network analysis
Medical Imaging
Multiresolution Analysis
Examining data at different resolutions
(e.g., seeing the forest, the trees, the
leaves, or the dew)
yields different
information about the
structure of the data.
Multiresolution analysis is effective
because it sees the forest (the overall
structure of the data)
without losing sight of
the trees (data
singularities)
Beyond Wavelets
Multiresolution analysis is a powerful
tool, but what about…
Non-Gaussian
problems?
Edges?
Image Edges?
Nongaussian
noise?
Inverse problems?
Piecewise polynomial- and plateletbased methods address these
issues.
Computational Harmonic
Analysis
I.
Define Class of Functions to Model
Signal
A. Piecewise Polynomials
B. Platelets
II. Derive basis or other representation
III. Threshold or prune small
coefficients
IV. Demonstrate near-optimality
Approximating Besov
Functions with Piecewise
Polynomials
Error Decay Rate : Od
 2r

  d-r


d
r

 
Discrete Error Decay : O 


 
 N N
N
 

3
2
Approximation with Platelets
Consider approximating this image:
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
50
100
150
200
250
E.g. Haar analysis
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
50
100
150
Terms = 2068, Params = 2068
200
250
Wedgelets
Original Image
Haar Wavelet Partition
Wedgelet Partition
E.g. Haar analysis with
wedgelets
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
50
100
150
Terms = 1164, Params = 1164
200
250
E.g. Platelets
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
50
100
Terms = 510, Params = 774
150
200
250
Error Decay
Platelet Approximation
Theory
b,2


Holder (Cb )
a,1


Holder (Ca )
Error decay rates:
• Fourier: O(m-1/2)
• Wavelets: O(m-1)
• Wedgelets: O(m-1)
• Platelets: O(mmin(a,b))
0,0
0,0
0,0
1,0
1,1
1,0
1,0
2,0
3,0
4,0
2,1
3,1
4,1
1,1
4,2
3,2
4,3
4,4
2,2
3,3
4,5
1,1
4,6
3,4
4,7
4,8
4,9
2,3
3,5
4,10
4,11
3,6
4,12
4,13
3,7
4,14
4,15
A Piecewise Constant Tree
A Piecewise Linear Tree
Maximum Penalized
Likelihood Estimation
Goal: Maximize the penalized likelihood
L  (μ)  log p( x | μ)   {# θ}
where
p( x | μ) is the likelihood
 is a smoothing parameter
{# θ} is the number of parameters in θ
So the MPLE is
ˆ  arg max L  (μ(θ ))
θ
θ
ˆ)
ˆ  μ(θ
μ
The Algorithm
Data

Const Estimate

Wedge Estimate

Platelet Estimate


Wedged Platelet
Estimate
Inherit from
finer scale
Algorithm in Action
Penalty Parameter
Penalty parameter balances between
fidelity to the data (likelihood) and
complexity (penalty).
1
0.8
Bias
Choose   c log N
2
0.6
Different  values
0.4
0.2
0
0
2
4
6
Variance
 = 0  Estimate is MLE: μˆ  x
   Estimate is a constant: μˆ  x
8
10
Penalization
Bowl
0
10
Ave. Counts/Pixel = 1
Ave. Counts/Pixel = 10
Ave. Counts/Pixel = 100
-1
Mean MSE
10
-2
10
-3
10
-4
10 -2
10
-1
0
10
10
b / log(# of photon counts)
1
10
Density Estimation - Blocks
Density Estimation - Heavisine
Density Estimation - Bumps
Density Estimation
Simulation
Medical Imaging Results
Inverse Problems
Goal: estimate m from observations
x ~ Poisson(Pm)
EM algorithm
(Nowak and Kolaczyk, ’00):
E  Step : Q(m , m )  Emi [L  (m ) | x]
i
M - Step : max Q(m , m )
i
m
Confocal Microscopy: An
Inverse Problem
Platelet Performance
2.6
MLE (min error level shown in dots)
Piecewise Constant
Platelet
2.4
2.2
Error
2
1.8
1.6
1.4
1.2
2
4
6
8
Iteration
10
12
14
Confocal Microscopy:
Real Data
Hellinger Loss
H pm , pm '  
2

pm ( x )  pm ' ( x )
• Upper bound for affinity
A(p, q)   p( x )q( x )
1/ 2
(like squared error)
• Relates expected error to Lp
approximation bounds

2
Bound on Hellinger Risk
(follows from Li & Barron ’99)
e
EH p ,p 
pen( m ' )
If
m '
2
m
mˆ
 1 , then risk of mˆ satisfies
 min KL pm ,pm'   pen(m' ) 
m '
KL distance
Approximation error
Estimation
error
Bounding the KL
• We can show:
1
n 2N
2
KL pμ , pμ' 
μ  μ' 
2
N
N c
• Recall approximation result:
  d-r


d
r  


O


 
 N N
N
 

3
2
μ - μ' 
2
• Choose optimal d
2
Near-optimal Risk

 1

O  
 N

2r
2r 1
2r



2
2
r

1
  1 H2 (p , p )  O  log (N)  
μˆ
μ
 N  
 N






• Maximum risk within logarithmic
factor of minimum risk
• Penalty structure effective:
  c log N
Conclusions
CHA with Piecewise
Polynomials or Platelets
•
•
•
•
Effectively describe Poisson or
multinomial data
Strong approximation capabilites
Fast MPLE algorithms for
estimation and reconstruction
Near-optimal characteristics
Major
Contributions
• Risk analysis for
piecewise
polynomials
• Platelet
representations
and
approximation
theory
Future Work
• Shift-invariant
methods
• Fast algorithms
for wedgelets and
platelets
• Risk Analysis for
platelets
Approximation Theory
Results
Consider the class of functions
f ( x, y )  f1 ( x, y )  I{ y  H ( x )}  f 2 ( x, y )  (1  I{ y  H ( x )} )
b ,2
a ,1




where f i  Holder (C b ), and H  Holder (Ca ).
If fˆ is the m - term, J - scale,  resolution platelet
approximat ion, so 2  m  2 , then
J
f  fˆ
L2
 Ka , b m
 min( a , b )
.
Why don’t we just find the
MLE?
- λi
e λ
p(x | λ )  
x i!
i
xi
i
xi

log( p(x | λ ))    1   0
λ
i
i
λˆ  x
MPLE Algorithm (1D)
Multiscale Likelihood
Factorization



Probabilistic analogue to orthonormal
wavelet decomposition
Parameters   wavelet coefficients
Allow MPLE framework, where
penalization based on complexity of
underlying partition
Poisson Processes
• Goal: Estimate spatially varying
function, (i,j), from observations
of Poisson random variables x(i,j)
with intensities (i,j)
• MLE of  would simply equal x. We
will use complexity regularization
to yield smoother estimate.
Accurate
Model
Complexity
Regularization
Parsimonious
Model
Penalty for each constant region
 results in fewer splits
Bigger penalty for each polynomial or
platelet region
 more degrees of freedom, so more
efficient to store constant if likely
Astronomical Imaging