Mathematics and Computation in Imaging
Science and Information Processing
July-December, 2003
• Organized by Institute of Mathematical Sciences and
Center for Wavelet. Approximation, and Information
Processing, National University of Singapore.
• Collaboration with the Wavelet Center for Ideal Data
Representation.
• Co-chairmen of the organizing committee:
• Amos Ron (UW-Madison),
• Zuowei Shen (NUS),
• Chi-Wang Shu (Brown University)
Conferences
• Wavelet Theory and Applications: New
Directions and Challenges, 14 - 18 July
2003
• Numerical Methods in Imaging Science and
Information Processing, 15 -19 December
2003
Confirmed Plenary Speakers for
Wavelet Conference
•
•
•
•
•
•
Albert Cohen
Wolfgang Dahmen
Ingrid Daubechies
Ronald DeVore
David Donoho
Rong-Qing Jia
•
•
•
•
•
Yannis Kevrekidis
Amos Ron
Peter Schröder
Gilbert Strang
Martin Vetterli
Workshops
• IMS-IDR-CWAIP Joint Workshop on Data
Representation, Part I on 9 – 11, II on 22 - 24 July 2003
• Functional and harmonic analyses of wavelets and frames,
28 July - 1 Aug 2003
• Information processing for medical images, 8 - 10
September 2003
• Time-frequency analysis and applications, 22- 26
September 2003
• Mathematics in image processing, 8 - 9 December 2003
• Industrial signal processing (TBA)
• Digital watermarking (TBA)
Tutorials
• A series of tutorial sessions covering various
topics in approximation and wavelet theory,
computational mathematics, and their applications
in image, signal and information processing.
• Each tutorial session consists of four one-hour
talks designed to suit a wide range of audience of
different interests.
• The tutorial sessions are part of the activities of
the conference or workshop associated with.
Membership Applications
• To stay in the program longer than two
weeks
• Please visit http://www.ims.nus.edu.sg
for more information
Wavelet Algorithms for High-Resolution
Image Reconstruction
Zuowei Shen
Department of Mathematics
National University of Singapore
http://www.math.nus.edu.sg/~matzuows
Joint work with (accepted by SISC)
T. Chan (UCLA), R.Chan (CUHK) and L.X. Shen (WVU)
Outline of the talk
Part I: Problem Setting
Part II: Wavelet Algorithms
What is an image?
image = matrix
pixel intensity
= matrix entry
Resolution = size of the matrix
I. High-Resolution Image Reconstruction:
Resolution = 64  64
Resolution = 256  256
Four low resolution images (64  64) of the same scene.
Each shifted by sub-pixel length.
Construct a high-resolution
image (256 256) from them.
Boo and Bose (IJIST, 97):
#4
#2
taking lens
#1
partially silvered
mirrors
relay
lenses
CCD sensor
array
Four 2  2 images merged into one 4  4 image:
a1
a2
b1
b2
a3
a4
b3
b4
a1 b1 a2 b2
c1 d1 c2 d2
a3 b3 a4 b4
c1
c2
d1
d2
c3 d3 c4 d4
d4
Observed highresolution image
c3
c4
d3
Four low resolution images
By permutation
Four 64 64 images merged into one by permutation:
Observed highresolution image by
permutation
Modeling
Consider:
Low-resolution
pixel
1
4
1
2
1
4
1
2
1
1
2
1
4
1
2
1
4
High-resolution
pixels
Observed image: HR image passing through a
low-pass filter a.
LR image: the down samples of observed image
at different sub-pixel position.
After modeling and adding boundary condition, it can be
reduced to :
L
f
=
g
,
Where L is blurring matrix, g is the
observed image and f is the original
image.
The problem L f = g is ill-conditioned.
Regularization is required:
( L* L   R)f  L*g.
Here R can be I, . It is called Tikhonov method
( or the least square )
g
( L* L) 1 L*g
( L* L   R) 1 L*g
Wavelet Method
• Let â be the symbol of the low-pass filter. Assume:
d ˆ ˆd
ˆ
a
, b, b can be found such that
•
aˆ d aˆ 
ˆ d bˆ  1
b
  
Z 22 \{ 0}
• One can use unitary extension principle to obtain a
set of tight frame systems.
Let  be the refinable function with refinement mask a, i.e.
  4  a( ) (2   ) .
Z 2
Let  d be the dual function of  :
 ,  d       0  .
We can express the true image as
f  2  v   d 2    ,
Z 2
where v() are the pixel values of the high-resolution picture.
The pixel values of the observed image are given by
a * v   ,
  Z2
The observed function is
g 
d


(
a


)


(   / 2) .

Z 2
The problem is to find v( ) from (a * v)().
From 4 sets low resolution pixel values reconstruct f, lift
1 level up. Similarly, one can have 2 level up from 16 set...
Do it in the Fourier domain. Note that
aˆ d aˆ 

bˆd bˆ  1 .
 Z 22 \{ 0}
We have

aˆ aˆvˆ    bˆd bˆ
  Z 2 \0
 2
d

vˆ  vˆ .


or

aˆ a * v    bˆd bˆ
  Z 2 \0
 2
d

vˆ  vˆ .


(1)
Generic Wavelet Algorithm:
(i) Choose
vˆ0  L2   ,  ;
2
(ii) Iterate until convergence:

vˆn 1  aˆ a * v    bˆd bˆ
  Z 2 \0
 2
d

vˆn .


Proposition Suppose that 0  a
ˆda
ˆ  1and nonzero
almost everywhere. Then || vˆn  vˆ || 2  0for
arbitrary v̂0 .
Regularization:
Damp the high-frequency components in the current iterant.
Wavelet Algorithm I:
(i) Choose
ˆv0  L2   , 2 ;
(ii) Iterate until convergence:


d
vˆn 1  aˆ a * v  (1   )  bˆ bˆ  vˆn .
  Z 2 \0 
 2

d
Matrix Formulation:
The Wavelet Algorithm I is the stationary iteration for
( Ld L    H d H  )f  Ld g .

Different between Tikhonov and Wavelet Models:
• Ld instead of L*.
• Wavelet regularization operator.
Both penalize high-frequency components uniformly by .
Wavelet Thresholding Denoising Method:
Decompose the n-th iterate, i.e. bˆ vˆn , into different
scales: ( This gives a wavelet packet decomposition of n-th
iterate.)
 
J 1
 
bˆ vˆ n  aˆ d aˆ  bˆ vˆn   aˆ d
J
Before reconstruction,
j 0

j ˆ
d ˆ
ˆ
b b aˆ  b vˆn ,
Z 22 \0, 0 
j
• Denoise these coefficients bˆ aˆ  bˆ vˆn of the wavelet
packet by thresholding method.
Wavelet Algorithm II:
(i) Choose
vˆ0  L2   ,  ;
2
(ii) Iterate until convergence:
ˆvn1  aˆ d a * v 

 Z 22 \ 0 ,0 
 
ˆb d T bˆ vˆ

 n
Where T is a wavelet thresholding processing .
4  4 sensor array:
Original
LR Frame
Tikhonov
Algorithm I
Observed HR
Algorithm II
4  4 sensor array:
Tikhonov
Algorithm II
Numerical Examples:
22 sensor array: 1 level up
SNR
(dB)
30
40
Tikhonov
Algorithm I
Algorithm II
PSNR
RE
PSNR
RE
PSNR
RE
Iter.
32.55 0.0437 33.82 0.0377 34.48 0.0350
9
33.88 0.0375 34.80 0.0337 35.23 0.0321
12
44 sensor array: 2 level up
SNR
(dB)
30
40
Tikhonov
Algorithm I
Algorithm II
PSNR
RE
PSNR
RE
PSNR
RE
Iter.
29.49 0.0621 29.70 0.0601 30.11 0.0579
30
30.17 0.0573 30.30 0.0566 30.56 0.0549
45
1-D Example: Signal from Donoho’s Wavelet Toolbox.
Blurred by 1-D filter.
Original Signal
Tikhonov
Observed HR Signal
Algorithm II
Calibration Error:
High-resolution
pixels
Displacement error
Ideal low-resolution
pixel position
Displaced lowresolution pixel
ex
Problem no
longer spatially
invariant.
The lower pass filter is perturbed
The wavelet algorithms can be modified
Reconstruction for 4  4 Sensors: (2 level up)
Original
Tikhonov
LR Frame
Observed HR
Wavelets
Reconstruction for 4  4 Sensors: (2 level up)
Tikhonov
Wavelets
Numerical Results:
2  2 sensor array (1 level up) with calibration errors:
Least Squares Model
Our Algorithm
SNR(dB)
PSNR
RE
*
PSNR
RE
Iterations
30
28.00
0.0734
0.0367
30.94
0.0524
2
40
28.24
0.0715
0.0353
31.16
0.0511
2
4  4 sensor array (2 level) with calibration errors:
Least Squares Model
Our Algorithm
SNR(dB)
PSNR
RE
*
PSNR
RE
Iterations
30
24.63
0.1084
0.0492
27.80
0.0752
5
40
24.67
0.1078
0.0505
26.81
0.0751
6
Super-resolution: not enough frames
Example: 4  4 sensor with missing frames:
(0,0)
(0,1)
(0,2)
(0,3)
(1,0)
(1,1)
(1,2)
(1,3)
(2,0)
(2,1)
(2,2)
(2,3)
(3,0)
(3,1)
(3,2)
(3,3)
Super-resolution: not enough frames
Example: 4  4 sensor with missing frames:
(0,1)
(1,0)
(0,3)
(1,2)
(2,1)
(3,0)
(2,3)
(3,2)
Super-Resolution:
Not enough low-resolution frames.
i.
Apply an interpolatory subdivision scheme to obtain
the missing frames.
ii. Generate the observed high-resolution image w.
iii. Solve for the high-resolution image u.
iv. From u, generate the missing low-resolution frames.
v. Then generate a new observed high-resolution image g.
vi. Solve for the final high-resolution image f.
Tikhonov
Algorithm I
Algorithm II
PSNR
RE
PSNR
RE
PSNR
RE
27.44
0.0787
27.82
0.0753
27.76
0.0758
Reconstructed Image:
Observed LR
Final Solution