Superresolution and Blind Deconvolution
of Images and Video
Filip Šroubek, Jan Flusser, and Michal Šorel
Institute of Information Theory
and Automation
Academy of Sciences of the
Czech Republic
Prague
04/12/10
SIAM Imaging Science 2010
1
Traffic surveillance
Can we read license plates?
04/12/10
SIAM Imaging Science 2010
2
Empirical observation
• One image is not enough
– ill-posed problem
• Solution
– strong prior knowledge of blurs and/or the
original image
OR
– more images
– techniques how to combine them
04/12/10
SIAM Imaging Science 2010
3
Outline
•
•
•
•
Mathematical model
Algorithm
Examples
Extension to the space-variant case
04/12/10
SIAM Imaging Science 2010
4
Multichannel Acquisition Model
noise
channel K
+
channel 2
channel 1
original image
D[u
u(x)
04/12/10
acquired images
*
hk](x)
+ nk(x)
SIAM Imaging Science 2010
= zk(x)
5
Multichannel Deconvolution
Super-resolution
04/12/10
SIAM Imaging Science 2010
6
Misregistration
04/12/10
SIAM Imaging Science 2010
7
Misregistration
original image
PSF
degraded image
• Incorporating between-image shift
04/12/10
SIAM Imaging Science 2010
8
Superresolution & Blind Deconv.
• Acquisition model
• Optimization problem
Data
term
04/12/10
SIAM Imaging Science 2010
Image
regularization
term
Blur
Regularization
term
9
Regularization Terms
04/12/10
SIAM Imaging Science 2010
10
PSF Regularization
u
z1 = u
z1 * h2
=
* h1
u *
h2
h1 * h2
* u
=
z2
h1 * h2 * u = h1 * z2
0
04/12/10
SIAM Imaging Science 2010
11
AM Algorithm
• Alternating minimizations of F(u,{hk})
• Input: blurred LR images and
estimation of PSF size
• Output: HR image and PSFs
• Blind deconvolution in the SR framework
04/12/10
SIAM Imaging Science 2010
12
Blind Deconvolution
04/12/10
SIAM Imaging Science 2010
13
04/12/10
SIAM Imaging Science 2010
14
Superresolution
rough registration
Optical zoom (ground truth)
Superresolved image (2x)
04/12/10
SIAM Imaging Science 2010
15
Space-variant Case
04/12/10
SIAM Imaging Science 2010
16
Space-variant Case
• Video with local motion
• Space-variant PSFs and/or misregistered
images
04/12/10
SIAM Imaging Science 2010
17
interpolated
SR
interpolated
t-2
SR
t-1
t
t+1
t+2
interpolated
t-2
SR
t-1
t
SR + masking
t+1
t+2
Out-of-focus Blur
04/12/10
SIAM Imaging Science 2010
21
Camera-motion Blur
04/12/10
SIAM Imaging Science 2010
22
Space-variant Superresolution
04/12/10
SIAM Imaging Science 2010
23
04/12/10
SIAM Imaging Science 2010
24
Close-up
Original
Input LR
Space-variant
Reconstruction
04/12/10
SIAM Imaging Science 2010
Space-invariant
Reconstruction
25
Misregistered Images
04/12/10
SIAM Imaging Science 2010
26
Misregistered Images - Results
Space-invariant
Space-variant
04/12/10
SIAM Imaging Science 2010
27
MATLAB Application
zoi.utia.cas.cz/download
04/12/10
SIAM Imaging Science 2010
28
Thank You for
Your Attention
04/12/10
SIAM Imaging Science 2010
29