Image Sampling and
Resizing
Yao Wang
Polytechnic Institute of NYU, Brooklyn, NY 11201
With contribution from Zhu Liu
Partly based on A. K. Jain, Fundamentals of Digital Image Processing
Lecture Outline
• Introduction
• Nyquist
N q ist sampling and interpolation
theorem
• Common sampling and interpolation filters
• Sampling rate conversion of discrete
images (image resizing)
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
2
Illustration of Image Sampling and
Interpolation
dx=dy=2mm
y
16 mm
dx=dy=1mm
Yao Wang, NYU-Poly
How to choose dx, dy to reach a good trade off between accuracy and cost of
storage, transmission,
processing?
EL5123: Sampling
and Resizing
3
Uniform Sampling
• f(x,y) represents the original continuous
image, fs(m,n) the sampled image, and
fˆ ( x, y ) the reconstructed image.
• Uniform sampling
f s (m, n)  f (mx, ny ),
m  0,..., M  1; n  0,..., N  1.
– ∆x and ∆y
y are vertical and horizontal
sampling intervals. fs,x=1/∆x, fs,y=1/ ∆y are
vertical and horizontal sampling frequencies.
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
4
Image Sampling as
Product with Impulse Train
p ( x, y )    ( x  m , y  n
Periodic impulse
M 1 N 1
•
~
x
m 0 n 0
sequence
y
)
M 1 N 1
f s ( x, y )  f ( x, y )  p ( x, y )   f (m x , n y ) ( x  m x , y  n y )
m 0 n 0
M 1 N 1
  f s (m, n) ( x  m x , y  n y )
m 0 n 0
m=0
∆x
1
p(x,y)
…
M-1
∆y
x
Yao Wang, NYU-Poly
n= 0
1
2
EL5123: Sampling and Resizing
N-1
y
5
Fourier Transform of Impulse Train
• 1D
p (t )    (t  nt )  P (u ) 
m ,n
where
fs 
1
  (u  nf s )
t n
1
t
• 2D
p ( x, y )    ( x  mx, y  ny )  P(u, v) 
m,n
where
f s,x 
Yao Wang, NYU-Poly
1
 (u  mf s , x , v  nf s , y )

xy m ,n
1
1
, f s, y 
x
y
EL5123: Sampling and Resizing
6
Frequency Domain Interpretation of
Sampling
• Sampling is equivalent to multiplication of
the original signal with a sampling pulse
sequence.
f ( x, y )  f ( x, y ) p ( x, y )
s
where
p( x, y )    ( x  mx, y  ny )
m,n
• In
I frequency
f
domain
d
i
Fs (u , v)  F (u , v)  P(u , v)
P (u , v) 
where
1
xy

f s, x 
1
1
, f s, y 
x
y
Yao Wang, NYU-Poly
 (u  mf s , x , v  nf s , y )  Fs (u , v) 
m, n
EL5123: Sampling and Resizing
1
xy
 F (u  mf
s , x , v  nf s , y )
m, n
7
Frequency Domain
p
of Sampling
p g in 1D
Interpretation
Original signal
The spectrum of the
sampled signal
includes the original
spectrum and its
aliases (copies) shifted
to k fs , k=+/- 1,2,3,…
Sampling
impulse train
Sampled
S
l d signal
i
l
fs > 2fm
When fs< 2fm , aliases
overlap with the
original spectrum ->
>
aliasing artifact
Sampled signal
fs < 2fm
(Aliasing effect)
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
8
Sampling of 1D Sinusoid Signals
Sampling above
Nyquist
yq
rate
s=3m>s0
Reconstructed
=original
i i l
Sampling under
Nyquist rate
s=1.5m<s0
Reconstructed
!= original
Aliasing: The reconstructed sinusoid has a lower frequency than the original!
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
9
Frequency Domain Interpretation of
Sampling in 2D
• The sampled signal contains replicas of
the original spectrum shifted by multiples
of sampling frequencies.
u
fs,x
u
fs,x>2fm,x
fs,y>2fm,y
fm,x
mx
v
fm,y
Original spectrum F(u
F(u,v)
v)
Yao Wang, NYU-Poly
v
fs,y
Sampled spectrum Fs(u,v)
(u v)
EL5123: Sampling and Resizing
10
Illustration of Aliasing Phenomenon
u
u
fm,x
fs,x
s x<2fm,x
fm x
fs,y<2fm,y
fm,y
Original spectrum F(u,v)
Yao Wang, NYU-Poly
fs,x
fs,y
v
v
Sampled
p
spectrum
p
Fs((u,v))
EL5123: Sampling and Resizing
11
Nyquist Sampling Rate
• Nyquist sampling rate
– To be able to preserve the original signal in
the sampled signals, these aliasing
components should not overlap with the
original one. This requires that the sampling
frequency fs,x
, , fs,y
,y must be at least twice of the
highest frequency of the signal, known as
Nyquist sampling rate.
• Interpolation
– Remove all the aliasing components
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
12
Nyquist Sampling and Reconstruction
Theorem
• A band-limited image with highest frequencies at fm,x, fm,y
can be reconstructed perfectly from its samples,
provided that the sampling frequencies satisfy: fs,x >2fm,x,
fs,y>2fm,y
• The reconstruction can be accomplished by the ideal
low-pass filter with cutoff frequency at fc,x = fs,x/2, fc,y =
fs,y/2, with magnitude ∆x∆y.
f
f

xy | u | s , x , | v | s , y
H (u , v)  
2
2
0 otherwise


sin f s , x x sin f s , y y
h ( x, y ) 

f s , x x
f s , y y
• The interpolated image
sin f s , x ( x  mx) sin f s , y ( y  my )
ˆf ( x, y ) 
f s (m, n)

f s , x ( x  mx) f s , y ( y  my )
m n
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
13
Applying Nyquist Theorem
• Two issues
– The signals are not bandlimited.
• A bandlimiting filter with cutoff frequency fc=fs/2 needs to be
applied before sampling. This is called prefilter or sampling
filter.
filter
– The sinc filter is not realizable.
• Shorter, finite length filters are usually used in practice for
b th prefilter
both
filt and
d interpolation
i t
l ti filt
filter.
• A general paradigm
B
A
C
Prefilter
Interpolation
(postfilter)
D
Sampling pulse fs
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
14
Non-ideal Sampling and
Interpolation
Aliased
Component
Pre-filtered
Signal
-fs
0
Non-ideal
Interpolation
filter
Aliased
Component
fs
Aliasing Imaging
Non ideal prefiltering causes Aliasing
p
filter causes Imaging
g g
Non ideal interpolation
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
15
Sampling a Sinusoidal Signal
f ( x, y )  cos(4x  2y ) 
1
F (u , v)   (u  2, v  1)   (u  2, v  1)
2
S
Sampled
l d att ∆x=∆y=1/3
∆ ∆ 1/3 fs,x=ffs,y=3
3
Original Spectrum
v
(-2,1)
Sampled Spectrum
v
3
3
(-2,1)
-3
3
-3
3
u
(2,-1)
(2,
1)
Original pulse
fˆ ( x, y )  cos(2x  2y )
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
-3
3
u
(2,-1)
Ideal
-3
3
interpolation
filter
Replicated pulse
Replication center
16
Sampling in 2D:
Sampling a 2D Sinusoidal Pattern
f(x,y)=sin(2*π*(3x+y))
Sampling: dx=0
dx=0.01,dy=0.01
01 dy=0 01
Satisfying Nyquist rate
fx,max=3, fy,max=1
fs,x=100>6, fs,y=100>2
Yao Wang, NYU-Poly
f(x,y)=sin(2*π*(3x+y))
Sampling: dx=0.2,dy=0.2
(Displayed with pixel replication)
Sampling at a rate lower than Nyquist rate
EL5123: Sampling and Resizing
17
A Simple Prefilter – Averaging Filter
• Each sampling value is the mean value of the
original continuous function in a rectangular
n
region of dimension ∆x and ∆y, i.e:
f s (m, n) 
1
f ( x, y )dxdy

xy ( x , y )Dm ,n
m
where
Dm ,n  ( m  1 / 2) x  x  ( m  1 / 2) x, ( n  1 / 2) y  y  ( n  1 / 2) y
The equivalent prefilter is
 1
| x | x / 2, | y | y / 2
sin xu sin yv

h( x, y )   xy
 H (u , v)  xy
xu yv

0 otherwise
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
18
How good is the averaging Filter?
• Look at its frequency response, how far is
it from the ideal low pass filter
Ideal low p
pass filter with cutoff at fs/2
Averaging filter
-3f
3fs
Yao Wang, NYU-Poly
-2f
2fs
-ffs
fs
EL5123: Sampling and Resizing
2fs
3fs
19
Reconstruction of continuous image
from Samples
• The interpolation problem
• Interpretation as weighted average of sample
values
• Interpretation as filtering
• What filter should we use?
– Ideal interpolation filter: remove the replicated
spectrum using ideal low-pass filter with cutoff
frequency at fs/2 (the sinc function)
sin f s , x ( x  mx ) sin f s , y ( y  ny )
ˆf ( x, y ) 
f s ( m, n )

f s , x ( x  mx ) f s , y ( y  ny )
m n
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
20
Interpretation as a weighted average
of sample values
•
The value of a function at arbitrary point (x, y) is estimated from a
weighted sum of its sample values in the neighborhood of ([x/∆x],
[y/∆y]):
– Let h(x,y; m,n) specifies the weight assigned to sample m,n, when
determining the image value at x,y
fˆ ( x, y )   h( x, y; m, n) f s (m, n)
m ,n
H(x;7)
H(x;8)
H(x;9)
x
H(x;6)
H(x;10)
H(x;5)
0
Yao Wang, NYU-Poly
1
2
3
4
5
6
7
8
9 10 11 12 13 14
EL5123: Sampling and Resizing
n
21
Desirable Properties of the Weight
Function
• The weighting function h(x,y;m,n) should depend only on
distance between (x,y) and the spatial location of (m,n),
ie
i.e.
h( x, y; m, n)  h( x  mx, y  ny ).
• Should be a decreasing function of the distance
– Higher weight for nearby samples
• Should be an even function of the distance
– L
Left
ft neighbor
i hb and
d right
i ht neighbor
i hb off same di
distance
t
have
h
the
th same
weight
– h1(x)=h1(-x)
• Generally Separable:
– h2(x,y)=h1(x) h1(y)
• To retain the original sample values, should have
– h(0,0)=1,
h(0 0)=1 h(mx,ny)=0
h(mx ny)=0
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
22
Interpretation as Filtering
• The weighted
fˆ ( x, y )   h( x  mx, y  ny ) f s (m, n)
m ,n
average operation is ~
equivalent to filtering f s ( x, y)   f s (m, n) ( x  mx, y  ny )
m,n
fs(x,y) with h(x,y)
~
h ( x, y )  f s ( x, y )
• Usually,
Usually h(x
h(x,y)
y) is
~
separable h(x,y) =   h( x   , y   ) f s ( ,  )dd
hx(x)hy(y)
  h( x   , y   ) f s (m, n) (  mx,   ny )dd
m,n
• To retain the original
  f s (m, n)  h( x   , y   ) (  mx,   ny )dd
sample values,
m ,n
should have
  f (m, n)h( x  mx, y  ny )
– h(0,0)=1,
h(mx,ny)=0
– Nyquist filter
Yao Wang, NYU-Poly
s
m ,n

~
fˆ ( x, y )  h( x, y )  f s ( x, y )
EL5123: Sampling and Resizing
23
A Simple Interpolation Filter:
Sample-And-Hold
Sample
And Hold (pixel replication)
• The interpolated value at a point is
obtained from that of its nearest sample
fˆ ( x, y )  f s (m, n) (m  1 / 2)x  x  (m  1 / 2)x, (n  1 / 2)y  y  (n  1 / 2)y
• Corresponding interpolation filter is
1  x / 2  x  x / 2, y / 2  y  y / 2
h ( x, y )  
0 otherwise

0th order interpolation filter
n
(m-1/2)∆x
(m+1/2)∆x
m
(n-1/2)∆y
(n
1/2)∆y
(n+1/2)∆y
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
24
Bilinear Filter
f1
fa
f0
0
a
(m,n+1)
(m,n)
b
Step 1 :
a
1
f a  (1  a ) f 0  af1
1D Linear interpolation
f (q1 )  (1  a) f (m, n)  af (m  1, n)
q1
p
(m+1,n)
q2
f (q2 )  (1  a ) f (m, n  1)  af (m  1, n  1)
S 2:
Step
f ( p )  (1  b) f (q1 )  bf (q2 )
(m+1,n+1)
2D bilinear interpolation
Corresponding interpolation filter
 | x |  | y | 
 1
  x  x  x, y  y  y
 1 
h( x, y )  
y 
x 

0
otherwise
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
25
Which filter is better?
• Recall what happened in the frequency
domain when we sample an image
• Ideal filter: ½ band ideal low pass filter
• Quantitatively we can evaluate how far is
the filter from the ideal filter
• But we should also look at visual artifacts
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
26
Frequency Domain Interpretation of
Sampling in 2D
• The sampled signal contains replicas of
the original spectrum shifted by multiples
of sampling frequencies.
u
fs,x
u
fs,x>2fm,x
fs,y>2fm,y
fm,x
mx
v
fm,y
Original spectrum F(u
F(u,v)
v)
Yao Wang, NYU-Poly
v
fs,y
Sampled spectrum Fs(u,v)
(u v)
EL5123: Sampling and Resizing
27
Ideal Interpolation Filter
• The ideal interpolation filter should be a low-pass filter
with cutoff frequency at fc,x = fs,x/2, fc,yy = fs,yy/2, with
magnitude ∆x∆y
f
f

xy | u | s , x , | v | s , y
H (u , v)  
2
2

0 otherwise

sin f s , x x sin f s , y y
h ( x, y ) 

f s , x x
f s , y y
The sinc filter
• The interpolated image
Weight function h(x,y;m,n)
sin f s , x ( x  mx) sin f s , y ( y  ny )
ˆf ( x, y ) 
f s (m, n)

f s , x ( x  mx) f s , y ( y  ny )
m n
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
28
Comparison of Different Interpolation
Filters
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
29
Image Resizing
• Image resizing:
– Enlarge or reduce the image size (number of pixels)
– Equivalent to
• First reconstruct the continuous image from samples
• Then Resample
p the image
g at a different sampling
p g rate
– Can be done w/o reconstructing the continuous image
explicitly
• Image down-sampling (resample at a lower rate)
– Spatial domain view
– Frequency domain view: need for prefilter
• Image up-sampling (resample at a higher rate)
– Spatial domain view
– Different interpolation filters
• Nearest neighbor, Bilinear, Bicubic
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
30
Image Down-Sampling
• Example:
– reduce a 512x512 image to 256x256 = factor
of 2 downsampling in both horizontal and
vertical directions
– In general, we can down-sample by an
arbitraryy factor in the horizontal and vertical
directions
• How should we obtain the smaller image ?
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
31
Down Sampling by a Factor of Two
8x8 Image
4x4 Image
• Without Pre
Pre-filtering
filtering (simple approach)
f d (m, n)  f (2m,2n)
• Averaging Filter
f d (m, n)  [ f (2m,2n)  f (2m,2n  1)  f (2m  1,2n)  f (2m  1,2n  1)] / 4
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
32
Problem of Simple Approach
• Aliasing if the effective sampling rate is below
the Nyquist sample rate = 2 * highest frequency
in the original continuous signal
• We need to p
prefilter the signal
g
before downsampling
• Ideally the prefilter should be a low-pass filter
with a cut-off frequency half of the new sampling
rate.
– In digital frequency of the original sampled image, the
cutoff frequency is ¼.
• In practice,
practice we may use simple averaging filter
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
33
Down Sampling by a Factor of K
f(m,n)
fˆ (m, n)
Hs
↓K
Pre-filtering
Down-sampling
fd(m,n)
f d (m, n)  fˆ ( Km, Kn)
For factor of K down sampling, the prefilter should be low pass
filter with cutoff at fs/(2K), if fs is the original sampling frequency
I terms
In
t
off digital
di it l ffrequency, the
th cutoff
t ff should
h ld be
b 1/(2K)
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
34
Example: Image Down-Sample
Without prefiltering
With prefiltering (no aliasing, but blurring!)
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
35
Down-Sampling Using Matlab
• Without prefiltering
– If f(,) is an MxN image, down-sampling by a factor of K can be
done simply by
>> g=f(1:K:M,1:K:N)
• With prefiltering
p
g
– First convolve the image with a desired filter
• Low pass filter with digital cutoff frequency 1/(2K)
– In matlab, 1/2 is normalized to 1
– Then subsample
>> h=fir1(N, 1/K)
%design a lowpass filter with cutoff at 1/K
1/K.
>> fp=conv2(f,h)
>> g=fp(1:K:M,1:K:N)
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
36
Image Up-Sampling
•
Produce a larger image from a smaller one
– Eg. 512x512 -> 1024x1024
– More
M
generally
ll we may up-sample
l b
by an arbitrary
bit
ffactor
t L
•
Questions:
– How should we generate a larger image?
– Does the enlarged image carry more information?
•
Connection with interpolation of a continuous image from discrete
image
– First interpolate to continuous image, then sampling at a higher sampling
rate, L*fs
– Can be realized with the same interpolation filter, but only evaluate at
x=mx’, y=ny
x=mx
y=ny’, x
x’=x/L
=x/L, y
y’=y/L
=y/L
– Ideally using the sinc filter!
sin f s , x ( x  mx) sin f s , y ( y  my )
ˆf ( x, y ) 
f s (m, n)

f s , x ( x  mx) f s , y ( y  my )
m n
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
37
Example: Factor of 2 Up-Sampling
(m,n)
(m,n+1)
(2m,2n) (2m,2n+1)
(2m+1 2n) (2m+1,2n+1)
(2m+1,2n)
(2m+1 2n+1)
(m+1,n)
(m+1,n+1)
Green samples are retained in the interpolated image;
Orange samples are estimated from surrounding green samples.
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
38
Nearest Neighbor Interpolation
(pixel replication)
(m n)
(m,n)
(m n+1)
(m,n+1)
b
a
(m+1,n)
(m’/M,n’/M)
(m+1,n+1)
O[m’,n’] (the resized image) takes the value of the sample nearest to
(m’/M,n’/M) in I[m,n] (the original image):
O[m' , n
O[m
n' ] = I[(int) (m + 0.5),
0 5) (int) (n + 0.5)]
0 5)] , m = m
m' /M,
/M n = nn' /M.
/M
Also known as pixel replication: each original pixel is replaced by MxM pixels of
the sample value
Equivalent to using the sample-and-hold interpolation filter.
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
39
Special Case: M=2
(m,n)
(m,n+1)
(2m,2n) (2m,2n+1)
(2m+1 2n) (2m+1,2n+1)
(2m+1,2n)
(2m+1 2n+1)
(m+1,n)
(m+1,n+1)
Nearest Neighbor:
O[2m,2n]=I[m,n]
O[2m,2n+1]= I[m,n]
O[2m+1,2n]= I[m,n]
O[2m+1,2n+1]= I[m,n]
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
40
Bilinear Interpolation
(m,n)
(m,n+1)
b
a
(m+1,n)
(m’/M,n’/M)
(m+1,n+1)
• O(m
O(m’,n’)
,n ) takes a weighted average of 4 samples nearest to (m’/M,n’/M)
(m /M,n /M) in I(m,n).
• Direct interpolation: each new sample takes 4 multiplications:
O[m’,n’]=(1-a)*(1-b)*I[m,n]+a*(1-b)*I[m,n+1]+(1-a)*b*I[m+1,n]+a*b*I[m+1,n+1]
• Separable interpolation:
i) interpolate along each row y: F[m,n’]=(1-a)*I[m,n]+a*I[m,n+1]
ii) interpolate along each column x’: O[m’,n’]=(1-b)*F[m’,n]+b*F[m’+1,n]
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
41
Special Case: M=2
(m,n)
(m,n+1)
(2m,2n) (2m,2n+1)
(2m+1 2n) (2m+1,2n+1)
(2m+1,2n)
(2m+1 2n+1)
(m+1,n)
(m+1,n+1)
Bilinear Interpolation:
O[2m,2n]=I[m,n]
O[2m,2n+1]=(I[m,n]+I[m,n+1])/2
O[2m+1,2n]=(I[m,n]+I[m+1,n])/2
O[2m+1,2n+1]=(I[m,n]+I[m,n+1]+I[m+1,n]+I[m+1,n+1])/4
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
42
Bicubic Interpolation
(m-1,n)
(m,n-1)
(m-1,n+1)
(m,n)
(m,n+1)
(m,n+2)
b
a
(m+1,n-1)
(m’/M,n’/M)
(m+1,n)
(m+1,n+1)
(m+2,n)
(m+2,n+1)
(m+1,n+2)
• O(m’,n’)
O( ’ ’) iis iinterpolated
t
l t d ffrom 16 samples
l nearestt to
t (m’/M,n’/M)
( ’/M ’/M) in
i I(m,n).
I(
)
• Direct interpolation: each new sample takes 16 multiplications
• Separable interpolation:
i) interpolate along each row y: I[m,n]->F[m,n’] (from 4 samples)
ii) interpolate
i t
l t along
l
each
h column
l
x’:’ F[
F[m,n’]->
’] O[m’,n’]
O[ ’ ’] (from
(f
4 samples)
l )
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
43
Interpolation Formula
(m-1,n-1)
(m n 1)
(m,n-1)
(m-1,n)
(m n)
(m,n)
(m-1,n+1)
(m-1,n+2)
(m n+1)
(m,n+1)
(m n+2)
(m,n+2)
b
F(m’,n-1)
( , )
F(m’,n)
F(m
n)
(m+1,n-1)
(m+1,n)
(m+1,n+1)
((m+2,n)
, )
((m+2,n+1)
,
)
F(m’,n+1)
F(m
n+1)
a (m’/M,n’/M)
(m+2,n-1)
F(m’,n+2)
F(m
n+2)
(m+1,n+2)
(m+2,n+2)
F [m' , n ]  b(1  b) 2 I [m  1, n ]  (1  2b 2  b3 ) I [m, n ]  b(1  b  b 2 ) I [m  1, n ]  b 2 (1  b) I [m  2, n ],
]
m'
m'
m
where m  (int) , b 
M
M
O[m' , n' ]  a (1  a ) 2 F [m' , n  1]  (1  2a 2  a 3 ) F [m' , n ]  a (1  a  a 2 ) F [m' , n  1]  a 2 (1  a ) F [m' , n  2],
n'
n'
where n  (int) , a 
n
M
M
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
44
Special Case: M=2
(m-1,n)
(m,n-1)
(2m,2n) (2m,2n+1)
(m-1,n+1)
(m,n)
(m,n+1)
(m,n+2)
(2m,2n+1)
(m+1,n-1)
(2m+1 2n) (2m+1,2n+1)
(2m+1,2n)
(2m+1 2n+1)
(m+1,n)
(m+1,n+1)
(m+2,n)
(m+2,n+1)
(m+1,n+2)
Bicubic interpolation in Horizontal direction
F[2m,2n]=I[m,n]
F[2m,2n+1]= -(1/8)I[m,n-1]+(5/8)I[m,n]+(5/8)I[m,n+1]-(1/8)I(m,n+2)
Same operation then repeats in vertical direction
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
45
Comparison of
Interpolation Methods
Resize_peak.m
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
46
Up-Sampled from w/o Prefiltering
Original
Nearest
neighbor
Bilinear
Bicubic
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
47
Up-Sampled from with Prefiltering
Original
Nearest
neighbor
Bilinear
Bicubic
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
48
Matlab for Image Resizing
[img]=imread('fruit.jpg','jpg');
%downsampling without prefiltering
i 1 i
img1=imresize(img,0.5,'nearest');
i (i
0 5 '
')
%upsampling with different filters:
img2rep=imresize(img1,2,'nearest');
i 2li i
img2lin=imresize(img1,2,'bilinear');
i (i 1 2 'bili
')
img2cubic=imresize(img1,2,'bicubic');
%down sampling with filtering
img1=imresize(img,0.5,'bilinear',11);
%upsampling with different filters
img2rep=imresize(img1 2 'nearest');
img2rep=imresize(img1,2,
nearest );
img2lin=imresize(img1,2,'bilinear');
img2cubic=imresize(img1,2,'bicubic');
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
49
Filtering View: Up Sampling by a Factor
of K
f(m,n)
~
f (m, n)
↑K
Hi
Zero-padding
fu(m,n)
Post-filtering
l l off K
 f (m / K , n / K ) iff m, n are multiple
~
f (m, n)  
0
otherwise

~
f u (m, n)   h(k , l ) f (m  k , n  l )
k ,l
Ideally H should be a low pass filter with cutoff at 1/2K in digital frequency, or
fs/2K in continuous frequency
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
50
Homework (1)
1.
2.
Consider a 1D signal f(t) = sin(4πt). Illustrate the original and the sampled
signal f(n) obtained with a sampling interval ∆t = 1/3. Draw on the same
figure the interpolated signal from the sampled one using the sampleand-hold and the linear interpolation filter
filter, respectively
respectively. Explain the
observed phenomenon based on both the Nyquist sampling theorem as
well as physical interpretation. What is the largest sampling interval that
can be used to avoid aliasing?
Consider a function f(x, y) = cos2π(4x + 2y) sampled with a sampling
period of ∆x = ∆y = ∆ = 1/6 or sampling frequency fs = 1/∆ = 6.
a) Assume that it is reconstructed with an ideal low-pass filter with cut-off
frequency fcx = fcy = 1/2fs. Illustrate the spectra of the original, sampled, and
reconstructed signals. Give the spatial domain function representation of the
reconstructed signal
signal. Is the result as expected?
b) If the reconstruction filter has the following impulse response:
1   / 2  x, y   / 2
h ( x, y )  
0 otherwise

Illustrate the spectra of the reconstructed signal in the range -fs ≤ u,v ≤ fs.
Give a spatial domain function representation of the reconstructed signal if the
, ≤ fs)). ((i.e.,, this filter remains the
reconstruction filter is band-limited to ((-fs ≤ u,v
same for the frequency range -fs ≤ u,v ≤ fs, and is set to 0 outsize this range.)
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
51
Homework(2)
3.
(Computer Assignment) Write your own program or programs
which can: a) Down sample an image by a factor of 2, with and
without using the averaging filter; b) Up-sample the previously
d
down-sampled
l d iimages b
by a ffactor
t off 2
2, using
i th
the pixel
i l replication
li ti
and bilinear interpolation methods, respectively. You should have
a total of 4 interpolated images, with different combination of
down-sampling and interpolation methods. Your program could
either
ith directly
di tl display
di l on screen th
the processed
d iimages d
during
i
program execution, or save the processed images as computer
files for display after program execution. Run your program with
the image Barbara. Comment on the quality of the down/up
sampled
l d iimages obtained
bt i d with
ith diff
differentt methods.
th d
Note: you should not use the ”imresize” function in Matlab to do
this assignment. But you are encouraged to compare results of
yyour p
program
g
with ”resize”.
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
52
Reading
• R
R. Gonzalez
Gonzalez, “Digital
Digital Image Processing
Processing,”
Section 2.4
• A.K.
A K Jain
Jain, “Fundamentals
Fundamentals of Digital Image
Processing,” Section 4.1-4.4
Yao Wang, NYU-Poly
EL5123: Sampling and Resizing
53