Computer Vision –
Sampling
Hanyang University
Jong-Il Park
Introduction
 Topics
 Television Standards (NTSC, SECAM, PAL)
 Multi-dimensional Sampling Theory
 Practical Limitations in Sampling and Reconstruction
 Image Re-Sampling
Department of Computer Science and Engineering, Hanyang University
Department of Computer Science and Engineering, Hanyang University
Television Standards
 Frame
 525 lines/frame
(or 625 lines/frame)
 frame rate : 30 frames/s
(or 25 frames/s)
 Field
 even field, odd field
 262.5 lines/field
(or 312.5 lines/field)
Department of Computer Science and Engineering, Hanyang University
NTSC
 NTSC
 525 scan lines/frame, 30 frames/s
 line frequency : 15,750 Hz ( = 30 x 525 Hz )
 2:1 line interlacing
 color video composite signal - (Y,I,Q)
 Bandwidth : Y - 4.2MHz,
I - 1.3MHz, Q - 0.5MHz
 color sub-carrier : 3.583125 MHz
( = 30 x 525 x 455/2 Hz)
 phase change of 180 : between lines, between
frames
 Korea, North America, Japan etc.

Never Twice Same Color!
Department of Computer Science and Engineering, Hanyang University
Other Standards
 SECAM(Sequential Couleur a Memoire)
 Idea
 avoid the quadrature demodulation and corresponding
chrominance shift due to phase detection errors in
NTSC
 France, Eastern Europe.
 625 lines/frame, 25 frames/s with 2:1 line interlace.
 color video composite signal - (Y,U,V)
 color sub-carrier : 4.25 MHz (for U) and 4.41 MHz (for
V)

Something Essentially Contradictory to American
Method!
Department of Computer Science and Engineering, Hanyang University
Other Standards (cont.)
 PAL (Phase Alternating Line)
 Idea
 changes by 180 degree between successive line in
the same field
 cross talk can be suppressed
• Germany, UK, South America
• 625 lines at 25frames/s with 2:1 line interlace.
• color video composite signal - (Y,U,V)


(Bandwidth) Y - 4.2MHz, U - 1.3MHz, V - 1.3MHz
Peace At Last!
Department of Computer Science and Engineering, Hanyang University
Sampling Theory
 For One-Dimensional Signal
g (t )
g S (t )
Periodic
Sampling
s(t )
g S (t )  g (t )  s(t )
where s(t )    (t  mt )
m
S g S ( f )  S g ( f )  SS ( f )
(Fourier T ransform)
Department of Computer Science and Engineering, Hanyang University
Sampling Theory (cont.)
 For One-Dimensional Signal (cont.)
Ss(f)
Sg(f)
...
...
-B
B
f
-2fs
-fs
0
fs
2fs
f
Nyquist Sampling Rate
fs > 2B
fs =1/ t
reconstruction filter
Sgs(f)
...
...
-2fs
-fs -B
B fs
2fs
3fs
4fs
f
Department of Computer Science and Engineering, Hanyang University
Sampling Theory(cont.)
 For Two-Dimensional Signal
 Band-limited Image
Fourier Transform of
a bandlimited function
Its region of support
Department of Computer Science and Engineering, Hanyang University
Sampling Theory (cont.)
 For Two-Dimensional Signal (cont.)
 Structure
 Orthogonal Structure (Rectangular Tesselation)
 Field Quincunx Structure (Triangular Tesselation)
g ( x, y )
g S ( x, y)
Periodic
Sampling
s(t )
g S ( x, y)  g ( x, y)  s( x, y)
SgS (u, v)  Sg (u, v)  SS (u, v)
(Fourier T ransform)
Department of Computer Science and Engineering, Hanyang University
Sampling Theory(cont.)
 For Two-Dimensional Signal(cont.)
 Structure(cont.)
Department of Computer Science and Engineering, Hanyang University
 On a rectangular samplinggrid,
theNyquist samplinginterval; Δx  Δy  Δ1  1
 On a triangular samplinggrid (quincunx structureor triangular
structure), with theintersample distanceof Δ2 , the
spectrumwill repeat with spaceing Δ12
 if Δ2  2 , then ther
e will be no aliasing
 T hus,if theimage does not contain the high frequencies
in bot h dimension,thesamplingratecan be reduced
by a factorof 2
Department of Computer Science and Engineering, Hanyang University
2D sampling
 For Two-Dimensional Signal (cont.)
 Orthogonal Structure (Rectangular Tesselation)
SgS (u, v)  Sg (u, v)  SS (u, v)
g S ( x, y)  g ( x, y)  s( x, y)
(Fourier T ransform)

Sampling Function
s ( x, y )    ( x  mx, y  ny )
n
m
S s (u , v)  uv   (u  ku , v  lv)
k
where u 
l
1
1
, v 
x
y
Department of Computer Science and Engineering, Hanyang University
2D sampling - Spectrum
 For Two-Dimensional Signal (cont.)
 Orthogonal Structure (cont.)
 Spectrum of sampled signals
By
v
y
y
x
x
(a) sampling function
v
u
(b) spectrum of sampling
function and signal
Bx
u
v
v
u
u
(c) spectrum of sampled signal
Department of Computer Science and Engineering, Hanyang University
2D sampling - Reconstruction
 For Two-Dimensional Signal(cont.)
 Orthogonal Structure(cont.)
 Reconstruction of the original image from its samples
If 2 Bx  u and 2 B y  v, then
1

xy 
, (u , v)  
H (u , v)  
uv

, ot herwise
 0
g ( x, y) 


m  

 sin(xu  m)  sin( yv  n) 



g
(
m
,
n
)

S
n  
 ( xu  m)  ( yv  n) 

Nyquist Sampling Rate(or Frequency) and Nyquist
Interval
u Nyquist  2 Bx , vNyquist  2 By ,
1
1
1
1
xNyquist 

, y Nyquist 

,
u Nyquist 2 Bx
vNyquist 2 By
Department of Computer Science and Engineering, Hanyang University
Reconstruction Filter
 sin  x x  sin  y y 


hr ( x, y )  K 


  x x   y y 
hc ( x, y ) 

20 J1  0 x 2  y 2

x2  y2
where J1 () is a first - order Bessel function
Department of Computer Science and Engineering, Hanyang University
Aliasing effect
 For Two-Dimensional Signal(cont.)
 Orthogonal Structure(cont.)
 Aliasing Effect
If u  uNyquist or v  vNyquist , then
the original image cannot be reconstructed from its samples.
Bx
u
nu
(n  1)u
Department of Computer Science and Engineering, Hanyang University
Eg. Aliasing
 For Two-Dimensional Signal (cont.)
 Orthogonal Structure (cont.)
 Aliasing Effect (cont.)
Zone Plate image ( = 1)
Aliasing ( = 2)
Department of Computer Science and Engineering, Hanyang University
Eg. Aliasing
 Examples
Little aliasing due to an effective antialiasing filter
Noticeable aliasing
Department of Computer Science and Engineering, Hanyang University
Practical limitations in sampling
 Practical Limitations
 Real-world images are not band-limited.
 aliasing errors
 can be reduced by LPF before sampling
 LPF attenuate higher spatial frequencies
 Resolution loss
 blurring
 No ideal LPF at reconstruction stage.
Department of Computer Science and Engineering, Hanyang University
Sampling aperture
 Finite aperture (finite duration pulse)
H 0    e
 j T 2
 2 sin  T 2 



Department of Computer Science and Engineering, Hanyang University
Sampling aperture - Spectrum
 Practical Limitations (cont.)
 Sampling Aperture/ LPF operation
Department of Computer Science and Engineering, Hanyang University
Reconstruction
 Reconstruction of a signal from its sample using
interpolation
Department of Computer Science and Engineering, Hanyang University
Linear interpolation
 linear interpolation
Department of Computer Science and Engineering, Hanyang University
Department of Computer Science and Engineering, Hanyang University
Sampling Theory(cont.)
Department of Computer Science and Engineering, Hanyang University
Geometrical Image Resampling
 Bilinear interpolation
F ( p, q)  (1  a)[(1  b) F ( p, q)  bF( p, q  1)]
 a[(1  b) F ( p  1, q)  bF( p  1, q  1)]
Department of Computer Science and Engineering, Hanyang University
Geometrical Image Resampling(Cont.)
 Bicubic interpolation
F ( p, q) 
2
2
  F ( p  m, q  n) R [m  a]R [(n  b)]
m  1 n  1
c
c
1
Rc ( x)  [(x  2)3  4( x  1)3  6( x)3  4( x  1)3 ] ,
6
where ( z ) m  z m for z  0, 0 for z  0
Department of Computer Science and Engineering, Hanyang University
Resampling by Convolution
 Convolution methods : integer zoom(ex. 2 : 1)
 Zero Interleaving
 Convolution
Pyramid : 1 1

Peg :

Cubic B-spline :
1 1
1 1


2 1
2 4 2

4
1 2 1 
1
4
1 
6
64 
4
1
Bell :
1

1 3
16 3

1
3
9
9
3
3
9
9
3
1
3
3

1
4 6 4 1
16 24 16 1
24 36 24 6

16 24 16 1
4 6 4 1
Department of Computer Science and Engineering, Hanyang University
Eg: Resampling by Convolution
Original
Zero interleaving
Department of Computer Science and Engineering, Hanyang University
Eg: Resampling by Convolution(Cont.)
peg
Bell
Pyramid
Cubic B-spline
Department of Computer Science and Engineering, Hanyang University
Image Warping
 image filtering: change range of image
g(x) = h(f(x))
f
f
h
x
x
 image warping: change domain of image
g(x) = f(h(x))
f
f
h
x
x
Department of Computer Science and Engineering, Hanyang University
Image Warping
 image filtering: change range of image
g(x) = h(f(x))
f
g
h
 image warping: change domain of image
g(x) = f(h(x))
f
g
h
Department of Computer Science and Engineering, Hanyang University
Parametric (global) warping
 Examples of parametric warps:
translation
affine
rotation
perspective
aspect
cylindrical
Department of Computer Science and Engineering, Hanyang University
2D Coordinate Transformations
x’ = x + t
x = (x,y)
rotation:
x’ = R x + t
similarity:
x’ = s R x + t
affine:
x’ = A x + t
perspective:
x’  H x
x = (x,y,1)
(x is a homogeneous coordinate)
 translation:




These all form a nested group
Department of Computer Science and Engineering, Hanyang University
Image Warping
 Given a coordinate transform x’ = h(x) and a source
image f(x), how do we compute a transformed image
g(x’) = f(h(x))?
h(x)
x
f(x)
x’
g(x’)
Department of Computer Science and Engineering, Hanyang University
Forward Warping
 Send each pixel f(x) to its corresponding location x’ =
h(x) in g(x’)
• What if pixel lands “between” two pixels?
h(x)
x
f(x)
x’
g(x’)
Department of Computer Science and Engineering, Hanyang University
Forward Warping
 Send each pixel f(x) to its corresponding location x’ =
h(x) in g(x’)
• What if pixel lands “between” two pixels?
• Answer: add “contribution” to several pixels,
normalize later (splatting)
h(x)
x
f(x)
x’
g(x’)
Department of Computer Science and Engineering, Hanyang University
Inverse Warping
 Get each pixel g(x’) from its corresponding location x =
h-1(x’) in f(x)
• What if pixel comes from “between” two pixels?
h-1(x’)
x
f(x)
x’
g(x’)
Department of Computer Science and Engineering, Hanyang University
Inverse Warping
 Get each pixel g(x’) from its corresponding location x =
h-1(x’) in f(x)
• What if pixel comes from “between” two pixels?
• Answer: resample color value from
interpolated (prefiltered) source image
x
f(x)
x’
g(x’)
Department of Computer Science and Engineering, Hanyang University
Interpolation
 Possible interpolation filters:
 nearest neighbor
 bilinear
 bicubic (interpolating)
 sinc / FIR
 Needed to prevent “jaggies”
and “texture crawl”
Department of Computer Science and Engineering, Hanyang University