Digital image processing
Chapter 3. Image sampling and quantization
IMAGE SAMPLING AND IMAGE QUANTIZATION
1. Introduction
2. Sampling in the two-dimensional space
Basics on image sampling
The concept of spatial frequencies
Images of limited bandwidth
Two-dimensional sampling
Image reconstruction from its samples
The Nyquist rate. The alias effect and spectral replicas superposition
The sampling theorem in the two-dimensional case
Non-rectangular sampling grids and interlaced sampling
The optimal sampling
Practical limitations in sampling and reconstruction
3. Image quantization
4. The optimal quantizer
The uniform quantizer
5. Visual quantization
Contrast quantization
Pseudo-random noise quantization
Halftone image generation
Color image quantization
Digital image processing
Chapter 3. Image sampling and quantization
1.
f(x,y)
Sampling
fs(x,y)
Introduction
Quantization
u(m,n)
Computer
Digitization
Computer
u(m,n)
D/A
conversion
Display
Analog display
Fig 1 Image sampling and quantization / Analog image display
Digital image processing
Chapter 3. Image sampling and quantization
2.
Sampling in the two-dimensional space
Basics on image sampling
f(x,y)
x
y
Digital image processing
Chapter 3. Image sampling and quantization
The concept of spatial frequencies
- Grey scale images can be seen as a 2-D generalization of time-varying signals (both in the analog
and in the digital case); the following equivalence applies:
1-D signal (time varying)
2-D signal (grey scale image)
Time coordinate t
Space coordinates x,y
Instantaneous value: f(t)
Brightness level, point-wise: f(x,y)
A 1-D signal that doesn’t vary in time (is constant)
= has 0 A.C. component, and only a D.C.
component
A perfectly uniform image (it has the same
brightness in all spatial locations); the D.C.
component = the brightness in any point
The frequency content of a 1-D signal is
proportional to the speed of variation of its
instantaneous value in time:
νmax ~ max(df/dt)
The frequency content of an image (2-D signal) is
proportional to the speed of variation of its
instantaneous value in space:
νmax,x ~ max(df/dx); νmax,y ~ max(df/dy)
=> νmax,x , νmax,y = “spatial frequencies”
Discrete 1-D signal: described by its samples => a
vector: u=[u(0) u(1) … u(N-1)], N samples; the
position of the sample = the discrete time moment
Discrete image (2-D signal): described by its
samples, but in 2-D => a matrix: U[M×N],
U={u(m,n)}, m=0,1,…,M-1; n=0,1,…,N-1.
The spectrum of the time varying signal = the real
part of the Fourier transform of the signal, F(ω);
ω=2πν.
The spectrum of the image = real part of the
Fourier transform of the image = 2-D
generalization of 1-D Fourier transform, F(ωx,ωy)
ωx=2πνx; ωy=2πνy
Digital image processing
Chapter 3. Image sampling and quantization
Images of limited bandwidth
•
Limited bandwidth image = 2-D signal with finite spectral support:
F(νx, νy) = the Fourier transform of the image:


 
F  x , y    f x, y e
 j  x  x  j  y  y
e
 
 
dxdy    f x, y e

 j 2   x  x  y  y
dxdy.
 
|F(νx, νy)|
νy
F ( x , y )  0,| x |  x0 ,| y |  y0
νx0
νx
νy0
νx
-νx0
νx0
-νy0
-νy0
νy
The Fourier transform of the
limited spectrum image
The spectral support region
The spectrum of a limited bandwidth image and its spectral support
Digital image processing
Chapter 3. Image sampling and quantization
Two-dimensional sampling (1)
The common sampling grid = the uniformly spaced, rectangular grid:
y

g(x, y ) ( x, y)  
  ( x  mx, y  ny)
m   n  
x
y

x
Image sampling = read from the original, spatially continuous, brightness function f(x,y), only in the black
dots positions ( only where the grid allows):
 f ( x, y ), x  mx, y  ny
 f s ( x, y )  
,
0
,
otherwise

m, n  Z.

 f s ( x, y )  f ( x, y ) g (x, y ) ( x, y )  

 f (mx, ny ) ( x  mx, y  ny )
m   n  
Digital image processing
Chapter 3. Image sampling and quantization
Two-dimensional sampling (2)
• Question: How to choose the values Δx, Δy to achieve:
-the representation of the digital image by the min. number of samples,
-at (ideally) no loss of information?
(I. e.: for a perfectly uniform image, only 1 sample is enough to completely represent the image =>
sampling can be done with very large steps; on the opposite – if the brightness varies very sharply => very
many samples needed)
 The sampling intervals Δx, Δy needed to have no loss of information depend on the spatial frequency
content of the image.
 Sampling conditions for no information loss – derived by examining the spectrum of the image  by
performing the Fourier analysis:
f s ( x, y )  f ( x, y ) g ( x, y ) ( x, y )  Fourier tr ansform :
 
FS ( x , y )    f S ( x, y )  e  j 2  x  x  e
 j 2  y  y
 
 
dxdy    f ( x, y )  g ( x, y ) ( x, y )  e  j 2  x  x  e
 j 2  y  y
dxdy.
 
The sampling grid function g(Δx, Δy) is periodical with period (Δx, Δy) => can be expressed by its Fourier
l
series expansion:
k
j 2   x j 2   y

g ( x, y ) ( x, y )  

 a(k , l )  e
k   l  
x
e
y
,
where :
k
l
 j 2   x  j 2   y
1 1 x y
y
x  e
a (k , l ) 
    g ( x, y ) ( x, y )e
dxdy.
x y 0 0
Digital image processing
Chapter 3. Image sampling and quantization
Two-dimensional sampling (3)
Since:
1, if x  y  0
for ( x, y )  [0; x)  [0; y ), g ( x, y ) ( x, y )  
,
0
,
otherwise

1 1
 a(k , l ) 
 , k , l   ;  .
x y
Therefore the Fourier transform of fS is:
2 l  y 
2 k  x

j
j
  

 j 2  y  y
1
FS ( x , y )    f ( x, y )    
 e x  e y   e  j 2  x  x  e
dxdy

x


y
 
 k   l  



 


l 
k 

 j 2  x x    j 2  y  y   
1
y 
x   e


dxdy
 FS ( x , y ) 
      f ( x, y )  e
x  y      k   l  


   





 

l 
k 

 j 2  x x    j 2  y  y  
y 
x   e


f ( x, y )  e
dxdy
 FS ( x , y ) 
1
    
x  y k   l      
 FS ( x , y ) 



1
k
l 
   F  x  , y  .
x  y k   l   
x
y 
 The spectrum of the sampled image = the collection of an infinite number of scaled spectral replicas
of the spectrum of the original image, centered at multiples of spatial frequencies 1/Δx, 1/ Δy.
Digital image processing
Original image
Chapter 3. Image sampling and quantization
Original image spectrum – 3D
Original image spectrum – 2D
2-D rectangular sampling grid
Sampled image spectrum – 3D
Sampled image spectrum – 2D
Digital image processing
Chapter 3. Image sampling and quantization
Image reconstruction from its samples
y
1/x
ys
 xs  2 x 0 ,  ys  2 y 0
ys-y0
y0

2y0
2
1/y
2x0
xs-x0
x0
xs
1
2 x 0
,y 
1
2 y 0
1

,( , )  

H ( x , y )   ( xs ys ) x y
0,
otherwise
1
3
x 
xs
x
Fig.4 The sampled image spectrum
~
F ( x ,  y )  H ( x ,  y ) Fs ( x ,  y )  F ( x ,  y ) 
~
f  x, y   h  x , y   f s  x, y 
~
 f  x, y  



 f s mx, ny hx  mx, y  ny 
m   n  
Let us assume that the filtering region R is rectangular, at the middle distance between two spectral
replicas:
 ys

 xs
1
sin x xs  sin y ys
,


and



x
y
H ( x ,  y )   ( xs  ys )

2
2  hx, y  

x

y ys
xs
0,
otherwise


~
 f x, y  





sinx xs  m sin  y ys  n






f
m

x
,
n

y
h
x

m

x
,
y

n

y

f
m

x
,
n

y

  s
  s



x


m
 y ys  n
xs
m   n  
m   n  




Digital image processing
~
 f x, y  


Chapter 3. Image sampling and quantization

 f s mx, ny sincx xs  msincy ys  n, where sinca  
m   n  
sin a
a
Since the sinc function has infinite extent => it is impossible to implement in practice the ideal LPF
it is impossible to reconstruct in practice an image from its samples without error if we sample it
at the Nyquist rates.
Practical solution: sample the image at higher spatial frequencies + implement a real LPF (as close to
the ideal as possible).
1-D sinc function
2-D sinc function
Digital image processing
Chapter 3. Image sampling and quantization
The Nyquist rate. The aliasing. The fold-over frequencies
y
 xs  2 x 0 , ys  2 y 0
y0
2 y0
0
2 x0
0
The Moire effect
x0
x
Fig. 5 Aliasing – fold-over frequencies
Note: Aliasing may also appear in the reconstruction process,
due to the imperfections of the filter!
How to avoid aliasing if cannot increase the sampling frequencies?
By a LPF on the image applied prior to sampling!
“Jagged” boundaries
Digital image processing
Chapter 3. Image sampling and quantization
Non-rectangular sampling grids. Interlaced sampling grids n
n
νy
1/2
F(νx, νy)=1
2
1/ 2
νx
1/2
-1/2
2
1
-1 0 1 1
2
-2
3 m
-1
1
2
1
0
2 m
-1/2
2
2
1
x
x
-1
c) Interlaced grid G2
b) Rectangular grid G1
a) Image spectrum
0
x
1
x
1
1
d) The spectrum using G1
e) The spectrum using G2
Interlaced sampling
Optimal sampling = Karhunen-Loeve expansion:
f ( x, y)  

a
m ,n  0
m ,n
 m ,n
Digital image processing
Chapter 3. Image sampling and quantization
Image reconstruction from its samples in the real case
The question is: what to fill in the “interpolated” (new) dots?
Several interpolation methods are available; ideally – sinc function in the spatial domain; in
practice – simpler interpolation methods (i.e. approximations of LPFs).
Digital image processing
Chapter 3. Image sampling and quantization
Image interpolation filters:
The 1-D
interpolation
function
Graphical
representation
p(x)
The 2-D
interpolation
function
pa(x,y)=p(x)p(y)
Frequency
response
pa(1,2)
pa(1,0)
1/x
Rectangular
(zero-order
filter)
p0(x)
-x/2
0
x/2
1
x
p0 ( x ) p0 ( y )
  
  
sinc 1  sinc 2 
 2x 0 
 2y 0 
x
0
4x0
1/x
Triangular
(first order
filter)
p1(x)
1
 x
rect  
 x 
x
-x
0
x
1
 x
tri  
x  x 
x
1
p1 ( x ) p1 ( y )
      
sinc 1  sinc 2  
  2x 0   2y 0  
2
p0 ( x )  p0 ( x )
x
0
4x0
1
n-order filter
n=2, quadratic
n=3, cubic spline
pn(x)
0
x
p0 ( x) p0 ( x) pn ( x ) pn ( y )
n convolu@ii
      
sinc 1  sinc 2  
  2x 0   2y 0  
n 1
x
0
4x0
1
 x2 1
 ( x2  y2 ) 
exp
 2 2  2 exp
2
2

2
2 2 

1
Gaussian
pg(x)
0

exp  2 2 2 (12  22 )

x
x
0
2
1
Sinc
0
2x
x
1
 x
sinc 
 x 
x
1
 x  y
sinc  sinc 
xy  x   y 
  
  
rect  1  rect  2 
 2x 0 
 2y 0 
0
2x0
Digital image processing
Chapter 3. Image sampling and quantization
Image interpolation examples:
1. Rectangular (zero-order) filter, or nearest neighbour filter, or box filter:
1/x
-x/2
0
Original
x/2
x
Sampled
Reconstructed
Digital image processing
Chapter 3. Image sampling and quantization
Image interpolation examples:
2. Triangular (first-order) filter, or bilinear filter, or tent filter:
1/x
-x
0
Original
x
x
Sampled
Reconstructed
Digital image processing
Chapter 3. Image sampling and quantization
Image interpolation examples:
3. Cubic interpolation filter, or bicubic filter – begins to better approximate the sinc function:
0
x
2x
Original
Sampled
Reconstructed
Digital image processing
Chapter 3. Image sampling and quantization
Practical limitations in image sampling and reconstruction
Input
image
Scanning
system
aperture
ps(-x,-y)
g(x,y)
Ideal sampler
x,y
gs(x,y)
Analog display g~(x,y)
pa(-x,-y)
Real scanner model
Fig. 7 The block diagram of a real sampler & reconstruction (display) system
Pa(1,0)
Interpolation filter or
display system spectrum
xs/2
-xs/2
Sampled
image
spectrum
Reconstructed
image spectrum
Input image spectrum
Spectral losses
0
0
xs/2
-xs/2
Interpolation error
Fig. 8 The real effect of the interpolation
1
Digital image processing
Chapter 3. Image sampling and quantization
3. Image quantization
3.1. Overview
u
Quantizer
Quantizer’s
output
u’
rL
rk
t1
t2
tk
r2
tk+1
tL+1
Quantization
error
r1
Fig. 9 The quantizer’s transfer function
Digital image processing
Chapter 3. Image sampling and quantization
3.2. The uniform quantizer
The quantizer’s design:
• Denote the input brightness range:
u  lmin ; LMax 
• Let B – the number of bits of the quantizer => L=2B reconstruction levels
E.g. B=2 => L=4
• The expressions of the decision levels:
Uniform quantizer transfer function
t1  lmin ; t L 1  LMax
L
l
t
t
q  L 1 1 ,t k  t k 1  q  q  Max min
L
L
Reconstruction levels
t k  t k 1  t k 1  t k  constant  q
r4=224
r3=160
r2=96
• The expressions of the reconstruction levels:
r1=32
t t
q
rk  k k 1  rk  t k 
2
2
t1=0
t2=64
t3=128
Decision levels
t4=192
• Computation of the quantization error: for a given image of size M×N pixels,
U – non-quantized, and U’ – quantized => we estimate the MSE:
L t k 1
1 M 1N 1
2

  um, n  u' m, n    (u  rk ) 2 hlin,U (u)du
MN m  0 n  0
k 1 t k
t5=256
Digital image processing
Chapter 3. Image sampling and quantization
Examples of uniform quantization and the resulting errors:
B=1 => L=2
Non-quantized image
Uniform quantizer transfer function
Quantized image
Reconstruction levels
r2=192
r1=64
t1=0
t2=128
Decision levels
t3=256
Quantization error; MSE=36.2
1000
900
800
700
600
500
400
300
200
100
0
0
50
100
150
200
250
The histogram of the non-quantized image
Digital image processing
Chapter 3. Image sampling and quantization
Examples of uniform quantization and the resulting errors:
B=2 => L=4
Non-quantized image
Quantized image
Uniform quantizer transfer function
Reconstruction levels
r4=224
r3=160
r2=96
r1=32
t1=0
t2=64
t3=128
Decision levels
t4=192
t5=256
Quantization error; MSE=15
1000
900
800
700
600
500
400
300
200
100
0
0
50
100
150
200
250
The histogram of the non-quantized image
Digital image processing
Chapter 3. Image sampling and quantization
Examples of uniform quantization and the resulting errors:
B=3 => L=8; false contours present
Non-quantized image
Uniform quantizer transfer function
Quantized image
r8=240
Reconstruction levels
r7=208
r6=176
r5=144
r4=112
r3=80
r2=48
r1=16
t1=0
t2=32
t3=64
t4=96 t5=128 t6=160
Decision levels
t7=192
t8=224
t9=256
Quantization error; MSE=7.33
1000
900
800
700
600
500
400
300
200
100
0
0
50
100
150
200
250
The histogram of the non-quantized image
Digital image processing
Chapter 3. Image sampling and quantization
3.2. The optimal (MSE) quantizer (the Lloyd-Max quantizer)
L ti 1
t L1
    (u  ri ) 2 hu (u )du
e  E[(u  u' ) ]   (u  u' ) hu (u)du
2
2
i 1 ti
t1



 (tk  rk 1 ) 2  (tk  rk ) 2 hu (tk )  0
tk
t k 1

 2  (u  rk )hu (u )du  0 1  k  L
rk
t
k
t k 1
r r
t k  k k 1
2
 uhu (u)du
rk 
tk
t k 1
 E u | u  k 
 hu (u)du
tk
pu (u)  pu (tj ),
tj 
z k  t1
A
t k 1 
1
(t  t ),
2 j j 1
 [hu (u )]
1 / 3
du
 t1
t1
t L 1
 [hu (u )]
t1
1 / 3
du
t j  u  t j 1
Digital image processing
Chapter 3. Image sampling and quantization
3
1

12 L2
t

1/ 3
  [hu (u )] du 
 t

L 1
1
pu(u)
u
t1
t2
  (u   ) 2 

hu (u ) 
exp 
2

2
2
 2

1
2 
2

(
tj
tj+1
tL+1

hu (u ) ,or exp   u   
(Gaussian)
2
variance , - mean)
(Laplacian)
Digital image processing
Chapter 3. Image sampling and quantization
Examples of optimal quantization and the quantization error:
B=1 => L=2
Non-quantized image
Nivelele de reconstructie
Functia de transfer a cuantizorului optimal
Quantized image
r2=153
r1=24
t1=0
t2=89
Nivelele de decizie
t3=256
The quantization error;
MSE=19.5
1000
The evolution of MSE
in the optimization, starting
from the uniform quantizer
900
800
38
700
36
600
34
500
32
400
30
300
28
200
26
100
24
0
22
0
50
100
150
200
250
The non-quantized image histogram
20
18
1
2
3
4
5
6
7
8
Digital image processing
Chapter 3. Image sampling and quantization
Examples of optimal quantization and the quantization error:
B=2 => L=4
Non-quantized image
Nivelele de reconstructie
Functia de transfer a cuantizorului optimal
Quantized image
r4=181
r3=156
r2=115
r1=20
t1=0
t2=68
t3=136 t4=169
Nivelele de decizie
t5=256
The quantization error;
MSE=9.6
1000
The evolution of MSE
in the optimization, starting
from the uniform quantizer
900
800
15
700
600
14
500
13
400
300
12
200
11
100
0
10
0
50
100
150
200
250
The non-quantized image histogram
9
1
2
3
4
5
6
7
8
9
10
Digital image processing
Chapter 3. Image sampling and quantization
Examples of optimal quantization and the quantization error:
B=3 => L=8
Non-quantized image
Quantized image
Functia de transfer a cuantizorului optimal
Nivelele de reconstructie
r8=224
r7=181
r6=165
r5=147
r4=125
r3=101
r2=54
r1=14
t1=0
t2=34
t3=78
t4=113 t5=136 t6=156t7=173
Nivelele de decizie
t8=203
t9=256
The quantization error; The evolution of MSE
MSE=5
in the optimization, starting
from the uniform quantizer
1000
900
800
700
7.5
600
7
500
400
6.5
300
6
200
100
5.5
0
5
0
50
100
150
200
250
The non-quantized image histogram
4.5
0
2
4
6
8
10
12
14
Digital image processing
Chapter 3. Image sampling and quantization
3.3. The uniform quantizer = the optimal quantizer for the uniform grey level
distribution:
 1
, t1  u  t L1

hu (u )   t L1  t1
0 otherwise

(t k21  t k2 ) t k 1  t k
rk 

2(t k 1  t k )
2
tk 
q
t k 1  t k 1
2
t L 1  t1
,
L
t k  t k 1  t k 1  t k  constant  q
t k  t k 1  q ,
rk  t k 
q
2
q /2
1
q2
2

u du 
q  q/ 2
12

 2  2B , therefore SNR  10log 10 2 2B  6  B dB
2
u
Digital image processing
Chapter 3. Image sampling and quantization
3.4. Visual quantization methods
• In general – if B<6 (uniform quantization) or B<5 (optimal quantization) => the "contouring"
effect (i.e. false contours) appears in the quantized image.
• The false contours (“contouring”) = groups of neighbor pixels quantized to the same value <=>
regions of constant gray levels; the boundaries of these regions are the false contours.
• The false contours do not contribute significantly to the MSE, but are very disturbing for the
human eye => it is important to reduce the visibility of the quantization error, not only the MSQE.
 Solutions: visual quantization schemes, to hold quantization error below the level of visibility.
 Two main schemes: (a) contrast quantization; (b) pseudo-random noise quantization
Uniform quantization, B=4
Optimal quantization, B=4
Uniform quantization, B=6
Digital image processing
Chapter 3. Image sampling and quantization
3.4. Visual quantization methods
a. Contrast quantization
• The visual perception of the luminance is non-linear, but the visual perception of contrast is linear
 uniform quantization of the contrast is better than uniform quantization of the brightness
 contrast = ratio between the lightest and the darkest brightness in the spatial region
 just noticeable changes in contrast: 2% => 50 quantization levels needed  6 bits needed with a
uniform quantizer (or 4-5 bits needed with an optimal quantizer)
Brightness
u
f( )
brightnesscontrast
c
MMSE
quantizer
c’
-
f 1( )
contrast brightness
u’
c   ln(1  u ),0  u  1; typ.   6... 18,    / ln(1  )
or
c  u  ; typ.   1;   1 / 3
Digital image processing
Chapter 3. Image sampling and quantization
Examples of contrast quantization:
• For c=u1/3:
The transfer function of the contrast quantizer
250
1
0.9
200
0.8
Reconstruction levels
0.7
0.6
0.5
150
100
0.4
0.3
50
0.2
0.1
0
t2=3.9844
t1=0
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
t3=31.875
t4=107.5781
Decision levels
1
1000
900
800
700
600
500
400
300
200
100
0
0
50
100
150
200
250
t5=255
Digital image processing
Chapter 3. Image sampling and quantization
Examples of contrast quantization:
The transfer function of the contrast quantizer
• For the log transform:
250
5
4.5
200
Reconstruction levels
4
3.5
3
2.5
2
150
100
1.5
50
1
0.5
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0
t1=0
1
t2=46.0273
1000
900
800
700
600
500
400
300
200
100
0
0
50
100
150
200
250
t3=102.2733
t4=171.0068
Decision levels
t5=255
Digital image processing
Chapter 3. Image sampling and quantization
b. Pseudorandom noise quantization (“dither”)
u(m,n)
+
v(m,n)

K bits
v’(m,n)
quantizer
+
+
u’(m,n)

-
(m,n)
Uniformly distributed
pseudorandom noise,
[-A,A]
Uniform quantization, B=4
Prior to dither subtraction
Large dither amplitude
Small dither amplitude
Digital image processing
Chapter 3. Image sampling and quantization
Fig. 13
a
c
b
d
a.
b.
c.
d.
3 bits quantizer =>visible false contours;
8 bits image, with pseudo-random noise added in the range [-16,16];
the image from Figure b) quantized with a 3 bits quantizer
the result of subtracting the pseudo-random noise from the image in
Figure c)
Digital image processing
Chapter 3. Image sampling and quantization
Halftone images generation
Luminance
v(m,n)
0  u(m,n)  A +
Thresholding
v’
v’(m,n)
+
+
0  (m,n)  A
Halftone
display
A
Pseudorandom matrix
Fig.14 Digital generation of halftone images
 40
 80

H1  140

120
 20
10 
170 240 200 110

210 250 220 130

190 230 180 70 
100 160 50 30 
60
150
90
 52
 60

 68

76
H2  
132

200
212

204
44
4
12
36
28
20
84
92
140 148
228 236
252 244
196 188
Demo: http://markschulze.net/halftone/index.html
Fig.15 Halftone matrices
124 132 140 148 156
116 200 228 236 164
108 212 252 244 172

100 204 196 188 180
156 52 44 36 124

164 60
4
28 116
172 68 12 20 108

180 76 84 92 100
Digital image processing
Chapter 3. Image sampling and quantization
Fig.3.16
Digital image processing
Chapter 3. Image sampling and quantization
Color images quantization
RN’
T1’
T1
RN
Quantizer
GN
BN
Color space
transformation
T 2’
T2
Quantizer
Color space
inverse
transformation
T3’
T3
Quantizer
Fig.17 Color images quantization
GN’
BN’