What is image processing?
Analog Image
CAMERA
x(n1,n2)
STORAGE
DIGITIZER
PROCESS
Sampling +
Quantization
x(t1,t2)
• Display
Sensor with RGB color filters
•
x(t1,t2) : ANALOG SIGNAL
• Perform analysis
• Reconstruct x(t1,t2)
x : real value
(t1,t2) : pair of real continuous space (time) variables
•
x(n1,n2) : DISCRETE SIGNAL (DIGITAL)
x : discrete (quantized) real or integer value
(n1,n2) : pair of integer indices
EEE 498/591- Real-Time DSP
1
Examples
• Sampled Black & White Photograph: x(n1,n2)
x (n1,n2) scalar indicating piel intensity at location (n1,n2)
For example: x = 0
Black
x=1
White
0<x<1
In-between
• Sampled color video/TV signal
xR(n1, n2, n3)
xG(n1, n2, n3)
xB(n1, n2, n3)
EEE 498/591- Real-Time DSP
2
Examples
EEE 498/591- Real-Time DSP
3
How do we process images?
•
Use DSP concepts as tools
•
Exploit visual perception properties
EEE 498/591- Real-Time DSP
4
Visual Perception
• We represent pixels as amplitude values (gray scale).
256 levels
1
0
128 levels
1
0
64 levels
1
0
32 levels
1
0
• How much to sample (quantize) the gray scale?
• Humans can distinguish in the order of 100 levels of
gray (about 40 to 100).
EEE 498/591- Real-Time DSP
5
Visual Perception: Examples
Original image, 8 bits per pixel
EEE 498/591- Real-Time DSP
Processed image, 0.35 bits per pixel
6
Visual Perception
RMSE = 8.5
EEE 498/591- Real-Time DSP
RMSE = 9.0
7
Image Enhancement
• Image Enhancement
 Objective: accentuate or improve appearance of features, for
subsequent analysis or display (possibly, but not necessarily
degraded by some phenomenon).
 Examples of features: edges, boundaries, dynamic range and
contrast.
 Examples of applications:
 TV: enhance image for viewer (image quality, intelligibility, visual
appearance).
 Preprocessing for machine identification.
 Enhancement is not necessarily needed because of degradation
but can be used possibly to remove degradation.
EEE 498/591- Real-Time DSP
8
Image Enhancement
Blurred or faint
edges

Sharpen faint or
blurred edges
EEE 498/591- Real-Time DSP
Low contrast or
dynamic range

Modify low
dynamic range
9
Noise
 Remove noise
Image Enhancement
• Types of distortions to correct:
 Blur (Blurring due to camera motion, defocusing, …)
 Noise (assumed to be additive often for simplicity, although not
necessarily the case).
 Contrast
 Blocking artifacts in block-based transform coders.
 Examples:
 Space photography
 Underwater photography
 Film grain noise
EEE 498/591- Real-Time DSP
10
Contrast and Dynamic Range Modification
• Contrast stretching
 Degradation is commonly due to poor lighting.
 Image probability distribution function (pdf) has narrow peak 
poor contrast.
p(x)
3500
3000
Number of
occurrences of
pixel with a
particular
intensity x
2500
2000
1500
1000
500
0
0
50
a
100
150
b
200
250
x
 Image intensities are clustered in a small region  available
dynamic range is not very well utilized.
EEE 498/591- Real-Time DSP
11
Contrast and Dynamic Range Modification
 Possible solution: increase overall dynamic range
  resulting image would appear to have a grater contrast
  expand the amplitudes from a to b to cover available intensity range.
p(xnew)
3500
3000
2500
2000
1500
1000
500
0
0
EEE 498/591- Real-Time DSP
50
100
150
12
200
250
xnew
Contrast and Dynamic Range Modification
 Idea: Gray scale or intensity level of an input image x(n1,n2) is
modified according to a specific transformation (function) f(·).
 Note: f(·) is usually constrained to be a monotonically non-decreasing
function of x  ensures that a pixel with higher intensity than another
will not became a pixel with a lower intensity in output image xnew.
 Typical stretching operator:
xnew
b’
slope 
slope 
xnew
a’
slope 
a
EEE 498/591- Real-Time DSP
b
L
x
13
x ,
0 xa



 x  a   a ,
a xb
 x  b    b  a   a , b  x  L

Contrast and Dynamic Range Modification
 Specific desired transformation depends on the application
Example: compensation of display non-linearity  most suitable
transformation depends on display non-linearity.
 In most applications, a good or suitable transformation can be
identified by computing and analyzing the histogram of the input
image to be enhanced.
– The histogram is a scaled version of the image pdf.
– The histogram gives pdf when scaled by the total number of pixels in the
image.
EEE 498/591- Real-Time DSP
14
Histogram Modification and Equalization
• Definition: The histogram of an image h(x) represents the
number of pixels that have a specific intensity x  number of
pixels as a function of intensity x.
h(x) = scaled version of pdf p(x)
normalized
pdf px  
h x 
 hn x 
T otalnumber of pixelsin image
L
Normalization ensures that
 h x   1,0  x  L
x 0
EEE 498/591- Real-Time DSP
n
15
Histogram Modification and Equalization
• Remarks:
 Histogram modification methods popular because computing and
modifying histogram of an image requires little computations.
 Experienced person can easily determine needed transformation
by analyzing histogram characteristics. But if too many images 
automatic method is desired.
 For typical natural images, the desired histogram has a maximum
around the middle of the dynamic range and decreases slowly as
the intensity increases or decreases.
hd(xnew)
hi(x)
Desired histogram of output image
xnew = f(x)
x
Lmin
 Problem: Determine f(·) such that houtput(xnew) = hd(xnew)
EEE 498/591- Real-Time DSP
16
Lmax
xnew
Histogram Modification and Equalization
• Histogram equalization: special case of histogram modification
where
hd(x) = constant
hd(x)
Redistribute pixels by
assigning pixels uniformly
to the given levels.
const
constant
Lmin
Lmax
xnew
T otalnumber of pixelsin image
Number of intensitylevels(Lmax  L min  1) in dynamicrange
 For a 256256 image, with 256 intensity levels:
2

256
const 
256
 256 pixels assigned to each level
 How can we do assignment?
EEE 498/591- Real-Time DSP
17
Histogram Modification and Equalization
• Method 1: Collect and redistribute pixels
Example: 128 pixels at 8 levels
16
h(x)
const 
35 35
16
21
7
21
7
1
1
0 1 2 3 4 5 6 7
L1 L2 L3 L4 L5 L6 L7 L8
x
ho(x)
16 16 16 16 16 16 16 16
0 1 2 3 4 5 6 7
L1 L2 L3 L4 L5 L6 L7 L8
EEE 498/591- Real-Time DSP
128
 16
8
18
x
Collect and redistribute pixels.
Get bundles of 16 regardless of
their position in image.
Histogram Modification and Equalization
 How to group them?
 One approach is to split bins  can choose pixels within a bin
randomly  poor results that tend to be noisy.
 Compute average values of neighboring pixels and match to closest
average  better, but still noisy.
 A better approach is to use a cumulative method.
EEE 498/591- Real-Time DSP
19
Histogram Modification and Equalization
• Cumulative method for histogram equalization
 Histogram equalization  desired histogram is constant at all
levels.
 Problem: Find transformation xo=f(xi) such that that houtput(xo) = const
hi(xi )
ho(xo )
xo = f(xi)
xmin
xmax
xi
const
0
L-1
xo
xmax
const 
 h x 
i  xmin
L
If normalized histogram 
EEE 498/591- Real-Time DSP
i

T otalnumber of pixels
Number of levels
xmax
1  uniform distribution

const



h
x

1
 i
L
i  xmin
20
Histogram Modification and Equalization
 Solution:
pi x  
 Compute input pdf
hi x 
xmax
 h x 
x  xmin
= normalized histogram
i
 Choose
f xi   F xi  
xi
 p  x   p x  x 
x 0
or xmin
i
 Why?
if y  F ( xi ) 
i
= cumulative probability distribution of xi
+ scaling needed
xi
 p x dx
i
i
i
 y uniformly distributed between (0,1)
0
 histogram uniformly distributed
 need also to scale y because y  (0,1) instead of (0,L-1) or (Lmin,Lmax)
EEE 498/591- Real-Time DSP
21
Histogram Modification and Equalization
 Note: if y  F ( xi ) 
xi
 p x dx
i
i
i
 y uniformly distributed between (0,1)
0
Proof:
Prob[ y  a]  Prob[ xi  F 1 (a)]  F (F 1 (a))  a
where 0  a  1  y is uniformlydistributed
EEE 498/591- Real-Time DSP
22
Histogram Modification and Equalization
 Since xi is a discrete variable, integral is replaced by summation:
y
xi
 p x only approximately uniformly distributed (because of discretization)
x  xmin
i
 ymin not necessarily 0 since
ymin  pxi  xmin   pi xmin 
 Scaling can be done as follows
y
xi
 p x 
x  xmin
i
 y  ymin

Lmax  Lmin   Lmin 
xo  Round
 1  ymin

y  ymin  xo  Lmin
y  1  xo  Lmax
EEE 498/591- Real-Time DSP
23
Histogram Modification and Equalization
•
•
If input and output range from 0 to L – procedure can be described
as follows:
Procedure: xk = k; k=0,…, L = input amplitude levels
yk; k=0,…, L = output amplitude levels
1. Compute the histogram of the image to be improved.
2. Normalize histogram;
 Normalize amplitudes so that the sum of all values is equal to one and
you have a pdf, pi(·).
3. Compute
 k

yk  R L pi xl 
where R
 - rounding operation
 l 0

4. Move bins in xk to locations in yk.
5. Scale yk to desired amplitude range (linear mapping).
EEE 498/591- Real-Time DSP
24
Histogram Modification and Equalization
• Example:
L=7
pi(xk)
0.25
0.20
0.15
0.10
0.05
0
Find yk such that xk maps into yk:
0.25
 k

 k

yk  RL pi xl   R7 pi xl 
 l 0

 l 0

0.2
0.15
0.14
1
0.1
2
3
4
0.07
5
0.05
6
0.04
xk; 0≤xk≤7
7
y4  R7(0.74  0.1)  R7(0.84)  6
y0  R7 pi x0   R7(0.14)  1
 1

y1  R7 pi xl   R7(0.14  0.25)  3
 l 0

y2  R7(0.39  0.2)  R7(0.59)  5
y5  R7(0.84  0.07)  R7(0.91)  7
y3  R7(0.59  0.15)  R7(0.74)  6
y7  R7(0.96  0.04)  R7(1.0)  7
EEE 498/591- Real-Time DSP
25
y6  R7(0.91 0.05)  R7(0.96)  7
Histogram Modification and Equalization
po(yk)
0.25
0.20
0.15
0.10
0.05
0
0.25
0.25
0.2
0.16
0.14
1
3
EEE 498/591- Real-Time DSP
5
6
yk; 0≤yk≤7
7
26
Histogram Modification and Equalization
Original Image
EEE 498/591- Real-Time DSP
Equalized Image
27
Thresholding
• Thresholding can be used to extract objects from image
(segmentation)
• Histogram statistics can be used to define single or multiple
thresholds to classify an image pixel-by-pixel.
• A simple approach:
 Bimodal histogram  set the threshold to gray value
corresponding to the deepest point in the histogram value.
 Multimodal histogram  set thresholds to correspond to points in
“valleys” of histogram.
 Classify every pixel f(x,y) by comparing its gray level to the
selected threshold.
1 if f ( x, y)  T
g x, y   
0 if f ( x, y)  T
EEE 498/591- Real-Time DSP
28
Pixel-based direct classification methods
Original image
Thresholded, T=12
EEE 498/591- Real-Time DSP
Image histogram
Thresholded, T=166
29
Thresholded, T=225
Thresholding
EEE 498/591- Real-Time DSP
30