CS 485 / 685
Computer Vision
Instructor: Mircea Nicolescu
Lecture 4
Alternative Color Spaces
• Various other color representations can be
computed from RGB
• This can be done for:
− Decorrelating the color channels:
− principal components
− Bringing color information to the fore:
− Hue, saturation and brightness
− Perceptual uniformity:
− CIELuv, CIELab, …
2
Alternative Color Spaces
•
•
•
•
•
•
•
•
•
•
•
•
RGB (CIE), RnGnBn (TV - National Television Standard Committee)
XYZ (CIE)
UVW (UCS de la CIE), U*V*W* (UCS modified by the CIE)
YUV, YIQ, YCbCr
YDbDr
DSH, HSV, HLS, IHS
Munsel color space (cylindrical representation)
CIELuv
CIELab
SMPTE-C RGB
YES (Xerox)
Kodak Photo CD, YCC, YPbPr, ...
3
Processing Strategy
Green
Blue
Red
T
Processing
Red
T-1
Green
Blue
4
Color Transformation - Examples
5
Skin Color
RGB
rg
r
g
6
Skin Detection
M. Jones and J. Rehg, Statistical Color Models with Application to
Skin Detection, International Journal of Computer Vision, 2002.
7
Image File Formats
• Many image formats adhere to the simple model
shown below (line by line, no breaks between lines)
• The header contains at least the width and height of
the image
• Most headers begin with a signature or “magic
number” – short sequence of bytes identifying the file
format
8
Common Image File Formats
•
•
•
•
•
•
•
GIF (Graphic Interchange Format)
PNG (Portable Network Graphics)
JPEG (Joint Photographic Experts Group)
TIFF (Tagged Image File Format)
PGM (Portable Gray Map)
FITS (Flexible Image Transport System)
…
9
PGM Format
• A popular format for grayscale images (8 bits/pixel)
• Closely-related formats are:
− PBM (Portable Bitmap) – for binary images (1 bit/pixel)
− PPM (Portable Pixelmap) – for color images (24 bits/pixel)
ASCII:
•
ASCII or binary (raw) storage:
Binary:
10
Image Filtering
f(x,y)
g(x,y)
filtering
filtering
11
Early Vision – One Image
• Classification of image operations
− Spatial domain methods
−
−
−
−
Point Processing Transformations
Area/Mask Processing Transformations
Frame Processing Transformations
Geometric Transformations
− Frequency domain methods
12
Image Filtering Methods
• Spatial Domain
• Frequency Domain (i.e., uses Fourier Transform)
13
Spatial Domain Methods
f(x,y)
g(x,y)
f(x,y)
g(x,y)
14
Point Processing Methods
• The most primitive, yet essential, image processing
operations
• Intensity transformations that convert an old pixel into a
new pixel based on some predefined function
• Operate on a pixel based solely on that pixel’s value
• Used primarily for image enhancement
Transformation function
15
Point Processing Methods
• Identity transformation
• Negative transformation
16
Point Processing Methods
• Contrast stretching /
compression
− Stretch gray-level
ranges where we
desire more
information
17
Point Processing Methods
• Thresholding
18
Point Processing Methods
• Intensity-level slicing
− Highlight a specific
range of gray-levels
only
− Similar to double
thresholding
19
Point Processing Methods
• Non-linear transformations
− We may use any function, provided that is gives a one-to-one or
many-to-one (i.e., single-valued) mapping.
• Logarithmic
− Useful for enhancing details in
the darker regions of the image
at the expense of detail in the
brighter regions.
20
Point Processing Methods
• Exponential
− The effect is the reverse of that
obtained with logarithmic
mapping.
21
Point Processing Methods
• Histogram equalization
− Low contrast images are usually mostly dark, mostly bright, or
mostly gray.
− Good contrast images exhibit a wide range of pixel values (i.e., no
single gray level dominates the image).
22
Point Processing Methods
• Histogram equalization
− The histogram of an image
(i.e., a plot of the gray-level
frequencies) provides
important information
regarding the contrast of an
image
− Histogram clustered at the
low end: dark image
− Histogram clustered at the
high end: bright image
− Histogram with a small
spread: low contrast image
− Histogram with wide
spread: high contrast image
23
Point Processing Methods
• Histogram equalization
− is a transformation that stretches the contrast by redistributing the
gray-level values uniformly
− is fully automatic compared to other contrast stretching techniques
24
Point Processing Methods
• Histogram equalization
25
Area Processing Methods
• Need to define:
(1) Area shape and size
(2) Operation
output image
26
Area Shape and Size
• Area shape is typically defined
using a rectangular mask
• Area size is determined by mask
size
• e.g., 3x3, 5x5, 7x7, …
• Mask size is an important
parameter
27
Operation
• Typically, linear
combinations of pixel values
− e.g., weigh pixel values
and add them together
• Different results can be
obtained using different
weights
− e.g., smoothing,
sharpening, edge
detection
mask
28
Example
Local image
neighborhood
Mask
Modified
image data
29
Common Linear Operations
• Correlation
• Convolution
30
Correlation
• A filtered image is generated as the center of
the mask visits every pixel in the input image
h(i,j)
nxn mask
g(i,j)
f(i,j)
filtered
image
31
Handling Pixels Close to Boundaries
wrap around
pad with zeroes
0 0 0 ……………………….0
0 0 0 ……………………….0
or
32
Correlation – Example
33
Geometric Interpretation of Correlation
• Suppose x and y are two n-dimensional vectors:
x  ( x1 , x2 ,..., xn )
y  ( y1 , y2 ,..., yn )
• The dot product of x with y is defined as:
x
x. y  x1 y1  x2 y2  ...  xn yn
using vector
notation:
x . y | x || y |cos( )
θ
y
• cos(θ) measures the similarity between x and y
34
Geometric Interpretation of Correlation
xy
x . y | x || y |cos( ) or cos( ) 
| x || y |
• Correlation generalizes the notion of dot product:
Normalized correlation (divide by lengths)
n
2
n
2
  h(k , l ) f (i  k , j  l )
N (i, j ) 
k 
n
2
[
n
2

n
n
k  l 
2
2
n
n
l 
2
2
n
2
h 2 (k , l )]1/ 2 [ 
n
2

f 2 (i  k , j  l )]1/ 2
n
n
k  l 
2
2
35
Normalized Correlation
• Measure the similarity between images or parts of
images
mask
=
36
Application: TV Remote Control
37
Application: TV Remote Control
38
Application: TV Remote Control
39
Application: TV Remote Control
40
Application: TV Remote Control
41
Application: TV Remote Control
42
Normalized Correlation
• Traditional correlation cannot handle changes due to:
• size
• orientation
• shape (e.g., deformable objects)
?
43
Convolution
• Same as correlation, except that the mask is flipped,
both horizontally and vertically
g (i, j ) 
n
2
n
2
n
2
n
2
  h(k , l ) f (i  k , j  l )    h(i  k , j  l ) f (k , l )
k 
n
n
l 
2
2
For symmetric masks (h(i,j)=h(-i,-j)),
convolution is equivalent to correlation
k 
n
n
l 
2
2
Notation:
h*f=f*h
44
Correlation/Convolution Examples
Correlation:
Convolution:
45
How to Choose the Mask Weights?
• Depends on the application
• Usually by sampling certain functions and their
derivatives
Gaussian
Good for
image smoothing
1st derivative
of Gaussian
2nd derivative
of Gaussian
Good for
image sharpening
46
Normalization of Mask Weights
• Sum of weights affects overall intensity of output
image
• Positive weights
− Normalize them so that they sum to one
• Both positive and negative weights
− Should sum to zero (but not always)
1/9
1/16
47