3.2 Color Processing



Spectral information perceived by human eye
 visible wavelength ; 380-700nm
 Cone: good spatial resolution, low sensitivity for
daylight, 3 types for r g b ),
 Rod : High sensitivity, but b/w
Physiopsycological Phenomena
Image Interpretation
 human beings perceive thousands of color shades
and intensities, compared to only two-dozen
shades of gray.
Electromagnetic Spectrum
1
Spectral Reflectance
80
Vegetation
70
Soil
60
Clear River Water
Turbid River Water
50
40
30
20
10
0
0.4
0.6
0.8
1.0 1.2
1.4
1.6
1.8
2.0
2.2
2.4
2.6
Wavelength (µm)
Rods and Cones
http://hyperphysics.phy-astr.gsu.edu/hbase/vision/retina.html#c1
2
Rods and Cones
The Retina
The retina is a light-sensitive layer at the back of
the eye that covers about 65 percent of its interior
surface. Photosensitive cells called rods and cones
in the retina convert incident light energy into
signals that are carried to the brain by the optic
nerve. In the middle of the retina is a small dimple
called the fovea or fovea centralis. It is the center
of the eye's sharpest vision and the location of
most color perception.
"A thin layer (about 0.5 to 0.1mm thick) of light receptor cells covers the inner surface of the
choroid. The focused beam of light is absorbed via electrochemical reaction in this pinkish
multilayered structure. The human eye contains two kinds of photoreceptor cells; rods and
cones. Roughly 125 million of them are intermingled nonuniformly over the retina. The
ensemble of rods(each about 0.002 mm in diameter in some respects has the characteristics of
a high-speed, black and white film(such as Tri-X). It is exceedingly sensitive, performing in
light too dim for the cones to respond to, yet it is unable to distinquish color, and the images it
relays are not well defined."(Hecht)
"In contrast, the ensemble of 6 or 7 million cones (each about 0.006 mm in diameter) can be
imagined as a separate, but overlapping, low-speed color film. It performs in bright light,
giving detailed colored views, but is fairly insensitive at low light levels."(Hecht)“
http://hyperphysics.phy-astr.gsu.edu/hbase/vision/retina.html#c1
Rods and Cones



The retina contains two types of photoreceptors, rods and cones. The rods are
more numerous, some 120 million, and are more sensitive than the cones.
However, they are not sensitive to color. The 6 to 7 million cones provide the
eye's color sensitivity and they are much more concentrated in the central
yellow spot known as the macula. In the center of that region is the " fovea
centralis ", a 0.3 mm diameter rod-free area with very thin, densely packed
cones.
The experimental evidence suggests that among the cones there are three
different types of color reception. Response curves for the three types of cones
have been determined. Since the perception of color depends on the firing of
these three types of nerve cells, it follows that visible color can be mapped in
terms of three numbers called tristimulus values. Color perception has been
successfully modeled in terms of tristimulus values and mapped on the CIE
chromaticity diagram.
http://hyperphysics.phy-astr.gsu.edu/hbase/vision/rodcone.html
3
Cone Details




Current understanding is that the 6 to 7 million cones can be divided into "red"
cones (64%), "green" cones (32%), and "blue" cones (2%) based on measured
response curves. They provide the eye's color sensitivity. The green and red
cones are concentrated in the fovea centralis . The "blue" cones have the
highest sensitivity and are mostly found outside the fovea, leading to some
distinctions in the eye's blue perception.
The cones are less sensitive to light than the rods, as shown a typical day-night
comparison. The daylight vision (cone vision) adapts much more rapidly to
changing light levels, adjusting to a change like coming indoors out of sunlight
in a few seconds. Like all neurons, the cones fire to produce an electrical
impulse on the nerve fiber and then must reset to fire again. The light adaption
is thought to occur by adjusting this reset time.
The cones are responsible for all high resolution vision. The eye moves
continually to keep the light from the object of interest falling on the fovea
centralis where the bulk of the cones reside.
http://hyperphysics.phy-astr.gsu.edu/hbase/vision/rodcone.html
The Color-Sensitive Cones

In 1965 came experimental
confirmation of a long expected
result - there are three types of colorsensitive cones in the retina of the
human eye, corresponding roughly to
red, green, and blue sensitive
detectors.


Painstaking experiments have yielded
response curves for three different kind of
cones in the retina of the human eye. The
"green" and "red" cones are mostly packed
into the fovea centralis. By population,
about 64% of the cones are red-sensitive,
about 32% green sensitive, and about 2%
are blue sensitive. The "blue" cones have
the highest sensitivity and are mostly found
outside the fovea. The shapes of the curves
are obtained by measurement of the
absorption by the cones, but the relative
heights for the three types are set equal for
lack of detailed data. There are fewer blue
cones, but the blue sensitivity is comparable
to the others, so there must be some
boosting mechanism. In the final visual
perception, the three types seem to be
comparable, but the detailed process of
achieving this is not known.
http://hyperphysics.phyastr.gsu.edu/hbase/vision/colcon.html#c1
4
Color Representation


Color Mixing System
 All Color can be created by mixing 3 primary colors in
appropriate proportions
 Physical Approach
 Easy for machines to compose color
 Typical Primary Color
 Red, Green, Blue
 Additive color composite
for light
 Subtractive color composite
for pigments
Color Appearance System
 Describe Color Qualitatively using Color Code
 Easy for human beings to describe or control color
 Munsell Color System
 HSI
Color Mixing System




All Color can be created by mixing 3 primary
colors in appropriate proportions
Typical Primary Color
 Red, Green, Blue
 3 wavelengths are selected as that 1 color
cannot be created by mixing other 2 colors.
Additive color composite
for light
Subtractive color composite for pigments
5
Color Mixing System
Light and Pigment
Mixtures of Pigments
(Subtractive Primaries )
Mixtures of Light
(Additive Primaries )
Yellow
Green
Yellow
Cyan
White
Black
Blue
Red
Green
Red
Magenta
Blue
Cyan
Magenta
CIE RGB Color Matching Function


defined by CIE in 1931
color matching function is defined through color
matching experiment.
r(),
 g(), r() : ColorMatchingFunction
L e() : SpectralIrradiance
C = RR + GG + BB
C : Color Stimulus
R=
R,G,B: Tristimulus Value
780
380
r() L e() d
780
G=
380
g() Le() d
780
B=
380
b() L e() d
6
Color Matching Experiment and Function
Screen
L-R
L-G
L-B
CIE
Commission Internationale de l'Eclairage
the International Commission on Illumination
Red: 700nm
Green 546.1nm
Blue 435.8nm
L
Single Wavelength Light
Control
intensities of
Light of R,G, B
and make same
color with L
CIE XYZ Color System

Mathematically Derived from CIE RGB System
 RGB system includes negative in color matching function
 Derived virtual color matching Function is always positive
X = 0.49000R + 0.31000G + 0.20000B
Y = 0.17697R + 0.81240G + 0.01063B
Z=
0.01000G + 0.99000B
7
XYZ Color System

Mathematically Derived from CIE RGB System
 RGB system includes negative in color matching function
 Derived virtual color matching Function is always positive
r(),
 g(), r() : Color MatchingFunction
L e() : Spectral Irradiance
780
R=
380
r() L e() d
780
G=
380
g() Le() d
780
B=
380
b() L e() d
StimulusComponent–>ChromaticityDiagram
x = X/ ( X + Y+ Z)
y = Y/ ( X + Y+ Z)
z = Z / ( X+ Y+ Z )
x+y+z = 1
XYZ Color System
8
CIE Chromaticity Diagram
Actual Possible Color by
Standard computer displays
by RGB phosphors
http://www.biyee.net/v/cie_diagrams/
Tristimulus value
C
= (Lr/lr)R + (Lg/lg)G + (Lb/lb)B
= RR + GG + BB
C Color Stimulus
Lr ,Lg,Lb: Brightness
R,G,B: Tristimulus value
relative to white which has a certain amount of energy;
when C is white R:G:B=1:1:1
Physiopsycological Value using brightness coefficient as a unit
Stimulus Componet
r = R/(R+G+B)
g = G/(R+G+B)
b = B/(R+G+B)
r+g+b = 1
9
CIE RGB Color System I

defined by CIE in 1931
Commision Internationale de l’Eclairage
 (the International Commision on Illumination)


Primary Color
 Red
700 nm
 Green 546.1nm
 Blue
435.8nm
3.3 Color Composite



Allocate RGB to 3 gray scale images
True Color Composite
 reproduction of color image
• visible Red band
-> R
• visible Green band
-> G
• visible Blue band
-> B
False Color Composite
 Allocate invisible band to RGB or any other color
combination other than True Color
• Infrared band
-> R
• visible Red band
-> G
• visible Green band
-> B
 Any other band combination for R,G,B
10
Color Composite Example: LANDSAT
Band 1
Visible Blue
B
True Color
Band 2
Visible Green
G
R
Band 3
Band 4
Visible Red
Near IR
B
G
R
False Color
Band 5
Band 7
Middle IR
B
R
Middle IR
G
Any other Color Combination
Color Composite II

Natural Color Composite
 looks like a true color
image using infrared band
 allocate G to infrared that
is strongly reflected by
vegetation
• visible Red band
-> R
• Infrared band
-> G
• visible Green band
-> B
11
Color Appearance System


Munsell Color System by Prof. A.H. Munsell
Color is represented by
 Hue(color)
5 basic colors
 Saturation(Chroma;color purity)
0 to 16 or 30
Intensity(Value; brightness)
0 to 10
 defined as a psychological response
Munsell Color sample is available in the commercial market;
http://www.munsell.com



Munsell Color Cube
5RP (red-purple) is achieved at 5/26
http://www.adobe.com/
support/techguides/col
or/colormodels/munsell
.html
5RP 5/26
12
Conversion between RGB and HSI


Mathematical conversion between RGB Color cube and HSI model
Specify color in HSI


easy for human beings to get color
Color Enhancement Operation in HSI color space

RGB -> HSI -> Operation -> H’S’I’ -> R’G’B’


Data Fusion between RGB and B/W( Highreso, Radar … )
Overlay one image on an original image.
• To keep the color of original image, replace only I


SPOT : HSI Composite of 20m False Color image and 10m
Monochrome color image
• Spectral Info. 20m Multi-band false color H&S
• Texture Info
10m Monochrome band
I
IKONOS-Pan Sharpened ( Algorithm ? )
White
I
Conversion between
RGB and HSI
Double Hexagon
G
Y
B
M
C
R
Hexagonal Dipyrami
d
Black
H
S
Hexagonal Pyramid
http://www.couleur.org/index.php?page=transformations#HSI
13
Conversion RGB<->HIS
Example of Hexagonal Pyramid
Remote Sensing Note
3.4 Pseudo-color

Allocate different color to each gray level in one gray scale image

Enhance gray scale image using human being's good sensitivity for
colors
 distinguish small gray level difference
 grasp same gray scale level area
Same as index color images
 Easy to create and implement your own color palette

14
Pseudo-color Example
Creating Color Table


ENVI, ERMapper –Color Table is
in text format
 R G B for Level 1
 R G B for Level 2
 ….
Photoshop Simple binary RGB for
Level0, RGB for Level1,….
for(i=0;i<256;i++){
if( i<=lim1 ){
r[i]=0;
g[i]=(double)(255-0)/(lim1-lim0)*(i-lim0);
b[i]=255;
} else if( i<=lim2 ) {
r[i]=0;
g[i]=255;
b[i]=(double)(0-255)/(lim2-lim1)*(i-lim1)+255;
} else if( i<=lim3 ) {
r[i]=(double)(255-0)/(lim3-lim2)*(i-lim2);
15
RGB Color Image Display
Full Color Type
8 bits image
0
1
2
0R
40
8
D/A
8 bits
LUT for R
8 bits
LUT for G
0
1
2
0R
40
8
254 255
255 2550
CRT
8 bits
LUT for B
0 0R
1 40
2 8
254 255
255 2550
254 255
255 2550
RGB Color Image Display
Color Map Type
Color Code Image
or
Gray Scale Image
for pseude-color
Look Up Table
LUT( Color Table)
R G
B
8 bits
0 0 0
0 R
1 255 255 255
2 255 0
0 8 bits
G
CRT
3 - 8 bits
8 bits
B
254 0
255 0
0 200
0 255
D/A
16
4. Image Conversion
4.1 Math Operation: NDVI

Arithmetic Operation is possible because the image
is nothing more than numerical data
 + - * / ,log,. . .
 Single Band/Band Math
 B1/B2
 Contrast Enhancement, Radiometric Correction
 Change detection, Calc. Index( Vegetation Index,
etc. )
4. Image Conversion
4.1 Math Operation: NDVI


Arithmetic Operation
 + - * / ,log,. . .
 Single Band/Band Math
 Contrast Enhancement, Radiometric Correction
 Change detection, Calc. Index( Vegetaion Index,
etc. )
Logical Operation
 Operation for binary image( False or True )
 NOT, AND, OR, XOR
 Creat new regions
17
Scaling in Arithmetic Operation




Output Range e.g.
 8 bit image + - 8 bit image -> 9bit
 8 bit image */ 8 bit image -> 16bit
Scaling to integer image(usually 8 bit )
integer source image -> floating point calculation
-> scaling
-> integer result image
some functions create badly distributed data if we
apply linear scaling to the result
 logarithm Transformation
 NDVI vs. simple dividing VI
Vegetation Index
Vegetation absorbs Visible Red, reflects
Near Infrared
 ratio of NIR and VR
 NIR/VR
 log( NIR/VR )
 ( NIR - VR ) / ( NIR+VR)

18
Spectral Reflectance
80
Vegetation
70
Soil
60
Clear River Water
Turbid River Water
50
40
30
20
10
0
0.4
0.6
0.8
1.0 1.2
1.4
1.6
1.8
2.0
2.2
2.4
2.6
Wavelength (µm)
NDVI: Normalized Differential
Vegetation Index
NDVI = (NIR - VR)/(NIR+VR)
= tany'/x'
a
NDVI=1.0
forest
NIR = VR
NDVI = 0
bare land
y'
NIR
x'
= cos /4 sin /4
– sin /4 cos /4
y'
y' NIR – VR
tan  = x' = NIR
+ VR
VR
NIR
x'
q
NDVI= -1.0
VR
19
Bands used for NDVI

NDVI
= ( NIR - VR ) / ( NIR+VR)
NIR
VR
 Landsat MSS:
 Landsat TM
 SPOT XS:
 NOAA AVHRR
NDVI Example
Landsat B3: VR
B7
B4
3
2
B5
B3
2
1
NDVI Image
NDVI= ( NIR - VR ) / ( NIR+VR)
Landsat B4: NIR
Bareland
Good Forest
20
Scaling of NDVI






-1.0 < NDVI < 1.0
Save as Floating Point Value, or
Save as 8 bit Integer for easy handling, but
with an appropriate scaling factors.
Adjust -1.0 to 1.0 to 8 bit Integer 0-255
1.
0 < (NDVI+1.0)*100 < 200
2.
0 < (NDVI+1.0)*128 < 256
Some Application Software automatically
decide scaling factors, which makes it
impossible to retrieve original NDVI value.
We should always consider Scaling Factors
DN to Real Value

Always check scaling factor
 NDVI, Reflectance, Temperature, Radiance ….
If the data is not in floating point
21
Accuracy of NDVI Calculation
Dividing sometimes enhances noise
 NDVI
 If NIR and VR are near to 0, the
operation will enhances noise.
 Exclude low level area from calculation
 water, shade area

4.2 Logical Image & Operation



Indicate area by binary image
 False or True
Operation on binary Image
 Logical Operator( NOT,AND,OR,XOR )
Create New Regions
 Broad-leave Tree OR Conifer Tree -> Forest
 Bare-land AND Steep Area -> Slope Failure
Area
22
Logical Operator
AND
00 0
10 0
01 0
11 1
NOT
0
1
1
0
0: False
1: True
OR
00
10
01
11
0
1
1
1
XOR
00 0
10 1
01 1
11 0
Clear Image
Cursor Display
Logical Image



ROI
 Region of Interest
 Destination area for operations
 Get statistics
 Class definition
Mask Image ( e.g. Cloud Mask, Land Mask … )
 0 : False, not 0 ( usually 255 ):true
Bit Plane Image ( Nowadays not using so much )
 treat each bit in 8 bit image as a image.
23
4.3 Principal Component Analysis




Individual bands of a multispectral image are
commonly highly correlated.
Principal components transformation is a technique for
removing or reducing this spectral redundancy.
The principal component images are uncorrelated each
other.
Principal components are used for color composite to
visualize more than 3 band data using first 3 PC bands,
where almost of information is being concentrated.
Principal Component
255
Principal Component 2
Principal Component 1
Band 2
255
0
Band 1
24
Principal Component Example
PC1
PC2
PC3
PC4
PC5
PC6
PC7
PC1,2,3: RGB Composite
6. Spatial Filtering
Image Feature Extraction
 Noise suppression
 Image Enhancement

25
Fourier Series

Describe a signal as an addition of all harmonic
sin &
cos wave
a
x(t) =
0
2
+ a1 cos2t + a2 cos 4t + ..... + an cos 2nt + .
T
T
T
+b 1 sin2t + b 2 sin 4t + ..... + b n cos 2nt + ...
T
T
T

a
2nt
= 20 +  an cosT2nt + b n sin T
n =1
an = 2
T
bn = 2
T
T
0
T
0
x(t) cos 2nt dt
T
x(t) sin 2nt dt
T

an cos 2nt + b n sin 2nt = an2 + b n2 cos(2nt/T –  n)
T
T
bn
–1
n = tan a
n
Complex Fourier Series

– i ( ei  + e– i  )
( e i  + e– i  )
cos  =
, sin  =
2
2



x(t) =
A ei 2nt/T +
B e–i 2nt/T
n =0 n
n =0 n



an – i b n 1
=
2
T
an + i b n 1
Bn =
=T
2
T
x(t) e–i 2nt/Tdt
An =

x(t) =
0
T
x(t) ei 2nt/Tdt
0

c ei 2nt/T
n =
– n
cn = 1
T
T
x(t) e– i 2nt/T dt
0
26
Fourier Transform
Fourier Transform
Spatial Domain x(t)<-> Frequency Domain X(f)


x(t) =
–
X(f) ei2ftd f

x(t) e– i2ftdt
X(f) =
–
ei = cos + i sin
Power Spectrum
P(f) = X(f)
2
Discrete Fourier Transform
1-dim Discrete Fourier Transform

N–1
G(n) = 1  g(k) exp( – j2nk/N)
N k =0
N–1
g(k) =
 G(n) exp( j2nk/N)
n=0
2-dim Discrete Fourier Transform

M–1 N–1
G(m,n) = 1   g(k, l) exp( – j2mk/M) exp( – j2nl/N)
M N k = 0l = 0
M–1 N–1
g(k,l) =
 
m = 0n = 0
G(m,n) exp( j2mk/M) exp( j2nl/N)
27
FFT



Fast Fourier Transform
Reduce Number of Multiplications and additions
MNMN -> MN log2MN
M,N
MNMN
MNlog2MN
Direct FT
FFT
2
16
8
512
68,719,476,736
4,718,592
1024
1,099,511,627,776
20,971,520
Modulation

Contrast
 C1 = GLmax/GLmin
 C2 = GLmax - GLmin
= sGL
Modulation
 C3


M=
GL max – GLmin C1 – 1
=
GL max + GL min C 1 + 1
MTF ( Modulation Transfer Function )
 MTF = M’/M
 M:INPUT IMAGE, M’ OUTPUT IMAGE
28
S/N
SIGNAL TO NOISE RATIO
 S/N = 20 log( Ps/Pn) (dB)
 S/N = 20 log( As/An) (dB)
 P:Power
 A:Amplitude
 Example: Speech Signal
 Vs=5.0V, Vn=5mv
 S/N = 20 log(5.0/0.005)=60dB

Convolution
IMPULSE RESPONSE
PSF(POINT SPREAD FUNCTION)
INPUT
f(x,y)
LINEARITY
Addition
OUTPUT
g(x,y)
Shift Invariance
Same Response
+
g(x,y)=
PSF(x,y) * f(x,y)

=
=


–
–


–
–
PSF(x',y') f(x–x', y–y') dx'dy'
f(x',y') PSF(x–x', y–y') dx'dy'
29
Spatial Filter Operation
f(x,y)
PSF(x,y)
( FILTER )
f(x,y)
Filter
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
i
10x1/9+20x1/9+30x1/9
+50x1/9+60x1/9+70x1/9
+30x1/9+20x1/9+10x1/9
= 33
j
5
6
7
8
9
11
10
20
30
9
31
50
60
70
9
40
30
20
10
9
58
95
105
106
98
g(i,j) = 33

g(i, j) =
i + W/ 2

j+ W / 2

m = i – W / 2 n = j– W / 2
f( m,n ) PSF( i – m, j– n )
Filter Operation in Spatial Domain and Frequency Domain
g(x,y) = PSF(x,y) * f(x,y)
 G(u,v) = OTF(u,v) F(u,v)
 G, OTF, F: Fourier Transform of g,PSF,f
 OTF: Optical Transfer Function

30
Various Filters
Smoothing
Laplacian
1
1
1
-1
-1
-1
1/9 1
1
1
-1
1
-1
1
1
1
-1
-1
-1
Sharpening
1
-8
1
1/9 -8
37
-8
-8
1
1
Example
Smoothing
Original
Sharpening
31