Image Formation
•
•
Image Formation occurs when a sensor registers
radiation.
Mathematical models of image formation:
1.
2.
3.
4.
5.
6.
Image function model
Geometrical model
Radiometrical model
Color model
Spatial Frequency model
Digitizing model
E.G.M. Petrakis
Image Formation
1
E.G.M. Petrakis
Image Formation
2
1. Image Function
• Mathematical representation of a (digital)
image.
– Relates to digitization: conversion from
continuous signal to discrete function
• Black & White image f x   f x, y   d




• Color image f x    f R x , fG x , f B x 
• Multispectral image f = (f1, f2, …, fn)
E.G.M. Petrakis
Image Formation
3
2. Geometrical Model
• Determines where in the image plane the projection of a
point will be located.
– the projected image is inverted
– (x,y,z) is projected on (x’,y’)
– f: focal length
E.G.M. Petrakis
Image Formation
4
• Avoid inversion by assuming that the image
plane is in front of the center of projection
– done automatically by cameras or by the human
brain
• Apply Euclidean geometry
– x’ = x f /z and y’ = y f/z
– depth z is lost !
E.G.M. Petrakis
Image Formation
5
Depth Computation
• Acquire a pair of images of the same scene using
two cameras (or two images by a moving camera)
• Two identical cameras separated in the x direction
by a baseline distance b
• The image planes are coplanar
E.G.M. Petrakis
Image Formation
6
• A point is projected at two different
positions on the two camera planes
– their displacement is called “disparity”
bf
z '
'
xl  xr
E.G.M. Petrakis
Image Formation
7
• In certain systems (human eyes) the optical axes of
the cameras intersect in space
– for any angle there is a surface in space corresponding
to d = 0.
– the disparities may be d = 0, d < 0 or d > 0.
E.G.M. Petrakis
Image Formation
8
• Epipolar constraint: even if the cameras are
in arbitrary positions and orientation the
projections lie on the intersection of camera
- epipolar planes
E.G.M. Petrakis
Image Formation
9
• Correspondence problem: detection of
conjugate pairs in stereo images:
– for each point in the left image find the
corresponding point in the right image
– measure the similarity between points
– the points to be matched should be distinctly
different from their surrounding points
– both region and edge features can be used in
stereo matching
– the epipolar constraint limits the search space
for finding conjugate pairs.
E.G.M. Petrakis
Image Formation
10
3. Radiometrical Model
• Measures the intensity of the reflected light at a
point (x’,y’) of the image plane
– it is determined by the physics of imaging
• The proper term of image intensity is image
irradiance but
– intensity, brightness, gray value are also used
• Image irradiance is the power per unit area of
radiant energy falling into the image plane
– Irradiance is incoming energy
– Radiance is outgoing energy (from reflecting surface)
E.G.M. Petrakis
Image Formation
11
•
•
The irradiance at point (x’,y’) of the image plane
depends on the amount of energy radiated by
points (x,y,z) in the scene
Two factors determine the radiance emitted by a
patch of scene surface:
1. The illumination falling on a surface (depends in its
position relative to the distribution of light)
2. The fraction of incident illumination reflected by the
surface (depends on surface properties e.g., dull, flat,
mirror-like etc.)
•
E.G.M. Petrakis
The reflectance of a surface is given by the Bi-directional
Reflectance Distribution Function (BRDF)
Image Formation
12
d 2
Wallts
• Scene Radiance L 
2
dA

cos


d

m
 steradiar
– Φ: light energy flux
– Α: area of source
– θ: angle (surface normal & direction of emission)
– dω: incremental solid angle
E.G.M. Petrakis
Image Formation
13
• Image irradiance
2
1D
4
E    cos   L
4  f p 
• Ideally, an imaging device should be
calibrated so that the variation in sensitivity
as a function of a is removed.
E.G.M. Petrakis
Image Formation
14
4. Color Model
• Visible light is an electromagnetic wave in
the 400nm – 700nm range
• The light we see is combination of many
wavelengths
– spectra: the profile below
E.G.M. Petrakis
Image Formation
15
• Each neuron on the retina is either a “rod” or a
“cone” (rods are not sensitive to color).
• Cons come in 3 types: red, green, blue
– each responds differently to various frequencies of
light.
• Spectral response functions of cones:
E.G.M. Petrakis
Image Formation
16
• The color signal to the brain is obtained by adding
the responses of the 3 cones
– the color signal consists of 3 numbers.
– R,G, B sensors filter the scene radiance E(λ).
– each sensor has a different spectral response S(λ) .
E.G.M. Petrakis
Image Formation
17
• CIE primaries: this figure shows the amounts of
the 3 primaries needed to match all the
wavelengths of the visible spectrum
– the negative value indicates that some colors cannot be
exactly produced by adding up the 3 primaries.
E.G.M. Petrakis
Image Formation
18
CIE XYZ
• Based on the CIE primaries
– negative values are transformed to positive
– chromaticity values x=X/(X+Y+Z), y=Y/(X+Y+Z),
z=Z/(X+Y+Z)
– x+y+z=1: two values represent all colors
E.G.M. Petrakis
Image Formation
19
Chromaticity Diagram
• Visible colors: points
in the bell
• Non-visible colors
outside the bell
• Primaries at edges
• A white point at centre
• Saturated colors along
the radii from edge
E.G.M. Petrakis
Image Formation
20
Color Representation
• Several methods
– Hardware-oriented: defined to properties of
devices (TV, printers) that reproduce colors
(RGB, CMY etc.)
– User-Oriented: based on human perception of
colors (HIS, L*u*v etc.)
– Colorimetric (CIE), Physiological (CIE XYZ,
RGB), Psychological (HIS, L*u*v etc)
E.G.M. Petrakis
Image Formation
21
RGB Color Space
• The most popular
hardware oriented scheme
• The colors form a unit
cube
– r = R/(R+G+B)
– g = G/(R+G+B)
– b = B/(R+G+B)
• RGB is good for
acquisition and display but
not for the perception of
colors
E.G.M. Petrakis
Image Formation
22
CMY Color Space
• Cyan, Magenta, Yellow are complements of
Red, Green, Blue
• Obtained by subtracting light from white
• For color printing
• Conversion from RGB to CMY
–R=1–C
–G=1–M
– B=1–Y
E.G.M. Petrakis
Image Formation
23
Munsell Color Space
• Represented in
cylindrical coordinates
based on
– Brightness: vertical
axis
– Hue: angular
displacement
– Saturation: cylindrical
radius
E.G.M. Petrakis
Image Formation
24
Color Definitions
• Brightness: intensity of color, average intensity
over all wavelengths
• Hue: is roughly proportional to the average
wavelength of the color percept
• Saturation: amount of white light in color
– highly saturated colors have no white
– deep red has S=1, pinks have S 0
• P = SH+(1–S)W: Think of a color P as an additive
mixture W and H where S controls the proportions
of W and H
E.G.M. Petrakis
Image Formation
25
HIS Color Space
• Represented as a
double cone
– Intensity: the main
axis (white at the top,
black at the bottom)
– Hue: angle around the
axis
– Saturation: distance
from axis
– Saturated colors on
maximal circles
E.G.M. Petrakis
Image Formation
26
HSV Color Space
• Similar to HIS
1
V  ( R G  B)
3
– Value
– Hue
H  cos1
– Saturation
2 R G  B
2 ( R G ) 2 ( R  B )( G  B )
S  1  RG3 B min(R, G, B)
– H = undefined for S = 0
– H = 360 – H if B/V > G/V
E.G.M. Petrakis
Image Formation
27
Color Models for Video (YIQ)
• YIQ is used for color TV broadcasting
Y  0.299 0.587 0.114   R 
 I   0.596  0.275  0.321 G 
  
 
Q  0.212  0.528 0.311   B 
E.G.M. Petrakis
Image Formation
28
4. Spatial Frequency Model
• Describe spatial variations in the frequency
domain of the Fourier Transform:
1
F u, v  
M N

 m u nv 
f m, nexp j 2 
 

N 
M
m 0 n 0

M 1 N 1

 m u nu 
f m, n   F u, v exp j 2 


N 
M
u 0 v 0

M 1 N 1
0  m  M  1,0  n  N  1
0  u  M  1,0  v  M  1
E.G.M. Petrakis
Image Formation
29
• f(m,n): linear combination of periodic waveforms
–
–
–
–
exp{j2π(ux + vy)}
F(u,v): weight factor of frequency u,v
High u,v  image detail (edges, points etc.)
Low u,v  no detail, smooth areas
e
E.G.M. Petrakis
Image Formation
30
Fourier Transform Pairs
E.G.M. Petrakis
Image Formation
31
Fourier Transform Pairs (2)
E.G.M. Petrakis
Image Formation
32
Sinc(x,y)
E.G.M. Petrakis
Image Formation
33
(a)
Original
image
(b)
Edge
Enhanced
image
(c)
Smoothed
image
Fourier
Transform
of (a)
Fourier
Transform
of (b)
Fourier
Transform
of (c)
E.G.M. Petrakis
Image Formation
34
Convolution
• Convolution of f and g:
h  x, y   f  g 
 f (a, b) g ( x  a, y  b)dadb
  a ,b
• Invert g by 1800, pass pass g over f and
compute h on each point of f
• Theorem: F  f g   F  f F g 
E.G.M. Petrakis
Image Formation
35
6. Digitizing Model
• Digitization: Conversion of continuous
signals to discrete.
– f(x,y)  f(m,n) , 0<= m <= M-1,0<=n<=N-1
– f(m,n) = k (intensity value) 0 <= k <= K-1.
– f(m,n): samples taken at equal intervals.
• Perfect sampling: It is possible to
reconstruct f(x,y) from f(m,n)
– K,m,n must be large enough
E.G.M. Petrakis
Image Formation
36
Image Sampling
• Multiply f(x,y) by com bx, y  


  x  m, y  n
m   n  
Sampling
function
E.G.M. Petrakis
Image Formation
37
• One way to reconstruct an image from its
samples f(kT) would be to interpolate
suitably between the samples
• Consider one dimensional signals
f x  

 f k  T  g x  kT 
k  
– in the frequency domain
G   
2n 
F   
F  


T n 
T 
– g(x-kT) interpolation function
– T: sampling period
E.G.M. Petrakis
Image Formation
38
F(u) for 1D band limited function
fc: max frequency
Non-overlapping copies of F(u)
Overlapping copies of F(u)
E.G.M. Petrakis
Image Formation
39
F(u,v) 2D band-limited function
Non-overlapping copies of F(u,v)
E.G.M. Petrakis
Image Formation
40
• Select G(w) that isolates F(w) from its
samples
T ,   2f c
G    
0, otherwise
• Whittaker-Kotelnikov-Shannon theorem:
f(x) can be reconstructed if the time distance
between the samples is at least 1/2f
– 2fc: sampling rate
– If the signal is not band-limited we have
aliasing (interference from high frequencies)
– Smooth before sampling
E.G.M. Petrakis
Image Formation
41
E.G.M. Petrakis
Image Formation
42
E.G.M. Petrakis
Image Formation
43
K
E.G.M. Petrakis
Image Formation
44