Image Similarity
Longin Jan Latecki
CIS Dept. Temple Univ., Philadelphia
latecki@temple.edu
Image Similarity
•Image based,
e.g., difference of values of corresponding pixels
•Histogram based
•Based on similarity of objects contained in images,
requires image segmentation
Mathematical Representation of Images
 An image is a 2D signal (light intensity) and can be
represented as a function f (x, y).
 coordinates (x, y) represent the spatial location of
point (x, y) that is called pixel (picture element)
 value of f (x, y) is the light intensity
called gray value (or gray level) of image f
Images are of two types: continuous and discrete
 A continuous image is a function of two variables,
that take values in a continuum.
E.g.: The intensity of a photographic image recorded on
a film is a 2D function f (x, y) of two real-valued
variables x and y.
A discrete image is a function of two variables,
that take values over a discrete set (an integer grid)
E.g.: The intensity of a discretized 320 x 240
photographic image is 2D function f (i, j) of
two integer-valued variables i and j.
Thus, f can be represented as a 2D matrix I[320,240]
A color image is usually represented with three matrices:
Red[320,240], Green[320,240], Blue[320,240]
Pixel based image similarity
Let f and g be two gray-value image functions.
d ( f , g) 
n
m
2
  f (i, j )  g (i, j )
i 1 j 1
e.g . :
  4 3 7  5 3 5



d   0 0 1 ,  0 0 0    1  4  1  1  16  23
 9 5 5  8 5 1



Let a and b bet two images of size w x h.
Let c be some image characteristics that assigns a number
to each image pixels, e.g., c(a,x,y) is the gray value of the pixel.
Pixel to pixel differences:
w
h
pd1(a, b)   | c(a, x, y )  c(b, x, y ) |
x 1 y 1
w
h
pd 2(a, b)   (c(a, x, y )  c(b, x, y )) 2
x 1 y 1
We can use statistical mean and variance to add stability to
pixel to pixel image difference:
d ( a, b)
spd (a, b)  wh
sd ( a, b)
1 w h
d ( a, b) 
d (a, b, x, y )

wh x 1 y 1
w h
1
2
s ( a, b) 
(
d
(
a
,
b
,
x
,
y
)

d
(
a
,
b
)
)

wh  1 x 1 y 1
2
d
d (a, b, x, y )  c(a, x, y )  c(b, x, y )
Let v(a) be a vector of all c(a,x,y) values assigned to all pixels
in the image a.
Image similarity can be expressed as normalized inner products
of such vectors. Since it yields maximum values for equal frames,
a possible disparity measure is
v(a)  v(b)
vd (a, b)  1 
v(a)  v(b)
Image histogram is a vector
If f:[1, n]x[1, m]  [0, 255] is a gray value image,
then H(f): [0, 255]  [0, n*m] is its histogram,
where H(f)(k) is the number of pixels (i, j) such that
F(i, j)=k
Similar images have similar histograms
Warning: Different images can have similar histograms
d H ( f ), H ( g )  
255
 H ( f )(i)  H ( g )(i)
i 0
2
Image Histogram
(3, 8, 5)
Hr
Hg
Hb
Histograms
Histogram Processing
4
5
0
3
1
5
1
Number of Pixels
1
gray level
Histogram-based image similarity
Let c be some image characteristics and h(a) its histogram
for image a with k histogram bins.
k
hd1(a, b)   | hi (a )  hi (b) |
i 1
k
hd 2(a, b)   (hi (a )  hi (b)) 2
i 1
(hi (a )  hi (b)) 2
hds (a, b)  
i 1 max( hi ( a ), hi (b))
k
Homework 5
• Implement in Matlab a simple image search
engine (no GUI needed).
Simply compare the performance of at least
two image distances on a small set of
images.