Region labelling
Giving a region a name
Introduction

Region detection


Region description


isolated regions
properties of regions
Region labelling

identity of regions
Image Processing and Computer Vision: 6
2
Contents

Template matching






Rigid
Non-rigid templates
Graphical methods
Eigenimages
Statistical matching
Syntactical matching
Image Processing and Computer Vision: 6
3
Template matching

Define a template


a model of the object to be recognised
Define a measure of similarity

between template and similar sized image
region
Image Processing and Computer Vision: 6
4
Similarity
Measure dissimilarity between image f[i,j]
and template g[i,j]
Place template on image and compare
corresponding intensities
Need a measure of dissimilarity
f g
max
 
i, j R

f g
i, j R
  f g 
2
i, j R
Last is best....
Image Processing and Computer Vision: 6
5
Expanding
  f g    f
2
i, j R
i, jR
2

g
2
i , j R
2
 fg
i , j R
If f and g fixed
-fg a good measure of mismatch
fg a good measure of match
Compute match between template and image
with cross-correlation
Mi, j  
k m
l n
  gk,l f i  k, j  l
k   m l  n
Image Processing and Computer Vision: 6
6
g is constant, f varies and so influences M
Normalisation
k m l n
Ci, j  
  gk,l f i  k, j  l 
k   m l  n
k m l n

2


i

k,
j

l
f




k   m l  n

C is maximum where f and g are same.
Limitations



number of templates required
rotation and size changes
partial views
Image Processing and Computer Vision: 6
7
Input
Output
120
100
80
60
40
20
Position
426
401
376
351
326
301
276
251
226
201
176
151
126
76
51
101
Image Processing and Computer Vision: 6
26
0
1
Non-Normalised Correlation
Template
8
Flexible Templates


Shapes are seldom constant
Variation




in shape itself
in image of same shape
viewpoint
Non-rigid representations capture
variability
Image Processing and Computer Vision: 6
9
Structure


Flexible image structures
Linked by virtual springs
Image Processing and Computer Vision: 6
10
Recognition

Deform image structure


Move image structures


To equate model and image
To colocate model and image
Matching
Etotal  WinternalEinternal  WexternalEexternal
Image Processing and Computer Vision: 6
11
Learning the model


Accuracy of model determines success
Model

For each control point


average, variance of location
To be learnt with minimum external
variation

size, orientation, inconsistency of location
Image Processing and Computer Vision: 6
12
Parametric Models

Parametrically define the shape


straight line, circle, parabola, …
Update parameters to match model and
object
Image Processing and Computer Vision: 6
13
Example – Face tracking

Eyes and mouth



circles and parabolas
locations, sizes, orientations
Templates define image structures
Image Processing and Computer Vision: 6
14
Flexible templates,
EigenImages


Attempt to capture intrinsic variability of
data
Mathematical representation of variation
Image Processing and Computer Vision: 6
15
Mathematical Foundation


Take samples from a population
plot values of parameters on a scatter
diagram
Image Processing and Computer Vision: 6
16

Rotate axes:



one axis encodes most of information
other axis encodes remainder
Generalise to multiple dimensions
Image Processing and Computer Vision: 6
17
Images

Use




outline co-ordinates
image values
As the variables
Normalise as much variability
Image Processing and Computer Vision: 6
18
Hand Eigenimages
Image Processing and Computer Vision: 6
19
Hand Gestures
Image Processing and Computer Vision: 6
20
Range of Eigenimages
Image Processing and Computer Vision: 6
21
Face Eigenimages
Image Processing and Computer Vision: 6
22
Recognition



Retain n eigenvectors with largest
eigenvalues
Form dot product of these with image
data
Find nearest neighbour from training
set
Image Processing and Computer Vision: 6
23
Statistical Classification
Methods



Derive characteristic feature
measurements from image
Form a feature vector that identifies
object as belonging to a predefined
class
Need decision rules to make
classification
Image Processing and Computer Vision: 6
24
Linear Discriminant Analysis


Samples from different classes occupy
different regions of feature space
Can define a line separating them
Image Processing and Computer Vision: 6
25
Feature 2
Class A
Class B
Feature 1
Image Processing and Computer Vision: 6
26
Decision
d(X) = F2 - mF1 - c
d(X) > 0 for points in class A
d(X) = 0 for points on line
d(X) < 0 for points in class B
Image Processing and Computer Vision: 6
27
Nearest Neighbour Classifier

Assign the new
height
sample to the
population whose
centroid is
closest.
N

d   ui  f ij
j
i 1

2
?
basketball players
jockeys
weight
d
Image Processing and Computer Vision: 6
N
R
 m ind j
j 1
28
Most Likely

Incorporate range of possible class
values
pC A  
 x  x A

Image Processing and Computer Vision: 6
2
2
29
Bayesian Classifiers




Take population variation
into account
Assume prior probability of
observing class j is P(j)
e.g. 10% of population are
jockeys
Assume a conditional
probability distribution for
each feature, x, of each
population p(x|j).
Image Processing and Computer Vision: 6
height
?
basketball players
jockeys
weight
30



Multiply these curves
by P(j) to give
probability of a
measurement
belonging to each
class.
Divide by total
probability of
measuring x, to
normalise.
This gives the
probability of the
sample being from
each class.
p(x|1)
p
p(x|2)
x

   

 px| P 
P  j| x 
p x|  j P
j
N
j
j
j1
Image Processing and Computer Vision: 6
31
Syntactic Recognition

Objects’ structure (outline) can be
described linguistically


Primitive shape elements = words
Grammatically correct sentences = a valid
shape
Image Processing and Computer Vision: 6
32
Shape Grammar

A set of pattern primitives


A set of rules that define combinations
of primitives (sentences)


the grammar
A start symbol


terminal symbols
represents a valid object
Non-terminal symbols

represent substructures in the shape
Image Processing and Computer Vision: 6
33
Recognition


Grammar is generative
Recognition is degenerative

Recognition uses rules in reverse

Terminal symbols are rewritten until a valid
start symbol is attained
Image Processing and Computer Vision: 6
34
Chromosome Grammar
submedian chromosome  arm pair arm pair
arm pair  side arm pair
arm pair  arm pair side
arm pair  arm right part
arm pair  left part arm
left part  arm c
right part  c arm
Image Processing and Computer Vision: 6
35
Chromosome Grammar
arm  b arm
arm  arm b
arm  a
side  b side
side  side b
side  b
side  d
Image Processing and Computer Vision: 6
36
The Primitives
a
b
c
d
b
b
a
b
b
b
c
c
d
a
b
a
d
Image Processing and Computer Vision: 6
b
a
b
37
Example
b
b
a
b
b
a
d
b
c
c
d
a
b
a
b
b
Image Processing and Computer Vision: 6
38
dbabcbabdbabcbab
<side> b <arm> c <arm> <side> b <arm> c <arm>
<side> <arm> c <arm> <side> <arm> c <arm>
<side> <arm> <right part> <side> <arm> <right part>
<side> <arm pair> <side> <arm pair>
<arm pair> <arm pair>
<submedian chromosome>
Image Processing and Computer Vision: 6
39
Evaluation
Classification rate
 Confusion matrix

Image Processing and Computer Vision: 6
40
Classification Rate
How often does the classifier
get the correct answer?
 Selection of training and test
data must be carefully done.

Image Processing and Computer Vision: 6
41
Confusion matrix
C(i,j) = number of times
pattern i was recognised as
class j.
 Want off-diagonal elements to
be zero.

Image Processing and Computer Vision: 6
42
Summary




Template matching
Deformable templates
Flexible templates
Statistical classification
Image Processing and Computer Vision: 6
43