Machine Vision for Robots
Modeling Cameras
• Processing Images
– Geometric/Moment based Features
– Other Features
Histograms and Thresholding
 Histogram: graphical presentation of the
frequency count of the occurrence of each
intensity in an image
 Magnitude of histogram at a specific pixel
value, hist(g), is the probability of the gray
value, g, occurring in any picture element in the
frame
Histograms and Thresholding
 Histogram:
 determine if image has sufficient contrast
 equalize image histograms to perform operations
(e.g. subtraction)
 determine appropriate threshold values
Histograms and Thresholding
 Threshold:
 Segmentation of the image (classify each pixel)
according to gray value – I.e. for threshold, T
 Puts pixel into the set of background Sb or foreground
Sf pixels
 binary segmentation of the image
Connect Component Analysis
 Identify individual components (blobs) within
the image:
 Input: a binary image with objects having value 1
and background having value 0.
 Output: an image with each region having value
equal to the region label (and the background
having value 0).
C
• 4-connected vs. 8-connected
• A is not connected to C in 4-connectivity
• A is connected to C in 8-connectivity
D
B
A
Connect Component Algorithm
 One Algorithm (4-connectivity):
 Pass 1:
 If A==1 and D==1 then EqualLabels (A,D)
 If A==1 and B==1 then EqualLabels(A,B)
 If A==1 and Both(B==1) and (D==1) then

If (Label(B) != Label(D) then EqualLabels(B,D)
 Pass 2:
 Relabel each pixel within each equivalence class
C
B
D
A
In Matlab™ the routine bwlabel performs
a connected component analysis
Geometric Properties
 Assume: binary image with ONE object having
value 1 and background having value 0.
 Moment definition:
Geometric Properties
 Assume binary image with ONE object having
value 1 and background having value 0.
 Moment definition:
 Note m00 is the area
Geometric Properties
 Center of mass: point
such that if all the
object’s mass were concentrated at that point,
the first moments would not change
Geometric Properties
 Center of mass: point
such that if all the
object’s mass were concentrated at that point,
the first moments would not change
 Note m00 is the area
 The zero and first moments give the centroid
Geometric Properties
 Centralized Moment definition:
 Note m00 is still the area but the first centralized
moments are now zero
Geometric Properties
 Orientation: axis that passes through the object
such that the second moment of the object
about that axis is minimal
 Or

where
= minimum distance from pixel (c,r) to the line
 To solve?
 Parameterize line with (r, q)
 Compute partial derivatives of w.r.t. (r, q)
 Find minima by setting partial derivatives to zero
Geometric Properties
 Line parameterization
 Normal to line is
 Perpendicular distance from line to origin is
 Now our task is to find:
Geometric Properties
 Derivative
 Set to zero
 Defines a line through centroid
 shift origin
 Line is now
Geometric Properties
 Derivative, set to zero
 Solve (algebra…)
 Or, using double angle formulas
Geometric Properties
 Principal axes from second moments
 Orientation of principal axis w.r.t. image x-axis
 Or using double angle formula we can also write:
Geometric Properties
 Alternate point of view
 Think ellipses again
 Eigenvectors along major/minor axes
 Eigenvalues give length of major/minor axes
Geometric Properties
 Area, Centroid
 Principal axes from second moments
(centralized)
 Orientation
 Major/ Minor axis length
Geometric Properties
 Normalized moments
 Moment Invariants
 Functions whose values are approximately
constant under various transformations
 E.g. the zeroth moment does not change under
rotation about the optical axis (i.e. rotation in the
image plane)
Geometric Properties
 Moment Invariants
Image Features
 Measured quantities – can be used to
“represent” the object




Centroid
Best fit ellipse (centroid + axes)
Area
Compactness
 Perimeter2/(4p Area) ¸ 1
 Eccentricity
 (major axis length)/(minor axis length)
 Moment Invariants
 # Holes
Features & Patterns
 We can represent the features associated with
an object as a vector
area
compactness
Mi4
Mi5
Features & Patterns
 We can represent the features associated with
an object as a vector
 A set of samples, described by their statistical
properties with respect to a number of features
defines a pattern.
Features & Patterns
 From a set of samples we can find the mean
value of each feature
 The covariance is
Features & Patterns
 The covariance is
Features & Patterns
 The statistics (mean & covariance) can be used
to classify an object.
 Given a set of samples for objects (O1,…,OM)
we can compute for each object class
 For a new object, onew, we measure the
features
Features & Patterns
 Pattern classification:
 Which object (O1,…,OM) is this new object closest
too?
 How do we measure distance?
 Euclidean:
Example: Euclidean Distance
 Two features measured for two objects
Feature 2
O1
Feature 1
O2
Example: Euclidean Distance
 Compute statistics
Feature 2
O1
Feature 1
O2
Example: Euclidean Distance
 Compute statistics
Feature 2
O1
Feature 1
O2
Example: Euclidean Distance
 Measure features for new object
Feature 2
O1
O2
Measured
feature values
for new object
Feature 1
Example: Euclidean Distance
 Compute distance
Feature 2
O1
O2
Euclidean
distance to
mean of
features for
Objects 1 and 2
Feature 1
Example: Euclidean Distance
 Statistically – which object is more likely?
Feature 2
O1
Feature 1
O2
Features & Patterns
 Pattern classification:
 How do we measure distance taking into account
statistics?
 Inversely weight distance with variance
 Mahalanobis distance:
Weights the distance
by the inverse of the
(co)variance
Example: Mahalanobis Distance
 Two features measured for two objects
Feature 2
O1
Feature 1
O2
Example: Mahalanobis Distance
 Compute statistics
Feature 2
O1
O2
Feature 1
Example: Mahalanobis Distance
 Use statistics to measure disance
Feature 2
O1
O2
Feature 1
Features & Patterns
 For M objects we need to store the mean of the
feature vector and the covariance
 For a new object, onew, we measure the
features and compute the distance to each
object class (M distance computations) and find
the minimum