Outline
• Classification
Pattern Recognition
• It is natural and desirable that we should seek
to design and build machines that can
recognize patterns
–
–
–
–
Automated speech recognition
Fingerprint identification
Optical character recognition
DNA sequence identification
5/29/2016
Visual Perception Modeling
2
An Example
• Fish sorting
– A fish-packing plant wants to automate the
process of sorting incoming fishes on a conveyor
belt according to species using optical sensing
• Separate sea bass from salmon
– Physical differences between sea bass and
salmon
• Length, lightness, width, number and shape of fins,
position of mouth .....
• Noise
5/29/2016
Visual Perception Modeling
3
Components of A Recognition System
• A Typical pattern recognition system
–
–
–
–
–
–
Input
Sensing
Segmentation
Feature extraction
Classification
Post-processing
5/29/2016
Visual Perception Modeling
4
Sensing
• The input to the recognition system
– Digital cameras
– Lasers
– Some kind of a transducer
• Characteristics and limitations of the transducer
–
–
–
–
–
Its bandwidth
Resolution
Sensitivity
Distortion
Signal-to-noise ratio
5/29/2016
Visual Perception Modeling
5
Segmentation and Grouping
• Segmentation
– To segment out the object we are interested in
from all other objects
– This is a very difficult problem
• Grouping
– Group pixels that correspond to an object
together
– Perceptual organization
– Figure-ground segregation
5/29/2016
Visual Perception Modeling
6
Feature Extraction
• Features
– Some characteristics of the input that can
separate objects in different types very
effectively
– Invariant features
• Translation invariance
• Rotation invariance
• Scale invariance
– Occlusion
– Projective distortion
5/29/2016
Visual Perception Modeling
7
Feature Extraction – cont.
• Deformation
– Domain specific highly complex transformations
• Feature extraction is domain specific
– That is, good features depend on what you want
to do
• Feature selection
– Techniques to select the best features among a set
of features
5/29/2016
Visual Perception Modeling
8
Training
• Design or training samples
– One needs to make measurements of each pattern
class
– This is often done by specifying examples
5/29/2016
Visual Perception Modeling
9
Classification
• Definition
– Given a set of classes, represented by the
corresponding feature values, assign the new
input object to a category
– The degree of the difficulty depends on the
variability in the feature values for objects in the
same object with respect to the difference
between feature values for objects in different
categories
5/29/2016
Visual Perception Modeling
10
Post-Processing
• Post-processing makes recommendations or
takes actions based on the output from the
classifier
– Error rate
– Risk
• Cost of a mistake
– Context
– Multiple classifiers
5/29/2016
Visual Perception Modeling
11
Design Cycles
• Data collection
• Feature choice
– Prior knowledge
• Model choice
• Training
• Evaluation
– Over-fitting
5/29/2016
Visual Perception Modeling
12
Computational Complexity
• Pattern recognition problems can be “solved”
using algorithms that are highly impractical
– Polynomial vs. exponential
• The computational resources needed and
computational complexity are of practical
importance
– The system may have to make a decision within a
time interval
5/29/2016
Visual Perception Modeling
13
Learning and Adaptation
• Supervised learning
– There is a teacher which provides a category
label for each pattern in a training set
• Unsupervised learning
– There is no explicit teacher
– The system forms clusters of the input patterns
• Reinforcement learning
– Some feedback information about the system’s
performance
5/29/2016
Visual Perception Modeling
14
Neural Networks
• Based on the connections in the brain
5/29/2016
Visual Perception Modeling
15
Neural Networks – cont.
5/29/2016
Visual Perception Modeling
16
Statistical Pattern Recognition
• Given a set of features and cost associated
with each decision, classification is to decide
a decision boundary in the feature space or
make a decision rule
– We want to minimize the total cost
• Generalization
– The classifier is designed to suggest actions for
novel patterns
5/29/2016
Visual Perception Modeling
17
Pattern Theory
• Pattern theory proposed by Ulf Grenander
– The analysis of the patterns generated by the
world in any modality, with all their naturally
occurring complexity and ambiguity, with the
goal of reconstructing the processes, objects and
events that produced them and of predicting these
patterns when they reoccur
5/29/2016
Visual Perception Modeling
18
Bayesian Decision Rule
• A two-class example
– 1 for sea bass
– 2 for salmon
• Prior probability
– P(1)
– P(2)
5/29/2016
Visual Perception Modeling
19
Bayesian Decision Rule – cont.
• Class conditional probability density
– P(1 | x)
– P(2 | x)
• Bayes formula
p( x |  i ) P( i )
P( i ) 
P( x)
5/29/2016
Visual Perception Modeling
20
Bayesian Decision Rule – cont.
• Bayes decision rule
– Decide 1 if P(1 | x) > P(2 | x)
– Otherwise decide 2
– The optimal decision rule
• Minimize the average error we make
5/29/2016
Visual Perception Modeling
21