Autonomous Learning of Object Models
on Mobile Robots
Xiang Li
Ph.D. student supervised by Dr. Mohan Sridharan
Stochastic Estimation and Autonomous Robotics Lab (SEARL)
Department of Computer Science
Texas Tech University
October 23, 2012
1
Object Recognition
Challenges:
• How to identify ROI in the image (Region Of Interest) ?
• What features to extract in ROI (Object model) ?
• Efficient Implementation.
ROI
match
Learn Object Model
color, texture or shape
Test
2
Related Work
• Object model
O. Bjorn, PAMI12; R. Fergus, CVPR03; P. Felzenszwalb, IJCV04; N.E.K. Roman,
AR10.
• Mobile robot
C. Guo, ICRA11; M. Sridharan, IROS07; D. Parikh, PAMI12; J. Hoey, CVIU10.
• Local image feature
D. Lowe, IJCV05; J. Matas, BMVC02; S. Park, IROS09; M. Calonder, ECCV10.
3
Motivation
• Goal
Learn and recognize objects autonomously in dynamic
environments.
• Our Work
 Identify ROIs autonomously based on a limited number of images
with moving objects.
 Build probabilistic object models using the complementary
properties of different visual cues.
 Fuse the information by generative model and energy minimization
algorithm.
4
Autonomously Learning
• Supervised Learning
 Images with the labeled regions
• Unsupervised Learning
 Images without any labeled regions
 Track and cluster local image gradient features [128D vector]
 A short sequence of images (motion cue)
ROI
ROI
t
t+1
5
Object Model
•
•
Given ROI (autonomous or manual)
Use the complementary properties of different visual cues
6
Motivation of SCV and Undirected Graph
• Match by Nearest Neighbor algorithm(shortest Euclidean distance).
correct
Incorrect
•
The difference between correct and incorrect match
 The spatial arrangements of local features
 The connection between the local features
7
SCV from gradient features
• The individual gradient features may not be unique.
• The spatial arrangement of local gradient features corresponding to a
specific object is difficult to duplicate.
8
Connection Potentials
• Connection potential is computed as the color distribution of pixels
between gradient features in the image ROI.
• Build an undirected graph of connection potentials to model the
neighborhood relationships between gradient features.
9
Parts from image segments
• Considers the arrangement of object parts made up of image segments.
• Pixels within a part have similar values, while pixels in neighboring parts
have dissimilar values.
10
Color-based Representation
• Computes the distance between every pair of pdfs and models the
distribution of distances as a Gaussian.
……
Color histogram (pdf)
Second order color distribution statistics
11
Local context from image segments
• Probabilistic mixture models
• Relative positions (on, under, beside)
12
Information Fusion
• Generative model
 considers the relationship between the components
• Energy minimization algorithm
 Identifies the ROI for recognizing the stationary objects
13
Robot Platforms: Erratic
• 1.6GHz Core2 Duo CPU
• 2 cameras (monocular &
stereo)
• Laser range finder
• 640 × 480 Resolution
• Wi-Fi
• On board computation
14
Experimental Trial
Test Image
Match Probabilities
Net Match
15
Experimental Trial (Cont)
Test Image
Match Probabilities
Net Match
16
Object Categories
17
The Classification Accuracy
18
Conclusions
• Mobile robot can identify interesting objects based on motion
cues, and autonomously and efficiently learn object models
that exploit the complementary properties of appearancebased and contextual visual cues.
• Exploiting the complementary properties of these visual cues
enables the robot to use generative model and energy
minimization algorithms to reliably and efficiently recognize
the learned.
19
Future Work
• Consider image sequences with multiple moving objects.
• Add Shape representation into object model.
• Extend to a team of robots collaborating in dynamic
environments.
20
Q&A
21
Convex Hull
• The convex hull of a set of points is the smallest convex set
that contains the points.
• Quickhull algorithm1 computes the convex hull.
1. Barber, C.B., Dobkin, D.P., and Huhdanpaa, H.T., “The Quickhull
algorithm for convex hulls”, ACM Trans. on Mathematical Software,
22(4):469-483, Dec 1996, http://www.qhull.org
Gamma distribution
• Context Probability
Matched
Not Matched
context
0.28±0.15
0.03±0.03
Other components
0.7
0.3
• Context Probability Using Gamma
Matched
Not Matched
context
0.5±0.25
0.09±0.10
Other components
0.7
0.3
Experiment Example
• Learned robot model (Corridor)
• Testing
Corridor(1)
Prob = 0.89
Corridor(2)
Prob = 0.77
Office
Prob = 0.12
Generative Model
• Randomly generating observable data
• Typically given some hidden parameters
• Specifies a joint probability distribution over
observation and label sequences.
25
Generative model (Cont)
26