Comparative Evaluation of Random Forest and Fern
classifiers for Real-Time Feature Matching
I. Barandiaran1, C.Cottez1, C.Paloc1 , M.Graña2
1Departamento de Aplicaciones Biomédicas
Asociación VICOMTech, San Sebastián, {ibarandiaran,ccottez,cpaloc}@vicomtech.org
2University
of Basque Country Computer Science School, Pº. Manuel de Lardizabal, 1 20009, San Sebastián, Spain
ccpgrrom@si.ehu.es
VISUAL INTERACTION AND COMMUNICATIONS TECHNOLOGIES
WSCG2008, Plzen, 04-07, Febrary 2008
Summary
1. Introduction.
2. Random Forest, FERNS
1. Mixed/Augmented Reality Application.
2. Conclusions/Questions.
2
Introduction
Motivation and objectives
•
•
•
Motivated by the work of Vincent LePetit
Real-Time Augmented Reality.
Camera Pose Estimation.
• Markerless tracking.
• Model-based tracking.
• Tracking by detection.
•
Test and compare different parameters.
• Scale.
• Size of the Training Set.
• Number of Classes.
• Training Time.
(CVLab).
3
Introduction
Augmented Reality Features:
• Mix Virtual and Real Objects..
• Real-Time.
• Portable Devices (Head Mounted Display, Tablet PC, PDA Device, Movil Phone..)
4
Introduction
Problems:
• Rendering.
• Real-Time(Delay).
• Registration/Pose Estimation.
5
Introduction
Non model-based Tracking
• No a priori knowledge of the object to be tracked.
• Updates/Propagates an estimation over time.
• Partial object occlusions.
• Tend to tracking reinitialization.
Model Based Tracking
• Some a priori knowledge is available.
• May not depend on the past.
• Frame by Frame estimation.
• Robust against partial object occlusion.
• Automatic tracking initialization.
6
Summary
1. Introduction
2. Random Forest, FERNS
3. Mixed/Augmented Reality
4. Conclusions/Questions
7
Random Forest, FERNS
Tracking of Planar Surfaces.
The Classifiers are applied for interest point (feature) matching.
Matched Points are used during camera pose estimation Process.
8
Random Forest, FERNS
Building the training set.
• Frontal view of the object to be detected.
• Feature Point extraction FAST (Rosten06) and YAPE (CvLab).
• Sub-images (patches) are generated for each class.
Classes to Be recognized
by the Classifier
9
Random Forest, FERNS
Building the training set.
• Generate Random Affine transformations.
• Generate new examples of each Class.
Training Set (examples)
Random Affine
transformations
…..
10
Random Forest
Multiclassifier based on Randomized Trees.
Firstly introduced in 1997 handwritten recognition (Amit, Y.,German, D.)
Developed by Leo Breiman (Medical Data Analisys).
Recently Applied to tracking by detection (LePetit06).
Main Features
• Fast Training Step, and execution.
• Good Precision.
• Random selection of the independent variables (features).
• Random selection of Examples.
• Easy to Implement and paralelizable.
11
Random Forest
Classifier Training.
• N Binary-Trees are Grown.
• Pixel intensity tests are executed in any non-terminal node.
• Pixels can be selected at Random.
• Posterior Distributions P(Y=c |T=Tk,n) are stored in leave nodes.
12
Random Forest
Example Classification.
• Every example is dropped down the trees.
• The Example traverse the tree towards the leaf nodes.
Pixels to be tested
13
Random Forest
Combine Results
• The example labeling is obtained as a combination of partial results
obtained by every tree in the forest.
Random Forest
T1
Tn
T2
Exampleclass _ label  arg max t

T
k 1

P Y  c i | t kn

14
FERNS
Introduced in 2007 (Mustafa Özuysal).
Multiclassifier.
Applied to 3D keypoint recognition.
Successfully applied to image recognition/retrieval (Zisserman07).
Main Features
• Non hierarchical structure.
• Semi Naive-Bayes Combination Strategy.
• Random selection of the independent variables (features).
• Random selection of Examples.
• Easy to Implement and paralelizable.
15
FERNS
Semi-Naive Bayes Combination.
c i  arg max PC  c i | f 1 , f 2 ,.... f n 
PC  c i | f 1 , f 2 ,.... f n  
P ( f 1 , f 2 ,.... f n | C  c i ) PC  c i 
P f 1 , f 2 ,.... f n 

P f 1 , f 2 ,.... f n | C  c i  
 P f
N
j
| C  ci

1 if p j ,1  p j , 2  t
fj 
otherwise
0
c i  arg max P f 1 , f 2 ,.... f n | C  c i 

j 1
P f 1 , f 2 ,.... f n | C  ci  
M
 P F
k
| C  ci 
Fk
j 1
16
FERNS
Classifier Training
Posterior Distributions
(Look-up Tables)
23
x
x
PFk | C  c i 
Possible Outputs
x
7
0
.
.
.
17
FERNS
Classifier Training
Posterior Distributions
.
.
(Look-up Tables)
.
.
Class 1
Fern 1
1
1
0
6
0
1
0
0
1
1
Class 2
.
2
.
.
Class 1
0
Fern 2
0
0
0
Class 2
.
.
3
.
.
.
.
Fern n
.
18
FERNS
Example Classification.
Fern 1
Fern 2
Fern 3
Posterior Distributions
(Look-up Tables)
2
6
1
M
Exampleclass _ label  arg max
f
 P F
k
| C  c i 
1
19
Random Forest vs FERNS
Rotation Range
• 20 Trees, 15 Depth.
• 225 Different Clases.
• 400 Images per class.
Rotation Range
% Classification Rate
100
95
90
85
80
0
PI/2
PI
FERNS
3PI/2
2PI
Random Forest
20
Random Forest vs FERNS
Scale Range
• 20 Trees, 15 Depth.
• 225 Different Clases.
• 400 Images per class.
% Classification Rate
Scale Range
100
90
80
70
60
50
40
30
20
10
0
0,8-1,0
0,5-1.0
1,0-1,2
FERNS
1,0-1,5
0,8-1,2
0,5-1,5
Random Forest
21
Random Forest vs FERNS
Size of the training Set
• 20 Trees, 15 Depth.
• 225 Different Classes.
• [0.5-1.5] Scale Range.
Training Set Size
80
% Classification Rate
75
70
65
60
55
50
45
350
550
750
1000
FERNS
1300
1500
1800
2100
2800
3800
4550
Random Forest
22
Random Forest vs FERNS
Number of different Classes.
• 20 Trees, 15 Depth.
• [0.8-1.2] Scale Range.
• 1500 Training images per class.
Number of Classes
% Classification Rate
100
95
90
85
80
200
325
FERNS
425
525
625
725
Random Forest
23
Random Forest vs FERNS
Training time.
• 20 Trees, 15 Depth.
• 225 Different Classes.
• [0.5-1.5] Scale Range.
300
Trainingo Time (s)
250
200
150
100
50
0
350
550
750
1000
1500
1800
2500
2800
3800
24
Pose Estimation
Homography Estimation
•
•
•
Robust Estimation (RANSAC).
Non-Linear Minimization (Levenberg-Marquardt).
25
Summary
1. Introduction.
2. Random Forest, FERNS.
3. Mixed/Augmented Reality Application.
4. Conclusions/Questions.
26
Augmented Reality Application
European Project IMPROVE (Improving Display and Rendering
Technology
for Virtual Environments)
• Develop of new interaction metaphors.
• Develop of new Displays.
• Photo Realistic Rendering.
• Development of Markerless Tracking Techniques.
27
Augmented Reality Application
Architectural Scenario
Automotive Scenario
28
Augmented Reality Application
Marker-Less tracking (InDoor Scenario)
Textured plane
Image Augmentation
Feature Points Tracking
29
Augmented Reality Application
Marker-Less tracking (OutDoor Scenario)
Image Acquisition
Feature points Tracking
Image Augmentation
30
Augmented Reality Application
Performance
• 20 Trees.
• Full Rotation Range and [0.8-1.2] Scale Range.
• 1000 images per Class.
• 250 Different Classes.
CPU Type
25
20
15
10
5
0
Intel Core2 Duo 1,6 Ghz
AMD 2,1+Ghz
Pentium4 1,4Ghz
Intel Centrino 1Ghz
31
Summary
1. Introduction.
2. Random Forest, FERNS.
3. Mixed/Augmented Reality Application.
4. Conclusions/Questions.
32
Conclusions
Both Approaches are very Similar.
The classifier is more sensitive to variations in scale.
The classifier is robust against variations in object orientation.
When the classifier converges, increase the number of trees does not improve
accuracy.
The node test can be selected at random.
FERNS Requires more Memmory Than Random Forest.
Training and classification Time is Higher in FERNS than in Random Forest.
Random Forest are Faster than FERNS (without heuristics)
FERNS Supports more classes than Random Forest.
The Output of both classifiers must be filtered.
The higher the classification accuracy, the better the performance of the
tracking.
33
Thanks For Listening
Iñigo Barandiaran Martirena
(ibarandiaran@vicomtech.org)
Researcher, VICOMTech
Paseo Mikeletegi 57
20009 San Sebastián
Tfno: +34 943 30 92 30 Fax : +34 943 30 93 93