Algorithms and Applications in
Computer Vision
Lihi Zelnik-Manor
lihi@ee.technion.ac.il
ROC
Precision-Recall
circles
Truth:
Not circles
Result:
circles
Not circles
Precision-Recall
circles
Not circles
Truth:
Not circles
FN
Result:
circles
TP FP TN
Confusion matrix
P = Positive
N = Negative
Ground-truth
E1
E2
E1
obtained
result
E2
tp
(true
positive)
fn
(false
negative)
fp
(false
positive)
tn
(true
negative)
ROC curve
TP
TPR 
P
AUC = Area
Under Curve
FP
FPR 
N
Precision-Recall
TP
Re call 
TP  FN
TP
Pr ecision 
TP  FP
F-measure
The harmonic mean (mean of rates) of the precision and recall
precision recall
F
precision recall
True/false positives
50
75
true match
200
false match
feature distance
The distance threshold affects performance
• True positives = # of detected matches that are correct
• False positives = # of detected matches that are incorrect
Evaluating the results
How can we measure the performance of a feature matcher?
1
0.7
true
# true positives
# matching features (positives) positive
rate
0
0.1
false positive rate
# false positives
# unmatched features (negatives)
1
Evaluating the results
How can we measure the performance of a feature matcher?
ROC curve
(“Receiver Operator Characteristic”)
1
0.7
true
# true positives
# matching features (positives) positive
rate
0
0.1
false positive rate
1
# false positives
# unmatched features (negatives)
ROC Curves
•
•
•
•
Generated by counting # correct/incorrect matches, for different threholds
Want to maximize area under the curve (AUC)
Useful for comparing different feature matching methods
For more info: http://en.wikipedia.org/wiki/Receiver_operating_characteristic