Compact Hyperplane Hashing with Bilinear Functions Wei Liu (Columbia
Columbia), Jun Wang (IBM
IBM), Yadong Mu (Columbia
Columbia), Sanjiv Kumar (Google
Google), and Shih‐Fu Chang (Columbia
Columbia)
June, 2012 Point‐to‐Point Search vs. Point‐to‐Hyperplane Search
normal
vector
t
nearest nearest
neighbor
nearest nearest
neighbor
hyperplane
h
perplane
query
point query
Exhaustive point‐to‐hyperplane
p
yp p
search costs linear time. Hashing hyperplane queries to near neighbors, achieving sublinear
2
time or faster.
Hashing Principle: Point‐to‐Hyperplane Angle
1
1
‐1
‐1
‐1
1
The ideal neighbors ┴ w
The ideal neighbors ┴
3
Proposed Idea: Bilinear Hashing
Proposed Idea: Bilinear Hashing
• A
A bilinear
bilinear hash function outputs the same bit for parallel hash function outputs the same bit for parallel
inputs and tends to output different bits for ┴ inputs:
Bilinear‐Hyperplane Hash (BH‐Hash) Any input vector: query normal w or query normal w
or
database point x.
random projection vectors
4
A Single Bit
A Single Bit
x1
u
v
1
x2
‐1
‐1
Probability?
1
// bin
// bin
┴ bin
5
Theoretic Analysis
Theoretic Analysis
is the highest collision probability till now.
Jain 2010
Ours is highest
6
Multiple Bits
Multiple Bits
u1 u2
v1
v2
┴ bin1
┴ bin2
neighbor region = bin1 ∩ bin2 ∩ bin3…
7
Learning Bilinear Projections
Learning Bilinear Projections
1
u
w
1
-1
x1
u
w
1
x1
v
v
Make h(x) yield the same bit for nearly // inputs.
8
Learning Bilinear Projections
Learning Bilinear Projections
1
1
w
w
u
x2 1
v
u
x2 -1
v
Make h(x) yield different bits for nearly ┴ inputs.
9
Active Learning Experiments
Active Learning Experiments
• 20 Newsgroup: about 20K documents from 20 classes.
20 Newsgroup: about 20K documents from 20 classes.
• Use
Use linear SVMs as base classifiers and run 300 active linear SVMs as base classifiers and run 300 active
learning iterations per class.
• Hyperplane hashing in each AL iteration: use 16 bits to do hash lookup within Hamming radius 3, and acquire a short list containing neighbors from the found hash buckets; scan the list to return the final neighbor that has the smallest point‐to‐hyperplane
i
h
l
di
distance. 10
20 Newsgroup
20 Newsgroup
LBH‐Hash outperforms
exhaustive search.
LBH‐Hash returns much better neighbors than the others.
11
Please come to our poster for more details!
Thanks! Questions?
Thanks! Questions?
12