SCALABLE IMAGE
MATCHING
David
Strickland
ENGN 256
Spring 2013
REFERENCE PAPER: INVERTED INDEX COMPRESSION
FOR SCALABLE IMAGE MATCHING
 Chen, D.M.; Tsai, S.S.; Chandrasekhar, V.; Takacs, G.;
Vedantham, R.; Grzeszczuk, R.; Girod, B., "Inverted Index
Compression for Scalable Image Matching," Data Compression
Conference (DCC), 2010 , vol., no., pp.525,525, 24 -26 March
2010
SCALABLE IMAGE MATCHING FOR THE
BLINDFIND PROJECT
 BlindFind aims to help the blind navigate unfamiliar indoor
environments to locate places via a wearable navigation
device powered by crowd -sourced maps
 The device needs to know its map location
 Image matching is required
 Image matching must be fast and real time
 Needs to be scalable
IMAGE MATCHING
 Most solutions are base on local image features:
 SIFT (Scale-Invariant Feature Transform)
 SURF (Speeded Up Robust Features”
 CHoG (Compressed Histogram of Gradients)
 Process:
1. Detect Features
2. Extract Feature Locations and Descriptors
3. Compare the Features of two images to determine similarity
EXAMPLE OF FEATURES
http://en.wikipedia.org/wiki/File:Sift_keypoints_filtering.jpg
PROBLEM
 Comparing an image with every image in a large database
takes an extremely long amount of time
 Doesn’t scale
 Some databases contain millions of images
SOLUTION: VOCABULARY TREE + INVERTED
INDEX
1.
2.
3.
4.
5.
Detect Features
Extract Feature Locations and Descriptors
Quantize Descriptors into a Vocabulary Tree
Score Database Images using Inverted Index
Pairwise Match using Geometric Check
VOCABULARY TREE + INVERTED INDEX
 The Vocabulary Tree is a tree -structured vector quantizer
constructed by hierarchical k-means clustering of feature
descriptors.
 Inverted Index: Each node has two lists
 Image IDs
 Array of counts
1Image
from Chen et al.
SIMILARIT Y SCORING & MEMORY USAGE
 Each image i k1 in the database of N images is given a
similarity score
 For each node visited by query descriptors the node’s
inverted list of images all have the scores incremented :
Where:
NEW PROBLEM
 Inverted index requires lots of memory
 Memory usage for V T with K leaf nodes, where N k database
images have visited each node:
 Need to find a way to reduce memory usage of the inverted
index
NEW SOLUTIONS FOR DEALING WITH LARGE
AMOUNTS OF DATA
 Fast decoding methods
 Carryover Code (32 bit word)
 Recursive Bottom Up Complete (RBUC)
 Inverted Index Compression
 Encode Image IDs by consecutive differences
 Reorder database to minimize differences
 Soft-binned feature descriptor histograms
 Improves accuracy of VT search
FAST DECODING METHODS
 Carryover Code (32 bit word)
 2-bit selector + 30 bit data portion
 Selector indicates the precision of the data portion
 E.g. 30 1 bit data symbols, 15 2 bit data symbols, etc.
 Recursive Bottom Up Complete (RBUC)
 Similar to carryover code encoding
 The precision for each pair is calculated as the max of the two
precisions
 Then P’ itself is encoded
INVERTED INDEX COMPRESSION
 Encode each inverted index’s Image IDs by consecutive
dif ferences
 Inverted index compression techniques can significantly reduce
memory usage by up to 5X without any loss in recognition accuracy
 Reorder database to minimize dif ferences
 Minimize:
SOFT-BINNED FEATURE DESCRIPTOR
HISTOGRAMS
 Classify a feature descriptor to k nearest tree nodes instead
of just nearest tree node
 Soft-binned tree gives improvement in classification accuracy
 Disadvantage:
 Each database feature now appears in k different inverted lists
 Inverted Index is k times as large
SCHEDULE I: VT/II IMPLEMENTATION
 Week 1: Research Vocabulary Tree / Inverted Index,
Determine which libraries to use
 Week 2: Implement Feature Locator/Descriptors
 Week 3: Implement Quantization of Descriptors in V T
 Week 4: Implement Database scoring scheme using Inverted
Index
 Week 5: Milestone: Mid Project Presentation, Combine
Previous parts, Pairwise Match to retrieve a single image
SCHEDULE II: COMPRESSION




Week
Week
Week
Week
6:
7:
8:
9:
Inverted Index Image ID storage
Fast Decoding
Soft Binned Tree, Analysis
Final Project Presentation
REFERENCES
 David M. Chen, Sam S. Tsai, Vijay Chandrasekhar, Gabriel
Takacs, Ramakrishna Vedantham, Radek Grzeszczuk, Bernd
Girod, “Inverted Index Compression for Scalable Image
Matching”