IIIT HYDERABAD
Image-based walkthroughs from
partial and incremental scene
reconstructions
Kumar Srijan
Syed Ahsan Ishtiaque
C. V. Jawahar
Center for Visual
Information Technology,
IIIT-Hyderabad
http://cvit.iiit.ac.in
Sudipta N. Sinha
Microsoft Research,
Redmond
http://research.microsoft.com
Introduction
IIIT HYDERABAD
Problem
IIIT HYDERABAD
• Efficiently organize and browse these huge
image collections?
• Keep Incorporating an incoming stream of
images into an existing framework?
Related Work
IIIT HYDERABAD
•
•
•
•
•
•
•
•
World-Wide Media Exchange (WWMX)
PhotoCompas
Realityflythrough
Aspen Movie Map
Photowalker
Sea of Images
Google Streetview
Photo Tourism
Photo Tourism
IIIT HYDERABAD
Input Images
Computing correspondences
Detect
Match keypoints
Features in
between each
each image
pair of images
Incremental SfM
Select a good
initial pair to seed
reconstruction
For each pair,
estimate an F-matrix
and refine matches
Add new images
and triangulate new
points
Full Scene Reconstruction
Snavely et. al, Photo Tourism: Exploring image collections in 3D
Chain pairwise
matches into
tracks
Bundle adjust
Bottlenecks and Issues
IIIT HYDERABAD
• Global scene reconstruction via incremental
structure from motion (Sfm)
– Sensitivity to the choice of the initial pair
– Cascading of errors
– O(N4) in the worst case
Snavely et. al, Photo Tourism: Exploring image collections in 3D
Bottlenecks and Issues
IIIT HYDERABAD
• Timing Breakdown
Full Scene Reconstruction for Trafalgar Square dataset with 8000
images took > 50 days
Snavely et. al, Photo Tourism: Exploring image collections in 3D
Our approach
IIIT HYDERABAD
• “ In a walkthrough, users primarily observe
near by overlapping images.”
Independent Partial Scene Reconstructions
instead of
Global Scene Reconstruction
• Advantages:
–
–
–
–
Robustness to errors in incremental SfM module
Worst case linear running time
Scalable
Incremental
Partial Reconstructions
IIIT HYDERABAD
Compute
Matches
Match
Image
Refine
Matches
Correct Match
Incorrect Match
Standard
SfM
Compute partial
Reconstructions
User interface and navigation
IIIT HYDERABAD
Sample
image
Input images
Verified neighbors
Visualization
Interface
Partial reconstruction
Global vs. Partial
IIIT HYDERABAD
• Global : Allows transition to any image
• Partial : Allows transition to a limited number
of overlapping images
• A -> B implies B -> A
A
A
B
B
Incremental insertion
IIIT HYDERABAD
Geometric
Match
Verification
Compute
Partial Scene
Reconstruction
New Image
Improve
Connectivity
Dataset
IIIT HYDERABAD
Golconda Fort, Hyderabad
Fort Dataset
5989 images
Results
IIIT HYDERABAD
Results
IIIT HYDERABAD
Results
IIIT HYDERABAD
• Courtyard Dataset with
687 images
• Initialized with 200
images
• Added 487 image one
by one
• Largest CC of 674
images.
Conclusion
IIIT HYDERABAD
• Image navigation
system based on partial
reconstructions can
effectively be used to
navigate through large
collections of images.
• Robustness to errors
• Able incorporate more
images as they become
available.
Future Work
IIIT HYDERABAD
• Complete automation
– Download images
directly from the
internet
– Add into the framework
Acknowledgements
IIIT HYDERABAD
• “Photo tourism:
Exploring photo
collections in 3D“
– Noah Snavely, Cornell
University
– Steven M. Seitz,
University of Washington
– Richard Szeliski,
Microsoft Research
Acknowledgements
IIIT HYDERABAD
• “Visual Word based
Location Recognition in
3D models using
Distance Augmented
Weighting”
– Friedrich Fraundorfer,
Marc Pollefeys
ETH Zürich
– Changchang Wu ,JanMichael Frahm ,Marc
Pollefeys - UNC Chapel
Hill
Thank You
IIIT HYDERABAD
• Questions