masks

advertisement
Step-by-Step Model Buidling
MASKS © 2004
Invitation to 3D vision
Review
Feature
selection
Feature
selection
Feature correspondence
Camera Calibration
Landing
Augmented Reality
Euclidean Reconstruction
Vision Based Control
Sparse Structure and camera motion
MASKS © 2004
Invitation to 3D vision
Review
Feature
selection
Feature
selection
Feature correspondence
Camera Calibration
Epipolar Rectification
Dense Correspondence
Euclidean Reconstruction
Sparse Structure and motion
MASKS © 2004
Invitation to 3D vision
Texture mapping
3-D Model
Review
Feature
selection
Feature
selection
Feature correspondence
Projective Reconstruction
Partial Scene Knowledge
Partial Motion Knowledge
Partial Calibration Knowledge
Camera Self-Calibration
Epipolar Rectification
Dense Correspondence
Euclidean Reconstruction
Texture mapping
3-D Model
MASKS © 2004
Invitation to 3D vision
Examples
MASKS © 2004
Invitation to 3D vision
Feature Selection
Compute Image Gradient
•  Compute Feature Quality
• 
• 
The image
cannot be
displayed.
Your
measure for each pixel
Search for local maxima
Feature Quality Function
MASKS © 2004
Invitation to 3D vision
Local maxima of feature quality function
Feature Tracking
• 
Translational motion model
•  Closed form solution
1. 
2. 
3. 
4. 
Build an image pyramid
Start from coarsest level
Estimate the displacement at the coarsest level
Iterate until finest level
MASKS © 2004
Invitation to 3D vision
Coarse to fine feature tracking
0
1
2
1. 
2. 
3. 
4. 
5. 
compute
warp the window
in the second image by
update the displacement
go to finer level
At the finest level repeat for several iterations
MASKS © 2004
Invitation to 3D vision
The image
cannot be
displayed.
Your
Optical Flow
•  Integrate around over image patch
•  Solve
MASKS © 2004
Invitation to 3D vision
Affine feature tracking
Contrast change
MASKS © 2004
Invitation to 3D vision
Intensity offset
Tracked Features
MASKS © 2004
Invitation to 3D vision
Wide baseline matching
Point features detected by Harris Corner detector
MASKS © 2004
Invitation to 3D vision
Wide baseline Feature Matching
1. 
2. 
3. 
4. 
Select the features in two views
For each feature in the first view
Find the feature in the second view that maximizes
Normalized cross-correlation measure
Select the candidate with the similarity above selected threshold
MASKS © 2004
Invitation to 3D vision
More correspondences and Robust matching
• 
• 
Select set of putative correspondences
Repeat
1. Select at random a set of 8 successful matches
2. Compute fundamental matrix
3. Determine the subset of inliers, compute distance to
epipolar line
The
ima
ge
cann
4. Count the number of points in the consensus set
MASKS © 2004
Invitation to 3D vision
RANSAC in action
Inliers
MASKS © 2004
Invitation to 3D vision
Outliers
Epipolar Geometry
•  Epipolar geometry in two views
•  Refined epipolar geometry using nonlinear estimation of F
MASKS © 2004
Invitation to 3D vision
Two view initialization
calibrated
• 
Recover epipolar geometry
• 
Compute (Euclidean) projection matrices and 3-D struct.
uncalibrated
• 
unknown
Compute (Projective) projection matrices and 3-D struct.
MASKS © 2004
Invitation to 3D vision
Nonlinear Refinement
• 
Euclidean Bundle adjustment
Initial estimates of
are available
•  Final refinement, nonlinear minimization with respect
to all unknowns
• 
MASKS © 2004
Invitation to 3D vision
Example - Euclidean multi-view reconstruction
MASKS © 2004
Invitation to 3D vision
Example
Original sequence
MASKS © 2004
Invitation to 3D vision
Tracked Features
Recovered model
MASKS © 2004
Invitation to 3D vision
Euclidean Reconstruction
MASKS © 2004
Invitation to 3D vision
Epipolar rectification
Make the epipolar lines parallel
•  Dense correspondences along image scanlines
•  Computation of warping homographies
• 
1. Map the epipole
The
imag
e
The
imag
e
to infinity
Translate the image center to the origin
Rotate around z-axis for the epipole lie on the x-axis
Transform the epipole from x-axis to infinity
2. Find a matching transformation
T
h
e
is compatible with the epipolar geometry
is chosen to minimize overall disparity
MASKS © 2004
Invitation to 3D vision
Epipolar rectification
Rectified Image Pair
MASKS © 2004
Invitation to 3D vision
Epipolar rectification
Rectified Image Pair
MASKS © 2004
Invitation to 3D vision
Dense Matching
• 
• 
• 
Establish dense correspondences along
scan-lines
Standard stereo configuration
Constraints to guide the search
1. ordering constraint
2. disparity constraint – limit on disparity
3. uniqueness constraint – each point has
a unique
match in the second view
MASKS © 2004
Invitation to 3D vision
Dense Matching
MASKS © 2004
Invitation to 3D vision
Dense Reconstruction
MASKS © 2004
Invitation to 3D vision
Texture mapping, hole filling
MASKS © 2004
Invitation to 3D vision
Texture mapping
MASKS © 2004
Invitation to 3D vision
Steps
+
=
Images ! Points:
Points ! More points:
Points ! Meshes:
Meshes ! Models:
Images ! Models:
+
+
Structure from Motion
Multiple View Stereo
Model Fitting
Texture Mapping
Image-based Modeling
+
=
Bundle adjustment
• 
Theory:
The Levenberg–Marquardt algorithm
• 
Practice:
The Ceres-Solver from Google
MASKS © 2004
Invitation to 3D vision
Multiple View Stereo
State-of-the-art:
PMVS: http://grail.cs.washington.edu/software/pmvs/
Accurate, Dense, and Robust Multi-View Stereopsis, Y Furukawa and J Ponce, 2007
Benchmark:
http://vision.middlebury.edu/mview/
A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms.
SM Seitz, B Curless, J Diebel, D Scharstein, R Szeliski. 2006.
Baseline:
Multi-view stereo revisited. M Goesele, B Curless, SM Seitz. 2006.
How to get the intrinsic parameters?
• 
Auto-calibration
Self-Calibration and Metric Reconstruction in spite of Varying and Unknown
Internal Camera Parameters, M Pollefeys, R Koch and L Van Gool, 1998.
http://mit.edu/jxiao/Public/software/autocalibrate/autocalibration_lin.m
• 
Grid Search to look for the solution with minimal
reprojection error
for f=min_f:max_f
do everything, then obtain reprojection error after bundle adjustment
• 
• 
Optimize for this value in bundle adjustment
Camera Calibration (with checkerboard)
http://www.vision.caltech.edu/bouguetj/calib_doc/
• 
EXIF of JPEG file recorded from digital camera
Read the code of Bundler to understand how to convert EXIF into focal length value
http://phototour.cs.washington.edu/bundler/
Real World Applications
• 
Streetview Reconstruction and Recognition
http://vision.princeton.edu/projects/2009/ICCV/
http://vision.princeton.edu/projects/2009/TOG/
• 
• 
• 
Photo Tourism http://phototour.cs.washington.edu/
Microsoft Photosynth http://photosynth.net/
2d3, boujor (Matchmovers) and movies
http://www.2d3.com/
• 
http://www.vicon.com/boujou/
Robotics: SLAM http://openslam.org/
Simultaneous Localization And Mapping
Download