masks
advertisement
Step-by-Step Model Buidling MASKS © 2004 Invitation to 3D vision Review Feature selection Feature selection Feature correspondence Camera Calibration Landing Augmented Reality Euclidean Reconstruction Vision Based Control Sparse Structure and camera motion MASKS © 2004 Invitation to 3D vision Review Feature selection Feature selection Feature correspondence Camera Calibration Epipolar Rectification Dense Correspondence Euclidean Reconstruction Sparse Structure and motion MASKS © 2004 Invitation to 3D vision Texture mapping 3-D Model Review Feature selection Feature selection Feature correspondence Projective Reconstruction Partial Scene Knowledge Partial Motion Knowledge Partial Calibration Knowledge Camera Self-Calibration Epipolar Rectification Dense Correspondence Euclidean Reconstruction Texture mapping 3-D Model MASKS © 2004 Invitation to 3D vision Examples MASKS © 2004 Invitation to 3D vision Feature Selection Compute Image Gradient • Compute Feature Quality • • The image cannot be displayed. Your measure for each pixel Search for local maxima Feature Quality Function MASKS © 2004 Invitation to 3D vision Local maxima of feature quality function Feature Tracking • Translational motion model • Closed form solution 1. 2. 3. 4. Build an image pyramid Start from coarsest level Estimate the displacement at the coarsest level Iterate until finest level MASKS © 2004 Invitation to 3D vision Coarse to fine feature tracking 0 1 2 1. 2. 3. 4. 5. compute warp the window in the second image by update the displacement go to finer level At the finest level repeat for several iterations MASKS © 2004 Invitation to 3D vision The image cannot be displayed. Your Optical Flow • Integrate around over image patch • Solve MASKS © 2004 Invitation to 3D vision Affine feature tracking Contrast change MASKS © 2004 Invitation to 3D vision Intensity offset Tracked Features MASKS © 2004 Invitation to 3D vision Wide baseline matching Point features detected by Harris Corner detector MASKS © 2004 Invitation to 3D vision Wide baseline Feature Matching 1. 2. 3. 4. Select the features in two views For each feature in the first view Find the feature in the second view that maximizes Normalized cross-correlation measure Select the candidate with the similarity above selected threshold MASKS © 2004 Invitation to 3D vision More correspondences and Robust matching • • Select set of putative correspondences Repeat 1. Select at random a set of 8 successful matches 2. Compute fundamental matrix 3. Determine the subset of inliers, compute distance to epipolar line The ima ge cann 4. Count the number of points in the consensus set MASKS © 2004 Invitation to 3D vision RANSAC in action Inliers MASKS © 2004 Invitation to 3D vision Outliers Epipolar Geometry • Epipolar geometry in two views • Refined epipolar geometry using nonlinear estimation of F MASKS © 2004 Invitation to 3D vision Two view initialization calibrated • Recover epipolar geometry • Compute (Euclidean) projection matrices and 3-D struct. uncalibrated • unknown Compute (Projective) projection matrices and 3-D struct. MASKS © 2004 Invitation to 3D vision Nonlinear Refinement • Euclidean Bundle adjustment Initial estimates of are available • Final refinement, nonlinear minimization with respect to all unknowns • MASKS © 2004 Invitation to 3D vision Example - Euclidean multi-view reconstruction MASKS © 2004 Invitation to 3D vision Example Original sequence MASKS © 2004 Invitation to 3D vision Tracked Features Recovered model MASKS © 2004 Invitation to 3D vision Euclidean Reconstruction MASKS © 2004 Invitation to 3D vision Epipolar rectification Make the epipolar lines parallel • Dense correspondences along image scanlines • Computation of warping homographies • 1. Map the epipole The imag e The imag e to infinity Translate the image center to the origin Rotate around z-axis for the epipole lie on the x-axis Transform the epipole from x-axis to infinity 2. Find a matching transformation T h e is compatible with the epipolar geometry is chosen to minimize overall disparity MASKS © 2004 Invitation to 3D vision Epipolar rectification Rectified Image Pair MASKS © 2004 Invitation to 3D vision Epipolar rectification Rectified Image Pair MASKS © 2004 Invitation to 3D vision Dense Matching • • • Establish dense correspondences along scan-lines Standard stereo configuration Constraints to guide the search 1. ordering constraint 2. disparity constraint – limit on disparity 3. uniqueness constraint – each point has a unique match in the second view MASKS © 2004 Invitation to 3D vision Dense Matching MASKS © 2004 Invitation to 3D vision Dense Reconstruction MASKS © 2004 Invitation to 3D vision Texture mapping, hole filling MASKS © 2004 Invitation to 3D vision Texture mapping MASKS © 2004 Invitation to 3D vision Steps + = Images ! Points: Points ! More points: Points ! Meshes: Meshes ! Models: Images ! Models: + + Structure from Motion Multiple View Stereo Model Fitting Texture Mapping Image-based Modeling + = Bundle adjustment • Theory: The Levenberg–Marquardt algorithm • Practice: The Ceres-Solver from Google MASKS © 2004 Invitation to 3D vision Multiple View Stereo State-of-the-art: PMVS: http://grail.cs.washington.edu/software/pmvs/ Accurate, Dense, and Robust Multi-View Stereopsis, Y Furukawa and J Ponce, 2007 Benchmark: http://vision.middlebury.edu/mview/ A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms. SM Seitz, B Curless, J Diebel, D Scharstein, R Szeliski. 2006. Baseline: Multi-view stereo revisited. M Goesele, B Curless, SM Seitz. 2006. How to get the intrinsic parameters? • Auto-calibration Self-Calibration and Metric Reconstruction in spite of Varying and Unknown Internal Camera Parameters, M Pollefeys, R Koch and L Van Gool, 1998. http://mit.edu/jxiao/Public/software/autocalibrate/autocalibration_lin.m • Grid Search to look for the solution with minimal reprojection error for f=min_f:max_f do everything, then obtain reprojection error after bundle adjustment • • Optimize for this value in bundle adjustment Camera Calibration (with checkerboard) http://www.vision.caltech.edu/bouguetj/calib_doc/ • EXIF of JPEG file recorded from digital camera Read the code of Bundler to understand how to convert EXIF into focal length value http://phototour.cs.washington.edu/bundler/ Real World Applications • Streetview Reconstruction and Recognition http://vision.princeton.edu/projects/2009/ICCV/ http://vision.princeton.edu/projects/2009/TOG/ • • • Photo Tourism http://phototour.cs.washington.edu/ Microsoft Photosynth http://photosynth.net/ 2d3, boujor (Matchmovers) and movies http://www.2d3.com/ • http://www.vicon.com/boujou/ Robotics: SLAM http://openslam.org/ Simultaneous Localization And Mapping
