advertisement

3D‐2D Coordinate Transforms EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff 3D to 2D Perspective Transformation • We can project 3D points onto 2D with a matrix multiplication • Assuming that the 2D point is in homogeneous coordinates, we divide through by the last element • Recall perspective projection (x = f X/Z, y = f Y/Z), so x1 f x2 0 x 0 3 0 f 0 X 1 0 0 0 X Y 0 1 0 0 Y 0 0 1 0 Z Z 1 X X /Z ~ x Y Y /Z Z 1 X 0 1 0 0 0 Y 0 0 1 0 0 , x x1 / x3 , y x2 / x3 Z 1 0 0 1 0 If f=1, we sometimes call (x,y) 1 “normalized image coordinates” EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff Intrinsic Camera Matrix • We can capture all the intrinsic camera parameters in a matrix K f / sx K 0 0 0 f / sy 0 cx fx c y or K 0 0 1 0 fy 0 cx cy 1 • • • • Recall that if f is the focal length in mm, then sx,sy is the size of a pixel in mm Alternatively, can just use fx,fy in pixels The optical center of the image is at pixel location cx, cy So to project 3D points in camera coordinates onto the pixel image C X 1 0 0 0 x1 Y 0 1 0 0 x K 2 , x 0 0 1 0 Z 3 1 x x1 / x3 / y x x 2 3 1 1 EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff Extrinsic Camera Matrix • If 3D points are in world coordinates, we first need to transform them to camera coordinates C C C W WR P WH P 0 • tWorg W P 1 We can write this as an extrinsic camera matrix, that does the rotation and translation, then a projection from 3D to 2D M ext • C C W R C tWorg r11 r21 r 31 r12 r13 r22 r23 r32 r33 tX tY tZ Also note M ext C W R C tWorg C W R WC R W t Corg EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff Complete Perspective Projection • Projection of a 3D point WP in the world to a point in the pixel image (xim,yim) W x1 x2 K M ext x 3 X Y Z , 1 xim x1 / x3 , yim x2 / x3 EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff Example • If the robot in the earlier example had a camera instead of a range sensor, what pixel would P project to? • Assume f=512 pix, (cx,cy)=(256,256) EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff H_V_W = [ 1 0 0 0 -1 0 0 0 -1 0 0 0 H_S_V = [ 0 1 0 0 0 0 1 0 5; 0; 1; 1] 1 0 0 0 1; 0; -2; 1] P_W = [ 16; 0; -1; 1]; K = [512 0 0 512 0 0 256; 256; 1 ]; R_C_V = [ 0 0 1; 1 0 0; 0 1 0]; R_V_C = R_C_V'; R_W_V = [1 0 0; 0 -1 0; 0 0 -1]'; R_W_C = R_V_C * R_W_V; tCorg_V = [1; 0; -2; 1]; tCorg_W = H_V_W * tCorg_V; tCorg_W = tCorg_W(1:3); Mext = [ R_W_C -R_W_C * tCorg_W ]; p = K * Mext * P_W; p = p / p(3) EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff Weak Perspective • • Sometimes it is better to use an approximation to perspective projection, called “weak” projection or scaled orthography This works if the average depth Zavg to an object is much larger than the variation in depth within the object – Instead of x = f X/Z, y = f Y/Z – use x = f X/Zavg, y = f Y/Zavg C x1 f x x2 0 x 0 3 • 0 fy 0 c x 1 0 0 0 c y 0 1 0 0 1 0 0 0 Z avg X Y Z , 1 x x1 / x3 / y x x 2 3 1 1 This makes the image coordinates (x,y) a linear function of the 3D coordinates (X,Y,Z) EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff Special Case • Small planar patch – Often we want to track a small patch on an object – We want to know how the image of that patch transforms as the object rotates • Assume – Size of patch small compared to distance ‐> weak perspective – Rotation is small ‐> small angle approximation – Patch is planar • It can be shown that the patch undergoes affine transformation xB a11 yB a21 1 0 a12 a22 0 t x x A t y y A 1 1 EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff