# 05-3Dto2DTransforms ```3D‐2D Coordinate Transforms
EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff
3D to 2D Perspective Transformation
•
We can project 3D points onto 2D with a matrix multiplication
•
Assuming that the 2D point is in homogeneous coordinates, we divide through by the last element
•
Recall perspective projection (x = f X/Z, y = f Y/Z), so
 x1   f
  
 x2    0
x  0
 3 
0
f
0
X
 1 0 0 0    X 

 Y   
 0 1 0 0     Y 
 0 0 1 0  Z   Z 

 1   
 
X  X /Z
  

~
x Y  Y /Z 
Z  1 
  

X
0  1 0 0 0  

 Y 
0  0 1 0 0  , x  x1 / x3 , y  x2 / x3
Z




1  0 0 1 0  
If f=1, we sometimes call (x,y) 1
“normalized image coordinates”
 
EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff
Intrinsic Camera Matrix
•
We can capture all the intrinsic camera parameters in a matrix K
 f / sx

K  0
 0

0
f / sy
0
cx 
 fx


c y  or K   0
0
1 

0
fy
0
cx 

cy 
1 
•
•
•
•
Recall that if f is the focal length in mm, then sx,sy is the size of a pixel in mm
Alternatively, can just use fx,fy in pixels
The optical center of the image is at pixel location cx, cy
So to project 3D points in camera coordinates onto the pixel image
C
X
1
0
0
0
 x1 

  
 

 Y 

0
1
0
0
x
K
 2

  ,
x 
0 0 1 0  Z 
 3

 1
 
 x   x1 / x3 
  

/

y
x
x
   2 3
1  1 
  

EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff
Extrinsic Camera Matrix
•
If 3D points are in world coordinates, we first need to transform them to camera coordinates
C

C
C
W
WR
P WH P
 0
•
tWorg  W
 P
1 
We can write this as an extrinsic camera matrix, that does the rotation and translation, then a projection from 3D to 2D
M ext 
•
C

C
W
R
C
tWorg

 r11

  r21
r
 31
r12
r13
r22
r23
r32
r33
tX 

tY 
tZ 
Also note
M ext 

C
W
R
C
 
tWorg 
C
W
R  WC R W t Corg

EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff
Complete Perspective Projection
• Projection of a 3D point WP in the world to a point in the pixel image (xim,yim)
W
 x1 
 
 x2   K M ext
x 
 3
X
 
Y 
 Z ,
 
1
 
xim  x1 / x3 , yim  x2 / x3
EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff
Example
• If the robot in the earlier example had a camera instead of a range sensor, what pixel would P project to?
• Assume f=512 pix, (cx,cy)=(256,256)
EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff
H_V_W = [ 1 0 0
0 -1 0
0 0 -1
0 0 0
H_S_V = [ 0
1
0
0
0
0
1
0
5;
0;
1;
1]
1
0
0
0
1;
0;
-2;
1]
P_W = [ 16; 0; -1; 1];
K = [512 0
0 512
0
0
256;
256;
1 ];
R_C_V = [ 0 0 1;
1 0 0;
0 1 0];
R_V_C = R_C_V';
R_W_V = [1 0 0;
0 -1 0;
0 0 -1]';
R_W_C = R_V_C * R_W_V;
tCorg_V = [1; 0; -2; 1];
tCorg_W = H_V_W * tCorg_V;
tCorg_W = tCorg_W(1:3);
Mext = [ R_W_C
-R_W_C * tCorg_W ];
p = K * Mext * P_W;
p = p / p(3)
EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff
Weak Perspective
•
•
Sometimes it is better to use an approximation to perspective projection, called “weak” projection or scaled orthography
This works if the average depth Zavg to an object is much larger than the variation in depth within the object
– Instead of x = f X/Z, y = f Y/Z
– use x = f X/Zavg, y = f Y/Zavg
C
 x1   f x
  
 x2    0
x   0
 3 
•
0
fy
0
c x  1 0 0 0 


c y  0 1 0 0 
1  0 0 0 Z avg 
X
 
Y 
 Z ,
 
1
 
 x   x1 / x3 
  

/
y

x
x
   2 3
1  1 
  

This makes the image coordinates (x,y) a linear function of the 3D coordinates (X,Y,Z)
EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff
Special Case
•
Small planar patch
– Often we want to track a small patch on an object
– We want to know how the image of that patch transforms as the object rotates
•
Assume
– Size of patch small compared to distance ‐&gt; weak perspective
– Rotation is small ‐&gt; small angle approximation
– Patch is planar
•
It can be shown that the patch undergoes affine transformation
 xB   a11
  
 yB    a21
1  0
  
a12
a22
0
t x  x A 
 
t y  y A 

1 
 1 
EGGN 512 Computer Vision Colorado School of Mines, Engineering Division Prof. William Hoff
```