Derivation of Tranformation Parameter
Computation for a 2D/3D Coordinates System
from Laser Scanner Coordinates to Camera Coordinates
Conventions:
In a 3D coordinate system, Xs, Ys, Zs will be used for object coordinates in the scanner
coordinate system. This is the coordinate system from which the transformation is made.
Xc ,Yc ,Zc will be used for the object coordinates expressed in the camera coordinate
system at the location of the image plane in the camera (CCD sensor, Zc = f). This is the
coordinate system to which the scanner coordinates are transformed. Xi, Yi, Zi will be
used for the object coordinates transformed to a coordinate system which has as its
origin, the camera coordinate system origin. Xi, Yi, Zi are unaffected by the camera lens
optics. x, y, z will be used for the translation needed to place the origin of the scanner
coordinate system at the origin of the camera coordinate system.
In a 3D coordinate system Omega (ω) will describe rotation about the X-axis, Phi (Ф)
will describe rotation about the Y-axis, and Kappa (κ) will describe rotation about the Zaxis.
I
Derivation of the 3D transformation matrix
A right-hand coordinate system with right hand rotation will be used.
0
0 
1

Mx = 0 cos ω sin ω  Rotation about the camera x-axis
0 − sin ω cos ω 
cos φ 0 − sin φ 
My =  0
1
0 
 sin φ 0 cos φ 
 cos κ sin κ 0
Mz = − sin κ cos κ 0
 0
0
1
Rotation about the camera y-axis
Rotation about the camera z-axis
A single rotation about the X-axis would have the form shown below.
 X i  1
0
0   X s + x
 X s + x
X i 
 Y  = Mx *  Y + y   Y  = 0 cos ω sin ω   Y + y 
 i 


 s
 s
 i
 Z  0 − sin ω cos ω   Z s + z 
 Z s + z 
 Z i 
 
Rotation about the three camera axes can be done in any order, but must then be done
consistently throughout the computations in the same order. Generally, matrix
multiplication is not transitive, that is [M] * [N] does not equal [N] * [M]. Rotation
about the X-axis followed by rotations about the Y and Z-axes is the common sequence
Lionel White 5/31/2007
1
that is used. Thus, a transformation from the scanner coordinate system to the camera
coordinate system is accomplished with the following matrix operations.
The full transformation would have the form shown below.
 X s + x
X i 
 Y  = Mz * My * Mz  Y + y 
or, expressed in complete form below

 s
 i
 Z s + z 
 Z i 
 X i   cos κ
 Y  = − sin κ
 i 
 Z i   0
sin κ
cos κ
0
0 cos φ 0 − sin φ  1
0
0   X s + x




0  0
1
0  0 cos ω sin ω   Ys + y 
1  sin φ 0 cos φ  0 − sin ω cos ω   Z s + z 
The combined transformation matrix M can be expressed as M = Mz * My * Mx . And
the transformation from scanner to camera coordinates can be expressed as
 X s + x
X i 
Y  = M Y + y 

 s
 i
 Z s + z 
 Z i 
Multiplying out the rotation matrices Mz, My, and Mx is shown below
 cos φ cos κ cos ω sin κ + sin ω sin φ cos κ sin ω sin κ − cos ω sin φ cos κ 
M = − cos φ sin κ cos ω cos κ − sin ω sin φ sin κ sin ω cos κ + cos ω sin φ sin κ 

 sin φ
cos ω cos φ
− sin ω cos φ
Each element of the matrix is named according to it's row/column position.
m11 = cos φ cos κ
m12 = cos ω sin κ + sin ω sin φ cos κ
m13 = sin ω sin κ − cos ω sin φ cos κ
m 21 = − cos φ sin κ
m 22 = cos ω cos κ − sin ω sin φ sin κ
m 23 = sin ω cos κ + cos ω sin φ sin κ
m 31 = sin φ
m 32 = − sin ω cos φ
m 33 = cos ω cos φ
 X i   m11 m12 m13   X s + x 
 Y  = m21 m22 m23  Y + y 

 s
 i 
 Z i   m31 m32 m33  Z s + z 
Lionel White 5/31/2007
2
The camera coordinate system has the origin located at the focal point of the lens. The Zaxis of the camera is parallel to the axis of the lens system and is perpendicular to the
CCD sensor in the camera. The positive direction of the Z-axis is toward the object being
photographed. Since the CCD sensor is located behind the focal point the CCD sensor is
located at a Z-coordinate of (–f ), [f: focal length of the lens]. The CCD sensor
represents the X-Y plane of the camera coordinate system and is orthogonal to the Z-axis.
Note that with the CCD sensor (X-Y plane) located behind the focal point, inversion in
the X and Y dimensions occurs. This is corrected by virtually moving the CCD sensor a
distance (f) in front of the focal point.
The above transformation matrix transforms the scanner coordinates to camera
coordinates but does not scale the dimensions to the scale of the CCD sensor. That is, the
dimensional reduction that occurs due to the camera lens is not reflected in the
transformation matrix at this point.
An image dimension is related to the object dimension by the ratio of the focal length of
the lens and the distance to the object. Using the relationship of similar triangles as
shown below, Xc = (f/Zi)*Xs, where f is the focal length of the camera, Zi is the distance
from the focal point of the camera to the object and Xs is the dimension of the object.
The Xc and Yc values will be scaled by the ratio f/Zi. Thus, the matrix equation above
when translated to the plane of the CCD sensor can be expressed as
f   f


0 0
X i Z   Z
i
X c  
  m11 m12 m13   X s + x 
  i
f
Y  = Y f  =  0
0 m 21 m 22 m 23  Ys + y 
 c  iZ  

Zi
i



 Z i  


Z
0
0 1  m31 m32 m33  Z s + z 

 i  

 

The above complete transformation matrix is expressed in equation form as
Xc =
f
(m11( X s + x ) + m12(Ys + y ) + m13(Z s + z ))
Zi
Yc =
f
(m21( X s + x ) + m22(Ys + y ) + m23(Z s + z ))
Zi
Z i = m31( X s + x ) + m32(Ys + y ) + m33(Z s + z )
Lionel White 5/31/2007
3
What are called the collinearity equations are formed by substituting for Zi in the
equations for Xc and Yc. The collinearity equations are
f (m11( X s + x ) + m12(Ys + y ) + m13(Z s + z ))
Xc =
m31( X s + x ) + m32(Ys + y ) + m33(Z s + z )
Yc =
f (m21( X s + x ) + m22(Ys + y ) + m23(Z s + z ))
m31( X s + x ) + m32(Ys + y ) + m33(Z s + z )
It will be necessary to negate the values above to make the camera and the scanner
coordinate systems compatible. This will be done at the end of the derivation.
The following assignments are made in order to simplify the notation later, refer to the
numerators and denominators of the above equations.
U = f (m11( X s + x ) + m12(Ys + y ) + m13(Z s + z ))
V = f (m21( X s + x ) + m22(Ys + y ) + m23(Z s + z ))
W = m31( X s + x ) + m32(Ys + y ) + m33(Z s + z )
Xc and Yc are the coordinates of a point on the camera CCD sensor with the origin of the
coordinate system at the center of the CCD sensor. These are not UV coordinates. UV
coordinates have the origin at the upper left corner of a photograph and have values
between 0 and 1.0. Xc and Yc have dimensions which relate to the size of the CCD sensor
and range from -1/2 the sensor width to +1/2 the sensor width and from -1/2 the sensor
height to +1/2 the sensor height. In solving for the transformation parameters (ω, Φ, κ, x,
y, and z) four points are identified common between the photograph and the 3D model.
Thus we have four set of values for Xc, Yc, Xs, Ys, and Zs. We do not have values for Zi in
the camera coordinate system. By substituting for Zi in the equations for Xc and Yc, the
need to solve for Zi is eliminated.
Critical mathmatical functions to solve for the transformation parameters
A Taylor's series expansion is used to linearize the collinearity equations so that an
interative least squares method can be used to solve for the parameters. Only the first
two members of the Taylor's series are used.
f ( x ) = f (a ) + ( x − a ) f ' (a )...
Thus
∂f (a )
∂f (a )
∂f (a )
∂f (a )
∂f (a )
∂f (a )
dω +
dφ +
dκ +
dx +
dy +
dz
∂z
∂ω
∂φ
∂κ
∂x
∂y
∂f (b )
∂f (b )
∂f (b )
∂f (b )
∂f (b )
∂f (b )
f (Yc ) = f (b ) +
dω +
dφ +
dκ +
dx +
dy +
dz
∂ω
∂φ
∂κ
∂x
∂y
∂z
f ( X c ) = f (a ) +
In general form, the derivative of any function f(x) where f(x) = u/v is given as
d  u  1 du
u dv
−
 =
dx  v  v dx v ^ 2 dx
Lionel White 5/31/2007
4
The above expressions for f(Xc) and f(Yc) are linear equations and can be used in a least
squares method of estimating the transformation parameters.
An example of the partial derivative in the solution f(x) is provided below.
∂f (a )
U ∂W
∂  U  1 ∂U
−
=
 =
∂ω
∂ω  W  W ∂ω W ^ 2 ∂ω
There will be a similar expression for the derivative with respect to each of the six
transformation parameters (ω, Φ, κ, x, y, z).
The Taylor's series expansion for the collinearity equations are expressed in part below.
 1 ∂U
U ∂W 
U ∂W 
 1 ∂U
−
−
 o dφ .....
f ( X c ) = f a (ω o , φ o , κ o , xo , y o , z o ) + 
 o dω + 
 W ∂ω W ^ 2 ∂ω 
 W ∂φ W ^ 2 ∂φ 
 1 ∂V
V ∂W 
V ∂W 
 1 ∂V
 o dφ .....
f (Yc ) = f b (ω o , φ o , κ o , x o , y o , z o ) + 
−
−
 o dω + 
 W ∂ω W ^ 2 ∂ω 
 W ∂φ W ^ 2 ∂φ 
U
V
And
Where f a (ω o , φ o , κ o , xo , y o , z o ) =
f b (ω o , φ o , κ o , x o , y o , z o ) =
W
W
The following expressions will be used to simplify the Taylor's series:
f ∂U
fU ∂W
−
B11 =
W ∂ω W ^ 2 ∂ω
f ∂U
fU ∂W
−
B12 =
W ∂φ W ^ 2 ∂φ
f ∂U
fU ∂W
−
B13 =
W ∂κ W ^ 2 ∂κ
f ∂U
fU ∂W
−
B14 =
W ∂x W ^ 2 ∂x
f ∂U
fU ∂W
−
B15 =
W ∂y W ^ 2 ∂y
f ∂U
fU ∂W
−
B16 =
W ∂z W ^ 2 ∂z
f ∂V
fV ∂W
−
B 21 =
W ∂ω W ^ 2 ∂ω
f ∂V
fV ∂W
−
B 22 =
W ∂φ W ^ 2 ∂φ
f ∂V
fV ∂W
−
B 23 =
W ∂κ W ^ 2 ∂κ
f ∂V
fV ∂W
−
B 24 =
W ∂x W ^ 2 ∂x
f ∂V
fV ∂W
−
B 25 =
W ∂y W ^ 2 ∂y
B 26 =
f ∂V
fV ∂W
−
W ∂z W ^ 2 ∂z
Lionel White 5/31/2007
5
The Taylor's series expressed in simplified form:
f ( X c ) = f a (ω o , φ o , κ o , xo , y o , z o ) + (B11 ∆ω + B12 ∆φ + B13 ∆κ + B14 ∆x + B15 ∆y + B16 ∆z ) o
f (Yc ) = f b (ω o , φ o , κ o , xo , y o , z o ) + (B21 ∆ω + B22 ∆φ + B23 ∆κ + B24 ∆x + B25 ∆y + B26 ∆z ) o
And then expressed in matrix form
 X c   f a (ω o , φ o , κ o , xo , y o , z o )  B11
 Y  =  f (ω , φ , κ , x , y , z ) +  B
 c   b o o o o o o   21
B12
B13
B14
B15
B22
B23
B24
B25
∆ω 
 ∆φ 
 
B16   ∆κ 
 
B26   ∆x 
 ∆y 
 
 ∆z 
Expressed in the form necessary for an iterative least squares computation
 ∆ω 
 ∆φ 
 
 X c   f a (ω o , φ o , κ o , xo , y o , z o )  B11 B12 B13 B14 B15 B16   ∆κ 
 
 Y  −  f (ω , φ , κ , x , y , z ) =  B
 c   b o o o o o o   21 B22 B23 B24 B25 B26   ∆x 
 ∆y 
 
 ∆z 
Each of the partial derivatives B11
-> B26 will be derived.
 X s + x
cos ω sin κ + sin ω sin φ cos κ sin ω sin κ − cos ω sin φ cos κ ] Ys + y 
 Z s + z 
 X s + x
U ′(ω ) = [0 − sin ω sin κ + cos ω sin φ cos κ cos ω sin κ + sin ω sin φ cos κ ] Ys + y 
 Z s + z 
∂
U ′(ω ) =
[cos φ cos κ
∂ω
 X s + x
U ′(ω ) = [0 − m13 m12] Ys + y 
 Z s + z 
Lionel White 5/31/2007
6
∂
[cos φ cos κ
U ′(φ ) =
∂φ
U ′(φ ) = [− sin φ cos κ
 X s + x
cos ω sin κ + sin ω sin φ cos κ sin ω sin κ − cos ω sin φ cos κ ] Ys + y 
 Z s + z 
 X s + x
sin ω cos φ cos κ − cos ω cos φ cos κ ] Ys + y 
 Z s + z 
 X s + x
cos ω sin κ + sin ω sin φ cos κ sin ω sin κ − cos ω sin φ cos κ ] Ys + y 
 Z s + z 
 X s + x
U ′(k ) = [− cos φ sin κ cos ω cos κ − sin ω sin φ sin κ sin ω cos κ + cos ω sin φ sin κ ] Ys + y 
 Z s + z 
 X s + x
U ′(k ) = [m21 m22 m23] Ys + y 
 Z s + z 
∂
[cos φ cos κ
U ′(k ) =
∂k
 X s + x
cos ω sin κ + sin ω sin φ cos κ sin ω sin κ − cos ω sin φ cos κ ] Ys + y 
 Z s + z 
1 
U ′( x ) = [cos φ cos κ cos ω sin κ + sin ω sin φ cos κ sin ω sin κ − cos ω sin φ cos κ ]0
0
U ′( x ) = cos φ cos κ = m11
∂
U ′( x ) = [cos φ cos κ
∂x
Likewise
U ′( y ) = cos ω sin κ + sin ω sin φ cos κ = m12
U ′( z ) = sin ω sin κ − cos ω sin φ cos κ = m13
Lionel White 5/31/2007
7
∂
V ′(ω ) =
[− cos φ sin κ
∂ω
cos ω cos κ − sin ω sin φ sin κ
V ′(ω ) = [0 − sin ω cos κ − cos ω sin φ sin κ
 X s + x
sin ω cos κ + cos ω sin φ sin κ ] Ys + y 
 Z s + z 
 X s + x
cos ω cos κ − sin ω sin φ sin κ ] Ys + y 
 Z s + z 
 X s + x
V ′(ω ) = [0 − m23 m22] Ys + y 
 Z s + z 
 X s + x
cos ω cos κ − sin ω sin φ sin κ sin ω cos κ + cos ω sin φ sin κ ] Ys + y 
 Z s + z 
 X s + x
− sin ω cos φ sin κ cos ω cos φ sin κ ] Ys + y 
 Z s + z 
∂
[− cos φ sin κ
V ′(φ ) =
∂φ
V ′(φ ) = [sin φ sin κ
 X s + x
cos ω cos κ − sin ω sin φ sin κ sin ω cos κ + cos ω sin φ sin κ ] Ys + y 
 Z s + z 
 X s + x
V ′(k ) = [− cos φ cos κ − cos ω sin κ − sin ω sin φ cos κ − sin ω sin κ + cos ω sin φ cos κ ] Ys + y 
 Z s + z 
 X s + x
V ′(k ) = [− m11 − m12 − m13] Ys + y 
 Z s + z 
∂
[− cos φ sin κ
V ′(k ) =
∂k
 X s + x
cos ω cos κ − sin ω sin φ sin κ sin ω cos κ + cos ω sin φ sin κ ] Ys + y 
 Z s + z 
1
V ′( x ) = [− cos φ sin κ cos ω cos κ − sin ω sin φ sin κ sin ω cos κ + cos ω sin φ sin κ ]0 = − cos φ sin κ
0
V ′( x ) = m21
V ′( y ) = m22
V ′( z ) = m23
∂
V ′( x ) = [− cos φ sin κ
∂x
Lionel White 5/31/2007
8
∂
W ′(ω ) =
[sin φ
∂ω
− sin ω cos φ
 X s + x
cos ω cos φ ] Ys + y 
 Z s + z 
 X s + x
W ′(ω ) = [0 − cos ω cos φ − sin ω cos φ ] Ys + y 
 Z s + z 
 X s + x
W ′(ω ) = [0 − m33 m32] Ys + y 
 Z s + z 
 X s + x
− sin ω cos φ cos ω cos φ ] Ys + y 
 Z s + z 
 X s + x
∂
W ′(φ ) =
[cos φ sin ω sin φ − cos ω sin φ ] Ys + y 
∂φ
 Z s + z 
∂
[sin φ
W ′(ω ) =
∂ω
∂
[sin φ
W ′(κ ) =
∂κ
− sin ω cos φ
 X s + x
cos ω cos φ ] Ys + y 
 Z s + z 
 X s + x
W ′(κ ) = [0 0 0] Ys + y  = 0
 Z s + z 
 X s + x
− sin ω cos φ cos ω cos φ ] Ys + y 
 Z s + z 
1
W ′( x ) = [sin φ − sin ω cos φ cos ω cos φ ]0
0
W ′( x ) = sin φ = m31
W ′( x ) = − sin ω cos φ = m32
∂
W ′( x ) = [sin φ
∂x
W ′( x ) = cos ω cos φ = m33
Lionel White 5/31/2007
9
The following are the complete expressions for the coefficients B11 through B26.
Note that B11 through B26 are evaluated with the parameters found during the previous
iteration.
f
f *U * S
B11 =
* [0 − m13 m12]* S −
* [0 − m33 m32] * S
W *S
(W * S ) 2
f
B12 =
* [− sin φ cos κ sin ω cos φ cos κ − cos ω cos φ cos κ ] * S
W *S
f *U * S
−
[cos φ sin ω sin φ − cos ω sin φ ]* S
(W * S ) 2
f
B13 =
* [m21 m22 m23]* S
W *S
f
f *U * S
B14 =
* m11 −
* m31
W *S
(W * S ) 2
f
f *U * S
B15 =
* m12 −
* m32
W *S
(W * S ) 2
f
f *U * S
B16 =
* m13 −
* m33
W *S
(W * S ) 2
f
f *V * S
B 21 =
* [0 − m23 m22]* S −
* [0 − m33 m32]* S
W *S
(W * S ) 2
f
B 22 =
* [sin φ sin κ − sin ω cos φ sin κ cos ω cos φ sin κ ] * S
W *S
f *V * S
[cos φ sin ω sin φ − cos ω sin φ ]* S
−
(W * S ) 2
f
B 23 =
* [− m11 − m12 − m13] * S
W *S
f
f *V * S
B 24 =
* m21 −
* m31
W *S
(W * S ) 2
f
f *V * S
B 25 =
* m22 −
* m32
W *S
(W * S ) 2
f
f *V * S
B 26 =
* m23 −
* m33
W *S
(W * S ) 2
The final desired form is F = B∆
Uo 

X c − f * W 
o

F=
V
o

Y − f *
 c
Wo 
Lionel White 5/31/2007
10
B
B =  11
 B21
B12
B22
B13
B23
B14
B24
B15
B25
B16 
B26 
o
 ∆ω 
 ∆φ 
 
 ∆κ 
∆= 
 ∆x 
 ∆y 
 
 ∆z 
The form required for the iterative solution of the transformation parameters is
Uo 

X c − f * W  B
o
 =  11

V
 Y − f * o   B21
 c
Wo 
B12
B13
B14
B15
B22
B23
B24
B25
 ∆ω 
 ∆φ 
 
B16   ∆κ 

o
B26   ∆x 
 ∆y 
 
 ∆z 
F = B∆
Using matrix algebra, the unknowns contained in the matrix ∆ can be solved for with the
following equation.
∆ = (B ′ * B ) * B ′ * F
−1
At the completion of each iteration, the unknowns ω, Φ, κ, x, y, z are computed as shown
below.
ω i  ω i −1  ∆ω 
 φ   φ   ∆φ 
 i   i −1   
κ i  κ i −1   ∆κ 
+ 
 =
x
x
i
i
−
1
  ∆x 
  
 y i   y i −1   ∆y 
  
  
 z i   z i −1   ∆z 
Then, the next iteration uses the new values for ω, Φ, κ, x, y, z and they carry the
subscript "o".
The above equations are represented in the MatLab program named d23.m. These
equations will not produce the transformation parameters necessary to drape the
photographs over the TIN model of the Riegel Scanner point cloud. It is necessary to
negate the Zi values in order to achieve that transformation.
Lionel White 5/31/2007
11
The equations necessary for this transformation follow.
f *U * S
−f
B11 =
* [0 − m13 m12]* S +
* [0 − m33 m32] * S
W *S
(W * S ) 2
−f
B12 =
* [− sin φ cos κ sin ω cos φ cos κ − cos ω cos φ cos κ ] * S
W *S
f *U * S
+
[cos φ sin ω sin φ − cos ω sin φ ]* S
(W * S ) 2
−f
B13 =
* [m 21 m 22 m 23] * S
W *S
−f
f *U * S
B14 =
m11 +
* m31
W *S
(W * S ) 2
−f
f *U * S
B15 =
* m12 +
* m32
W *S
(W * S ) 2
−f
f *U * S
B16 =
* m13 +
* m33
W *S
(W * S ) 2
−f
f *V * S
B 21 =
* [0 − m 23 m 22]* S +
* [0 − m33 m32]* S
W *S
(W * S ) 2
−f
B 22 =
* [sin φ sin κ − sin ω cos φ sin κ cos ω cos φ sin κ ]* S
W *S
f *V * S
[cos φ sin ω sin φ − cos ω sin φ ]* S
+
(W * S ) 2
−f
B 23 =
* [− m11 − m12 − m13] * S
W *S
−f
f *V * S
B 24 =
* m 21 +
* m31
W *S
(W * S ) 2
−f
f *V * S
B 25 =
* m 22 +
* m32
W *S
(W * S ) 2
−f
f *V * S
B 26 =
* m 23 +
* m33
W *S
(W * S ) 2
 ∆ω 
 ∆φ 
Uo 

 
X c + f * W  B
 ∆κ 
B
B
B
B
B

12
13
14
15
16
o
 =  11
F=
F = B∆

o

 Y + f * Vo   B21 B22 B23 B24 B25 B26   ∆x 
 c
Wo 
 ∆y 
 
 ∆z 
Note that the values in the B- matrix here have been multiplied by -1. MatLab program
d23n.m provides the necessary transformation parameters.
Lionel White 5/31/2007
12