E
Essential Matrix
16-385 Computer Vision (Kris Kitani)
Carnegie Mellon University
Recall:Epipolar constraint
p
x
o
l
0
x0
l
e
Potential matches for
e
o0
0
x lie on the epipolar line l
0
The epipolar geometry is an important concept for stereo vision
Task: Match point in left image to point in right image
Left image
Right image
How would you do it?
The epipolar constraint is an important concept for stereo vision
Task: Match point in left image to point in right image
Left image
Right image
Epipolar constrain reduces search to a single line
How do you compute the epipolar line?
Essential Matrix
E
The Essential Matrix is a 3 x 3 matrix that
encodes epipolar geometry
Given a point in one image,
multiplying by the essential matrix will tell us
the epipolar line in the second view.
Ex = l
0
X
0
x
x
o
e
e0
l
0
o0
Representing the …
Epipolar Line
ax + by + c = 0
e
x
l
in vector form
If the point
2
3
a
l=4 b 5
c
x is on the epipolar line l then
>
x l= ?
Epipolar Line
ax + by + c = 0
e
x
l
in vector form
If the point
2
3
a
l=4 b 5
c
x is on the epipolar line l then
>
x l=0
Recall: Dot Product
c=a⇥b
b
a
c·a=0
c·b=0
dot product of two orthogonal vectors is zero
vector representing the line is
normal (orthogonal) to the plane
2
3
a
l=4 b 5
c
x
>
x l= ?
o
vector representing the point x
is inside the plane
Therefore:
>
x l=0
So if
>
x l=0
and
Ex = l
0
then
0>
x Ex = ?
X
0
x
x
o
e
e0
l
0
o0
So if
>
x l=0
and
Ex = l
0
then
0>
x Ex = 0
X
0
x
x
o
e
e0
l
0
o0
Motivation
The Essential Matrix is a 3 x 3 matrix that
encodes epipolar geometry
Given a point in one image,
multiplying by the essential matrix will tell us
the epipolar line in the second view.
Essential Matrix vs Homography
What’s the difference between the essential matrix and a homography?
Essential Matrix vs Homography
What’s the difference between the essential matrix and a homography?
They are both 3 x 3 matrices but …
Essential Matrix vs Homography
What’s the difference between the essential matrix and a homography?
They are both 3 x 3 matrices but …
0
0
l = Ex
x = Hx
Essential matrix maps a
point to a line
Homography maps a
point to a point
Where does the Essential matrix come from?
X
x0
x
o
o
t
R, t
0
x = R(x
t)
0
X
x0
x
o
o
t
R, t
0
x = R(x
t)
Does this look familiar?
0
X
x0
x
o
o
t
R, t
0
x = R(x
t)
Camera-camera transform just like world-camera transform
0
X
x0
x
o
o
t
These three vectors are coplanar
0
x, t, x
0
X
x0
x
o
o
t
If these three vectors are coplanar
>
0
x, t, x
x (t ⇥ x) = ?
then
0
X
x0
x
o
o
t
If these three vectors are coplanar
>
0
x, t, x
x (t ⇥ x) = 0
then
0
Recall: Cross Product
Vector (cross) product
takes two vectors and returns a vector perpendicular to both
c=a⇥b
b
a
c·a=0
c·b=0
X
x0
x
o
o
t
If these three vectors are coplanar
(x
>
0
x, t, x
t) (t ⇥ x) = ?
then
0
X
x0
x
o
o
t
If these three vectors are coplanar
(x
>
0
x, t, x
t) (t ⇥ x) = 0
then
0
putting it together
rigid motion
0
x = R(x
coplanarity
t)
0>
(x
>
t) (t ⇥ x) = 0
(x R)(t ⇥ x) = 0
putting it together
rigid motion
0
x = R(x
coplanarity
t)
0>
(x
>
t) (t ⇥ x) = 0
(x R)(t ⇥ x) = 0
0>
(x R)([t⇥ ]x) = 0
Cross product
2
a 2 b3
a ⇥ b = 4 a 3 b1
a 1 b2
3
a 3 b2
a 1 b3 5
a 2 b1
Can also be written as a matrix multiplication
2
0
a ⇥ b = [a]⇥ b = 4 a3
a2
a3
0
a1
32
3
a2
b1
a 1 5 4 b2 5
0
b3
Skew symmetric
putting it together
rigid motion
0
x = R(x
coplanarity
t)
0>
(x
>
t) (t ⇥ x) = 0
(x R)(t ⇥ x) = 0
0>
(x R)([t⇥ ]x) = 0
0>
x (R[t⇥ ])x = 0
putting it together
rigid motion
0
x = R(x
coplanarity
t)
(x
0>
>
t) (t ⇥ x) = 0
(x R)(t ⇥ x) = 0
0>
(x R)([t⇥ ]x) = 0
0>
x (R[t⇥ ])x = 0
0>
x Ex = 0
putting it together
rigid motion
0
x = R(x
coplanarity
t)
(x
>
t) (t ⇥ x) = 0
0>
(x R)(t ⇥ x) = 0
0>
(x R)([t⇥ ]x) = 0
0>
x (R[t⇥ ])x = 0
0>
x Ex = 0
Essential Matrix
[Longuet-Higgins 1981]
properties of the E matrix
Longuet-Higgins equation
0>
x Ex = 0
(points in normalized coordinates)
properties of the E matrix
Longuet-Higgins equation
Epipolar lines
>
x l=0
0
l = Ex
0>
x Ex = 0
0> 0
x l =0
T
l=E x
(points in normalized coordinates)
0
properties of the E matrix
Longuet-Higgins equation
Epipolar lines
>
x l=0
x Ex = 0
0> 0
x l =0
0
l=E x
0>
Ee = 0
l = Ex
Epipoles
0>
e E=0
T
(points in normalized camera coordinates)
0
How do you generalize to
uncalibrated cameras?