Exploiting Homography in
Camera-Projector Systems
Tal Blum
Jiazhi Ou
Dec 11, 2003
[Sukthankar, Stockton & Mullin. ICCV-2001]
Road Map






Introduction
Projector-Camera Homography
Automatic Keystone Correction
Lines Detection
Demo
Conclusion
Introduction




The Situation:
A presentation system with a computer and a
projector
The Problem:
The projected image is warped if the projector is
not aligned manually
The Solution:
Automatic keystone correction using a camera
The Assumption
Uncalibrated camera, uncalibrated projector, we
know the resolution of the screen
Projector-Camera
Homography
Presumption: Points on a plane (board)
Given
 x proji 


i
pi   y proj  ,


 1 
 xcam i 


i
qi   ycam  ,


 1 
we want to estimate H:
qi  H * pi
Projector-Camera
Homography
Linear Least-Squares:
Construct a 2N*9 matrix (N>=4):
 p1T

0

L
 ...

 0
0
 p1
T
...
 pN
T
1
T
xcam * p1 
1
T 
ycam * p1 

...
N
T
ycam * p N 
h equals to the eigenvector of L’*L
corresponding to the smallest eigenvalue
Automatic Keystone
Correction
Overview:
1. Compute
Projector-Camera
Homography
2. Compute
World-Camera
Homography
7.
Warp
Image
3. Compute
World-Projector
Homography
4. Compute
New Projected
Area on Board
6. Compute
Projector-Image
Homography
5. Compute
World-Image
Homography
Compute ProjectorCamera Homography
If we know p1, p2, p3, p4,
we can can estimate H1:
 x proj   xcam 

 

H1 *  y proj    ycam 
 1   1 

 

P1
P4
P3
P2
1 1 1024 1024 


H1 * 1 768 768
1    p1
1 1

1
1


p2
p3
p4 
Compute World-Camera
Homography
If we know p1, p2, p3, p4,
we can can estimate H2:
 xworld   xcam 

 

H 2 *  yworld    ycam 
 1   1 

 

P1
P4
P2
1
10000 10000 
1


H 2 * 1 10000 10000 10000    p1
1

1
1
1


P3
p2
p3
p4 
Compute World-Projector
Homography
 x proj   xcam 

 

H1 *  y proj    ycam 
 1   1 

 

 xworld   xcam 

 

H 2 *  yworld    ycam 
 1   1 

 

 x proj   xworld 

 

1
H 2 * H1 *  y proj    yworld 
 1   1 

 

1
H 3  H 2 * H1
Automatic Keystone
Correction
Overview:
1. Compute
Projector-Camera
Homography H1
2. Compute
World-Camera
Homography H2
3. Compute
World-Projector
Homography H3
4. Compute
New Projected
Area on Board
Compute New Projected Area
on Board
1. Find old projected area:
p
old _ area
1
, p2
old _ area
, p3
old _ area
, p4
old _ area

1 1 1024 1024 


 H 3 * 1 768 768
1 
1 1

1
1


Compute New Projected Area
on Board
2. Find a largest rectangle in the old
projected area:
old _ area
p1
p1
new _ area
p2
p2
new _ area
p4
new _ area
p3
new _ area
p4
old _ area
p3
old _ area
old _ area
Compute World-Image
Homography
new _ area
new _ area
new _ area
new _ area
p3
p2
p4
Now we know p1
we can can estimate H4:
 ximage   xworld 

 

H 4 *  yimage    yworld 
 1   1 

 

1 1 1024 1024 


new _ area
H 4 * 1 768 768
1   p1
1 1
1
1 


p2
new _ area
p3
new _ area
p4
new _ area

Compute Projector-Image
Homography
 x proj   xworld 

 

H 3 *  y proj    yworld 
 1   1 

 

 ximage   xworld 

 

H 4 *  yimage    yworld 
 1   1 

 

 x proj   ximage 

 

1
H 4 * H 3 *  y proj    yimage 
 1   1 

 

1
H5  H 4 * H3
Automatic Keystone
Correction
Overview:
1. Compute
Projector-Camera
Homography H1
2. Compute
World-Camera
Homography H2
7. Warp Image:
For each pixel in
projector, find the
corresponding pixel in
the image using H5
3. Compute
World-Projector
Homography H3
4. Compute
New Projected
Area on Board
6. Compute
Projector-Image
Homography H5
5. Compute
World-Image
Homography H4
Road Map






Introduction
Projector-Camera Homography
Automatic Keystone Correction
Lines Detection
Demo
Conclusion
Lines Detection


We used our
implementation for the
lines detection
Problems in lines
detection include:
– Noise due to low quality
camera
– Need to be invariant to
different room settings
and different lighting
conditions
– The camera might be in
different distances from
the screen.
Lines Detection
implementation

Stages
– Brightness Normalization
– Canny edge detection
for k=1 to 4
Compute the parameter distribution
Smooth the parameter space
Find the point in the k’th parameter space (R_k,Theta_k)
that has the maximal value and satisfy constraints.
Remove the points belonging to the lines found so far
from the parameter distribution
end
Brightness Normalization

Adjusting the intensity by linear
transformation so that the intensity range
would be [0,1]
Canny edge detection
For detecting the
projection coordinates


We use the
difference of an
image with a white
projection and an
image without it.
The canny image is
much cleaner &
easier to deal with.
Counting Over Parameter
Space



Lines are represented as
(R,Theta)
Count for each line how many
points go through it
Smooth with a Gaussian kernel
– Depends on distance from the
center

Sampling problems
– Lines more densely sampled near
the center
– Solution: Representing the points
relative to the center point &
Sample more densely
Representation Problems
Y
X
Choosing 4 lines



Iteratively choose 4 lines
Order the best lines by
their counts
Choose the best line that
satisfy constraints
– Constraints include that the
intersections are within the
image & that each line has
exactly 2 intersection with
the other lines

Weighting lines with
different angles differently
to correct for vertical lines
Finding the intersection
Road Map






Introduction
Projector-Camera Homography
Automatic Keystone Correction
Lines Detection
Demo
Conclusion
Conclusion




We built a presentation system that
corrects keystone automatically
We exploited camera-projector
homography
We implemented our own line
detection algorithm
The authors also use this homography
to define virtual buttons on the
projector screen
Thank You!
A502 Newell-Simon Hall
{blum,jiazhiou}@cmu.edu