Simultaneous surveillance camera calibration
and foot-head homology estimation from
human detection1
Author : Micusic & Pajdla
Presenter : Shiu, Jia-Hau
Advisor : Wang, Sheng-Jyh
1. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Outline
•
•
•
•
Introduction
Human Detection
Foot-head homology estimation
Conclusion
Introduction
• This paper uses people to calibrate the camera
• Human contour detection (green)
• Refined human detection with camera calibration
parameters (blue)
• Foot-head homology(o:foot,x:head)
Concept
• Objects are human
• Estimate camera parameters by observing a person
standing at several positions
3-D scene
2-D projection Image
System Flow
Sequential Images
Human Detection
Foot-head
Homology
Estimation
Output
System Flow
Sequential Images
Human Detection
Foot-head
Homology
Estimation
Output
Background
• Shape-based detector(Global search)
– Detection rate drop significantly in presence of
occluded humans
• Part-based detector(Local search)
C. Beleznai and H. Bischof. ,“Fast Human Detection in Crowded Scenes by Contour
Integration and Local Shape Estimation”, In CVPR,2009.
Background
Left - Shape based :
Template matching with head and body
Right - Part based :
Obtain foreground image by background
subtraction
Segmentation of detected human
Result : Contour template
Human Detection
• Line edges model a human
• Offline: Create around 1000 human contours based
on 3D model and moving and rotating camera
Draw foot-head lines in
one image
2-D Image
3-D scene
System Flow
Sequential Images
Human Detection
Foot-head
Homology
Estimation
Output
Background : Camera Model
u  zc K *[ R t ]X
 xw 
u 
y 
 v   z K *[ R t ]  w  , u is camera point , X is 3-D point
  c
 zw 
 1 
 
1
Intrinsic parameters
Extrinsic parameters
 x  u0 
K=  0  y v0 
 0 0 1 
 x =f * mx  y =f * my
Pc  [ R t ]
Homography matrix
 xw 
 xw 
u 
y 
 y   H11
 v   z K *[ R t ]  w   H *  w    H
3*4
  c
 zw 
 zw   21
 1 
 
   H 31
1
1
H12
H 22
H 32
H13
H 23
H 33
 xw 
H14   
yw 


H 24 
 zw 
H 34   
1
H11 xw  H12 yw  H13 zw  H14
H 21 xw  H 22 yw  H 23 z w  H14
u
, v
H 31 xw  H 32 yw  H 33 zw  H 34
H 31 xw  H 32 yw  H 33 zw  H 34
[ xw yw zw 1 0 0 0 0 -u  xw -u  yw -u  zw -u]* h  0
[0 0 0 0 x w yw zw 1 -v  xw -v  yw -v  zw -v]* h  0
One pair(2D-3D) of points
2 equation
, where h  [ H11 H12 H13 H14 H 21 H 22 H 23 H 24 H 31 H 32 H 33 H 34 ]T
11 DOF
Simple Calibration Example
• Measure 3-D position of special object points
in 3-D scene
z
Correspond to camera 2-D point
(u1,v1)
y
(30,30,40)
(0,0,0)
(0,30,0)
(u2,v2)
x
Foot-head Homology Estimation
•
•
•
•
1. Camera model : Shifted Homographies
2. Focal length, Rotation, Translation
3. Quadratic Eigenvalue Problem(QEP)
4. Foot-head Homology
Camera model
•
• Extrinsic parameters rotation R and translation t
Camera Parameters
• Assumptions intrinsic parameters
– Square pixels
– No principal point offset : Image coordinate at
center point (principal point)
– No skew : angle of horizon axis and vertical axis =
y
90’
90’
• Intrinsic parameters K = |f 0 0|
|0 f 0|
|0 0 1|
x
(x3,y3,z0)
z
(x1,y1,z0)
(x2,y2,z0)
y (x3,y3,0)
(x1,y1,0)
x
(x2,y2,0)
If x1,x2,x3,y1,y2,y3 are known
Six points => 12 equation
Compute homography of H
(x3,y3,z0)
z
If x1,x2,x3,y1,y2,y3 are unknown
How to find homography of H?
(x1,y1,z0)
(x2,y2,z0)
y (x3,y3,0)
(x1,y1,0)
x
(x2,y2,0)
(x3,y3,z0)
(0,0,0) & (0,0,z0) two point
are known 4 equation
z
(0,0,z0)
(x2,y2,z0)
(x3,y3,0)
y
(0,0,0)
(x2,y2,0)
x
(x3,y3,z0)
z
(0,0,z0)
z
(0,0,z0)
(x3,y3,0)
y
(0,0,0)
(0,0,0)
x
x1
y1
x1 = x+dx1
y1 = y+dy1
z
(0,0,z0)
z
(0,0,z0)
z
(0,0,z0)
y2
x2 = x+dx2
y2 = y+dy2
(0,0,0)
y
(0,0,0) x2
(0,0,0)
x
x1
y1
x1 = x+dx1
y1 = y+dy1
Shifted Homographies
 x  dx 
 x
u 
 y  dy 
 y
v    K R t 
   K  r1 r 2 r 3 r1 dx  r 2 dy  t   


 
 z 
z
 1 


 
 1 
1
only pick two points , foot and head =>x  0, y  0
u 
 v    K r 3 r1 dx  r 2 dy  t  z 

 
 
1

 1 
3+3K unknown,
Dof = 3+3K-1
unknown
K=1
6
K=2
9
K=3
12
add equation
4
8
12
Shifted Homographies
• The 3D point X = (x, y, z,1) can be simplified
assuming x = 0
• ri : is the i-th column of R
• 6+3K unknowns, K : number of detections
Shifted Homographies
•
•
Finding Homographies
• This equation is extended with all the known point
correspondences to form this equation:
M contains all the point correspondences
h contains h1, h2 and the unknown h3 of the
homographies
h is fixed for standard
camera calibration
[ xw yw zw 1 0 0 0 0 -u  xw -u  yw -u  zw -u]* h  0
[0 0 0 0 x w yw zw 1 -v  xw -v  yw -v  zw -v]* h  0
Focal Length、Rotation and Translation
•
•
• form 3 equations
Where
K=1
K=2
equation
3
6
unknown
4
6
Minimum Solution : two detectors case
• The equations in (7) give
Six equation with six unknown
Overdetermined Solution
• More than two homologies :
solvable as a Quadratic Eigenvalue Problem (QEP)
• Find scalars λ and nonzero vectors x, satisfying
(λ2D3 + λD2 + D1)x = 0
• The authors create D1, D2, D3 using the known
values in (7), λ = f.
Overdetermined Solution
• Solve
With
• D1, D2, D3 very sparse containing only:
Solving QEP
• One approach to solving the QEP :
Convert it to a linear system (remove the f2):
• Solving ( A - f B ) v = 0
Foot-head Homology
• Result of QEP : K, R, t, f
• From this construct the homology HFH with
uH ≃ HFH*uF
H
u
– uH : image points of head
– uF :image points of feet
k
H
(x0k,y0k,l)
HFH
uF
Hk
Camera Image
(x0k,y0k,0)
3-D points
Result
Conclusion
• Use 3D-2D point correspondences (model to
contour)
• Encode camera parameters that define
relation between 3D 2D as a matrix H
• Solve H and get the camera parameters