CCU
VISION
LABORATORY
Object Speed Measurements
Using Motion Blurred Images
林惠勇
中正大學電機系
lin@ee.ccu.edu.tw
C
V
Images…
H.Y.Lin, CCUEE
L
CCU Vision Lab
2
C
V
Blur Images…

Defocus blur:
H.Y.Lin, CCUEE
L

CCU Vision Lab
Motion blur:
3
What Do They Tell Us?

Motion of Object
H.Y.Lin, CCUEE

CCU Vision Lab
C
V
L
Region of Interest:
4
Information from Blur Images

C
V
L
Two types of image blur:

Defocus blur – due to the limitation of optical sensors




Image restoration
Identification of region of interest
Depth measurement
Motion blur – due to the relative motion between the
camera and the scene





Image restoration
Motion analysis
Increase still resolution from video
Special effect
Speed measurements?
H.Y.Lin, CCUEE
CCU Vision Lab
From the movie: “Chicken Run”
5
Defocus Blur
C
V
L
q
p
Image
Detector
1
3
D
Focused
Position
f
Blur circle
Defocused
Position
d
4
2
Z
z
H.Y.Lin, CCUEE
CCU Vision Lab
6
Motion Blur
C
V
L
p
q
Image
Detector
1
s
2
D
3
x
4
f
H.Y.Lin, CCUEE
CCU Vision Lab
7
Speed Measurements

L
Why measure speed? (motivation)





C
V
Wind
Experiments
Sports (baseball, tennis ball), athletes
Vehicle speed detection
How?




RADAR (Radio Detection And Ranging)
LIDAR (Laser Infrared Detection And Ranging)
GPS
Video-Based Analysis
H.Y.Lin, CCUEE
CCU Vision Lab
8
Image-Based Speed Measurement L
C
V

Key idea:

For a fixed camera exposure time:
Relative motion between
object and static camera
H.Y.Lin, CCUEE
Motion blur appeared in
the dynamic image region
CCU Vision Lab
9
Geometric Formulation

C
V
L
Simple pinhole camera model:
d
d

Ks x
L
K
v
z
f
zKs
x
Tf
v
z

f
KL
Tl
Key components:


Focal length, exposure time, CCD pixel size
Object distance, blur length (blur extent)
H.Y.Lin, CCUEE
CCU Vision Lab
10
Image Degradation

Characterized by its point spread function (PSF) h(x,y)
g ( x, y ) 




 
h ( x   , y   ) f ( ,  ) d  d 
Degradation under uniform linear motion (whole image)
1
 ,
h( x, y )   R

0,

L
Image degradation – linear space invariant system


C
V
x 
R
cos  , y  x tan 
2
otherwise
How about space variant case? (partial blur & total blur)
H.Y.Lin, CCUEE
CCU Vision Lab
11
Blur Parameter Estimation

L
Edge detection ABC:



C
V
Sharp edge  step response
Blur edge  ramp response
How to use this fact to estimate blur extent?
H.Y.Lin, CCUEE
CCU Vision Lab
12
Image Deblurring
L
g(x,y)
Degradation
function
H
f(x,y)
C
V
Restoration
filter(s)
+
f(x,y)
Noise (x,y)
Restoration
Degradation

If H is linear, space invariant:



Inverse filtering
Wiener filter
Bad news:


Our case is space variant
Region segmentation
T
H (u, v)   e
0
H.Y.Lin, CCUEE
 j 2ux0 ( t )
dt 
T
ua
sin( ua)e
 jua
CCU Vision Lab
13
More General Case – I

C
V
L
What if the object is not moving parallel to the
image scanlines?


Motion direction estimation
Image rectification
H.Y.Lin, CCUEE
CCU Vision Lab
14
Motion Direction Estimation

Fourier spectrum analysis:

It can also be implemented in spatial domain
H.Y.Lin, CCUEE
CCU Vision Lab
C
V
L
15
More General Case – II

C
V
L
What if the object is not moving parallel to the
image plane?
d
l
L
z
H.Y.Lin, CCUEE
CCU Vision Lab
f
16
Extended Camera Model


pk
C
V
L
d sin 
f
d cos   
k
d 
v
v
H.Y.Lin, CCUEE
CCU Vision Lab

z
f
zk
f cos   ( p  k ) sin 
zKs
x
T [ f cos   s x ( P  K ) sin  ]
zKs
x
Tf cos 
17
Required Parameters

Size of the softball (physical measurement)
Vehicle speed detection – “parallel case”


Distance to the object, camera orientation
Softball speed measurement


Focal length, CCD pixel size, exposure time
Extrinsic camera parameters


L
Intrinsic camera parameters


C
V
Length of the vehicle (from manufacturer’s data sheet)
Vehicle speed detection – “non-parallel case” ?

How to obtain the parameters z, , etc.?
H.Y.Lin, CCUEE
CCU Vision Lab
18
Vehicle Speed Detection



C
V
L
Parameters:
 K = 22 pixels, sx = 11 m, f = 10 mm, T = 1/160 sec.
 l = 560 pixels, L = 4750 m
Detected speed – 104.86 km/hr
Video-based speed – 106.11 km/hr, speed limit – 110 km/hr
H.Y.Lin, CCUEE
CCU Vision Lab
19
Camera Pose Estimation

L
Theorem:


C
V
Given a parallelogram in 3-D space with known image projection
of four points, their relative depths can be determined.
To obtain the unknown scale factor:


D
Absolute metric between two 3-D points
License plate with standard size
W
C
A
Pi  ( X i , Y i , Z i )  (  i sx i ,  i sy i ,  i f )
 i  i / 0
B

di 
d
X
  cos
H.Y.Lin, CCUEE
2
i
1
 Yi  Z
 
n c
 
n c
2
2
i
c
a
b
I
P
CCU Vision Lab
20
Vehicle Speed Detection



C
V
L
Parameters:
 K = 22 pixels, sx = 6.8 m, T = 1/400 sec., l = 560 pixels, L = 4750 m
 W = 320 mm,  = 48.25, f = 26 mm
Detected speed – 112.97 km/hr
Video-based speed – 110.22 km/hr
H.Y.Lin, CCUEE
CCU Vision Lab
21
Fully Automated? How?



Motion blur analysis
Region segmentation


JPEG EXIF header
Target identification


L
Intrinsic camera parameters?


C
V
Region growing
Additional image capture
Robust blur extent estimation
Image synthesis

Deblurred target region + static background region
H.Y.Lin, CCUEE
CCU Vision Lab
22
Initial Target Segmentation




Horizontal ramp edge detection
Run-length coding or projection
Vertical continuity checking
Multiple direction analysis
H.Y.Lin, CCUEE
CCU Vision Lab
 1

1

 1

 1
 1

 1
 1

C
V
L
2
3
0
3
2
2
3
0
3
2
2
3
0
3
2
2
3
0
3
2
2
3
0
3
2
2
3
0
3
2
2
3
0
3
2
1

1

1

1
1

1
1
23
Spherical Object in Motion


Accuracy, robustness, precision (subpixel resolution…)
Spherical object  circular from any viewpoint
Initial blur extent identification + circle detection


L
Problems on parameter estimation


C
V
Circle fitting, Hough transform
More problems

Motion blur due to rotation, three-dimensional translation,
shading, etc.
H.Y.Lin, CCUEE
CCU Vision Lab
24
Speed Measurement Flowchart
Motion Blurred
Image
C
V
L
Object Speed
Two Images
Target Identification
Environment Parameter
Estimation
Image Segmentation
Speed Measurement
Horizontal
Motion Blur
no
Image Rotation
Circle Fitting
yes
Initial Blur Length
Estimation
H.Y.Lin, CCUEE
Image Deblurring
CCU Vision Lab
25
Motion Direction Estimation


C
V
L
Camera pose estimation – non-parallel case
Two or more captures with fast shutter speed




Vertical projection
Post-processing
Fixed object size
Could be blurred
z
Q
r
P
a
Z
1
 z  z1 
1

  tan  2

x

x
1 
 2
2
2
I
p
q
f
x
O
H.Y.Lin, CCUEE
CCU Vision Lab
26
Softball Speed Measurement



C
V
L
Parameters:
 K = 26 pixels, T = 1/320 sec., l = 72 pixels, d = 97.45 mm
Detected speed – 40.5 km/hr
Video-based speed – 40.9 km/hr
H.Y.Lin, CCUEE
CCU Vision Lab
27
Conclusion


Vehicle speed detection
Softball speed measurement
Advantages





L
Object speed measurement using a single motion blurred
image


C
V
Low cost – off-the-shelf digital camera
Passive device – can avoid anti-detection
Passive device – no radiation, light
Large measurement range – through adjustable shutter speed
Limitation


Lighting condition
Accuracy?
H.Y.Lin, CCUEE
CCU Vision Lab
28
C
V
L
Thank you for your attention!
Any questions?
H.Y.Lin, CCUEE
CCU Vision Lab
29