MOVING OBJECT DETECTION ON A
RUNWAY PRIOR TO LANDING USING
AN ONBOARD INFRARED CAMERA
Dr. Gerard Medioni
Cheng Hua Pai
Yu Ping Lin
Introduction

Input: Infrared runway sequence

Goal: Detect moving objects on runway
Approach

We do it in two steps:
1.
Stabilize the sequence
2.
Detect motion on the stabilized sequence
Flow chart of the system
Reference
Frame
Runway Identification
Update Reference
frame
Image Stabilization
Motion Detection
Yes
Update
Reference
frame?
Blobs in motion
H ref ,i
Stabilization

Issues:



Planar region containing the runway
Feature choice and matching
Transformation between consecutive frames
Stabilization

Approach



Manually Label
planar region
SIFT provides
sufficient and
descriptive features
RANSAC to
estimate best
transformation
Stabilization

Result:
Stabilized runway sequence
Adaptive Reference Frame

Issues:

For longer sequence
Small errors accumulate
 Big scale difference

Beginning of a Sequence
End of a Sequence
Adaptive Reference Frame

When to change reference frame?

Check the lower edge length ratio
Stabilization algorithm
Landing UAV
image sequence
Manually labeled
planar region
input
1. Extract SIFT features
4. Use RANSAC to
remove outliers and
estimate homography
2. Region of Interest
5. Update reference
frame if necessary
3. Match features to
previous frame to
establish
correspondence
6. Warp to the reference
frame
output
Locally stabilized image sequence and H i ,ref for all i s
Adaptive Reference Frame

Result:
Original Sequence
Locally Stabilized Sequence
Detection module

Issues:



Detection method
Global intensity variation
Noise
Moire in the sequence
 Poor stabilization
 Local intensity variation
 Random noise

Detection

Approach:

Use simple Gaussian background model
t = (1-) * (t-1) +  * (It)
t2 = (1-) * (t-1) 2 +  * (It- t)

Foreground: More than 4t2 from mean
Foreground
Background
4t2
4t2
t
Intensity distribution of an image
Global intensity variation

Approach:

Compensate gain with affine transformation
[Yalcin 05]
I t  mt It 1  bt   t
Before compensation
After compensation
Noise reduction

Approach:

Moire in the sequence


Compare 8 neighbouring background pixels
Poor stabilization

Restabilize with gradient map (also SIFT)
To Gradient
Noise reduction

Approach:

Local intensity variation

Intensity normalization on the foreground pixels
I i fg= miuifg  bi   i

Random noise

Compare consecutive foreground masks
With random noise
Without random noise
Detailed flow chart of Motion
Detection Module
Locally
Stablized
Runway
Sequence
Motion Detection Module
Homographies
Hi,ref
Reference
Frame update?
Yes
Update Runway
Filter
No
Filter Runway
Intensity
Normalization
Runway
Filter
Reference
Frame
Update Reference
Frame
Intensity
compensated
runway image
Image Subtraction
Noise reduction
Foreground
mask &
Quality
Indicator
QI. Score
Bad
Good
Foreground
mask
Background
model
Update Background
Model
Detection Result

Result:
Locally Stabilized Sequence
Foreground mask
Evaluation

Tested on 150 synthesized and 18 realworld sequences
Results (synthetic data):
Speed vs. Detection Rate
1.2
Detection rate

1
Obj. size
2x2
3x3
4x4
5x5
6x6
0.8
0.6
0.4
0.2
0
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Speed (pixel/frame)
Conclusion

Detection affected by:



Program limitation:



Object speed and size
Threshold parameters
Moving objects fade in and out
Bad result near the end of the sequence
Future work:


More test on larger dataset
Speed improvement
Reference









W. G. Chris Stauffer. Adaptive background mixture models for real-time tracking. 1999 IEEE
Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'99), 2:2246, 1999.
M. Fischler and R. Bolles. Random sample consensus: A paradigm for model fitting with applications
to image analysis and automated cartography. Communications of the ACM, 24(6):381-395, 1981.
C. Harris and M. Stephens. A combined corner and edge detector. Proceedings of The Fourth Alvey
Vision Conference, (1):147-152, 1988.
T. G. R. Kasturi, O. Camps and S. Devadiga. Detection of obstacles on runway using ego-motion
compensation and tracking of signicant features. Proceedings 3rd IEEE Workshop on Applications of
Computer Vision, 1996 (WACV'96), pages 168-173, 1996.
D. G. Lowe. Distinctive image features from scale invariant keypoints. International Journal of
Computer Vision, 60(2):91-110, 2004.
R. S. B. Sridhar and B. Hussien. Passive range estimation for rotor-craft low-altitude flight. Machine
Vision and Applications, 6(1):10-24, 1993.
S. Sull and B. Sridhar. Runway obstacle detection by controlled spatiotemporal image Low disparity.
IEEE Transactions on Robotics and Automation, 15(3):537-547, 1999.
R. C. H. Yalcin and M. Hebert. Background estimation under rapid gain change in thermal imagery.
Second IEEE Workshop on Object Tracking and Classification in and Beyond the Visible Spectrum
(OTCBVS'05), 2005.
Q. Zheng and R. Chellappa. Motion detection in image sequences acquired from a moving platform.
Proc. Int. Conf. Acoustics, Speech, and Signal Processing, Minneapolis, 5:201-204, 1993.