Real-time foreground object
detection & tracking with moving
camera
P93922005 Martin Chang
Motivation

More and more moving cameras





Handheld devices
Cell phone
PDA
Hard to track object with moving
camera
Hard to learn background with moving
camera
Previous Work

Thompson, W.B. and Pong, T.C. Detecting moving objects.
International Journal of Computer Vision, 4(1):39-57. (January
1990).


K Daniilidis, C Krauss, M Hansen, G Sommer. Real-Time Tracking
of Moving Objects with an Active Camera. Real-Time Imaging,
1998.


Stationary Camera
Two degrees of freedom of a camera platform
E Hayman, JO Eklundh - Procs. Statistical Background
Subtraction for a Mobile Observer. IEEE Intl. Conf. on Computer
Vision.



Moving foreground object
static background
Mobile observer
Steps
1.
2.
3.
4.
5.
Find good feature to track
Track features
Classify foreground and background
features
Decide region of foreground object
Track foreground object
Video Demo
Step 1: Find good feature to
track

Finding good feature to track

Shi and Tomasi ‘s method
Step 2: Track features

Optical flow
Step 3: Classify foreground
and background features

Classify feature points

Optical flow



Moving direction of feature
Length of moving direction
MTF of neighbor image patch

Doesn’t work, due to


With cheap camera
Low resolution video
Idea: how to identify
foreground features? 1/3

Case 1: The camera rotates

The background image moves more
Background
Object
Idea: how to identify
foreground features? 2/3

Case 2: The background moves

The background image moves more
Background
object
Idea: how to identify
foreground features? 3/3

Case 3: The object moves

The foreground image moves more
Background
object
Classify Features

KMeans


Marginal KMeans


Hard to separate them well
Filter unreliable features
Angle issue

1° is similar to 359 °
KMeans
Marginal KMeans
(Margin=1/2)
Marginal KMeans
(Margin=1/4)
Step 4: Foreground Object
Detection 1/2

Two two-class problems

Classify foreground and background
features


Cluster features
Calculate the occlusion rate

The region of foreground object should be


Compact
Less noise (background features)
Foreground Object Detection 2/2

Measure our confidence


Geometry approach
Check foreground and background regions
Step 5: Foreground Object
Tracking
1.
2.
Object detection
If foreground object is never detected
Go to Step 1
3.
4.
Object tracking
Go to Step 1
Development Platform



Microsoft Visual C++ .NET 2003
Cheap webcam (USB 1.1)
OpenCV
Future Work


Find parameters by machine learning
Detect finite candidate objects


Cue: color moment
Multiple object detection(!)
Conclusion



The bottleneck is camera’s data
transporting speed (USB 1.1)
Real time is possible
OpenCV is useful
Demo
Thanks!