OBJECT TRACKING FRAMEWORK FOR ACTIVITY
RECOGNITION
Nowadays video surveillance systems are applied worldwide in many different
sites such as parking lots, museums, hospitals and railway stations. Video surveillance
helps a supervisor to overlook many different camera fields of views from the same room
and to quickly focus on abnormal events taking place in the control space. However one
question arises: how a security officer can analyse in real time and simultaneously dozens
of monitors with a minimum rate of missing abnormal events? Moreover, the observation
of screens for a long period of time becomes boring and draws the supervisor’s attention
away from the events of interest. The solution to this issue lies in three words: intelligent
video monitoring.
Intelligent video systems belong to the domain that studies the content of a video.
This term expresses a research direction fairly large, and is applied in different fields: for
example in robotics and homecare. In particular, a lot of research and work in this area
are already achieved in video surveillance. This doctoral thesis focuses on one of the
most important issue in video analysis: mobile object tracking. There are many methods
proposed to track mobile objects such as Kalman filter, particle filter and probabilistic
data association filters. Some other researches have focused on modelling the scene in
order to improve the tracking algorithm. Their main idea consists in providing the
contextual information of the scene such as: the positions, directions of paths (i.e tracked
objects), the sensitive zones in the scene where the system can lose object tracks with a
high probability, the zones where mobile objects appear and disappear usually… These
elements can help the system to give better analysis and prediction on object trajectory.
All these works listed above have obtained satisfactory results, but each approach can
only be applied in some video types. When there is a large variation of scene dynamics,
the object tracking usually fails.
To solve this issue, this doctoral thesis aims at constructing a general tracking
framework for object tracking. The proposed framework includes the following elements:
tracking algorithms, evaluation methods for assessing the tracking algorithm
performance, method for learning, supervising, and optimizing the algorithm parameters.
The proposed framework will be able to control the tracking process by choosing the
most effective tracking algorithm for the current scene. The tracking algorithms can be
changed dynamically throughout the time to adapt to scene changes.
In the first step, a metric will be defined for evaluating the performance of
tracking algorithms. To define this measure, we have to select the suitable trajectory
features influencing the tracking process quality (eg. trajectory temporal length, spatial
length…). Then we learn and regulate the importance of these features using the
annotated trajectories in ETISEO data. The proposed metric will be tested on several
tracking algorithms (Genius tracking algorithm, Global tracking [7]), CARETAKER
tracking algorithm [8] ...) in many different videos.
The second step will study the characteristics of the scene environment that
influence the performance of tracking algorithms (people number in the scene,
illumination, camera position, occlusion...). Thanks to the first step results, a novel
method will be designed to learn the relations between these parameters and the tracking
algorithms. The system can then decide which tracking algorithm is most compatible to a
given scene.
A third step will consist in constructing a framework that is able to interpret
automatically the scene. In this step, the system will try to detect the scene variations and
then will be able to change the current tracking algorithm by a better one. The framework
also could configure automatically the parameters of the tracking algorithms. To do that,
we propose to study and apply online machine learning techniques.
The evaluation of this work will be conducted using video data captured over
several days or weeks containing mainly human activities (GERHOME,...).




Desired contributions:
A method to evaluate automatically the performance of tracking algorithms.
Establishing the relations between the tracking algorithm performance and the
environment parameters
A new method to detect automatically the scene variations and to adapt the current
tracking algorithm based on machine learning.
Publications in international conferences such as ECCV, ICCV, PAMI, ECAI...
Time table:
Time period
Task
6 months
- Study of tracking techniques: F2F tracking, longterm tracking,
global tracking; and different approaches such as optical flow,
appearance model based tracker, particle filter...
- Evaluating available tracking algorithms in the SUP platform
on ETISEO database and on GERHOME database
- Study the influence of environment parameters on tracking
performance
- Evaluation of ETISEO tracking algorithms
6 months
- Study of optimization techniques, machine learning techniques
for tracking algorithms (in particular, online machine learning)
- Learn the relations between the environment features and
tracking algorithm parameters
6 months
- Propose a new framework for object tracking processes
enabling the dynamic supervision of tracking algorithms.
6 months
- Using online machine learning to optimize object tracking
according to the environment changes
6 months
- Evaluation and improvement of the proposed framework
6 months
- Writing of thesis report
References
1. 1. A.T Nghiem F. Bremond M. Thonnat and R. Ma. New Evaluation Approach
for Video Processing Algorithms. In the WMVC 2007 IEEE Workshop on Motion
and Video Computing - Austin - Texas - USA, February 23-24, 2007.
2. F. Bremond and A.T Nghiem. ETISEO Performance Evaluation for Video
Surveillance Systems. Bristish Machine Vision Association symposium on
Security and surveillance: performance evaluation, London - UK, December 12,
2007.
3. Richard S. Sutton and Andrew G. Barto, Reinforcement learning: An
introduction, The MIT Press Cambridge, Massachusetts
London, England.
4. Rania Y.Khalaf and Stephen S.Intille, Improving Multiple People Tracking Using
Temporal Consistency, In MIT Dept.of Architecture House N Project Technical
Report, 2001.
5. V. Tomasevic (Serbia and Montenegro), The Dynamic Approach to the Tracking
Process in the Explanation Systems, Proceedings of the 3th IASTED International
Conference ARTIFICIAL INTELLIGENCE AND APPLICATIONS, ISSN 14827913, Benalmadena, Spain, September 8-10, 2003, pp. 204-209.
6. Kyuengbo Min "Tracking of Human Walking Based on Hybrid Method Using
Dynamic Model" International Symposium on Large-Scale Knowledge
Resources(LKR2007), Tokyo, Japan, pp.129-132 (2007-3)
7. D.P. Chau, F. Bremond, E. Corvee and M. Thonnat. Repairing People
Trajectories Based on Point Clustering. In the International Conference on
Computer Vision Theory and Applications (VISAPP 09), Lisboa, Portugal,
February 5-8, 2009.
8. http://www.ist-caretaker.org/