Video Data Mining: An Architectural Review
ABSTRACT
Acquisition and storage of video data can be done easily. But to retrieve the useful information
from video is a very challenging task. To solve this problem video data mining technique is used
to extract the semantic information as per user demand automatically. Extraction of implicit
knowledge including interesting patterns, summarization, comparison and video data
relationships can be done by Video data mining. It is an interdisciplinary endeavor that draws
upon expertise in computer vision, video processing, video retrieval, data mining, machine
learning, and artificial intelligence. This paper describes the proposed architecture and concept
of video data mining to extract the useful information from video applications.
Keywords
Video data mining, Preprocessing, Feature extraction, Association and clustering.
1. INTRODUCTION
To retrieve the video effectively and to
facilitate new and better ways of
entertainment, advanced technologies must
be developed for searching and mining the
vast amount of videos. Although valuable
information may be hiding behind the data
and the overwhelming data volume makes it
difficult for human beings to extract them
without powerful tools. Video mining
system that can automatically extract
semantically meaningful information from
video databases. While numerous papers
have appeared on data mining [1-4], some
deals with video data mining directly [5-6].
In video databases, knowledge discovery
deals with non-structured information, for
this reason we need tools for discovering
relationships between objects or segments
within video components. In general video
must be first preprocessed to improve their
quality. Subsequently, these video files
undergo various transformations and
features extraction to generate the important
features from the video databases. With the
generated features, mining can be carried
out using data mining techniques to discover
significant patterns. These resulting patterns
are then evaluated and interpreted in order to
obtain
relevant
knowledge
related
application.
Some research work has been done
on video mining like [7] where authors
focused on mining on medical video to
detect the events from medical video. But in
our paper we describe video data mining for
video application in general, which can be
applied to retrieve information from
different video applications more frequently.
2. VIDEO DATA MINING PROCESS
Figure 1 depicts the video data mining
system architecture for applying in different
video applications. Video data is the starting
point of a learning system, then, the data
pre-processing is used to discover important
features from video data. Data preprocessing
includes data cleaning, normalization,
transformation, feature selection, etc.
Detailed procedure depends highly on the
nature of video data and problem’s domain.
To facilitate the relational dataset video
shots are detected. The set of rules will be
applied on video shots. Given a set of rules
on specified video shots, a video data
mining algorithms is applied to extract
useful information from video data.
Video Data
Data preprocessing
Video Shots
Mining Rules
goal, we adopt some video processing
techniques to segment a video sequence into
shots, and applying data mining techniques
to explore relationships among shots.
5. VIDEO MINING ALGORITHMS
Video Data
Mining
Algorithms
Figure 1. System Architecture of Video
Data Mining
To adopt the video data mining algorithms
for data extraction an unsupervised
technique is considered to present video files
into groups. Following section describes
various steps for video mining.
5.1 Clustering
3. FEATURE EXTRACTION
There are two kinds of features: description
based and content based. The former uses
metadata, such as keywords, caption, size
and time of creation. The later is based on
the content of the object itself [8]. Higherlevel information from video includes:
•Detecting trigger events (for example any
vehicles entering a particular area, people
exiting or entering a particular building)
•Determining typical and anomalous
patterns of activity, generating personcentric or object-centric views of an
activity.
•Classifying
activities
into
named
categories (for example walking, riding a
bicycle),
•Clustering and determining interactions
between entities.
The first stage for mining raw video data is
grouping input frames to a set of basic units,
which are relevant to the structure of the
video.
4. PREPROCESSING OF VIDEO DATA
Before applying video data mining on video
data to extract the useful information the
most important tasks is to transform the
original video sequence into a relational
dataset, since most traditional data mining
techniques work on the relational database
where the relationships between data items
are explicitly specified. To facilitate this
Grouping of a given collection of unlabeled
video files into meaningful clusters
according to the video content without a
priori knowledge is done by the
unsupervised classification.
Clustering
algorithms can be categorized into
partitioning methods, hierarchical methods,
density-based methods, grid-based methods,
and model-based methods. An excellent
survey of clustering techniques can be found
in [9].
5.2 Association Rules
First association rules described in general.
The most association rules studies have been
focusing on the corporate data typically in
alphanumeric databases [10]. There are three
measures of the association: support,
confidence and interest. The support factor
indicates the relative occurrences of both X
and Y within the overall data set of
transactions. It is defined as the ratio of the
number of instances satisfying both X and Y
over the total number of instances. The
confidence factor is the probability of Y
given X and is defined as the ratio of the
number of instances satisfying both X and Y
over the number of instances satisfying X.
The support factor indicates the frequencies
of the occurring patterns in the rule, and the
confidence factor denotes the strength of
implication of the rule. The interest factor is
a measure of human interest in the rule. For
example, a high interest means that if a
transaction contains X, then it is much more
likely to have Y than the other items.
Relatively little research has been conducted
on mining video data [11]. There are
different types of associations: association
between video content and non-video
content features. Association mining in
video data can be transformed into problems
of association mining in traditional
transactional databases. Therefore, mining
the frequently occurring patterns among
different video contents becomes mining the
frequent patterns in a set of transactions. In
[12], the concept of content-based video
association rules using feature localization is
extended by the authors. They introduced
the concept of progressive refinement in
discovery of patterns in video.
6. KNOWLEDGE DISCOVERY FROM
VIDEO DATA
Video Data Mining, also popularly known
as Knowledge Discovery in video Databases
refers to the nontrivial extraction of implicit,
previously unknown and potentially useful
information from video data. Data mining is
actually part of the knowledge discovery
process (KDD). Video data mining is the
extension of data mining. The Knowledge
Discovery in Databases process comprises
of a few steps leading from raw data
collections to some form of new knowledge.
7. APPLICATIONS
Applications that require data mining of
video databases includes:
 Web mining: to extract meaningful
information and hidden patterns that may
implicitly exist in video databases on web.
 Crime preventation: to infer potentially
useful
and
previously
unrecorded
information with spatial and temporal
proximity relationships.
 Geographical information system: the
widespread use of GIS by local and federal
governments
and
other
institutions,
necessitate the development of efficient data
mining algorithm for geo referenced –data.
Others: News broadcasting, military, video,
education and training, cultural heritage,
advertising, and interior design, sports
videos [13], traffic video [14] etc.
These applications have vast collection of
images in the corresponding video databases
and can be mined to discover new and
valuable knowledge.
8. CONCLUSION AND FUTURE
WORK
This paper describes a survey and proposed
architecture of video data mining for
extraction of knowledge. There are a
number of applications where these
techniques could be applied and extract
hidden predictive pattern or information
from video database. Further research work
is to develop video data mining algorithms
for information retrieval.
REFERENCES
[1]U. Fayyad and R. Uthurusamy, data
mining and knowledge discovery in
database, Communication of ACM 39(11),
1996.
[2]R. Agrawal, T. Imeielinski, and A.
Swami, data mining: A performance
perspectives, IEEE TKDE, 5(6), 1993.
[3]M.S. Chen, J. Han and P.S. Yu, data
mining: An overview from a database
perspective, IEEE TKDE, 8(6), 1996.
[4]J. Han and M. Kamber, data mining:
Concepts
and
techniques,
Morgan
Kaufmann Publishers, 2001.
[5]J. Fan, X. Zhu and X. Lin, mining of
video database, Book chapter in Multimedia
data mining, Kluwer, 2002.
[6]J.-Y. Pan, C. Faloutsos, VideoCube: A
new tool for video mining and classification,
ICADL Dec., 2002, Singapore.
[7]Xingquan Zhu, Walid G. Aref, Jianping
Fan, Ann C. Catlin, Ahmed K. Elmagarmid,
Medical Video Mining for Efficient
Database Indexing, Management and
Access,
Proceedings
of
the
19th
International
Conference
Engineering,2003.
on
Data
[8]Yoshitaka A. and Ichikawa T., A survey
on content-based retrieval for multimedia
databases. IEEE Trans.on Knowledge and
Data Engineering, Vol 11, 1999, pp. 81-93.
[9]Jain A.K., Murty M.N., Flyn, P.J.
(1999), Data Clustering: A Review, ACM
Computing Surveys, Vol. 31, No. 3, 1999,
pp. 264 - 323.
[10]Agrawal R., R. Srikant, Fast Algorithms
for Mining Association Rules, Proc. of the
20th Int'l Conference on Very Large
Databases, Santiago, Chile, 1994, pp. 487499.
[11]Shyu Mei-Ling, Shu-Ching Chen and R.
L. Kashyap, Generalized Affinity-Based
Association rule Mining for Multimedia
Database
Queries,
Knowledge
and
Information Systems, Vol, 3, 2001, pp. 319337.
[12]Za O., a Han, H. Zhu, in mining
recurrent items in multimedia with
progressive resolution refinement. Proc. of
the IEEE International Conference on Data
Engineering, 2000, pp. 461-470.
[13]Kobla V., D. DeMenthon, and D.
Doermann. Identification of sports videos
using replay, text, and camera motion
features. In SPIE Conference on Storage and
Retrieval for Media Databases, volume
3972, 2000, pp. 332-343.
[14]Cucchiara R., M. Piccardi, and P.
Mello.Image analysis and rule-based
reasoning for a traffic monitoring system.
IEEE
Transactions
on
Intelligent
Transportation Systems, Vol. 1, June 2000,
pp. 119–130.