I.
INTRODUCTION
With the rapid development of modern electronic equipment, the amount of video data
increase tremendously in recent years. For the video browsing, editing and sharing, video
similarity search (VSS) which can provide content based video retrieval service in large storage
device has been proposed as important research area.
Video similarity in this paper means visual similarity which is one of basis of content-based
video retrieval. Besides, the visual similarity does not contain strict restriction on video objects
and scenes because they are related to users’ interests. Video similarity search includes two
important aspects: one is similarity measure and another is efficient search technique. The
feature representation is the basis of similarity measurement.
Video similarity in this project means visual similarity which is one of basis of content-based
video retrieval. Besides, the visual similarity does not contain strict restriction on video objects
and scenes because they are related to users’ interests.
Literature Study:
ViSig: method was introduced based on randomized summarization which randomly selects a
number of seed frames and assigns a small collection of closest frames to each seed.
Gao et al. proposed a video summarization tool to remove redundant video content with the
hierarchical agglomerative cluster in the shot level.
Shen et al. employed bipartite graph to map relationship between the query and database video.
The irrelevant subsequences to the query were pruned by a filter-and-refine search strategy.
Besides, several other feature extraction algorithms such as distinctive local descriptors, color
fingerprint, DCT and visual rhythm spectrum etc were also proposed. For similarity
measurement, many approaches have also been proposed such as neighborhood similarity
considering local sample and label distribution, distance measure based on Kullback-Leibler
divergence and high-dimensional statistical measure according to probability density function
etc. Though these techniques are quite valuable for the further studies on image or video feature
extraction and similarity measure, they are usually too complicated to accomplish feature
extraction in large video database.
An Efficient Video Search Algorithm Project Report
1
Methodology:
Video similarity search includes two important aspects: one is similarity measure and
another is efficient search technique. The feature representation is the basis of similarity
measurement. In the past decade, the effective feature representation of image and video has
been extensively studied.
Efficient indexing and search is another important aspect of video similarity search. Recently,
there were numerous efficient strategies designed to improve similarity search in database, such
as max nearest distance, local constant embedding and optimal indexing etc. The indexing
methods based on locality sensitive hashing (LSH) have been widely used for approximate
neighbor search problem in high-dimensional space. However, it has been shown in that when
the dimensionality exceeds about ten, the indexing data structures based on space partitioning are
slower than the brute-force, linear-scan approach. The existing LSH techniques, usually
containing numerous of hash tables, are inefficient for scalable distributed computing and fast
query in large database. Although these techniques provide valid solutions for high-dimensional
search in large database, the search methods with complex index structure are not efficient
enough for fast search in large database.
In this paper, an efficient VSS approach is proposed. The feature extraction of image and video
was achieved by image characteristic code (ICC) based on the statistics of spatial temporal
distribution. The similarity of video was measured according to the computation of the number
of similar video components. Besides, a fast search approach for scalable computing was
presented based on clustering index table (CIT). By virtue of proposed feature computation and
index clustering, the VSS can be implemented very efficiently with satisfying recall and
precision rate. The experimental results indicate this approach performs well in large database.
An Efficient Video Search Algorithm Project Report
2
II.
VIDEO FEATURE COMPUTATION AND SIMILARITY
A. Feature Computation
Video feature computation is generally based on image feature extraction. Several lowlevel features such as color, texture, edge are usually adopted for image fingerprint. It has been
shown that YCbCr histogram is an effective video signature because of its valuable information
for video matching. Besides, YCbCr coding is widely used in consumer electronic equipment
such as TV, DVR and DVD etc. Since the volume of video database grows very fast, video
feature extraction should not be too complicate to compute for the computational efficiency. In
proposed approach, the mean of YCbCr was employed for image feature computation, where M
and N are the width and height of image, respectively. Yij, Cbij, Crij stand for the value of Y, Cb
and Cr components of each pixel in the image. Since there is no strict restriction on video object
and scene, the feature computation based on the spatial statistics of image can obtain the major
information including luminance and color which is the basis of visual similarity. For video
similarity search and noise resistance, the mean statistics were four digits rounding off integers.
Image characteristic code (ICC) c is a joint feature representation made up of three statistical
integers of every pixel components: Y, Cb and Cr. In this way, high dimensional feature was
transformed into compact characteristic code and video similarity search can be implemented as
text search.
B. Similarity Measure
According to the characteristics of spatial-temporal distribution (STD) of image frame
feature in video, video similarity is related to image frame feature, shots type, length and their
temporal variation. The ICC in video makes up a characteristic time series of video according to
STD of image frame. Therefore, the statistics of STD based on the ICC containing the general
An Efficient Video Search Algorithm Project Report
3
information of similar shots and their temporal distribution can be adopted for video similarity
measure. Since the type and length of shots are usually different, the video similarity is
computed based on the statistics of video components. The video component (VC) is the set of
image frames with the same ICC, which can be identified as a statistical feature cluster based on
STD. According to the statistics of video component, the video similarity measure can be defined
as following: giving two video clips whose video component set is Ci, Cj (2) respectively, nk is
the number of common video component clusters in their intersection set. The video similarity D
is the sum of common component clusters. In this way, the video similarity is measured by fine
computation of video component based on the statistics of STD.
An Efficient Video Search Algorithm Project Report
4
III.
FAST SIMILARITY SEARCH
A. Clustering Index Table
By virtue of the compact image feature representation ICC, the VSS can be implemented
in high efficiency. For the efficiency improvement and scalable computing, Clustering Index
Table (CIT) search method is proposed in this paper. The indexes of tables are ICC for video
feature clustering. The video components are arranged into the same video clustering index table
according to the definition mentioned before. The VSS can be carried out by index clustering and
redundant dissimilar video search is no longer executed, so the search efficiency can be greatly
improved. Compared with the existing search techniques whose computational complexity are
O(n) or O(logn), the CIT search approach can perform with higher efficiency, which is
implemented on the basis of hashing technique. There are four types of table in the proposed
search strategy. The first is video CIT (VCIT) containing the number of video components. The
second is database CIT (DCIT) recording video ID and the number of video component of each
video. The third is search table (ST) including the corresponding database CIT of each video
clip. The forth is video similarity table (VST) recording similar video ID and video similarity
measure. Every video clip in database has its video CIT, search table and video similarity table.
There is database CIT for the video similarity measure and video similarity search.
B. Similarity Search based on CIT
The proposed video similarity search system is on the basis of four types of table
mentioned above, which can improve the similarity search efficiency and meet the requirement
of scalable computing. In video database, every video clip has its video CIT recording the result
of video component clustering. Since the index of video CIT and database CIT is the same ICC,
the search table can keep recording the indexes of its corresponding database CIT with the
computation of video component clustering. For the video similarity search, the similar video
clips can be easily found in corresponding database CIT according to the records in search table.
In database CIT, the similarity measure can be computed by the statistics of video components.
In fact, the database CIT can be identified as the intersection set of video components shared by
several video clips. After the entire search in corresponding database CIT, video similarity table
including video ID and similarity measure can be generated with the computation of video
similarity. Since every video has its video similarity table, the search engine only needs to show
the content of video similarity table according to query video, which is also in the video
database. If there are new video clips to be added in the database, the video similarity search
An Efficient Video Search Algorithm Project Report
5
system will accomplish two tasks: one is video processing and another is database update. The
video processing implements the video feature computation and ICC generation. A series of
video CIT based on video component clustering can be created simultaneously. Meanwhile, the
search table containing the index of database CIT will be generated according to the video CIT.
The video ID and the number of video components will be added at the end of corresponding
database CIT. Then, video similarity can be calculated according to the number of video
components and the video similarity tables can be updated by searching in corresponding
database CIT. The tables of related video in database will also update. All these operation can be
implemented simultaneously for computational efficiency and video similarity search system can
meet the requirement of scalable computing.
The framework of this project VSS system is shown in Fig.1.
In this system, feature computation of every video clip in video database DB is implemented by
ICC based on the statistics of STD. Then, a series of video CIT can be generated by index
clustering according to video components. ST is created according to video CIT and
corresponding database CIT will update. VST is generated by searching corresponding database
CIT according to ST to provide the list of similar video clips and their similarity ranking. For the
video similarity search, the content of VST will be shown directly by the search engine to give
the similarity search result of the query video clip. In this way, the efficiency of VSS can be
greatly improved.
An Efficient Video Search Algorithm Project Report
6
IV.
Implementation Details
Tools Used:
MATLAB
MS-Access
Ffmpeg
An Efficient Video Search Algorithm Project Report
7
V.
EXPERIMENTAL RESULTS
A. Database Setup
The database in the experiments was made up of a few hundred video clips downloaded
from internet. The total length of video data is approximately one hundred and sixty hrs,
including various contents such as news, sports, movie, cartoon, teleplay and natural scene etc.
The experiments were performed on Intel processor 1.5 GHz, 2G Memory.
The database used is MS-ACCESS. There four tables as mentioned in the above.
Cluster Mechanism: Cluster Indexing.
Due to this cluster indexing it is possible to group similar frames together so as there is no need
to store same frame information more than once.
An Efficient Video Search Algorithm Project Report
8
Feature Extraction:
Extracted Key frames from the input query video.
Results:
Query By Example:
Given input search Query:
An Efficient Video Search Algorithm Project Report
9
Retrieved Videos are:
An Efficient Video Search Algorithm Project Report
10
Search Query2:
Retrieved videos are
An Efficient Video Search Algorithm Project Report
11
An Efficient Video Search Algorithm Project Report
12
VI.
CONCLUSION & FUTURE WORK
In this project, an efficient video similarity search approach is proposed. A new video feature
representation ICC is computed based on the statistics of video spatial-temporal distribution. The
video similarity is measured by computing the number of similar video components. To meet the
scalable computing requirement, an efficient search method based on clustering index table is
proposed and several types of tables are employed for the improvement of search efficiency. The
experimental results show this method is efficient and effective and it can be deployed in various
storage devices for video similarity search.
This project can be further improved by adding several other local and global descriptors.
An Efficient Video Search Algorithm Project Report
13
VII.
REFERENCES
[1] An Efficient Video Similarity Search Algorithm. Zheng Cao, Ming Zhu. IEEE Transactions
on Consumer Electronics, Vol. 56, No. 2, May 2010.
[2] http://www.mathworks.com/help/toolbox/images/f12-12267.html
[3] http://www.physicsforums.com/showthread.php?t=24029
[4] http://www.mathworks.com/products/viprocessing/
[5]http://www.mathworks.com/company/events/webinars/index.html?id=&language=en&b
y=application
[6]http://www.mathworks.com/company/events/webinars/wbnr43666.html?id=43666&p1=7239
07038&p2=72390756
[7] Cheung S, Zakhor “A Efficient video similarity measurement with video signature”. IEEE
Trans. on Circuits and Systems for Video Technology, vol.13 pp.59-74 2003
[8] Yue Gao Weibo Wang, Junhai Yong. “A video summarization tool using two-level
redundancy detection for personal video recorders”, IEEE Trans. Consumer Electronics. vol.54
pp.521-526 2008
[9] Heng Tao Shen, Jie Shao, Zi Huang, Xiaofang Zhou “Effective and Efficient Query
Processing for Video Subsequence Identification” IEEE Trans. on Knowledge and Data
Engineering, vol.21 pp. 321-334 2009
[10] Yan Ke Rahul Sukthankar Larry Huston “An efficient parts-based nearduplicate and subimage retrieval system” Proc. of the 12th annual ACM international conference on Multimedia
pp.869-876 2004
[11] Yang X, Tian Qi, Chang E-C “A color fingerprint of video shot for content identification”
Proc. of the 12th annual ACM int’l conf. on Multimedia pp.276-279 2004
[12] Xuefeng Pan, Jintao L, Yongdong Zhang,, Sheng Tang, Lejun Y, “Format-Independent
Motion Content Description based on Spatiotemporal Visual Sensitivity” IEEE Trans. Consumer
Electronics. vol.53 pp.769-774 2007
[13] Kwang-deok Seo, Seong Park, Soon-heung Jung, “Wipe scene-change detector based on
visual rhythm spectrum” IEEE Trans. Consumer Electronics. vol.55 pp.831-838 2009
[14] Meng Wang, Xian-Sheng Hua, Jinhui Tang, Richang Hong. “Beyond Distance
Measurement: Constructing Neighborhood Similarity for VideoAnnotation”
An Efficient Video Search Algorithm Project Report
14