IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS, VOL. 51, NO. 6, JUNE 2021
3939
KNN-BLOCK DBSCAN: Fast Clustering
for Large-Scale Data
Yewang Chen , Lida Zhou, Songwen Pei , Senior Member, IEEE, Zhiwen Yu , Senior Member, IEEE,
Yi Chen, Xin Liu, Member, IEEE, Jixiang Du , and Naixue Xiong , Senior Member, IEEE
Abstract—Large-scale data clustering is an essential key for
big data problem. However, no current existing approach is
“optimal” for big data due to high complexity, which remains
it a great challenge. In this article, a simple but fast approximate DBSCAN, namely, KNN-BLOCK DBSCAN, is proposed
based on two findings: 1) the problem of identifying whether
a point is a core point or not is, in fact, a kNN problem and
2) a point has a similar density distribution to its neighbors,
and neighbor points are highly possible to be the same type
(core point, border point, or noise). KNN-BLOCK DBSCAN uses
a fast approximate kNN algorithm, namely, FLANN, to detect
core-blocks (CBs), noncore-blocks, and noise-blocks within which
all points have the same type, then a fast algorithm for merging CBs and assigning noncore points to proper clusters is also
Manuscript received December 29, 2018; revised May 31, 2019 and July 25,
2019; accepted November 18, 2019. Date of publication December 18, 2019;
date of current version May 18, 2021. This work was supported in
part by the National Natural Science Foundation of China under Grant
61673186, Grant 61972010, Grant 61975124, Grant 61722205, Grant
61751205, Grant 61572199, and Grant U1611461, in part by the Funds
from State Key Laboratory of Computer Architecture, ICT, CAS under Grant
CARCH201807, in part by the Open Project of Provincial Key Laboratory
for Computer Information Processing Technology, Soochow University under
Grant KJS1839, in part by the Quanzhou City Science and Technology
Program of China under Grant 2018C114R, in part by the Open Project
of Beijing Key Laboratory of Big Data Technology for Food Safety under
Grant BTBD-2019KF06, in part by the Key Research and Development
Program of Guang Dong Province under Grant 2018B010107002, and in
part by the Grant from the Guang Dong Natural Science Funds under
Grant 2017A030312008. This article was recommended by Associate Editor
G. Nicosia. (Corresponding authors: Songwen Pei; Zhiwen Yu.)
Y. Chen is with the College of Computer Science and Technology, Huaqiao
University (Xiamen Campus), Xiamen 361021, China, also with the Beijing
Key Laboratory of Big Data Technology for Food Safety, Beijing Technology
and Business University, Beijing 100048, China, also with the Provincial
Key Laboratory for Computer Information Processing Technology, Soochow
University, Suzhou 215301, China, and also with the Fujian Key Laboratory
of Big Data Intelligence and Security, Huaqiao University (Xiamen Campus),
Xiamen 361021, China (e-mail: ywchen@hqu.edu.cn).
L. Zhou and X. Liu are with the College of Computer Science and
Technology, Huaqiao University, Quanzhou 362021, China.
S. Pei is with the Shanghai Key Laboratory of Modern Optical Systems,
University of Shanghai for Science and Technology, Shanghai 200093, China
(e-mail: swpei@usst.edu.cn).
Z. Yu is with the School of Computer Science and Engineering,
South China University of Technology, Guangzhou 510640, China (e-mail:
zhwyu@scut.edu.cn).
Y. Chen is with the Beijing Key Laboratory of Big Data Technology for
Food Safety, Beijing Technology and Business University, Beijing, China.
J. Du is with the College of Computer Science and Technology, Huaqiao
University, Quanzhou 362021, China, and also with the Fujian Key Laboratory
of Big Data Intelligence and Security, Huaqiao University, Quanzhou 362021,
China.
N. Xiong is with the Department of Mathematics and Computer Science,
Northeastern State University, Tahlequah, OK 74464 USA.
Color versions of one or more figures in this article are available at
https://doi.org/10.1109/TSMC.2019.2956527.
Digital Object Identifier 10.1109/TSMC.2019.2956527
invented to speedup the clustering process. The experimental
results show that KNN-BLOCK DBSCAN is an effective approximate DBSCAN algorithm with high accuracy, and outperforms
other current variants of DBSCAN, including ρ-approximate
DBSCAN and AnyDBC.
Index Terms—DBSCAN,
DBSCAN.
FLANN,
kNN,
KNN-BLOCK
I. I NTRODUCTION
LUSTERING analysis is the task of grouping objects
according to measured or perceived intrinsic characteristics or similarity, aiming to retrieve some natural groups from a
set of patterns or points. It is a fundamental technique in many
applications, such as data mining, pattern recognition, etc., and
many researchers believe that clustering is an essential key for
analyzing big data [1].
Currently, there are thousands of clustering algorithms have
been proposed, for example, k-means [2], mean shift [3],
DBSCAN [4], spectral clustering [5], [6], mixtures of dirichlet model [7], [8], clustering based on supervised learning [9], and clustering by local cores [10], [11]. According
to Jain et al. [12], different categories of these clustering
approaches are recognized: centroid-based clustering, partitioning clustering, density-based clustering etc.
The goal of density-based clustering is to identify densely
regions with arbitrary shape, which can be measured by the
density of a given point. An identified cluster is usually a
region with high density, while outliers are regions with low
densities. Hence, density-based clustering is one of the most
popular paradigms. There are many algorithms of this kind,
such as DBSCAN [4], OPTICS [13], DPeak [14]–[16], mean
shift [3], DCore [11], etc., where DBSCAN [4] is the most
famous one and has been widely used.
Unfortunately, most of the current existing clustering
approaches do not work well for large-scale data, due to their
high complexities. For example, the complexity of k-means
is O(ktn) where t is the iterations times, DBSCAN runs in
O(n2 ). In this article, a fast approximate algorithm named
KNN-BLOCK DBSCAN,1 is proposed to speedup DBSCAN,
which is able to deal with large-scale data. We also concentrate
on comparing our algorithm with DBSCAN, ρ-approximate
DBSCAN [17], and AnyDBC.
C
1 https://github.com/XFastDataLab/KNN-BLOCK-DBSCAN
c 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
2168-2216 See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
3940
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS, VOL. 51, NO. 6, JUNE 2021
TABLE I
D ESCRIPTION OF M AIN VARIABLES AND S YMBOLS
U SED IN T HIS A RTICLE
The main contributions of this article are listed as follows.
1) We find that the key problem in DBSCAN of identifying
the type of each point is a kNN problem in essence.
Therefore, many techniques of this field, such as
FLANN [18], kd-tree [19], cover tree [20], etc., can be
utilized.
2) According to a general rule that a point has similar density distribution to its neighbors, and neighbor points are
likely to be the same type (core, border, or noise). Then,
a technique is proposed to identify blocks within which
all points have the same type, such as CBs, noncore
blocks, and noise blocks.
3) A fast algorithm is also invented for merging CBs and
assigning noncore points to corresponding clusters.
Before introducing the proposed algorithm, we would like to
present the main variables and symbols used in this article as
follows. Let P be a set of n points in D-dimensional space RD ;
pi ∈ P be the ith point of P; dp,q (or dist(p, q)) be the distance
between points p and q, where the distance can be Euclidean
or Chebychev distance; be the scanning radius of DBSCAN;
dp,(i) be the distance from p to its ith nearest neighbor, and
p(i) be the ith nearest neighbor of p. More symbols are shown
in Table I.
The remainder of this article is organized as follows. Section II introduces the related work of DBSCAN
and nearest neighbor query. Section III revisits FLANN,
DBSCAN, and ρ-approximate DBSCAN. Section IV
addresses the proposed method, KNN-BLOCK DBSCAN,
in detail, including basic ideas, processes, and algorithms.
Section V shows experiments and makes comparison with
ρ-approximate DBSCAN on some data sets. Section VI gives
the final conclusion, and our future work that could improve
the proposed method.
II. R ELATED W ORK
A. Variants of DBSCAN
DBSCAN is designed to discover clusters of arbitrary shape.
It needs two parameters, one is scanning radius , and the other
is MinPts which is used as a density threshold for deciding
whether a point is a core point or not.
If a tree-based spatial index is used, the average complexity is reduced to O(n log(n)) [4]. However, this turns out to
be a misclaim: as pointed out by Gunawan and de Berg [21],
DBSCAN actually runs in O(n2 ) time, regardless of and
MinPts. Unfortunately, this misclaim is widely accepted as
a building brick in many research papers and textbooks,
e.g., [22]–[24], etc. Furthermore, DBSCAN is almost useless in high dimension, due to the so-called “curse of
dimensionality.”
Mahran and Mahar [25] introduced an algorithm named
GriDBSCAN to enhance the performance of DBSCAN
by using grid partitioning and merging, yielding a high
performance with the advantage of a high degree of
parallelism. But this technique is inappropriate for highdimensional data because the effect of redundancy in this
algorithm increases exponentially with dimension. Similarly,
Gunawan and de Berg [21] proposed an algorithm named
Fast-DBSCAN to improve DBSCAN for two-dimensional
(2-D) data, which also imposes an arbitrary grid √
T on 2D space, where each cell of T has side length / 2. If a
nonempty cell c contains at least MinPts points, then this
cell is called core cell, and all points in this cell are core
points because the maximum distance within this cell is .
Therefore, it is unnecessary to compute densities for each
point in a core cell. Gan and Tao [17] proposed an algorithm
named ρ-approximate DBSCAN also based on grid technique
for large data set, and achieved an excellent complexity O(n)
in low dimension. But it degenerates to an O(n2 ) algorithm
in high even relative high-dimensional data space. Besides,
parallel GridDBSCAN [26] and GMDBSCAN [27] are also
grid-based DBSCAN.
AnyDBC [28] compresses the data into smaller densityconnected subsets called primitive clusters and labels objects
based on connected components of these primitive clusters
for reducing the label propagation time. To speedup the range
query process, it uses kd-trees [14] for indexing data, and performs substantially fewer range queries compared to DBSCAN
while still guaranteeing the exact final result of DBSCAN.
There are some other variants of DBSCAN as following. IDBSCAN [29] is a sampling-based DBSCAN, which
is able to handle large spatial databases with minimum
I/O cost by incorporating a better sampling technique, and
reduces the memory requirement for clustering dramatically.
KIDBSCAN [30] presents a new technique based on the concept of IDBSCAN, in which k-means is used to find the
high-density center points and then IDBSCAN is used to
expand clusters from these high-density center points. Based
on IDBSCAN, Quick IDBSCAN [31] (QIDBSCAN) uses
four marked boundary objects (MBOs) to expand computing
directly.
Moreover, because exact clustering is too costly, this has
generated interest in many approximate methods, including our algorithm, to speed up original DBSCAN in the
past two decades. Here, the approximation means that the
clustering result may be different from that of the original
DBSCAN. For example, in original DBSCAN, a data point
p may be classified into one cluster, while in approximate
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
CHEN et al.: KNN-BLOCK DBSCAN: FAST CLUSTERING FOR LARGE-SCALE DATA
Algorithm 1 [18] SearchKmeansTree
1: Input: query point q; the K value of kNN; the maximum
number of examined points L; k-means tree T;
2: count := 0;
3: PQ := empty priority queue
4: R := empty priority queue
5: curNode := T
6: TraverseKmeansTree(curNode, PQ, R, count, q)
7: while PQ <> NULL and count < L do
8:
curNode := top of PQ
9:
TraverseKmeansTree(curNode, PQ, R, count, q)
10: end while
11: Return K top points from R
DBSCAN, it may be designated into another cluster. A scalable RNN-DBSCAN [32] solution was investigated to improve
DBSCAN by using an approximate kNN algorithm. NGDBSCAN [33] is an approximate density-based clustering
algorithm that operates on arbitrary data and any symmetric distance measure. The distributed design of this algorithm
makes it scalable to very large data sets; its approximate nature
makes it fast, yet capable of producing high-quality clustering
results.
B. Nearest Neighbors Searching Algorithms
In the past few decades, many researchers have launched
large amounts of fruitful researches in the field of nearest
neighbor query, many techniques have been proposed and
successfully applied to accelerate the processes of searching neighbors. For example, partition trees (kd-tree [34], [35],
semi-convex hull tree [36]), hashing techniques such as ANN
based on trinary-project tree [37].
Because the exact search is time-consuming for many
applications, then the approximate nearest neighbor query is
optional in some cases, which returns nonoptimal results, but
runs much faster. For example, FLANN [18], [38] uses the
priority search k-means tree or the multiple randomized kd
forest [39] which can give the best performance on a wide
range of dimensional data space. In this article, we mainly
use it to improve the performance of DBSCAN.
III. FLANN, ρ-A PPROXIMATE DBSCAN REVISITED
FLANN: In this article, we use FLANN with the priority
search k-means tree to perform the nearest neighbor query,
where the priority k-means tree is constructed by k-means
(see [18, Algorithm 1]) that partition the data points at each
level into χ (in [18], it is denoted as K which represents
to the cluster number K of k-means tree. In this article,
we use character χ to replace with K in order to make it
different from the K value of kNN), which is called the
branching factor with default value 512, distinct regions recursively, until the total number of points in a region is less
than χ .
As Algorithm 1 shows, given a query point q, the priority
k-means the tree is searched by the following steps.
3941
Algorithm 2 [18] TraverseKmeansTree
1: Input: current node curNode; priority queue PQ; priority
queue R count; query point q
2: if curNode is leaf then
3:
search all points in curNode and add them to R
4:
count := count + |curNode|
5: else
6:
subNodes := sub nodes of curNode
7:
nearestSubNode := nearest node of subNodes to q
8:
subNodes := subNodes -nearestSubNode
9:
PQ := PQ + subNodes
10:
TraverseKmeansTree(nearestSubNode, PQ, R)
11: end if
Fig. 1. Example of core cells. Core cells are shown in gray, and each point
in the core cell is a core point [17].
1) Initially traverse the tree from root to the q s nearest leaf
node, followed by nonleaf node with the closest cluster
center to q, and add all unexplored branches along the
path to a priority queue [(PQ): lines 7–9 in Algorithm 2],
which is sorted in increasing distance from q to the
boundary of the branch being added to the queue.
2) Restart to traverse the tree in the queue from the top
branch (line 10 in Algorithm 2).
Let I be the maximum iteration times of k-means, and L be
the number of examined points by FLANN. The height of the
tree is about log(n)/ log(χ ) if the tree is balanced. During each
traversal from top to down, there are about O(log(n)/ log(χ ))
inner nodes and one leaf node should be checked. Thus, the
complexity of FLANN is about O(LD(log(n)/ log(χ ))), where
L is the number of examined points.
ρ-Approximate DBSCAN: For simplicity, the basic concepts and terms of DBSCAN [4] (e.g., core points, densityreachable, cluster, noise, etc.) are not presented here. Aiming
to improve DBSCAN, ρ-approximate algorithm imposes a
simple quadtree-like hierarchical grid T on D-dimensional
space, and divides the data space into a set of nonempty cells.
Each
√ cell is a D-dimensional hyper-square with side length
/ D. Fig. 1 shows an example in 2-D space. Then, it builds
a graph by redefining the definition and computation of the
graph G = (V, E): 1) each vertex is a core cell and 2) given
two different core cells c1 , c2 .
1) ∃p1 ∈ c1 , p2 ∈ c2 , such that dist(p1 , p2 ) ≤ , there is an
edge between c1 and c2 .
2) If ∃p1 ∈ c1 is within the (1 + ρ)-neighborhood of
p2 ∈ c2 , there is no edge between c1 and c2 .
3) Otherwise, don’t care.
Based on the graph G and the quadtree-like hierarchical
grid, an approximate range counting algorithm is designed to
solving the problem of DBSCAN.
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
3942
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS, VOL. 51, NO. 6, JUNE 2021
IV. P ROPOSED A LGORITHM
A. Drawbacks of DBSCAN Analysis
DBSCAN runs in O(n2 ), and most of its variants still do not
work well for large-scale data. In order to find the underlying
causes, we analyzed fundamental techniques used in traditional
clustering approaches, and find that there are some significant
deficiencies as follows.
1) Brute force algorithm is used in original DBSCAN to
compute density for an arbitrary data point, the complexity is O(n). However, there are many redundancies.
Suppose di,k and dj,k are already known, while di,j is
unknown. Suppose |di,k −dj,k | > or di,k +dj,k ≤ , then
we can infer di,j > or di,j ≤ according to triangle
inequality, respectively. Then, the distance computation
for di,j is also unnecessary.
2) In the case of the grid technique
is used, the side length
√
of each cell is fixed to / D, which implies that it is
almost useless in high dimension [40].
eps
eps/2
eps
Fig. 2. Example of a CB. MinPts = 8, = eps and there are eight red points
which are within N(/2) (p), then all red points are core points.
r−eps
eps
q p eps
r
Fig. 3. Example of an NCB. MinPts = 22, = eps, r > eps, and the total
number of points within Nr (p) (the outer circle) is 21, then all red points are
noncore points, because they are all within Nr− (p).
B. Basic Ideas
As mentioned above, DBSCAN cannot deal with large-scale
data due to its high complexity. According to our observation
and analysis on DBSCAN, there are two findings as follows.
1) The key problem of DBSCAN is to find core points,
which is a kNN problem in essence, because the density
defined in DBSCAN is the total number of points within
a specified neighborhood, and all neighbors of a core
point should be reported for merging.
2) Point p and point q should have similar neighborhoods,
provided p and q are close; the closer they are, the
more similar neighborhood they have. Thus, it is highly
possible that a point has the same type as its neighbors.
Hence, it is reasonable to utilize kNN technique to solve
the problem of DBSCAN. Formally, let K = MinPts and
p(1) , . . . , p(K) be the first K nearest neighbor points of p, where
1 ≤ i ≤ K, then we have the following.
Theorem 1:
1) If dp,(K) ≤ , then p is a core point.
2) p is a noncore point if dp,(i) > , where 1 ≤ i ≤ K.
Proof: 1) Because dp,(K) ≤ , which means dp,(1) ≤ dp,(2)
≤, . . . , ≤ dp,(K) ≤ , |N (p)| ≥ K = MinPts, p is a core point.
2) Because 1 ≤ i ≤ K and dp,(i) > , < dp,(i) ≤ dp,(K) .
Thus, |N (p)| < K = MinPts, i.e., p is a noncore point.
As a result of Theorem 1, we argue that the problem of
identifying whether a point is a core point or not is a kNN
problem.
Theorem 2: If dp,(K) ≤ (/2), p(1) , p(2) , . . . , p(K) are all
core points.
Proof: Because dp,(K) ≤ (/2) ≤ , according to triangle
inequality, we have ∀i, j ∈ [1, K] dist(p(i) , p(j) ) ≤ . Therefore,
∀i ∈ [1, K] we have |N (p(i) )| ≥ K, i.e., p(1) , p(2) , . . . , p(K)
are all core points.
Definition 1 (Core-Block (CB)): Nξ (p) is a CB with respect
to p and ξ , if ∀q ∈ Nξ (p) is core point. It is noted as CB(p, ξ ),
and p is called the center of CB(p, ξ ).
As Fig. 2 shows, all red points are within N(/2) (p), and
the total number of red points is 8 which is equal to MinPts,
p
pi
r−eps
p
r−2*eps
eps
r
Fig. 4. Example of a noise-CB. MinPts = 22, = eps and r > 2, then all
red points within green circle are noise, because Nr− (p) is noncore block
which implies there is no core point within the red circle.
then according to Theorem 2 all red points are core points.
Therefore, N(/2) (p) is a CB.
Theorem 3: Let dp,(K) = r, (1) if r > , then ∀q ∈ Nr− (p)
is noncore point. (2) if r > 2, then ∀q ∈ Nr−2 (p) is noise.
Proof:
1) Because dp,(K) = r > , which means ∀q ∈ Nr− (p),
N (q) ∈ Nr (p), therefore, |N (q)| < |Nr (p)| = MinPts. Thus,
q is a noncore point.
2) Because dp,(K) = r > 2, then ∀q ∈ Nr−2 (p), we have
N (q) ∈ Nr− (p), and because Nr− (p) is a noncore-block
(NCB), which implies there is no core point in N (q), then q
is noise.
Definition 2 (None-Core Block (NCB)): Nξ (p) is an NCB
with respect to p and ξ , if ∀q ∈ Nξ (p) is noncore point. It is
noted as NCB(p, ξ ), and p is called the center of NCB(p, ξ ).
Definition 3 (Noise-Block (NOB)): Nξ (p) is an NOB with
respect to p and ξ , if ∀q ∈ Nξ (p) is noise. It is noted as
NOB(p, ξ ), and p is called the center of NOB(p, ξ ).
Obviously, an NOB is NCB, but an NCB may not be NOB;
neither NCB nor NOB is CB, and vice versa.
Fig. 3 addresses an example of Theorem 3 (1). Because
MinPts = 22, = eps and r > eps, it is impossible for each
point within the blue circle to find enough neighbors within
its -neighborhood, (because the total number of points within
Nr (p), i.e., the outer circle, is 21). Thus, all points within the
blue circle are noncore points, i.e., Nr− (p) is an NCB.
Fig. 4 is another example to explain Theorem 3 (2). Because
r > 2, all points within green circle are noncore points, and
it is also impossible for any point p within green circle to
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
CHEN et al.: KNN-BLOCK DBSCAN: FAST CLUSTERING FOR LARGE-SCALE DATA
Fig. 5. Framework of KNN-BLOCK DBSCAN. It uses FLANN to identify
CBs, NCBs, and NOB, then merges CBs, assigns points in NCBs to proper
clusters and discards noises.
Algorithm 3 KNN-BLOCK DBSCAN(P, , MinPts)
1: Input: P is input data; [, MinPts];
2: Output: cluster id of each point;
3: Initialize core-blocks set CBs = {φ}
4: Initialize non-core-blocks set NCBs = {φ}
5: K := MinPts, cur_cid := 0
// current cluster id
6: for each unvisited point p ∈ P do
7:
{p(1) , . . . , p(K) } := FLANN :: kNN(p, P)
8:
ξ := dp,(K) , Nξ (p) := {p(1) , p(2) , . . . , p(K) }
9:
if ξ ≤ then
10:
cur_cid := cur_cid + 1
11:
if ξ ≤ 2 then
12:
push Nξ (p) into CBs //a core block found
13:
∀s ∈ Nξ (p) mark s as core-point and visited
14:
else
15:
push N0 (p) into CBs //single core point
16:
mark p as core-point and visited
17:
end if
18:
curCorePts:= core points already found in Nξ (p)
19:
exist_cids:= clusters found in curCorePts
20:
merge exist_cids into cur_cid
21:
assign Nξ (p) to cluster cur_cid
22:
else if < ξ ≤ 2 then
23:
push Nξ − (p) into NCBs
24:
mark all points within Nξ − (p) as visited
25:
else if ξ > 2 then
26:
mark ∀q ∈ Nξ −2 (p) as noise and visited
27:
end if
28: end for
29: CBCENT := extract all center points from CBs
30: Create a index tree by FLANN from CBCENT
31: MergeCoreBlocks(CBs, CBCENT, cbIDs, )
32: AssignNonCoreBlocks(NCBs, CBs, CBCENT, )
find any core point from which p is directly density-reachable,
because Nr− (p) is noncore block which implies there is no
core point within the red circle. Thus, points within Nr−2 (p)
are all outliers, i.e., Nr−2 (p) is an NOB.
Definition 4: A core block CB(p, ξ1 ) is density-reachable
from another core block CB(q, ξ2 ), if ∃s ∈ CB(p, ξ1 ) and
w ∈ CB(p, ξ2 ), such that s is density-reachable from w.
Definition 5: A point p is density-reachable from core block
CB(q, ξ ), if ∃s ∈ CB(q, ξ ) such that p is density-reachable
from q.
Comprehensively, based on the two findings mentioned
above, the difference of between this article and other variants
3943
Algorithm 4 MergeCoreBlocks(CBs, )
1: Input: CBs: core-blocks; CBCENT: core-block centers
set; is the parameter of DBSCAN ;
2: for each core-block CB(p, ξ1 ) do
3:
Neibs := FLANN::RangeSearch(p, 2, CBCENT)
4:
for each q ∈ Neibs do
5:
CB(q, ξ2 ) be the core-block of q
6:
if p and q are in different cluster then
7:
if dp,q > ξ1 + ξ2 + then
8:
BruteForceMerge(CB(p, ξ1 ), CB(q, ξ2 ))
9:
end if
10:
end if
11:
end for
12: end for
Algorithm 5 AssignNonCoreBlocks(NCBs, CBs, )
1: Input: NCBs: non-core-blocks; CBs: core blocks; is the
parameter of DBSCAN;
2: for each non-core-block NCB(p, ξ1 ) do
3:
r := ξ1 + 1.5;
4:
Neibs := FLANN::RangeSearch(p, r,CBCENT)
5:
if ∃q ∈ Neibs s.t. dp,q ≤ ( − ξ1 ) then
6:
merge NCB(p, ξ1 ) into the cluster of q
7:
process next non-core-block
8:
else
9:
for each unclassified o ∈ NCB(p, ξ1 ) do
10:
if ∃q ∈ Neibs s.t. dp,q ≤ ( + ξ1 + ξ2 ) then
11:
if ∃s ∈ CB(q, ξ2 ) s.t.do,s ≤ then
12:
assign o to the cluster of q
13:
process next unclassified point o
14:
end if
15:
end if
16:
end for
17:
end if
18: end for
of DBSCAN mainly lies in: 1) kNN is used, instead of using
range query algorithm, to identify core points and noncore
points by block (CBs, NCBs, and NOBs); 2) each block has a
dynamic range, while the width of grid used in ρ-approximate
DBSCAN and fast DBSCAN is a constant; and 3) CBs can
be processed by a simple way which is far more efficient than
grid.
C. Algorithms
In this section, we outline the proposed method. The framework of KNN-BLOCK DBSCAN is shown in Fig. 5. First, it
uses FLANN to identify CBs, NCBs, and NOB. Second, for
any two pairs of CBs, it merges them into the same cluster
provided they are density-reachable from each other. Third, for
each point p in NCBs, KNN-BLOCK DBSCAN may assign
p to a cluster if there exists a core point from which it is
density-reachable. The details are shown in Algorithm 3, 4, 5,
and 6, respectively.
1) Types and Blocks Identification: As Algorithm 3 shows,
for each an unvisited point p in P, it uses FLANN::kNN to
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
3944
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS, VOL. 51, NO. 6, JUNE 2021
Algorithm 6 BruteForceMerge(CB(p, ξ1 ), CB(q, ξ2 ))
1: Input: CB(p, ξ1 ): a core-block; CB(q, ξ2 ): another coreblock;
2: Initialize two points set O = {φ} and S = {φ}
3: for each point o in CB(q, ξ2 ) do
4:
push o to O if do,p < + ξ1
5: end for
6: for each point s in CB(p, ξ1 ) do
7:
push s to S if ds,q < + ξ2
8: end for
9: if ∃o ∈ O, s ∈ S, s.t. do,s ≤ then
10:
merge CB(p, ξ1 ) and CB(q, ξ2 ))
11: end if
Fig. 6. Three cases of two CBs. (a) Two CBs can be merged directly. (b) Is
a case that can skip directly for they are far from each other. (c) Addresses
the third case that is necessary to check in detail.
retrieve the first K (K = MinPts) nearest neighbors of p.
According to Theorem 1, the type of p can be identified.
If p is a core point, we may find a core block according
to Theorem 2 (lines 11–13). If p is not a core point, we may
find an NCB (lines 22–24) or noise block (lines 25 and 26)
according to Theorem 3.
2) Blocks Merging: Let CB(p, ξ1 ) and CB(q, ξ2 ) be two
CBs, there are three cases as described below.
Case 1 (dp,q ≤ ): As image (a) in Fig. 6 shows, because p
is directly density-reachable from q, both CBs can be merged
into a same cluster directly.
As shown from lines 20 and 21 in Algorithm 3, suppose
CB(p, ξ1 ) is a newly identified CB, and if there are some
points that have already been assigned to other clusters within
CB(p, ξ1 ), then these clusters can be merged directly.
Case 2 (dp,q > ( + ξ1 + ξ2 )): As illustrated in Fig. 6 (b),
they are far away from each other, there is no need to merge
them, because according to triangle inequality, there is no point
in CB(p, ξ1 ) that is density-reachable from another point in
CB(q, ξ2 ).
Case 3 ( < dp,q ≤(ξ1 + ξ2 + )): As Fig. 6(c)
addresses, CB(p, ξ1 ) and CB(q, ξ2 ) have no intersection,
and they can be merged if there exists a pair of points
(o1 , o2 ) where dist(o1 , o2 ) ≤ , o1 ∈ CB(p, ξ1 ) and
o2 ∈ CB(q, ξ2 ).
In order to detect this case effectively, a simple method is
proposed as Algorithm 6 illustrates. First, we select point set
O ⊆ CB(q, ξ2 ) such that ∀m ∈ O s.t. dp,m ≤ + ξ1 , and
point set S ⊆ CB(p, ξ1 ) such that ∀s ∈ S s.t. dp,m ≤ + ξ2 .
Then, we simply utilize brute force algorithm to check whether
there exist two points o ∈ O, s ∈ S that are directly
density-reachable from each other, and merge two CBs if
yes. As Fig. 7 shows, set O is within the right shadow
region, while S is within the left shadow region. Only points
ξ2
ξ1
ε+ξ 1
p
q
ε+ξ 2
Fig. 7. Example of case (3) for merging CBs. CB(p, ξ1 ) is a CB, CB(q, ξ2 )
is another CB, only points in the two shadow region are possible directly
density-reachable from each other.
in the two shadow regions are checked, instead of whole
two CBs.
3) Borders Identification: At last, given a CB CB(p, ξ1 ) and
an NCB NCB(q, ξ2 ), Algorithm 5 (AssignNonCoreBlocks) is
called to identify border points in NCB(q, ξ2 ) that are densityreachable from CB(p, ξ1 ). Similar to Fig. 6, there are also three
cases as described below.
Case 1 (dp,q >( + ξ1 + ξ2 )): NCB(q, ξ2 ) is far from
CB(p, ξ1 ), then it is unnecessary to merge them.
Case 2 (dp,q ≤( − ξ2 )): Because NCB(q, ξ2 ) is totally
contained in N (p), all points within NCB(q, ξ2 ) are densityreachable from p. Therefore, all points in NCB(q, ξ2 ) are
assigned to the cluster of p directly.
Case 3 (( − ξ2 )< dp,q ≤( + ξ2 )): it is necessary to check
whether each point within NCB(q, ξ2 ) is density-reachable
from p. Similar to Fig. 7, only points within two shadow
regions are checked.
D. Complexity Analysis
Let n be the cardinality of data set, b0 = b1 + b2 + b3 be the
total number of all blocks, where b1 , b2 , and b3 are the total
number of CBs, NCBs, and NOBs, respectively. Averagely,
b0 = β(n/MinPts), where β is a factor about the distribution of the data, and b0 is usually far less than n provided
[, MinPts] are well chosen (how to choose good parameters
for DBSCAN is another big topic, such as OPTICS [13] and
others [41]–[43], which is out of the scope of this article). The
complexity of Algorithm 3 is analyzed as follows.
Space Complexity: As shown in the above algorithms, we
can see that each block should be saved, thus the space cost
is about O(MinPts ∗ b0 ) = O(βn).
Time Complexity:
1) From lines 6–29 of Algorithm 3, we can infer that
FLANN::kNN will be called about b0 times. As we
know, in the case of using priority search k-means tree,
FLANN::kNN runs in O(L D log(n)/ log(χ )) expected
time [18] for each query, where L is a data points examined by FLANN, D is dimension, and χ is a branching
factor of the tree used in FLANN. Thus, the complexity
of finding blocks is about O(b0 [L D log(n)/ log(χ )]).
2) The complexity of creating a tree by FLANN from
CBCENT is about O(b1 D log(b1 )).
3) The complexity of Algorithm 4: There are two main
parts as follows.
CBs,
for
each
CB
a) There
are
b1
FLANN::RangeSearch is called to find its
2-neighbors from CBCENT, the complexity is
about O(b1 [L d log(b1 )/log(χ )]).
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
CHEN et al.: KNN-BLOCK DBSCAN: FAST CLUSTERING FOR LARGE-SCALE DATA
b) For each CB, the total number of points in a CB
is usually far less than n, i.e., MinPts << n, then
the complexity of Algorithm 6 is averagely about
O(MinPts).
Hence, since MinPts << n can be regarded as
a constant, the complexity of Algorithm 4 is about
O(b1 [L D log(b1 )/ log(χ )]).
4) The complexity of Algorithm 5: there are also two main
parts as follows.
a) There are b2 NCBs. For each NCB we call
FLANN::RangeSearch to find its (ξ1 + 1.5)neighbors from CBCENT, the complexity is about
O(b2 [L D log(b1 )/log(χ )]).
b) The average complexity of assigning an unclassified point in NCBs to a cluster (from line 5 to line
17) is about O(MinPts[L D log(b1 )/ log(χ )]).
Hence, the complexity of Algorithm 5 is less
log(b1 )/ log(χ )])
<
than O(b2 MinPts [L D
O(UCPtsNum [L D log(b1 )/ log(χ )]), where UCPtsNum is
the total number of unclassified points in all NCBs.
As mentioned above, b0 = b1 + b2 + b3 =
(βn/MinPts) is far less than n provided [, MinPts]
are well chosen, then the overall time complexity is
about as O(b0 [L D log(n)/ log(χ )]) = O ([βn/MinPts]
[L D log(n)/ log(χ )]) < O(L D n log(n)/ log(χ )).
In the case of dealing with very high dimensional data
sets, FLANN::kNN degenerates to be an O(n) algorithm, and
then the complexity of KNN-BLOCK DBSCAN is about
O(b0 [L D n/ log(χ )]).
In the worst case, if there is none CB and FLANN::kNN
runs in O(n), the complexity of KNN-BLOCK DBSCAN is
O(n2 ).
V. E XPERIMENTS
A. Algorithms and Set Up
In this section, to evaluate the correctness and effectiveness
of the proposed approach, several experiments are conducted
on different data sets at Intel Core i7-3630 CPU @2.50 GHz,
8G RAM. We mainly compare the proposed algorithm with
ρ-approximate DBSCAN, AnyDBC [28] and pure kNN-based
DBSCAN.
1) “KNN-BLOCK” is KNN-BLOCK DBSCAN which is
coded in C++ and runs on Windows 10 64-bit operating system, the tree used in FLANN is priority search
k-means tree, and the cluster number χ of k-means is
10.
2) Approx is ρ-approximate DBSCAN which is also written in C++ and runs on Linux (Ubuntu 14.04 LTS)
operating system.
3) AnyDBC is the efficient anytime density-based clustering algorithm [28].
4) kNN-based DBSCAN is an algorithm which only uses
FLANN::kNN technique to accelerate DBSCAN, as
shown in Algorithm 7, and the complexity is about
O(L D n log(n)/ log(χ )), where L is a data points examined by FLANN, D is dimension, and χ is a branching
factor of the tree used in FLANN.
3945
Algorithm 7 Pure kNN-Based DBSCAN
1: Input: data set P, and , MinPts;
2: coreSet := {φ}
3: for each unclassified p ∈ P do
4:
neibors:= FLANN::kNN(p,MinPts);
5:
if dp,(MinPts) ≤ then
6:
push p into coreSet
7:
end if
8: end for
9: for each core point p ∈ coreSet do
10:
neibCores := find core points from k-neighbors of p
11:
merge neibCores and p into one cluster
12: end for
13: for each two pair of clusters c1 and c2 do
14:
merge c1 and c2 if ∃p1 ∈ c1 and p2 ∈ c2 s.t. p1 is
density reachable p2
15: end for
16: find border points and assign them
B. Data Sets
Data sets come from UCI (https://archive.ics.uci.edu/ml/ind
ex.php), including PAM (PAMPA2), HOUSE (household),
USCENCUS (USCensus 1990), gas sensor, FMA (dataset for
music analysis), AAS-1K (Amazon access samples), HIGGS,
etc., where AAS-1K is a 1000-dimensional data set which
is extracted from 20 000-dimensional data set AAS. For each
data set, all duplicate points are removed to make each point
unique, all missing values are set to 0, and each dimension of
these data sets is normalized to [0, 105 ]. The following part
of this section lists brief descriptions of these data sets.
PAM39D is a real 39-dimensional dataset, PAMAP2, with
cardinality n = 3, 850, 505; PAM4D is a real dataset obtained
by taking the first four principle components (PCA) of
PAMPA2; Household: dim = 7, n = 2049280; USCENCUS:
dim = 36, n = 365100; GasSensor (Ethylene-CO): dim = 16,
n = 4208261; MoCap: dim = 36, n = 65536; APS (APS
Failure at Scania Trucks): dim = 170, n = 30000; Font
(CALIBRI): dim = 36, n = 19068; HIGGS: dim = 28,
n = 11000000; FMA: dim = 512, n = 106574; AAS − 1K:
AAS is a large sparse data set, and AAS-1K is a subset extracted
from AAS with dim = 1000, n = 30000.
C. Two Examples of Clustering
We benchmark KNN-BLOCK DBSCAN on two 2-D test
cases to reveal the processes in detail, as shown in Fig. 8.
The left case is aggregation [44], and the right case comes
from [1].
Specifically, in Fig. 8(a) presents the original data distribution. Fig. 8(b) draws CBs, NCBs, and NOBs plotted by black,
green, and red circles, respectively. The radius of each circle is
different, which means each block has a different size. We also
can see that NCBs usually distribute along the border of CBs,
and NOBs appears far from CBs. Fig. 8(c) illustrates the result
of merging CBs, which is the most important step to identify
clusters; in Fig. 8(d), as mentioned in Section IV-C3, there
are three cases to process NCBs: the green circles represent
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
3946
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS, VOL. 51, NO. 6, JUNE 2021
30
30
30
20
20
20
10
10
10
0
0
20
40
0
0
20
40
0
0
30
30
30
20
20
20
10
10
10
0
0
20
(d)
40
0
0
20
(e)
20
40
(c)
(b)
(a)
40
0
0
20
40
(f)
0.5
0.5
0.5
0
0
0
−0.5
−0.5
0
(a)
0.5
−0.5
−0.5
0
(b)
0.5
−0.5
−0.5
0.5
0.5
0.5
0
0
0
−0.5
−0.5
0
(d)
0.5
−0.5
−0.5
0
(e)
0.5
−0.5
−0.5
0
0.5
0
0.5
(c)
(f)
Fig. 8. Two examples present the processes of KNN-BLOCK DBSCAN. (a) is the original data distribution; (b) shows 3 kinds of blocks found by
KNN-BLOCK DBSCAN, where black circles are core blocks, green circles are NCBs and red are NOBs; (c) illustrates clusters found after merging CBs;
(d) addresses the assignment of NCBs to corresponding clusters, the red balls are NCBs that can be assigned to their nearest clusters, and green circles in
(d) are those who find no cluster to assign; (e) exhibits the final result of KNN-BLOCK DBSCAN, where black points are noise; (f) demonstrates the result
of original DBSCAN.
TABLE II
RUNTIME C OMPARISONS ON S UBSETS OF HOUSE AND PAM W ITH n I NCREASING . T HE S PEEDUP OF KNN-BLOCK
DBSCAN OVER I TS C OMPETITOR I S G IVEN IN B RACKETS . (U NIT: S ECOND )
case (1), because they are far from all core-points, all points
within these NCBs are classified as noise; the red balls illustrate case (2), each of them is assigned to one cluster from
which it is density-reachable; in case (3), for each point p
within unclassified NCBs, if q is identified as a core from
which p is density-reachable, then p is classified to the cluster of q. Fig. 8(e) exhibits the final result of KNN-BLOCK
DBSCAN, where black points are noise; and Fig. 8(f) shows
the result obtained by original DBSCAN.
It is observed that KNN-BLOCK DBSCAN nearly obtains
the same result as DBSCAN with high efficiency, because
it processes data by blocks, and reduces a large number of
redundant distance computations.
D. Runtime Comparisons With ρ-Approximate DBSCAN
The first experiment is conducted on a set of subsets
of HOUSE and PAM4D to observe the complexities of the
proposed algorithm and ρ-approximate DBSCAN with different [, MinPts]. Figs. 9 and 10 present the results of two
algorithms, and Table II reveals more details. We also conduct experiments on the whole data sets of HOUSE, PAM4D,
KDD04, USCENCUS, REACTION, MOPCAP, BODONI,
HIGGS, FMA, and AAS1K, respectively, and Table III shows
the comparison of runtime with different [, MinPts].
From two figures and two tables, we can observe as follows.
1) Both algorithms prefer large and small MinPts. For
example, on data set HouseHolod, both KNN-BLOCK
DBSCAN and ρ-approximate DBSCAN run best when
[, MinPts] = [5000, 100], and the worst case happens
when [, MinPts] = [1000, 200]. On other data sets,
things are similar as shown in Table III.
2) Both algorithms run in linear expected time in low
dimensional data sets.
3) We can see that on large-scale data sets
PAM4D, HOUSEHOLD, and HIGGS, our algorithm
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
CHEN et al.: KNN-BLOCK DBSCAN: FAST CLUSTERING FOR LARGE-SCALE DATA
3947
TABLE III
RUNTIME C OMPARISONS ON D IFFERENT DATA S ETS W ITH D IFFERENT AND M IN P TS . T HE S PEEDUP OF
KNN-BLOCK DBSCAN OVER I TS C OMPETITOR I S G IVEN IN B RACKETS . (U NIT: S ECOND )
is much better, the speedup of KNN-BLOCK
DBSCAN over its competitor is about 2.5–6 times
on HOUSEHOLD, 1.4–3 times on PAM4D, and 16
and 17 times on HIGGS (28 dim). On other relative
high-dimensional data sets, e.g., MOPCAP (36 dim)
APS (170 dim), BODONI (256 dim), FMA (512 dim),
and AAS-1K(1000 dim) KNN-BLOCK DBSCAN still
performs well, while ρ-approximate degenerates to be
an O(n2 ) algorithm which conforms to our analysis
mentioned in Section II. It is also notable that the
performance of KNN-BLOCK DBSCAN drops with
the dimension, e.g., the proposed algorithm spends
much more time on HIGGS than that on PAM4D, and
the should be relatively larger in high dimension than
that of low dimension.
From these experiments, we can see that KNN-BLOCK
DBSCAN accelerates ρ-approximate DBSCAN greatly, and
is promising for processing such large-scale data.
Fig. 9.
Runtime comparisons on subsets of HOUSE with n increasing.
E. Runtime Comparisons With AnyDBC
To make comparisons with AnyDBC, we conduct experiments on the two same data sets, namely, GasSensor
(Ethylence-co) and PAM39D, as shown in Fig. 11 (the result
of AnyDBC is obtained by running the binary program
provided by the authors on our machine). It is observed
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
3948
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS, VOL. 51, NO. 6, JUNE 2021
Fig. 10.
Runtime comparisons on subsets of PAM4D with n increasing.
TABLE IV
RUNTIME C OMPARISONS W ITH P URE K NN-BASED DBSCAN
Fig. 11. Runtime comparisons with AnyDBC and ρ-approximate DBCAN
on Gas Sensor and PAM39D, MinPts is fixed to 50.
that KNN-BLOCK DBSCAN outperforms AnyDBC and
ρ-approximate DBSCAN, especially, on PAM39D KNNBLOCK DBSCAN runs far faster than AnyDBC.
F. Runtime Comparisons With Pure kNN-Based DBSCAN
In this part, KNN-BLOCK DBSCAN is compared with
pure kNN-based DBSCAN on some data sets, and the results
are shown in Table IV. From this table, we can see that
KNN-BLOCK DBSCAN runs far faster than pure kNN-based
algorithm, and the speedup varies from 1.42 to 5.48. Clearly,
in most cases, the speedup is larger than 2, which proves that
the block techniques plays an important role in our algorithm,
and greatly speedup DBSCAN.
G. Effect of and MinPts
In this section, we check the effect of [, MinPts] on the
proposed algorithm. PAM4D is used in this experiment with
Fig. 12. Runtime distributions with the changing of and MinPts on PAM4D
and HOUSEHOLD, respectively.
cardinality 3 850 505 and the dimension is 4. Table V reveals
the execution details of kNN, MergeCB (Algorithm 2) and
AssignNCB (Algorithm 3), as well as the numbers of CBs,
NCBs, and NOBs.
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
CHEN et al.: KNN-BLOCK DBSCAN: FAST CLUSTERING FOR LARGE-SCALE DATA
3949
TABLE V
E XECUTION T IMES OF K NN, M ERGE CB, AND A SSIGN NCB, AS W ELL AS B LOCKS F OUND ON PAM W ITH D IFFERENT [, MinPts]
TABLE VI
T OTAL N UMBERS OF CB S , NCB S , AND NOB S F OUND ON D IFFERENT
DATA S ETS
TABLE VIII
C OMPARISONS OF NMI FOR KNN-BLOCK DBSCAN AND
ρ-A PPROXIMATE DBSCAN (n = 5000)
TABLE VII
C OMPARISONS OF Omega-index FOR KNN-BLOCK DBSCAN AND
ρ-A PPROXIMATE DBSCAN (n = 5000)
TABLE IX
E XAMPLE OF C OMPUTING P RECISION FOR KNN-BLOCK DBSCAN
BASED ON T HREE M ATCHED L ABEL PAIRS : (“A1,” “B2”), (“A2,” “B1”),
AND (“A3,” “B4”) F OUND BY K UHN –M UNKRAS
As the two bold columns show, the execution times of kNN
is the same as the number of blocks found by KNN-BLOCK
DBSCAN. It is observed: 1) the runtime and execution times
of kNN linearly increase with MinPts; 2) while the execution
times of MergeCB rapidly decrease with MinPts; and 3) the
less CBs the more NCBs and NOBs.
Fig. 12 also provides more details of the runtime distribution
on PAM4D and HOUSEHOLD with the changing of and
MinPts, respectively. Hence, we can infer: 1) the complexity
of KNN-BLOCK DBSCAN mainly depends on the execution
times of kNN and 2) KNN-BLOCK DBSCAN prefers large
and small MinPts, which yields less executions of kNN due
to larger number of CBs identified.
1) In the case of MinPts is small and is large, most
blocks will be identified as CBs, and the number is about
N/MinPts. For example, as Table V shows, [7000, 100]
runs fastest, followed by [5000, 100], then [3000, 100],
and then [1000, 100].
2) When MinPts is large and is small, few CBs are
found, thus kNN will be called more frequently, and
it will degenerate to be an O(n2 ) algorithm in the worst
case. As shown in Table III, when the parameters are
[1000, 30 000] and [3000, 30 000], the runtime is much
longer than others.
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
3950
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS, VOL. 51, NO. 6, JUNE 2021
TABLE X
ACCURACY, R ECALL , AND F1-S CORE OF KNN-BLOCK DBSCAN AND ρ-A PPROXIMATE
DBSCAN ON S UBSETS OF HOUSE, PAM4D, MOPCAP, AND APS
H. Statistics of Three Kinds of Blocks
J. Accuracy of KNN-BLOCK DBSCAN
In this section, to observe the numbers of three kinds
of blocks with respect to different and MinPts, some
experiments are conducted on some whole data sets, including HOUSE, PAM4D, KDD04, USCENCUS, REACTION,
MOPCAP, and BODONI, respectively.
Table VI exhibits some statistics of CBs, NCBs, and NOBs
with respect to different and MinPts on all data sets. From
this table, we can see: the total number of blocks, especially
CBs, is far less than the cardinality n, which reveals that many
distance computations are filtered.
To evaluate the accuracy of KNN-BLOCK DBSCAN, some
experiments are conducted based on an assumption that the
clustering labels obtained by DBSCAN are ground truth. The
reason is as follows.
1) This article is only motivated to accelerate the speed
of DBSCAN, without concerning the clustering result
is good or not which is another topic out of the scope
of this article. It is expected that the clustering results
should be the same as original DBSCAN provided the
parameters (, MinPts) are the same.
2) Both KNN-BLOCK DBSCAN and ρ-approximate
DBSCAN are approximate algorithms, the more similar of their clustering results to the original DBSCAN,
the better. Hence, we argue it is reasonable to use the
clustering results of DBSCAN as ground truth.
Specifically, the idea is that each data point belongs to a
unique predefined cluster and its predicted cluster should correspond either to only one predefined cluster or to none [11].
Any pair of data points in the same predefined cluster is considered to be incorrectly clustered if the predicted cluster does
not match the predefined cluster to which they belong, even
if both points appear in the same predicted cluster. Therefore,
we evaluate the precision of two approaches as follows.
Step 1 (Clustering): Given a data set and [, MinPts], suppose Lab1 = {A1 , A2 , . . . , Ak } and Lab2 = {B1 , B2 , . . . , Bm }
I. Omega-Index and NMI Evaluations
Omega-Index [45] and normalized mutual information
(NMI) [46] are two well known methods to evaluate clustering
result, then similar to [47], we use them to make comparisons
for KNN-BLOCK DBSCAN and ρ-approximate DBSCAN.
Because the complexities of Omega-index and NMI are high
(O(n2 )), we only conduct experiments on sub sets of HOUSE,
PAM4D, MOPCAP, and APS with n = 5000.
In these experiments, we compute the Omega-index
and NMI scores of both algorithms by comparing the
results with those obtained from the original DBSCAN. As
Tables VII and VIII show the performances of both algorithms
are similar, and the results are all close to 1, which indicate
that both algorithms nearly agree on original DBSCAN.
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
CHEN et al.: KNN-BLOCK DBSCAN: FAST CLUSTERING FOR LARGE-SCALE DATA
are clustering labels obtained by the DBSCAN and KNNBLOCK DBSCAN.
Step 2 (Matching): It is well known that different clustering
algorithms may yield different labels on the same data set. For
example, cluster “A1” labeled by DBSCAN may be the same
as “B2” obtained by KNN-BLOCK DBSCAN. Hence, it is
reasonable to match labels first, and use the matched labels
to compute Accuracy. In this article, Kuhn–Munkras [48] performs the task of maximum matching two different cluster
label sets, which has been used in [11] and [49].
Step 3 (Computing Accuracy): Suppose there are three clusters with labels “A1,” “A2,” and “A3” obtained by DBSCAN
on one data set, but KNN-BLOCK DBSCAN labels them
with “B1,” “B2,” “B3,” and “B4,” and Kuhn–Munkras finds
there are three matched pairs: (“A1,”‘ “B2,”) (“A2,” “B1,”) and
(“A3,” “B4.”) If the labels of point p obtained by DBSCAN
and KNN-BLOCK DBSCAN match, then the prediction of p
is correct, e.g., (“A1” and “B2,”) otherwise it is wrong, e.g.,
(“A1” and “B1”). Table IX shows more details. Suppose there
are eight points in the data set, the second row lists labels
obtained by DBSCAN, and the third line is the clustering
result of KNN-BLOCK DBSCAN. We can see that there are
two cases that are wrongly predicated because (A1, B4) and
(A2, B3) are not matched pairs. Therefore, the total precision
is (8 − 2)/8 = 75%.
Because the original DBSCAN has high complexity, we
only test on small data sets. Here, we extract four subsets from
HOUSE, PAM4D, APS, and MOCAP, and use them as test
cases. Also because DBSCAN is nondeterministic (sensitive
to iteration order), some border points may be assigned to different clusters according to the order they appear. Therefore,
the accuracy is computed only by comparing core points.
Table X shows that both algorithms achieve high accuracy.
In low-dimensional data sets (HOUSE and PAM4D), the
precision, recall, and F1-score of both approximate algorithms are about 98%–100%, and there is only a little drop
in high-dimensional data sets (MOCAP and APS) which are
about 94.5%–97.7%.
VI. C ONCLUSION
DBSCAN runs in O(n2 ) expected time and is not suitable
for large-scale data. ρ-approximate DBSCAN is designed to
replace with DBSCAN for big data, however, it only can work
in a very low dimension. In this article, we analyze the underlying causes that current approaches fail in clustering large
scale data, and find that the grid technique is nearly useless
for high-dimensional data.
Aiming to tame problems mentioned above, an approximate approach named KNN-BLOCK DBSCAN is proposed
for large-scale data based on two findings: 1) the key
of DBSCAN to find core points is a kNN problem
in essence and 2) a point has similar density distribution to its neighbors, which implies it is highly possible
that a point has the same type (core/border/noise) as its
neighbors.
Therefore, we argue that the kNN technique, e.g., FLANN,
can be utilized to identify CBs, NCBs, and NOBs, which only
3951
includes core points, border points, and noises, respectively.
Then, we proposed an algorithm to merge CBs that are densityreachable from each other and assign each point in NCBs to
a proper cluster.
The superiority of KNN-BLOCK DBSCAN to
ρ-approximate DBSCAN is that it processes data by blocks,
each of which has a dynamic range, instead of grids used
in ρ-approximate DBSCAN with a fixed width, and fast the
kNN technique is used to identify the types of points. Given a
fixed intrinsic dimensionality, the complexity of the proposed
algorithm is about O([βn/MinPts][L D log(n)/ log(χ )])
where L is a constant, D is dimension, β is a factor of data
distribution, and χ is the branching factor of the tree used in
FLANN.
Experiments address that KNN-BLOCK DBSCAN runs
faster than ρ-approximate DBSCAN and pure kNN-based
DBSCAN with high accuracy, even on some relative highdimensional data sets, e.g., APS (170 dim), BONONI
(256 dim), FMA (512 dim), and AAS-1K (1000 dim), where
ρ-approximate DBSCAN degenerates to be an O(n2 ) algorithm, KNN-BLOCK DBSCAN can still run very fast.
Our future work is to improve the proposed algorithm and
apply it in real applications in the following aspects.
1) Try to use other precise the kNN technique, such as
cover tree, semi-convex hull tree [36], etc., to improve
the accuracy of KNN-BLOCK DBSCAN.
2) Parallelize KNN-BLOCK DBSCAN on GPUs with a
highly efficient strategy for scheduling data to make the
proposed algorithm faster.
3) Apply it in our other researches, such as image
retrieval [50], vehicle reidentification [51], [52], vehicle crushing analysis [53], and auditing for shared cloud
data [54]–[56].
R EFERENCES
[1] A. K. Jain, “Data clustering: 50 years beyond K-means,” Pattern
Recognit. Lett., vol. 31, no. 8, pp. 651–666, 2010.
[2] A. Likas, N. Vlassis, and J. J. Verbeek, “The global k-means clustering
algorithm,” Pattern Recognit., vol. 36, no. 2, pp. 451–461, 2003.
[3] Y. Cheng, “Mean shift, mode seeking, and clustering,” IEEE
Trans. Pattern Anal. Mach. Intell., vol. 17, no. 8, pp. 790–799,
Aug. 1995.
[4] M. Ester, H.-P. Kriegel, J. Sander, and X. Xu, “A density-based algorithm
for discovering clusters in large spatial databases with noise,” in Proc.
KDD, vol. 96, 1996, pp. 226–231.
[5] U. Von Luxburg, “A tutorial on spectral clustering,” Stat. Comput.,
vol. 17, no. 4, pp. 395–416, 2007.
[6] H. Chang and D.-Y. Yeung, “Robust path-based spectral clustering,”
Pattern Recognit., vol. 41, no. 1, pp. 191–203, 2008.
[7] W. Fan, H. Sallay, and N. Bouguila, “Online learning of hierarchical
Pitman–Yor process mixture of generalized Dirichlet distributions with
feature selection,” IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 9,
pp. 2048–2061, Sep. 2017.
[8] W. Fan, N. Bouguila, J. Du, and X. Liu, “Axially symmetric data clustering through Dirichlet process mixture models of Watson distributions,”
IEEE Trans. Neural Netw. Learn. Syst., vol. 30, no. 6, pp. 1683–1694,
Jun. 2019.
[9] L. Duan, S. Cui, Y. Qiao, and B. Yuan, “Clustering based on supervised learning of exemplar discriminative information,” IEEE Trans.
Syst., Man, Cybern., Syst., to be published.
[10] D. Cheng, Q. Zhu, J. Huang, Q. Wu, and L. Yang, “A novel cluster
validity index based on local cores,” IEEE Trans. Neural Netw. Learn.
Syst., vol. 30, no. 4, pp. 985–999, Apr. 2019.
[11] Y. Chen et al., “Decentralized clustering by finding loose and distributed
density cores,” Inf. Sci., vols. 433–434, pp. 649–660, Apr. 2018.
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
3952
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS: SYSTEMS, VOL. 51, NO. 6, JUNE 2021
[12] A. K. Jain, M. N. Murty, and P. J. Flynn, “Data clustering: A review,”
ACM Comput. Surveys, vol. 31, no. 3, pp. 264–323, 1999.
[13] M. Ankerst, M. M. Breunig, H.-P. Kriegel, and J. Sander, “Optics:
Ordering points to identify the clustering structure,” in Proc. ACM
SIGMOD Rec., vol. 28, 1999, pp. 49–60.
[14] A. Rodriguez and A. Laio, “Clustering by fast search and find of density
peaks,” Science, vol. 344, no. 6191, pp. 1492–1496, 2014.
[15] Y. Chen et al., “Fast density peak clustering for large scale data based
on KNN,” Knowl. Based Syst., vol. 187, Jan. 2020, Art. no. 104824.
[16] D. Cheng, Q. Zhu, J. Huang, Q. Wu, and Y. Lijun, “Clustering with
local density peaks-based minimum spanning tree,” IEEE Trans. Knowl.
Data Eng., to be published.
[17] J. Gan and Y. Tao, “DBSCAN revisited: Mis-claim, un-fixability, and
approximation,” in Proc. ACM SIGMOD Int. Conf. Manag. Data, 2015,
pp. 519–530.
[18] M. Muja and D. G. Lowe, “Scalable nearest neighbor algorithms for
high dimensional data,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 36,
no. 11, pp. 2227–2240, Nov. 2014.
[19] J. L. Bentley, “Multidimensional binary search trees used for associative
searching,” Commun. ACM, vol. 18, no. 9, pp. 509–517, 1975.
[20] A. Beygelzimer, S. Kakade, and J. Langford, “Cover trees for nearest
neighbor,” in Proc. 23rd Int. Conf. Mach. Learn., 2006, pp. 97–104.
[21] A. Gunawan and M. de Berg, “A faster algorithm for DBSCAN,” Ph.D.
dissertation, Dept. Math. Comput. Sci., Univ. Eindhoven, Eindhoven,
The Netherlands, 2013.
[22] V. Chaoji, M. Al Hasan, S. Salem, and M. J. Zaki, “SPARCL: Efficient
and effective shape-based clustering,” in Proc. 8th IEEE Int. Conf. Data
Min., 2008, pp. 93–102.
[23] E. H.-C. Lu, V. S. Tseng, and P. S. Yu, “Mining cluster-based temporal mobile sequential patterns in location-based service environments,”
IEEE Trans. Knowl. Data Eng., vol. 23, no. 6, pp. 914–927, Jun. 2011.
[24] S. K. Pal and P. Mitra, Pattern Recognition Algorithms for Data Mining.
Boston, MA, USA: CRC Press, 2004.
[25] S. Mahran and K. Mahar, “Using grid for accelerating density-based
clustering,” in Proc. 8th IEEE Int. Conf. Comput. Inf. Technol. (CIT),
2008, pp. 35–40.
[26] K. Sonal, G. Poonam, S. Ankit, K. Dhruv, S. Balasubramaniam, and
N. Goyal, “Exact, fast and scalable parallel DBSCAN for commodity
platforms,” in Proc. 18th Int. Conf. Distrib. Comput. Netw., 2017, p. 14.
[27] X. Chen, Y. Min, Y. Zhao, and P. Wang, “GMDBSCAN: Multi-density
DBSCAN cluster based on grid,” in Proc. IEEE Int. Conf. e-Bus. Eng.,
2008, pp. 780–783.
[28] S. T. Mai, I. Assent, and M. Storgaard, “AnyDBC: An efficient anytime
density-based clustering algorithm for very large complex datasets,” in
Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Disc. Data Min., 2016,
pp. 1025–1034.
[29] B. Borah and D. K. Bhattacharyya, “An improved sampling-based
DBSCAN for large spatial databases,” in Proc. Int. Conf. Intell. Sens.
Inf. Process., 2004, pp. 92–96.
[30] C.-F. Tsai and C.-W. Liu, “KIDBSCAN: A new efficient data clustering algorithm,” in Proc. Artif. Intell. Soft Comput. (ICAISC), 2006,
pp. 702–711.
[31] C. Tsai and T. Huang, “QIDBSCAN: A quick density-based clustering technique,” in Proc. Int. Symp. Comput. Consum. Control, 2012,
pp. 638–641.
[32] A. Bryant and K. Cios, “RNN-DBSCAN: A density-based clustering
algorithm using reverse nearest neighbor density estimates,” IEEE Trans.
Knowl. Data Eng., vol. 30, no. 6, pp. 1109–1121, Jun. 2018.
[33] A. Lulli, M. Dell’Amico, P. Michiardi, and L. Ricci, “NG-DBSCAN:
Scalable density-based clustering for arbitrary data,” Proc. VLDB
Endow., vol. 10, no. 3, pp. 157–168, 2016.
[34] F. Gieseke, J. Heinermann, C. E. Oancea, and C. Igel, “Buffer KD trees:
Processing massive nearest neighbor queries on GPUs,” in Proc. ICML,
2014, pp. 172–180.
[35] Y. Chen, L. Zhou, Y. Tang, N. Bouguila, and H. Wang, “Fast neighbor
search by using revised k-d tree,” Inf. Sci., vol. 472, pp. 145–162, 2019.
[36] Y. Chen, L. Zhou, and N. Bouguila, “Semi-convex hull tree: Fast nearest
neighbor queries for large scale data on GPUs,” in Proc. IEEE Int. Conf.
Data Min., 2018, pp. 911–916.
[37] J. Wang et al., “Trinary-projection trees for approximate nearest neighbor search,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 2,
pp. 388–403, Feb. 2014.
[38] M. Muja and D. G. Lowe, “Fast approximate nearest neighbors with
automatic algorithm configuration,” in Proc. Int. Conf. Comput. Vis.
Theory Appl. (VISSAPP), 2009, pp. 331–340.
[39] C. Silpa-Anan and R. Hartley, “Optimised KD-trees for fast image
descriptor matching,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit. (CVPR), 2008, pp. 1–8.
[40] Y. Chen, S. Tang, N. Bouguila, C. Wang, J. Du, and H. L. Li, “A fast
clustering algorithm based on pruning unnecessary distance computations in DBSCAN for high-dimensional data,” Pattern Recognit., vol. 83,
pp. 375–387, Nov. 2018.
[41] A. Karami and R. Johansson, “Choosing DBSCAN parameters automatically using differential evolution,” Int. J. Comput. Appl., vol. 91, no. 7,
pp. 1–11, 2014.
[42] H. Zhou, P. Wang, and H. Li, “Research on adaptive parameters determination in DBSCAN algorithm,” J. Xian Univ. Technol., vol. 9, no. 7,
pp. 1967–1973, 2012.
[43] F. O. Ozkok and M. Celik, “A new approach to determine eps parameter
of DBSCAN algorithm,” Int. J. Intell. Syst. Appl. Eng., vol. 4, no. 5,
pp. 247–251, 2017.
[44] A. Gionis, H. Mannila, and P. Tsaparas, “Clustering aggregation,” in
Proc. Int. Conf. Data Eng. (ICDE), 2005, pp. 341–352.
[45] L. M. Collins and C. W. Dent, “Omega: A general formulation of
the rand index of cluster recovery suitable for non-disjoint solutions,”
Multivariate Behav. Res., vol. 23, no. 2, pp. 231–242, 1988.
[46] A. Strehl and J. Ghosh, “Cluster ensembles: A knowledge reuse framework for combining partitionings,” in Proc. 18th Nat. Conf. Artif. Intell.,
2002, pp. 93–99.
[47] M. A. Patwary, D. Palsetia, A. Agrawal, W.-K. Liao, F. Manne, and
A. Choudhary, “Scalable parallel optics data clustering using graph algorithmic techniques,” in Proc. Int. Conf. High Perform. Comput. Netw.
Storage Anal. (SC), 2013, pp. 1–12.
[48] H. W. Kuhn, “The Hungarian method for the assignment problem,”
Naval Res. Logist. Quart., vol. 2, nos. 1–2, pp. 83–97, 1955.
[49] Y. Chen, S. Tang, S. Pei, C. Wang, J. Du, and N. Xiong, “DHeat: A
density heat-based algorithm for clustering with effective radius,” IEEE
Trans. Syst., Man, Cybern., Syst., vol. 48, no. 4, pp. 649–660, Apr. 2018.
[50] X. Liu, Z. Hu, H. Ling, and Y. Cheung, “MTFH: A matrix trifactorization hashing framework for efficient cross-modal retrieval,”
IEEE Trans. Pattern Anal. Mach. Intell., to be published.
[51] J. Hou, H. Zeng, L. Cai, J. Zhu, J. Chen, and K.-K. Ma,
“Multi-label learning with multi-label smoothing regularization
for vehicle re-identification,” Neurocomputing, vol. 345, pp. 15–22,
Jun. 2019.
[52] J. Zhu et al., “Vehicle re-identification using quadruple directional deep
learning features,” IEEE Trans. Intell. Transp. Syst., to be published.
[53] Y. Zhang, X. Xu, J. Wang, T. Chen, and C. H. Wang, “Crushing
analysis for novel bio-inspired hierarchical circular structures subjected to axial load,” Int. J. Mech. Sci., vol. 140, pp. 407–431,
May 2018.
[54] H. Tian, F. Nan, C.-C. Chang, Y. Huang, J. Lu, and Y. Du,
“Privacy-preserving public auditing for secure data storage in fogto-cloud computing,” J. Netw. Comput. Appl., vol. 127, pp. 59–69,
Feb. 2019.
[55] H. Tian, F. Nan, H. Jiang, C.-C. Chang, J. Ning, and Y. Huang,
“Public auditing for shared cloud data with efficient and secure group
management,” Inf. Sci., vol. 472, pp. 107–125, Jan. 2019.
[56] H. Tian et al., “Public audit for operation behavior logs with error locating in cloud storage,” Soft Comput., vol. 23, no. 11, pp. 3779–3792,
Jun. 2019.
Yewang Chen received the B.S. degree in management of information system from Huaqiao
University, Quanzhou, China, in 2001, and the
Ph.D. degree in software engineering from Fudan
University, Shanghai, China, in 2009.
He is currently an Associate Professor with
the School of Computer Science and Technology,
Huaqiao University, and the Fujian Key Laboratory
of Big Data Intelligence and Security, Huaqiao
University (Xiamen Campus), Xiamen, China. He
is also with the Beijing Key Laboratory of Big Data
Technology for Food Safety, Beijing Technology and Business University,
Beijing, China, and the Provincial Key Laboratory for Computer Information
Processing Technology, Soochow University, Suzhou, China. His current
research interests include machine learning and data mining.
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.
CHEN et al.: KNN-BLOCK DBSCAN: FAST CLUSTERING FOR LARGE-SCALE DATA
Lida Zhou received the B.S. degree in computer
science from the College of Computer Science
and Technology, Central China Normal University,
Wuhan, China, in 2012. He is currently pursuing the Post-Graduation degree with the School
of Computer Science and Technology, Huaqiao
University (Xiamen Campus), Xiamen, China.
His current research interests is machine learning
and pattern recognition.
Songwen Pei (SM’19) received the B.S. degree in
computer science from the National University of
Defence and Technology, Changsha, China, in 2003,
the M.S. degree in computer science from Guizhou
University, Guiyang, China, in 2006, and the Ph.D.
degree in computer science from Fudan University,
Shanghai, China, in 2009.
He is currently an Associate Professor with
the University of Shanghai for Science and
Technology, Shanghai. Since 2011, he has been a
Guest Researcher with the Institute of Computing
Technology, Chinese Academy of Sciences, Beijing, China, a Research
Scientist with the University of California at Irvine, Irvine, CA, USA from
2013 to 2015 and the Queensland University of Technology, Brisbane, QLD,
Australia, in 2017. His research interests include heterogeneous multicore
system, cloud computing, and big data.
Dr. Pei is a board member of CCF-TCCET and CCF-TCARCH. He is a
member of ACM and CCF in China.
Zhiwen Yu (SM’14) received the Ph.D. degree
in computer science from the City University of
Hong Kong, Hong Kong, in 2008.
He is a Professor with the School of Computer
Science and Engineering, South China University
of Technology, Guangzhou, China, from 2015 to
2019. He has been published more than 140 referred
journal papers and international conference papers,
including 40 IEEE T RANSACTIONS papers. His
research areas focus on data mining, machine learning, pattern recognition, and intelligent computing.
Prof. Yu is a Distinguishable Member of China Computer Federation and
the Vice Chair of ACM Guangzhou Chapter. He is a Senior Member of ACM.
Yi Chen received the Ph.D. degree in computer
science from the Beijing Institute of Technology,
Beijing, China, in 2002.
She is currently a Professor of computer science
with Beijing Technology and Business University,
Beijing, where she is the Director of Beijing
Key Laboratory of Big Data Technology for Food
Safety. Her research interests mainly focuses on
information visualization, visual analytics and big
data technology for food quality and safety, including high-dimensional, hierarchical, spatio-temporal,
and graph data visual analytics.
3953
Xin Liu (M’08) received the M.S. degree in applied
mathematics from Hubei University, Wuhan, China,
in 2009, and the Ph.D. degree in computer science
from Hong Kong Baptist University, Hong Kong, in
2013.
He was a Visiting Scholar with the Computer
and Information Sciences Department, Temple
University, Philadelphia, PA, USA, from 2017 to
2018. He is currently an Associate Professor with the
Department of Computer Science and Technology,
Huaqiao University, Quanzhou, China, and also with
the State Key Laboratory of Integrated Services Networks, Xidian University,
Xi’an, China. His present research interests include multimedia analysis,
computer vision, pattern recognition, and machine learning.
Jixiang Du received the B.Sc. and M.Sc. degrees
in vehicle engineering from the Hefei University
of Technology, Hefei, China, in September 1999
and July 2002, respectively, and the Ph.D. degree
in pattern recognition and intelligent system from
the University of Science and Technology of China,
Hefei, in December 2005.
He is currently a Professor with the College
of Computer Science and Technology, Huaqiao
University, Quanzhou, China.
Naixue Xiong (SM’12) received the first Ph.D.
degree in software engineering from Wuhan
University, Wuhan, China, in 2007, and the second
Ph.D. degree in dependable networks from the Japan
Advanced Institute of Science and Technology,
Nomi, Japan, in 2007.
He worked with Colorado Technical University,
Colorado Springs, CO, USA, Wentworth Technology
Institution, Boston, MA, USA, and Georgia State
University, Atlanta, GA, USA, for many years. He
is currently a Professor with Northeastern State
University, Tahlequah, OK, USA. His research interests include cloud computing, security and dependability, parallel and distributed computing, networks,
and optimization theory.
Authorized licensed use limited to: Thangal Kunju Musaliar College of Engineering. Downloaded on March 20,2023 at 14:57:36 UTC from IEEE Xplore. Restrictions apply.