— Chapter 2 —
April 14, 2020 Data Mining: Concepts and Techniques 1


General data characteristics

Basic data description and exploration

Measuring data similarity
April 14, 2020 Data Mining: Concepts and Techniques 2


Collection of data objects and their attributes
Attributes


An attribute is a property or characteristic of an object

Examples: eye color of a person, temperature, etc.

Attribute is also known as variable, field, characteristic, or feature
Objects
A collection of attributes describe an object

Object is also known as record, point, case, sample, entity, or instance
April 14, 2020 Data Mining: Concepts and Techniques
Tid Refund Marital
Status
Taxable
Income Cheat
1 Yes
2 No
3 No
4 Yes
5 No
6 No
7 Yes
8 No
9 No
10
10 No
Single 125K
Married 100K
Single 70K
Married 120K
Divorced 95K
Married 60K
Divorced 220K
Single 85K
Married 75K
Single 90K
No
No
No
No
Yes
No
No
Yes
No
Yes
3





Dimensionality

Curse of dimensionality
Sparsity

Only presence counts
Resolution

Patterns depend on the scale
Similarity

Distance measure
April 14, 2020 Data Mining: Concepts and Techniques 4


Attribute values are numbers or symbols assigned to an attribute

Distinction between attributes and attribute values

Same attribute can be mapped to different attribute values

Example: height can be measured in feet or meters

Different attributes can be mapped to the same set of values

Example: Attribute values for ID and age are integers

But properties of attribute values can be different

ID has no limit but age has a maximum and minimum value
April 14, 2020 Data Mining: Concepts and Techniques 5






Nominal

E.g., profession, ID numbers, eye color, zip codes
Ordinal

E.g., rankings (e.g., army, professions), grades, height in {tall, medium, short}
Binary

E.g., medical test (positive vs. negative)
Interval

E.g., calendar dates, body temperatures
Ratio

E.g., temperature in Kelvin, length, time, counts
April 14, 2020 Data Mining: Concepts and Techniques 6


The type of an attribute depends on which of the following properties it possesses:

Distinctness: = 



Order:
Addition:
Multiplication:
< >
+ -
* /




Nominal attribute: distinctness
Ordinal attribute: distinctness & order
Interval attribute: distinctness, order & addition
Ratio attribute: all 4 properties
April 14, 2020 Data Mining: Concepts and Techniques 7

Attribute
Type
Nominal
Ordinal
Description Examples Operations
The values of a nominal attribute are just different names, i.e., nominal attributes provide only enough information to distinguish one object from another. (=,

)
The values of an ordinal attribute provide enough information to order objects. (<, >) zip codes, employee
ID numbers, eye color, sex: { male, female } hardness of minerals,
{ good, better, best }, grades, street numbers mode, entropy, contingency correlation,

2 test median, percentiles, rank correlation, run tests, sign tests
Interval
Ratio
April 14, 2020
For interval attributes, the differences between values are meaningful, i.e., a unit of measurement exists.
(+, - )
For ratio variables, both differences and ratios are meaningful. (*, /) calendar dates, temperature in Celsius or Fahrenheit mean, standard deviation, Pearson's correlation, t and F tests temperature in Kelvin, monetary quantities, counts, age, mass, length, electrical current geometric mean, harmonic mean, percent variation
Data Mining: Concepts and Techniques 8



Discrete Attribute

Has only a finite or countably infinite set of values


E.g., zip codes, profession, or the set of words in a collection of documents
Sometimes, represented as integer variables

Note: Binary attributes are a special case of discrete attributes
Continuous Attribute


Has real numbers as attribute values
Examples: temperature, height, or weight


Practically, real values can only be measured and represented using a finite number of digits
Continuous attributes are typically represented as floating-point variables
April 14, 2020 Data Mining: Concepts and Techniques 9




Record

Data Matrix


Document Data
Transaction Data
Graph


World Wide Web
Molecular Structures
Ordered

Spatial Data



Temporal Data
Sequential Data
Genetic Sequence Data
April 14, 2020 Data Mining: Concepts and Techniques 10

April 14, 2020
Important Characteristics of Structured Data

Dimensionality

Curse of Dimensionality

Sparsity

Only presence counts

Resolution

Patterns depend on the scale
Data Mining: Concepts and Techniques 11


Data that consists of a collection of records, each of which consists of a fixed set of attributes
Tid Refund Marital
Status
Taxable
Income Cheat
1 Yes
2 No
3 No
4 Yes
5 No
6 No
7 Yes
8 No
9 No
10
10 No
Single 125K
Married 100K
Single 70K
Married 120K
Divorced 95K
Married 60K
Divorced 220K
Single 85K
Married 75K
Single 90K
No
No
No
No
Yes
No
No
Yes
No
Yes
April 14, 2020 Data Mining: Concepts and Techniques 12


If data objects have the same fixed set of numeric attributes, then the data objects can be thought of as points in a multi-dimensional space, where each dimension represents a distinct attribute

Such data set can be represented by an m by n matrix, where there are m rows, one for each object, and n columns, one for each attribute
April 14, 2020 Data Mining: Concepts and Techniques 13


Each document becomes a `term' vector,
 each term is a component (attribute) of the vector,
 the value of each component is the number of times the corresponding term occurs in the document.
April 14, 2020
Document 1
Document 2
Document 3
3
0
0
0
7
1
5
0
0
0
2
0
2
1
1
6
0
2
0
0
2
2
3
0
0
0
3
2
0
0
Data Mining: Concepts and Techniques 14


A special type of record data, where
 each record (transaction) involves a set of items.

For example, consider a grocery store. The set of products purchased by a customer during one shopping trip constitute a transaction, while the individual products that were purchased are the items.
TID Items
1
4
5
2
3
Bread, Coke, Milk
Beer, Bread
Beer, Coke, Diaper, Milk
Beer, Bread, Diaper, Milk
Coke, Diaper, Milk
April 14, 2020 Data Mining: Concepts and Techniques 15


Examples: Generic graph and HTML Links
5
2
2
5
1
<a href="papers/papers.html#bbbb">
Data Mining </a>
<li>
<a href="papers/papers.html#aaaa">
Graph Partitioning </a>
<li>
<a href="papers/papers.html#aaaa">
Parallel Solution of Sparse Linear System of Equations </a>
<li>
<a href="papers/papers.html#ffff">
N-Body Computation and Dense Linear System Solvers
April 14, 2020 Data Mining: Concepts and Techniques 16


Benzene Molecule: C
6
H
6
April 14, 2020 Data Mining: Concepts and Techniques 17


Sequences of transactions
Items/Events
April 14, 2020
An element of the sequence
Data Mining: Concepts and Techniques 18


Genomic sequence data
GGTTCCGCCTTCAGCCCCGCGCC
CGCAGGGCCCGCCCCGCGCCGTC
GAGAAGGGCCCGCCTGGCGGGCG
GGGGGAGGCGGGGCCGCCCGAGC
CCAACCGAGTCCGACCAGGTGCC
CCCTCTGCTCGGCCTAGACCTGA
GCTCATTAGGCGGCAGCGGACAG
GCCAAGTAGAACACGCGAAGCGC
TGGGCTGCCTGCTGCGACCAGGG
April 14, 2020 Data Mining: Concepts and Techniques 19


Spatio-Temporal Data
Average Monthly
Temperature of land and ocean
April 14, 2020 Data Mining: Concepts and Techniques 20


General data characteristics

Basic data description and exploration

Measuring data similarity
April 14, 2020 Data Mining: Concepts and Techniques 21





Motivation

To better understand the data: central tendency, variation and spread
Data dispersion characteristics
 median, max, min, quantiles, outliers, variance, etc.
Numerical dimensions correspond to sorted intervals

Data dispersion: analyzed with multiple granularities of precision

Boxplot or quantile analysis on sorted intervals
Dispersion analysis on computed measures

Folding measures into numerical dimensions

Boxplot or quantile analysis on the transformed cube
April 14, 2020 Data Mining: Concepts and Techniques 22




Mean (algebraic measure) (sample vs. population):


Weighted arithmetic mean:
Trimmed mean: chopping extreme values
Median: A holistic measure x
 x

1 i n 

1 i n 

1 w i x i w i n i n 

1 x i
 
 x
N

Middle value if odd number of values, or average of the middle two values otherwise

Estimated by interpolation (for grouped data ):
Mode median

L
1

(
N

Value that occurs most frequently in the data
/ 2

(
 freq ) l
) width freq median


Unimodal, bimodal, trimodal
Empirical formula: mean
 mode

3

( mean
 median )
April 14, 2020 Data Mining: Concepts and Techniques 23


Median, mean and mode of symmetric, positively and negatively skewed data symmetric
April 14, 2020 positively skewed negatively skewed
Data Mining: Concepts and Techniques 24



Quartiles, outliers and boxplots




Quartiles : Q
1
(25 th percentile), Q
3
(75 th percentile)
Inter-quartile range : IQR = Q
3
– Q
1
Five number summary : min, Q
1
, M, Q
3
, max
Boxplot : ends of the box are the quartiles, median is marked, whiskers, and plot outlier individually

Outlier : usually, a value higher/lower than 1.5 x IQR
Variance and standard deviation ( sample: s, population: σ)

Variance : (algebraic, scalable computation) s
2  n
1

1 i n 

1
( x i
 x )
2  n
1

1
[ i n

1 x i
2
1 n
( n    i

1 x i
)
2
]


2

1
N i n 

1
( x i
 
)
2

1
N i n 

1 x i
2  
2
Standard deviation s (or σ) is the square root of variance s 2 ( or σ 2)
April 14, 2020 Data Mining: Concepts and Techniques 25



Five-number summary of a distribution:
Minimum, Q1, M, Q3, Maximum
Boxplot




Data is represented with a box
The ends of the box are at the first and third quartiles, i.e., the height of the box is IQR
The median is marked by a line within the box
Whiskers: two lines outside the box extend to
Minimum and Maximum
April 14, 2020 Data Mining: Concepts and Techniques 26


Graph displays of basic statistical class descriptions

Frequency histograms


A univariate graphical method
Consists of a set of rectangles that reflect the counts or frequencies of the classes present in the given data
April 14, 2020 Data Mining: Concepts and Techniques 27



The two histograms shown in the left may have the same boxplot representation

The same values for: min, Q1, median, Q3, max
But they have rather different data distributions
April 14, 2020 Data Mining: Concepts and Techniques 28



Displays all of the data (allowing the user to assess both the overall behavior and unusual occurrences)
Plots quantile information

For a data x i data sorted in increasing order, indicates that approximately 100 below or equal to the value x i f i f i
% of the data are
April 14, 2020 Data Mining: Concepts and Techniques 29



Graphs the quantiles of one univariate distribution against the corresponding quantiles of another
Allows the user to view whether there is a shift in going from one distribution to another
April 14, 2020 Data Mining: Concepts and Techniques 30



Provides a first look at bivariate data to see clusters of points, outliers, etc
Each pair of values is treated as a pair of coordinates and plotted as points in the plane
April 14, 2020 Data Mining: Concepts and Techniques 31



Adds a smooth curve to a scatter plot in order to provide better perception of the pattern of dependence
Loess curve is fitted by setting two parameters: a smoothing parameter, and the degree of the polynomials that are fitted by the regression
April 14, 2020 Data Mining: Concepts and Techniques 32

April 14, 2020


The left half fragment is positively correlated
The right half is negative correlated
Data Mining: Concepts and Techniques 33

April 14, 2020 Data Mining: Concepts and Techniques 34



Why data visualization?


Gain insight into an information space by mapping data onto graphical primitives
Provide qualitative overview of large data sets


Search for patterns, trends, structure, irregularities, relationships among data
Help find interesting regions and suitable parameters for further quantitative analysis

Provide a visual proof of computer representations derived
Typical visualization methods:



Geometric techniques
Icon-based techniques
Hierarchical techniques
April 14, 2020 Data Mining: Concepts and Techniques 35



Visualization of geometric transformations and projections of the data
Methods

Landscapes

Projection pursuit technique

Finding meaningful projections of multidimensional data



Scatterplot matrices
Prosection views
Hyperslice

Parallel coordinates
April 14, 2020 Data Mining: Concepts and Techniques 36

April 14, 2020
Matrix of scatterplots (x-y-diagrams) of the k-dim. data
Data Mining: Concepts and Techniques 37

news articles visualized as a landscape


Visualization of the data as perspective landscape
The data needs to be transformed into a (possibly artificial) 2D spatial representation which preserves the characteristics of the data
April 14, 2020 Data Mining: Concepts and Techniques 38



 n equidistant axes which are parallel to one of the screen axes and correspond to the attributes
The axes are scaled to the [minimum, maximum]: range of the corresponding attribute
Every data item corresponds to a polygonal line which intersects each of the axes at the point which corresponds to the value for the attribute
• • •
April 14, 2020
Attr. 1 Attr. 2 Attr. 3
Data Mining: Concepts and Techniques
Attr. k
39

April 14, 2020 Data Mining: Concepts and Techniques 40



Visualization of the data values as features of icons
Methods:

Chernoff Faces




Stick Figures
Shape Coding:
Color Icons:
TileBars: The use of small icons representing the relevance feature vectors in document retrieval
April 14, 2020 Data Mining: Concepts and Techniques 41



A way to display variables on a two-dimensional surface, e.g., let x be eyebrow slant, y be eye size, z be nose length, etc.
The figure shows faces produced using 10 characteristics--head eccentricity, eye size, eye spacing, eye eccentricity, pupil size, eyebrow slant, nose size, mouth shape, mouth size, and mouth opening): Each assigned one of 10 possible values, generated using
Mathematica (S. Dickson)


REFERENCE: Gonick, L. and Smith, W. The
Cartoon Guide to Statistics.
New York:
Harper Perennial, p. 212, 1993
Weisstein, Eric W. "Chernoff Face." From
MathWorld --A Wolfram Web Resource. mathworld.wolfram.com/ChernoffFace.html
April 14, 2020 Data Mining: Concepts and Techniques 42



Visualization of the data using a hierarchical partitioning into subspaces.
Methods





Dimensional Stacking
Worlds-within-Worlds
Treemap
Cone Trees
InfoCube
April 14, 2020 Data Mining: Concepts and Techniques 43



Screen-filling method which uses a hierarchical partitioning of the screen into regions depending on the attribute values
The x- and y-dimension of the screen are partitioned alternately according to the attribute values (classes)
MSR Netscan Image
April 14, 2020 Data Mining: Concepts and Techniques 44

April 14, 2020 Data Mining: Concepts and Techniques 45


General data characteristics

Basic data description and exploration

Measuring data similarity (Sec. 7.2)
April 14, 2020 Data Mining: Concepts and Techniques 46




Similarity

Numerical measure of how alike two data objects are


Value is higher when objects are more alike
Often falls in the range [0,1]
Dissimilarity (i.e., distance)

Numerical measure of how different are two data objects


Lower when objects are more alike
Minimum dissimilarity is often 0

Upper limit varies
Proximity refers to a similarity or dissimilarity
April 14, 2020 Data Mining: Concepts and Techniques 47


Data matrix

 n data points with p dimensions
Two modes







 x x
11
...
i1
...
x n1
...
...
...
...
...
x
1f
...
x if
...
x nf
...
...
...
...
...
x x
1p
...
ip
...
x np









Dissimilarity matrix
 n data points, but registers only the distance


A triangular matrix
Single mode
April 14, 2020







0 d(2,1) d(3,1 d (
: n , 1
)
)
0 d ( 3 , 2 )
: d ( n , 2 )
0
:
...
...
0







Data Mining: Concepts and Techniques 48

3
2 p1
1
0
0 1 p2
2 p3
3 4 p4
5 6 point p1 p2 p3 p4
2
3 x
0
5
Data Matrix
0
1 y
2
1 p1 p2 p3 p4 p1
0
2.828
3.162
5.099
p2
2.828
0
1.414
3.162
p3
3.162
1.414
0
2 p4
5.099
3.162
2
0
Distance Matrix (i.e., Dissimilarity Matrix) for Euclidean Distance
April 14, 2020 Data Mining: Concepts and Techniques 49


Minkowski distance : A popular distance measure

 d ( i , j )
 q
(| x i
1
 x j
1
| q 
| x i
2
 x j
2
| q 
...

| x i p
 x j p
| q
) where i = ( x i1
, x i2
, …, x ip
) and j = ( x j1
, x j2
, …, x jp
) are two p -dimensional data objects, and q is the order
Properties

 d(i, j) > 0 if i ≠ j, and d(i, i) = 0 (Positive definiteness) d(i, j) = d(j, i) (Symmetry)
 d(i, j)  d(i, k) + d(k, j) (Triangle Inequality)
A distance that satisfies these properties is a metric
April 14, 2020 Data Mining: Concepts and Techniques 50


 q

= 1: Manhattan (city block, L
1 norm) distance
E.g., the Hamming distance: the number of bits that are different between two binary vectors d ( i , j )

| x i
1
 x j
1
|

| x i
2
 x j
2
|

...

| x i p
 x j p
| q = 2: (L
2 norm) Euclidean distance d ( i , j )

(| x i
1
 x j
1
|
2 
| x i
2
 x j
2
|
2 
...

| x i p
 x j p
|
2
)


 q

  . “supremum” (L max norm, L
 norm) distance.
This is the maximum difference between any component of the vectors
Do not confuse q with n numbers of dimensions.
, i.e., all these distances are defined for all
Also, one can use weighted distance, parametric Pearson product moment correlation, or other dissimilarity measures
April 14, 2020 Data Mining: Concepts and Techniques 51

point p1 p2 p3 p4
April 14, 2020
2
3
5 x
0
0
1
1 y
2
L2 p1 p2 p3 p4
L1 p1 p2 p3 p4
L
 p1 p2 p3 p4 p1
0
4
4
6 p1
0
2.828
3.162
5.099
p1
0
2
3
5 p2
4
0
2
4 p2
2.828
0
1.414
3.162
p2
2
0
1
3
Distance Matrix p3
4
2
0
2 p3
3.162
1.414
0
2 p3
3
1
0
2 p4
6
4
2
0 p4
5.099
3.162
2
0 p4
5
3
2
0
Data Mining: Concepts and Techniques 52






A contingency table for binary data
Distance measure for symmetric binary variables:
Object j
1 0 sum
Object i
1
0 a c b d a
 b c
 d d ( i , sum j )
 a
 c b
 d p a
 b b

 c c
 d
Distance measure for asymmetric binary variables:
Jaccard coefficient ( similarity measure for asymmetric binary variables): d ( i , j )
 sim
Jaccard
( i , j ) a
 b
 a
 b c

 a b
 c c
A binary variable is symmetric if both of its states are equally valuable and carry the same weight.
April 14, 2020 Data Mining: Concepts and Techniques 53


Example
Name Gender Fever Cough Test-1 Test-2 Test-3 Test-4
Jack M
Mary F
Jim M
Y N
Y N
Y P
P
P
N
N
N
N
N
P
N
N
N
N


 gender is a symmetric attribute the remaining attributes are asymmetric binary let the values Y and P be set to 1, and the value N be set to 0 d ( jack , mary )

2
0


0
1

1

0 .
33 d d
(
( jack jim ,
, jim mary
)
)


1
1
1 1

1
1


2
1


1

2


0 .
67
0 .
75
April 14, 2020 Data Mining: Concepts and Techniques 54




A generalization of the binary variable in that it can take more than 2 states, e.g., red, yellow, blue, green
Method 1: Simple matching
 m : # of matches, p : total # of variables d ( i , j )
 p
 p m
Method 2: Use a large number of binary variables
 creating a new binary variable for each of the M nominal states
April 14, 2020 Data Mining: Concepts and Techniques 55




An ordinal variable can be discrete or continuous
Order is important, e.g., rank
Can be treated like interval-scaled

 replace x if by their rank r
 if
{ 1 ,..., M } f map the range of each variable onto [0, 1] by replacing i -th object in the f -th variable by
 z if
 r if
M f

1

1 compute the dissimilarity using methods for intervalscaled variables
April 14, 2020 Data Mining: Concepts and Techniques 56



Ratio-scaled variable: a positive measurement on a nonlinear scale, approximately at exponential scale, such as Ae Bt or Ae -Bt
Methods:
 treat them like interval-scaled variables— not a good choice! (why?—the scale can be distorted)

 apply logarithmic transformation y if
= log(x if
) treat them as continuous ordinal data treat their rank as interval-scaled
April 14, 2020 Data Mining: Concepts and Techniques 57



A database may contain all the six types of variables
 symmetric binary, asymmetric binary, nominal, ordinal, interval and ratio
One may use a weighted formula to combine their effects d ( i , j )

 p f

 f p
1


( ij
1
 f ij
(
) d ij
( f ) f )


 f is binary or nominal: d ij
(f) = 0 if x if
= x jf
, or d ij
(f) = 1 otherwise f is interval-based: use the normalized distance f is ordinal or ratio-scaled


Compute ranks r if
Treat z if and as interval-scaled z if
 r
M if f

1

1
April 14, 2020 Data Mining: Concepts and Techniques 58





Vector objects: keywords in documents, gene features in micro-arrays, …
Applications: information retrieval, biologic taxonomy, ...
Cosine measure: If cos( d
1
, d
2 d
) = (
1 and d
1
 d d
2
2 are two vectors, then
) /|| d
1
|| || d
2
|| , where  indicates vector dot product, || d ||: the length of vector d
Example: d
1
= 3 2 0 5 0 0 0 2 0 0 d d
2
1

= 1 0 0 0 0 0 0 1 0 2 d
2
= 3*1+2*0+0*0+5*0+0*0+0*0+0*0+2*1+0*0+0*2 = 5
|| d
1
||= (3*3+2*2+0*0+5*5+0*0+0*0+0*0+2*2+0*0+0*0) 0.5
=(42) 0.5
= 6.481
|| d
2
|| = (1*1+0*0+0*0+0*0+0*0+0*0+0*0+1*1+0*0+2*2) 0.5
=(6) 0.5
= 2.245
cos( d
1
, d
2
) = .3150
April 14, 2020 Data Mining: Concepts and Techniques 59


Correlation coefficient (also called Pearson’s product moment coefficient ) r p , q


( p
( n
 p )( q

1 )
 p
 q
 q )


(
( n pq )

1 )

 n p
 q p q

 q means of p and q, σ p and σ q are the respective standard deviation of p and q, and Σ(pq) is the sum of the pq cross-product.
If r p,q
> 0, p and q are positively correlated (p’s values increase as q’s). The higher, the stronger correlation.
r p,q
= 0: independent; r pq
< 0: negatively correlated
April 14, 2020 Data Mining: Concepts and Techniques 60



Correlation measures the linear relationship between objects
To compute correlation, we standardize data objects, p and q, and then take their dot product p k
 
( p k
 mean ( p )) / std ( p ) q k
 
( q k
 mean ( q )) / std ( q ) correlatio n ( p , q )
 p
  q

April 14, 2020 Data Mining: Concepts and Techniques 61

Scatter plots showing the similarity from
–1 to 1.
April 14, 2020 Data Mining: Concepts and Techniques 62