Introduction to Data Mining
Why Data Mining?


The Explosive Growth of Data: from terabytes to petabytes
o Data collection and data availability = Automated data collection tools, database
systems, Web, computerized society
o Major sources of abundant data = Business: Web, e-commerce, transactions, stocks.
Science: Remote sensing, bioinformatics, scientific simulation. Society and everyone:
news, digital cameras, YouTube
We are drowning in data, but starving for knowledge!
What Is Data Mining?




Data mining (knowledge discovery from data) = Extraction of interesting (non-trivial, implicit,
previously unknown and potentially useful) patterns or knowledge from huge amount of data.
Alternative names = Knowledge discovery (mining) in databases (KDD), knowledge extraction,
data/pattern analysis, data archeology, data dredging, information harvesting, business
intelligence, etc.
Is everything “data mining”? = Simple search and query processing. (Deductive) expert systems.
KDD Process = Data Cleaning (from Databases) -> Data Integration (from multiple sources) ->
Data Warehouse -> Task-Relevant Data (Data cube construction) -> Data selection for data
mining -> Data Mining -> Presentation of the mining results-> Pattern Evaluation -> Knowledge
Data Mining in Bussiness Intelligence
KDD Process: A View from ML and Statistics
This is a view from typical machine learning and statistics communities
Multi-Dimensional View of Data Mining




Data to be mined = Database data (extended-relational, object-oriented, heterogeneous), data
warehouse, transactional data, stream, spatiotemporal, time-series, sequence, text and web,
multimedia, graphs & social and information networks
Knowledge to be mined (or: Data mining functions) =
o Characterization, discrimination, association, classification, clustering, trend/deviation,
outlier analysis, …
o Descriptive vs. predictive data mining
o Multiple/integrated functions and mining at multiple levels
Techniques utilized = Data-intensive, data warehouse (OLAP), machine learning, statistics,
pattern recognition, visualization, high-performance, etc.
Applications adapted = Retail, telecommunication, banking, fraud analysis, bio-data mining,
stock market analysis, text mining, Web mining, etc.
Kinds of Data Can Be Mined


Database-oriented data sets and applications =
o Relational database, data warehouse, transactional database
o Object-relational databases, Heterogeneous databases and legacy databases
Advanced data sets and advanced applications = Data streams and sensor data, Time-series
data, temporal data, sequence data (incl. bio-sequences), Structure data, graphs, social networks
and information networks, Spatial data and spatiotemporal data, Multimedia database, Text
databases, The World-Wide Web.
Kinds of Patterns Can Be Mined







Generalization
Pattern Discovery
Classification
Cluster Analysis
Outlier Analysis
Time and Ordering: Sequential Pattern, Trend and Evolution Analysis
Structure and Network Analysis
Kinds of Technologies Are Used








Apps
Algorithm
Database Technology
High-Performance Computing
Visualization
Statistics
Pattern Recognition
Manchine Learning
Applications of Data Mining








Web page analysis: classification, clustering, ranking
Collaborative analysis & recommender systems
Basket data analysis to targeted marketing
Biological and medical data analysis q Data mining and software engineering
Data mining and text analysis
Data mining and social and information network analysis
Built-in (invisible data mining) functions in Google, MS, Yahoo!, Linked, Facebook, …
Major dedicated data mining systems/tools q SAS, MS SQL-Server Analysis Manager, Oracle Data
Mining Tools)
Major Issues in Data Mining





Mining Methodology
o Mining various and new kinds of knowledge
o Mining knowledge in multi-dimensional space
o Data mining: An interdisciplinary effort
o Boosting the power of discovery in a networked environment
o Handling noise, uncertainty, and incompleteness of data
o Pattern evaluation and pattern- or constraint-guided mining
User Interaction
o Interactive mining
o Incorporation of background knowledge
o Presentation and visualization of data mining results
Efficiency and Scalability
o Efficiency and scalability of data mining algorithms
o Parallel, distributed, stream, and incremental mining methods
Diversity of data types
o Handling complex types of data
o Mining dynamic, networked, and global data repositories
Data mining and society
o Social impacts of data mining
o Privacy-preserving data mining
o Invisible data mining
Summary





Data mining: Discovering interesting patterns and knowledge from massive amount of data
A natural evolution of science and information technology, in great demand, with wide
applications
A KDD process includes data cleaning, data integration, data selection, transformation, data
mining, pattern evaluation, and knowledge presentation
Mining can be performed in a variety of data
Data mining functionalities: characterization, discrimination, association, classification,
clustering, trend and outlier analysis, etc.
Data
Type of Data Sets




Record Data
o Relational records
o Data matrix, e.g., numerical matrix, crosstabs
o Transaction data
o Document data: Term-frequency vector (matrix) of text documents
Graphs and Networks
o Transportation network
o World Wide Web
o Molecular Structures
o Social or information networks
Ordered Data
o Video data: sequence of images
o Temporal data: time-series
o Sequential Data: transaction sequences
o Genetic sequence data
Spatial, image and multimedia Data
Important Characteristics of Structured Data




Dimensionality
o Curse of dimensionality
Sparsity
o Only presence counts
Resolution
o Patterns depend on the scale
Distribution
o Centrality and dispersion
Data Objects
A data object represents an entity. Examples: sales database: customers, store items, sales. medical
database: patients, treatments. university database: students, professors, courses. Also called samples ,
examples, instances, data points, objects, tuples. Data objects are described by attributes

Database rows → data objects; columns → attributes
Attributes


Attribute (or dimensions, features, variables)
o A data field, representing a characteristic or feature of a data object.
o E.g., customer _ID, name, address
Types:
o Nominal (e.g., red, blue)
o Binary (e.g., {true, false})
o Ordinal (e.g., {freshman, sophomore, junior, senior})
o Numeric: quantitative
o Interval-scaled: 100C is interval scales.
o Ratio-scaled: 100K is ratio scaled since it is twice as high as 50K
Attribute Types



Nominal: categories, states, or “names of things”
o Hair_color = {auburn, black, blond, brown, grey, red, white}
o marital status, occupation, ID numbers, zip codes
Binary
o Nominal attribute with only 2 states (0 and 1)
o Symmetric binary: both outcomes equally important q e.g., gender
o Asymmetric binary: outcomes not equally important. q e.g., medical test (positive vs.
negative) Convention: assign 1 to most important outcome (e.g., HIV positive)
Ordinal
o Values have a meaningful order (ranking) but magnitude between successive values is
not known
o Size = {small, medium, large}, grades, army rankings
Histogram Analysis


Histogram: Graph display of tabulated frequencies, shown as bars
Differences between histograms and bar charts
o Histograms are used to show distributions of variables while bar charts are used to
compare variables
o Histograms plot binned quantitative data while bar charts plot categorical data
o Bars can be reordered in bar charts but not in histograms
o Differs from a bar chart in that it is the area of the bar that denotes the value, not the
height as in bar charts, a crucial distinction when the categories are not of uniform width
Data Visualization


Why data visualization?
o Gain insight into an information space by mapping data onto graphical primitives
o Provide qualitative overview of large data sets
o Search for patterns, trends, structure, irregularities, relationships among data
o Help find interesting regions and suitable parameters for further quantitative analysis
o Provide a visual proof of computer representations derived
Categorization of visualization methods:
o Pixel-oriented visualization techniques
o Geometric projection visualization techniques
o Icon-based visualization techniques
o Hierarchical visualization techniques
o Visualizing complex data and relations
Similarity, Dissimilarity, and Proximity



Similarity measure or similarity function
o A real-valued function that quantifies the similarity between two objects
o Measure how two data objects are alike: The higher value, the more alike
o Often falls in the range [0,1]: 0: no similarity; 1: completely similar
Dissimilarity (or distance) measure
o Numerical measure of how different two data objects are
o In some sense, the inverse of similarity: The lower, the more alike
o Minimum dissimilarity is often 0 (i.e., completely similar)
o Range [0, 1] or [0, ∞) , depending on the definition
Proximity usually refers to either similarity or dissimilarity
Summary





Data attribute types: nominal, binary, ordinal, interval-scaled, ratio-scaled
Many types of data sets, e.g., numerical, text, graph, Web, image.
Gain insight into the data by:
o Basic statistical data description: central tendency, dispersion, graphical displays
o Data visualization: map data onto graphical primitives
o Measure data similarity
Above steps are the beginning of data preprocessing
Many methods have been developed but still an active area of research
Data Preprocessing
What is Data Preprocessing?




Data cleaning
o Handle missing data, smooth noisy data, identify or remove outliers, and resolve
inconsistencies
Data integration
o Integration of multiple databases, data cubes, or files
Data reduction
o Dimensionality reduction
o Numerosity reduction
o Data compression
Data transformation and data discretization
o Normalization
o Concept hierarchy generation
Why Preprocess the Data?

Measures for data quality: A multidimensional view
o Accuracy: correct or wrong, accurate or not
o Completeness: not recorded, unavailable, …
o Consistency: some modified but some not, dangling, …
o Timeliness: timely update?
o Believability: how trustable the data are correct?
o Interpretability: how easily the data can be understood?
Data Cleaning

Data in the Real World Is Dirty: Lots of potentially incorrect data, e.g., instrument faulty, human
or computer error, and transmission error
o Incomplete: lacking attribute values, lacking certain attributes of interest, or containing
only aggregate data q e.g., Occupation = “ ” (missing data)
o Noisy: containing noise, errors, or outliers q e.g., Salary = “−10” (an error)
o Inconsistent: containing discrepancies in codes or names, e.g., q Age = “42”, Birthday =
“03/07/2010” q Was rating “1, 2, 3”, now rating “A, B, C” q discrepancy between
duplicate records
o Intentional (e.g., disguised missing data) q Jan. 1 as everyone’s birthday?
Incomplete (Missing) Data



Data is not always available
o E.g., many tuples have no recorded value for several attributes, such as customer income
in sales data
Missing data may be due to
o Equipment malfunction
o Inconsistent with other recorded data and thus deleted
o Data were not entered due to misunderstanding
o Certain data may not be considered important at the time of entry
o Did not register history or changes of the data
Missing data may need to be inferred
How to Handle Missing Data?



Ignore the tuple: usually done when class label is missing (when doing classification)—not
effective when the % of missing values per attribute varies considerably
Fill in the missing value manually: tedious + infeasible?
Fill in it automatically with
o a global constant : e.g., “unknown”, a new class?!
o the attribute mean
o the attribute mean for all samples belonging to the same class: smarter
o the most probable value: inference-based such as Bayesian formula or decision tree
Noisy Data



Noise: random error or variance in a measured variable
Incorrect attribute values may be due to
o Faulty data collection instruments
o Data entry problems
o Data transmission problems
o Technology limitation
o Inconsistency in naming convention
Other data problems
o Duplicate records
o Incomplete data
o Inconsistent data
How to Handle Noisy Data?




Binning
o First sort data and partition into (equal-frequency) bins
o Then one can smooth by bin means, smooth by bin median, smooth by bin boundaries,
etc.
Regression
o Smooth by fitting the data into regression functions
Clustering
o Detect and remove outliers
Semi-supervised: Combined computer and human inspection
o Detect suspicious values and check by human (e.g., deal with possible outliers)
Data Integration




Data integration
o Combining data from multiple sources into a coherent store
Schema integration: e.g., A.cust-id = B.cust-#
o Integrate metadata from different sources
Entity identification:
o Identify real world entities from multiple data sources, e.g., Bill Clinton = William Clinton
Detecting and resolving data value conflicts
o For the same real world entity, attribute values from different sources are different
o Possible reasons: different representations, different scales, e.g., metric vs. British units
Data Reduction



Data reduction:
o Obtain a reduced representation of the data set q much smaller in volume but yet
produces almost the same analytical results
Why data reduction?—A database/data warehouse may store terabytes of data
o Complex analysis may take a very long time to run on the complete data set
Methods for data reduction (also data size reduction or numerosity reduction)
o Regression and Log-Linear Models
o Histograms, clustering, sampling
o Data cube aggregation
o Data compression



Regression analysis: A collective name for techniques for the modeling and analysis of numerical
data consisting of values of a dependent variable (also called response variable or measurement)
and of one or more independent variables (also known as explanatory variables or predictors)
Linear regression: Y = w X + b
o Data modeled to fit a straight line
o Often uses the least-square method to fit the line
o Two regression coefficients, w and b, specify the line and are to be estimated by using
the data at hand
o Using the least squares criterion to the known values of Y1, Y2, …, X1, X2, ….
Nonlinear regression:
o Data are modeled by a function which is a nonlinear combination of the model
parameters and depends on one or more independent variables
o The data are fitted by a method of successive approximations
Data Transformation





A function that maps the entire set of values of a given attribute to a new set of replacement
values s.t. each old value can be identified with one of the new values
Methods
o Smoothing: Remove noise from data
o Attribute/feature construction New attributes constructed from the given ones
Aggregation: Summarization, data cube construction
Normalization: Scaled to fall within a smaller, specified range min-max normalization z-score
normalization normalization by decimal scaling
Discretization: Concept hierarchy climbing
Dimensionality Reduction



Curse of dimensionality
o When dimensionality increases, data becomes increasingly sparse
o Density and distance between points, which is critical to clustering, outlier analysis,
becomes less meaningful
o The possible combinations of subspaces will grow exponentially
Dimensionality reduction
o Reducing the number of random variables under consideration, via obtaining a set of
principal variables
Advantages of dimensionality reduction
o Avoid the curse of dimensionality
o Help eliminate irrelevant features and reduce noise
o Reduce time and space required in data mining
o Allow easier visualization
Summary





Data quality: accuracy, completeness, consistency, timeliness, believability, interpretability
Data cleaning: e.g. missing/noisy values, outliers
Data integration from multiple sources:
o Entity identification problem; Remove redundancies; Detect inconsistencies
Data reduction, data transformation and data discretization
o Numerosity reduction; Data compression
o Normalization; Concept hierarchy generation
Dimensionality reduction