Databases:
Visualization, Data Mining,
New DB Paradigms
Thomas Weik
FH Münster
9. Basic Mining Strategies
9.0 References
9.1 Motivation
9.2 Classification
9.3 Clustering
9.4 Association Rule Discovery
9.5 Challenges of Data Mining
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
2
9.0 References: Books
Books:
 Witten, Eibe, Hall: Data Mining – Practical Machine Learning Tools and
Techniques; 3rd Edition, Morgan Kaufman 2011
 Han et al.: Data Mining – Concepts and Techniques, Morgan Kaufman
2011
 North: Data Mining for the Masses: http://docs.rapidi.com/files/DataMiningForTheMasses.pdf
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
3
9.0 References: Software
Software:
 WEKA: http://www.cs.waikato.ac.nz/ml/weka/
 Rapid Miner: http://www.rapidminer.com
 Manual: http://docs.rapid-i.com/files/rapidminer/rapidminer-5.0-manualenglish_v1.0.pdf
 KNIME (Konstanz Information Miner): http://www.knime.org
 R: CLI for Statistical Computing, Graphics and Data Mining:
http://www.r-project.org/
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
4
9.1 Why Mine Data?
 There is often information “hidden” in the data that is
not readily evident
 Human analysts may take weeks to discover useful information
 Much of the data is
never analyzed at all
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
5
9.1 Orders of Magnitude
 1 PB is enough to store the DNA of every person in the US –
with cloning it twice ...
 AT&T transfers 30 PB of data through its network per day.
 Until July 2012 CERN amassed about 200 PB of data about 800
trillion collisions in search for the Higgs boson.
 1 PB of MP3 encoded music plays continously for about 2000
years.
 IDC: Total amount of global data was expected to grow to 2.7
ZB in 2012, which is an increase of 48% from 2011.
 Whistleblower: NSA's Utah Data Center will have a capacity of
about 5 ZB when completed.
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
6
9.1 Orders of Magnitude
 According to an IDC paper sponsored by EMC Corporation, 161
exabytes of data were created in 2006, "3 million times the
amount of information contained in all the books ever written",
with the number expected to hit 988 exabytes in 2010.
(Wikipedia.org)
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
7
9.1 Why Mine Data? Commercial Viewpoint
 Lots of data is being collected
and warehoused
 Web data, e-commerce
 purchases at department/
grocery stores
 Bank/Credit Card
transactions
 Computers have become cheaper and
more powerful
 Competitive Pressure is Strong
 Provide better, customized services for an edge (e.g. in Customer Relationship
Management)
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
8
9.1 Why Mine Data? Scientific Viewpoint
 Data collected and stored at
enormous speeds (GB/hour)
 remote sensors on a satellite
 telescopes scanning the skies
 microarrays generating gene
expression data
 scientific simulations
generating terabytes of data
 Traditional techniques infeasible for raw data
 Data mining may help scientists
 in classifying and segmenting data
 in Hypothesis Formation
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
9
9.1 What is (not) Data Mining?
What is not Data
Mining?


What is Data Mining?
– Look up phone
number in phone
directory
– Certain names are more
– Query a Web
search engine for
information about
“Amazon”
– Group together similar
documents returned by search
engine according to their context
(e.g. Amazon rainforest,
Amazon.com,)
Thomas Weik: DWH and Data Mining
prevalent in certain US locations
(O’Brien, O’Rurke, O’Reilly… in
Boston area)
WS 2014 / 2015
10
9.1 Origins of Data Mining
 Draws ideas from machine learning/AI, pattern
recognition, statistics, and database systems
 Traditional Techniques
may be unsuitable due to
 Enormity of data
 High dimensionality
of data
 Heterogeneous,
distributed nature
of data
Statistics/
AI
Machine Learning/
Pattern
Recognition
Data Mining
Database
systems
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
11
9.1 What is Data Mining?
 Many Definitions
 Non-trivial extraction of implicit, previously unknown and potentially useful
information from data
 Exploration & analysis, by automatic or
semi-automatic means, of
large quantities of data
in order to discover
meaningful patterns
 Data Mining needs a process!
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
12
9.2 Classification: Definition
 Given a collection of records (training set )
 Each record contains a set of attributes, one of the
attributes is the class.
 Find a model for class attribute as a function of
the values of other attributes.
 Goal: previously unseen records should be
assigned a class as accurately as possible.
 A test set is used to determine the accuracy of the
model. Usually, the given data set is divided into
training and test sets, with training set used to build
the model and test set used to validate it.
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
13
9.2 Illustrating Classification
Task
Tid
Attrib1
Attrib2
Attrib3
Class
1
Yes
Large
125K
No
2
No
Medium
100K
No
3
No
Small
70K
No
4
Yes
Medium
120K
No
5
No
Large
95K
Yes
6
No
Medium
60K
No
7
Yes
Large
220K
No
8
No
Small
85K
Yes
9
No
Medium
75K
No
10
No
Small
90K
Yes
Learning
algorithm
Induction
Learn
Model
Model
10
Training Set
Tid
Attrib1
Attrib2
Attrib3
11
No
Small
55K
?
12
Yes
Medium
80K
?
13
Yes
Large
110K
?
14
No
Small
95K
?
15
No
Large
67K
?
Apply
Model
Class
Deduction
10
Test Set
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
14
9.2 Example of a Decision
Tree
Tid Refund Marital
Status
Taxable
Income Cheat
1
Yes
Single
125K
No
2
No
Married
100K
No
3
No
Single
70K
No
4
Yes
Married
120K
No
5
No
Divorced 95K
Yes
6
No
Married
No
7
Yes
Divorced 220K
No
8
No
Single
85K
Yes
9
No
Married
75K
No
10
No
Single
90K
Yes
60K
Splitting Attributes
Refund
Yes
No
NO
MarSt
Single, Divorced
TaxInc
< 80K
NO
Married
NO
> 80K
YES
10
Training Data
Thomas Weik: DWH and Data Mining
Model: Decision Tree
WS 2014 / 2015
15
9.2 Classification Techniques
 Decision Tree based Methods
 Rule-based Methods
 Memory based reasoning
 Neural Networks
 Naïve Bayes and Bayesian Belief Networks
 Support Vector Machines
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
16
9.2 Ex. for Classification
 Sky Survey Cataloging
 Goal: To predict class (star or galaxy) of sky objects,
especially visually faint ones, based on the telescopic
survey images (from Palomar Observatory).
 3000 images with 23,040 x 23,040 pixels per image.
 Approach:




Segment the image.
Measure image attributes (features) - 40 of them per object.
Model the class based on these features.
Success Story: Could find 16 new high red-shift quasars,
some of the farthest objects that are difficult to find!
Thomas Weik: DWH and Data Mining
From [Fayyad, et.al.] Advances in Knowledge Discovery and Data Mining, 1996
WS 2014 / 2015
17
9.2 Classifying Galaxies
Early
Class:
• Stages of Formation
Attributes:
• Image features,
• Characteristics of light
waves received, etc.
Intermediate
Late
Data Size:
• 72 million stars, 20 million galaxies
• Object Catalog: 9 GB
• Image Database: 150 GB
Courtesy: http://aps.umn.edu
Thomas Weik: Data Mining
WS 2014 / 2015
18
9.2 Predicting
Examples
of
Classification
tumor cells as benign or malignant
 Classifying credit card transactions
as legitimate or fraudulent
 Classifying secondary structures of protein
as alpha-helix, beta-sheet, or random
coil
 Categorizing news stories as finance,
weather, entertainment, sports, etc
 Gene defect analysis
 Customer Rating
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
19
9.2 Constructing Decision
Trees: Another Example
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
20
9.2 Constructing Decision
Trees: Generic Algorithm
Generic recursive algorithm:
 Select an attribute to place at the root node
 Make one branch for every possible value

Thus the example set is split up into subsets

One for every value of the attribute
 Repeat this process recursively for each branch

Use only instances that actually reach this branch
 If all instances at a node have the same class value, then stop developing that
part of the tree
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
21
9.2 Constructing Decision
Trees: Problem
Which attribute
should we split on??
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
22
9.2 Resulting Decision Tree
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
26
9.3 Clustering: Application 1
 Market Segmentation:
 Goal: subdivide a market into distinct subsets of customers
where any subset may conceivably be selected as a market
target to be reached with a distinct marketing mix.
 Approach:
 Collect different attributes of customers based on their
geographical and lifestyle related information.
 Find clusters of similar customers.
 Measure the clustering quality by observing buying patterns of
customers in same cluster vs. those from different clusters.
Thomas Weik: Data Mining
WS 2014 / 2015
29
9.3 Clustering: Application 2
 Document Clustering:
 Goal: To find groups of documents that are similar to each other
based on the important terms appearing in them.
 Approach: To identify frequently occurring terms in each
document. Form a similarity measure based on the frequencies
of different terms. Use it to cluster.
 Gain: Information Retrieval can utilize the clusters to relate a
new document or search term to clustered documents.
Thomas Weik: Data Mining
WS 2014 / 2015
30
9.3 Illustrating Document
Clustering
 Clustering Points: 3204 Articles of Los Angeles Times.
 Similarity Measure: How many words are common in these documents
(after some word filtering).
Category
Total
Articles
Correctly
Placed
555
364
Foreign
341
260
National
273
36
Metro
943
746
Sports
738
573
Entertainment
354
278
Financial
Thomas Weik: Data Mining
WS 2014 / 2015
31
9.4 Association Rule
Discovery: Definition
 Given a set of records each of which contain some number of items from
a given collection;
 Produce dependency rules which will predict occurrence of an item based on
occurrences of other items.
TID
Items
1
2
3
4
5
Bread, Coke, Milk
Beer, Bread
Beer, Coke, Diaper, Milk
Beer, Bread, Diaper, Milk
Coke, Diaper, Milk
Thomas Weik: Data Mining
Rules Discovered:
{Milk} --> {Coke}
{Diaper, Milk} --> {Beer}
WS 2014 / 2015
32
9.4 Association Rule Discovery:
An Application
 Marketing and Sales Promotion:
 Let the rule discovered be
{Bagels, … } --> {Potato Chips}
 Potato Chips as consequent => Can be used to determine
what should be done to boost its sales.
 Bagels in the antecedent => Can be used to see which
products would be affected if the store discontinues selling
bagels.
 Bagels in antecedent and Potato chips in consequent =>
Can be used to see what products should be sold with
Bagels to promote sale of Potato chips!
Thomas Weik: Data Mining
WS 2014 / 2015
33
9.4 Association Rule Discovery:
An Application II
 Supermarket shelf management.
 Goal: To identify items that are bought together by sufficiently
many customers.
 Approach: Process the point-of-sale data collected with
barcode scanners to find dependencies among items.
 A classic rule - If a customer buys diapers and milk, then he is very likely to buy
beer.
 So, don’t be surprised if you find six-packs stacked next to diapers!
Thomas Weik: Data Mining
WS 2014 / 2015
34
9.5 Challenges of Data Mining
 Scalability
 Dimensionality
 Complex and Heterogeneous Data
 Data Quality
 Data Ownership and Distribution
 Privacy Preservation
 Streaming Data
Thomas Weik: DWH and Data Mining
WS 2014 / 2015
39