SAS Halifax Regional User Group
April 29, 2011
Decision Trees Using SAS
Sylvain Tremblay
SAS Canada – Education
Copyright © 2010 SAS Institute Inc. All rights reserved.
They come in all shapes and forms
2
Copyright © 2010, SAS Institute Inc. All rights reserved.
They come in all shapes and forms
3
Copyright © 2010, SAS Institute Inc. All rights reserved.
Agenda
 What is a decision tree?
 Decision trees using SAS Enterprise Miner
 Decision trees using JMP
 Conclusion / Questions
4
Copyright © 2010, SAS Institute Inc. All rights reserved.
A Decision Tree is
 A predictive model
 A representation of the relationship between a target
(dependant variable) and a set of inputs (independant
variables)
 A supervised learning method
 A recursive partitionning algorithm
 Also know by the name of algorithms that were
commercialized:
CART (Classification And Regression Tree)
CHAID (CHi-squared Automatic Interaction Detector)
5
Copyright © 2010, SAS Institute Inc. All rights reserved.
Simple Prediction Illustration
Training Data
1.0
0.9
Predict P(Y=1| X1, X2)
0.8
0.7
0.6
x2
0.5
0.4
0.3
0.2
0.1
0.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
x1
6
Copyright © 2010, SAS Institute Inc. All rights reserved.
...
Decision Tree Split Search
1.0
<0.63
x2
0.9
0.8
≥0.63
0.7
0.63
0.6
x2
0.5
0.4
Create a partition rule
from the best partition
across all inputs.
0.3
0.2
0.1
0.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
x1
7
Copyright © 2010, SAS Institute Inc. All rights reserved.
...
Decision Tree Prediction Rules
root node
<0.63
x2
1.0
0.9
≥0.63
<0.52
4
1
2
0.8
0.7
interior node
x1
3
≥0.52
<0.51
x1
0.6
≥0.51
x2
0.5
0.4
0.3
0.2
70%
40%
leaf node
3
60%
1
2
55%
4
0.1
0.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
x1
8
Copyright © 2010, SAS Institute Inc. All rights reserved.
...
Decision Tree Prediction Rules
Predict:
<0.63
x2
Decision =
Estimate = 0.70
1.0
0.9
≥0.63
0.8
0.7
<0.52
x1
≥0.52
<0.51
x1
0.6
≥0.51
x2
0.5
0.4
0.3
0.2
70%
40%
55%
60%
0.1
0.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
x1
9
Copyright © 2010, SAS Institute Inc. All rights reserved.
...
Agenda
 What is a decision tree?
 Decision trees using SAS Enterprise Miner
 Decision trees using JMP
 Conclusion / Questions
10
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees with SAS Enterprise Miner
Trees can be created manually or automatically
Copyright © 2010, SAS Institute Inc. All rights reserved.
11
Decision Trees with SAS Enterprise Miner
manually
12
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees with SAS Enterprise Miner
manually
13
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees with SAS Enterprise Miner
manually
14
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees - Assessment
Pruning
15
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees - Pruning
manually
16
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees with SAS Enterprise Miner
automatically
EM Tree Parameters
17
Copyright © 2010, SAS Institute Inc. All rights reserved.
Agenda
 What is a decision tree?
 Decision trees using SAS Enterprise Miner
 Decision trees using JMP
 Conclusion / Questions
18
Copyright © 2010, SAS Institute Inc. All rights reserved.
Decision Trees with SAS JMP
19
Copyright © 2010, SAS Institute Inc. All rights reserved.
Agenda
 What is a decision tree?
 Decision trees using SAS Enterprise Miner
 Decision trees using JMP
 Conclusion / Questions
20
Copyright © 2010, SAS Institute Inc. All rights reserved.
Questions?
THANK YOU!
Sylvain.Tremblay@sas.com
Copyright © 2010 SAS Institute Inc. All rights reserved.