Using Tableau with SAS
A Case Study with Recommendations
2009 Tableau Customer Conference
July 20-23 – Seattle, WA
All rights reserved. © 2009 Tableau Software Inc.
Michael W Cristiani
Market Intelligence Group, LLC
Should you be in this room?
+ Questions on the table:
+ Can output from SAS procedures (what’s that
mean?) be used with Tableau?
+ Should SAS output be used with Tableau?
+ Can SAS output be used in concert with Tableau,
SQL, web apps, without breaking the bank?
+ Can you contribute by crowdsourcing these questions
with
+ Answers?
+ More questions?
All rights reserved. © 2009 Tableau Software Inc.
Wisdom from the field
“Remember, it’s not about the tools, it’s about the intelligence.”
- Ted Cuzzillo
Ted Cuzzillo is “a freelance writer who writes lead-generating journalism, PR and marketing for clients
within the greater business intelligence community.” He also gifts the BI community with
www.datadoodle.com. Ted graces friends and family with fabulous cuisine and delightful stories of Sicily.
Ted is here at the 2009 Tableau Customer Conference.
All rights reserved. © 2009 Tableau Software Inc.
Market Intelligence Group, LLC
+
Established 1997
+
Database development
+
Database management
+
Decision support and comprehensive business analytics
+
Tableau application development
+ Hosted database marketing and business operations
applications running on the DataWell® platform
All rights reserved. © 2009 Tableau Software Inc.
Market Intelligence Group, LLC
Michael W Cristiani
+
Principal
+
Urban and Regional Planning
+
Telecommunications demand forecasting and analysis
+
Business analytics and decision support systems development
All rights reserved. © 2009 Tableau Software Inc.
The Business Problem -- 2002
All rights reserved. © 2009 Tableau Software Inc.
The Business Problem
All rights reserved. © 2009 Tableau Software Inc.
Evaluation framework
Evaluators
+ Self, Bosses, Peers, Staff (reports)
+ Self
+ 3 bosses
+ 3 groups of 7 peers
+ 3 groups of 7 staff
+ Feedback requested for each evaluated behavior
+ Actual (% of time behavior is observed)
+ Preferred (% of time behavior is preferred)
+ Feedback Report
+ Report Gap, average Actual, Preferred, and (Preferred –
Actual) scores for each behavior within each evaluator group
+
All rights reserved. © 2009 Tableau Software Inc.
First Gen Solution
+
+
+
+
Gather business rules
Process data sets using PC-SAS (Base)
+ Summarize data into multiple “tables” to support multiple report
formats with unique rules for calculating averages
+ Apply complex business and privacy rules
Develop Excel reports application
+ Graphic data presentation per client’s requirements
+ Lots of VBA
Deliver employee level results in Excel application
+ Lots of formulas
+ Huge (typically 100MB/file)
+ Monthly files
+ Point and click hard copy reports
All rights reserved. © 2009 Tableau Software Inc.
Current process flow
Evaluation
Process
Survey Administration – consider
business rules
Employee Assessment – consider
business rules
Raw
Data Set
(weekly,
on demand)
All rights reserved. © 2009 Tableau Software Inc.
SAS
Processing
(as new data
Arrives)
Cubed
Data Set
(cumulative)
PEAK™
Report
Generator
(Excel
deliverable)
Some business rules
Protect privacy of evaluators
+ Minimum of 3 paired ratings per question within each rater
group
+ Collapse remaining matched pairs into generic rater groups
+ Apply these rules for questions and core competency groups
+ Allow self and superior evaluations to flow through without
matching ratings
+
All rights reserved. © 2009 Tableau Software Inc.
Some raw and summarized data
All rights reserved. © 2009 Tableau Software Inc.
SAS easily summarizes raw data while
applying complex business rules
qindex
n
participant_id
lvl2
lvl3
q
a
p
F110.009BossGroup1
1
F110.009
Boss
Group1
1
93
97
F110.009BossGroup2
1
F110.009
Boss
Group2
1
100
100
0 F110
F110.009PeerGroup2
4
F110.009
Peer
Group2
1
89.5
95
5.5 F110
F110.009PeerGroup3
4
F110.009
Peer
Group3
1
97.5
97.5
0 F110
F110.009_SelfSelf
1
F110.009
Self
Self
1
95
90
-5 F110
F110.009StaffGroup1
5
F110.009
Staff
Group1
1
85.8
84.4
-1.4 F110
F113.058PeerPeerGroup
4
F113.058
Peer
Peer Group
1
97.5
87.5
-10 F113
F113.058_SelfSelf
1
F113.058
Self
Self
1
50
100
50 F113
F113.058StaffStaffGroup
4
F113.058
Staff
Staff Group
1
93.75
95
1.25 F113
F113.060_SelfSelf
1
F113.060
Self
Self
1
85
95
10 F113
F113.060StaffStaffGroup
3
F113.060
Staff
Staff Group
1
F115.013BossGroup1
1
F115.013
Boss
Group1
1
70
F115.013PeerGroup1
3
F115.013
Peer
Group1
1
90.33333333
F115.013PeerGroup2
5
F115.013
Peer
Group2
1
71
95
F115.013_SelfSelf
1
F115.013
Self
Self
1
85
91
6 F115
F115.013StaffGroup1
5
F115.013
Staff
Group1
1
79.2
94
14.8 F115
F115.013StaffGroup2
4
F115.013
Staff
Group2
1
96.25
96.5
0.25 F115
F122.033BossGroup1
1
F122.033
Boss
Group1
1
90
90
0 F122
F122.033PeerGroup1
4
F122.033
Peer
Group1
1
84.75
89.75
5 F122
F122.033_SelfSelf
1
F122.033
Self
Self
1
100
100
0 F122
F122.034PeerGroup2
3
F122.034
Peer
Group2
1
90
95
5 F122
F122.034PeerGroup3
3
F122.034
Peer
Group3
1
90 91.66666667 1.666666667 F122
F122.034_SelfSelf
1
F122.034
Self
Self
1
85
90
5 F122
F122.035_SelfSelf
1
F122.035
Self
Self
1
75
100
25 F122
All rights reserved. © 2009 Tableau Software Inc.
g fcnumber
4 F110
88.33333333 96.66666667 8.333333333 F113
90
20 F115
93 2.666666667 F115
24 F115
Deliverable description for First Gen Solution
Two MS-Excel huge workbooks (100MB and 30MB)
+ Reports [Focus of this presentation]
+ Verbatim Comments
+ Each tool features
+ Extensive VBA
+ In-workbook database (fact and lookup tables)
+ Point and click functionality
+ Report previews
+ Flexible printing
+ Batch
+ Individual employee
+
All rights reserved. © 2009 Tableau Software Inc.
Example from First Gen Solution
All rights reserved. © 2009 Tableau Software Inc.
Example from First Gen Solution
All rights reserved. © 2009 Tableau Software Inc.
Successes of First Gen Solution
+
Significant improvements in productivity
+ Speedy data processing using PC-SAS
+ Batch report preparation takes minutes of SME’s time
instead of days
+ HR organizational development specialists have time to
counsel / write courses / teach
Why change this blissful situation?
All rights reserved. © 2009 Tableau Software Inc.
Challenges with First Gen Solution
+
+
+
+
+
+
+
Frequent business rules changes require extensive SAS code changes
Limited access to PC-SAS on occasion
Data still not in a proper database
Data error tracking is cumbersome, usually occurring after employee
has one-time hard copy report
Report format changes are a real challenge in Excel (100s K
formulas)
Report distribution is offline / incurs significant mail costs
Administrative tasks handled manually by staff and a third party vendor
What to do?
All rights reserved. © 2009 Tableau Software Inc.
Successes / Challenges of First Gen Solution
NO
YES
NO
YES
YES
YES
NO
All rights reserved. © 2009 Tableau Software Inc.
Next Gen Solution Requirements
+
+
+
+
+
+
+
Automate administrative tasks online
Secure self-serve report generation and delivery online
Store data in proper database
Improve data error detection and remediation
Eventual online data processing
Evaluate SAS processing environment for speed, cost, and
reliability
Include restricted views of data for research and tracking
All rights reserved. © 2009 Tableau Software Inc.
Next Gen Solution: Two Cycles
+
First Cycle:
+ Leverage investment in PC-SAS data processing code
+ Process data in PC-SAS
+ Store summarized data tables and lookup tables in a proper
PostgreSQL database
+ Connect in Tableau to summarized data tables
+ Design reports for Tableau
+ Employee take away reports/views
+ Administrative and research views
+ Develop Tableau reports workbook
+ Apply business and privacy rules
+ Allow data exploration
+ PDF export
+ User filters
+ Publish Tableau reports workbook to Tableau Server
+ Secure login for up to 300 participants at a time (rolling)
+ Data exploration
+ PDF export
All rights reserved. © 2009 Tableau Software Inc.
Next Gen Solution: Two Cycles
+
Second Cycle:
+ Start with First Cycle Tableau reports workbook
+ Test two approaches to data processing and make a decision
+ Connect in Tableau to summarized data tables
+ Incorporate SAS data processing logic into PostgreSQL
code
+ Process data online
+ Requires minimal re-tooling of visualizations in Tableau
+ Connect in Tableau to unsummarized (raw) data tables
+ Incorporate SAS data processing logic into Tableau
calculated fields or custom SQL
+ Requires some further re-tooling of Tableau report
workbook
+ Publish Tableau reports workbook to Tableau Server
+ Secure login for up to 300 participants at a time (rolling)
+ Data exploration
+ PDF export
All rights reserved. © 2009 Tableau Software Inc.
Evaluation of Next Gen Concepts
Tableau demo comes here.
All rights reserved. © 2009 Tableau Software Inc.
Successes/Challenges of Second Gen Solution
YES
1
YES
YES
ONLINE
YES – but customizable
YES
YES
1 DataWell®
All rights reserved. © 2009 Tableau Software Inc.
Why use summarized SAS output with
Tableau?
SAS is a ubiquitous data management and analysis tool set:
+ Installed base of many millions
+ Base SAS handles large data sets with ease
+ Stable platform that runs in many operating environments
+ Proven analytical and statistical techniques for data exploration and
discovery
+ Widely used in academia for research and teaching
Training
+ Very large body of official SAS published knowledge about SAS and
its usage
+ Endless resources online
Accessibility
+ Easy to export the results of SAS data management and analytics to
flat files
+ SQL interface is available
All rights reserved. © 2009 Tableau Software Inc.
Why not use summarized SAS output with
Tableau?
+Learning cycle:
+ Becoming very proficient with SAS can take years of experience
+ SAS specialists tend to concentrate on its specialized use within
their specific discipline
+ Learning materials can be very expensive over a short time
+Expense:
+ SAS licensing can be quirkily expensive
+ Special modules are also expensive, and are often needed to
accomplish specific tasks
+ SQL access is not free
+ Server based applications are well outside the budget of most
potential users and consultants
All rights reserved. © 2009 Tableau Software Inc.
Why use a SQL database with Tableau
Tableau easily connects to many flavors of SQL:
+ Many SQL implementations are open source
+ PostgreSQL is MIG flavor of choice
Online database management is a snap since
+ Hosted database management solutions permit
+ Automated inbound data feeds from any third party data
provider (real time, online, or batch)
+ Seamless interaction with PC-SAS if necessary
+ Simple/complex querying of unified summarized database
Security and accessibility
1
+ A PHP / PostgreSQL front-end to this application will provide
assistance in managing administrative functions Tableau is not
designed to address.
1 DataWell®
All rights reserved. © 2009 Tableau Software Inc.
Why not use a SQL database with Tableau
+Learning and evaluation:
+ Becoming very proficient with the SAS/SQL/DataWell interaction
may take some time
+ The SAS processing necessary to prepare summarized data for
either the First Gen or Second Gen solutions is complex.
+ Will PostgreSQL choke on it?
+ If no SAS processing of raw data is done, is Tableau up to the
complex processing steps needed to summarize the data from a
PostgreSQL database?
+ For security reasons, clients may not want raw data exposed in
hosted end-user facing environment
All rights reserved. © 2009 Tableau Software Inc.
Evaluation of Next Gen Concepts
From a business perspective, what makes sense is:
+ Leverage SAS data processing investment
+ Continue to process raw data feeds in PC-SAS
+ Produce SAS output files in near real-time,
+ stored on DB server,
+ for access by DataWell® using PostgreSQL and PHP
scripting
+ Build unified results database from stored SAS data sets or
output files
+ provides for multiple levels of aggregation in one Tableau
worksheet vs multiple tables (SAS/Excel)
All rights reserved. © 2009 Tableau Software Inc.
Evaluation of Next Gen Concepts
From a client perspective, what makes sense is:
+ In Tableau, connect to unified results database
+ Employee focused report views
+ Basics research views
+ Train client to go further with Tableau’s cycle of analysis
+ Employee filter
+ Publish workbook to Tableau Server
+ In DataWell®,
+ Integrate Tableau views
+ Automatic view filtering on log-in
+ Add administrative functions outside Tableau’s feature set.
All rights reserved. © 2009 Tableau Software Inc.
Questions and Answers?
All rights reserved. © 2009 Tableau Software Inc.
Resources
http://www.migsite.com
119 East Court St
Cincinnati, OH 45202
Robert Migliara robert@migsite.com
Michael W Cristiani mike@migsite.com
All rights reserved. © 2009 Tableau Software Inc.