Visualizing Big Data In Drupal
Using Data Visualizations to Drive Knowledge Discovery
David Hazel - dhazel@uw.edu
Assistant Director - Center for Web and Data Science
Brad Porter - bjporter@uw.edu
Undergrad. Research Assist. - Center for Web and Data Science
PNWDS - 2012
University of Washington, Tacoma
10/21/2012
UW Tacoma / WebDataScience.org
Why We Use Drupal?
•
•
•
•
•
Consistent Look and Feel
Content Management
Modules
Secure and Solid Code Base
Great Community
10/21/2012
UW Tacoma / WebDataScience.org
But...I want
•
•
•
•
•
HTML5
Slick UI
Fast Query Management
Explore Really Big Data
SCALE
10/21/2012
UW Tacoma / WebDataScience.org
So...We turn To
10/21/2012
UW Tacoma / WebDataScience.org
django
10/21/2012
UW Tacoma / WebDataScience.org
RoR
10/21/2012
UW Tacoma / WebDataScience.org
10/21/2012
UW Tacoma / WebDataScience.org
10/21/2012
UW Tacoma / WebDataScience.org
The Problem
•
•
Tradeoffs
Drupal vs Custom
10/21/2012
UW Tacoma / WebDataScience.org
World of BIG DATA
•
•
•
•
Really large data sets
Complex queries
Inconsistent load times
Visualize Data in meaningful ways
PNWDS - 2012
10/21/2012
UW Tacoma / WebDataScience.org
Some Real World Examples
• DDRS
• CHF Risk of Readmission
• Adoptopedia
PNWDS - 2012
10/21/2012
UW Tacoma / WebDataScience.org
Dietary Data Recording System
PNWDS - 2012
10/21/2012
UW Tacoma / WebDataScience.org
CHF Risk of Readmission
•
•
•
•
PNWDS - 2012
10/21/2012
No reimbursement for readmission
within 30 days
Examine Factors that Contribute to
Readmission
Better Care
Reduced Costs
UW Tacoma / WebDataScience.org
Adoptopedia
PNWDS - 2012
10/21/2012
UW Tacoma / WebDataScience.org
What do all these have in common
PNWDS - 2012
10/21/2012
UW Tacoma / WebDataScience.org
10/21/2012
UW Tacoma / WebDataScience.org
10/21/2012
UW Tacoma / WebDataScience.org
OSPI Data Dashboard Project
OSPI
State Needs Projects
10/21/2012
Center for Web and Data Science
Center for Strong Communities & Schools
UW Tacoma / WebDataScience.org
Lots of Data
• 10 Sites across WA State
• 100's of Trainings
PNWDS - 2012
10/21/2012
UW Tacoma / WebDataScience.org
Data + Challenges
•
Homemade survey system
o
o
MySQL database (LAMP)
How to merge data


10/21/2012
Seemingly missing data
Hardcoded modifications
UW Tacoma / WebDataScience.org
Data + Challenges Cont.
•
Drupal surveys
o
o
Separate PostgreSQL database
Webform module

10/21/2012
Lacking queue system
UW Tacoma / WebDataScience.org
Data Flow
Standardized questions integrated into
Webform
Import into external datamart
Drive visualizations
10/21/2012
UW Tacoma / WebDataScience.org
Visualizing the Data
•
Custom modules
• pull data from External Datamart
• Load JSON into Browser
• Visualize the Data
10/21/2012
UW Tacoma / WebDataScience.org
wds_viz.module
PNWDS - 2012
10/21/2012
UW Tacoma / WebDataScience.org
Why D3.js?
• Transitioning From Flash
o
o
Printer Compatibility
No plugins necessary
• Image Embedding
o
o
10/21/2012
Lower Bandwidth
Dynamic Graphs
UW Tacoma / WebDataScience.org
Pie Chart
10/21/2012
UW Tacoma / WebDataScience.org
Choropleth
10/21/2012
UW Tacoma / WebDataScience.org
Bar Graph
10/21/2012
UW Tacoma / WebDataScience.org
Parallel Coordinates
10/21/2012
UW Tacoma / WebDataScience.org
Implementation / Integration
10/21/2012
UW Tacoma / WebDataScience.org
Implementation / Integration
•
•
10/21/2012
Under 40 lines of code
Completely customizable
UW Tacoma / WebDataScience.org
Implementation / Integration
10/21/2012
UW Tacoma / WebDataScience.org
Implementation / Integration
10/21/2012
UW Tacoma / WebDataScience.org
Heat map Constructions
•
•
•
•
•
•
Map is Drawn
Plot svg circles
Calculate x,y = long/lat
Set color palette based on count
Set r = 0
Time series change r
10/21/2012
UW Tacoma / WebDataScience.org
CODE DEMO
•
•
Heatmap transition
function changeHeat()
10/21/2012
UW Tacoma / WebDataScience.org
Center for Web and Data Science
VISION: TO BE THE ROSETTA STONE FOR
SOCIETAL BIG DATA
Q&A
10/21/2012
UW Tacoma / WebDataScience.org