1/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Data Exploration, Analysis, and Representation:
Integration through Visual Analytics
Remco Chang, PhD
UNC Charlotte
Charlotte Visualization Center
2/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Problem Statement
• The growth of data is
exceeding our ability to
analyze them.
• The amount of digital
information generated is
growing exponentially…
– 2002: 22 EB (exabytes, 1018)
– 2006: 161 EB
– 2010: 988 EB (almost 1 ZB)
1: Data courtesy of Dr. Joseph Kielman, DHS
2: Image courtesy of Dr. Maria Zemankova, NSF
3/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Problem Statement
• The data is often complex,
ambiguous, noisy. Analysis of
which requires human
understanding.
– About 2 GB of data is being
produced per person per year
– 95% of the Digital Universe’s
information is unstructured
• There isn’t enough man-power to
analyze all the data, and the
problem is getting worse!
• Solution: help the user
– Find patterns
– Filter out noise
– Focus on the important stuff
1: Data courtesy of Dr. Joseph Kielman, DHS
2: Image courtesy of Dr. Maria Zemankova, NSF
4/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Example: What Does (Wire) Fraud Look Like?
• Financial Institutions like Bank of America have legal responsibilities
to report all suspicious wire transaction activities (money laundering,
supporting terrorist activities, etc)
• Data size: approximately 200,000 transactions per day (73 million
transactions per year)
• Problems:
– Automated approach can only detect known patterns
– Bad guys are smart: patterns are constantly changing
– Data is messy: lack of international standards resulting in ambiguous
data
• Current methods:
– 10 analysts monitoring and analyzing all transactions
– Using SQL queries and spreadsheet-like interfaces
– Limited time scale (2 weeks)
5/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
WireVis: Financial Fraud Analysis
• In collaboration with Bank of America
– Develop a visual analytical tool (WireVis)
– Visualizes 7 million transactions over 1 year
– Currently beta-deployed at WireWatch
• Uses interaction to coordinate four perspectives:
–
–
–
–
Keywords to Accounts
Keywords to Keywords
Keywords/Accounts over Time
Account similarities (search by example)
R. Chang et al., Scalable and interactive visual analysis of financial wire transactions for fraud detection. Information Visualization,2008.
R. Chang et al., Wirevis: Visualization of categorical, time-varying data from financial transactions. IEEE VAST, 2007.
6/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
WireVis: A Visual Analytics Approach
Heatmap View
(Accounts to Keywords
Relationship)
Search by Example
(Find Similar
Accounts)
Keyword Network
(Keyword
Relationships)
Strings and Beads
(Relationships over Time)
7/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Introducing Visual Analytics
• Visual analytics is the science of analytical reasoning
facilitated by interactive visual interfaces [Thomas &
Cook 2005]
• Since 2004, the field has
grown significantly. Aside
from tens to hundreds of
domestic and international
partners, it now has a IEEE
conference (IEEE VAST), an
NSF program (FODAVA), and a
forthcoming IEEE Transactions
journal.
Graphics &
Visualization
Interaction
&
Reasoning
Computing
8/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Visual Analytics, A Graphics Perspective
9/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Visual Analytics, A Graphics Perspective
• Master’s Thesis -– Simulating dynamic motion
based on kinematic motion
• Jiggling of muscles
– Skinnable Mesh
• Volumetric deformation
– Compared 3 types of massspring systems
• Regular (unconstrained) massspring
• Reduced degree of freedom
• Approximate finite element
method with implicit
integration
• Is this applicable beyond
graphics and simulation?
R. Chang, Simulation Techniques for Deformable Animated Characters. Master’s Thesis, Brown University, 2000.
10/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
From Graphics to Visual Analytics:
An Example in Urban Simplification
• (left) Original model, 285k polygons
• (center) e=100, 129k polygons (45% of original)
• (right) e=1000, 53k polygons (18% of original)
R. Chang et al., Legible simplification of textured urban models. IEEE Computer Graphics and Applications, 28(3):27–36, 2008.
R. Chang et al., Hierarchical simplification of city models to maintain urban legibility. ACM SIGGRAPH 2006 Sketches, page 130 , 2006.
11/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Urban Simplification
• Which polygons to remove?
Original Model
Our Textured Model
Simplified Model
using QSlim
Our Model
Visually different, but quantitatively similar!
Intro
12/33
VA
Graphics
Computing
Interaction
Urban Simplification
• The goal is to retain the “Image of the City”
• Based on Kevin Lynch’s concept of “Urban
Legibility” [1960]
–
–
–
–
–
Paths: highways, railroads
Edges: shorelines, boundaries
Districts: industrial, historic
Nodes: Time Square in NYC
Landmarks: Empire State building
Wrap-up
13/33
Intro
VA
Graphics
Computing
Interaction
Algorithm for Preserving Legibility
• Paths & Edges
– Hierarchical (singlelink) clustering
• Nodes
– Merging clusters
– Polyline
simplification using
convex hulls
• Landmarks
– Pixel-based skyline
preservation
• That’s pretty good,
right?
Wrap-up
14/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Urban Visualization with Semantics
• How do people think about a city?
– Describe New York…
• Response 1: “New York is large, compact, and crowded.”
• Response 2: “The area where I live has a strong mix of
ethnicities.”
Geometric, Information, View Dependent (Cognitive)
15/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Urban Visualization
• Geometric
– Create a hierarchy of shapes based on the rules of legibility
• Information
– Matrix view and Parallel Coordinates show relationships between clusters and
dimensions
• View Dependence (Cognitive)
– Uses interaction to alter the position of focus
R. Chang et al., Legible cities: Focus-dependent multi-resolution visualization of urban relationships. IEEE Transactions on Visualization
and Graphics , 13(6):1169–1175, 2007
16/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Probe-based Interface
• Using Probes allows for comparing multiple
regions-of-interest simultaneously
R. Chang et al., Multi-focused geospatial analysis using probes. Visualization and Computer Graphics, IEEE Transactions on, 14(6):1165–
1172, Nov.-Dec. 2008.
17/33
Intro
VA
Graphics
Computing
Urban Visualization
Graphics + Visual Analytics
• Applying graphics approaches
– Data transformation (clustering,
LOD, simplification)
– Screen-based metrics
– Hardware acceleration
• Applying visual analytics
principles
– Multi-dimensional data
representation
– Interactive exploration
– Broader applicability
Interaction
Wrap-up
18/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Extending Visual Analytics Principles
Who
• Global Terrorism
Database
– With University of
Maryland
– Application of the
investigative 5 W’s
Where
What
Evidence
Box
Original
Data
• Bridge Maintenance
– With US DOT
– Exploring subjective
inspection reports
• Biomechanical Motion
– With U. Minnesota
and Brown
– Interactive motion
comparison methods
R. Chang et al., Investigative Visual Analysis of Global Terrorism, Journal of Computer Graphics Forum, 2008.
When
19/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Extending Visual Analytics Principles
• Global Terrorism
Database
– With University of
Maryland
– Application of the
investigative 5 W’s
• Bridge Maintenance
– With US DOT
– Exploring subjective
inspection reports
• Biomechanical Motion
– With U. Minnesota
and Brown
– Interactive motion
comparison methods
R. Chang et al., An Interactive Visual Analytics System for Bridge Management, Journal of Computer Graphics Forum, 2010. To Appear.
20/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Extending Visual Analytics Principles
• Global Terrorism
Database
– With University of
Maryland
– Application of the
investigative 5 W’s
• Bridge Maintenance
– With US DOT
– Exploring subjective
inspection reports
• Biomechanical Motion
– With U. Minnesota
and Brown
– Interactive motion
comparison methods
R. Chang et al., Interactive Coordinated Multiple-View Visualization of Biomechanical Motion Data, IEEE Vis (TVCG) 2009.
21/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Human + Computer
A Mixed-Initiative Perspective
• Our approach is great and successful! But it’s mostly user-driven…
• Human vs. Artificial Intelligence
Garry Kasparov vs. Deep Blue (1997)
– Computer takes a “brute force” approach without analysis
– “As for how many moves ahead a grandmaster sees,” Kasparov concludes: “Just
one, the best one”
• Artificial Intelligence vs. Augmented Intelligence
Hydra vs. Cyborgs (2005)
– Grandmaster + 1 chess program > Hydra (equiv. of Deep Blue)
– Amateur + 3 chess programs > Grandmaster + 1 chess program1
• How to systematically repeat the success?
– Unsupervised machine learning + User
– User’s interactions with the computer
1. http://www.collisiondetection.net/mt/archives/2010/02/why_cyborgs_are.php
Computer
Process
(Translate)
Human
Intro
22/33
VA
Graphics
Computing
Interaction
Wrap-up
Human + Computer:
Dimension Reduction – Lost in Translation
• Dimension reduction using principle component analysis (PCA)
• Quick Refresher of PCA
– Find most dominant eigenvectors as principle components
– Data points are re-projected into the new coordinate system
• For reducing dimensionality
• For finding clusters
height
• For many (especially novices), PCA is easy to understand mathematically,
but difficult to understand “semantically”.
0.5*GPA + 0.2*age + 0.3*height = ?
age
23/33
Intro
VA
Graphics
Computing
Interaction
Human + Computer:
Exploring Dimension Reduction: iPCA
R. Chang et al., iPCA: An Interactive System for PCA-based Visual Analytics. Computer Graphics Forum (Eurovis), 2009.
Wrap-up
24/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Human + Computer:
Comparing iPCA to SAS/INSIGHT
• Results
– Users seem to understand the
intuition behind PCA better
– A bit more accurate
– Not faster
– People don’t “give up”
• Overall preference
– Using letter grades (A through
F) with “A” representing
excellent and F a failing grade.
• Problem is worse with non-linear
dimension reduction
• A lot more work needs to be
done…
25/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Human + Computer:
User Interactions
Computer
Process
(Translate)
Human
• Capture a user’s interactions
in a visual analytics system
• Translate the interactions
into something that would
affect the computation in a
meaningful way
• Challenge:
• Can we capture and extract a user’s
reasoning and intent through capturing a
user’s interactions?
26/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
What is in a User’s Interactions?
• Goal: determine if a user’s reasoning and intent
are reflected in a user’s interactions.
Grad
Students
(Coders)
Compare!
(manually)
Analysts
Strategies
Methods
Findings
Guesses of
Analysts’
thinking
Logged
(semantic)
Interactions
WireVis
Interaction-Log Vis
27/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
What’s in a User’s Interactions
• From this experiment, we find that interactions contains at least:
– 60% of the (high level) strategies
– 60% of the (mid level) methods
– 79% of the (low level) findings
R. Chang et al., Recovering Reasoning Process From User Interactions. IEEE Computer Graphics and Applications, 2009.
R. Chang et al., Evaluating the Relationship Between User Interaction and Financial Visual Analysis. IEEE Symposium on VAST, 2009.
Intro
28/33
VA
Graphics
Computing
Interaction
Wrap-up
User Interactions, A Computational Approach
• Now that we’ve shown that (interaction ~= reasoning )
– Can we automate the process?
• Consider each of a user’s interactions as a fixed-length vector
(Design Galleries [Marks et al. Siggraph 97]).
Computer
•
•
Process
(Translate)
Human
User interaction in the left application can be represented as a single dimensional
vector <P>
User interaction in the right application can be represented as a two dimensional
vector <P, S>
29/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Conclusion
• Visual Analytics is a growing new
area that is looking to address
some pressing needs
– Too much (messy) data, too little
time
Graphics &
Visualization
Interaction
&
Reasoning
Computing
• By integrating interaction,
graphics, and data computation,
we have demonstrated that
– There are some great benefits
– But there are also some difficult
challenges
• With great challenges come great
opportunities…
– Government agencies
– Industrial partners
Intro
30/33
VA
Graphics
Computing
Interaction
Wrap-up
Future Work (Funded Projects)
•
NSF SciSIP:
– Title: A Visual Analytics Approach to Science and Innovation Policy.
•
•
PI: William Ribarsky, Co-PIs: Jim Thomas, Remco Chang, Jing Yang.
$746,567. 2009-2012 (3 years).
– Abstract: developing metrics and visual tools for identifying patterns in science policies.
•
NSF/DOD (Minerva Initiative):
– Title: Collaborative Project: Terror, Conflict Processes, Organizations, & Ideologies: Completing
the Picture.
•
•
PI: Remco Chang
$100,000. 2009-2010 (2 years).
– Abstract: design and develop visual analytical tools to identifying the causal relationships in
government policies and domestic conflicts.
•
DHS International Program:
– Title: Deriving and Applying Cognitive Principles for Human/Computer Approaches to
Complex Analytical Problems.
•
•
PI: William Ribarsky, Co-PIs: Brian Fisher, Remco Chang, John Dill.
$200,000. 2009-2010 (1 year).
– Abstract: identifying new evaluation methods for visual analytical systems, and applying
computational methods for analyzing user interactions.
•
Quantitative Analysis Division at Bank of America
– Exploration and analysis of financial risk
31/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Future Work (On-going Collaborations)
• With NSF FODAVA Center at Georgia Tech (Dr. Haesun Park, director)
– Interpreting user interactions to affecting machine learning algorithms
– Visual PCA: using perceptual metrics to finding principle components
– Applying perceptual constraint to dimension reduction: for animating temporal
data in dimension reduction, find methods to maintain hysteresis
• With University of Kentucky (Drs. Judy Goldsmith, Jinze Liu, Phillip Chang, MD)
– Integrating data mining (KDD), POMDP, and visual analytics to prevent sepsis by
identifying biomarkers (Proposal in submission to NSF CDI)
• With geographer and architect at UNC Charlotte (Dr. Jean-Claude Thill and Eric
Sauda)
– Designing computational methods for identifying neighborhood characteristics
(Proposal in submission to NSF IIS)
– Applying the UrbanVis system to analyzing crime (proposal in preparation for
DOJ/NIJ)
• With Virginia Tech (Dr. Chris North) and Pacific Northwest National Lab (Dr. Bill
Pike and Richard May)
– Developing a research agenda for analytic provenance (Workshop proposal in
submission to DHS)
32/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Thank you!
Graphics &
Visualization
Interaction
&
Reasoning
Computing
rchang@uncc.edu
http://www.viscenter.uncc.edu/~rchang
33/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Acknowledgement
From the Data Visualization Group (DVG) at UNC Charlotte
Bill Ribarsky
Zach Wartell
Dong Hyun Jeong, Tom Butkiewicz, Xiaoyu Wang, Wenwen Dou, Tera Green
34/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Acknowledgement
From the Urban Visualization Group at UNC Charlotte
Eric Sauda
Jean-Claude Thill
Ginette Wessel
Elizabeth Unruh
35/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Acknowledgement
More Collaborators…
Clockwise, starting on the left:
Nancy Pollard (CMU), Evan Suma (UNCC), Heather Lipford (UNCC), Dan Keefe (UMN),
Caroline Ziemkiewicz (UNCC), Robert Kosara (UNCC), Mohammad Ghoniem
36/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Acknowledgement
• And many many others…
Joseph Kielman (DHS), Bill Pike (PNNL), Theresa
O'Connell (NIST), Seok-Won Lee (UNCC), Brian Fisher
(Simon Fraser), Alvin Lee (BofA), Jing Yang (UNCC),
Daniel Kern (BofA), Agust Sudjianto (BofA), Erin Miller
(UMD), Kathleen Smarick (UMD), Felesia Stukes
(UNCC), Marcus Ewert (UMN), Larry Hodges
(Clemson), Michael Butkiewicz (UC Riverside), Josh
Jones (BofA), Alex Godwin (Charles River Analytics),
Edd Hauser (UNCC), Shenen Chen (UNCC), Bill Tolone
(UNCC), Wanqiu Liu (UNCC), Rashna Vatcha (UNCC)
37/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Final Thought…
• “The sexy job in the next 10 years will be
statisticians,” said Hal Varian, chief economist at
Google. “And I’m not kidding.”
Graphics &
Visualization
Interaction
&
Reasoning
Computing
• Yet data is merely the raw material of knowledge.
“We’re rapidly entering a world where everything
can be monitored and measured,” said Erik
Brynjolfsson, an economist and director of the
Massachusetts Institute of Technology’s Center
for Digital Business. “But the big problem is going
to be the ability of humans to use, analyze and
make sense of the data.”
• “The key is to let computers do what they are
good at, which is trawling these massive data sets
for something that is mathematically odd,” said
Daniel Gruhl, an I.B.M. researcher whose recent
work includes mining medical data to improve
treatment. “And that makes it easier for humans
to do what they are good at — explain those
anomalies.”1
1. New York Times. “For Today’s Graduate, Just One Word: Statistics “, August 5, 2009.
38/33
Intro
VA
Graphics
Computing
Interaction
Backup Slides – Visual Analytics
Wrap-up
39/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Individually Not Unique
• Data Mining
• Machine
Learning
• Databases
• Information
Retrieval
• etc
Analytical
Reasoning
and
Interaction
Data
Representation
Transformation
Production,
Presentation
Dissemination
• Tech Transfer
• Report Generation
• etc
•
•
•
•
Interaction Design
Cognitive Psychology
Intelligence Analysis
etc.
Visual
Representation
•
•
•
•
InfoVis
SciVis
Graphics
etc
Validation
and
Evaluation
• Quality Assurance
• User studies (HCI)
• etc
40/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
In Combinations of 2 or 3…
• Data Mining
• Machine
Learning
• Databases
• Information
Retrieval
• etc
Analytical
Reasoning
and
Interaction
Data
Representation
Transformation
Production,
Presentation
Dissemination
Visual
Representation
Validation
and
Evaluation
•
•
•
•
InfoVis
SciVis
Graphics
etc
41/33
Intro
VA
Graphics
Computing
Interaction
In Combinations of 2 or 3…
Analytical
Reasoning
and
Interaction
Data
Representation
Transformation
Production,
Presentation
Dissemination
• Tech Transfer
• Report Generation
• etc
•
•
•
•
Interaction Design
Cognitive Psychology
Intelligence Analysis
etc.
Visual
Representation
Validation
and
Evaluation
Wrap-up
42/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
This Talk Focuses On…
• Data Mining
• Machine
Learning
• Databases
• Information
Retrieval
• etc
Analytical
Reasoning
and
Interaction
Data
Representation
Transformation
Production,
Presentation
Dissemination
•
•
•
•
Interaction Design
Cognitive Psychology
Intelligence Analysis
etc.
Visual
Representation
Validation
and
Evaluation
•
•
•
•
InfoVis
SciVis
Graphics
etc
43/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Eureka: Visual Analytics!!
“Saunders, perhaps you’re getting a bit carried away
with the visual analytics!”1
1: Slide courtesy of Dr. Maria Zemankova, NSF
44/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Case Study on WireVis
• User Centric
– Designed system based on
domain expertise
• Visual Interface
– Multiple coordinated views
that link multiple dimensions
• Interactive
– Overview, drill-down,
reclustering
• Data Clustering
Analytical
Reasoning
and
Interaction
Data
Representat
ion
Transformat
ion
Production,
Presentatio
n
Disseminati
on
Visual
Represent
ation
– Clustering by accounts, and
search by example
• Production
– Connected to a live database
and beta-deployed at BofA
Validation
and
Evaluation
• (Validation)
– Expert evaluation
45/33
Intro
VA
Graphics
Computing
Interaction
Backup Slides – Urban Simplification
Wrap-up
46/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Algorithm to Preserve Legibility
• Identify and preserve Paths and Edges
• Create logical Districts and Nodes
• Simplify model while preserving Paths, Edges, Districts, and
Nodes
• Hierarchically apply appropriate amount of texture
• Highlight Landmarks and choose models to render
47/33
Intro
VA
Graphics
Computing
Identifying and Preserving
Edges
Interaction
Wrap-up
Paths and
48/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Identifying and Preserving
Paths and Edges (1)
a
b
c
d
bc
• Single-Link Clustering
– Iteratively groups the “closest” clusters
together based on Euclidean distance
– produces a binary tree (dendrogram)
– Penalizes large clusters to create a more
balanced tree
e
de
def
abc
bcdef
abcdef
f
49/33
Intro
VA
Graphics
Computing
Identifying and Preserving
Paths and Edges (2)
Interaction
Wrap-up
50/33
Intro
VA
Creating logical
and Nodes
Graphics
Computing
Interaction
Wrap-up
Districts
51/33
Intro
VA
Creating logical
Nodes (1)
Graphics
Computing
Interaction
Wrap-up
Districts and
52/33
Intro
VA
Graphics
Computing
Creating logical
Nodes (2)
Interaction
Wrap-up
Districts and
• Merge two clusters by combining footprints
(a)
(b)
(c)
• (c) The resulting “Merged Hull”
• (d) The Introduced Error, or “Negative Space”
(d)
53/33
Intro
VA
Graphics
Computing
Simplification while preserving
Edges, Nodes, and Districts
Interaction
Paths,
Wrap-up
54/33
Intro
VA
Graphics
Computing
Simplification while preserving
Edges, Nodes, and Districts (1)
6000 edges
Interaction
Wrap-up
Paths,
1000 edges
Demo!
55/33
Intro
VA
Graphics
Computing
Simplification while preserving
Edges, Nodes, and Districts (2)
Interaction
Wrap-up
Paths,
• After the polylines have been simplified
– Create “Cluster Meshes”
– The height of the Cluster Mesh is the median height of all buildings
in the cluster
56/33
Intro
VA
Graphics
Hierarchical Textures
Computing
Interaction
Wrap-up
57/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Hierarchical Textures (1)
• Each Cluster Mesh contains 6 textures
– 1 Side Texture
– 1 top-down view of the roof texture
– 4 roof textures from 4 angles
(south, west, east, north)
Top-down
South
Side texture
West
East
North
58/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Hierarchical Textures (2)
• Clusters are divided into “bins” based on their visual importance
• Each bin contains a texture atlas
• Texture atlases from all bins have the same dimension
n/2
n/4
n/8
….
…
59/33
Intro
VA
Graphics
Computing
Runtime Levels of Detail
Interaction
Wrap-up
60/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Runtime Levels of Detail
• Starting with the root node of
abcdef
the dendrogram
– Approximate the “Negative
abc
Space” as a 3D box – shown as
the red box
def
– Project the visible sides of the
bc
box onto screen space
– Reject if the number of pixel is
above a user-defined tolerance
a
b
de
c
d
e
f
61/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Landmark and Skyline Preservation (1)
Original Skyline
Without Landmark
Preservation
With Landmark
Preservation
62/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Landmark and Skyline Preservation (2)
– Project a user-defined pixel tolerance (α) onto the top of each cluster
– If any building within that cluster is taller than the projected
tolerance (shown in green), it is drawn separately from the cluster
mesh.
63/33
Intro
Results
VA
Graphics
Computing
Interaction
Wrap-up
64/33
Intro
VA
Graphics
Computing
Backup Slides – VA Systems
Interaction
Wrap-up
Intro
65/33
VA
Graphics
Computing
Interaction
Wrap-up
(2) Investigative GTD
Who
Where
What
Evidence
Box
Original
Data
R. Chang et al., Investigative Visual Analysis of Global Terrorism, Journal of Computer Graphics Forum (Eurovis), 2008.
When
66/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
(2) Investigative GTD:
Revealing Global Strategy
This group’s attacks
are not bounded by
geo-locations but
instead, religious
beliefs.
Its attack patterns
changed with its
developments.
67/33
Intro
VA
Graphics
Computing
Interaction
(2) Investigative GTD:
Discovering Unexpected Temporal Pattern
A geographicallybounded entity in the
Philippines.
The ThemeRiver shows
its rise and fall as an
entity and its modus
operandi.
Domestic Group
Wrap-up
68/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
(3) Analysis of Biomechanical Motion
• Biomechanical motion
sequences (animation) are
difficult to analyze.
• Watching the movie repeatedly
does not easily lead to insight.
• Collaboration with Brown University and Univ. of Minnesota
to examine the mechanics of a pig chewing different types
and amounts of food (nuts, pig chow, etc.)
• The data is typically organized by the rigid bodies in the
model, where each rigid body contains 6 variables per frame
-- 3 for translation, and 3 for rotation.
69/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
(3) Analysis of Biomechanical Motion
R. Chang et al., Interactive Coordinated Multiple-View Visualization of Biomechanical Motion Data, IEEE Vis (TVCG) 2009. To Appear.
70/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
(3) Analysis of Biomechanical Motion
• Our emphasis is on “interactive comparison.”
Following the work by Robertson [InfoVis
2008], comparisons can be performed using:
– Small Multiples
– Side by side comparison
– Overlap
• Between two datasets
• Different cycles in the same data
71/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Human + Computer:
Dimension Reduction – Lost in Translation
• Biomechanical motion analysis revisited…
– 6 degrees of freedom (x, y, z rotation and x, y, z translation)
– One single joint
• Applying a non-linear
dimension reduction method
– Isomap
– MDS embedding
• We found:
– 3 latent dimensions
– 2 of which are ambiguous…
72/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
What is in a User’s Interactions?
Keyboard, Mouse, etc
Input
Visualization
Human
Output
Images (monitor)
• Types of Human-Visualization Interactions
– Word editing (input heavy, little output)
– Browsing, watching a movie (output heavy, little input)
– Visual Analysis (closer to 50-50)
73/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
What’s in a User’s Interactions
• Why are these so much
lower than others?
– (recovering “methods” at
about 15%)
• Only capturing a user’s
interaction in this case is
insufficient.
74/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Discussion
• What interactivity is not good for:
– Presentation
– YMMV = “your mileage may vary”
• Reproducibility: Users behave differently each time.
• Evaluation is difficult due to opportunistic discoveries..
– Often sacrifices accuracy
• iPCA – SVD takes time on large datasets, use iterative
approximation algorithms such as onlineSVD.
• WireVis – Clustering of large datasets is slow. Either
pre-compute or use more trivial “binning” methods.
75/33
Intro
VA
Graphics
Computing
Discussion
• Interestingly,
– It doesn’t save you time…
– And it doesn’t make a user more
accurate in performing a task.
• However, there are empirical
evidence that using interactivity:
– Users are more engaged (don’t
give up)
– Users prefer these systems over
static (query-based) systems
– Users have a faster learning curve
• We need better measurements
to determine the “benefits of
interactivity”
Interaction
Wrap-up
76/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Human + Computer:
User Interactions – Lessons Learned
• Showing reasoning and intent are capturable.
– Although the study is limited in scope, it establishes a
foundation for interaction-capturing related research
• With interaction capturing, we might be able to collect all
the thinking of expert analysts and create a knowledge
base that is useful for
– Training: many domain specific analytics tasks are difficult to
teach
– Guidance: use existing knowledge to guide future analyses
– Verification, and validation: check to see if everything was done
right.
• Automating the process of extracting thinking is the key.
– By formulating user interactions as high dimensional vectors, we
can apply analytical methods
77/33
Intro
VA
Graphics
Computing
Interaction
Backup Slides – Professional Activities
Wrap-up
Intro
78/33
VA
Graphics
Computing
Interaction
Wrap-up
Professional Activities
•
Committee / Panelists
–
–
–
–
–
•
Program Committee: IEEE Conference on Visual Analytics, 2010
Program Committee: SIG CHI Workshop on BELIV, 2010
Program Committee: AAAI Spring-09 Symposium on Technosocial Predictive Analytics, 2009
Panelist: 3rd Annual DHS University Summit. Panel: Research to Reality, 2009
Panelist: 3rd Annual DHS University Summit. Panel: Visual Analytics and Discrete Science Integration into the DHS Center of
Excellence Program, 2009
Invited Talks
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
Dec 13, 2006 Google Inc. Simplification of Urban Models based on Urban Legibility
July 6, 2007 Naval Research Lab. Urban Visualization
Oct 4, 2007 Charlotte Viscenter. Urban Visualization
Oct 17, 2007 Charlotte Metropolitan GIS Users Group. GIS and Urban Visualization
Nov 19, 2007 START Center at University of Maryland. Integrated Visual Analysis of the Global Terrorism Database
Nov 29, 2007 Charlotte Viscenter. Integrated Visual Analysis of the Global Terrorism Database
Jan 25, 2008 DoD/DHS Social Science Modeling and Information Visualization Symposium. Social Science and Information
Visualization on Terrorism and Multimedia
May 14, 2008 Charlotte Metropolitan GIS User Group. Multi-Focused Geospatial Analysis Using Probes
Aug 27, 2008 DoD/DHS Symposium for Overcoming the Information Challenge in Federated Analysis: From Concept to Practice.
Roadmap of Visualization
Mar 19, 2009 DHS University Summit. Panel: Research to Reality
Mar 19, 2009 DHS University Summit. Panel: Visual Analytics and Discrete Science Integration into the DHS Center of Excellence
Program
Apr 27, 2009 University of Kentucky. Thinking Interactively with Visualization
May 29, 2009 University of Victoria. Thinking Interactively with Visualization
Jul 28, 2009 Pacific Northwest National Lab. Thinking Interactively with Visualization
Jul 30, 2009 Microsoft Research. Thinking Interactively with Visualization
Aug 19, 2009 National Visual Analytic Consortium. What Are Your Interactions Doing For Your Visualization?
Sep 30, 2009 University of Kentucky (Grand Rounds at the Department of Surgery). Preventing
Sepsis: Artificial Intelligence, Knowledge Discovery, and Visualization
Jan 21, 2010 Charlotte Viscenter. UrbanVis Research Group: Urban Analytics
Feb 25, 2010 University of Georgia (AI Institute). Thinking Interactively with Visualization
79/33
Intro
VA
Graphics
Computing
Summary of Contributions
• Contributions
– Graphics/Visualization
• Urban modeling and visualization
– Visualization + Interaction
• Role of interactivity in visual thinking
• Applying principles to real-world
problems such as financial analytics,
terrorism studies, bridge management,
biomechanical motion analysis, etc.
– Interaction + Computing
• Exploring principle component analysis
• Study of user interactions in visual
analytics systems
• In particular, foundations in computer
graphics help the development of a
human + visual computing research
agenda
Interaction
Wrap-up
Intro
80/33
VA
Graphics
Computing
Interaction
Wrap-up
Journal Publications (16)
•
Urban Visualization
–
–
–
–
•
Visualization and Visual Analytics
–
–
–
–
–
–
–
•
X. Wang, W. Dou, S.E. Chen, W. Ribarsky, and R. Chang. An interactive visual analytics system for bridge management. Computer
Graphics Forum (Eurovis 2010), 2010. Conditional acceptance.
D. Keefe, M. Ewert, W. Ribarsky, and R. Chang. Interactive coordinated multiple-view visualization of biomechanical motion data.
Visualization and Computer Graphics, IEEE Transactions on (IEEE Visualization Conference), 15(6):1383–1390, 2009
X. Wang, D.H. Jeong, W. Dou, S.W. Lee, W. Ribarsky, and R. Chang. Defining and applying knowledge conversion processes to a visual
analytics system. Computers & Graphics, July 2009. [Online] doi:10.1016/j.cag.2009.06.004
D.H. Jeong, C. Ziemkiewicz, B. Fisher, W. Ribarsky, and R. Chang. iPCA: An interactive system for PCA-based visual analytics. Computer
Graphics Forum, 28(3):767–774, 2009.
R. Chang, C. Ziemkiewicz, T.M. Green, and W. Ribarsky. Defining insight for visual analytics. IEEE Computer Graphics and Applications,
29(2):14–17, 2009.
R. Chang, A. Lee, M. Ghoniem, R. Kosara, W. Ribarsky, J. Yang, E. Suma, C. Ziemkiewicz, D. Kern, and A. Sudjianto. Scalable and interactive
visual analysis of financial wire transactions for fraud detection. Information Visualization, 7:63–76(14), 2008.
X. Wang, E. Miller, K. Smarick, W. Ribarsky, and R. Chang. Investigative visual analysis of global terrorism database. Computer Graphics
Forum, 27(3):919–926, 2008.
Interaction & Provenance
–
–
•
R. Chang, T. Butkiewicz, C. Ziemkiewicz, Z. Wartell, N. Pollard, and W. Ribarsky. Legible simplification of textured urban models. IEEE
Computer Graphics and Applications, 28(3):27–36, 2008.
T. Butkiewicz, R. Chang, Z. Wartell, and W. Ribarsky. Visual analysis of urban change. Computer Graphics Forum, 27(3):903–910, 2008.
T. Butkiewicz, R. Chang, W. Ribarsky, and Z. Wartell. Understanding Dynamics of Geographic Domains, chapter Visual Analysis of Urban
Terrain Dynamics, pages 151– 169. CRC Press/Taylor and Francis, 2007.
R. Chang, G. Wessel, R. Kosara, E. Sauda, and W. Ribarsky. Legible cities: Focus-dependent multi-resolution visualization of urban
relationships. Visualization and Computer Graphics, IEEE Transactions on, 13(6):1169–1175, Nov.-Dec. 2007.
W. Pike, J. Stasko, R. Chang, and T. O’Connell. Science of interaction. Information Visualization, 8:263–274, 2009.
W. Dou, D.H. Jeong, F. Stukes, W. Ribarsky, H. Lipford, and R. Chang. Recovering reasoning process from user interactions. IEEE Computer
Graphics and Applications, 29(3):52–61, 2009
VR & Interface Designs
–
–
–
T. Butkiewicz, R. Chang, Z. Wartell, and W. Ribarsky. Alleviating the modifiable areal unit problem with probe-based geospatial analyses.
Computer Graphics Forum (Eurovis 2010), 2010. Conditional acceptance
T. Butkiewicz, W. Dou, Z. Wartell, W. Ribarsky, and R. Chang. Multi-focused geospatial analysis using probes. Visualization and Computer
Graphics, IEEE Transactions on, 14(6):1165–1172, Nov.-Dec. 2008.
D.H. Jeong, C. Song, R. Chang, and L. Hodges. User experimentation: An evaluation of velocity control techniques in immersive virtual
environments. Springer-Verlag Virtual Reality, 13(1):41–50, Mar. 2009.
81/33
Intro
VA
Graphics
Computing
Interaction
Wrap-up
Conference/Workshop (22)
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
R. Chang, C. Ziemkiewicz, R. Pyzh, J. Kielman, and W. Ribarsky. Learning-based evaluation of visual analytics systems. In ACM SIGCHI BELIV Workshop, 2010.
Conditional acceptance.
D. H. Jeong, T. Green, W. Ribarsky, and R. Chang. Comparative evaluation of two interface tools in performing visual analytics tasks. In ACM SIGCHI BELIV
Workshop, 2010. Conditional acceptance.
G. Wessel, E. Unruh, R. Chang, and E. Sauda. Urban user interface: Urban legibility reconsidered. In Southwest ACSA, 2010.
D. H. Jeong, W. Dou, W. Ribarsky, and R. Chang. Knowledge-oriented refactoring in visualization. In IEEE Visualization Workshop on Refactoring Visualization
From Experience, 2009.
D. H. Jeong, W. Ribarsky, and R. Chang. Designing a PCA-based collaborative visual analytics system. In IEEE Visualization Workshop on Collaborative
Visualization, 2009.
W. Dou, D. H. Jeong, F. Stukes, W. Ribarsky, H. Lipford, and R. Chang. Comparing usage patterns of domain experts and novices in visual analytical tasks. In ACM
SIGCHI Sensemaking Workshop 2009.
X. Wang, W. Dou, R. Vatcha, W. Liu, S. E. Chen, S. W. Lee, R. Chang, and W. Ribarsky. Knowledge integrated visual analysis of bridge safety and maintenance. In
SPIE 2009.
X. Wang, W. Dou, W. Ribarsky, and R. Chang. Integration of heterogeneous processes through visual analytics. In SPIE 2009,.
M. Butkiewicz, T. Butkiewicz, W. Ribarsky, and R. Chang. Integrating timeseries visualizations within parallel coordinates for exploratory analysis of incident
databases. SPIE 2009.
T. Butkiewicz, D. H. Jeong, W. Ribarsky, and R. Chang. Hierarchical multitouch selection techniques for collaborative geospatial analysis. In SPIE Defense, Security
and Sensing 2009.
D. H. Jeong, R. Chang, and W. Ribarsky. An alternative definition and model for knowledge visualization. In IEEE Visualization Workshop on Knowledge Assisted
Visualization, 2008.
X. Wang, W. Dou, S. W. Lee, W. Ribarsky, and R. Chang. Integrating visual analysis with ontological knowledge structure. In IEEE Workshop on Knowledge
Assisted Visualization, 2008.
D. H. Jeong, W. Dou, F. Stukes, W. Ribarsky, H. Lipford, and R. Chang. Evaluating the relationship between user interaction and financial visual analysis. In Visual
Analytics Science and Technology. IEEE Symposium on, 2008.
G. Wessel, R. Chang, and E. Sauda. Towards a new (mapping of the) city: Interactive, data rich modes of urban legibility. In Association for Computer Aided
Design in Architecture, 2008.
G. Wessel, R. Chang, and E. Sauda. Visualizing GIS: Urban form and data structure. Seeking the City: Visionaries on the Margins, ACSA, 2008.
G. Wessel, E. Sauda, and R. Chang. Urban visualization: Urban design and computer visualization. In CAADRIA 2008.
T. Butkiewicz, R. Chang, Z. Wartell, and W. Ribarsky. Visual analysis for live lidar battlefield change detection. SPIE, 2008.
J. Jones, R. Chang, T. Butkiewicz, and W. Ribarsky. Visualizing uncertainty for geographical information in the global terrorism database. SPIE, 2008.
A. Godwin, R. Chang, R. Kosara, and W. Ribarsky. Visual analysis of entity relationships in the global terrorism database. SPIE, 2008.
T. Butkiewicz, R. Chang, Z. Wartell, and W. Ribarsky. Analyzing sampled terrain volumetrically with regard to error and geologic variation. SPIE, 2007.
R. Chang, M. Ghoniem, R. Kosara, W. Ribarsky, J. Yang, E. Suma, C. Ziemkiewicz, D. Kern, and A. Sudjianto. Wirevis: Visualization of categorical, time-varying
data from financial transactions. In Visual Analytics Science and Technology, 2007, IEEE Symposium on, 2007.
R. Chang, T. Butkiewicz, C. Ziemkiewicz, Z. Wartell, N. Pollard, and W. Ribarsky. Hierarchical simplification of city models to maintain urban legibility. In
SIGGRAPH ’06: ACM SIGGRAPH 2006 Sketches, 2006.
R. Chang, R. Kosara, A. Godwin, and W. Ribarsky. Towards a role of visualization in social modeling. AAAI 2009 Spring Symposium on Technosocial Predictive
Analytics, 2009.
G. Wessel, E. Sauda, and R. Chang. Mapping understanding:Transforming topographic maps into cognitive maps. GeoVis Hamburg Workshop, 2009.