How Human Brain Understand Visualisation
and Graph Visualisation Evaluation
Kai Xu
National ICT Australia
Sydney, Australia
Part I: How Human Braid Understand Visualisation
Acknowledgement
• Most of the content can be
found in the book “Information
Visualization – Perception for
Design”.
• It’s author Colin Ware at
University of New Hampshire
is a close friend of the
information visualisation
research group at School of IT.
Table of Contents
• The human visual perception system.
• Visual Attention and pre-attentive patterns.
Visual Perception System
Information flow
Visualisation
Eyes
Brain
A
B
C
D
Parallel processing
of orientation, texture,
Object Identification,
color and motion
Working Memory
features
Detection of 2D patterns,
contours and regions
Visual Perception System
A
B
C
D
Parallel processing
of orientation, texture,
Object Identification,
color and motion
Working Memory
features
Detection of 2D patterns,
contours and regions
Brain
processing
speed
Task
complexity
Visual Perception System
Stage 1
A
B
C
D
Parallel processing
of orientation, texture,
Object Identification,
color and motion
Working Memory
features
Detection of 2D patterns,
contours and regions
• Rapid parallel processing: billions of neurons;
• Extraction of orientation, texture, color, and motion features.
Visual Perception System
A
B
C
D
Parallel processing
of orientation, texture,
Object Identification,
color and motion
Working Memory
features
Detection of 2D patterns,
contours and regions
Stage 2
• Slower processing than stage 1;
• Detection of 2D patterns, contours and regions.
Visual Perception System
Stage 3
A
B
C
D
Parallel processing
of orientation, texture,
Object Identification,
color and motion
Working Memory
features
Detection of 2D patterns,
contours and regions
• Slow serial processing;
• Involve both working and long-term memory;
• Object identification and eye-hand coordination.
Visual Angle
• Visual angle: the angle subtended by an object at the
eye of an observer.
– A thumbnail held at arm length subtends about 1 degree of
visual angle.
Human Visual Field
100
• Visual field of view when
gazing straight ahead.
80
60
– Can see slightly more than
180 degrees horizontally;
– Much less angle vertically.
40
20
LEFT
RIGHT
• The irregular boundary of left
and right fields are caused by
facial features such as nose.
• The dark grey area is the
region of binocular overlap.
Visual Acuity
• If focus on the central
point, every character
is about equally
distinct.
• This is because the
visual acuity
decreases quickly
with the distance from
fovea.
Acuity test
Line test
Points test
Acuity distribution
Pixels and
Brain Pixels
Brain pixel
screen pixel
1 bp
Small Screen
Big Screen
Ultimate Display
High resolution
binocular insert (25x19)
Left eye
background
(82.5x66.7)
Right eye
background
(82.5x66.7)
Binocular overlap
Visual Attention and Pre-Attentive Patterns
• In the stage 1 of perception system, the whole visual
field is processed in parallel and very fast;
• The information that can be captured in this stage are
easily distinguished.
• Pre-attentive patterns (pop-out effects).
• Should be considered when designing visualisation.
• Some examples.
Color
Orientation
Motion
Size
Simple shading
Conjunction (does not pop out)
Compound features (do not pop out)
Surrounded colors do not pop out
Laws of pre attentive display
• Must stand out on some simple dimension
–
–
–
–
color,
simple shape = orientation, size
motion,
depth
• Conjunctions of pre-attentive dimensions do not always
work.
Lessons for Information Visualisation
• Can be used for individual symbols or areas;
• Avoid possible negative effect:
– Do not use large areas of strong color.
• Orthogonality: use a different channel for a different type of
information.
• Example: Mapping high dimensional data to display variables.
–
–
–
–
–
–
–
Position (2)
Orientation (1)
Size (1)
Motion (2)++
Blinking (1)
Color (3)
…
Part II: Graph Visualisation Evaluation
Table of Contents
•
•
•
•
•
•
Quick review
Introduction
Evaluation of graph drawing aesthetics
Evaluation of graph layout methods
Evaluation of large graph visualization
Conclusions
Review - Drawing conventions
Review - Graph Layout
• Important part of graph visualization
• Finds “good” positions for nodes and edges
– To improve “graph readability”, i.e., facilitate people's
understanding of the graph structure
• Review: Layout algorithm covered in the previous
lectures:
– Tree layout
– Layered layout (Sugiyama method)
– Force-directed layout (spring algorithm)
Tree
• Layered drawing
– nodes are placed on
horizontal layers
• Radial drawing
– the layers are mapped to
concentric circles
• HV drawing
– places the edges horizontally
or vertically
• Space-filling methods
(Treemap)
– Inclusion indicates parentchild relationship;
– Improved space efficiency
Directed Graph
Layered drawing (framework)
1. Cycle removal: if there is directed
cycles, temporarily reverses the
direction of some to make the
graph acyclic;
2. Layer assignment: nodes are
assigned to horizontal layers, and
thus determines their ycoordinate;
3. Crossing reduction: within each
layer the nodes are ordered to
reduce the number of crossings;
4. Horizontal coordinate
assignment: the x-coordinates of
each vertex is determined.
Undirected Graph
• Force-directed methods
– A graph is treated as a system
of entities with force acting
between them.
– The algorithm seeks a
configuration with locally
minimal energy, i.e. , a
position for every entity such
that the sum of the forces on
each entity is zero.
– Common example
• Spring embedder
Introduction
• What’s covered so far:
– Various Graph layout algorithms
• This lecture: how these affect people's understanding of
the graph.
– Are they effective at all?
– Which one is relatively more effective?
• Also: visualization of large graphs
– Where the traditional aesthetics and layout algorithms do not
really work
Graph Drawing Aesthetics
Aesthetics are the graphic properties
layout algorithm try to optimise.
• Crossings:
– Minimization of the total crossing
number
•
Area
– Minimization of drawing area
– Only meaningful to some layout.
Example, grid drawing with integer
coordinates
•
Aspect ratio
– The ratio of the long and short
edge length of its covering
rectangle
– Ideal case is to obtain any aspect
ratio in a given range (so the
drawing can fit into differently
shaped screen space)
Graph Drawing Aesthetics
• Edge length (several variations):
–
–
–
–
minimization of the sum of the edge length;
minimization of the maximum edge length;
minimization of the variance of the edge length;
only meaningful to some layout algorithm.
• Bends (several variations):
–
–
–
–
minimization of the total number of bends;
minimization of maximum number of bends on an edge;
minimization of the variance of the number of bends on the edge;
trivially satisfied by straight-line drawing.
Graph Drawing Aesthetics
• Angular resolution:
– maximization of the smallest angle;
– especially relevant for straight-line drawing.
• Symmetry:
– display the symmetries of the graph in drawing
– reflective and rotational symmetry
• Orthogonality:
– how well the edges are parallel to the axes,
and how well the nodes match to a grid;
• Upward flow:
– for directed graph only,
– how well edges are pointing to a specified
direction (usually upward);
Evaluation
• Measuring the performance of subjects (users)
completing certain task(s).
• Graph-related tasks are used for graph visualization.
– Example: find the shortest path between 2 nodes in a graph.
• Performance is usually measured by
– Accuracy
– Completion time
Graph-Related Task Performance
There are many factors affect the performance
• The difficulty level of the task
– Simple: find all the neighbors of a node
– Hard: find all the nodes have graph distance 2 to two given nodes.
• Size of the graphs
– Small
– Large
• Subjects background
– Whether they are familiar with graph visualization or not
– Whether they are familiar with certain application domain (for domainspecific tasks).
• And many more …
• Should consider/control as many as possible when doing a test.
Table of Contents
•
•
•
•
•
Introduction
Evaluation of graph drawing aesthetics
Evaluation of graph layout methods
Evaluation of large graph visualization
Conclusions
Do Aesthetics Affect Graph Readability ?
•
Problem: Most aesthetics are proposed based on
experience
– Later becomes something the research community agree on
•
Study: readability of abstract graph
– Tasks are not domain specific
Purchase, H.C. et al. (1996)
Dataset
•
Three aesthetics are tested:
1. Minimizing edge crossings,
2. Minimizing bends, and
3. Showing symmetry.
•
Two planar graphs are used
– one with 16 nodes and 18 edges
– the other with 16 nodes and 28
edges
•
Nine drawings are produced for
each graph,
– with three levels (few, some and
many) of bends, crossings, and
symmetry respectively.
•
To isolate the effect of each
aesthetic, the drawings with
different bend levels shows no
crossings or symmetry.
– the same for the two other
aesthetics (manual layout).
•
Tasks:
1. Shortest path: the length of the shortest path between two
nodes;
2. Connections between nodes: minimum number of nodes
needs to be removed to disconnect two nodes;
3. Connections between subgraphs: minimum number of edges
needs to be removed to disconnect two subgraphs.
•
Results
– Effective: increasing bends or crossings decreases readability;
– Not clear: symmetry.
Caveats
• Dataset is fairly simple
– Small planar graph
• The selected tasks are similar
– All focus on path between nodes or subgraphs,
– this hardly cover all the information a graph structure can
possibly convey.
• It is possible that change in the dataset and/or tasks can
alter the results
Which Aesthetic is the most important?
•
•
The relative importance among aesthetics
Including 5 aesthetics:
1.
2.
3.
4.
5.
minimizing edge crossings,
minimizing bends,
symmetry.
minimum angle
orthogonality
Purchase, H.C (1997)
Dataset (Similar to last work)
• Planar graph with 16 nodes and 28 edges is used
• 5 aesthetics and 10 drawings
– 2 for each aesthetics: representing a strong or weak presence.
• b: bends, c: crossings, m: minimal angle, o: Orthogonality, s:
symmetry
•
Tasks (the same as the last work)
1. Shortest path: between two nodes;
2. Connections between nodes: number of nodes to disconnect
two nodes;
3. Connections between subgraphs: number of nodes to
disconnect two subgraphs.
•
Results
– Most important: reducing the number of crossing;
– Less effective: minimizing the number of bends and
maximizing symmetry;
– Not obvious: maximizing the minimum angle and orthogonality.
Does Aesthetics Affect Cognitive Load?
• From a cognitive psychology angle
• Testing aesthetics that affect shortest path task
performance:
– Continuity (path bendiness): the angular deviation from a
straight line.
– Number of crossings and average crossing angles: the
crossings on the shortest path, and the angle of crossing.
– Number of branches: the number of edges connect to the
nodes on the shortest path but not part of the path.
– Shortest path length and total edge length
Ware, C. et al. (2002)
• Task
– find the shortest path between 2 given nodes
• Dataset
– 180 drawings with per-defined parameters.
– 42 nodes in each graph,
– 2 examples
Results:
• Important: path continuity.
• Edge crossings
– Neutral: the total number of edge crossings in the graph.
– Important: those cross the shortest path.
• Important: the number of branches emanating from
nodes on the path.
Table of Contents
•
•
•
•
•
Introduction
Evaluation of graph drawing aesthetics
Evaluation of graph layout methods
Evaluation of large graph visualization
Conclusions
How People Read Graph?
• Eye movement: when people reading graph .
– using eye-tracking device.
• Dataset: 3 small social networks.
• Drawings:
– Four drawings each graph ;
– two circular layout and two radial layout;
– The pair of same layout: one has more crossings than the other.
• Task: shortest path between nodes
• Results:
– Graph layout can affect:
• slow down
• trigger extra eye movements.
– Caused by edges:
• incident to nodes concerned,
• going toward to the target node
• alongside the paths.
Huang, W., Eades, P. (2005)
Layout vs. Graph Readability
• Comparing different layout methods.
• A planar graph of 17 nodes and 29 edges .
• maximum node degree is 4:
– So it is applicable to orthogonal drawing ;
– A quite strong constraint.
• 3 layout algorithms:
– Force-directed: 3 variations;
– Planar orthogonal grid drawing: 2 variation;
– Planar grid drawing: 3 variations.
Tasks:
1. Shortest path between two nodes;
2. Disconnect two nodes;
3. Disconnect two subgraphs.
Results:
• One planar grid drawing method (SEIS) produced
significantly more errors than the rest;
• For the rest, the average response times were not
significantly different.
– So there is not much difference between layout algorithms!
Purchase, H.C. (1998)
Are the Drawings Actually Different?
• For one drawing method, there are usually many variations
– How different are the drawings?
• But first, what are the measurements?
• A study to find good measurements for orthogonal drawing.
• What’s interesting:
– The authors tried to find out the measurements through evaluation,
rather than just pick some.
Bridgeman, S.S. and Tamassia, R. (2001)
What difference needs to be measured?
• Rotation: the minimum angle needs to be rotated to make one
drawing the same as the other;
• Ordering: which drawing is more similar to the original;
• Magnitude: one drawing is k times more similar to the original than
the other drawing.
The tests:
• Rotation:
– Left: original drawing; right: same drawing with different orientations.
– The user chooses the one that looks most similar.
• Ordering:
– Left: original drawing; the other two: modified drawings;
– The user chooses the more similar one.
Magnitude
• Measures response times:
– Assumption: user will complete the task quicker if the drawings
are similar.
• Left: original drawing; Right: modified drawing;
• Identify missing vertex in the right drawing:
– Vertices have labels, otherwise the task is too difficult.
Evaluated Measures (a lot)
•
•
•
•
•
•
•
•
•
•
•
Corresponding Objects.
Point Set Selection.
Drawing Alignment.
Suitability for Ordering vs.
Rotation and Ordering.
Notation.
Undirected Hausdorff
Distance.
Paired Hausdorff Distance.
Average Distance.
Nearest Neighbor between.
Orthogonal Ordering.
Ranking.
•
•
•
•
•
•
…
Average Relative Distance
λ-Matrix
Nearest Neighbor Within
ε-Clustering
Separation-Based Clustering
Shape.
Results
• Rotation: absolute and relative point positions are
important.
• Ordering: point positions are less significant.
• Magnitude: no clear measurement is found.
• To recognize the graph as similar, the most important are
:
– the perimeter of the drawing;
– The position and shape of few key features.
• To find a specific change (magnitude)
– The drawings need to look very much alike, or
– Some other cues (color change, more distinctive vertex names,
etc.) are needed to highlight the change.
Table of Contents
•
•
•
•
•
Introduction
Evaluation of graph drawing aesthetics
Evaluation of graph layout methods
Evaluation of large graph visualization
Conclusions
Is Large Graph Readable?
• A graph with 3200 nodes.
• Laid out using force-directed
method.
• The “readability” will be
more or less the same with
other layout methods.
• Many real-world networks
are much larger.
• What can we do?
Clustered Graph
• Clustering can show the structure
of a large graph.
– The nodes are partitioned into
clusters;
– The connections between clusters
reveal the underlying graph
structure.
• Example: a multi-level 2.5D
drawing for hierarchically-clustered
graphs.
• Difficult to evaluate:
– Tightly coupled with interaction.
Extending Layout to 3D
Many 2D layout methods can be
extended to 3D:
• Natural extension:
– Force-directed method;
– Orthogonal drawing.
•
Layered tree drawing
– cone tree
•
Poly-plane drawing
– Placing subtree on poly-planes in
3D
•
Layered drawing of directed
graphs
– Placing nodes on the parallel
circles on the surface of a cone or
cylinder
– Placing nodes on parallel layers in
vertical planes (“walls”)
Does 3D Really Improve Readability?
Comparing various approaches to
visualize a graph in 2D and 3D
• 2D: orthographic (parallel)
projection
• Static Perspective:
perspective projection
• Stereo: shutter glasses
• Passive rotation: automatic
• Hand coupled rotation:
mouse-controlled rotation;
• Head coupled perspective:
head-controlled perspective;
• Combination of previous
approach: stereo, head
coupled perspective.
“Stereo, head coupled
perspective” setup
• Task: whether two nodes are connected by a path
of length 2.
• Dataset: randomly laid out graph.
Ware, C., Franck, G. (1996)
Results:
• A static perspective is only slightly better than a 2D
diagram;
• 3D motion and stereo viewing both help but not
particularly important;
– Both are more significant than stereo cues.
• Stereo viewing alone increases the understandable
graph size by a factor of 1.6;
• Head coupling alone increases by a factor of 2.2;
• Combine the two (head-coupled stereo viewing)
increases by a factor of 3;
What Else Can We Do With 3D?
• Improve graph readability with “geon diagram”
• Geon: simple 3D shape with color and texture.
• Geon diagram:
– Nodes: geons;
– Edges: connections between geons.
– Attributes of nodes and edges: geon color and texture.
• Experiment 1
– UML diagrams: geon vs. 2D;
– The subjects were first shown a structure in either geon or UML form,
and later asked to identify it in a series of diagrams.
• Results:
– Geon diagrams have half the errors and significantly faster.
– Geon diagrams can be recalled much more reliably.
• Experiment 2:
– Geon diagram vs. 2D geon
• Dataset and tasks are the same.
• Results:
– Geon is much more accurate and
reliable compare to 2D geon.
Motion
• Using the forth dimension — time — to improve graph
readability
• Different from dynamic or time-series graph visualization
– Graph is not changed
– Improving rather than reducing readability: it is usually more
difficult to understand the graph when it changes.
C. Ware and R. Bobrow. 2004.
Motion highlighting
• Circular:
– A circular motion around the center position
• Jolt:
– Moves in pulses;
– Similar to an object oscillating from being
struck briefly.
• Crawl:
– Animated sawtooth patterns radiating out
from the selected node
• Expanding nodes:
– Grow larger and smaller periodically
Comparing Different Motion Highlighting
•
Experiment 1
– Task: whether there are two red nodes within two links from a
circled node.
– Also included “no highlighting” and “static highlighting”, i.e., the
selected subgraph is marked with different color.
•
Results:
– Motion highlighting requires half times and more accurate
comparing to “no highlighting”;
– Static highlighting is the most accurate, but slower than motion
highlighting.
Experiment 2: Node and Edge Motion Highlighting
• Tasks:
– Are there at least two red nodes within two links of the specified
subgraph?
– Are there at least two red links within two links of the specified
subgraph?
• Results:
– Motion highlighting both nodes and links is as good as or better
than separate highlighting of nodes and links;
– The static highlighting:
• As good as the motion techniques for revealing links
• Not as good as circular motion for revealing nodes.
Experiment 3: Complex Pattern
• Task:
– Identify a chain of three red nodes
connected by blue edges.
• Results:
– Motion highlighting had similar
performance as static highlighting.
– The reason may be the pattern
was relatively easy to identify,
because it always started at the
ringed node.
Motion in Large Graph
•
Comparing 4 highlighting
methods:
1.
2.
3.
4.
•
Static highlighting:
–
–
•
No highlighting;
Static highlighting;
Motion highlighting;
Static and motion
highlighting.
increase node size;
Change edge color.
Combined highlighting
–
–
Combine static highlighting
with pulse highlighting;
The larger nodes and links
are in pulse highlighting
mode.
• Dataset:
– Graph of five sizes are used: 32, 100, 320, 1000, and 3200.
• Task 1:
– If there was a red node within 2 links of a specified node.
• Results 1:
– Motion highlighting is accurate even for the largest diagrams.
– Without highlighting, error rates were high even for the smallest
network.
– Motion highlighting and static highlighting were equally effective.
• Task 2: if 2 subgraphs had nodes in common.
• Highlighting methods:
– One with static highlighting and one with motion highlighting, or
– Two with different motion highlighting.
• Results 2:
– Best performer: one with motion highlighting and the other with
static highlighting.
Ware, C., Bobrow, R. (2005)
Beyond Node-Link Diagram
• Matrix Representation
• A graph can be represented by
a connectivity matrix.
• Advantage:
– no edge crossing
• Disadvantage:
– large empty space for spare
graph
A comparison between node-link diagram
and matrix representation
• Tasks:
– estimating the number of nodes;
– estimating the number of links;
– finding the most connected node;
– finding a node with a given label;
– finding a link between two specified nodes;
– finding a common neighbor between two specified nodes;
– finding a path between two nodes.
(more variety of tasks)
• Dataset
– Random graph of size: 20, 50, and 100 nodes;
– For each size, different link density: 0.2, 0.4 and 0.6.
Ghoniem, M. et al. (2004)
Results:
• When graphs are bigger than 20 vertices, matrix outperforms nodelink diagrams on most tasks.
– Only path finding is consistently in favor of node-link diagrams.
• For small graphs:
– Node-link diagrams are always more readable than matrices;
• For larger graphs:
– Matrices are 30% more accurate;
– Matrices have comparable or better answer time.
• For more complex tasks such as “path finding”, interaction is
needed:
– For example, displaying all the possible paths after selecting two nodes;
– For matrix, path can be displayed by connecting cells using curves (mix
matrix with node-link diagram).
Summary
• Effectiveness of aesthetics:
– Most effective: edge crossings;
– Less effective: bends and symmetry;
– Not obvious: minimum angle and orthogonality.
• Effectiveness of layout:
– No significant difference between force-directed, orthogonal and planar
layout methods.
• Large graphs:
– 3D is only better when coupled with interactions (and stereo vision if
possible).
– The 3D Geon diagram works better than 2D version;
– Motion highlighting is better than static highlighting in some tasks.
– Matrix representation is better than node-link diagram except path
finding.
Caveat
• These conclusion are only valid in their test setup.
• For instance, “different layout methods have little impact
on graph readability” when
– Graph is small and simple
– For shortest path task only
• It is possible that change in any test conditions can alter
the results.