Viz: A Visual Analysis Suite
for Explaining Local Search Behavior
http://www.comp.nus.edu.sg/~stevenha/viz
Steven Halim
Roland Yap Hock Chuan
Lau Hoong Chuin
(School of Computing, NUS)
(School of Computing, NUS)
(School of Information Systems, SMU)
User Interface Software and Technology 2006
Montreux, Switzerland
Outline
 Short Background:
 Local Search (LS) for attacking NP-hard
Combinatorial Optimization Problems (COPs),
 The problem: Understanding and Tuning LS…
 Our Approach:
 Explaining Local Search using Visualization (Viz)
 Viz DEMO
 Q &A
Background – Local Search
 Local Search
 e.g. Tabu Search, Iterated Local Search, Simulated Annealing
 …is commonly used for attacking NP-hard (intractable)
Combinatorial Optimization Problem (COP):
 e.g. Traveling Salesman Problem (TSP)
 Simplified explanation of Local Search:
Background – Tuning Problem
 Local Search algorithms have tunable configurations:
 Parameter values,
 Components, and/or
 Search strategies.
 These configurations affects its behavior  performance…
 To make a LS perform well in context is not straightforward.
 Its heuristic/stochastic behavior is not well understood.
 Empirical analysis is needed.
Common Practice: Ad Hoc Tuning
Implement the local search
Evaluate its performance
Textually/Statistically…
Good or
Give up
Stop
Modify the local search
using something??
(“ad hoc”)
Not good
Bottleneck !!!
We propose: Visual Diagnosis
Implement the local search
Evaluate its performance
Using Visualization Tool…
Good or
Give up
Stop
Not good
Modify the local search
after understanding what
is happening.
The tool helps us to think
more systematically.
Why Use Visualization?
 Visualization is a cognitive tool.
 Human brain is a powerful pattern finding machine.
 As long as the data is visualized in a way that conform
with the rules of human visual perception.
 Information Visualization: “The use of interactive visual
representations of abstract data to amplify cognition.”
 Facilitates hypothesis formation in empirical analysis.
 Allows users to answer questions they didn't know they had.
Explaining Local Search Behavior
via Visualization
http://www.comp.nus.edu.sg/~stevenha/viz
Explaining Local Search Behavior
It is more useful to understand how and why something works,
Rather than just simply know that something works…
By understanding the underlying principles,
We can adjust properly if presented with different scenarios!
To understand LS, we need to answer these questions:
1.
2.
3.
4.
5.
Does it behave like as what we intended?
How good is the local search in intensification?
How good is the local search in diversification?
Is there any sign of cycling behavior?
How does the local search algorithm make progress?
Explaining Local Search Behavior
Where in the search space does the search spend
most of its time?
7.
How far is the starting (initial) solution to the global
optima/best known solution?
8.
Does the search quickly find the global optima/best known
solution region or does it wander around in other regions?
9.
How wide is the local search coverage?
10. What is the effect of modifying a certain search
parameter/component/strategy w.r.t the search behavior?
11. How do two different algorithms compare?
6.
The behavior of heuristic and stochastic local search
is hard to explain without appropriate tools!
How to Understand LS Behavior?
 Various local search analysis tool:
 Data Analysis (text-based):
 Solution Quality
 Robustness
 Speed/Running Time
 Run Time/Length Distribution [Hoos 98; Hoos & Stuetzle, 05]
 Fitness Landscape (Statistical) Analysis [Jones, 95; Watson et al., 05]
 Information Visualization:
 Visualization of Fitness Landscape/properties/search behavior
Visualizations for Local Search
 Existing Visualizations:
 Objective Value over Time
 Fitness Distance Correlation [Jones, 95; Watson et al. 05]
 Problem Specific, e.g. TSP [Klau et al., 02]
 2-D Animation of 2 Dimensional Problem [Syrjakow & Szczerbicka, 99]
 N-to-2-Space Mapping [Kadluczka, 04]
 Search Trajectory Analysis [Lau et al., 05; Halim et al., 06a,b]
Analogy of Search Trajectory Visualization:
Finding the Highest Mountain
1. Without anchor points, the behavior of the pink
trajectory is hard to be explained.
2. Do several local search runs with different configurations,
record diverse local optima/anchor points (circled).
3. With anchor points, the behavior of the pink
trajectory is as follows: trapped in region that
contains red/blue anchor points, thus failed to visit
good solutions, the green/orange anchor points.
4. The behavior of the pale blue trajectory is as follows: after
reaching a local optima, it diversifies to another place. It
manages to reach green and orange anchor points, and thus
its performance is better than the pink trajectory in Figure 3.
Search Trajectory Visualization
In Practice
Layout the points in Abstract 2-D Space
• Points that are close in N-dimensional
space in terms of distance metric
(hamming, permutation distance, etc)
are laid out close to each other in the
abstract 2-D space and vice versa.
• This utilizes human strength in discerning
2-D spatial information.
Anchor Points are quite
close to each other.
Trapped in cycling
near Anchor Point C
Only covers regions near
Anchor Point D and E
Layout First Phase:
• The anchor points are measured with
each other using distance metric.
• The anchor points are installed greedily
in abstract 2-D space
• Re-optimize using the Spring Model
layout algorithm.
Layout Second Phase:
• Again, using Spring Model algorithm,
the points along search trajectory are
aid out in abstract 2-D space using
these anchor points as reference.
Presentation Aspects:
• Color coding is used to enhance our
understanding: blue: good, green:
medium, brown: poor anchor points.
• The search trajectory is animated
over time.
Anchor Points are quite
close to each other, but
their quality are different.
Trapped in cycling near
Anchor Point C because it is
attractive (green)
Only covers regions near Anchor
Point D and E, which are good
regions (blue and green)
Explaining Local Search Behavior
using
Viz: Local Search Visual Analysis Suite
http://www.comp.nus.edu.sg/~stevenha/viz
Note: we use the updated version of Viz for the demo.
Design Goals
 Enable algorithm designer (human) to understand LS behavior.
 Must obey Information Visualization Rules:
 Overview first, Zoom and Filter, then Details on Demand.
 Utilize human visual perception strengths.
 Interactive.
 Responsive.
 Aim for widespread use:
 Generic:
 Support various (/all) Local Searches.
 Support various (/all) Combinatorial Optimization Problems.
 Viz as Analysis Suite.
 Facilitates information sharing with other algorithm designers.
Using Viz
Viz Features (Overview)
 Multi-Source Visualizations
 Visual Comparison
 Animated Search Playback
 Information Visualization + Data Analysis
 Color and Highlighting
 Multiple Level of Details
 Viz GUI is:
 User friendly, use familiar widgets
 Interactive
 Responsive
 Customize-able
Multi-Source Visualizations
 The local search playback are viewed
from several angles:
 Generic
 Search Trajectory
 Objective Value
 Fitness Distance Correlation
 Event Bar
 Algorithm Specific
 Problem Specific
 CoordinatedVisualizations
Visual Comparison
 Human is creature of comparison.
 Actually, human is not a perfect judge.
 But, good at relative comparison when given a baseline.
 Two comparison modes:
 Juxtapose
 Superimpose
 Useful for understanding:
 Behavior: Same LS & configuration, different parts of search
 Robustness: Same LS & configuration, two runs
 Effect of Modifications: Same LS, different configurations
 Overall behavior: Two totally different LS
Animated Search Playback
 Animations
 Local Search info is too big: cannot be presented at one time.
 But rather: animated over time.
(the search trail is drawn using alpha blending).
 Local Search typically runs in thousand iterations…
 Analyzing from start to end may be overwhelming
 We use Event Bar for quick reference to interesting regions.
 Viz computes highlights/index points for generic events
(e.g. new best found solution)
Info Visualization + Data Analysis
 We combine strengths from both worlds.
 Information Visualization
 Data Analysis (Textual/Statistics)
Viz DEMO
http://www.comp.nus.edu.sg/~stevenha/viz
Viz is Useful for:
 Local Search Algorithm Analysis tool:
 Individual
 Addressing Tuning Problem
 Debugging
 Improving algorithm
 Collaborative
 Researchers can annotate and publish *.viz file for information exchange.
 Benchmarking
 Pedagogical tool:
 For Studying/Teaching Local Search Algorithms.
 Presentation tool:
 To illustrate behavior of a new Local Search Algorithm.
Summary
 We have seen that proper visualization can be used to help
explaining/understanding local search behavior.
 Some of these are hard to derive without proper visualizations.
 A necessary step for tuning/improving local search.
 To be useful, a visualization tool should utilize
human visual perception strengths.
 Viz’s capability in explaining local search is so far promising.
 Lessons for UIST:
 Regardless whether your research is on Local Search or not. Hopefully you
can get something from what I share here with my visualization tool Viz 
 Careful application and extrapolation of existing techniques in a new domain
can result in fascinating extensions. (multidisciplinary).
 Many other fields can be enhanced using proper Information Visualization
tools, developing specific visualization for these domains may be fruitful.
End of Talk
Time for Q & A
Thank you for your attention 
For further details and to download Viz, please visit:
http://www.comp.nus.edu.sg/~stevenha/viz