Using Visualization to Understand
the Behavior of Computer Systems
Robert P. Bosch Jr.
Stanford University
May 3, 2001
Motivation
• Explosion in complexity of computer systems
• Development of rich data collection tools
–
–
–
–
Complete Machine Simulation: SimOS
Software Monitoring: Performance Co-Pilot (PCP)
Firmware Instrumentation: FlashPoint
Hardware Monitoring: DCPI
• Challenge: how do we fully exploit the large,
detailed data sets these tools can generate?
2
Data Analysis Challenge
• How do we typically handle this data?
– Visual inspection of huge log files
– Summarize through statistics and aggregation
– Focus on restricted data subsets
• Alternative approach: data visualization
– Display large amounts of data at once
– Enable interactive exploration of entire data set
• Overview, zoom and filter, details-on-demand
– Use human perception to discover patterns,
trends, and interesting information
3
Computer Systems Visualization:
Existing Work
• Pedagogical examples
– Processors: DLXView, Pentium Pro Tutorial
– Memory: Cache Visualization Tool (CVT)
• Parallel systems performance
– AIMS, Pablo, Paradyn, ParaGraph, PARvis,
StormWatch, VAMPIR…
• Other examples
– Network performance, file systems, etc.
4
Computer Systems Visualization:
Existing Work
• Demonstrate potential of visualization
• Limitations
– Focused on particular system components
– Integrated with specific data collection tools
– Limited to a fixed set of visual representations
• Conclusion: rich, flexible data sources
require an equally powerful and flexible
visualization system
5
Outline
• Motivation
• The Rivet visualization environment
– Architecture
– Implementation
• Focused visualization systems
–
–
–
–
SUIF Explorer
Thor
PipeCleaner
Visible Computer
• Ad hoc analysis and visualization
• Contributions
6
Rivet Architecture: Goals
• Learn once, apply to wide range of problems
– Decouple visualization and data collection
• Interactive exploration of large data sets
– Couple visualization and analysis
• Rapid prototyping of visualizations
• Extensibility
– Allow users to add new components
7
Rivet Architecture: Approach
•
•
•
•
Identify visualization “building blocks”
Mix and match to create visualizations
Users can add new components as needed
Three basic object types:
– Data management: tuples, tables, and transforms
– Visual representation: primitives and metaphors
– Mapping from data to visual: encodings
8
Data Model
• Simplified relational model
– Tuple: collection of data fields
– Table: set of tuples with common data format
– Familiar, homogeneous model
• Load data by parsing text files
• Directly save/load tables in binary format
Procedure: Redraw()
PID:
1717
Page Faults:
129
Tuple
.
.
.
Table
9
Data Transforms
• Transforms enable users to operate on data
• Can be composed to form data networks
– Active: data changes are propagated
• Rivet includes a set of standard transforms
– Filter, sort, group, merge, join, aggregate, etc.
• Users may design and incorporate their own
– Example: clustering algorithms
.
.
.
GroupBy
Transform
..
...
...
.
10
Visual Objects and Data Mapping
• Primitives draw individual tuples
• Metaphors draw entire tables
– Draws table attributes: axes, labels, etc.
– For each tuple: compute bounding box, select primitive
• Encodings map tuple contents to visual properties
– Metaphors use spatial encodings
• Map tuple fields to bounding box parameters
– Primitives use attribute encodings
• Map tuple fields to retinal properties: color, fill pattern, size…
• Encodings encapsulate datavisual mapping
– Metaphors and primitives are data independent
11
The Encoding Process: Example
Tuple
Graph Metaphor:
Spatial Encodings
Procedure: Redraw()
PID:
1717
Page Faults:
129
Display
X
Y
S
0.5
F
C
Rectangle Primitive:
Attribute Encodings
12
Coordination and Interaction
• Implicit coordination: sharing of objects
– Examples:
• Primitives share attribute encodings: brushing
• Metaphors share primitives: common appearance
• Metaphors share spatial encodings: common axes
– Listener mechanism
• Explicit coordination: events and bindings
– Rivet objects can raise events
– User can bind actions to these events
– Example: details-on-demand
13
Rivet Implementation
• Goal: balance performance and flexibility
– Interactive visualizations of large data sets
– Rapid prototyping of visualizations
• C++ and OpenGL for performance
– Also provides platform independence
• Scripting language interfaces for flexibility
– Simplified Wrapper and Interface Generator (SWIG)
– Create visualizations by writing scripts
• Interpreter is not in the main loop
– Listener mechanism for high-frequency events
– Event bindings for low-frequency user interaction
14
Sample Rivet Script
set table [DataVector]
set parser [CSVParser -args $table]
rparse $parser poletops.csv
set primitive [GLPoint]
Import data into table
Create point primitive
rwindow .radios
rgeometry .radios W 500 H 675
rglob Graph .radios.map
.radios.map SetData $table
Create graph metaphor
set gran [expr 1.0 / 3600.0]
set lmin [$table GetMin Longitude]
set lmax [$table GetMax Longitude]
set long [QUniformRangeMap –args $lmin $lmax $gran]
.radios.map EncodeAsXPosition \
[QRangeEncoding -args $long Longitude]
Encode Longitude as X
set lmin [$table GetMin Latitude]
set lmax [$table GetMax Latitude]
set lat [QUniformRangeMap –args $lmin $lmax $gran]
.radios.map EncodeAsYPosition \
[QRangeEncoding -args $lat Latitude]
Encode Latitude as Y
set hue [list 0.0 0.5]
set sat [list 0.0 1.0]
set val [list 0.6 1.0]
set ramp [IsomorphicColorMap –args $hue $sat $val]
set rmin [$table GetMin Radios]
set rmax [$table GetMax Radios]
set radios [QNumberMap -args $Log $rmin $rmax]
$ramp SetDomainMap $radios
$primitive EncodeAsColor \
[QColorEncoding -args $ramp Radios]
Encode # Radios as Color
15
Outline
• Motivation
• The Rivet visualization environment
– Architecture
– Implementation
• Focused visualization systems
–
–
–
–
SUIF Explorer: Interactive user-directed parallelization
Thor: Detailed memory profiling on FLASH
PipeCleaner: Superscalar processor pipelines
Visible Computer: System and cluster monitoring
• Ad hoc analysis and visualization
• Contributions
16
SUIF Explorer:
Interactive Parallelization
17
SUIF Explorer: Background
• Goal: enable sequential codes to run fast on MPs
• Compiler parallelizes loops when possible
• Otherwise:
– Compiler presents loop analysis results to user
– User applies application knowledge to assist compiler
• Data source: SUIF dynamic analyzers
– Performance metrics for all loops in program
– Coverage, granularity, loop level
• Visualization
– Displays data in context of program source code
– Focuses on loops most deserving of user attention
18
SUIF Explorer: Discussion
• Benefits
– Combines data with source, instead of loop IDs
– Allows user to filter uninteresting lines of code
– Facilitates comparisons between runs
• Limitations
– Loosely coupled with rest of SUIF Explorer
– Short loops less prominent than long loops
• Add sortable table of results as a linked view
19
Thor: Detailed Memory Profiling
20
Thor: Background
•
•
•
•
Memory getting slower relative to CPU
High remote access latencies on NUMA systems
Memory system is often performance bottleneck
Data source: FlashPoint protocol on FLASH
– Collects all cache and TLB misses in firmware
– Classified as local/remote, read/write
– Attributed to CPU, procedure, data structure
• Visualization
– Stacked bar charts showing miss counts
– Collection of UI controls for configuring the display
21
Thor: Discussion
• Both post-mortem and real-time analysis
– Live connection to FLASH via socket
– Useful for multi-phase applications
• Simple, familiar visualization
• Interactive filtering, sorting, aggregation
• Drill down from overview to data of interest
22
PipeCleaner: Superscalar Processors
23
PipeCleaner: Background
• High peak performance
• Complex implementation techniques
– Speculation, multiple functional units, out-of-order
execution
• Intended to be transparent to the programmer
– True for correctness, not necessarily for performance
• Data sources: MXS and MMIX simulators
– Detailed superscalar processor pipeline models
• Visualization: three linked views
– Overview: occupancy strip charts
– Detail: animated pipeline display
– Context: program source code
24
PipeCleaner: Discussion
• Applications of PipeCleaner
–
–
–
–
–
Program development
Compiler optimizations
Hardware design
Education
Simulator development
• Possible extensions
– Other computer pipelines: graphics hardware
– Physical pipelines: assembly lines, etc.
25
Visible Computer:
System and Cluster Monitoring
26
Visible Computer: Background
• Real-time analysis of system behavior
– Observe behavior of system in its entirety
– Explore interesting phenomena in detail
• Data sources: PCP, SimOS
– Comprehensive data sources
– Collect low-level hardware events
• Cache misses, disk requests, CPU utilization, etc.
– Classify using high-level structures
• Process name, user/group, CPU mode, etc.
27
Visible Computer: Visualization
• Organizes data using nested physical hierarchy
• Provides overviews at each level of detail
– Active icons representing system components
– User defines data ranges of interest for each component
– Icons activate when components are in range
• Allows users to “remove the cover”, show next level
• Displays detailed charts on demand
– Data filtered/colored using high-level classifiers
28
Visible Computer: Discussion
• Unified interface for computer systems data
– Focus-plus-context view of system
– Physical layout, virtual classification
• Provides an overview of system behavior
• Draws attention to potential problem areas
• Suggests targets for further exploration
29
Outline
• Motivation
• The Rivet visualization environment
– Architecture
– Implementation
• Focused visualization systems
–
–
–
–
SUIF Explorer
Thor
PipeCleaner
Visible Computer
• Ad hoc analysis and visualization
• Contributions
30
Ad hoc Analysis and Visualization
• SimOS: Complete machine simulator
– Full non-intrusive access to HW, OS, SW state
– Flexible data collection mechanism
– Deterministic execution
• Combine with Rivet
– Flexible data import mechanism
– Rapid prototyping of visualizations
• Result: powerful analysis framework
31
Simulation & Visualization Cycle
Change HW model or SW
Configure simulated
machine and software
Change annotations
Perform simulation
Change visualization
Visualize results
32
Case Study: Argus
• Parallel, multithreaded graphics library
– NURBS rendering application
– Implemented using fork & shared memory region
• Performance limitations on SGI Origin
– Linear speedup up to 26 processors
– Rapid performance falloff beyond 26 CPUs
• Imported into SimOS environment
– Same performance characteristics observed
• Expectation: memory system is the problem
33
Memory Visualization
34
Memory Visualization
35
1. Histograms of memory stall time vs. physical/virtual address
Memory Visualization
2. Memory stall time incurred by each line of source code
36
Memory Visualization
3. Local/remote stall time and idle time for each process
37
Memory Visualization
Very
Large
littleamounts
memoryofstall
unexpected
time in process
idle time
view
38
Visualization of Process Data
Processes go idle in kernel pfault/vfault calls
39
Process Scheduling & Kernel Locks
CPU scheduling
kernel lock
If If
the
the
lock
lock
is is
unavailable,
available, itthe
is immediately
process is descheduled
granted
40
Process Scheduling & Kernel Locks
process is descheduled
kernel lock is held
Kernel lock is heavily contended
41
Results With Processes Pinned
Lots of idle time and kernel lock contention still remain
42
Results Using sproc
• Minimal kernel time
• No idle time at all
• Completes nearly twice
as fast as original version
• 95% parallel efficiency
43
Argus: Conclusion
• Five simulation & visualization iterations
• Initial suspicions were totally incorrect
• Flexibility of Rivet & SimOS enabled us to
follow leads
• Visualizations led us to the bottleneck
– Single scheduling event out of over 50,000 events
44
Contributions
• Rivet computer systems visualization environment
– Rapid prototyping support
• Quick ‘rough cut’ visualization of data
• Incremental development of sophisticated data displays
– Modular architecture
• Mix and match basic building blocks to create visualizations
• Enables coordination through object sharing
– Data transforms as first-class citizens in viz environment
• Unifies the analysis and visualization process
• Collection of computer systems visualizations
– Emphasize interactive data exploration
– Use relatively conventional visual representations
– Provide extensive support for filtering, sorting, brushing
45
Acknowledgments
• Orals committee: Joel Ferziger, Mendel
Rosenblum, Pat Hanrahan, Mark Horowitz,
Monica Lam
• Mendel
• Visualization collaborators
–
–
–
–
–
PipeCleaner: Donald Knuth
Visible Computer: John Gerth
Argus: Gordon Stoll
Thor: Jeff Gibson
SUIF Explorer: Shih-Wei Liao
46
Acknowledgments
• Groupmates: Rivet, SimOS, FLASH, Graphics
• Studio 354
– Steve Herrod and John Heinlein
– The foosball table
– Chris Stolte and Diane Tang
• Staff:
– John, Charlie, Thoi, Kevin
– Ada, Heather, Chris, …
• Funding agencies: ONR, [D?]ARPA, ASCI
• Robert Bosch Corporation
47
Acknowledgments
•
•
•
•
Rains 7A: Steve, Hoa, Marco
Friends
Family
Ming
48