Evaluating Modern Graphics – New Standards or Old?
Hilary Green
Macquarie University, Sydney
Summary
This paper investigates whether there is a need for new criteria for the
evaluation of new data display methods. Two modern graphical display
methods are examined using well documented guidelines from the literature of statistical graphics. It appears that established criteria, while still
valid, do not extend to important features of some modern visual displays.
Keywords: dynamic graphs, information visualisation, graphical perception.
Introduction
Due to rapid advances in computer technology and to the increasing ability
to obtain and display some datasets on demand, development of new
graphics is inevitable. In particular, the use of colour and movement are
fascinating features of modern graphs. Their dynamic features and ability
to offer instantaneous feedback make them invaluable as a tool for data
mining or prospecting. Typically, the engaged user needs to focus on the
task of discovery without the distraction of superfluous detail. On the
other hand, the user will often wish to capture pertinent findings for reexamination at some later stage. And still later, the user may wish to
communicate discoveries to an audience, thus requiring a static graphical
display. Can we evaluate modern graphics using historical criteria - or is
it necessary to develop new criteria? To address this issue, two examples
of modern graphics, a mosaic plot and a Grapharti are presented and examined in the light of historic criteria for good graphics. I conclude that the
historic criteria remain valid; however, modern graphics require additional
standards for the evaluation of features that were not formerly present.
968
Two Examples of New Graphics
An example of a recent mosaic plot, originally developed by Hartigan &
Kleiner in 1981, is shown in Figure 1. Mosaic displays show the frequencies in n-way contingency tables by nested rectangular regions whose areas
are proportional to cell frequencies. Interactive mosaic plots permit the
addition and deletion of variables as well as the ordering of the variables
and choice of colours. Figure 1 shows there is an association between sex,
hair and eye colour: females with brown hair are most likely to have brown
eyes, while males are unlikely to have both brown hair and brown eyes.
Brown
Standardized
residuals:
Hazel
Green Blue
<4 4:2 2:0 0:2 2:4 >4
Model (Hair) (Eye) (Sex)
Male Female
Black
Brown
Red
Blond
Fig. 1: Mosaic display for hair color, eye color, and sex.
(from http://www.math.yorku.ca/SCS/mosaics.html).
Grapharti, is designed to display data from surveys, opinion polls,
financial data - wherever there is a long list of items each associated with
at least two quantitative variables. The method is demonstrated using data
from daily opinion polls carried out on the Australian ninemsn website
(ninemsn.com.au). The dates, questions and the numbers of yes and no responses are stored in a database and retrieved from a web page in a format
as shown in the Grapharti of Figure 2. A rectangle represents each question; the width is determined by the number of responses and the colour by
the proportion of yes responses. These images are tiled alongside each
other. By rolling the mouse over the image the user extracts further infor-
969
mation. The user has the opportunity to explore the data by variable selection, grouping, ordering and filtration. “The spatial representation may be
visually browsed and analyzed in ways that avoid language processing and
thus reduce the analyst’s mental workload”. In the example in Figure 2,
the mouse was over the tile representing the outcome of the poll of the 16th
October, 2002, showing a large and positive response.
Both these graphs summarize proportions. The proportions of each
of gender, eye colour and hair colour are simultaneously depicted on the
mosaic plot in Figure 1. The mosaic plot reveals information about a single
sample. In Grapharti each tile reveals information from different sources.
Both these displays are fairly recent innovations; mosaic plots in 1981,
Grapharti in 2005. However, the mosaic plot considered in Figure 1 includes two enhancements designed to facilitate the perception of relations
among the variables: the shading of each tile and the re-ordering of the
categories [1].
Fig. 2: Screen shot of Grapharti from ninemsn daily news polls.
Background
The search for the understanding of the processes that produce a
successful graphic has been taken up by many individuals from a variety of
fields, including statisticians, psychologists and computer scientists. In the
visualization of information, there must be two component processes. The
first is that the brain translates visual representations to arrive at cognition,
and the second, that visual representations assist in the amplification of
cognition. The first of these processes is sometimes is called perceptual organization – “the issue of how the brain really makes us see, in the sense of
970
how does it produce conscious experience, remains a mystery”. The approach to the second process, where visual representations assist in the
amplification of cognition, relies on the fact that visual representations are
tools. These tools are most useful because they “facilitate the comprehension of huge amounts of data, allow the perception of emergent properties
that were not anticipated, enable problems with the data to become apparent, facilitate understanding of both large scale and small scale features
and patterns in the data and facilitate hypothesis formation”. There has
been much research into discovering how the tools work and most importantly, which tools are effective: some of this research has been carried out
by statisticians. The works of Jaques Bertin, John Tukey, Edward Tufte
and William Cleveland are seminal and together they have compiled a set
of guidelines by which we can judge the efficacy of statistical graphics. In
the following section, mosaic plots and Grapharti are investigated within
the framework of these guidelines.
Evaluation/Assessing New Graphics
William Playfair (1759–1823) is often regarded as the progenitor of modern statistical graphics, The French king, Louis VI said of Playfair’s
graphs that “they spoke all languages and were very clear and easily understood”. His appraisal of Playfair’s graphs initiated a recurring theme;
graphs must speak a universal language, they “must have immediate and
strong impact and show phenomena not numbers”.
Applying
these
notions to our two examples, it seems relatively easy to discover the
method of reading mosaic plots and Grapharti. However, deciphering the
meaning of the graphs may take a little more concentration. The use of
colour in the mosaic graph refers to scaled residuals, a concept that needs
to be learned, and the rectangles in the mosaic plots vary in both width and
length, making comparison of areas difficult. With a Grapharti, we need to
explore the detail before the essence can be perceived.
Graphs are encoded with objects that are perceived at the preattentional level such as points, lines and shapes, so it is not surprising that
they can be deciphered across all cultures and all languages. Cleveland and
McGill have determined, experimentally, a ranking of elements that assist
with processing elementary tasks and thus facilitate accurate decoding of
graphs. These are, from best to worst; position on a common scale, position on identical non-aligned scales, length, direction, angle, area, volume,
curvature, shading, and colour saturation. These rankings, though not rigid,
provide a basis on which to judge graphs. The mosaic plot employs area
971
and colour saturation, while Grapharti makes use of length and colour saturation. Both the displays feature lower level pre-attentive elements.
A graphic should “serve a reasonably clear purpose; description,
exploration, tabulation or decoration”. The main purpose of the mosaic
plot is description, whereas that of a Grapharti is to support exploration.
Both graphs use rectangles to depict quantitative information: the size,
shape, colour and position of each rectangle show variation in data. They
are examples of Tufte’s small multiples: “Information slices positioned
within the eyespan, … Constancy of design puts the emphasis on changes
in data”. Small multiples are effective because they invite comparison and
can show shifts in relationships. The mosaic in Figure 1 shows, for example, that most people had brown hair (width of the horizontal brown category). By using small multiples, the mosaic plot is able to achieve its purpose - to describe the data and facilitate comparisons. The main purpose
of Grapharti is to encourage exploration of the data, or data mining. By lining up tiles in date or colour order the “multiples create visual lists …
helping viewers to analyze, compare, differentiate, decide”, and thus the
user is encouraged to discover what sort of questions provoke a large number of responses, and how strongly respondents feel about certain issues.
Tukey also maintained that we should be clear about two distinct functions
of visual displays – ‘prospecting’ and ‘transference’, and they must be
used quite differently in each situation. Prospecting, or searching the data
for phenomena, would typically require many displays. The transference of
“a recognition of the appearances that indicate the phenomena that deserve
report”, would require a clearly labeled static graph. Thus the function of
the mosaic plot is transference, while that of Grapharti is prospecting. It
follows that dynamic graphs are more useful for prospecting and static
graphs for transference.
Other salient features of statistical graphs are the use of gridlines,
tick marks, legends and labels. Whilst neither of the examples uses gridlines or tick marks, each employs separators; white spaces in the mosaic
plots separate groups within categories and fine muted borders separate
tiles in Grapharti. These are necessary to distinguish different events by
enhancing boundary detection, and so do not constitute “chart junk” nor
unnecessary “data ink”. Comparisons of scale are facilitated by the dimensions of the rectangles. The legend in the mosaic plot clearly provides an
ordinal value to the colour scheme, and so enables the reader to interpret
this variable in the context of the data. In the same way, a simple legend
for Grapharti should be provided. In each graph, the necessity of constantly referring to a legend to decode the colours is eliminated by the use
972
of sensible colour coding. The categories in the mosaic plot have been
clearly labeled. In Figure 2, labeling is not required as the tiles are ordered
horizontally and wrapped within the data frame; the title provides adequate
description of what is shown. However, Grapharti can be organized to
display more complex results, thus necessitating the use of axes and labels.
Tufte suggests that the display should “reflect a balance, a proportion, a sense of relevant scale”. Balance has been achieved in the mosaic
plot in Figure 1, by the ordering of the hair/eye categories, so that the most
prominent effects are displayed in diagonally opposite extremes of the display. This has the effect of highlighting one of the main conclusions of the
investigation– that the three variables are not independent. From an aesthetic point of view as well, there is a sense of balance in the variety and
intensity of colours, and the sizes of the rectangles. In the same way, reordering the tiles in Grapharti, Figure 2, has produced a sense of balance
suggesting that the questions have evoked a variable proportion of yes voters. Data density is defined as the number of data entries per square centimeter. Grapharti is clearly more “data dense” than the mosaic plot (19.31
to 1.33) but far less meaningful in its static form.
Further guidelines needed
The mosaic plot appears to satisfy most criteria for an effective graphic
while a static Grapharti reveals little meaningful information. However, as
the purpose of Grapharti is to facilitate data exploration, the use of dynamic features is an essential component of this graphic. A list of guidelines for dynamic graphs would be useful. Statisticians have contributed to
this burgeoning area, and, again, they propose that common sense should
prevail. “For a dynamic method to be useful the visual and manual tasks
must be easy to perform”. For dynamic graphs, “The basic ideas – impact,
easy comparison, balanced emphasis, etc, can and still should be the
same”. “Dynamic graphics can be viewed as methods that allow us to
move rapidly through a long sequence of static views until the desired or
static view is found, having found it we can present it to others”.
To discover some guidelines for the display of dynamic graphs, we can investigate the research of Ben Shneiderman relating to human-computer interaction and user interface. In 1997, he proposed “well designed, effective
computer systems generate positive feelings of success, competence, mastery, and clarity” - “users have to be seen as heroes”. A generally accepted
Visual Information Seeking strategy is: “Overview first, zoom and filter,
then details-on-demand”. A particularly relevant goal for designers of in-
973
terfaces that enable prospecting or visual information seeking is to “offer
usability plus reliability. Guidelines to achieve this goal are, briefly; to
strive for consistency, cater to universal usability, offer informative feedback, permit easy reversal of actions, and reduce short-term memory load.
I investigate Grapharti in the light of these guidelines.
Consistency is achieved by employing the same design features
such as colour scheme for each display of a Grapharti image. On the page
introducing Grapharti, the user is able to select some or all of the data to
view in table format or as Grapharti, and sort in various ways. The use of
familiar features such as buttons ensures the user is not distracted by the
interface. Such features have become widely known and intuitive for users
and possibly speak a universal language alluded to previously. Grapharti
provides informative feedback by the information yielded on rolling the
cursor over the tiles. This feature is further enhanced by being able to click
on tiles to retrieve further information. However, in the format shown in
this paper, Grapharti goes no further in assisting with the task of prospecting. To successfully use a dynamic visualization for prospecting it is essential that a user have the ability save useful instances that may provide insight. “Users should be able to bundle search sessions as well as save
individual portions of a given search session, and flexibly access and modify each”. Incorporating such features into Grapharti would further reduce
short term memory load. In particular, a search option and a shopping
basket facility would help generate positive feelings of success and clarity
in users.
Conclusion
For graphs that are generated dynamically, then, it is apparent that some
further guidelines need to be substantiated in the literature. Some graphs,
like the mosaic plot shown in Figure 1, survive extremely well in their
static form. This is largely because of adherence to details such as careful
labelling and appropriate legends that are shown at each rendition. The historic criteria referred to previously seem to suffice for evaluating such
graphs. These graphs are extremely useful for transference of information.
Dynamic graphs are far less informative in their static form but function
well as the tools for data visualization. For these graphs, well researched
and functional guidelines can be gathered from others, particularly from
computer scientists. We argue that dynamic graphs need not adhere to the
traditional guidelines applicable to static graphics, particularly when the
manipulation of the interactive features on the graph is intuitive. However,
the user of dynamic graphs needs the ability to reorganise, retrieve and
974
store potentially useful discoveries. It is reasonable to conclude that static
graphics may need one set of criteria; while dynamic graphics may need
another (closely related but distinct) set!
Reference List
Cleveland, W. S. and M. E. McGill (1988). Dynamic Graphics for
Statistics. FL, USA, CRC Press, Inc.
Cleveland, W. S. and R. McGill (1985). "Graphical Perception and
Graphical Methods for Analyzing Scientific Data." Science,
New Series 229(4716): 828-833.
Friendly, M. (1994). "Mosaic displays for n-way contingency tables."Journal of the American Statistical Association 89: 190200.
Green, H. (2005). Grapharti. Statistics Education and the Communication of Statistics, Sydney, Australia.
Hearst, M. (1999). User Interfaces and Visualization. Modern Information Retrieval. R. Baeza-Yates and B. Ribeiro-Neto. New
York, Addison Wesley Longman Publishing Co. Inc.
McLaughlin, J. (2005). Looking for the Mind's Eye. 2005.
Shneiderman, B. (1997). "Designing Information-Abundant Websites: Issues and Recommendations." International Journal of
Human-Computer Studies (1997) 47(1): 5-29.
Tufte, E. R. (1983). The visual display of quantitative information. Cheshire,Conn., Graphics Press.
Tufte, E. R. (1997). Visual explanations : images and quantities,
evidence and narrative. Cheshire, Conn., Graphics Press.
Tukey, J. (1990). "Data-Based Graphics: Visual Display in the
Decades to Come." Statistical Science 5(3): 327-339.
Wainer, H. (2004). Graphic Discovery: A Trout in the Milk and
Other Visual Adventures. Princeton, New Jersey, Princeton
University Press.
Ware, C. (2004). Information visualization : perception for design. San Francisco, Morgan Kaufman.
Wise, J. A., Thomas, J.J., Pennock, K., Lantrip, D., Pottier, M.,
Schur, A., & Crow, V. (1995). Visualizing the Non-Visual:
Spatial analysis and interaction with information from text
documents. Readings in Information Visualization: using vision to think. S. K. M. S.K. Card, and B. Shneiderman. San
Francisco, Calif., Morgan Kaufmann Publishers.