>> Steven Drucker: So a brief introduction. I think I first met you when you were
interning here with George Robertson. So that was 1994?
>> Tamara Munzner: 1998.
>> Steven Drucker: Yeah, so it’s been awhile. I wasn’t in visualization at the time, but
now I come here and I think you are one of the premier professors teaching information
visualization today. She just has a new book out, which I’ve had a chance to read and is
fabulous. She is here today and [indiscernible] tomorrow. So please, if you have any
interest in speaking with her let me know. Her schedule has been filling up pretty
quickly. So please welcome Tamara.
>> Tamara Munzner: Thank you. All right. So yes, thanks for the kind introduction. It
has been awhile since I have been back at Microsoft research. So I have been working on
this book project for quite some time. It turns out books take awhile, who knew? So
what I am going to do today is not try and cover the entire book, which is not possible in
an hour no matter how fast I were to talk. I am just going to hit a few key ideas, give a
big picture and particularly talk about ways I try and frame the field. It might be a little
new or different than what you might have seen before, so trying to focus on the more
interesting bits.
So to start out with let me talk about my own definition of visualization and some of its
implications. To save myself a few syllables over the hour I will call visualization vis,
because that’s the word I will use quite a bit. So computer based vis systems provide
visual representation of datasets designed to help people carry out tasks more effectively.
Now why did I pick this particular definition? What are the implications of it? So for
one thing there is this idea that we’ve got datasets and people together in the same
sentence. And in particular that vis is suitable when there’s some need to augment the
humans instead of replacing them with something purely computational.
So in what context does this happen? So I argue there’s a lot of situations we don’t need
vis. We don’t need vis if we’ve got something completely automatic that we trust. So
the problem is there are a lot of situations where you’ve got an analysis problem that’s
not well specified. There might be one of dozens, or hundreds, or thousands, or tens of
thousands of questions that you might want to be asking and you don’t know a priori
which they are. So that’s a situation where vis is typically a lot more suitable.
Now there’s actually a spectrum of possibilities and it could be that many of you might
think really on only one flight of that spectrum. So what I’ll call the classic use case for
vis is you intend it to be used in the long-term indefinitely by some group of end users.
So that classic case is something like exploratory analysis often with something like
scientific data, but that’s only one use case. So that kind of discovery use case is very
common. There is also not just discovery of the unknown, but also presentation of the
known. That might be anything from things like graphics in the New York Times to any
other situation in which you are not trying to discover something new with the vis, you
are trying to present something you know to some audience of other people.
But, there’s also these three other use cases that I want to make sure people are thinking
about. These are all in some sense visualization trying to work itself out of a job. So one
would be you are using vis with the intent of later developing an automatic solution, but
you need to understand it more so you want to have some exploratory analysis towards
the goal of model building. Another might be that you’ve already started building
something automatic and you could use the vis to try to refine it, or debug it, or figure out
the right set of parameters.
So often in that case the vis is something for the developers of the algorithm, not for the
target end users or it might be that you’ve even finished building the automatic system,
but you’ve got to get the people that are going to use it to actually trust that it’s right. So
before you actually let it run unmonitored you might have some situation where again the
visualization is trying to bridge the gap between the old practice and something that’s
new and fully automatic.
Now here’s another interesting word in that sentence “visual representation”. So why
what is the goal of having this visual representation, which is something external to your
own head? So the idea with this always is that anytime we can replace cognitive with
something perceptual then we have a win where we’ve suddenly freed up internal
resources for people to do something higher level. So here’s a very classic example, we
have a table full of numbers, in this case these numbers are gene expression levels that
are measured after applying some experimental drugs.
Now if I ask you things like, “find the minimum value,” “find the maximum value,” even
those really simple sounding tasks what do you have to do? You have to read a number,
you have to remember it in your head, you have to read the next number and compare it.
And this is trivial if you’ve got two numbers, but once you start getting a few dozen or I
ask you more complex questions like, “Well what’s the distribution not just min and
max?” This actually starts using significant amounts of resources internally. If in
contrast I take these number and in this case I’m coding them according them to a
diverging color scale where you are now able to use your perceptual system to read off
things like low values, high values and C distributions quite quickly without using a
bunch of internal resources, the idea being that now you can actually ask higher level
questions than just what’s the range of the distribution.
Now one question is: Why is it useful to actually represent all that data? After all we’ve
got statistics, aren’t we done, and are we there yet? And the answer typically is that
statics is always necessary, but in many cases not sufficient. Sometimes the loss of
information that you have in a summary matters and the details are something that it
would help if the person is able to see. So part of what you want to do is confirm what
you expected to see in the summary, but maybe look for unexpected patterns or
particularly back to this idea of model building. You wan to gage whether or not the
particular sophistical model you are using is actually a good fit with your data. So now in
this room there’s going to be some ringers, but I’m curious who here has heard of
Anscombe’s Quartet? So many, but not all the people in the room have.
So this is this great example that was designed by the statistician Anscombe in the early
70s where we have 4 datasets and they have identical mean and variance, x and y and x/y
correlation. So from this particular flavor of summary alone you can’t tell the difference.
The instant I draw you a picture of course humans can tell the difference. We see a very
classic distribution that’s probably what you expected. We see something that’s clearly
non-linear verses linear. You know we see something where that correlation line is a bit
off and then of course we see this nice terrible example where it’s deeply misleading.
This of course was a synthetic dataset that was designed to mislead in that case, but the
key thing here is you saw these patterns very quickly and this is 20 data points. What
happens when we have thousands or millions of data points? What happens when we
have dynamic data that’s changing over time? It just get’s worse, so this is the opposite
of it get’s better. There are a lot of situations where we really do want to show people a
lot of the details and that is where visualization shines.
Now there’s another interesting idea which is that there is actually some resource
limitations of three very different flavors and computers and humans and displays all
have their limits. Now for those of you who came out of computer science the idea that
there are computational limits that we care about like CPU time or system memory this is
obvious and we are used to thinking about them as computer scientists. But, there are
also very different limits, even though some of these words are the same, for humans.
And what it means to have limits of human memory are quite different than the limits of
computer memory, certainly [indiscernible] is not going to help us, but more to the point
it’s a very different model where we have particularly for things like short-term working
memory some really strict limits on what we can do and particularly what we can attend
to at once.
Now there’s a third limit that people might not have thought about quite as much, which
is this idea that there is limits on the displays. So keep in mind that display technology
has gotten better at a far slower pace than computer technology. So one of the most
precious things you have are pixels. It’s what you run out of even before human attention
you start running out of pixels on your display. So it’s useful to think about those as a
constraint in your situation as a designer. And one thing that we struggled a lot with is
this tradeoff of information density. When are you encoding information verses having
white space that’s not telling you anything and the reason it’s tricky is of course it’s a
trade off. You don’t simply want to put everything on the screen at once all on top of
each other, that’s too much clutter. But, if you spread things out too much then it takes
huge amounts of navigation and zooming to even see anything at all and it’s hard to see
details in the overview at the same time. So this idea that there is a tradeoff of
information density is one useful thing to think about.
Now finally this definition has words like tasks and effectiveness in this. This should
immediately make you wonder: what does effectiveness mean? Is there an operational
definition of that? There is an idea that a lot of people have, which is that vis is intended
to let you do new things that you could never do before. That’s definitely one of the use
cases of it, but it’s not the only use case. Perhaps the even more common case is to speed
up something that people are doing already, but to do it dramatically faster. So there is
both novel and fast. Now what makes life both terrible and good is that this is hard. It
turns out that most of the possible designs you could have for vis system don’t work; they
are ineffective.
So one of the arguments in my book of why I’m presenting things the way I do is that I
want to increase the chances that people do good designs by having them think
systematically about the entire space of possible designs. It’s a large space; it’s easy if
you don’t know much about vis to get stuck in a very small part of that. And as I will be
talking this idea is that well how would you know what to do? You are constrained by
the tasks in addition to your data. So it’s not totally surprising for people outside the field
that your dataset has got to dictate something about how you show it, but keep in min that
there are many, many possible pictures of the same dataset and the task of what the user
is trying to do is a way to actually try to constrain your choices as a designer, because as
we’ll talk about some today, representations of data don’t all server the same tasks
equally. Some are better for some tasks and some for others and likewise for the data.
So the book is built around an analysis framework that thinks about 4 different levels and
3 different questions. So let me give you an overview of that. So in what I call the
nested model of thinking about visualization design I’m actually splitting things up into
levels to help us think about them separately. So at what I think of as the very top level,
the outside level is a domain situation. There are some particular people you might have
in mind to use the system and they have characteristics. They have maybe particular
domain knowledge. They might be physicists, they might be chemists, they might be in
finance or they will have particular data. So there’s a lot of questions you can ask about
who is trying to use this and in some sense this is a very classic HCI question. Who are
you designing for?
Then there’s something that’s really important and perhaps far too easy to ignore, which
is the idea that we need to abstract from some domain problem in domain language to
something that we can actually reason about cross domain. So this idea that we need
abstraction’s where we translate from the domain to generic is a really, really key
question. I’m going to focus on two different questions: One is what is being shown? So
I’m going to call that the data abstraction and also why is it being shown to the user? I’m
going to think of that as the task abstraction. Now this is a very interesting terrain
because first of all there’s this question of translation to something generic and a lot of
this talk will be the building blocks of the generic vocabulary for thinking about data and
tasks.
But, in addition what’s particularly interesting about the data abstraction is you might do
something different than what you are given. The easy case in visualization is someone
hands you a dataset and you draw a picture of that. The interesting case, which almost
always happens with large complex real world data is you are given a dataset and then
you do some series of transformations, such that you derive new data so that when you
draw the picture of that derived data you actually help them with what they had in mind,
so this idea that your data abstraction is actually a design choice, not just what you’re
given is one of the really interesting and hard parts. So really thinking about this
abstraction layer explicitly as its own thing is something that I argue is a really good idea
and that we’re thinking about the what and why at that level.
Now finally to move on to what you might have thought about as central to vis I’m
calling this an idiom. A lot of people call it technique or a method. In particular it’s
“How is it shown?” So how do we actually make these interactive visual
representations? And I’m going to split that into what I’m going to call visual encoding:
How is it that we draw these pictures? And interaction which is: How do you manipulate
these pictures? For some idioms these are actually very intertwined and you can’t easily
separate them, but some fall more on one side or on the other. So I’ll talk some about
visual encoding and interaction.
And one of the reasons I focused on this question of idiom as it’s own level is I want to
separate out this question of designing how you draw and interact with the picture from
the algorithm level, which is how do you instantiate that efficiently in some
computational format? You can have different possible algorithms for the same idioms.
So it’s useful to distinguish between them. One thing about these four different levels is
that anything you figure out at one level above actually cascades down. So decisions you
make about abstractions cascade into idioms. Once you’ve specified your idiom well that
dictates what your algorithm needs to do. So this idea of these nested dependencies is
important.
Now one of the reasons that I find this model an interesting age for thinking is it helps
you answer the question of: Did it work? How do you know if your design has actually
succeeded? The interesting thing is that vis is this very highly interdisciplinary field
where you need quite different methods to answer the question of: Did I help at every one
of these levels? So let’s actually start with that central level, the algorithm level. Well
that’s something that as computer scientists we learn about. How do you run a
computational benchmark? It’s interesting and tricky and complexity analysis for those
who are more theoretically inclined. So the methods of computer science serve us very
well at that inner most box of algorithm. But, there are a lot of questions that we can’t
answer that way.
So questions like: Do I actually have a way to draw this picture that works with human
brains and processing? That tends to be something where we often want methods more
out of say cognitive psychology. If you’re actually running a controlled experiment in a
lab with a stopwatch measuring human performance there’s a lot of trickiness to that
experimental design that you are not taught in a typical CS class. So a lot of issues about:
How do you do lab study, controlled experiments? Those are methods we often get out
of cognitive psychology.
Then how do you think about whether or not this thing that you’ve designed is even
reasonable? So often there are a lot of design guidelines for these. So there’s a lot of
methodology out of design, generation of multiple alternatives, trying to justify some of
your choices, the idea that there’s not a single correct perfect answer, but there are many
possible good answers. There are a giant number of terrible answers and distinguishing
between those is interesting, but actually there’s even more.
So that whole question of the domain situation and the abstraction of translating from a
domain problem into something generic. How do you know if you’ve got that right?
That’s an interesting question that is not well served by controlled experiments in a lab.
This is something where you actually want to get methods less out of cognitive
psychology and much more out of domains like anthropology and ethnography, often as
filtered through HCI. But, often much more qualitative methods where you actually talk
to people, that’s necessary, but absolutely not sufficient.
You often have to observe them. You can see what happens in a field study where you
deploy your system to people and see whether what they can do changes and try to
understand how that changes. That’s typically not something where you’re looking for
statistical significance, but something where you’re doing a very different
methodological approach of trying to understand what’s now possible or what is now
sped up, but not so much with a stopwatch.
Now what’s important about these divisions is one of the big areas of weak work in
visualization is when people try to validate choices at one level with methods appropriate
for another level. So in particular if you are trying to convince me that people can use an
idiom well you are not going to convince me of that by telling me a CPU benchmark
about how fast your algorithm was, because that’s a different level of design. And
likewise I mentioned this idea that it’s pretty hard to figure out the abstraction stuff with a
user experiment and why is that?
In a controlled experiment in a lab you tell the user what to do. If you are trying to figure
out whether you are understanding what users do is correct, well you’re not going to
succeed with that. So this approach to thinking about validation methods at each level is
something I’ve found has really helped my own thinking and also helped me understand
different aspects of this design. Let me emphasize one thing, this is not something where
you start at the top, go all the way down to the bottom and declare victory. It’s very,
very, very iterative as with many design issues.
Now let me address this question of: So why am I talking about analyzing visualization
systems so much? It’s in the title of the book, why is this? And I argue that analysis is a
way to try to impose a structure on a very, very large design space. So it provides some
kind of scaffolding to help people think in a systematic way of some of these design
options. My argument is that analyzing existing systems can be a useful stepping stone
for a designer who wants to design a new one because it get’s them to think about the
design space more generally.
So let me actually give you a quick preview of this before I talk about the framework
much at all. Let me show you can example of comparing two systems which in some
ways are very similar and in other ways are very different. If we just look at the pixels on
the screen they are doing pretty different things. So what’s really going on here? Here
are two systems for looking at large trees. SpaceTree is some work out of Maryland.
TreeJuxtaposer is some work out of our own lab.
Now if I use this why, what and how analysis system that I talked about these questions
of: What are we doing with the data, why is the user doing it with a task and how are we
actually doing the idiom? Well from the “what” point of view they are both looking at
tree data. I’m positing that users are doing the same task in both situations and that task
is to try to find a path between two nodes. So out of all the possible paths in the tree what
is the one that we are looking for and then that question of how? Now notice how both
SpaceTree and TreeJextaposer are somehow visually encoding that data. There is a
picture that we are drawing. That’s pretty pervasive in all vis systems is that we’re
visually encoding. They both have an interaction technique that involves some flavor of
navigating. You change over time as you zoom in.
They both have this idea that you can select an object, but then there’s a place where they
differ. With SpaceTree once you select something much of the rest of the tree is actually
filtered out and parts are aggregated together. That’s actually these aggregated things
where depending on what you’ve chosen you see a lot of the rest of the tree in a very
compact, simplified cartoon style version and TreeJuxtaposer does something very
different. When you select something with TreeJuxtaposer what it does is it changes the
spatial arrangement of that layout. It’s actually stretching and squishing. So it’s
stretching out part of it and compressing the rest. So these are different approaches at a
high level arranging in space verses aggregating and changing the visual representation to
this same question. So a lot of my arguments with the analysis framework is to help us
think in this quite systematic way about systems.
So what is this framework I’m talking about? Well, there’s a lot of stuff here. So let’s
start with this question of what? What are some ways we can think about our data in a
generic way that’s not tied to a particular domain? Now I mentioned that there’s a lot of
book and a small number of minutes in this talk so I am not going to slog through every
detail in all of these diagrams. I will be showing you a lot of big picture diagrams to give
you flavor. I’m just going to pick a few things out of this to give you a sense of flavor of
this kind of analysis. So in particular I’m going to talk about this idea of data types and
dataset types.
So I argue that in visualization there are really 3 main types of data we have to worry
about and I’m going to start out with tables of data, that’s a common case right. You’ve
got a spreadsheet, it’s in Excel and what have you got in it? The vocabulary I’m going to
use to talk about this is that in the really simple, nice, easy case where you’ve got a flat
table your items are your rows, your attributes are your columns and you can think of
having some value in there. Now of course there can be more complex and interesting
multidimensional tables. As a computer scientist think about this as an array with
multiple indicies or keys and looking at interesting slices of those tables or roll ups is
where a lot of the action is.
But, there’s something I want to have you think about first which is: What’s happening
with these attributes? What is the type of data that you’ve got? Is it categorical, so is it
something where you can tell the difference between things? There are apples, oranges
and bananas, but that’s really all you know or is there some intrinsic ordering of your
data? The classic case is you’ve actually got something that’s quantitative where you’ve
got the weights of things like your cats. Your cat could be 20 pounds, another cat could
be 15 pounds and you can do arithmetic and say there is a 5 pound difference between
those cats and that is a meaningful statement.
There’s also orderings where you can’t do that kind of quantitative analysis on it. My
favorite example is T-Shirts. It is not defined what a large T-Shirt minus a small T-Shirt
means and yet everybody will put the medium T-Shirt in between the small and the large.
So there is an ordering, but you can’t actually do quantitative mathematics on it. So this
question of do you have an ordered data attribute or something that’s more of a category
we’ll come back to very shortly as being a really central question.
But tables are not the only kind of data. So another interesting thing we can think about
explicitly is what if you’ve got a network? What if your items actually have some
connections between them? In this case we often call the items nodes and we call these
links between them links or edges. Sometimes these networks are called graphs. There’s
a field of graph drawing in visualization. I am going to avoid the word graphs here and
mostly try and use the word network. There is also a third type of data and that data is
where you have spatial information that’s intrinsic to your dataset. So this idea that you
have tables, or networks or spatial data is really what I think of as this holy trinity of the
three data types in vis.
Now there are flavors of spatial data. One is that you’ve actually got continuous data that
you’re sampling on a regular or an irregular grid. So if we think about this room for
example we could think about sampling in many places in 3 dimensional spaces to get
things like maybe the airflow, the temperature and so suddenly we are dealing with some
of the interesting mathematics of how do you sample continuous spatial data and
interpolate?
So there’s a lot of interesting implications once you start being in a regime where you are
thinking about continuous data that you have samples of rather than more discrete data
and of course you could represent spatial fields in a table, but I really want people to
think about when is data spatial? There’s even the case of where you simply have shape
information. There is geometric structure and that might be points, or lines, or surfaces,
or volumes. So this question of tables, verses networks, verses spatial data is the way I
think of fundamental division.
>>: Is it a specific attribute then you were thinking?
>> Tamara Munzner: Well so this get’s into the stuff, there’s more complex stuff in the
book that I wasn’t going to go into deeply, but let me mention briefly. So the way I think
of temporal data is –. I wonder if I can skip forward.
>>: [indiscernible].
>> Tamara Munzner: The short answer is time could be a key to your data. You can
think of time as actually being one of the ways that you are indexing your data or time
could be more of a value. So you might have something which is, “I took this sample on
Tuesday,” that’s time as a key or you might say, “There was a horse race and this race
lasted 47 minutes,” but that’s not something you are really using as a key, that’s more of
a value. So time alone doesn’t tell you whether the semantics are key or value semantics.
And there are certainly a lot of datasets where you actually have both space and time, but
there’s a lot of non-spatial data where you also have time. So in some sense I think of
time as one of the keys that you could have, but in this kind of super high level picture
it’s not quite first class in the sense that it can be attached, it can be an attribute type for
any of these. So it’s important, but it’s not a way to distinguish between. If we have lots
of time at the end we can go deeper, but I’ll keep going for now.
So that was just a few little flavors of this question of what do we have? What are these
building blocks of abstract data? Now I’m going to spend a little bit more time on the
why. This question of what is an abstract vocabulary of tasks that people have to do? So
again I’m not going to walk through absolutely every part of this, but I am asking you to
notice there is this column of actions and there’s this column of targets and what’s that all
about? So a lot of things that people do are these action target pairs. So I’m thinking
about that systematically. So if you start looking through visualization papers you will
see a lot of these phrases like, “The goal is to discover distribution,” or “To compare the
trends,” or “Find the outliers,” or “Browse topology structure.” So, this idea that there’s
an action and a target and let me actually walk through in a little bit more detail what
some of these actions and targets can be.
So with the actions there are really three separate levels where you can ask these
questions and they are actually decoupled and they can go together in any way. So I’m
going to walk through each of them separately. So one is this question of: What are we
doing at the analysis level when we are analyzing data? And already by that statement
I’ve made the world into the part I’m talking about and the part I’m not. So when I’m
talking about analyzing data that means specifically that I’m not addressing the use of
data for artistic purposes. So I’m talking about the use of data for analysis much more
than an evocative emotional experience, although there are some interesting areas about
where those might overlap, but that’s not what I’m focusing on today.
So the really highest level split is you could be consuming data that already exists. A
very classic distinction in visualization is this idea of discover something new verses
something known. I already even eluded to that my previous slides. Sometimes that’s
called exploratory verses explanatory visualization, different vocabulary for that basic
idea. And moreover within discovery there is often a distinction between verifying an
existing hypothesis or generating a new hypothesis.
Now what’s gotten a bit of press lately in the vis world is this idea of a third category
that’s still not so well understood. We’ve called it “enjoy” for now, which is people are
having fun with their data, they are playing with it, it’s not their day job and yet here they
are. It might be baseball statistics on the web or baby names, even if you’re not about to
name a baby. All kinds of things you might do with data, sometimes that’s called casual
or social vis. So we have that as sort of a first class citizen for you to think about as the
goals of the user. Now you might not just consume existing data, you might create new
data as part of the visual analysis process. So we are calling that the case of produce.
Now one thing you might do is you might annotate. You might take an existing dataset
and add information to it. You might record an analysis session. Maybe you do that in
order just to play it back later to show someone else to present. Maybe you do that to
really make sure you’ve got providence of what’s happened with your data.
The third one here is very crucial and I want to put that front and center and get people
thinking about it, because this is the idea that you might derive new data from your
existing data and I alluded to that as a crucial question in that data abstraction step. You
can derive new things. You are not bound by what you are given. You are actually free
to change that. So I will be alluding to this as we go along in the talk and point out
examples where we’ve derived data along the way.
So those are some questions that we can ask at what I call the “high level”, the analysis
level. There are some other questions that we can ask for actions. One is what does the
user know? So words like search, explore and browse have all been used quite a bit in
vis and we wanted to encourage people to think a bit more systematically about that. By
saying, “Let’s think about the target and the location of the knowledge.” So in particular
if you know what you are looking for and you know where it is I’m going to call that
lookup. In contrast you might know what you are looking for, but you don’t know where
that is in the visualization so let’s call that “locate”.
Now there’s the flip side of that which is you don’t exactly know what you are looking
for, but you’ve identified a neighborhood that might actually be interesting. That’s the
location known so let’s think about that as browsing. Then finally the somewhat terrible
case where you don’t really know what you are looking for, you don’t know where it is,
and then you’re really in what we are going to call exploration. I mean this is both
terrible and wonderful of course, but by using this slightly more specific vocabulary
rather than just saying, “Oh we’re exploring,” we find that helps us think.
There’s one more question you could ask which is again decoupled from these other two
which is: How much stuff matters? Specifically are they trying to identify one thing?
That is the example that I showed you before, you are finding a particular path or are you
comparing a number of things to each other? So that’s you are looking at some of the
data. Or are you actually trying to summarize all of the data. So this one, verses some,
verses all, identify, verses compare, verses summarize, it turns out that a lot of what
people are doing in vis comes down to understanding which one of those scopes are you
actually trying to build a tool for.
And it’s very common for people to realize that, “Oh if I have to support the compare
task then all of the apparatus that I needed for identify is necessary, but not sufficient and
we need to build something more in.” And for summarizing the whole dataset, well
suddenly that’s actually the heart of what a lot of vis is. It’s let’s show people in
overview. How do we do that? Well that would be the devil in the details hard part, but
this question of how to provide useful and meaningful summaries in overviews as the
datasets grow bigger and bigger maybe at multiple scales is a really central question.
So now this idea of targets, well there are some targets that actually have to do with any
kind of dataset whatsoever. So things like finding the trends or finding the outliers that’s
generic to any kind of dataset. Something I’m calling a feature is essentially any kind of
domain specific thing a user cares about that generically I’m just going to think of as a
feature. Now thinking about attributes in particular a lot of core questions can be
answered by thinking about one attribute all by itself. And one of the most common
things in visualization is understand the distribution of values within the attribute or the
special case of looking at extremes like min and max. But, there are some questions that
are about relationships between multiple attributes.
So for example is one attribute dependent on another, are the correlated, and are they
similar? These are questions about multiple attributes, not just one and there are also
some targets that depend on the kind of data. So once we have network data where we’ve
got relationships between items suddenly we can ask questions about that topological
structure of the network. And path tracing that I mentioned before is just one special
case. There’s another really interesting one to think about, which is what are people
doing with spatial data? Often understanding the shape of that, the geometric shape is the
task in and of itself. One of the reasons I emphasize that is there are many other tasks
where understanding shape is not crucial. So it’s important to always understand whether
or not the spatial shape of something is a central question.
So that was our kind of whirlwind tour through the why. Now in the time that we have
left I’m going to talk about certain parts of this question of how and really that’s at the
heart of the book. How do we encode this information and interact with it. So there are a
lot of choices. I’m not actually going to talk in great detail about the more understood
parts and that is this part on the left. How do we visually encode data? I’m going to talk
about just a little bit of that to give you a glimmer where a lot of the questions of visual
encoding come down to the use of space. How are you arranging things in space? Then
there are a number of other visual channels including color and others that we also use to
communicate data visually.
So just to step through the real basics of that what if I have this question of: So I’ve got
some visualization idiom how can I analyze this? And it starts at the level of saying,
“Marks and channels.” So let me give you that vocabulary. What I am going to call a
mark, a geometric primitive, is a way to –. So some mark has meaning. It might be a
point, it might be a line or a curve, it might be an area, and think of these as geometric
primitives and now the interesting part is what can we do with the geometric primitives?
We can control their appearance through these things I’m going to call visual channels
and spatial position is a really central one. So it could be just horizontal, or vertical, or
the combination of both together. We could color code, we could shape code, we could
have orientation, or angle, or tilt. We can size code things 1 dimensionally with length, 2
dimensionally with area, and 3 dimensionally with volume. So here’s all these ways to
control the appearance of one of these marks in a way to communicate data about
attributes to users.
Once we have this idea of marks and channels then we can start to actually analyze
things. I am picking these very, very simple examples just to walk us through. What can
we say about a bar chart? Well it’s encoding using vertical position a mark that is a line.
What about a scatter plot? Now we are actually encoding two different attributes,
vertical and horizontal and we’ve switched from a line mark to a point mark. Then we
can start to add more channels. We can say, “Ah, once we add the channel of color now
we can encode a third attribute.” Then once we add a forth channel of size now we can
encode a fourth attribute. So these are simple examples, but they give us a vocabulary.
Now an interesting question is: How many channels are there? The answer is quite a
few. So what’s happening here? Here are a lot of these channels I already mentioned. I
seem to have them in particular orders. What’s going on here? This get’s into one of the
fundamental questions. This is actually originally proposed back in the 80s by
[indiscernible], this principal of expressiveness. There’s a crucial idea which is some of
these visual channels intrinsically to human brains communicate magnitude information.
They should be used to show ordered attributes.
So this question of how much, that’s something where if your brain is thinking how much
in a channel you want to make sure you are encoding data that has an ordering to it.
Because the flip side is there are also some channels that don’t communicate magnitudes.
I’m calling these the identity channels. They are saying this is something or it’s not
something and these are a good match for categorical attributes. So what region is it in?
What color is it? What direction is it moving in? What’s the shape? As opposed to how
much? How long is it? How saturated is it? How dark is it?
So these how much verses these what questions, these magnitude verses identities, this is
the part that you really want to make sure you have a match. If you show unordered data
with an ordered channel you are going to communicate something that’s not there. If you
communicate ordered data with this non-ordered channel you’ve lost information.
So that’s one important principle of matching up those characteristics. The other point is
there’s actually a spectrum here of effectiveness where we are going from the most
highly effective at the top to the least effective on the bottom and you want to try to
understand if you’ve got multiple attributes in your datasets and code the important ones
with these higher ranked channels.
Now an obvious question you might ask is: Where did these rankings come from and
how can we actually use them? Let’s actually think if we wanted to have lots of extra
time, like if we say had an extra hour today, I would go much more deeply into these
questions of justifying these rankings, talking about where they came from and actually
walking you through a lot of these choices for using space, and color and the other
channels. But, that’s actually not what I’m going to do today. So this is I would argue
the part of visualization that is more well understood that has had a lot more camera time.
Although I think there’s a lot of interesting stuff in the book that’s a somewhat different
way to think about arranging space in particular through this lens of either expressing
quantitative values, or this idea of separating it out into regions that your order and align,
or just use given spatial data, that to really discuss this is something I can’t do and also
talk about these at the same time. So I am not going to go deep into these questions of
space, color and other channels. Instead today I actually want to focus more on this
question not just of how you draw the picture, but of a systematic way to think about the
ways you might manipulate that picture. So in the time I have left today I will focus on
that and just with the caveat that there’s a lot of other stuff going on with space and other
channels, but time is finite.
So the part I want people to think about is this question of: If your dataset is really simple
like those tiny little scatter plots and bar charts I showed you then well you draw the
whole thing. What happens as your datasets get bigger and as they are dynamic? How
do we handle complexity? What I’m arguing is that it’s useful to think about essentially
all the techniques we have in visualization as fitting into one of four different strategies.
So the three that I haven’t talked about yet are this idea of manipulating, faceting and
reducing and the one I already have talked about is the idea of deriving new data.
So a classic example would be you could have two different line charts, you could draw
them both and make the user actually think about what the difference between them is.
You know that’s actually this more cognitive act if you can just derive the difference
between them and how that directly. You could visually un-code exactly what they need
to know. That’s a simple case of deriving, but it’s something that we are going to be
using all through a lot of the examples.
These other three approaches to handling complexity: one is this idea of manipulation.
You are changing something over time. You’ve got some view you are looking a data in
and you can change it. A special case even of that is to navigate or to select. So these are
all things about having a view that changes. Now there’s another set of choices that has
to do with saying, “Ah, I could have more than one view at the same time.” So a lot of
design choices about faceting your data across multiple views. Then finally a suite of
choices of saying, “Well within a single view how do I reduce the complexity?” I might
filter stuff out, I might aggregate it, I might do complex combinations of that. So these
three design choices plus that fourth one of derived are the ones I’m going to focus on a
bit today with just a few examples.
So let’s start out with this idea of manipulate and in particular having a view that changes
over time, well that’s actually obvious right. That’s what most people would just sort of
bring to the table and say, “Well I could navigate, I could zoom, I could pan, I could
change stuff.” So that has a lot of power and it’s also extremely flexible. So let me show
you just one example of that. An idiom that’s used a lot in visualization is this idea that
you would have an animated transition from one state to another. Well why is this? It
actually gives you some support for seeing how one configuration changes into another
where you don’t have to, for each item, separately try to figure out, well where did that
go? Where did that go?
So it’s an alternative to jump cut which has relatively high cognitive load and as long as
there is not too much changing in between these two views then you can actually have
items smoothly move between you have a much better ability to track. So this particular
example is looking at multilevel matrix. So here’s where we start out. We see a few
snapshots of it to its next state and what we are doing is the user has actually selected this
part in the middle and that’s now getting gradually expanded larger and larger to a final
state here.
And if you just had a jump cut from here to there it might be very hard to track the fact
that this square here now became that square there. Whereas actually having these
intermediate views where you are gradually increasing the size of one and fading out to
the other is a way to support the user’s ability to track between these. So this is just one
example of change over time. There is a huge amount that people do, particularly things
like zooming and panning that all fit into this category of something changes.
Now let’s contrast that with some of these other choices. So you might say, “Well
instead of just having a single view of the changes over time maybe I’ll have multiple
views.” Now let me point out actually just briefly going back all four of these choices are
often all done at the same time. So it’s not that if you pick one you can’t do the others.
In fact it’s extremely common to combine them, but for the purposes of analysis I’m
going to think about them separately, so this idea of faceting across multiple views. Well
what are some things we could do? We could have multiple views side by side.
We could think about how it is that we chop up the data between the views and we can
think even of putting two layers on top of each other. So let’s start with this question of:
If you were to use multiple views side by side how can you do that? What are your
design choices? Well one question you could say is: Am I going to visually encode the
data the same way across the views or differently? And particularly if you’ve got
different encodings it can be very interesting to day, “And if I highlight a section in one
view how does that correspond in another?”
Another way to think about this is are the views actually showing the same data, or is one
view a subset of the other, or are they showing completely different data? And finally
you could actually also share navigation between the views. You could have linked
navigation where navigating in one causes things to move in another or not. So the slides
are not quite in the order I thought they were, but that’s all right.
So jumping forward to this idea of linked highlighting let’s actually see an example. A
crucial question in linked highlighting, which is perhaps the most common interactive
visualization idiom out there is and it’s common because it’s powerful is: How is a region
that’s contiguous in one view distributed within another? So here’s an example from
baseball statistics where what they’ve done is we’ve got a number of different views. So
this is actually an example of a multiform visual encoding where you’ve got some
different views. You’ve got scatter plots, we’ve got bar charts, we’ve got these
histograms and the data is shared between them.
So, different ways to encode the data, but every view is actually showing you all of that
same dataset. So what happens when you actually select the high salary players? This is
log scale salary and the question is: Do the guys getting the big bucks, how are they
distributed? Well they are really distributed across different seniorities and they are
pretty distributed across the positions and even about this question of: Are you actually
getting people out? But, they seem to actually be clustered on the high side of the
number of hits people have per year.
So okay, we are rewarding the people with a lot of hits with the big money that seems
plausible. What about another way to look at the data? In this view what they’ve done is
they’ve selected what seems like this interestingly [indiscernible] distribution of people
that actually get a lot of the other people out and this is distributed across salary, across
years and across the hits. But we see that it’s very much segmented into specific
positions. So, specific people end up tagging people out in baseball. That’s how the
game is structured. So this is just one example that this example that linked highlighting
can show you relationships between these different visual encodings in a way that gives
you a lot of power.
What’s another way to think of [indiscernible] view of side by side? Well a very
different idiom in this case is these bird’s-eye maps. So these are pretty pervasive,
actually at the moment Google Maps is now taken it out of their system, but it is still
quite common of having these geographic maps. We’ve got a detailed view and an
overview and in this case what we’ve got is the same visual encoding. These are actually
both geographic views, but you’ve got a subset of the data. What we’ve done is here’s
our detailed view and here’s our overview. So we are seeing more in one view than in
the other and in many cases you do have this bidirectional link to navigation. So this
idiom of having overview and a detail together is extremely common, another way to get
power.
Now a third flavor of idiom, the buzzword for this is small multiples, is where you have
the same visual encoding in multiple views showing different parts of the data. So
typically disjoint sets of the data. So in this example, and this is one that you actually
saw earlier where we were looking at this gene expression data, at different time points in
experimental conditions we’ve actually got these completely different colorings of the
same fundamental base layout of the graph and again this one’s got shared navigation.
So when we start thinking about these possibilities there’s a way you can think about this
as a 3x2 matrix. What’s interesting about this is only four of those six possibilities are
really very interesting.
So if you think about your choices of do you show all, or some, or none of the data in
common between the views and are you encoding it the same or different between the
views well then what do we have? Well if it’s the same encoding and it’s the same data
that’s just redundant and there is no point. The interesting cases are this one where you
might have different encodings of all of the data or this sort of overview detail you can do
either with the same encoding or with different encoding. These are both extremely
common and then this case that the buzzword is small multiples is where you really have
the same encoding across different partitioned disjoint datasets. The thing I want you to
notice is this cell is also boring. This cell is if you’ve got different encodings and none of
the data is shared you can’t actually link the views. So that’s not such an interesting case.
Now the point I want to make here is why would we do this? We already talked about
the idea that you could just have a single view that changes over time. What is the
benefit of having these multiple side by side views? There’s a section of the book,
there’s a whole chapter on rules of thumb, that talks about sort of cross cutting interarching themes and one of them is eyes beat memory. So to unpack that anything that
you can do with a human’s eyes is much better than relying on their internal memory. So
the thing is there’s a lower cognitive load for this kind of side by side view compared to a
single changing view because it’s easier for you to move your eyes between views and
compare them in that way then to remember in your head what you saw before and
compare that to what you’re seeing now.
So if this is so great why do we ever do anything else? Well there’s a cost, just like
superpowers; with great powers come great costs. So yes you are lowering the cognitive
load, but you have just used up the pixels and you now have finite display areas so there
is a very finite number of views that you can actually fit before each one of them
becomes so small that you can’t see anything at all. So this tradeoff between the costs in
pixels and the benefits of scaffolding your memory is one of the core things that you then
struggle with as a vis designer. But, it’s useful to think about that very explicitly.
So there are some other questions: How do we partition into views? So I’ve said we have
multiple side by side views, well how do we chop things up between them? How do we
divide the data between these views? Let me point out something explicitly that might
sound obvious and trivial, but I think is actually a really important and central question:
What are you doing when you divide data between views? You are encoding the
association between items with spatial proximity. You’re saying, “Here’s one view and
here’s the other,” and it turns out that back to the way human brains work this spatial
proximity is a very, very strong signal to your brain that things are associated with each
other.
So it turns out that by partitioning into views, having different regions and making
choices about which items go into which regions that is one of the most crucial questions
in how you are able to detect patterns. So this choice of partitioning is actually very, very
central. So how might we do that? Well one thing we could do is say we could split
according to attributes. Typically we are doing this in situations where we don’t just
have one attribute we often have many. We have multi-attribute tables that we are
splitting in various interesting ways. So there are a whole lot of design choices. I’m not
going to go into the design choices for all of the things, but just to give you an example
that for every one of these design possibilities I’ve talked about in the book it goes into
much more detail about the set of choices.
For example how many splits do you do? Do you go all the way down until where
you’ve got a single mark per region like you do with a simple bar chart or do you actually
end up going part of the way down where you are doing some partitioning and then
you’ve got a region that actually has quite a bit of interesting structure within it. Then
you might draw something as a more complex glyph made of multiple simple marks. So
this whole question of when do you show things a glyph’s verses when do you show
things as marks this comes down to: How did you partition the data? What is the order
that you split things in? As we’ll see it’s got a big consequence. How many views?
Open research questions nobody knows. At least 2, not 6,000, in fact not even 100, in
fact probably not even more than a few dozen. Is a few dozen completely pushing it or
very viable? It depends on your circumstance. So this question of how many views is
definitely an interesting open problem.
So let me give a concrete example. Let’s think about the bar chart case because that’s
easy to think about and what do we see that’s different with two different partitioning
choices? So if we start out with a table of data and the first thing we do is split –. So I’ve
got a dataset where I’ve got a demographic attribute and I’ve got it by states in the US.
So, if I first split this by the state and then stop and say, “Okay, I have one region per
state, now what?”, now I’ve got these multiple attributes within each region that I want to
visually encode. In this case I’m going to do it where each one of them has their own
bar.
So this is basically a grouped bar chart. So what’s the point here? It’s really easy to
make comparisons within a state. We can think, “How did the demographics of
California work? Oh, look there’s this dip in the middle and then it spikes way up. Is
that different or the same from the patterns in these other states?” So you can make
comparisons within the state. That’s easy, but it’s kind of hard to compare how many
people are in that middle demographic group because there’s all this stuff in between. In
contrast you could split the other way.
You could say, “First I’m going to split by age,” and within each of those regions what
am I going to do? Well actually now I could just have one chart in each region. I could
have a single bar chart per the region. What can I do now? I can very easily compare
across age. I can say, “Oh this is relatively flat whereas this one is actually descending.”
So comparisons now within age are easier, but across states are harder. So by making a
choice about partitioning we have dictated what kinds of visual patterns are easy to see
verses hard to see. This is easy to understand with bar charts, but things can get more
complex as you combine these with some of the other more exotic layouts.
So let’s just look at a few other examples. Here’s the dataset called HIVE from the folks
at the GI Center and they’ve got a dataset where they have housing sales in London. So
the kind of house, the neighborhood it was in and exactly when it was sold. So first let’s
think about what’s one way to look at this data? In this case rather than just looking at it
as a list of things or even a matrix we’re actually recursively subdividing. So this is part
of the discussion about spatial layouts that I’m not going to get deep into, but I’ll just
mention that this is an example of this relatively complex recursive subdivision structure.
So what they’ve done here is they’ve first split it by the type of the house. It’s an
apartment, a flat, semi-detached, a terraced house, a completely detached house with a
yard, like a lot of folks have out here. Then they split by neighborhood. So these are a
bunch of the neighborhoods within London, each of which is in its own little box and
then they’ve got time where the rows are the years, the columns are the months so you
can start to see some temporal patterns. Now what you can see in this view is allowing
you to very easily look at things like, “What’s happening within a neighborhood? Let me
compare one neighborhood to another. Ah, this one’s much darker meaning the houses
sold for higher prices.”
So you can start to immediately start to see things like, “Well which ones are the rich
neighborhoods, which ones are the poorer neighborhoods? Which ones don’t have
actually very many houses at all being sold?” So what you see in this view is different
than if we partitioned the other way. So what if instead we say, “First I’m going to split
my neighborhood and then I’m going to split by the type of the house?” Now you can
see very different patterns. Now it’s much easier to say, “Ah, I see, some of these
neighborhoods are actually dominated by one house type verses another.” And we can
certainly see things like, “Ah I see, certain types are in fact more expensive.” The
detached houses in general tend to cost more than the apartments. So there’s some
analysis of these other categories that are easier to see because we did a different
partition.
Now just to wrap up this example partitioning is not just for rectangles and squares.
Here’s an example where first they partitioned into the house type and then they said,
“Okay, within that now let’s actually encode the data where we use spatial information.”
So in this case instead of just having little squares and rectangles we are actually saying,
“Let’s show the geographic boundaries and we’ll still color code by the attribute.”
Technically that’s called a [indiscernible] map and so the point is there are many ways of
visually encoding the data within these regions and some of those choices are very
interesting and central as we think about partitioning into views.
Now I’m not going to go deeper on this front. I’m going to come back again to this
question of we’ve now talked about a changing view; we’ve talked about partitioning into
multiple views. I’m just going to briefly talk about this idea of reducing the amount of
data that you show in a single view. So what’s the idea here? Well first let me mention
when I talk about reducing the amount of data half the time I actually mean increasing.
So these are these inverse things. You can reduce, you can increase, so these are ways to
throttle the amount of information you have. Now a core choice is to filter, right. Filter
means some stuff you simply don’t show. What’s nice about this is first of all incredibly
easy to explain to people and often computationally very nice and straightforward and
suddenly you’ve dramatically reduced the size of your dataset and everything is great, life
is fantastic.
So why don’t we just do this all the time? The answer is that this get’s back to how
human brains are, we have this terrible tendency to have out of sight out of mind not just
be a proverb, but actually a reality of how we visually process data. Even if you yourself
hit the filter button three minutes ago to stop drawing things you immediately start
drawing conclusions as if what’s hidden doesn’t even exist. And it’s so built in that it’s
really hard to avoid that. So even though filtering is very straightforward it can really
end up with you being misleading if you just don’t show some of the data.
Well what’s the alternative? Aggregation is a much more complex thing where what you
say is, “I’m still drawing a smaller set of things than my original, but rather than just
completely ignoring some of them I’m actually going to say, “I’ve got a proxy item that
stands for a number of original items.” So what’s nice is that proxy item in a perfect
world informs you about something about the entire set that it replaces. But, how do we
do these proxies? Well that would be the hard part. So how do you build these aggregate
proxies in a way that doesn’t lose the entire signal you were trying to encode?
So it’s very well know that if for example you just start averaging things you can easily
smooth out the interesting signal in your data. So a lot of interesting questions come to
doing something more sophisticated often with aggregation and filtering both. You are
doing something interactive where you are changing over time. So a lot of interesting
visualization and design choices comes down to combinations of filtering and
aggregation, either statically or dynamically. And let me mention you can do this either
for items or for attributes. So either the dimensions, the attributes you are showing or the
items you are showing. And again I really want to emphasize what’s very common is to
combine these things. In fact the technique, this so called focus plus context techniques
where you’re embedding focus and context information together are really sophisticated
combinations of filtering and aggregation.
So not only can you combine filtering and aggregation, but those four major design
choices of reducing, and faceting, and changing and deriving, these are all things that are
typically combined. So let me show you an example of, in this case, a well known static
aggregation technique, the boxplot. So I want to walk you through that just to put this
into context, this is designed for the task of finding distributions. And we start out with
some table of data and then what do you do? You do something interesting which is it
doesn’t matter how big that table was we are aggregating that into a very small number of
numbers that we can then show visually. So we are visually encoding the derived data of
five quantitative attributes that actually express a statistical summary of that original
attribute.
So of course what we’ve got is this central line showing the median. You can see this
upper and lower quartile’s explicitly encoded as the height of the box and then showing
the points beyond which things are outliers. And if there are outliers sometimes
explicitly showing those. So this is an example of a static aggregation technique. A lot
of the interesting action comes when we start doing things that are more interactive and
dynamic. Let me, just to give you the other side of the fence, show you a particularly
complex example of aggregation. So, this idea that actually we might aggregate not just
items, but we could aggregate the attributes themselves. Think of these as dimensions of
data.
So here’s an idea of a pretty complex idiom and task of: How do you look at a huge
document collection? What do we do now? Well documents weren’t even in our set of
data that we knew how to encode. You know you’ve got free form text, what do you do?
So first of all a very common transformation is turn the document into a table. Well how
could you make a table out of a document? You could say for each word in the document
I’m going to think of that as an attribute and actually just keep a count of the number of
times the word appears in the documents, this so called “bag of words” model. That’s a
row in this table and then I have one row for each of these documents.
What do I have? A really big table and looking at that table directly is not very
interesting. It’s incredibly sparse and it doesn’t help the human do much. But, now what
could we do? We could say, “Well I’m going to take that very large table and I want to
derive a much smaller table that has only two attributes, two dimensions.” So that is
attribute aggregation. Now the question of how we do that? Many, many algorithmic
papers, it’s an interesting topic and that’s a whole another hour long talk, which we won’t
be doing today. Lots of fun algorithmic meat in there, but let’s pause it for a moment that
we’ve found some way to actually not lose too much of the signal, because we assumed
that there’s latent structure in the data that was gathered verbosely so it’s reasonable to
actually throw away quite a lot of redundant information.
And now what do we do? Now suddenly we have points in two dimensional spaces
instead of points in high dimensional space. What we can do is start to use a visual
encoding like drawing a scatter plot to look for clusters. So now our task is to say, “Oh
are there documents that are relatively similar to each other?” So dimensionality
reduction is a hypothesis about is this an interesting space to look at? Then you can look
at the documents themselves and try to see, “Do I see clusters? Do I believe the clusters I
see?” You might actually do a lot of exploration and navigation to go in and actually
select. This point represents a document and maybe you read that document. You might
try to say, “All right, what is this cluster even about?” Is there a way I can come up with
a label for that cluster that’s meaningful?
So the thing to notice here is I’ve actually structured this as three cascading tasks. So
when I mention this idea that you want to analyze things in terms of tasks sometimes you
actually want to think about sequences of tasks, not just a simple task in and of itself.
The most interesting complex visualization systems handle these sequences of tasks and a
lot of this question of the [indiscernible] of the analysis question. At the beginning I
started out by saying, “What if people don’t know what they are asking?” Often
sequences of tasks clustered together are a way to actually go back and forth between a
lot of different possible questions. So in this one I’ve framed it as three different tasks
where the output of one is the input to the next. So it’s really focusing on deriving data
along the way. So this is an example of quite complex attribute aggregation.
Now this was a whirlwind tour through a subset of what the entire book goes into in more
detail. So I introduce these ideas of the four levels: domain, abstraction, idiom and
algorithm. I introduce this idea that there are these three questions that you can frame:
two of them are the abstraction level, the what, and the why and the how are at this idiom
level. I haven’t talked at all about algorithms, except to allude to the fact that they are
fun and we want to do a lot of them, but that would be another 17 hours of talking. In
fact the book itself does not get into the algorithm level. The book itself is staying mostly
at the abstraction and the idiom level in order to make it 400 pages rather than 4,000
pages.
So speaking of lots of detail we saw briefly this idea that there is this topology of what
questions, topology of why questions, there’s a lot of how questions and we just alluded
to some of them. So if folks are curious as I mentioned there is this book and it goes into
a lot more detail. If people actually feel like getting the book and eBook together for less
this is a magic promo code that will work on the publishers page. The talk slides
themselves are also on the web. Let me mention that all these diagrams I have, if people
want to use them in their own talks, are available as creative commons licensed and
Eamonn Maguire did a fantastic job with those. Finally there’s a lot more information
about all the other stuff we do, actually my own research, which I didn’t talk about very
much at all, both on our group’s page and then my own personal page.
So with that I’m going to switch over from talking at you to getting more questions from
you, either those of you in this room or on the screen. I see there’s a question from a long
time ago.
[Applause]
>> Tamara Munzner: So let me start with this poor question that has been languishing on
my screen and forgotten from [indiscernible]. I’m sorry about that; I’m not used to
having questions come in through this modality. So your question is not sure why the
display limit is the most constrained resource. So that was actually way back on slide 2 I
think.
>> [inaudible].
>> Tamara Munzner: So what I would argue is typically you’ve got quite a lot of
computer memory to play around with and even the humans have a fair amount of
resources, but often what we end up with, a situation as a designer, where the first thing
that causes you pain, and agony and swearing is that you’ve run out of pixels. You can
partially address this by saying, “Let’s add more displays.” These days a lot of people do
at least have one high resolution display or even a couple of high resolution displays.
You can even have more complex things like display walls, but then you start getting into
the constraints of how much of that you can actually attend to in high resolution at once?
>>: So it inner operates with human intention as well.
>> Tamara Munzner: Yes, and one reason I kind of emphasize this a bit more is that it’s
the resource that I think people tend to think about the least. People are used to thinking
about computational resources and at least people with an HCI background are used to
thinking about human limitations. But, thinking about pixels as something in and of itself
particular when you start thinking of the multiple displays. So then when I started talking
about this issue of faceting where every time you add a new window you take away the
pixels available for the other windows it helps to think about that really explicitly as a
tradeoff of a finite resource.
Any questions from people in this room? Yes, all the way in back.
>>: So I just want to get some insight from you, when you are using multiple encoding
channels to design [indiscernible]. So I remember I read somewhere, I forgot where
specifically which book, but there are two perceptual structures from psychology: integral
and separable. So, integral means that the two characteristic are perceived more
integrally with the human brain such as X and Y position and [indiscernible]. And
separable means for example size and color, there are two different encodings and when
people perceive them they tend to be separately. So what are your insights on that when
you are using those multiple channels and [indiscernible] those multiple channels to
multiple data attributes, because you have so many design choices.
>> Tamara Munzner: Yea let me actually bring up the relevant slide for that. I have to
remember where I’ve put it, but I believe I’ve got it here. Yea, so this question of
separability verses integrality is this sort of interesting and complex one. So because I
had this whole table of channels and then the question was: When are you allowed to say,
“I can combine this channel and this channel and have them actually read off the
information separately verses when life is not so good?” So this is the place where
there’s actually been quite a bit of study in the cognitive psychology literature on human
vision. So some of this is understood, but there are still a lot of frontiers where it’s not
fully understood.
But, let me just actually walk through this example because I think it’s evocative where
for example spatial position and the hue aspect of color are very, very separable. So what
we can do is say, “What I’ve done here is basically two groups of two.” There are two
groups spatially right, there’s this group verses that group. So that’s one attribute having
two possibilities and I’ve also got another attribute where I’m coding by blue or by red.
So you are able to selectively attend to either one of these. You can either attend to
position or you can attend to color. So that means I’ve successfully encoded all of that
information here.
But, things then start getting worse. Here’s a situation where it’s really easy to tell apart
the big purple circles from the big blue circles, but it’s a little harder when I have size
coding. Size interacts with color such that it’s harder to tell apart, it’s not impossible, but
it’s harder to tell apart the small purple circles from the small blue circles. You’ve sort of
got less dynamic range in which to make that perceptual discrimination. So there’s still a
little bit of interference, but at least we’re still able to see that there are two groups.
There’s little stuff and big stuff and there’s purple stuff and blue stuff.
Now what about here? Here’s where life starts getting worse because what I’ve done is
I’ve tried to separately use horizontal length and vertical length to encode information.
And that sounds good if you just think, “Well there’s two different attributes. I’m
encoding them with two different things,” but it turns out that our brains are doing
something integral with this. And what we end up perceiving is area. So the problem is
what we really see is small stuff, big stuff and flattened stuff kind of independent of
whether it’s flattened this way or flattened that way. So fundamentally we are really
seeing three groups of things more so than the four we originally had in mind. So you
can if you try hard actually start to say, “Okay, well I’ll try to treat these differently from
these.” So you can sort of start to recover, but it’s harder.
Here’s a terrible situation where what I’ve done is, and I can tell you I’ve done this and it
doesn’t help, I can tell you that there are two values of the red channel and there are two
values of the green channel. And I can tell you this until my face turns blue and it
doesn’t help you actually see those, because this color is actually perceived as a single
perception. So you can’t actually separate that out. You can’t find the two values of red
verses the two values of green. You just really do see these four different colors. So
even if you try hard consciously you can’t pull apart those integrated dimensions.
So one issue is understanding what the relationships are between the channels. So one of
the things I do is talk a bit in the book about which ones are more separable and which
ones are more integral. Basically it turns out size coding ends up interfering with a lot of
the other channels and it particularly interferes with color perception. So that’s
something to be aware of. So the usual case in vis is that you are going for this kind of
experience where you want things to be as separable as possible. Sometimes you are
deliberately going for this where you are actually getting this integrated perception,
particularly designing glyphs where you have multiple marks in a small region.
So you want to be aware of this phenomenon. There is no magic answer, but
understanding and what we can use much more research is, for example, let me go back
to a list of these channels, some pairs have been explored and understood and others not
so much. So there’s been some work, [indiscernible] and [indiscernible] have done some
work for example on the motion channels. Right now it turns out that any kind of motion
verses something static really strongly pops out at people. So you’ll notice if something
is moving around. It’s conjectured to be unlikely by people like [indiscernible] that you
can have multiple motion channels that are easy to just [indiscernible].
So maybe it’s not a good idea to try to say, “Well I’m encoding one thing in Flickr
frequency, and another thing in direction of motion and a third thing.” That probably is
not very separable, but this is something which is literally not yet extremely well
understood, so some characterizations. Also like size coding interferes with shape
coding. As things get smaller you are less and less able to see. So there is some work,
but it’s not completely understood, so good question, long “ish” answer.
>>: This is an observation, not a question, but you were referring to the size coding
interfering with color and it seems that’s an example where that interference is true. We
can all grasp that, but it’s primarily true at very small sizes in that case. So there might
be a large range over which they don’t interfere much, but then other parts of the
continuum, if you will, where there would be interference and that may be true as well for
other angular displacement or other things too. But, it’s kind of an interesting angle, like
over what domain or what range do these things coexist very nicely, very orthogonal that
at some point they don’t?
>> Tamara Munzner: So part of what I didn’t have time to get into deeply with talking
about these is you can think about these channels in terms of many things: one is the
accuracy at which you can perceive things and another is the discriminability. How
many bins within each of these do you have? And so you can imagine a situation where
depending on the size coding you might have the full range of bins or fewer bins, also the
ability of something to actually visually pop out pre attentively without conscious
control. So these are all ways you can think about these channels.
So for example I didn’t have time to go deep into questions of color today, although it’s a
pretty glorious topic. One reason color is so difficult for people to think about first of all
is that some aspects of it are categorical and some are magnitude. So that’s one source of
confusion right there. And for example there is definitely some overlap between things
like saturation coding and luminance coding where at some point as you remove more
and more of the hue suddenly you start to be unable to distinguish.
So there’s a lot of stuff about color that also involves stuff that you wish would be
separable that ends up being far more integral than you realize. Transparency also, in
order to create visual layers you often have a foreground verses a background and you
might have a semitransparent background. It turns out that heavily interferes with things
like luminance and saturation coding. So there’s a lot of stuff involving color where it
really very much comes to the forefront.
Natalie?
>>: [inaudible]. So lots of ways you presented even though it seemed like you make it
design for exploration and presentation, but in my head I was thinking all of this
technique or different methods to [indiscernible] or manipulating things you couldn’t say
one is better than the other in many exploration cases. But, then I started thinking in fact
when you are in the presentation case where you are using [indiscernible] things or
maybe you are repeating things then that may become now some of them are better than
others. So, I was just wondering how much you thought about that. So first are all of
those thought about the exploration and how you divide all of those different things and
elements? Did you think really about those two cases: presentation and exploration or
maybe more one? Also, do you have some sort of idea about if we talked about visual
communication now if some would be preferable or not.
>> Tamara Munzner: Yea let me actually bring back a different slide to make a particular
point. So one point is that these questions of: When is something good? I heavily
emphasize that you can never say, “When is something good?” You always have to say,
“When is something good for a particular task and data abstraction combination?” So I
feel like a lot of interesting questions come down to characterizing design choices at the
how level with respect to specific configurations of what and why.
So it’s never is this better; it’s always is this better for this task? And so what you talked
about is weight. Here are two very different high level tasks: discover verses present.
And so now can we start characterize some design choices with respect to these tasks?
And my answer is absolutely, like the answer to almost everything in vis is it depends on
the task, but that’s a good thing and not a bad thing. So this question of so absolutely it
can depend on discover verses present. In fact it could depend on things even at a much
finer level of granularity than that.
I feel like one of the most interesting in vis right now is basically we’ve done a lot of
enumeration of blocks, if you think about these idioms as Lego bricks. The beginning
years of vis were all about, “Look at my new Lego brick; isn’t it great and here it is.”
Then at some point we became interested in questions of: And so what is it good for?
Can we characterize not just this new brick, but a whole family of bricks as being related
in an interesting way in a way that we can tie to these questions of why and what?
So I feel like these characterization guidelines for abstraction level into idiom level
mappings is where a lot of these interesting questions are. So some of that work has been
done and there’s just this enormous amount left to do. So it’s kind of a long winded
answer, but my answer is yes, yes, yes, absolutely it depends and not just on that, but on
all the tasks. So I feel like filling in this giant cookbook in the sky of one day will we be
able to fill the vis cookbook where you say, “This is my data and this is my task,” and we
have the complete vocabulary of these abstractions. So we’ve found a bunch of the Lego
building blocks at this level and we flip through and we find page 5,673 says, “Okay, so
for those here are some sets of idioms that are known to be good or known to be bad.”
That’s the 30 year research agenda for the field, but we totally need to do it.
Benjamin?
>>: [inaudible].
>> Tamara Munzner: I think of interaction not as a limit, but as a strategy, as a tactic. So
a lot of the really, I mean many of the interesting vis idioms are just intrinsically these
combinations of visual encoding and interaction. So with [indiscernible] like what is the
encoding part and what is the interaction part? They are linked and so I feel like
interaction is part of the strategies about how we do things. So a whole bunch of the
design choices come down to specific uses of interaction. So it’s not so much that
interaction is a limit, interaction is a technique to address a limit, which is human abilities
to do things like remember.
I mean interaction is very much a double edged sword, because actually to go back to
these bits here the thing about interaction is any time you are changing the view over time
you are now basically imposing a memory cost. You are asking people to remember,
“What did I do before?” With a lot of navigation the problem is, “So I navigated and
now I’m lost. Where am I, where is the root, where is home?” So a huge amount of the
techniques that we propose are ways to basically scaffold the frailties of human memory
to get some of the power of interaction, because the power of interaction is you the vis
designer didn’t have to figure everything out a priori. The user’s allowed to actually do a
bunch of this exploration, but then you are actually now taxing their cognitive system
more and more.
So, there’s this sort of tradeoff of power verses cost. So I think of all the interaction
techniques as being sort of apart of this system of design choices. One of my logics
behind this taxonomy the way that it is, is to say, “Let’s explicitly think about stuff that
involves changing something, which is at the heart of interaction, but a huge amount of
the power of all of these side by side views is the fact that you interact with them.” So as
you go in and you select an object in one view and see it highlighted in the other, well
that’s a combination of linking views and interacting.
So a lot of these reduction techniques of filtering and aggregation, the simple ones are all
static. The interesting and complicated ones are of course all dynamic where as you
interact, as you select things, as you do things like, “Well let me select a scale at which to
aggregate in this local region to me it feel that interaction is essentially a suite of design
choices that work in combination with these others to actually give you a lot of power of
the vis.” Again, sort of a long winded answer, but –.
>>: How confident are you that you’ve covered everything can exist?
>> Tamara Munzner: So the answer is in some ways I am reasonable confident and in
other ways I am absolutely sure that it’s incomplete. So let me give those caveats. What
I have tried to do is particularly in the how where I’ve thought about the use of space, the
use of the other channels and then manipulating and multiple views and reducing. I do
think of this as reasonably complete. However, if I had a hundred million dollars to bet
would I say that this is the final answer and we are all done? Well no, I wouldn’t bet a
hundred million dollars on that. On the other hand if we are talking about 20 bucks that
we are betting and you were to say three years from now could you go through pretty
much every paper at vis and end up categorizing it’s choices in to one of these bins
usefully I’d say, “Yeah, I’ll bet you 20 bucks on that.”
Now the place where my willingness to bet is going way further down, so for example I
don’t think this is complete. I think that this taxonomy of tasks is fairly high level. I
think one of the really interesting parts in vis is going to be coming up with either
hopefully not domain focused, but ideally at least like dataset set focused like, “What are
the set of tasks for networks?” There has been papers written by people like
[indiscernible] on let’s have a topology of network visualization tasks and there has been
other work as well. So we just had a paper last year at Believe on: What are the tasks for
working with dimensionally reduced data, where you actually end up adding quite a bit
more precision.
So I think this is a start. It’s a better start than I had before and even this, for those of you
who know the paper from last year on a multilevel taxonomy for tasks notice how there’s
a bunch of stuff in this version in the book that wasn’t in that, in part because of page
length, but in part because we spent more time thinking. So I feel like this is very much
sort of evolving and then on the dataset front I don’t think this is necessarily the final
answer either. In particular I went back and forth a lot about these questions of spatial
fields and geometric stuff in and of itself and notice how there’s this grab bag of other
stuff. This was the miscellaneous category.
So suddenly there are all these papers on sets. At InfoVis this year there were three
papers on sets. You know lists, clusters, so I don think any of these is the final answer,
but I think they let us try to come up with some kind of shared vocabulary for now and
people will propose additions to the vocabulary in changes and refinements. So I don’t
think that necessarily these particular diagrams will be the final answer 200 years from
now, but I hope they give us a place to have a discussion of the next set of building and
define these guidelines between them, which I think is the really interesting part.
[Applause]