>> Danyel Fisher: So good afternoon and welcome. I'm Danyel Fisher from the
VIBE research team, and I am excited to have Sheelagh Carpendale joining us
today. Sheelagh's in from the University of Calgary, where she's been a pioneer
in research on Information Visualization and tabletops and she's going to talk
today about the joint spaces between Information Visualization, art, and tabletop
design.
With that, I'd like to pass it over to Sheelagh.
>> Sheelagh Carpendale: All right. I don't actually -- well, I guess I kind of do
have to stand here for making my slides go forward. But because I'm sort of
behind this that you guys -- one of the disadvantages of being short.
Yeah, I put this title Walking the Line, though it is the first time that I've actually
done a public talk talking about this space between fine arts and computer
science. And so part of me is actually much more nervous than I usually and
because I think like oh, I don't know how this is all going to come together and if it
would make sense.
But I think that many of you actually have seen my work and you've seen the
papers, and if I just kind of, you know, like redid that, that will be kind of boring.
And I thought everybody actually knows that I have dual background and that
maybe it would be the time to try and talk a little bit about how that works for me.
So, yes, so that's what I'm going to try and do. So, you know, I have an art,
design, and computer science background. Way more time being a student than
one should ever do.
I went to Emily Carr, where I did art, not design. And I went to Sheridan College
where I did design. Actually worked teaching fine arts for almost a -- you know,
almost about ten years. Had my own studio, did all that kinds of stuff and
actually came back and went to Simon Fraser and did two degrees in computing
science, undergraduate and PhD.
And so with all of that of course didn't end up being a proff until moderately
recently, until 2000. So this still feels to me like kind of new. And I did actually
kind of you know get sucked into feel like I should be a little safe, pretenure and
kind of stay with the science thing in a computing science department. And you
know. So it's only been in the last three years that I've been kind of looking more
at how I can push this edge and bring it into what I do and that kind of thing, like
trying to actually -- so part of where I wanted to start is actually want to talk about
these terms that fly around, like what I actually think about what does
multidisciplinary meaning.
And to me, a multidisciplinary endeavor is one where people of many disciplines
needs to work together for a common goal, but there's integrity within the
disciplines. And I think a perfect example of that is the medical situation, where
you have, you know, your family doctor, your surgeon, your social worker, you
know, dietician, all kinds of different people, they have their own expertise, you
really need all of them and you actually really want them to stay in their own
expertise. It's where they belong.
And this is what I think of as a multidisciplinary effort. And a lot of what we do in
our society is multidisciplinary.
And there's this or space called interdisciplinary, and that's kind of more common
I think to the kind of space that we work in, where, you know, you have a
programmer and a designer and a human factor's expert and you bring a team
together I think it's like really common like in theater where the end result is a
kind of melding of the skills. Not necessarily the people change, but they're
working together like to put a play on where, you know, the skills of each are kind
of bumping into and feeding off each other and they're working in what I think of
as you know interdiscipline, between the discipline space.
And then there's this other space which is what I kind of inhabit what I think of
transdisciplinary, and you can sort of see the link. But it's where, you know you
take the time to actually try and understand both worlds to some extent so that
you actually at least to some extent embody both kinds of expertise. And in
education up until now, most of this is kind of an arduous path, kind of like the
kind of thing that I did that's sort of not actually something I really actually
recommend people doing. But I think that for our world there is this sort of huge
spectrum for somebody who is in, you know, right at the end of doing fine arts
and design and just factually wants to have a little bit more technology, program
type savvy so they can at least actually tell when their programmer's pulling the
wool over their eyes when they tell them they can't do something when it's just
they don't want to do something. They want to have a little bit more technological
power.
And you know, where computer scientists might actually just like to know a bit
more about colors so they can maybe make their interface kind of a little bit more
sexy, you know, a bit appealing in some way. And they're just kind of stepping
slightly. But there's this whole space in the middle about, you know, where the
two come more to be and what does one end up and where does it -- where does
that sort of change over and how does that work? And yes, so there's two parts
to this. And one is the methodology of how you actually go about doing your
work and one is dissemination, how you actually go about talking about your
work.
And I think they're really quite different. And I'm going to mostly today talk more
about my kind of exploration of the blurred methodology space, but I wanted to
have a little bit of talk about voice, the dissemination space. Because when I
work in -- and I do do a lot of interdisciplinary work where I'm working with groups
of people with more than one discipline, when one of the big differences to me is
sort of what you want to do with the end result and how you want the people who
interact with it to experience it. And if I'm in the science voice, all the efforts is to
actually make it clear, like ideally somebody reads a good science paper, they
can go back to their own place and they can reproduce it. If they've done a really
good job, they can rerun your study if they want, they could reimplement your
algorithm. You want and try to be clear that one of the reviewing questions is
could an intelligent grad student go from this paper. That's one of the common
things you want to answer.
And for an artist, they definitely want to get their stuff out there. They definitely
want people to see it, to experience it, to interact with it, but they want to raise
the level of questioning in the viewer. So they don't necessarily want to -- well, in
fact actually if I try and write about it, they might say you're spoiling the piece,
you're destroying it, right, you're taking away all the mystery here.
So there is a -- there is a tension here between, you know, like how people want
something disseminated and who they want to disseminate and whether me, you
know, thinking as a scientist would think it's being disseminated, right? Or
whether it's just been put out. But they really want people -- I don't know, you've
probably actually used that word, you know, problematize, which I hate, but
essentially one way of talking about it is they want to take some issue and
actually expose the problems in it so that people are sensitized to it, more aware
of it, that it may actually influence and impact and change their way of thinking.
Right?
Yes, so I think there's a lot of unexplored and unresolved for me differences in
what the voice was. But then as I went through kind of thinking about what I was
actually going to say about blurring of the methodologies, I found myself less and
less clear of which is what. And I think that you'll -- I'm going to talk about it a
little bit like as how I can see this as an art methodology, but I think you're all
going to say, yeah, I do that. We do that all the time. And actually think that, you
know, there is -- there is this, we use both methodologies, but I actually think like
there is a kind of richness to it.
So but I think that there is also a kind of fundamental difference which, you know,
can be talked about as a difference between vertical and horizontal. And I think
the scientific methodologies being a vertical methodology and I chose this image
of because I think of it like, you know, so I'm advising student to something, a
problem, say, hey, this is a really exciting problem this is what I want to work on, I
would actually say, okay, let's find out what's out there what's been done, who's
done what, let's actually make sure that you have a real good understanding of
what has come before you.
And part of what you're doing is like you kind of gathering all of the literature and
all of your antecedents, your ancestors, your academic ancestors behind you
almost to give a momentum to it. Like I think of it as actually you know getting
yourself -- where is my mouse, getting yourself up into the crest, into the foam of
the wave so that you have this kind of momentum behind you so that when you
take a step once you know that, it will be forward. That by the methodology itself,
you know what's been done already so that when you do something new, you'll
be actually moving forward. And, you know, this is a good methodology. This
works well. We use it a lot; I use it a lot. I don't -- this is not -- so I don't want to
actually be criticizing either, I just kind of think there is a kind of richness of trying
to actually look at both.
And so if I characterize an arts approach, I think of it as horizontal. Like that.
You have the whole world at your feet. That your anything is part of your
information, your inspiration any time, anywhere you are you could draw on in
any kind of way. So I thought this was actually kind of a nice broad horizon
picture.
So it's all about like learning how to sync laterally, learning how to get yourself
into a place where you happen to approach this problem from before, like -- and I
know that one of the things that I still use a lot of is the whole kind of contrariwise
approach, that if I'm thinking about something, what's the opposite? If I have a
solution that I'm not happy of it, what's a non solution? Because thinking about a
non solution will then have a mirror into the solution space that's somewhere
different. Like can I use playing between the positive and negative to find more
places in the space than I had before? So this sort of horizontal approach is kind
of like can you come from it from somewhere else?
And I know when it was teaching fine arts, the best thing the student ever said to
me was I see the whole world differently now. Because this is all about
observation and how you observe and how you're aware of your own reactions to
what you're observing. And that all of that is where inspiration comes from and I
think that's like -- I don't see any reason why we can't do both, why they don't
both fit together. And I think that there's -- you know, there's useful stuff to be
gathered from both.
And I also think that it hasn't been that comfortable for me operating in this
space. And I usually find that whatever space I first meet somebody in, they kind
of think that I can't really be very good in the other space. So, you know, if I'm
used to operating with somebody in the arts world, they'll think I must be a kind of
compromise scientist, and if I'm used to operating in the science world they say
but you don't really do art anymore. I mean this.
>>: [inaudible] in the reverse.
>> Sheelagh Carpendale: Oh, I don't know. Anyway, so it's like -- any way. So
there's -- it hasn't actually been that comfortable -- uncomfortable, but yet I do
think that it is, I guess, you know, very rich, and I think -- if I remember next, the
next slide is -- yeah, this is the gang at this point.
All of my students have -- though some of them are quite in the arts world
themselves, that's still a relatively few. Most of them have come from a very solid
science background, but all of them in working in my group have started to
actually -- they're coming at least an edging in a bit and that's some
understanding of this kind of transdisciplinary space. And part of the reason that
I put this up at the front is I do want to, as I start getting into examples, you know,
this -- these people are of course the reason why all of these examples exist.
And I do actually include theirs pictures when it's their stuff so you can see, if I
don't actually say them by name each time.
>>: I have a question [inaudible]. So earlier you were talking about like in the
purpose of our -- a lot [inaudible] about raising the question or raising the level of
questioning in audience [inaudible]. Do you see like -- I mean and there can also
be a message that [inaudible]. Do you see that as part of raising the awareness?
>> Sheelagh Carpendale: The message?
>>: Yeah.
>> Sheelagh Carpendale: It can be.
>>: [inaudible]. Okay.
>> Sheelagh Carpendale: Yes. It depends on the artist. I mean, sometimes
they're quite passionate about a message, some of the ones that I know, and
sometimes they're not -- they're, you know, they want the piece really actually to
irritate, like to cause consternation. It's kind of to try and make somebody rethink
some of their fundamental -- I mean, I don't -- there's a whole lot of art these
days that actually makes me feel really uncomfortable.
>>: Do you feel like the raising the question aspect is more fundamental maybe
[inaudible] to make use of [inaudible].
>> Sheelagh Carpendale: I think it would depend on the artist. For some of
them, definitely. I don't think necessarily for all. And there is also, you know,
there's a whole blurring. I mean, I said sort of art and design, but it's all blurring
between art and design itself. And those, you probably actually know like with a
-- they're two very different ways of thinking.
And your question is really is an art-flavored question. Where a designer is much
more likely to be swayed by, you know, functionality or appeal or those kinds of
things, so they [inaudible].
>>: [inaudible] there's work that is often [inaudible] because it does raise
questions but doesn't [inaudible]?
>> Sheelagh Carpendale: Yes.
>>: Some people love that kind of work, and some people hate it. So it's very
interesting ->> Sheelagh Carpendale: There is. There is.
>>: [inaudible] visualizations that [inaudible] there's a good visualization raises
[inaudible] question. So it's very appropriate.
>> Sheelagh Carpendale: Yes. It is. It is. I mean, yeah. On all this also -- this
is kind of the only kind of slide that I have like this. And it's just partly like the
kinds of things that -- so I want to -- I mean, I want to talk a lot about -- I'll start
with an example about representational transformation and part of the reason I
that I do this is I sort of stumbled across Infovis is it's kind of art in the science
world, you know. Take abstract concepts and you make them so other people
and you can see them. You know, it's kind of like it's almost like some of the
definition of art, right? It's like -- so I kind of really attached to the whole thing of
representational transformation and actually really like playing around with
different kinds of ways like doing exercises to kind of enable that kind of facility.
But largely this is about observation. And I just sort of picked out examples of
work that you will all go, oh, yeah, familiar work that you have seen that comes
from nature, from art, from other artwork, from everyday objects, from ideas and
concepts. Which is part of this kind of like horizontal approach.
And then at the end, I'll end up talking a bit about planned observations because
that's -- that's part of where this really blurring has actually got really exciting for
me is -- you know, I come from the art world, and it's all about observing. But
there wasn't anything -- so this is opposite thing like -- so in the science world
everybody knows you have to be inventive, creative, innovative. But nobody
teaches you how. You got it or you don't. You pick it up on the way by yourself,
right?
And yet, the they'll teach you how to observe or what it means to observe or how
to -- so they'll teach you that part. Of and in the arts world, they'll teach you how
to be innovative, creative. But -- and they tell you it's all based on observation.
But they don't teach you how to observe.
So this is -- so I think this kind of like a nice kind of coming together here of,
yeah, so anyway, I just go through a little bit through these examples. So the first
one is a representational transformation, and so it's -- I was going to just talk
about approximate keystrokes. So the idea behind keystrokes is you know so
much about communication is digital these days, and, you know, we've got this
wealth of emodecoms that are being developed because everybody knows that
it's not personal anymore. When you wrote a hand note, you know, the character
of your handwriting, whether you're in a rush or whether you took care or -- you
know that, was all present in the note. Now, people try to add that on because
it's not in the typing.
So I thought, well, let's just visualize typing patterns, you know, the letters and
the rhythms of how you type because that's got to do with those same kinds of
things. Your presence of mind at a particular time.
So this representational transformation is actually pretty straightforward. It's
based on the QWERTY keyboard, so it probably wouldn't work very well for
people who are doing Dvorak or something like that. Just slightly perturb them
so they use up the screen space better. But essentially the rose of typewriter are
there, the keyboard are there. And there's a line drawn between one letter and
the next. And the fatness of the line depends on how slow it is. The slower it is,
the fatter it is. Very simple representation of your speed and rhythms of typing
when you stop and think and when you're typing fast.
The couple other things that are added in, because back spacing is so much part
of how this all works, so this is a back space. It does take the leaf that you drew
out but it leaves a trace because backspacing is part of typing character. And if
you repeat a letter, it didn't show at all initially, so these little white dots are
repeating it. And they ended up calling this keystrokes because almost
immediately people know where the letters are and how to actually play with
typing and stuff like that. So people start painting with it. People started typing
nonsense to each other, but making little paintings out of the pattern.
So, yeah. These are examples. And I think I actually have -- it's going to run?
Yes. A little bit of a video of just typing.
Okay. So another example. So observing from nature. You know.
>>: [inaudible].
>> Sheelagh Carpendale: What?
>>: [inaudible] that you had [inaudible] the animation, that you extend the final
picture.
>> Sheelagh Carpendale: We actually had it set up -- I don't think it have
actually made it on to the Web. It was intended to, we wanted to make it like a
Web app, and actually what it would do is make a little card so that the
visualization of the typing -- of the message was the top of the card and -- or it
was like a post card. It would print as a post card and [inaudible].
>>: [inaudible] image [inaudible].
>> Sheelagh Carpendale: The final image was static. So you're right, actually,
that would be cool. It was for all three of the students who did it, it was a side
project. It was not, you -- and it was something that, you know, an idea that
came up, and they got all really excited about actually, you know, they did very
quickly, and it was really fun, but then when it came to making it a little bit more
product it didn't happen. Which happens so often. Yes, so now you actually see
most of this -- this is phyllotrees. So, you know, it's inspiration from nature that's
part of that whole thing, you know, sunflower seeds have this gorgeous double
spiral pattern. It's actually got all kinds of wonderful properties. It's provably
optimal. If you don't know how many seeds our nodes you're going to get ahead
of time, which of course the plant doesn't. But actually, you know, in Infovis we
actually don't know either. It's really good to be able to know where to place the
next one without having to disrupt your whole layout and actually know that you
have optimal use of space.
>>: [inaudible].
>> Sheelagh Carpendale: Yeah, I think ->>: [inaudible].
>> Sheelagh Carpendale: It's not actually. I think the artichoke leaf uses a
different -- let me see if I get to the next one.
>>: [inaudible].
>> Sheelagh Carpendale: So this is actually, you know, it's very, very simple
which is really nice because it means like we could have 200,000 nodes and it's
still interactive. It's really, really simple and the first one is placed a certain
constant out from the center, 137.5 degrees, that's the sunflower pattern, you
know, and times two, constant times root N gives you your radius. It's really
simple. So you get all these gorgeous patterns. If you change the angle and
nature changes the angle, too, you get different types of spirals so you could
have kind of characteristic spirals. We did the a really simple tree layout, ala
cone trees, right, but using these kinds of patterns to lay out the children instead
of just a simple circle. And you end up with absolutely gorgeous and fairly
readable trees that show pattern really clearly. So it's actually really kind of
satisfying how gorgeous they are. And I think this one I have a little animation of.
Is it going to animate? Come on. There we go.
So this one I think is actually 200,000 nodes. But structure actually seen though
it is really big, and it is in some ways too big, but structure really comes through
really well. And nature has a good idea, right? So that's phyllotrees. And I think
the next one is yet another tree. So this is an idea from inspired from other art,
so this is from Persian floral patterns. This is actually called Shamseh. It's this
concentric nesting of circles into kind of sun shape is, well, it's the inspiration for
this. And we have yet another tree layout. So here's a Shamseh tree. But step
back a little bit and actually explain what this is. This is actually a tree cut. This
is not a whole tree, so this was, you know, from the science side this is kind of
looking at WordNet, which is huge, huge enough that you never load the whole
thing in if you actually want to navigate it, you load a bit, and then you take a new
part from a different root and you load some kind of bit so you can actually
handle them. Even the parts like if you, you know, if you load something under
artifact, you still have, you know, several thousand nodes, right? I think it's over
75,000 nodes.
So this is a context tree, right? So if you take that node, this is node focus. If
you take that node, it shows children to two depth, it shows all of the ancestors
and ancestors children to two depths. So it's a tree cut. It's not a whole cut, but
it lets you load the whole tree. And now you've actually done this switch that my
students accused me of, talking about it more from a science perspective, from
what the problem is, what it does from a large tree.
You can actually load the whole tree and you can navigate around in the tree
really easily, it animates really well. But it doesn't ever show the whole story. I
don't know, but -- yeah. This is brand new, we just actually sent the camera in to
pay for it on Friday. So it's brand new.
But there's a small part of WordNet under artifact centered on the node hose.
But once again, you know, gorgeous patterns. Once again we actually use the
phyllotactic layout because by looking at how many node we had to put in the
children in one area, we can choose an appropriate angle will actually make
them fit into the same size. So different patterns actually work from different
angles. So --
>>: What are the [inaudible] maybe you're about to illustrate this. What are the
concepts that the [inaudible] illustrate, like what's happening [inaudible].
>> Sheelagh Carpendale: This is -- this dark one? This is the root.
>>: Right, but in the case of hose as the example and fire hose and airline hose
are inside.
>> Sheelagh Carpendale: So do you know how the [inaudible] works?
>>: No.
>> Sheelagh Carpendale: Okay. I wouldn't actually be able to tell you what hose
-- this is an ISA hierarchy, human created of the English language. So you know,
fire hose is a type of hose, a hose is a type of -- and I'm not sure what they
characterized hose as. I would have to look it up. By the time it all the way gets
out, the only other label that we've put on is artifact at the outside. You could
actually but labels all the way down through its ancestors.
>>: Okay. Artifact is the outside word?
>> Sheelagh Carpendale: Is the root of this particular branch. Actually, WordNet
is anchored for everything at entity, that all words are a type of entity of some
sort. That's how they categorized it. It's not an algorithmically done ISA tree, it's
a human done, but computational linguists actually use it a lot.
>>: Thank you.
>> Sheelagh Carpendale: So that was, yeah, for everyday objects. And so
here's something that I know everybody has seen. You probably recognize
what's going to -- what I'm going to show next, right? Which is now not that
knew, we've all seen, but this whole like if you want to actually share, if you want
to spread information around, what do we do in the physical word? We've got all
these kind of conveyor belts and sushi boats and things like so we just thought
that we'd stick it on a table, right? And actually found that it fundamentally
changed how people browsed to this sorted of rapid serial of access. And we
found that actually people doing the same tasks without currents and with
currents -- without currents they divided the task into sorting and assembling and
you even actually hear them while they're assembling saying, oh, you know, we
might have seen a better picture for that. They didn't want to go back to sorting.
And when they had that easy kind of access, they'd switch back and forth, they'd
assemble and then say, oh, no, we had a better picture, they'd go look if the
better picture. So I think arguably people might do better quality work.
From concepts. So this is sort of stepping into the actual art world like thinking I
have an idea that I want to express. So this first one is memory encode. So they
were -- this is two people, Holly and Uta. They were at that point paired. This is
interdisciplinary Uta was one of my students and Holly is a fine arts student. And
they were looking at manifesting memory somehow on the tabletop in an
interactive way, some way that would engage people. So they made all of these
different cells. Each cell has a memory. Anybody who walks up at any side of
the table you actually get a soft keyboard and you can type in your own
memories.
So when it was actually up in the gallery, people would come in and they would
type their memories in and get they are cells. These cells had half lives. So
gradually over time if nobody read them, they'd kind of fade away. So people
would come back and see if their memories were still there, if people had been
interested in them, if there thoughts were still there. Also the cells had the ability
to concatenate, so they would join, merge, and then there would be so
sometimes, you know, different thoughts and different memories would kind of
get elongated.
So it was actually -- it was interesting -- it was actually really interesting to us,
because both in what I'd experienced at that point, going into galleries with
interactive art pieces that I'm going, I'm trying to figure out how to use these -how to make these pieces, like what's happening, what is this interactive new
media art piece and this gallery and really hard to figure out what they were
doing, and I think partly it was because have been had been trying to figure it out
like me, and most of them are broke because people had done terrible things to
them. But it was just like, but nobody was interacting with them. Because
people are used to going to a gallery and just kind of, you know, you know what
it's like, you go to a gallery, you kind of look what's on the wall, maybe you read
the caption, and you go on. And you've got maybe, you know, 30 seconds to two
minutes for a piece, kind of what the average length of time that somebody gives
it.
It's not very long to attract somebody's attention. So one of the really cool things
about this is people wanted to do this. People wanted to do this, they wanted to
come back. In fact, actually we ended up putting double sided sticky tape around
the edge of the table to try and keep too many hands off in the -- to try and
actually put people off to slow some things down because we needed to actually
slow the interaction down and so it's so on the science side it actually started this
whole thing about okay, what is intriguing, what can -- how can we take things
like Infovis into a public space and make it intriguing? What was intriguing about
that? What would actually make somebody walk up and spend 10 to 20 minutes
with it, instead of it's kind of usual two-minute maximum, and can we actually use
that to enrich a gallery museum type experience. So this is actually, you know,
starting a whole direction for me of, like, you know, Infovis in public spaces, like
how does this actually work.
So the next piece was end up being an installation in the Glenbow, which is a
local gallery. They're having a traveling show of Emily Carr, and if you know
Emily Carr as a painter, one yes, she's very well known Canadian painter.
Unfortunately less well known now, but she painted at the same time as the
group of seven. But very much her own style. Show you could see of that genre.
Very much about Canadian nature, but west coast instead of Ontario. So, and
very much about trees. So the first part, the visualization had two parts, and one
part was this kind of cross-section of a tree where each tree ring represents a
decade of her life. Each bubble is either one of her own writings, one of her
paintings, one of her sketches, a photograph of her, a critique of her. You know,
and if some kind of information that would enrich the context of the show that
was there in the gallery at the time. So you could browse through it and see, you
know, other paintings and know what time they came in her life and how that -and how people had discussed them. So that was kind of one-half of it.
The other half is this more conceptual tree which was like talking, grouping in a
more comply type tree type way. The type of discussion that had gone on
around her critically like, you know, some -- there's different colored branches
and depending on what you chose there, this would show that branch that related
to the piece that you showed there. And if you go and you choose something
else in this branch it would actually then open up different things. So the two
kind of little linked. And I have a little bit of a video.
So this is actually at the gallery. You can hear kind of the -- you can't really hear
what people are saying, put you can hear that other people are talking about
what they're exploring. You can see that touching on [inaudible] comes back into
that one.
And there was other -- there was two things about it. So it was a tilted -- kind of
tilted tabletop so you could come up to it easier like that. But what was
happening on the tabletop was also projected really large on the wall adjacently.
So the walking side, we saw walking past it you would actually see the interaction
on the wall. And this was actually fairly well interacted, too. We actually just this
fall published -- we -- and this was there for, what, three, four months? So she
spent hours and hours and hours because we couldn't videotape anything sitting
at the backing taking notes, watching how long people interact with it, what kind
of people. It was different between the ages. So they're sort of like getting back
into making the science.
And this was the piece that was the finalist in the Canadian New Media Awards.
It was one of three finalists. So here we get into this whole planned observations
thing. So here I sort of come starting to do this like well used to thinking that like,
you know, observations is like that's where I get my inspiration from. And here's
this whole community that like, you know, well actually formalize observations.
Because I never actually talked about -- other people about where my
observations came from and where my inspirations came from. And I think
actually probably some of my most cited work at least has been from this kind of
like planning observations and actually watching and thinking and taking, you
know, like seriously looking at data in different kinds of ways with actually even
some of the same kind of terminology that comes right out of the other world like
looking at the data with different kinds of lenses like, you know, understanding
kind of what kind of -- because you can't not bring a bias to it, so you know, like
understanding what your bias is so you can use that as a lens and talk about it
that way and actually see what actually comes up from the data in that kind of
way. So this planned observations or observation for inspiration is totally like
works for me. It's a lot of where I come from.
The first two here that I'm going to just show really briefly is the whole beginning
of the territoriality stuff. We did lots of watching people on regular tables. All
kind of studies. We -- Stacey spend hours going through videotapes, counting
like dividing up into different segments and counting everywhere that everybody
touched, making activity plots and looking at the different kind of patterns and,
you know, relating them to human territory at and looking up developing this
whole kind of way about like how human territory at kind of manifests on
tabletops a little bit different, not exactly the same, but lots of similarities, and lots
of kind of useful information about how coordination and negotiation happens
over space when you have a larger space.
This actually I like this because part of this is like you can observe on so many
different levels you can -- you know, you can observe about the task. So here
they're doing a planning task you know, on how long they're taking at different
kinds of spaces, what kinds of things they do first, do they put where the rooms
first, do they put the big pieces of furniture. You could think about the task,
what's happening in the task. You can learn about the task. But if you to that,
what you would be able to get out of that, and you could probably get some really
good inspiration but you would end up making software that's for floor layout,
because that's what you're learning about.
But so what we were looking at is, you know, tabletop coordination like more than
one person sharing a table and how that works. And so one of the things is like
from counting all of the things, the dots where people would touch across the
table, like far away from themselves, they were really, really few. But they did
exist. But see, one of the really neat things to observe is so see here we have
two examples of people actually reaching across into other people's space. And
look at the other people. Look at the similarity of the hand posture the person in
the red shirt and this person here. There is this kind of moment out of time.
People don't object. It happenings, but there's a kind of pause in activity. And
it's almost exactly the same way they use their hands.
There's some -- anyway, there's something here that it's both like saying it's
okay, but I'm in suspended animation, so then the person who's doing this
reaching is really quick about it. But this information like I'm in suspended
animation, that information doesn't happen in software, and that's sort of what
keeps the person polite. Like they'll say excuse me and they'll apologize again
as they come out, but as they swipe the mouse across or like some kind of
cursor across underneath where people's hands are, they don't have the soulful
cues to help them be polite.
So I think there's a whole -- anyway, there's a whole lot of interest in tabletop
territoriality. I think almost everyone who thinks about tables thinks about this in
some way. Yeah, this is actually the first software that we did pose tabletop
study. I mean, one of the things that we saw is that there was the -- all of the
tables were flexible, and all of the territories were flexible and mobile. You know,
this private and personal group. But there's also these kind of storage territories,
you know, that's where you put your book and your lunch bag when you're
working with your friends. And what we figured was most missing was storage
territory. So that's a storage territory.
>>: Perfect.
>> Sheelagh Carpendale: And he's are people who had never seen it. At that
point hardly anybody had tables because this is three years ago, right. Four
years ago.
So I think we'll go on. I'm going to run out of time here. So this is the orientation
study, and maybe I should go through it a bit quicker. Similar type of study.
People doing puzzles on tables. I very clearly remember after we gathered all
the data Russell coming into my office and going I see people making puzzles on
table. What is there here? Right. But what we ended up like through looking
and looking and relooking is this whole kind of thing about how orientation is
actually used to communicate. But one of the key observations that led us into
this is this particular thing. One of the things that sort of talking about group
territories and personal territories as being established by how you orient the
objects, then we knew there had to be a time when they decided on group
orientation and how did that happen. And we're actually kind of lucky that it was
actually in most of our videos because it was right at the beginning and lots of the
groups actually had decided what group orientation is as they were sitting down.
And you can probably even remember this, like you can say, oh, I'm good at
reading upside down, let's just do it your way.
Like that's usually, it's something that people were very proud of, it was usually
decided almost before they were actually seated at the table. But these guys
here, I'll show you this, because this is so amusing. They were -- the arrow in
this is indicative of what will end up being group orientation.
[video played].
So thank goodness for outliers because they took so long doing it, we actually
could see what was happening. But then you could see when they're talking
about it. And they were the only ones who took like a proactive -- I mean, it was
real and they were laughing and they thought it was kind of hilarious. Most
people do this sort of really smoothly, happened really quickly.
Yeah, so that's -- but this whole orientation was used for coordination,
communication. I see you guys are all well familiar.
So the next thing, this is kind of where we're at right now, and we haven't got at
this point all kinds of data, not all of it analyzed yet and kind of in the middle of all
of this but it's in fact I actually think to some extent you know, like well, Petra has
been here recently, and we're kind of thinking, okay, let's actually do this for real,
like if somebody is actually going to work on the table, what do they need? And
we come back to observing people at work, observing people doing complex
tasks and observing people doing simple tasks, actually going out into the real
world where people are actually working and bringing people into the lab where
we do these kind of planned observations.
And there is a sort of -- and it's still like I say because this part, I think that the
sort of observation for inspiration is kind of already there. We're already kind of
like seeing how this comes together and how you actually can actually do real
applications. But the fullness of the data now this is not there yet with this data,
we're still looking at it. I've got quite a lot of it. Yeah, this is just actually saying
people actually -- these are not from art, but people really have real lot of data
that they still really look at paper, just -- these are just images. This is actually
my two students hamming it up. This is the study actually that really -- Petra
designed. I actually really like this study. She wanted to have -- she wanted
have been to be experts, and you know, you can't get a whole lot of experts
really easily.
So she actually found a data set on manners on where it's polite to run and
where it's not polite to run, where it's polite to burp and where it's not polite to
burp, and you know, we're all kind of experts on this, right? So a bunch of stats
about manners and people's opinions about manners and so we could ask them
some questions about. And with -- because part of the impetus for this is, you
know, we're going to try to put Informational Visualization on the table, all of the
stuff that we have about how people do visual analytics comes from people
already using software. So how do we actually know that that's how people
actually really do it? People -- that's how people do it with software. And
software actually, as we all know, is kind of opinionated. It kind of tells you what
you can do next and what's easy to do from where you're at and so you do it as
you learn the software, you do it the way it kind of opportunity get in your way so
much, right? So we again wanted to go back to pencil and paper, give people
visual information, simple tasks, data that they actually understood and would
find amusing. And just see how they solved it in small groups and how they
actually did the that. And this was actually -- is this your [inaudible] so this is just
recent. And so I'm not going to go into the details of this. But essentially most of
the categories of the different kinds of ways that we group things really relate to
everything else. So that's not too surprising. These are kinds of the things. One
separation that she made that's not common is strategizing and discussing
collaboration because she thought it was interesting for us that we're trying to
make collaboration tools to when they're actually talking about how I'm going to
work with you instead of what we're actually going to do. So that's the distinction
between those two. I think the rest of themselves are the ones that are familiar.
But the interesting thing about this is, yeah, temporal sequence. Or one of the
interesting things. But this is definitely one of them.
So we had groups of one, two, and three. And she had six or eight groups of
each category, so that's quite a few groups. There was no repetition in even
from one group to the next in temporal sequencing of how they would solve the
problems. So -- and it's also really clear, and this is the kind of observation -- you
know, like that the ability to like have insight and then jump over several steps
was how, you know, groups became efficient or you know, like this was -- this
was an important part of being able to work together and being able to work well
together. So this whole notion of actually you got to free up temporal sequencing
is the first place that we kind of looked at how do you actually incorporate that in
software and, yeah, so she's looking at tree comparison things on tabletops.
And I think here I have a little video. Yeah. So here you see these two people
are actually initially working quite independently. And our table's got quite a bit of
space. Each of those is 1400 by 1280, so that's quite a few pixels. And you'll
see where this person here on the right suddenly finds something that's
interesting and can watch -- see, you can see when you're observing, you can
actually see when the moment is that somebody gets interested and they say
something, right, and then the important part, like what we're trying to work
towards, is that it can become seamless from when they're working
independently to when they're working together. And the software kind of
supports both, and it has the flexibility to provide them with the space they need
for both, right?
And, yeah, I have actually kind of blurred back into talking about this from a
science perspective. Part of -- so there's two things that I wanted to end up with,
and this is -- there were all kinds of other examples of projects that I could have
talked about for these kinds of things. These are just a sort of set that I talked
about. But one of the things that, you know, one of the things that I want to kind
of leave with you is this whole idea that we have this kind of fixation in our
community that empirical work is about validation and verification. For me,
empirical work is about inspiration and design. And I really enjoy and find this
hugely rich to do, so the majority of the empirical work that I do before I design
anything to do observation for design. So that's I think kind of maybe my main
message. And this other one, you know, I talked about -- and I've talked about
this all today about working in this transdisciplinary space. I'm actually kind of
working within one self with some understanding of both worlds.
This is -- I also work in a lot of transdisciplinary projects. So these are all student
transdisciplinary projects where it's pairs of an arts and scientist working
together, and there's a whole wealth of other things there. But I do -- I think that
actually this space is just as interesting and just as worth exploring. And that's
actually where I was going to do that.
[applause].
>>: So perhaps I'll kick off the questions. The last slide you showed us may be
as an Infovis person sort of cringe a little bit. For the first time in this entire
presentation you've shown me bits on the screen that aren't data. Everywhere
else, everything you've shown me was someone manipulating manager for
information. And, you know, that was a warm and fuzzy space for me.
>> Sheelagh Carpendale: This is [inaudible] I'd like to talk to you about these
different projects. Some of them actually have some data kind of aspects to
them. But no, these are transdisciplinary art science projects where there's this
whole other kind of discussion that you could have about if you have an
transdisciplinary space what does it actually mean if it's in balance and both the
artist and scientists are equal? Can that actually work in nearly always
somebody ends up taking the lead, somebody ends up kind of being in service of
the other that is primarily science that's happening and the artist is helping them
make better science. Primarily art is happening and the scientist is helping them
make better art, like does balance actually work? If so, it's difficult, it's not
common usually you're in one space or another.
In the transdisciplinary classes that I have run, there has definitely been a
tendency for it to be art projects. I think that you know, by the time somebody
has done five or six years of computing science education and they're in a
situation where they can do an art installation, it's like, ooh, right, and they're all
for it. And that tends to be what happens.
Though there are some interesting things. So this one, this project won the
governor general's award. It's kind of interesting idea and it's sort of data. It's -so the idea was taken from, you know when you go and you get your photo taken
in the mall, so they wanted to make it kind of like a photo booth, but instead of
having your photo taken it was to watch a little video clip. And he said, okay, so
what is it, how do you actually choose what videos, what movies you're going to
rent, what movies you're going to see? You choose them based on your friend's
opinions. So let's not actually give them any title or tell them anything about the
actual movie, let's show them a person's reaction who watched the same clip
before, right?
So on the outside, you can see different people watching different clips and you
can go through seeing different reactions and it's that's and you go and you make
a choice and you go inside and you see some video clip. So they're using the
data of, you know, of people's response as little video clips as part of their art
piece. And it was interesting in our piece annoyed people a lot. But that's in this
space of people wanting to, you know, problematize this whole thing about does
that actually really work, do you actually want to really make your choices based
on your friend's opinions, is this actually a sensible way for you to make your
choices? That's what they kind of wanted to make people start thinking about.
And so they had achieved what they wanted, like, you know, in making people
relook at what's a common practice in society, right? And that was, you know,
their piece. And I would talk about all of them like that if you want. I mean, like
they're all -- they're all -- actually all the ones that I chose on here I actually think
are really quite interesting.
>>: Can you talk about them [inaudible].
>> Sheelagh Carpendale: This one? Okay. So these two, two young women,
they said, okay, what we want to do is we want to make an interactive piece that
encourages contemplation. Right. Okay. So good goal. Go for it. If you could
do it, I will be impressed, right? So they did. So they built this thing that looked a
little bit like a well. They filled it full of water. At the bottom of the well, they
made a sculpture that's a reproduction of you know the type of dome that's in
certain types of temples, Persian temples, sort of a nested 3D star pattern? So
they created one of these, they inverted it so the bottom of their well is this
gorgeous multi curve nested pattern. They had projector up above, projecting
through the water on to this. Several variations of the traditional type of fill agree
patterns that go with these types of domes, and the usual kind of vision, motion
sensor camera stuff happening. So the camera's watching what the audience is
doing. However, what they triggered is if the camera cached any motion, the
animation would restart. So if you want to see the full animation, you have to be
still, right? And, you know, people figured it out, but, you know, and so I was only
actually there at the opening for the gallery for this one. So there's always
somebody new coming up and people going shhh, don't move, shhh, don't move.
And they'll actually be doing this with their hands or something like -- but it was
really interesting to see that people figured it out and were trying, there was a
whole sort of community effort into trying to get it to become enough so they
could see the full pattern. And I thought it was pretty cool. I thought it was a very
cool piece.
>>: [inaudible].
>> Sheelagh Carpendale: Katayoon Etemad and Lia Rogers.
>>: Thank you.
>>: So I'd like to get back to your tabletop work. In particular, the -- what I notice
about your observation was that most [inaudible] based on [inaudible]
prototyping, right?
>> Sheelagh Carpendale: So taken off ->>: It's very nice because you get it started very quickly and you don't have to
worry about implementing things.
>> Sheelagh Carpendale: And we didn't have a table when we started.
>>: Right. Exactly. And but it's a great technique. But -- and so one interesting
thing, a consequence of doing [inaudible] is you might start into your software
thinking about various physics, metaphors for your interactions.
>> Sheelagh Carpendale: Absolutely.
>>: Right. And like on the one hand that's been I think one of the great strengths
of most of the stuff that we've seen in tabletop software is you have this sort of
pseudo tangible aspect.
>> Sheelagh Carpendale: There is some kind of aspect that is close to tangible,
I completely agree with you.
>>: But then when you -- but some of the stuff that you showed after that, where
you actually finish out the software prototype, breaks through that wall, right, and
you're showing these huge data sets, relatively huge, anyway, compared to the
paper prototype, and so you're now collapsing spaces, you're doing all this kind
of stuff, and I'm wondering if you could comment on this transition from -- I mean,
you have to be doing all this stuff when you're on a computer for some reason.
>> Sheelagh Carpendale: Yeah, yeah.
>>: That's one of the powers of moving to these interactive tabletops. But on the
other hand like what -- how do people react, do they -- what's sort of the
consequences for giving some of that away, the tangibility?
>> Sheelagh Carpendale: Okay. First of all, I think that -- I do think that -- I
mean and there is stuff that I didn't show that we're kind of looking at, you know,
four space interaction, I think the stuff that you've been doing is like really cool. I
really like both the last two things that I've seen come out of your group, the
shape, touch, and the bringing physics to the table, and I think this is like a really
hugely exciting direction for table top interaction and that it is getting much further
into this tangible, this in between tangible, and I -- yes, so I actually think this is
great. But I also think that like we do in the long run or I think I want in the long
run to actually or in the long run I guess I even initially thought that I wanted a
table like I did my PhD in screen real estate like in how to make -- how to get big
information on to small screens. So when I had the opportunity to get a piece of
equipment I thought I want a bigger screen on a bigger screen and I want more
resolution. And I thought about that even before I was thinking about like tables,
right. That's what I wanted to do. And because the whole purpose initially was to
get back to big information.
And I didn't do it initially because when we first got our table in 2004, 2005, the
interaction wasn't really there for big information. And I didn't feel like the
interaction was really there at all, it felt like we just needed to start to rethink this.
And part of me actually thinks that going back to big information like we have in
the last year instead of going on with new different types of kind of like four
spaced interaction is maybe the wrong point to have a hiatus in that direction,
and yet for me, it's been kind of like regrounding and the initial goal of like
actually thinking like can I really -- can I at this point actually think about real
applications?
I do know that the biologist that we're working at right now -- working with right
now it's completely flipped around that if we -- if we will give them access to our
table, because we actually have their data on it, it's not for them like a study or
something, it's a privilege to have access to it. They will give us any amount of
time because [inaudible] pay any attention to us, they're like muttering away to
themselves and doing stuff, which is really fascinating for us.
But they -- so I would think that we're a long way away of having what one really
ultimately wants to have in the way of tangibility of interaction on tabletops and
this feeling that you can actually get your hands actually into the data, which is I
think where we actually really want to go. But at the same time, coming back to
actually having big information on the table and seeing that there are people who
actually really want to use it is I mean, you know, I think it's -- I think -- I don't see
where I am at right now as the end point, I think of it as like a reconnection I
needed to make on to information visualization on the tabletop from doing what
was largely kind of HCI on the tabletop.
>>: So it seems like one of the big problems there is how to you take this kind of
intuitive tangible interactions that [inaudible] put them on big data [inaudible].
>> Sheelagh Carpendale: Yeah. Well ->>: Without being a slider or [inaudible].
>> Sheelagh Carpendale: It's a good thing we still ->>: [inaudible].
>> Sheelagh Carpendale: Good thing we still have work to do, right?
>>: What's your sense like when you're showing your work to people, maybe the
biologists are just random [inaudible] but what makes people gravitate to your
systems? Like why do they ->> Sheelagh Carpendale: I actually think ->>: What excites people about these devices?
>> Sheelagh Carpendale: I actually think that the resolution is a big issue. I
actually think that ->>: Wait a minute. The resolution on [inaudible] terrible.
>> Sheelagh Carpendale: Well, it's not on ours. We have a four up with -- so we
have like six megapixels on our table.
>>: Okay.
>> Sheelagh Carpendale: I would still actually like to go higher. But it's getting
close to the resolution. It's actually pretty much what you have on an average
high quality screen. So we have 1400 projectors instead of -- I would like them to
be 1600, but they're 1400 and they each do a size -- a space about that size.
So we have the whole table is quite big. It's about four feet by five feet. But it's
six megapixels.
>>: Do you know what DPI is off the top of your head? I can't do that math in my
head.
>> Sheelagh Carpendale: Me neither.
>>: Okay.
>> Sheelagh Carpendale: It's in some of my papers somewhere.
>>: Okay.
>> Sheelagh Carpendale: Yes?
>>: But [inaudible] because you've done a lot of work for [inaudible] high
resolution.
>> Sheelagh Carpendale: Actually that's not true. We went straight to high res.
I mean, when I actually first thought -- when I first got the money to buy a table, I
went and I looked at some tables. That point Simon Fraser had one, UBC had
one, Stanford had one, and we went to see [inaudible] gang, right? And I came
back actually, truthfully quite depressed, like I can't improve on any of those
devices. And actually had this whole negotiation with the grant agent to let me
build my own. And so the first table that I built was two 1280 projectors on this
same four by five screen. So quite big. But the one I have now is more than
twice the resolution of the first one. But still the first one was like three times the
resolution any of the other ones I saw in the labs at that point.
So we were really actually higher res. So -- and I think that a lot of the appeal of
the everyday person is they come in is pictures are crisp, they look good, they're
pleasant to look at, you don't have any blur, you could read the data -- you could
read the typing. So all of those things matter, right?
>>: But have you done ->> Sheelagh Carpendale: So ->>: Have you done the experiment where you compare it against ->> Sheelagh Carpendale: No, because I don't actually have a low res table.
>>: No, no, no, I was going to say that other possibility in the matrix, and that is
the high resolution and conventional [inaudible].
>> Sheelagh Carpendale: Ah, yes, but that's different. We have actually done
some sort of, you know, not -- I mean some you would say not formal, not
published, but we've done parts of studies of like looking at vertical and
horizontal. And so in vertical, just like John Tang's original paper said, people
tend to work serially, like, even if it's a group of close collaborators they take
turns. There's somebody -- you know, even if they're at a white board so it's not
digital, there's somebody pontificating and then there's other people, and then
there's me a change.
And if you have a table, everybody thinks it's a free for all. All hands go on at
once. You know? And it happens immediately, it happens in public places.
When SMART had their table at CSCW 2006, you know, David Martin is there
actually, you know, making some pronouncements and you know he's walking up
and putting his hands on the table and doing something ->>: [inaudible].
>> Sheelagh Carpendale: No, right in the middle of like everybody feels like this
is a table, tables are for everybody, tables are a kind of unifying, so it's different
when it's a table. It's like equals the playing field, it levels the playing field
literally, right? So I think there is that about tables.
I do know that -- I mean, I know that the community kind of goes around saying
where's the killer app, where's the killer app, and I've been kind of always sort of
perplexed about this, because my experience when people come into our lab the
they all say I want it, I want it now. Even some of our like really kind of not very
together demos, right? So there's -- I think that -- I mean, I do think that a lot of
the demos that are out on other tables, the interaction's a little bit different. It's
very -- we're a kind of -- we're in a kind of equivalence class in some ways, some
differences here, some differences there, so it's not like I think that our
applications are that different. So that's why I kind of didn't want to say, oh, we're
doing these absolutely -- you know that's not true. There's -- you know, we're
kind of like, you know, the tabletop community does like all the rest of the
community like this -- you know, it sort of does this. But we're kind of all coming
up together looking at much of the same issues.
So I guess that's why you do the this sort of immediate response. Well, one of
the things that is different about ours is the resolution. SMART doesn't like it
when I say that either. They kind of go like -- I mean, it's difficult to do, it's
unwieldy. You know, we have one in our lab that don't have -- you know, like it's
not the -- it's not the response they want to hear either. They would like me to
run a study and prove that I'm wrong. And they -- and you know. Yeah. Though
I think that.
>>: [inaudible].
>> Sheelagh Carpendale: That's right. Because if you could actually show that it
didn't matter, it would be much easier because actually, you know, like the
surfaces actually now, I mean, I really -- I think it's really nice. It looks good, it's
smooth, it's, you know, it's actually -- we actually finally actually have all the
touch that you want. There's a whole bunch of like really, really nice things about
them. But the resolution is low. And it's small. I want size. And I want
resolution. But I love the multi touch. I think it's -- you know, I think it's fabulous.
And I think that all the stuff that you're talking about with physics and like, you
know -- I mean, what we really -- what the killer app to me will actually end up
being is that people really can feel like not that their data is like this long way
away from them, but that they have their hands right in the data and they're
manipulating it, and they haven't had to go into a cave where they've got all this
encumbrances and they're tied and tethered. And actually, you know, I think
caves have the same problem. They're all low res. You know. But maybe not
all. There are some now, these big huge megaprojects that I think have lots of
resolution, so I'm not going to, you know, like they exist that's for sure. They
exist. And I haven't been there. I should actually try to get there because
supposedly they're fabulous at absolutely amazing experience.
But I actually think, you know, I think the resolution is important.
>>: Thank you.
>> Danyel Fisher: If there's no other questions, we'll thank the speaker once
more.
[applause]