>> Andy Wilson: So thank you guys for showing up. It's my pleasure to
introduce Steve Benford here today. He's actually here for today and tomorrow.
Steve is a professor of collaborative computing in the Mixed Reality Laboratory in
Nottingham, and he is the director of the EPSRC funded doctoral training center
in ubiquitous computing for the digital economy. And he's received numerous
best paper awards over the years. And has coauthored a book. It's actually right
in the back, if you want to check it out, Performing Mixed Reality. He told me he
has a few copies.
>> Steven Benford: It's actually one.
>> Andy Wilson: One copy. And has recently been awarded a two-year Dream
Fellowship by the UK's Engineering and Physical Sciences Research Council,
and I believe he's traveling under that funding at the moment.
And I think the most interesting thing about Steve's work is that he manages to
do lots of really interesting -- interesting performance based work and artistic and
creative performances while also making some really interesting scholarly
contributions on those works. And I the think that's the -- that's the trick, and
that's the really valuable thing.
>> Steven Benford: Thank you.
>> Andy Wilson: Thank you.
>> Steven Benford: Thank you very much.
[applause].
>> Steven Benford: Well, thank you very much for your introduction and thank
you for very much for inviting me to talk to you as well.
So, yes, for the past 10 to 15 years myself and people in the lab seem to have
fallen into a mode of working with artists, particularly those with a background in
kind of theater and performance to create and tour and study a series of kind of
interactive performance works. And to use that as a sort of a vehicle for driving
forward our interests in human computer interaction.
And today I'm going to draw on a kind of -- at least one of the examples of that
kind of work and try and pull out some general principles, perhaps, for how we
might design what I'm going to argue is kind of the sense of the extended user
experience.
Before we get into it, though, I'll just say a kind of few quick words about why we
choose to work with artists in particular in a way that we think of being artist led.
And what that means is very often it's the artist who come to us with the idea for
the project they want to do and we kind of figure out how we can help them
realize it technically and then we try and study it and figure out what went on.
And so why do we do an artist-led research? Well, I think there are three
reasons. One, artists are extremely creative, certainly compared to ourselves.
They have ideas that stretch technologies and do interesting things with
emerging technologies that we just wouldn't have. So that's number one.
Number two is public performance is a fantastic way of conducting research in
the wild, of getting these technologies into the hands of real people, albeit under
somewhat unusual situation and figuring out what happens and if you are kind of
working I think in highly regulated fields, medicine, transportation, whatever, you
might struggle to do that.
But many of these projects we've turned around from meeting the artist to rolling
up the project, you know, in kind of two to three months, it's as quick as that. So
it's a very flexible thing. And thirdly, and most importantly, this stuff is just
important. Creative and cultural uses of computing are extremely important thing
economically in terms of the creative industries and to us as human beings,
expressing ourselves and being creative is a core human value that we need to
support. So, dammit, it's just important.
So as I say worked I guess 10 to 15 years, probably with somewhere between 6
and 10 artist groups, probably created in that period 20 or so different works.
And here's some of the bits of work that I'm not going to talk about in this lecture.
One of our longest standing collaborations is with a group of artists called Blast
Theory, who we've worked with since kind of 1997. And the book documents
their work and the work we've done with them quite extensively.
So can you see me now is a piece of work in which people on the streets of a
real city who are actors chased people who are online in a virtual model of that
city and their audio is stringed out online as well, and the online players could
lead them a sort of merry dance through the city.
And we studied that. Our approach to studying these things in the wild is to use,
here I go esthematologically informed ethnography. I practiced that for many
years with Dr. Andy Crabtree hitting me with a stick until I said it right. And that
revealed all sorts of interesting things. In this case, it revealed lots of things
about how the scenes in the technologies, the partial coverage of Wi-Fi and the
flakiness of GPS had to be accommodated in how the experience was
orchestrated. And that was interesting.
Another Blast Theory work. I have spoken up here before. What I'll be talking
about today was day of the figurines. This was a month-long text messaging
adventure game for mobile phones. But you played very slowly. You only sent
and received a few messages each day. So the game was kind of -- was
interwoven or became interwoven with the fabric of your everyday life. There
was also a rather bizarre framing interface was this kind of tabletop interface
where the figurines would move around that was in a venue that you might only
ever see once during the whole month at the beginning. But somehow framed or
gave you a sense of what the experience was about. And we published -- we
published work about the episodic nature of the play and engagement with a
slow game. We managed to write a bit about the design of that table. But I'm not
talking about that today. 2008 we began a series of collaborations with Brendan
Walker. There he is. A bit of a TV personality in the UK. And thrill engineer. He
would start himself out as a part-time roller coaster designer, among other things.
Brendan first came to us because he wanted a series of technologies that would
in essence the spectator experience of roller coasters. So we built a telemetry
system that took video and audio from these head mounted cameras very close
up, quite intimate video of people's faces. Heart rate, galvanic skin's response.
And in later versions the movements of those two facial muscles that I can't
remember their names for smiling or frowning.
And the system didn't interpret this stuff but rather it was displayed for spectators
to see on large screens. You had a very close-up view of someone else's ride
experience.
And then more recently we've worked with Brendan to think about how you can
use some physiological measures to actually start to control ride experiences.
And we built a couple of rights in which you use breath or breathing to control the
ride. And breathing is very interesting because it's only under partial control. At
times you can control your breathing remarkably well. It's kind of what I'm doing
in being able to speak. At a other times you can't control it at all and you have to
breathe after a certain amount of time. And if you build a ride that responds to
breathing, you can bring people to a really delicious tipping point between control
and loss of control. And after all, what are amusement roller coasters about if it's
not about giving up control to something that's where a lot of the thrill lies.
In breathless, which was a powered swing, we went as far as to put the breathing
rest operator sensors inside a rubberized gas mask, so just to really up the ante
and make you tune into your own feelings about your breathing, you were
wearing this gas mask and controlling this powered swing.
So those are four bits of work that I'm not going to talk about. But I could have
probably picked any one to talk about this notion of trajectories that I'm going to
come to later in the talk, because they do all reflect it. But I'm going to just pick
one piece of work, and apologies if some of you have seen it before, because it
is quite an old one. It's 2003 now, which is pretty much ancient. This was what
-- a Blast Theory piece of work and it just -- I guess because it's quite a complex
structure it just serves to illustrate many of the points that I want to make later on.
So we'll quickly have a look on Uncle Roy all around you as a nice example of -well, I guess we're kind of calling the genre of mixed reality performance. Mixed
reality performance is a phrase, is really meant to convey a couple of things.
Firstly, the idea of performances that somehow makes the real and the virtual,
whatever that is, which Uncle Roy does, and, secondly, the idea of mixing digital
content and media with live performance from actors, possibly, or from the
participants who become actors in some sense.
Anyway, Uncle Roy all around you is a performance game that takes place
parallel, in parallel of the streets of the city and online. On the streets of the city,
you arrive at a venue. Your personal possessions are sort of taken away from
you, and instead you're given a device and asked to head out into the city and
find Uncle Roy. That was your job. And you followed a series of quite
ambiguous sort of location-based clues that led you on a dance through the city,
eventually to an office that it was claimed was Uncle Roy's office and you were
asked to go in, and from there to a phone box and from there to a car, and you
were asked to get in the car and various things happened to you.
And there's an online player. You could have the same experience but from a
different point of view. You could enter a virtual model, somewhat fantastical of
this part of the city. You could see where the street players were. You could
chat with other online players. But most importantly you could find out things that
these people didn't know, in particular where is Uncle Roy's office in the city.
And once you got that piece of information, you could try and build a relationship
to a street player by sending them messages and getting messages back and
guide them to Uncle Roy's office if you wanted to be helpful or mass them about
and get them lost if you didn't. At some point in the experience both kinds of
players, slightly different points, are asked a series of questions about whether
they were trusting strangers and to what extend they were trusting strangers and
would he be prepared to enter a year-long contract with a stranger in which if this
person ever calls them up, they would give them help. And if they say yes, we
take their contact details and we change it with someone later on in the game,
and they get a post card two weeks later that says if you need help, call this
person.
So I've got a bit of a video of Uncle Roy. Find it. It's fairly long. We won't look at
it all. But just to give you a bit of the flavor of the work. It takes a couple of
seconds to kick in.
>>: Uncle Roy All Around You is a game that sets online players alongside
players of the streets of the city. Equipped with a hand-held computer street
players are sent out into the city in search of Uncle Roy. Online prayers explore
a virtual model of the same area of the city. They can follow the progress of
street players and communicate with them via audio and text messages. Each
street player has only 60 minutes to find Uncle Roy. Online players can uncover
the location of Uncle Roy's office and so guide street players to their destination.
>>: This is a game in which you walk around the sprawling area [inaudible] this
map shows the game area. To move around it, [inaudible] the icon and drag it.
This is the microphone. You'll need to speak into this to record messages to
online players.
Your first instruction is as follows: Head for a location in the park. Uncle Roy will
send you a message indicating where this is. Once you get there tap on
[inaudible].
I'm doing well. I need to wait for someone coming from the opposite direction.
Turn and follow them.
>> Steven Benford: I'll just pause for a moment there. I mean I mentioned that
the clues were ambiguous. Give a longer talk about the nature of ambiguity in
interface design which a number of people have written about recently. Very
interesting example of ambiguity here. So I think the ambiguity here in part lies
in the -- in blurring the frame of the experience. So the frame of the experience
refers to having an understanding of what is within the experience and what is
without. It's the set of often unwritten conventions by which you know that
something is fictional or not which you're watching it, for example. And in this
case, this clue kind of plays with that. The idea that someone's going to walk
across the bridge in St. James' park and you should turn and follow them invites
a whole series of quite difficult to interpret possibilities here. Should I just turn
and follow the first stranger who walks across the bridge while waiting for an
actor. How will I know what they look like? Is everyone in London an actor who
is employed in the game? Who knows.
Anyway. So bit of an aside but that's what I kind of meant by ambiguity. And it's
an interesting tactic again for generating a bit of thrill and bit of excitement at
perhaps some risk.
So this video rolls for a while. Let's pick up Jen a bit later. So she finds Uncle
Roy's office. Or he does first. When she enters the office, the online player gets
to see her for the first time on the kind of video surveillance link and he gets his
interview about trusting strangers and contracts and things. Watch. He's asked
to look around the office, write a post card and do various other bits.
Then she darts into a phone box where she gets a phone call and from there to
the car.
>>: I would like to ask you a few questions. And I'd appreciate it if you would
answer the questions as honestly as possible.
>> Steven Benford: Okay. I bet. So and again ethnographic studies of Uncle
Roy all around you revealed all sorts of interesting stuff about how it's designed,
put together, how the technologies worked, all manner of things if a number of
papers flowed.
So over the years roll out a number of projects such as these, conduct a number
of studies, publish the rules. That's well and good. Academic life continues. But
you do begin to wonder slightly about the bigger picture and what we began to
wonder was what -- there's something interesting about these experiences.
Okay. We're working with performers who are definitely at the extreme end of
creating certain kinds of experiences. I'm not suggesting that Uncle Roy All
Around You is a mainstream game. But our view is they were -- they were kind
of pushing the envelope and taking environment entertainment or cultural
experiences into some new place. And we wanted to get a bigger sense of what
was it these experiences are about and how are they put together? In other
words, we wanted to sit back and do a bit of theorizing as much as anyone in our
neck of the woods does theorizing.
So this is what we started to figure. First of all we were struggling to characterize
this experience in terms of any of the kind of existing paradigms that are out
there. There are lots of paradigms, there's virtual reality. And that augmented
reality and mixed reality. But Uncle Roy All Around You seems to be
simultaneously several of those at the same time. There's tangible computing
and mobile computing. But data figurines had both tangible and mobile
interfaces. There's location based. There's this and that.
What's interesting is the artists don't really seem to care about that. They take
whatever is the hand and deliberately mix it up to create something else, these
experiences are online and on the streets and tangible and mobile and whatever
all at the same time.
So I think what characterizes them at first glance is two things. First of all is to
some extent they're extended. Of course it depends on what you mean by
extended. I mean, they're not extended yet into years. They're not life long. But
they're extended beyond the moment of interaction with a single interface. If you
were designing -- if you're an HCI person, you're no longer in the bills of
designing how you interact with one thing but rather how many things are
stitched together into something that's larger. And I would call that a user
experience.
I know that UX has become synonymous again with the design of the one thing
with the interface. But my sense is there is a bigger motion of having an
experience in Uncle Roy for an hour. And the structure of that experience is
pretty complex. It's got real and virtual spaces. Sometimes appear to be
overlaid. Sometimes that appear to be juxtaposed or somehow adjacent it's got
complex time scales. I won't talk about those. There isn't time.
It has quite complex role structures. There are online players and players on the
streets. There are act OS, there are people behind the scenes, there are
members of the public who are somehow involved as spectators or bystanders.
And it has a whole bunch of interfaces.
So that's not a very exciting conclusion in many ways. It just says well, they're
extended and they consist of lots of stuff.
So what we wanted to do is start to find of way of articulating what we thought
the artists were doing and what fundamentally these experiences were about.
And for that, we eventually turned to a notion of trajectories.
And so this part of the talk I'm going to make an argument, being a computer
scientists abstracting away just a few constructs providing a framework that all
experiences at least of this kind, can be described in terms of trajectories and oh,
that's it. Well, not really. And that there are just three fundamental kinds of
trajectory and oh, that's it. And I'll try and convince you of that that.
So let's have a good. So firstly the unifying principle of these things I think is of
taking people on a journey. That's actually quite important. It's not the
destination that matters. Uncle Roy and many of these experiences it's not about
reaching an end point. It's not a task or a goal. It is the journey that matters. It's
having a good experience on the way all the way is what you're trying to design
for.
And they can be described in terms of trajectories. Why is a trajectory a good
metaphor to use? One thing that we feel it captures is a set of continuity of
experience. We'll come back to that later on. I've put ideal because there are
episodes and breaks sometimes deliberate and sometimes unavoidable. But
underneath all of the things we looked at, it felt like there was something that was
meant to flow like a thread of experience.
Secondly, trajectories inherently capture the idea of there being individual roots
through experience. You have ours, I have mine. Trajectory is something that
could be steered and shaped by external forces. Perhaps not so overtly, but it
can be bent and moved around in certain ways. And that's something that I'll
come back to later on that we felt was saw going on.
Trajectories are interwoven with one another, the threads of experience to create
some kind of social fabric. It's a kind of -- I'm mixing my metaphors. But that's
interesting. We wanted something that captured the way in which difficult
people's experiences that sometimes overlap, sometimes not and would
interweave as part of a larger hole.
But frankly, also, it's not really our idea, the notion of interactional trajectories is
already out there in the HCI literature. And of course the notion of trajectories is
everywhere in all sorts of fields from physics to social sciences.
But in particular several people already published about the idea of there being
interactional trajectories. For example, Christian Heath and colleagues at Kings
College, London, has studied museum interactions and had pointed out how one
person's interaction with an exhibit often serves to configure the next person's
root into it an interaction with it. Christian has an example of somebody
interacting with a bolt on the frame of an exhibit in the science museum. Not
actually the exhibit itself. I don't know why they're interested in the boat.
Perhaps they're an engineer or perhaps they're looking to make a garden shed at
home on something. Who knows? But they interact with a boat and they head
on.
And of course the next person who comes in interacts with the boat and heads
on and the next person who comes in does the same. It's a beautiful example of,
you know, how there is a trajectory into an interface and through it, the moments
you feel like interaction begins quite a long time before you pick up the mouse,
press the button, are at the table or whatever. So that was already out there.
We just wanted to push it a bit further and to think about how it worked in the
large.
So okay. I said before, there are three kinds of trajectories out of which such
experiences appear to be made. And the first is the canonical trajectory. This, if
you like, is the plan, the designer's artist's performer's intent defined mostly
beforehand, mostly, and probably expressed in various ways through scripts, set
designs, lists of instructions and of course through code in computers.
I've shown the canonical trajectories being just sort of one line that flows through
the piece. But actually any experience may have several canonical trajectories.
Uncle Roy All Around You has at least a canonical trajectory for street players
and a different one that's designed for online players. Moreover, of course, these
things can branch and merge again in experiences that can steer you down
different routes.
So for example at a top level the canonical trajectory for Uncle Roy All Around
You looks something like -- starts at the host venue. This is for the street player.
There's a different one for online players. The Red Spot moment was something
that the artist talked about time and time again as being your first successful
engagement with the game, with the interface, with the clue trial. If you can do
this basic one, we know at some level you're on track or we've got you, you're in
it.
Blast Theory works with Uncle Roy mark out a number of anticipated routes to
the city. And they really did this. They documented it. I have the map. They
walked the city. They decided which routes they expected players to take. And
they made the clues steer them towards those routes.
So there was before it ever ran a sense of what the routes through the city ought
to be. And if people were on one of these routes, they were on track. And if
people were not on one of these routes, they were not on track.
Then the stuff that happens at uncle Roy's office, the phone box and the limo.
And you could -- it's a multiscale thing. You could probably take any one of these
and you'd find a trajectory could probably take any one of these, and you'd find a
trajectory through it. There is a trajectory through most definitely the interaction
in the limousine. It's scripted and rehearsed. What's meant to happen in there.
There is a trajectory of what's meant to happen at the host venue. You saw on
the video the person handing over the PDA and explaining how to use it. Well,
that was scripted and rehearsed and our observation showed rescripted and
rerehearsed until it was done just right.
I talked about ideal -- the idea of continuity. But in practice trajectories have to
negotiate various transitions. And these are key moments at which if you like the
whole enterprise is at risk. And what our studies showed is there were various
kinds of key moments and that the artists in their craft pay particular attention to
how to design those, get it right you may be able to create something that works
more or less continuously. Get it wrong and the whole thing may fall apart.
So what are they? Beginnings is a kind of an obvious one. But the initial ritual
and the initial framing of the experience, how someone's introduced, how the
mood is set, how the rules are understood is absolutely critical. And so you see
a lot of effort designed into this purpose-built briefing room scripted, rehearsed
briefings and so on.
Anything that involves handing over an interface is a really critical moment for
several reasons. I mean, if you've ever about in a virtual reality cave with some
kind of hedge tracking thing that one of the typical things that happens is you
kind of stand in there. And they -- as they put it on your head the graphics are
turned on. And the whole world is kind of jumping about like crazy until it's only a
head and it settles down. That's an example of a problem of handing over an
interface.
And sometimes that's designed by the artist to be minimal that laid the easiest
transition. How can I give this person the PDA they've not used before and get
them into the game? Sometimes the opposite. Brendan would make a great
fuss about attaching sensors to people's bodies before they go on the rides in a
very, very ritualized way to keep building the anticipation. But in each case it's
planned and it's thought about. You don't just hand it over and say get on with it.
Episodes. If you do have an experience that is spread over time, perhaps
deliberately data figurines you only play ideally the plan was you only play a bit
each day. You really have to think about how to reintroduce people back into it
quickly. How do you catch up with the stuff you've missed and get back into the
game within -- well, as we're using text messages, one text message with 160
characters to reintroduce that person back into the game. What have they
missed. Get them back into the position.
Physical virtual traversals for those who are interested in virtual worlds, how do
you get into a virtual world? And there are various ways of doing this. My
favorite was from a piece of work called desert rain when the artist had a screen
made of falling water. And you too actually pass into the image and you would
enter another piece of physical set or the performers would appear to emerge
from the image.
Physical resources such as Uncle Roy's Office to limousine are very different to
digital ones. And one of the fundamental differences is you cannot reproduce
them at will. If two people turn up at the limousine in Uncle Roy at the same
moment, it is a disaster. The whole illusion that you're in this fictional world
created just around you is broken, and even worse, they start chatting and
relaxing and comparing notes and going completely off track.
So the artists would work extremely hard to avoid two people ever turning up to
the limo at the same moment. They would delay some people and push other
people on. We'll talk about orchestration in a moment. To manage that moment.
If it was a digital resource it could be different. Could you replicate the office as
many times as you like, no problem. But as soon as it's a physical thing, one
table, one exhibit, one painting in the museum, one office, you have to think how
to imagine that.
How many of you worked with GPS? But it seems to me every time we do it and
every time we work with someone new, we say it doesn't work like you think it
works and they go that's right. Then we go outside and then someone stands
around for 15 minutes trying to make it work and it's the whole thing's a disaster.
So it does help to recognize from the outset that the seams in the experience, the
bits where the technical infrastructures aren't so quite knitted together you've
really got to be aware of those and design for them. If you leave somebody
standing in the middle of London with no connection, no way of knowing where
they are, no way of them knowing was to do, you've probably got a broken
experience.
And last of all, endings. And in particular, one of the things we noticed certainly
with Blast Theory there would very often be a sort of coder for the experience.
There would be some kind of thing that happened after what you would have
thought was the end to just kind of take you back to it. In Desert Rain they left a
box of sand of your pocket of your coat which you would find anything from five
minutes to two weeks later, depending on how you use your coat. Uncle Roy
Around You you got the post card sent to you through the post. But there was
some kind of little moment that just kind of took you back to it.
We won't talk about the transition this is Uncle Roy. There isn't time.
Okay. The second kind of trajectory. Participant trajectories. If -- to use Lucy
Suchman's words, if the canonical trajectory is the plan then the participant
trajectory is the situated action. It is what happens, what somebody actually
does during the experience not what was intended to happen.
And the most obvious thing to say about that is it can diverge from what was
meant to happen. In fact, you might argue that if you have an interactive
experience that's kind of inevitably what's going to happen, giving people the
choice, particularly -- if it's set of one of those branching narrative things there
may not be much choice. You end up with a canonical trajectory that just
branches a lot and comes back and the only real choice is which branch do I go
down next?
But as soon as you put someone in the real world if the city, on the streets, faced
with all of the things they could do, then there is real choice. They can turn
around and walk if the other direction because they're lost or bored and given up
and a divergence happens.
That's okay. But there is an opposing force which is that of orchestration. And
this is the set of processes, tools, procedures that you can use, you as people, or
perhaps you as the system if you can figure them out well enough to code them
and to push that line back to that line or perhaps to some other canonical
trajectory. Maybe they're so far off the original track you have to push them on to
one of your other tracks.
And again, our studies showed that time and time again this were very, very well
thought-out processes for how to do this. I mean, just one example, this is what
we saw with Uncle Roy. So this is the bit of the canonical trajectory where you're
going through the park on one of those -- ideally in one of those three planned
routes. And here's what the artists thought they should do if somebody -- if their
participant trajectory started to diverge. First of all you would get? Clues that
kind of subtly tried to push you back to the main trajectory. By subtly, I mean,
essentially they were still within the voice of the experience. They were perhaps
a bit less ambiguous. But it still felt like they were Uncle Roy's clues.
And then last year we had authored around the edge of the zone a series of
further clues that were much, much more explicit and clearly kind of broke the
voice of the game and said basically turn around and go back the other way.
And if you still continued, they would start to worry and try and figure out where
you might be and send somebody on the streets. They have about three or four
actors on the streets to deal with this, to try and find you. It's not that easy in this
city, but that have your photograph and that have a sense of where you were last
seen and when. And they try and track you down. They come up to you and
they try a misty speech in the voice of the game, be a bit mysterious, if you would
listen, that kind [inaudible] turn around, go back the other way.
Not many people in Uncle Roy were lost. We could talk about why. But I guess
if that fell down then you really had a problem. And there are all sorts of
technologies that were designed to support that. So this is a screenshot of the
uncle Roy orchestration interface. There's a map with the last known positions of
people. The most critical bit on the interface actually is that disconnection
history. That's the thing you really need to keep an eye on.
People who are kind of connected you can be pretty confident about they are
where the system says they are and you could talk to them if you wanted. But
people who are suffering from disconnections you've really got a problem
because you don't know where they are and you can't talk to them. So you have
to get them early.
>>: I'm sorry. Is each one of those lines a person.
>> Steven Benford: Yes, each one of those lines is one of the current people in
the street play. They would cycle the street players through. So you would have
more people coming into the games. It was a kind of rolling experience.
>>: So that green bar is showing where they are on ->> Steven Benford: The green bar is some kind of temporal representation of
their -- has their device been connected back to the game. Have we been
getting lots of messages and red's indicating some periods of disconnection.
[inaudible] what white is. Sorry.
And then the color coding, the map is kind of where they most recently were. So
we're still on participate trajectories. The other thing is to think about the social
design of the experience. And this is interesting too. So this diagram shows the
peoples participant trajectories can cross, they can diverge, they can come
together. In designing these experiences, again observations showed there were
kind of several key aspects. So first of all, many people when they design
collaborative systems design for this moment. It's all about encounters. What
happens when two people get together? Perhaps they meet in space and then
they can collaborate. Perhaps they meet in time but at different spaces. And
then they can collaborate. And, you know, a lot of attention in CSCW, Peter.
What kind of technologies can make this happen? Good.
This, however is really, really important. Nearly all of these experiences did
seem to involve moments of deliberate isolation of getting people apart from
other people and on their own I guess set of inwardly focused, maybe lost and
confused. Not distracted by having family members or friends present. And this
seemed to be a tactic that came up time and time again, Uncle Roy, December
certain rain, data figurines. There were moments either in the real world or in the
virtual space where you were kept apart.
And this also turned out to be a real problem. And the experience where you're
mixing that kind of physical and the virtual, so you've got this thing connection
between people or where it's episodic and a people dip in and out, keeping
people together once they formed a relationship is really hard. And so you need
to design for that. In Uncle Roy All Around You you had to be remarkably
tenuous as an online player to keep hold of the street player for long enough to
get them to the office. It was a real achievement if you could overcome all of the
potential transitions essentially.
And last of all, the historic trajectory, third kind of trajectory out of which
experiences are made. Historic trajectory supports the retelling of the
experience. So it's not what happened, it is what you want to say to somebody
else happened which is a very different thing. But it's really important to have it.
I guess on pretty much any experience we have we want to talk about, anybody
whose on Facebook knows that. But a lot of the experiences we have, cultural
ones, tends to be quite objectively shared. If we go to the cinema, okay, I know
that in your head you may interpret the film differently than I do. But let's be
clear. We saw pretty much the same film and we have a pretty good sense of
what the other person saw.
If you do Uncle Roy All Around You, I have no idea what happened to you, you
have no idea what happened to me. And people would come back from those
experiences, whether it had been isolation and time and time again say we need
to talk to somebody else, we need some way -- or I need to sit down with
someone and talk about what just happened and compare what happened to me
to what happened to them.
So historic trajectory supports this. They synthesize the view of what happened.
And this means four things. Firstly, you need to in some way document or record
the participant trajectory. That means the actual what happened. That might be
the system recording all of the messages that go through it. It might be people
taking photos. It might be whatever. But you need a pool of documentation.
You have to select and filter segments from -- well, if it's a repeated experience,
possibly many recordings. Put those into some order that suits the story you
want to tell and then find a way of publishing them, making them available.
So imagine it's -- imagine you're playing a computer game with, I don't know, how
many levels does a modern complex computer game have? Quite a few. Some
10s. And you repeat each level lots and lots and lots and lots of times. Imagine
that the system, the game records your interactions, quite a lot of 3D games can
do that so you can play them back or compete against yourself later on or
something.
Then the interesting question is, if I wanted to ask the question what happened,
say to the system to me what happens when I played whatever I was playing, it
could filter all of those recordings in all sorts of way. It could take my most recent
attempt at each level. It could take my best attempt at each level. Or if I'm telling
the story to Jonathan, it would take the versions where me and Jonathan are
both in the game because surely that's the one he's interested in knowing about.
So basically all I'm saying is you don't just replay the experiences happen but
you have to give people tools to tell the experience. Now, I don't have from
Uncle Roy a good example. It's an interesting, if you like, problem, perhaps even
weakness with all of these experiences that we made that there often weren't
good ways of retelling them and replaying them.
And so I'll now turn to the final bit of the talk, a quite different project where we
did see canonical and participant trajectories at work outside of a kind of
performance context in a much more main stream setting, but in particular we
were able to explore this idea of the historic trajectory. And the setting here is
the amusement park or the theme park. And in this case for two or three years
now we've been working with Alton Towers in particular and Thorpe Park which
are kind of two of the UK's bigger theme parks.
In general looking at technologies for entertainment, so some of the ride design
stuff we've done with breathing and those things have been kind of in partnership
with these folks. But also we looked -- there's one part of it at souvenirs and
story telling. And this kind of told us some quite interesting things about
supporting the historic trajectory.
So a change of setting. Let's get to the theme park. And we'll just spend a bit of
time thinking about how people might potentially construct their stories of a day in
the park. The first thing we did was an ethnographic study, a sensitizing study
where we followed a number of families around the park. We looked at how they
treated the souvenirs that were available in the park, how they talked about them.
We looked at their open photo work. We visited in their homes afterwards. We
talked to them about how they use those photos and what was on display. And
that uncovered a number of interesting things.
I mean, firstly, the theme park is rife with souvenirs of all sorts, mugs, T shirts,
whatever. But in particular with photo souvenirs. So you probably know. You go
on a ride, you come off the ride, probably, if it's a big enough ride, a system has
automatically taken your photo in an attempt to capture the picture that nobody
else can, and it may transcend you that photo in the case of Alton Towers within
two minutes of getting off the ride past the kiosk you buy then or you never buy it.
That's the way it works at Alton Towers. And then you get it mounted. But also
people of course take their own pictures or try to take their own pictures too.
So, without going to giving too much detail, our studies pulled out a number of
interesting things about this construction historic trajectory. First of all at Alton
Towers a common theme, thought the only theme of stories, but a common one,
was the right of package. Alton Towers is typified by having quite a lot of thrill
rides and people go time and time again. And as the kids grow up, one of the
thing is doing your next big ride in use of social bonding being part of the family,
you know, everyone's done the big ride. Now it's your turn. That's quite a scary
experience.
And as a result you see some contested stories. So this is the on ride shot of
Jamie. I don't know if you notice anything particularly striking about that photo
but in my opinion, apologies, Jamie if you're watching, because I know you don't
agree, because you told us, I think he looks a bit scared. Don't you see?
Certainly Jamie vetoed his parents buying this picture, absolutely vetoed it. He
just would not have it. Jamie was very happy to take a metal to school and tell
his school mates about how he had ridden Nemesis. But that was half the
question. The mum wished she had bought the photo and spent the rest of the
day and he was still talking about it when we visited in her home about regretting
not having the photo. For the family this photo was what the day was all about
and it was a -- and even then Jamie got ribbed throughout most of the day about
every time he appeared on one of these things he look terrified. And he said it
was the G force that was pulling his face. Maybe it. I don't know enough
physics. Who knows? All I can say is it's a contested story. It won't come down
on any one person's side.
Jamie did rather like his grand dad's photo it has to be said. Okay. So his grand
dad's photo shows him having conquered Nemesis, and that photo was just fine.
The other thing to say about the photo work of course is it's a routine part of
being in the theme park. Often spectators capture the ride experience, so that is
a classic role for the spectator is to be the documenter of the ride. Sometimes
photos are taken of other things. Jamie really liked bird life. He spent lots of the
day taking pictures of birds. And there are of course key photo opportunities
around the park.
So to cut a long story short, as a result of these studies we built a prototype
system, if you'd like, to generate the historic trajectory, to help people tell their
stories about going to the park in a new way. And I have to talk through it, but it's
an app for your smart phone. And the first thing it does is of course you can take
photographs. So you can take your own pictures. That's good. There are
various photo opportunities around the park. And we tried out some location
based prompts to encourage people to do this. This to say is a photo opportunity
nearby. We can tell each one about how successful various location based
prompts were or weren't.
Importantly there's a shared pole. So with this app you register as a group
before you go in you sign up as a family or a group of friends or whatever. And
the rule of the game is any picture I take through the app is shared with the rest
of you, and you get to see it. So collectively we pull our pictures. And then you
can caption the pictures. You can add speech bubbles to them. And you can
add captions that run along the bottom in a kind of cartoon like style. Any
captioning you do, again, the new captioned image is put back in the shared pool
and everyone gets to see it. You can caption whatever you'd like.
We use location based triggers here to encourage captioning when you in the
queue. So thinking about, again, the nature of the experience we figured out
queuing is a real pain which I think everyone in the theme park business would
agree is true, and that if you can recognize when people are in a queue and it's
not that difficult with GPS to do that, then you can give them some kind of prompt
that says now would be a really good time to look through that pool of photos and
do some captioning and have -- and do a bit of photo work.
We through the on ride photos into the pot so we were able to take the photos off
the ride systems. And they go in the shared pool the same way. You can
caption them, you can do whatever. And then when you've been on the ride, the
system encourages you to select three images from the pool for each ride you've
done, one that you think represents before, one that you think represents during,
one that you think represents after. And this is the simplest narrative structure
we could come up with for the right of package. I was scared, it was scary, I felt
great, for example.
So you do all of this work as you're going around the park, taking pictures,
sharing them automatically, annotating them, choosing the three pictures for
each big ride you went on. And then the system automatically generates these.
So it creates your one page photo story for each ride you do. And this again, it's
got your three photos in there, and it's got a bunch of stock material of course
and a bunch of branding because that's really important in the theme park
setting. And it all comes together and you get your cometic book page, your
story.
So this is a historical trajectory. What's interesting I suppose is firstly you get one
for your ride. That's quite important. If you read these ones careful you'll notice
different people differently want to tell different stories. There are some that are
suitable for children. There are some that are definitely more adult oriented
among groups of friends. So not everyone wants to tell the same story. But you
are telling up stories from a pool of shared material.
So in terms of the historic trajectory, I think it reflects most of the points I made
earlier about firstly you have to think about how you document the experience.
There are triggers and points in the application to encourage you to do that. How
you sift and select and how you reorder and represent to make your kind of
shared account.
I will skip over this very briefly. But although I've talked about the theme park
and the historic trajectory, it was quite evident that to us the other kinds of
trajectory I talked about, canonical and participant were also at play in the theme
park. So the canonical trajectory of a theme park, the plan can be quite
complicated. It's co-constructed by both the park and the visitors. But if you
want a general shape for it, it looks something like this. People arrive often
together, people split up. I want to do Nemesis. I want to do the squirrel nut ride.
People come together again, lunch, coffee. They replan. They split up again.
And at each ride think also often split up. I'll go on the right, I'll hold the bags.
Without going into more detail, orchestration work becomes difficult for people.
They spend a great deal of time trying to orchestrate where these trajectories
come together, in particular, meeting someone at the end of a ride is really quite
difficult because you don't know how long the queue is, you don't know how long
the ride is, there's -- I don't know, 20 people coming through every five minutes.
So you hang around for hours.
Meeting people for lunch is also really difficult because you don't know when that
ride is finished and they don't know when yours has.
Equally when it comes to interacting people, we think we learned some of our
notifications were good, some of our notifications were bad. So in terms of
opportune moments for mobile notifications, when the spectator is queuing -sorry, when the rider is queuing, before they get on the ride, good time, I'm
generalizing a great deal here, there are many fine nuances, when spectators
are waiting around quite a good time. When people are coming together, quite a
bad time to notify them. Because at that point they're engaged in lots and lots of
other social stuff like hi, haven't seen you in a bit. What's going on? The last
thing they need is a bunch of system notifications saying don't forget to do your
atomorrowic. And there are various other points. I mean, the rider when they're
on the ride, no point interrupting them because they can't get their mobile out.
It's disallowed.
So I guess I'm suggesting that these trajectories are also quite a useful way of
starting to think about things like mobile notifications, interruptions and possibly
how they work as well.
So I'm pretty much out of time, I'm sure. What I might try to do. Well, a bit of a
journey I think, first of all from some stuff that is very much at the kind of cutting
edge, I suppose, of kind of performance and art but can which have inspired
some things about main stream experiences, but then trying to suggest to you
that some of the ideas we got from that about what you've experiences are like
and what goes into them, canonical participants and historical trajectories that
are designed, how that also might be found in more main stream settings, the
theme park, full of trajectories. Each ride is a trajectory. And then the layouts of
the rides in the park are designed in some sense.
So I'm trying to suggest to you that there is perhaps a broader relevance of this
notion of designing trajectories, transitions, orchestrations, divergences, all of the
things I've said.
So that leaves me with a few questions that I don't have answers to. Assuming
that you believe it is a useful way of thinking even about cultural applications, I
do, is it useful outside of those? Can you think about the trajectory of having a
disease, for example, on going through the health care system? Can you think
about the trajectory of going on the holiday. Can you think about the trajectory of
online shopping and use and eventual disposal of an artifact? Does it have the
same sense of participant canonical and historic? Do you need to document and
reflect on them? Are there the same social overlappings?
>>: A career.
>> Steven Benford: Or having a career or a life. Is there a life long trajectory?
These are all good questions.
Do they divide a general basis for understanding this notion of extended user
experience? One particular thing that we haven't thought by think, hardly at all at
the moment, is how difficult trajectories, different experiences overlap. So
although I said we have two different participants in Uncle Roy, you need to think
about how they design, that was all still within Uncle Roy. Never said anything
about how Uncle Roy overlaps with receiving your work e-mail, overlaps with
being in the theme park, overlap for shopping. At any one moment you and I are
involved in probably several trajectories or threads of activity. And the whole
nature of mobile devices is that they are increasingly connected and rubbing up
against one another. The mobile is essentially a time slicing device. That's what
it does. It lets you cut up your time into finer, finer chunks. And so these
trajectories get potential more and more interwoven. So how do we design that?
And I'm comfortable, at least in my own head, because I understand what I'm
talking about, I think, that trajectories are kind of useful in reflecting all
experiences that I've seen, I can now take all sorts of experiences that we've
documented and say, look, here's the canonical trajectory, look, here's the
historical trajectory. But I'm not sure I can use one yet as a proactive design tool.
I'm not sure I can sit down with a blank page and no experience and say let's use
them from the start.
We're making our first attempt. We've got a project called Chess which is
working doing museum design in Necropolis Museum Greece and City of Space,
which is a big space museum in Toulouse. And there for the first time we are
trying to say to the designers from the outset, hey, let's think about trajectories
and see whether any of the issues that we raise make any sense. But I'll be
honest and say I don't know yet, the conversation is just starting.
>>: At the level of the -- when it's been time to interrupt or not interrupt does that
-- wouldn't that be something that you would build into the design or ->> Steven Benford: So that's what you would hope, yes. And what we try to
encourage is whether you can write these things down. But, you know, it's
interesting. There are all sorts of questions about what does the annotation look
like? Does it look like my wiggly lines? Does it look like whatever? So how you
express this stuff and how you write it down and really whether it has a kind of
proactive purchase and design I think is a -- yeah, it will be a definite question for
the feature.
So that's it. By a coincidence there is a book. The book documents the Blast
Theory works, Brendan's works and some works with other artists as well. But it
them does kind of pull out in a hopefully quite a grounded work this notion of
trajectories. So I'll stop there. Thank you very much for asking. Once again,
thanks for your time and attention. Thank you.
[applause].
>> Andy Wilson: Questions?
>>: I'll ask one. So, yeah, when you're talking about your trajectories, one of the
things that strikes me is that typical theme parks environment, one of the big
things that you try to prevent is everybody going the same way because that
generally leads to horrible congestion problems and so, you know, a lot of effort
is spent to try to make sure that the trajectories of, you know, integral people
tend to adversary as much as possible so you don't get that sort of buildup
whereas in contrast if you look at like an IKEA, you have, you know, a single path
that you were pretty much forced to take through the entire [inaudible] experience
everything, right.
>>: [inaudible].
>>: Yeah. [inaudible] [laughter].
>>: They really don't want you doing that because they want you to see
everything. Or, you know, as opposed to like a super market, right, where they
put the milk and stuff in the back to try to force your -- you know, so a lot of this
has to do I guess with the different goals, right, of what people are trying to
achieve, you know, so the theme park case the crowd control seems sort of
ancillary to the experience, but it's actually a pretty fundamental, you know,
pragmatic thing that you have to do.
>> Steven Benford: Yeah. I mean, at Alton Towers when we asked about the
design allowed to the theme park, the rationale there, I'm not saying that Disney
or not, the rationale there is that the big rides are spread out towards the back of
the park. So you come in and basically you're then immediately pushed into
[inaudible] it's three choices. There are three big zones, the X Factor, the Thrill
Zone, and Ugland or whatever. Yeah. And you're shunted out to those parts and
then later on you sort of find your way back. And that seems very deliberate.
That was quite interesting.
>>: How do you yield with explosively trying to have, you know, very large
number of trajectories? It's not -- you know, it's very hard, you know, the theme
park is you typically don't have just a handful of [inaudible], you know, hopefully
you have thousands of different ones at any given day.
>> Steven Benford: Yeah. So, I mean, it's -- it is multi-scale. And they're not -it's quite clear I think, you know, things like Uncle Roy All Around You is quite
and stream case and it's really a pretty scripted and designed experience. It
doesn't exactly run rails but it's quite clear what's meant to happen from the
outset.
The theme park is interestingly different. Each ride is very much like that. You
go on Nemesis. You're in the queue. And at that point you're in a -- like IKEA
you're in a linear trajectory, you're going through the queue, you're getting on the
ride, you're up the ramp, you're down the ranch, you're off. And that they're it's in
a line.
But, yeah, the layout of those within the park seems to then be based on some
more general trajectory principles. But what happens is people when they arrive
construct their own trajectory.
>>: Are they -- I assume that they analyze the different -- there's pattens in the
trajectories that people take.
>> Steven Benford: It feels like there's a time basic generalization so rider ride
and then [inaudible] go to the bathroom or they will like stand in a queue. That
list is short.
>> Steven Benford: Yeah. And, I mean, there's -- in the areas there's an
ecology of attractions. There is a few big rides, a bunch of eateries, some other
shops and some smaller stuff to do. So, again, that's part of it.
And the trajectory thing is again the extent to which people plan in advance what
they're going to do. Now, not everyone does. But again our experience
suggests that quite [inaudible] do plan their day in the car on the way. What are
we going to do?
>>: [inaudible].
>> Steven Benford: Yeah. And then the situation at lunch they tend to have to
replan. So, yeah, the canonical trajectory is kind of more flexibly constructed in
the theme park for sure. It's negotiated quite a bit more I think, yeah.
>>: I can think of one case when animal Kingdom first opened, one of the things
that was unplanned was that people quickly figured out that the animals are
awake in the morning and they sleep the rest of the day, and so it quickly came
to be that people would line up at the gate and as soon as it opened, they all run
like crazy for the safari ride and you get this instance huge line there and the rest
of the park is completely empty. And there was a lot of effort spent in trying to fix
that problem.
>>: Yeah. I'd love to [inaudible].
>>: So [inaudible] work with artists and the thing that you have ended with
something that's more in the entertainment and consumer aspect of it. And I
wanted to say first of all thank you for saying that industry and artists work
together collaboratively in interesting ways and opposed to sometimes now I
think culturally people think industry [inaudible] kind of dialogue that they can
[inaudible] each other. But so I guess I'm sort of curious why work with artists
when it seems like -- I mean, this could be something that could be done in
marketing or design. Or something that's more high -- that cannot trajectory very
highly and tightly descriptive but you are interested in working with -- at least in
the beginning you're sort of interested in working with artists who think [inaudible]
communication [inaudible].
The experience like having a trajectory is similar idea, but I think that the
intermediate parts are very difficult. When you work [inaudible] as opposed to
going to a theme park.
>> Steven Benford: Yeah. So I'm going to get -- yes, your observations
[inaudible] the question is still why work with artists in that context. I think it does
go back to the reasons I gave to the outset. I think artists are very, very -extremely creative and therefore trail blaze things you can do with technology
that I think the main stream entertainment entries for all of its fantastic research
and creativity may not quite trail blaze the same things. Not saying that research
in main stream industries isn't valuable, lots of deep, difficult problems being
solved. But I still think the artist goes somewhere unusual and I think they do act
as trail blazers. Not that every artist does commercial crossover work or wants to
do commercial crossover work. Some, I'm sure, would reject it out of hand. But
I've seen quite a lot of examples of absolutely where it does work, quite a few of
the industry projects have had industry partners involved to an extent. Uncle Roy
All Around You had involvement of British telecom, data figurines, had Nokia and
Sony who were kind of in there as partners. It felt mostly kind of watching,
seeing what was going on, kind of taking it in.
So there is that crossover, I think. I think [inaudible] because it does take you -takes you out there. Takes you somewhere unusual, makes you think what was
that that just got made? And then makes you think how is what just got made
perhaps more relevant to anything else.
>>: Now, although I also hike the other side that you got to at the end which is
going outside of the theme park and the -- to the real world, I think the
adjustment we're all sort of -- you saw the Adjustment Bureau, I think the movie
with Matt Damon -- well, okay, it's basically -- it's this on the [inaudible] story and
it's this side of [inaudible] scale. In fact, there is this choreography that's has
anyone seen The Adjustment Bureau? Do you agree that it's -- it has some
relevance here? And maybe [inaudible] would be worth reading. But I think that
is an interesting additional angle to it.
>>: One thing I was thinking of is if I wanted to use the notion of trajectories
project, future projects like maybe already have an -- trying to characterize what's
going on by sort of trajectories. So I look at the thing and I start writing things
down. I say oh, this event happened. Maybe I'll do a few of those diagrams you
have. [inaudible] realize [inaudible] this is starting to get really complicated and
there's so many different ways to experience the thing and I'm just -- I feel like I'm
going to be very quickly confirmed by the complexity of how to just write it down
and talk about it, manipulate it or, you know, worse yet what if I have a sensing
system that, you know, watches people moving through my experience and then,
you know, upchucks the representation based on the sense, you know, maybe I
did some data. I don't know, just, I'm worried that the -- you know, the complexity
of reality is going to just, you know, be really trouble some for these
representations.
>> Steven Benford: I think that's correct. I mean, first the -- yeah, I mean, first
yeah, the complexity of reality is great. It is really complex. And so what you're
trying to do whenever you're making one of these things you are designing a
complex multi-faceted thing.
I don't think that writing down trajectory diagrams will, in spite of the fact I say it,
there is only -- everything is made up of this. It won't give you the whole answer.
And I also think it would be impossible to write down in detail all of the
trajectories of any of these experiences. And I've never done it. So, yeah, I don't
think you soon you just give up [inaudible]. So in that sense I don't think that's
the right thing to try and do. What I think you need to try and do, though, is take
the key elements of the idea and ask the question of hmm where is the historic
trajectory in this experience? Do I -- do I have a way of retelling -- recapturing,
representing? If not, is that a problem? Should I have one or does it not need
one? The transitions I think are useful. You can take each of those transitions
and take them off and go, yeah, where what are the scenes, what should I do
about it, where do interfaces get handed over, what's the first engagement, how's
the framing done?
So I think at that level it -- at least it gives you, I would hope, a kind of checklist
that you can run that says okay, have I got one of these, is there one of these,
why without necessarily -- then the next question is do you need to put it -- do
you need to set it out in more detail? And that's where we are in the chess
project and saying is it useful in a more constructive sense.
>>: [inaudible], you know, model, you know, automatically as sort of what are the
things that everyone does, you know, at this moment in this experience, right,
sort of figuring on it -- finding things you don't already know through the use of
[inaudible].
>> Steven Benford: This certainly is hard to do. Probably impossible to do that
in advance. And I suppose you could think of it as being a bit like story boarding.
You know, story boarding in experience is useful but you probably can't write a
story board for every nuance of it. There's always -- there's always things you
miss in the interstitials between the frames. You know, I think it's part of that.
>>: [inaudible] opportunities to look at it. [inaudible] the number of [inaudible]
start with a linear [inaudible] movie there's no detours in the sense of interactive
gaming you have more so you kind of defined the detour points. Help, you know,
kind of at a life level I wonder if [inaudible] cell phones where that's often being
interrupted and kind of present you with new detours [inaudible] so maybe there's
filtering opportunities there with if you really want to stay on a light trajectory, you
know, [inaudible].
>>: Reminds me of -- so John [inaudible]y try to model trajectories of people
driving, right, and so he can attempt to predict where you're going just basically a
few miles out of your driveway, fairly reliable. It's kind of a little scary [inaudible]
how boring we are. And it's all data driven.
>> Steven Benford: Driving is really interesting. There's a sense in which if you
were trying to think about, as I said, the open question for us is to what on earth
does it mean to actually then put different experiences together, something like
driving is more interesting because that's one that feels like you probably
dominate others when you do it in quite interesting way, you know, when you
drive there are some things you really do have to do to be safe and there are
certain constraints. Spatially you are going to satisfy certain constraints, you're
going to satisfy not leaving each other if you're in the car with each other, you're
going to be there for the next three hours. So it's quite interesting. Whether
some trajectories essentially would shape others a lot more than have to fit
against them.
>> Andy Wilson: Thanks.
>> Steven Benford: Well, thank you very much. Thanks for your questions.
[applause]