>> John Tang: My name is John Tang, and I'm a member of Microsoft Research, but I
work out of Silicon Valley down here. And I'm joining by proxy which is research that we
do on social proxies for remote people.
And we're very glad to have Mr. Jeremy Bailenson from Stanford come to talk with us and
visit here at MSR. This is perhaps the first time that a proxy is introducing a real speaker.
But it's kind of fitting because Jeremy's research is on virtual realities and the kind of
impact of virtual presence. So here we go. This is kind of another aspect of that.
I've really enjoyed learning about Jeremy's research in the virtual human interaction lab
at Stanford, both in terms of the impact of virtual experiences on our daily behavior, but
also the clever experimental designs for measuring the impact of that work. And I'm sure
Jeremy will describe some of that, as well as some of the ways he's playing with artifacts
that in many ways can be enabled with Kinect and technologies like Kinect.
He's written a book called Infinite Reality which I'm sure he'll mention more information
about. But we're really glad to have Jeremy here visiting and sharing what he's been
working on and having a chance to meet many of the folks up here at Microsoft.
And I just want to say I'm going to drop off here because I can tune in remotely through
RESNET and get a better view, but I'll probably try to pop in at the end if there's any
questions. Okay. Thanks very much. And welcome, Jeremy.
>> Jeremy Bailenson: Thanks, John.
[applause].
>> Jeremy Bailenson: So my name is Jeremy, and I'm absolutely delighted to be here.
What an honor it is to come talk to us guys about the topic we're talking to today.
And just on a personal note, I had a fairly intense week at work where it's the final's week
and I was very fatigued, kind of sick of my own voice. And I've had four meetings this
morning already, and I just got completely energized by those meetings, I really -- the
time I spent already here has just actually really made me think about things that I hadn't
thought of in a while. So thanks for that, my morning people, and I look forward to the
rest of the time I'm here.
So the title of my talk is Infinite Reality. And what I'm going to do, what I decided to do
today, was to really give you a whirlwind of about 15 years of research, kind of sampling
big ideas.
To give you a sense of who I am, I got a PhD in cognitive psychology where I learned
how to run experiments about what goes on in the mind. I got a bit disillusioned with the
field I was in. I was at Northwestern. I decided to try something new. I got a post-doc at
UC Santa Barbara in a lab with -- Andy Beall, can you stand up a little bit? This is one of
my mentors, one of the ->>: Go Gauchos.
>> Jeremy Bailenson: Go Gauchos. One of the best people I've had the privilege to
work with and learn how to program virtual reality and also learn how to run social
psychology experiments. And so I'm kind of a jack-of-all-trades.
What you're going to hear about today is a lot of work that I've done, some of it in
collaboration with Andy, who is also the CEO of WorldViz who brought you this amazing
VR experience that you're about to get.
Talking about the psychological consequences of spending times in avatar is both good
and bad. And everything I'm going to do today is for the most part published. I'm going
to rocket you through about 40 or 50 experiments. This is the web page of my lab. All
the stuff is on line at IPDF. And the big question that we're going to explore today is what
are the wonderful and horrific things that can happen psychologically when people spend
time inside of avatars. And toward the end we're going to shift a focus to Microsoft, how
can we really think about opportunities for you guys and what are the research things you
can do and the applications you can build?
So to give you a bit of a roadmap, I'm going to talk about technology. What is VR? It's a
very jargonist term. And lots of people define it in different ways. So we'll talk about the
technology.
I'm then going to take you through lots and lots of experiments. One at a time. And then
I'm going to focus on some very specific applications that we're starting to work on a little
bit that are really going to affect the future.
So thanks to WorldViz, I don't have to fully describe what's going on here. So just show
of hands many how many of you guys went through the thing next door? Okay. All of
you will do that later on. I want to talk a little bit about the technology here.
So here is a gentleman who is walking the virtual plank. What's happening here is that
he's moving in the physical world. As he's moving in the physical world there are sensors
that are detecting -- see the XYZ position of his head or the pitch and roll of how he's
moving his neck. And as he's moving in the physical world, there's the three-pronged
process.
The first is track the movement. Understand if he's translated forward. Understand if
he's turned his head. Understand if he's opened his mouth. So that's called tracking.
Tracking physical movements.
The second step is called rendering, that is redrawing the virtual world some damage
scene as a function of that movement. So when I walk a meter forward, we should
redraw the world one meter backward based on the tracking data.
The final step is display that is updating what somebody sees, what she hear, maybe
what they feel. There's even a handful of people in the world changing what you smell as
a function of that tracking data. So virtual reality is a cycle of track the physical
movement, update a virtual scene as a function of that movement, and send it back to the
user's sensory organs.
And when this works well, it happens about a hundred times a second. And the person
has a very seems experience of walking around a virtual world. And to give you a sense
of how hard this is, so some of you have done this, in my lab at Stanford because the
technology has matured, I've been running pit demos like this for about 15 years.
It used to be the case that about one in 10 people would decide, they'd look and they'd
say no, I'm not even going to take a step forward toward this big hole that's looming in the
middle of the floor, which you guys have over there. We're now at the point where one in
three people in my lab will immediately closes their eyes and say take me out of here.
It is a fairly compelling simulation that I should stop talking about, because you all get to
experience it in a second. But inconsistently who doesn't believe that virtual reality feels
real can go next door and jump off of that 30 meter plank and we'll see if you still believe
that.
Stanford rebuilt my lab two years ago and I can brag about this because I had absolutely
nothing to do with it. Stanford paid for it and WorldViz built it. But it is the best virtual
reality lab in the world right now in terms of a fully immersive experience. We have the
vision that you'll experience next door which updating what you see as a function of your
movement. There is an array of 24 speakers in the wall that can spatialize sound to
correspond with your physical movements so if a virtual object flies by the sound
eminates from that virtual object. And then we do virtual touch in a variety of ways.
So this yellow part of the floor is very is very rigid airplane steel that can carry vibrations.
In the four corners of the room are what we call butt kickers which is the product name
that can basically use low frequency sound to make the floor boom and vibrate. We also
use commercial devices to use your hands for you. So basically in this lab, we integrate
what you see, what you hear, and what you feel as a function of your movements. And
so one way this talk happened is because John came down to my lab and brought a
bunch of folks from Microsoft and you're all encouraged as well after this talk, if you're in
the Bay area, one of the goals of the lab is to give outreach. You're all welcome to come
down and do something as amazing as you're seeing there, but also with virtual sound
and virtual touch added as well.
And here is where I thank you guys. So I've been studying VR for about 15 years, Andy
even longer than I. And up until this year, I can never -- I've never been able -- there's a
joke among VR guys which is next year VR's going to be in your living room. And it is like
we have been saying that for 20 years. But now because of you guys, VR is in the living
room, right? So what does the Kinect do? The Kinect tracks 20 simple points on your
body, the XYZ position of that and it does it about 30 frames a second.
So what you're seeing next door, which is in my humble opinion one of the best virtual
simulations you'll ever get, Andy is pulling that off with the XYZ position of one point and
then a accelerometer that gives pitch on roll. You guys are giving us 24.
So we have the raw materials to have the best amazing virtual experience that could ever
happen by tracking your physical movements in just about every living room. A second
non-Microsoft technology that has made VR more accessible to the common person is
something they call auto-stereo which is things like lenticular displays that can give you
an amazing 3D experience without even having to wear glasses.
So you may have seen the Nintendo 3DS. Lenticular displays are like those fancy
postcards -- sorry, those cheap postcards that can give you a sense of 3D. Lenticular
display is a fancy version of that. And in my lab, we bought one from a French company
called Alioscopy that you almost to have reach out and touch these 3D objects to see if
they're real when you walk in the waiting room. It feels as if they're there. So finally the
vision of all VR scientists which is track my body without me wearing anything, which we
can get from the Kinect, and render me 3D stuff without having to have a fancy room, it's
here. So VR is simply tracking and rendering and finally tracking and rendering are
available not for the scientist, not for the Stanford lab or for the WorldViz people, but for
everybody in the country.
But I don't just build VR, my specialty is social scientist. I'm a trained cognitive
psychologist and social psychologist and I study virtual people. So when you're a subject
who walks in my lab, we want to build your avatar. We study what happens when you
spend time in your avatar.
So this is my colleague Jack Loomis. If he were a subject in my lab, and I run a hundred
subjects a week like this, they walk in, we take two pictures of his head and face and then
what we can do is build a 3D model of it.
So this is a 3D model of Jack's head that we've done using photogrammetric software
which is software that from a -- if you take pictures from a series of angles and find
anchor points you can reconstruct that 3D object in the way that looks real.
So how real are these things? They're pretty real, okay? So if a subject comes in 15
minutes later he's wearing a head that captures about 96 percent of the variance in his
faces. These are real enough, such that we've worked with the federal government to do
police lineups using virtual busts as opposed to photographs of people. I just got an
amazing demonstration upstairs of using the Kinect to do something that's very similar to
this.
So the idea of building an avatar that looks like you, you don't even need people to do
this, you don't even need humans anymore, you can just simply have someone sit in front
of the Kinect, move around, and we've now captured your virtual bust.
But I want you to pause for a second. I want you to think about what -- what Napster did
to the idea of music and how it drastically transformed what we think about music.
If I build your virtual bust, it's around forever, it will never degrade. I can make a billion
copies of it for free. And I can send it to whoever I want. More importantly, I can change
it. I can manipulate it. So this virtual head, this idea of having a human, everything that
virtual songs did to the music industry, what does it mean to have a person where you
never change, you're going to live forever, and I can make a million versions of you. And
I want you to keep that in the back of your head as we move on with this talk.
It's not enough to make an avatar look like you. We can use computer vision devices like
the Kinect to capture somebody's gestures. And this is fairly important. So here is a
woman standing in front of a webcam. We're using facial tracking software. These aren't
markers on the face. Computer vision algorithms are catching the XYZ -- the XY
positions of 22 points on the face and then -- so the subject comes in 15 minutes later,
he's not only wearing an avatar that has a nose face, but this is the realtime video image,
this is me animate -- or this is the software company animate the 3D model as a function
of the XY position of those 22 points.
So if this looks amazing, it's because this is the demo video that the company produced.
It doesn't look this good in realtime, but it looks pretty good. And using fairly old
technology. So you've come as a subject in my lab. We've built an avatar that not only
looks like you but now it gestures just like you.
So here's where I pause and I say well, who cares? And I've been giving talks like this
for 15 years now. And I almost don't even need a point -- to make this point any longer.
But I want to focus on I don't need to talk to you guys about the Kinect, which is the
fastest selling product according to Guinness in U.S. history.
A Kaiser Family Foundation, my colleague, Don Roberts who just retired from Stanford
every five years he does a report for the Kaiser Family Foundation where he studies
children in the United States, their use of media. And in 2005 he stood up in front of
congress and he said children when they're not in school and they're not sleeping are
consuming about seven hours of digital media a day. And he said it simply is not
possible -- he said this in front of congress, it is not possible for them to consume any
more digital media. It's just not tangibly possible.
He redid the study in 2010. National random sample very rigorously run. In 2010 the
child between the ages of 8 and 18 is if you count multi-tasking, that is using two things at
once, they are consuming about 11 hours of media per day. And if you don't, meaning
just the pure amount of hours they're exposed to digital media, they're accumulating 8
hours of digital media a day.
So this is the average child in the United States is consuming not in class, not sleeping
for 8 hours a day has a thing on giving him or her digital media. Okay? For minorities,
African Americans and Latinos, it's actually higher than that. It's closer to 10 hours.
Okay?
So kids today eat digital media for a living. 1.25 hours of that are spent in video games
wearing avatars. Put that in perspective. Reading is how many -- how many hours do
you think reading is, reading print? 38 minutes. Yeah. 38 minutes on average. So twice
as much time in avatars as they are reading books.
So let me get to the theoretical point of my talk. And this is a big idea that Andy and
myself and Jim Blascovich and Jack Loomis came up with some years ago that certainly
wasn't an original idea. Science fiction authors have been talking about this for a while,
which is in the physical reality when I perform an action, whatever I do you see. Okay? If
I smile, you see it.
Over Skype if I smile you see it. If I lean forward, you see it. But how does virtual reality
work? So let's take -- let's take -- have you guys heard of Second Life that's online virtual
world? In Second Life if we were on in Second Life now we'd have versions of each
other's avatars, the 3D modeling stored locally on all of our machines, okay?
As I'm talking and gesturing, my computer senses I'm gesturing. How do you do that in
Second Life? You hit a button. But let's pretend the Kinect was doing it. The Kinect is
capturing my motions. My system is automatically mailing your systems telling you that
I'm gesturing, and you are redrawing my avatar gesturing locally on your machine.
So Skype works by capturing pixels and sending those pixels over the network. VR
works by categorizing or tracking of behavior locally sending the semantic definition of
that behavior over the network and you draw me doing what my computer said I was
doing locally on your machine. Is that clear? Is that difference between video
conference, which sends pixels and VR where it sends semantic information which you
draw later, that's the crux for the rest of my talk. Okay.
So why is that important? This allows for what we called transformed social interaction,
which is not -- I can send you what I'm doing or I can send you data, a strategically
optimized version of what I want you to think that I'm doing.
Moreover in a defraud like this, where there's 50 of you, instead of me sending the same
version of myself to each of you, I can strategically optimize myself differentially to all of
you because the way that the Second Life works the way that avatar Kinect works is that
everybody is simultaneously sending each other their behavior data and everyone
redraws each other locally. So I can simply change the version of myself simultaneously
to all of you.
And that may sound abstract and weird, but let me give you a lot of examples. So first let
me talk about how when I transform myself how I can make myself a better presenter,
better teacher, better salesperson, et cetera.
So one of the most powerful ideas in all of nonverbal behavior which is other than me
walking up to you and poking you in the chest, looking at you in the eye is the most
powerful nonverbal tool I have. And as humans we've known this for a long time. As
psychologists we've known it for 50 years is when I look at you your heart beats faster,
you retain for information, you're more likely to look back at me, you're more likely to be
persuaded what I say.
But in the physical world there's a problem, which is I can look at you guys, but in this
room each of you can only get on average my direct eye gaze about three or four percent
of the conversation.
In VR, it's simple for me to send different versions of myself out to all of you such that
each of you gets more eye gaze than you could ever get in a face-to-face situation.
So the first study that we ran in this area, we wanted to start very simple algorithmically
because we wanted to examine the psychological affect. We took three subjects. Three
subjects came in. One was the teacher, the other two were designated as the students.
And unbeknownst to the teacher, as the teacher is teaching the students, there's two
conditions. Of the either the students receive the natural eye gaze as rendered by head
movements or, again, we start very simple on the AI side.
Each of these two people received the direct eye gaze of the teacher for a hundred
percent have the social interaction. Okay? So you either got normal eye gaze, about 50
percent, or for the entire 20-minute presentation one person was staring and looking at
you for the entire time while we talked, okay.
And so I want to you think about that. In order for us to pull this off, this is maybe 20 lines
of Python code. This is not a hard thing to do technologically. We were simply just
redirecting the yaw of the head such that it either was being natural, according to the
tracking data or redirected to stare each -- both people at the same time.
So we've now replicated the study about three or four times, probably over 600 people in
a period of six or seven years. The findings I'm about to give you are very robust. So the
first thing we find is that people hate this, okay? It's extremely uncomfortable. You can
imagine if I was giving this lecture and staring at all of you the entire time you would not
enjoy that. Okay?
But despite the fact that they don't like it, nobody knows that it's not real. So afterward
we have them read a detailed paragraph about the teacher, and no one says well, it's
probably fake gaze redirected by an algorithm. They all feel like they are getting stared
at.
So despite the fact it was cheap and easy to do this, people were buying into that they're
actually getting looked at.
The third thing that you get is behavioral change in terms of attention. So if I'm going a
teacher, I'm a salesperson, half of my goal -- half the battle is getting you to pay attention
to me. And what we can do in VR, which makes us -- gives us a big leg up in terms of
other social scientists, we can capture all of your tracking data.
So what we can figure out is how often were these guys looking back at the presenter?
I'm using this super gaze algorithm, you guys are returning my eye gaze more often than
if I have just my normal power.
The fourth thing we get is social influence. Whether in some studies we mixture
persuasion and some studies we measure learning, regards if I've got a conversational
goal, if I give myself 20 lines of Python code such that all of you are getting stared at the
whole time as a class you've learned better, as a group of voters you've decided to vote
for me, I can change your mind simply by changing easily the nonverbal factors of my
avatar.
Next line of studies is based on a line of work by a woman named Tanya Chartrand
which leverage the notion that people love themselves. Okay. So one of the overriding
themselves of social psychology is that people -- similarity is one of the largest driver's of
human behavior out there. And Tanya Chartrand in the late '90s she was at NYU, she
ran a study where she sent people face-to-face into interviews and she had the people
either mimic nonverbally the gestures of the interviewer, so the interviewer, he closes his
legs, the interviewer, she crosses her legs, the interviewer, he leans forward, et cetera.
The people who were being interviewed were trained to nonverbal mimic the interviewer.
None of the interviewers consciously detect this, but the odds of you getting the job
increase if you employ this mimicry strategy compared to if you don't.
Since then there's probably been 50 studies, 100 studies that replicate this, in all sorts of
domains. One of my favorite if a waitress, this is a study done in the Netherlands. If she
says to you ham and eggs, actually repeats the order that you say, compared to if she
says thank you, sir, tips go up by about 20 percent.
So we know that mimicry is a huge factor in terms of behavior in the physical world. But
how does VR work? VR works in a way that is stunningly easy to do this. I send you my
tracking data, you get it from my Kinect, you'd use that to draw me locally on your
machine. And you're sending me your tracking data so you can draw you guys locally on
my machine. It's trivial. It's 10 lines of code for me to grab the packet that contains your
behavioral information, hold on to it, put my name on it and send it back to you. So think
about mimicry.
In the physical world if I'm trying to mimic you, I need to focus cognitive resources. It's
really hard for me to do that. I can screw up and you can consciously detect it. Moreover
in a room with 50 people, I can only mimic one of you guys. I can't mimic all of you. With
VR, you're just giving me your tracking data. You're handing it to me on a silver platter
saying here's how I'm moving. And so the first study we -- again we start really simple in
the algorithmic side. We just decided you know what, what happens if somebody mimics
the pitch, yaw, and roll of your head at a four-second lag. And the four-second number
didn't happen magically. We did a lot of pre-testing. If four seconds is kind of this magic
land between it's still interactive but nobody consciously detects it.
And what you're about to see is a persuasion study where this woman is trying to
persuade this man about a very unpopular policy at Stanford which is if you're ever
caught on campus without your student ID you're going to get in a lot of trouble. Nobody
wants to sign this petition to support this policy. It's fairly [inaudible] and while she's
presenting this policy to him, half the subjects get head movements that are normal, the
other half get a slow lag of her head movements so -- of his own head movements. So
let me show you this.
So she's telling him about this policy. He's looking at her. So he looks up one, two,
three, four, she looks up. So basically she's just a slow mirror of his movements, okay?
So what do we discover in this study? The first one, the first thing we discover in this line
of work is that nobody consciously detects this. So we thought about half of our subjects
were going to be clearly aware that someone is shamelessly stealing her head
movements.
But only one in 20 people on average, five percent of people put in this simulation
consciously detect that someone has completely stolen their gestures. And those tended
to be people that spun in their chairs or did something very unorthodox. [laughter].
And remember, we started with a very uncreative algorithm. It's just a slow mirror. If you
put in a tiny bit of logic to make this more impressive no one consciously detects it.
The second thing that you get is attention. So people were more likely to look at the
presenter when she's mimicking them than when not, even though they don't consciously
detect this. And the third thing that we get is social influence. So later on, the probability
when you're out of VR of you signing this petition that nobody wants to sign, which is if
you get caught on campus without your ID you're in trouble, the probability of you signing
that increases if you were mimicked by the presenter than if the presenter had normal
head movements.
So in addition you're rated as more intelligent, more credible, more attractive. We just
love ourselves. And when we see our gestures on other people, that transfers. And,
look, imagine what we're getting with the Kinect? We'll return to that later on.
Okay. It's not just your actions. So we have a whole line of work about simply grabbing
your appearance. So if I were to ask, you don't have to raise your hands, how many of
you have 10 pictures on Facebook of your face, 20 pictures, 100 pictures at a ridiculous
resolution of 4,000 by 2,000 pixels, I don't even want to know the answer to that. But I
could get all your faces quite easily.
And we wanted to ask the question, I want you to be very impressed by this video. So be
impressed because it was made in the year 2003 by a 19 year old communication major
using software that cost 19.95 and it took this student 30 minutes of training to produce
this video. So really cheap and easy to do this.
All right. I'm going to show you that again. This is the governor -- the new governor of
California back then, Arnold Schwarzenegger as he's morphed into Gray Davis. And all
this is is pixel blending software where you morph from one picture to another. Okay?
So it's trivial for me to make my face look as much as I want like yours and vice versa.
What do we know about people looking like you? So again one of the huge driving forces
of social influence is if you look like me. We like to think of humans as this very
advanced species where we're deep and we look inside people. And we are and we do.
But it turns out how you look has a huge influence on how I'm going to treat you.
I'll focus on these two bottom ones, Devidia [phonetic] demonstrated that if I'm clutching
my knee in agony on the street the odds of you stopping to help me increase if we look
alike. Lisa DeBruine, a researcher in England, she morphed faces and demonstrated
that the odds of me handing over my children for you to baby sit increase if your face
looks more like mine.
So we've known for a while that facial similarity has a big driver on people's behavior.
But I wanted to ask this in the domain of politics. So how big could these effects get?
How much do they scale up? And I collaborated with a colleague of mine named
Shanteau Iyengar who is a political scientist.
And what you're seeing here is pictures of two subjects. This is a woman named Julia
who works in my lab. This is a picture of me from 2003. You can see clearly that is from
before when I got tenure, very clean cut [laughter] and this is George W. Bush and this is
John Kerry. This is the morph that you saw on that video, which is 60-40 George Bush to
me and 60-40 John Kerry to Julia.
As long as you stay under 40 percent we have now run over a period of almost a decade
over 1300 people in these studies and ask them after the study to write a detailed
paragraph of the candidate's face as long as you stay under 40 percent. Not a single
person to date has had any idea that we have taken their own face and blended them
with the candidate.
So fairly trivial for me to do this. So we wanted to ask the question, if I was a politician
and a week before a major election I took your face and put it into mine would it change
how you voted? And the way that we did this experiment, we -- 6 months before the
2004 election which was between George W. Bush and John Kerry, we paid a focus
group company to acquire photographs passively of a national random sample of people.
So we got about 400 people from across the United States. Six months before the
election we had photographs of all of them, and we really had a national random sample
in terms of age, demographics, you name it, whatever feature we had it.
And then feverishly for six months we had Stanford undergraduates morphing faces of
the voters into either George Bush or John Kerry. We had three groups of subjects. A
week before the 2004 election -- remember, this is a very heated election. People either
loved George W. Bush or they really didn't love George W. Bush, all right?
A week before the election, we asked them who they're going to vote for, and we sent
them pictures of Bush and Kerry side by side. There's three conditions. In one condition
they saw the normal pictures of Bush and Kerry. In the second condition they saw their
own face morph with George W. Bush and in the third condition they saw their own face
morph with John Kerry with the prediction that perhaps this would change who they're
going to vote for based on this subconscious similarity that we've been talking about.
What we demonstrated -- we first looked at the gray bar. This is 46-44, Bush over Kerry,
which means we had a very good national random sample. We had the exact pattern
that you got in the election. In the red bars when the voter, when he was morphed with
George Bush, Bush wins in a landslide by about 15 points. When the voter was morphed
with John Kerry, Kerry actually wins by seven points. Okay? So statistically significant
differences demonstrating that a simple thing like making my face look more like you is
going to cause you to vote for me.
So as social scientists we were extremely excited by this. As citizens we were terrified.
And what we immediately did is we took this and we wrote it up and we sent it to the best
journal in political science. The editor rejected it without even sending it out for review.
He said voters are rational. Voters don't make decisions based on your face. [laughter].
And, in fact, the entire field of political science is founded upon the notion that people
make decisions based on substance.
So in order to publish this, which took literally seven or eight years because of the
controversy, we replicated this four, five different times, huge samples, different elections.
This effect just doesn't go away.
And I want to think -- I want you to think about -- you know, you're probably a little bit
freaked out right now. But I want you to think about all the data you're giving away on
Facebook with the Kinect, everywhere. Our digital identities are out there for people to
use. And what my studies have demonstrated is that even grabbing a tiny bit, grabbing
your face, grabbing some nonverbal data gives me advantage to where I'm interacting
with you.
All right. Let's shift. Let's shift. And let's not talk now about me influencing you guys. If
we are transforming and changing ourselves, I want to talk about the impact of an avatar
that's different from my physical self. How does that affect me?
And, you know, in Second Life the online world Second Life there is a ratio of female
avatars to female users is quite high. So you have lots of men who wear female avatars.
In games like world of war craft, people are even in different species. And these crazy
goblins, et cetera. So you have -- are you guys familiar with the game World of Warcraft?
The average player, what do you think his age is?
>>: 26.
>> Jeremy Bailenson: 29 is his average age. Average number of hours per week?
Average number of hours a week is around 20. Okay? So you have adults spending half
of a work week in these crazy avatars that are a different species from us. So this is an
issue that we're going to have to confront as a society.
And one study that we wanted to ask -- one big question -- this is a guy named Nick Yee.
These are his dissertation studies. If I spend a lot of time in an avatar that's very different
from myself, how does that change the way that I feel about myself? My self-esteem, my
psychology, my attitude. So I want you to watch this movie.
This is the classic mirror demo that you'll see in many VR labs. And this is Nick. He's
physically walking around this hotel room. It's a fairly old simulation. And he walks up to
the virtual mirror, and you can see he's a white male. And our typical studies you spend
about two minutes gesturing in front of the mirror, moving around, and you get a
lightening rod when this thing moves with you in the virtual mirror it really feels like you.
So he's now formed a psychological connection. He was a white male. He pops up and
he's now a woman of color. Okay? And what we ask is that now that he's wearing an
avatar that has social features different from him, namely we've changed his race and his
gender, how does he proceed to act when he meets someone else in VR? Does he act
male or does he act female, and what are the changes long-term to spending time in
avatars that are different from you? And the paradigm more most of these studies are
this is the subject in front of the mirror, he looks at himself in front of the mirror for about
two minutes, she then turns around and interacts with what we call a confederate. A
confederate is another person who is part of the study in VR with them whose job it is to
talk to them, to elicit behavior.
And I'm going to summarize about 12 years of research, and I'm going to talk about two
different identity queues that we change in the mirror, attractiveness and height. We've
looked at age. We've looked at race. We've looked at gender, we've looked at having
disabilities, we've looked at just about every social feature you can imagine, all of them
showing these similar effects. But I'm going to focus on these two today.
So the first one, attractiveness. In the United States, attractiveness is correlated with
extra version. It's correlated with self-esteem. So again we like to think of ourselves as
this deep race, but it turns out physical attractiveness is a driver of how we feel about
ourselves. But in VR beauty is free. So I challenge you to go into a world like Second
Life and try to find an unattractive avatar. It just doesn't exist.
In fact, those people that make the software have a hard time even designing these
features into the design. So Nick wanted to ask the question, when you're wearing an
avatar that's better looking than you or better looking than average, how does that
change your behavior?
And so in this line of studies what he did is subjects went up to the virtual mirror and
they're wearing avatars that were either one standard deviation, as determined by other
-- by code or ratings higher than average or one standard deviation lower than average.
So subjects went to the virtual mirror. They saw themselves either as better looking than
normal or as less attractive than normal. They then turned around, and the confederate
says why don't you come on over here, come on over here. And what we measure is
their behavior when they have an attractive face in the mirror or an unattractive face in
the mirror.
Which you have an attractive face and the subject -- the confederate asks you to come
over, you'll step a meter closer to the confederate than when you have an unattractive
face. So personal distance, which is one of the most unchangeable metrics of
confidence and social behavior, when I have a good looking face I've got the confidence
to get right up to you. Okay?
We also tape record everything that's -- that you say in a subsequent 20-minute
conversation, and we mail those tapes off to judges who are trained. They have no idea
what the experiment's about. They're trained to figure out how confident you are based
upon your voice. And when you have an attractive face in the mirror, even though you're
only looking at it for about two minutes and you turn around, you actually speak more
social, more personally and more confidently.
So very large effects. And anything I'm going to talk about today, I'm confident in these
data. Meaning when Nick had to publish this data, he had to replicate the study three or
four times. These are very, very robust findings. Let me give you another detail here.
There's -- this work is based on a theory called self perception, which is I look at myself
and by looking at myself I figure out how to behavior. So why do I dress up for a talk?
Because maybe if I'm having a hard time I can look down and see I'm wearing a fancy
shirt tucked in and that's going to give me confidence to be a good speaker.
And that's the idea that my own appearance is changing my own behavior. And that's
what Nick wanted to study.
There's a competing theory called behavioral confirmation which is because I'm dressed
up you guys are treating me more official and I then speak more confidently because
there's this behavioral feedback from you guys. And so in order to prevent that second
theory, all confederates only saw the subject in VR as average in attractiveness.
So the subject saw himself as high or low. The confederate saw the subject as average
and what we were measuring is the pure change in behavior based on the belief that you
were in a better looking or a worse looking avatar.
The next study Nick wanted to do was height. And in this in the United States income is
correlated with height, so taller people make more money. A fairly depressing study
came out of University of Chicago. They got all the back-end data from online dating
websites and they could do the actuarial math on lots of features as it predicts whether or
not you're going to get dates based on how many people click to date you. And it turns
out that for male every inch you are under 5-10 to get the same number of dates you
have to make an extra $30,000 per inch. Okay. [laughter]. So you can actually put -you can put a price on height.
But remember, in VR height is free. It's just a scaling on the Y axis. It's trivial to make
yourself any height that you want. So in Nick's studies he simply made subjects 10
centimeters taller or 10 centimeters shorter than the confederate. So the subject walks
up to the confederate, they start talking and there is a height differential. Nobody
consciously realizes that they're taller or shorter.
Later on we interviewed these subjects during debriefing. No one has any idea that
there's a height differential or attractive differential. This is all subconscious in what we
call implicit. So there's a 10 centimeter height differential. They then go and have
financial negotiations. We borrowed a paradigm from the business school in Stanford
called the ultimatum game, which is a classic money splitting task. And what we have
people actually haggle with real money. As they interact with avatars, which are the
same height, taller than the confederate or shorter than the confederate.
When you have an avatar that is 10 sent meters shorter than the confederate, you are
three times as likely to lose that negotiation than if your avatar is the same height or
equal. So tiny change and a scaling on the Y axis is drastically changing the amount of
money that you win.
This effect occurs independent of your physical height. So it's purely the virtual
manipulation, even when you take into account the physical height this occurs. And this
has been replicated a number of times. So fairly confident that a small change in how
your avatar looks in terms of how tall it is is going to change how much money you win.
But the question everybody always asks Nick when he was running this work is fine,
that's fine, but I don't spend that much time in VR. How much does it transfer to the
world when you're face to face? And so for both of these studies Nick ran the
face-to-face analog.
With the attractiveness study, you spend 20 minutes -- you saw yourself in the mirror for
two minutes as good looking and then you turned around, you interacted with the
confederate for 20 minutes. We then take the helmet off, pay you for your time, you sit
there, you fill out some forms or -- and then we walk you four flights of stairs down the -our building, and you're about to leave Stanford, you're going to your car and a graduate
student runs up to you and says wait, wait, wait. If you fill out this form I'm trying to finish
my dissertation, it takes five minutes, I'll give you $10, people help me out, I need you to
fill this out.
Everybody does it. Nobody has any idea it's associated with this task. And it's an online
-- it's a dating task where you're supposed to choose people that you think would go out
with you. So you're supposed to pick people with whom you think you have a shot to
date. When you've had a good looking avatar compared to a bad looking avatar a half
hour earlier in VR, you pick better looking people that you think would be interested in
you. So the confidence that you get in virtual reality extends face-to-face. And again, all
this is replicated and we're fairly confident that major, major social consequences by
spending two minutes in front of the mirror wearing an avatar that's better look or worse
looking.
With the height differential study Nick had them spend time in a VR taller or shorter, he
then took the helmets off the confederate and the subject. They never saw each other
physically. He then put them in chairs designed to make their heights exactly equal and
had them then do the negotiation. And literally identical results, meaning in the social
hierarchy that forms inside VR transfers face to face and the guy who was shorter in VR
still loses money.
So and I could talk about all the other examples if you'd like in the Q and A with other
features. But it's basically the same story for all this stuff, which is a small feature in
virtual reality whether you're good looking, whether you're tall, changes how you behave
not only in VR but later on.
We talk about the opposite phenomenon. And this, we call this veja du. It feels like I've
never been here before. Another word that we use for this is doppelgangers. And this
takes the big idea remember when I built that 3D model of Jack Loomis, okay? This is
the model of his head that looks just like him. But now I want you to think about every
animation that's ever been produced in the history of video games, all right? How many
is that? Is that closer to 100 or closer to 50,000? It's probably a million. I don't even
know how much it is. There's a lot of animations.
I can take that head and put on it a body such that any animation you can fathom that
head can do. Meaning we're at a strange point in the time of human history. So let's
take a step back and just think about self reflection and people. Humans have always
been able to look at each -- themselves in the mirror. Even when we were cavemen, we
could look over the water and see our own reflection.
In the last 50 years, we've been able to see ourselves asynchronously meaning you can
watch a videos of yourself doing something that you've done, but you're not doing it right
now. And you can see this via home video. For the first time ever as a specie we have a
new media event. And this takes a while to wrap your head around, but I think it's one of
the most profound things that I've ever encountered in my 20 years of social science
history, which is for the first time ever we can look in a mirror, see ourselves and it looks
just like us and our mirror image can all of a sudden do something that we've never done
physically.
And this experience of seeing yourself from the third person engaging in a behavior that
you either have never done or you couldn't do or is dangerous or is unethical or is
something that you hope to do is one of the most profound psychological experience I
think that we can even think about as a specie. And I want to talk about some of the
examples that we've done in my lab.
And the first one, this is the dissertation of a woman named Jesse Fox who really cares
about health behavior. So this guy is exercising in place. He's seeing his own avatar. It
looks just like him. We've modeled his face from the third person. Every three times that
his legs cross his waist in the physical world, his avatar loses a pound. So what he's
seeing is the realtime consequences of physical behavior on his idealized self. He's
getting to see the cause and effect of actions and actually being able to see the virtual
self that he wants to be.
So he wants to lose weight. We've selected people that want to do that. And what we
show them is over time an exaggerated sense of the cause and effect of exercise so that
they can see the ideal version of themselves as how they want to be.
And I'm going to summarize about 15 studies by Jesse Fox. When I give you this
simulation, compare to control conditions, for example, seeing someone of the same
gender who is not your exact face but looks like you, that doesn't work. Showing you
yourself but it doesn't change as the function of your action. Compared to those control
conditions, this changes your behavior drastically.
So all of us in America, you know, Seattle is probably like California where we're fairly fit
part of the country, but in America we've got a problem where people are not exercising.
And all of us know we're not -- we're supposed to be exercising more than we are. But
that's not enough. What VR gives you and what you experience when you go to that pit, I
want you to take all the angst when you're looking over this cliff and all that psychological
power that you're feeling. Imagine that you're directing this towards, you know, improving
your own bode size or shape based on your behavior.
And when you get the simulation in Jesse's studies we had people keep diaries for the
next 24 hours. When you get the simulation compared to control conditions, you'll
exercise for an hour longer when you get this compared to other control conditions.
That's self report.
She's also run studies where she leaves you in the lab and she says you know what,
your study's done, gives you your money, but if you want, you can kind of hang out here
and work out. Here's some weights.
And when you give them that simulation, people will exercise for 10 times as long with
when you've given them this compared to other ones. So we know exercise is the
hardest behavior to change. And we're getting unique leverage by using these
doppelgangers.
So on the negative side or positive side, depending on what you do for a living, Grace
Ahn, a PhD student in my lab, took the idea that has been long herald in science fiction
and built it, which is what if you were walking down a shopping mall and a camera took
your picture and then you got 50 feet later and then you were starting in a billboard using
a product you never used before and wonderful things were happening to you because
you were using that product. There are beautiful people fawning all over you or you are
smiling. She calls this self-endorsing.
So the goal of advertising is, you know, if you can have an endorser that matches the
demographic of your target market, who is a better match of your demographic than you?
And through a long line of studies Grace -- she just actually published this in the Journal
of Advertising demonstrated that whether you're showing yourself in the third person
using a product or whether you're overtaking someone's avatar and putting labels on their
arms about a product, any form of self-endorsing is a more effective advertising strategy
than lots and lots of control conditions.
A study by Hal Hershfield, who is now at the Business School at NYU, we did this in
collaboration with Laura Carstensen who runs the Center for Longevity at Stanford. And
the goal of the Center for Longevity is to try to figure out how we can have happier -- be
happier when we're older.
So it's not about extending life but it's about making sure that you're having a good time
when you're older. And in this study is about retirement savings. So the actuarial math
people have demonstrated that people in their 20s are not saving enough money. And
that demographic is going to be in big trouble 40, 50 years from now because they're just
simply not putting money away.
Hal was a PhD student of Laura's and said humph, you know, there's this philosopher
called Parfit and he's got this really cool idea about a connection with your future self.
Which is we have a hard time imagining future pain. We're more likely to be pleasurable
now because we can't make -- the pain in the future is very abstract.
So what he did is he brought subjects in, and we built their avatars and you can see this
a little bit. What we're doing is year algorithmically ageing them. So they go to the mirror
and they look in the mirror and they watch themselves turn from 20 years old to how
they're going to look when they're 65 in front of the mirror. And what they're getting is a
visceral connection with their future self.
He then gives them various ways to defer gratification. For example you can have $20
now, or if you put it in a savings account you can have more money later. And he looks
at the probability that you will defer gratification and put money in savings and
demonstrates that avatars do better than descriptions, better than typical ways of getting
people to save money.
And just to give you a sense of how these things are being applied, so, a big company
called financial engines in about three different stock companies are putting this as an
interface on their software. The come who did the program in this, Maria Javen
[phonetic], left my lab and works for LinkedIn, and it is now their most popular feature that
they've ever put on LinkedIn in terms of getting people to click on a job that was meant
for you. And the exercise studies are being used by a variety of different people who are
trying to make people healthy. So big, huge opportunities for people who want to do
behavior change.
In terms of where we're doing a lot of work now, we're at 50 minutes now, so I want to
wrap up to give you guys questions. But quickly we're doing a lot of work with energy
conservation. So one of the big themes from the work that you're getting thus far is that
VR gives you the opportunity to make behaviors that would have been abstract to make
them concrete.
So take global warming. Carbon molecules are invisible. Their changes are long term.
It's hard to imagine them. And we ran a line of studies to try to connect people in a very
visceral manner with the environmental consequences of their actions. So this is a study
by Grace Ahn. And there's an article -- an op ed that came out in the New York Times
about five years ago that said if you use that beautiful soft nonrecycled toilet paper, over
the course lifetime, you're sawing down two huge beautiful virgin trees. This doesn't
change behavior that much.
So Grace brought subjects in. She selected for people that use nonrecycled paper.
Everybody was given statistics that if you use nonrecycled paper you're killing two trees.
Half the subjects read a beautifully written narrative of what it would be like to cut down a
tree. The other half put the helmet on, had spatialized sound of birds in the forest and
used a haptic device that shook their hands and were forced to cut down these two trees.
The trees thunderously crashed, shook the floor. And it's -- everyone that comes to my
lab that does this, it's a fairly profound experience that really makes you think for a long
time afterward.
What she then does is measure your behavior later on in the physical world. She
measures your paper use. When you get this virtual simulation compared to reading a
really beautifully written narrative about what it's like to use a -- to cut down a tree, you
use 20 percent less paper in the physical world later on.
So, again, everybody likes to think of themselves as green. We'd like to say that we're
going to, you know, car pool and recycle. But talk is cheap. She's demonstrating large
behavioral changes as a function of an embodied experience.
We also have a line of work on diversity training. So it's hard to imagine what it's like to
be someone from an out group, whether that it's a different race, different gender. In
terms of sexual harassment, I can imagine what it's like to be harassed by my boss, but
what if you could put someone in a simulation and have them literally feel what it would
be to be an out group. And this is a line of work we're doing with disabilities.
So for example we took a very simple disability which I wouldn't even call it a disability,
which is red-green color blind. Subjects were either -- wore a helmet that filtered out so
you could either see colors or made it so you couldn't differentiate between red and
green and then they had to do a task that was very difficult to do without seeing colors.
So they're wearing a helmet and they're either robbed of the ability to see color or they
can have normal vision.
Later on we give them the opportunity to volunteer their time toward people with visual
disabilities. When they got this simulation, which is being forced to see color blind
compared to having to imagine what it would be like to be color blind, they're more willing
to volunteer their own time to help people with disabilities. So again, small virtual
simulation that has a lightening rod to experience that causes big behavioral changes.
I'm going to conclude very quickly and talk about four different challenges. I'm going to
spend about a minute on each one. The first is digital footprints. And, you know, to John
asked me specifically to talk about some work we're doing with the Kinect and everything
that I've talked about before, what the Kinect does is just make it so we can do it in a way
where you have more control of your body and where you can increase the scale
because it's in the homes. But there's a side effect of tracking and rendering, which is
when I track your motions I can store it in a data file. And in my book, infinite reality,
there's a chapter called digital footprint that summarizes experiments we're doing with the
Kinect and others which is what can I tell about you based on your digital footprint?
So I'll talk about -- there's about 10 different studies here, but the one I'll talk about is one
that I just got data from last week. We used the Microsoft Kinect to track the behavior of
two people. There was a teacher in the physical world and there was a student.
The teacher's job is to teach the student about environmental principles. The student is
listening. Both the teacher and the student are two subjects from my classes. We run
150 of these diads, okay? For each person they have a Kinect trained on their body.
We're not rendering anything, we're just capturing the XYZ position of all the points the
Kinect gives us. Okay?
With these 150 diads, we then take the learners. And we can partition them in the top
Core Tile and the bottom Core Tile. How do we know this? Because after this whole
thing ends, we have them take an exam. And we have education scholars that teach us
how to grade exams so that we're actually getting inferential learning. And we have a
very objective measure of who learned and who didn't.
What we then did is machine learning where we take the nonverbal digital footprint, that
is all the data that we dumped. The Kinect gives us XYZ position in 24 points, 30 frames
a second and it's a 10-minute conversation. We're running 150 diads, you can imagine
how much data we have. With 90 percent accuracy, based on the nonverbal synchrony
between the teacher and the learner. Not looking at what was said at all, I can predict
whether or not you are in the top or the bottom tier of learning.
So simply what your body is doing is telling me what the learning is. And, you know, I
had a wonderful conversation with Doug this morning about how you guys were thinking
about protecting privacy on this -- with this data. But to make a long story short based
upon the nonverbal footprint using devices that are much less sophisticated than Kinect I
can predict your gender, I can predict your personality, I can predict your race from the
way that you walk around Second Life. You know, the speed with which you walk and
how close you come to objects. I can predict your attitude towards things like women,
political parties, whether or not you're going to buy a product.
A side consequence of us spending all of this time in VR is you are giving away what we
call big data. And if you've got an objective truth on things like if you're going to purchase
something or not, if you are so the military's using this to find out what country you're
from. And, you know, there's wonderful applications of this. But there's this cartoon from
the New Yorker you guys have probably seen on the Internet nobody knows you're a
dog.
Right. So the Internet gives you this illusion you're anonymous. But based on your
nonverbal footprint that I can get from the Kinect, and not only do I know you're a dog, I
know what kind of dog you are, I've known what you've had for breakfast. And it's a side
consequence that in terms of things Microsoft could think about, I was very excited
talking to Doug to see how hard you are thinking about this.
The other thing -- I will be very brief on this because I'm running out of time. I want you to
take the immersion you're getting with that pit next door that WorldViz has built and
you're looking there, and you're afraid to take that step. I want to you port that to imagine
that online gambling felt like Vegas, imagine that pornography felt like you were there,
and I want you to think about the future. For better or worse when these video -- we call
these video games but you guys who have used the Kinect, you know it's not just a game
anymore. It feels like you're there. What happens when these things that are already
subsuming us feel like you're there?
A related point of this is not knowing what's really anymore. And a very brave woman
named Kathryn Segovia ran a study in my lab where they brought in a hundred
elementary school children. The elementary school children all got a novel experience,
which is they saw their doppelganger -- remember, once I built a version of yourself, I can
animate it to do anything. So they saw a version of themselves, that looked just like
them, swimming with whales.
A week later she brought the same eight and nine year olds back and said, hey, have you
guys ever been to Sea World and swam with whales? And 50 percent of them not only
said yes they had been to Sea World and swam with a whale, but could tell you about the
hot dog they ate earlier, about the different games they played with the whale, of the
temperature of the water. There's a fairly rigorous method for determining what's called a
false memory. And Kathryn has demonstrated that seeing your doppelganger do
something results in huge formations of false memories.
So things for us to think about as these video games get more and more realistic.
Multitasking very briefly. I know you guys were dab link in this. Have you guys seen the
video for Google glasses? Where a guy is walking and Skyping with his girl friend while
standing on the edge of a building and playing the ukulele at the same time? My
colleague, Clifford Nass at Stanford ran a line of studies to design to prove to the world
how multitaskers get more done than people who don't because Cliff is a -- will be the
first to tell you how much he loves multitasking.
He published this in the proceedings of the National Academy of Science because the
data were so surprising. People that use more than one media at once not only perform
poorly in the moment but tend to be people in general that perform worse at work, at life.
What's going to happen as these virtual experiences that we love so much can be
integrated with the physical world is something that I'd love for you guys to think about
more.
You know, Jaron Lanier who many of you guys know, he's one of the reasons I'm here at
this talk today. He really asked WorldViz to come to bring the VR. He and I have been
collaborating on something that is only possible now because of the Kinect. So Jaron's
big idea -- so this is a guy with an arm that's 50 meters long. And this emerge Jaron will
tell you about how he had a sign error or he had an extra zero so one of his simulations
in the early '80s when you moved your hands, when you translated your physical hand,
your arm was as big as a sky scraper. And to him this is very inspirational about well
how can we what's called remapping degrees of freedom? How can you put somebody
in a body that's different than their own? And his anecdotal evidence is a lobster. In
California a lobster has eight arms, okay?
And if you're tracking your physical arms, it's fairly simple how to track arms one and two.
It's not so easy figure how to track arms three through eight. And there's the line of
research I had going with Jaron which is can you increase people's function, meaning
can you teach people how to occupy bodies that are dramatically different in terms of
their physical bodies?
And I'll tell you some data that's fresh off the press. Andrea Wo, a woman who works in
my lab ran an experiment using the Kinect and other tracking devices as well where
when you move your physical arms, either your arms move and you move your legs and
your legs move, which is reality in an avatar, or she simply swapped tracking and
rendering. So when you move your physical arms your legs go whoop and when you
move your physical legs, your arms move.
And what she did was she gave someone a task in VR where they have to pop balloons
and she's measuring how quickly and easily someone adopts to these novel bodies. And
we've got some really great data showing that it takes about two minutes and how you
can literally embody configurations. She ran another study where people's -- your arms
have much more degree of motion than your legs, demonstrating giving someone the
ability to have the legs move like arms do and just showing how you can accommodate it.
And the answer is really quickly. But that's a brand new area of research.
I want to leave time for Q and A. So I will just -- have you guys seen this Orville
Redenbacher video? So not now, but when you leave this talk go to YouTube and
looking up virtual Orville received. Orville passes away. Orville is the spokesperson -the most famous spokesperson in all time. He is the -- literally the brand and the brand
name of a popcorn company, very successful company. Orville passes away. His
grandson gives ConAgra the right to hire an advertising agency to rebuild a 3D model of
him after he passes away from all of that video footage and there's a commercial where
Orville's dancing around with an iPod saying this baby's life and fluffy just like my
popcorn.
And so Orville continues to be a brand endorser from beyond the grave. He's never
given permission to do this, right, so you guys may make the conscious decision to have
your avatar built. You know, they can do it on the second floor here. And to have it
stored so that your great, great, great, great grandchildren can sit on your lap and you
can have a wonderful experience -- they can have a wonderful experience in the future.
And that's your decision, and that's fantastic.
But Orville didn't give that decision. Right? His data were out there. Somebody built a
3D model. And now he's dancing -- his legacy is dancing around with an iPod like idiot,
okay? [laughter]. That was old technology. So you watch this video and it looks very
creepy.
There's a concert in California called Coachella, which is a very big three-day big
conference. And this is artist called Tupac Shakur who passed away some time ago.
And what you're seeing here is the crowd went ballistic because this is Snoop Dogg on
stage. They used Pepper's ghost, which we talked about earlier, to basically put 2Pac on
stage with Snoop.
No one in the crowd had any idea what this technology was. They just looked up and
saw this guy who they all knew was passed away. And, you know, in a very nicely
interactive manner Snoop and 2Pac -- 2Pac sings a song he's never sang before and
integrates it with Snoop Dogg.
So, you know, the question is, so this Orville thing is a one-time thing. So with 2Pac, his
mom gave permission for this to happen. And, you know, how long is it until you guys put
him in the Kinect. Right? What is -- you know, we could talk about what the laws are on
this, and that's a different conversation. But there's a lot of big ideas here.
And the data we're collecting now, forget about the stuff I talked about early on in my talk
which is I could be a better teacher, a better persuader, you guys could fall victim to my
tools of persuasion. I mean, this is big stuff. Any of you guys that have a Facebook
profile with a hundred photos of you, the things that the people in this room could do with
that in terms of building a perfect version of you is wonderful or scary, depending on your
point of view.
So again, my book is called Infinite Reality. It gives lots of detailed discussions to all this.
If you enjoyed this talk, please do read it. And I look forward to your questions.
[applause].
>> Jeremy Bailenson: Yes, sir?
>>: Yes. I have a question regarding the [inaudible] simulation. When you mimic
someone. How good do we have to be to stop this belief? When we know for example
that if [inaudible] if the measurement lags by some value, then it disconnects. Do we
have this kind [inaudible].
>> Jeremy Bailenson: So there's -- so since we did this study, there's been about 10 or
15 studies in VR. And we haven't looked at lag in particular. But let me give you some
examples of things we have looked at.
So we've looked at the -- imagine that -- so the mimicry we did was the -- just take head
movements. Imagine that you swap the axis so when you did this, I did this. Okay? So
that's making the mimicry more like intra-- so it's a fine line between mimicry and
interactivity. And what we demonstrated in kind of a psychophysics manner is how the
effect goes -- gets smaller and smaller as a function is abstract in the technique. But the
one thing I will say is that the effects are fairly robust that a mere approximation -- so with
VR you'll see this. I mean, what makes this so amazing is that 60 times a second, no
matter where you're looking, they're capturing the position of your head perfectly and
rendering it in that position. So with field of view you're very sensitive. With the
behaviors of others you're a lot more forgiving. Yeah?
>>: I read your book, which I found very inspiring.
>> Jeremy Bailenson: Thank you.
>>: You speak a lots about the narcicisstic background here. But you also talk in your
book about the potential for education, therapy, phobias and things like that.
I guess I just wanted to ask of your knowledge what you find to be the most promising
use of this technology ->> Jeremy Bailenson: Yeah. I hope I didn't come off too cynical. I try to go 50-50 on
studies that are positive and negative. I guess I jump up and down a little bit more when
they're scary. So thank you for the kind words about the book.
To me the two home run applications in my mind are the environmental stuff. I really
think we've got a great potential to connect people in a visceral manner with, you know,
what is a carbon molecule, what is warming? We have a study -- I didn't talk about that's
funded by the Department of Energy.
We're trying to get Stanford kids to use less hot water in the shower. They take 40
minute showers at scalding hot water. And the simulation is they walk in, we build the
avatar that looks just like them. They're in a shower that has a window, and they see
themselves through the window. And they've given data about how much coal it takes for
a 15 seconds of hot water and in order to power that shower -- and this feels really real,
it's a really feeling shower, their avatar is forced to eat a chunk of coal. And then we later
remeasure their water use. And showering with this vividness you get a decrease fairly -so that's a good one. The education I think is -- you know, Stanford is taking a very huge
I think leadership role in doing these online courses.
And my take on this, I wrote an article for the Chronicle of Higher Education which is my
avatar could be a better teacher than I could ever be. And if you take face morphing,
mimicry, the augmented gaze and put them all in a teacher, imagine all of you got the
perfect version of me as a teacher.
And there's a -- many of you have probably read this book called The Diamond Age by
Neal Stephenson that -- you know, so we've got a lot of data on the home-run
applications for education. But, no, I -- sometimes during talks I get carried away. But I
do think especially when I'm talking to you guys that have so much power moving forward
to do the right thing I wanted to focus on the learning, but I appreciate you pointing I
should focus on what we can do well. Thank you.
Yes?
>>: [inaudible] for the avatar instead of the visual is speech?
>>: So as surprising as it sounds, it's 10 times easier for me to manipulate the features
of your face than it is in realtime to do your voice. So there's a book called Wired For
Speech by Clifford Nass which is the treatise on doing these things with voice. And
there's some studies with it. Just it's really hard to transform speech in realtime. Yeah?
>>: I was curious with the kind of self perception study you're talking about, is that kind
of limited to this fully emersed virtual reality or do you -- are you aware that there's same
effect from playing World of Warcraft or something where I'm not literally in the
perspective of that.
>> Jeremy Bailenson: So one of the most famous studies in -- so there's a line of
self-perception face to face. One of the most -- so self-perception states that the world is
a complicated place. And there's no blueprint for how I can behave. I often infer how to
behave from my own behavior. I know that sounds circular. But a good example of this
is if a famous study by a guy named Valen in the '60s, he had people look at Playboy
Playmates. While they were looking at Playmates they had a heart monitor showing their
own heart rate, boop, boop, boop. For half the Playmates he randomly sped up people's
heart rates. For half of them, he had them be the actual heart rate. And then later on
they rated how attractive they thought each of the Playmates were.
For people -- when somebody saw a Playmate for whom their heart rate had been jacked
up, they rated the model as better looking later on. Meaning they inferred their own
attitude from their own behavior. And there's a whole line of work on that Nick has done.
So one of my favorite studies in regard to this is with Nick Yee. He did a study of World
of Warcraft. He somehow got data from 80, 100,000 years of World of Warcraft. And he
looked at gender.
So a lot of people, a lot of males in the World of Warcraft play female avatars. And he
somehow got a very large simple size of men with multiple avatars, some that were male,
some that were female, and demonstrated that when males wear female bodies in World
of Warcraft with 2D nonimmersive they have a much higher proportion of healing to killing
than when they wear male avatars. And there's nothing inherent to the avatar that
causes that. It's just self-perception.
So send me an e-mail and I'll send you some of Nick's work on that.
Yes?
>>: How long does the aftereffect last?
>> Jeremy Bailenson: So I get this question every time I give a talk. And I've got two
answers to it.
>>: Well, and is it impacted by a positive -- like using less water versus negative like ->> Jeremy Bailenson: So let me answer the first one first. My first answer to that is that
we just haven't done enough work on that. And of the 80 studies I've run in this, I've
done longitudinal studies three or four times. And they exercise less. At least 24 hours
later.
I've done studies with obesity where I basically force your avatar to eat Reese's Pieces
and you see your gut grow and your thighs grow. And we've shown some fairly longer
term effects with that. But not enough. You know, now that I've tenure, I got tenure two
years ago. And social science doesn't really reward the longitudinal studies so much.
But the world does. So now I'm going to focus more on the longitudinal stuff.
But the second part of that answer is that kids are spending eight hours a day inside,
right? So as long as you can get them there once a day, I don't think you need to show
that it lasts 80 years. Right? It's -- that answer is kind of a cop out to your question. And
I think -- the second one I'm positive versus negative framing. The answer to that is that
we've manipulated that variable. How important is positive versus negative feedback?
Thus far haven't found many difference. That literature in general is a really complicated
one on what works better fewer appeals or positive appeals. And our work both have
worked. Yeah?
>>: So in your first -- the student lecturer example, one of the indirect effects is that the
student things that the lecturer is staring at them, is ignoring the other students, which
isn't true. And I guess that's the -- then the question is will over time we start to not trust
anything that's out there, and are we heading to Philip K. Dick-land?
>> Jeremy Bailenson: Well, Philip K. Dick inspired a lot of the work that you saw today
whether directly or indirectly. But the interesting idea of this arm's race. So I like to talk
about spam. You know, spam, there's this arm's race between the defenders and the
people who create it. And every once in a while I'll click on one of those things. You
know, they do it smart enough, they put my name in the title, they grab -- they grab
something that -- so you're going to have -- look, for everything I've talked about with this
subconscious persuasion stuff when you detect it consciously, it backfires.
So the mimicry when -- so we've done some studies where instead of it was four
seconds, it's only two seconds and you detect it. And when that mimicry becomes
mockery, it backfires. And, you know, in pilot studies before -- when we were doing the
face morphing before it perfected that technique, if you're conscious -- I mean,
knowledge is power.
How I sleep at night by doing this work is that, you know, I just make these discoveries
and I talk to you guys about it. I don't answer the phone calls from political consultants
that want me to go to Ohio and work in those five districts and get a roomful of monkeys
that are doing morphing the same way that they have people stuffing envelopes.
>>: You see yourself in the billboard you know it's happened, that it still works, right?
>> Jeremy Bailenson: So the way that she -- so whenever you want to experiment, you
want to avoid what's called demand characteristics, which is you know what the study's
about, so the way she solved that problem, that all the -- the woman who ran this, Grace,
she was from Korea, all the products were novel in Korean. And she gave you lots of
them.
So a month later, when she tested your product preference in recall, you didn't
consciously remember the association between yourself and the products, she just had
this implicit life. But, you know, yeah, it's fairly hitting you over the head. But what she
did demonstrate was that you could imagine getting very pissed off at an advertiser for
doing this. But she did demonstrate that. People weren't that pissed actually. Yeah?
>>: I'm curious. You've studied the difference in a lot of these things between children
and adults, specifically like identity queues. Like are children born with this notion of
attractiveness and being more confident when they're attractive versus ->> Jeremy Bailenson: So that's a question far above my pay grade, the nature versus
nurture. To answer the specific question, we have only run a couple of studies with
children. The universities make that a fairly complicated and a sad fact of social
sciences, 99 percent of the work as college sophomores.
Send me an e-mail, please, and I'll send you the three or four scholars I know who are
looking at stuff developmentally. What I will tell you is that VR -- kids take to VR like fish
to water, man. They are -- when you see this pit, you're going to be a bit hesitant. Kids
are doing canon balls in there, and they are fairly happy. [laughter].
Yes, sir?
>>: So to kind of build on what the gentleman over here mentioned about the whole
Philip K. Dick dynamic. I'm kind of gravitating back towards the children that swam with
the whales and how they kind of had these manufactured memories.
>> Jeremy Bailenson: Yes.
>>: Which kind of touches on the do we -- we can remember [inaudible].
>> Jeremy Bailenson: Yes. Yes. Thanks for using the correct [inaudible].
>>: In your -- in your opinion, do you think there's any -- anything positive that can come
from manufactured memories that sort of -- that VR creates?
>> Jeremy Bailenson: Look, again, I'm a social scientist, not an ethicist. And I
appreciate the question. And I'm going to cop out in the answer, which is you know in the
book we've got a chapter called the yin and the yang of the stuff. And I want you to think
of avatars and VR the same way you think about uranium. It can heat homes and it can
destroy nations. And, yeah, if I've got a fairly traumatic event and you can knock it out,
what makes these stories so brilliant is that there's an appeal there, right? It's not -- it's
not 1984 them telling me which country I should hate. This is -- you know, why wouldn't I
-- you know, travel -- why wouldn't I take a virtual vacation? You know. It's cheaper. I
don't get germs. There's an allure to it. But, again, I -- what I can tell you as a scientist is
that kids and adults have a hard time differentiate -- there's a theory called media
richness that really kind of explores, you know, the richer -- the richer a media event, the
more likely you get something called source confusion which is, you know, if you see
something on TV and it's very detailed and it's high def, you're more likely to, five years
later, to, you know, basically tell someone else you were there. And it's not because
you're being evil, it's just because that's how we are. Yes?
>>: I worked with Jaron back in the '80s developing the first [inaudible] and, you know,
over time [inaudible] more higher resolution we found that simulator sickness was a huge
problem because there's no gravitational cue unless you're on a motion platform a lot of
people will [inaudible] using these kinds of things.
Has there been any solving of that problem?
>> Jeremy Bailenson: So did you go next door and try it out yet?
>>: No, but I get simulator sickness ->> Jeremy Bailenson: Okay. I suggest you try it. You can always close your eyes, you
know, the two seconds. So nobody knows exactly why simulator sickness happens. And
Andy is probably better equipped to answer this than I, because he's more of an expert
on perception.
But what I can say is in our time running we get it a lot less. One of the reasons you got
it before was from lag. Meaning you moved physically and it takes a while for the world
to catch up. Andy -- neither WorldViz nor myself ever run at less than 60 frames a
second in each eye. You do get something called the convergence problem, which is
when I see an object far away, eyes focus in a way and everything in the HMD that looks
far is close. So that hasn't gone away. That's an H -- that's a VR thing. So that could be
driving it. But it's a complicated literature on simulator sickness. People don't ->>: I guess the second question is I'm doing a lot of research myself with
HealthPresence and autism. We're finding a lot of kids who use HealthPresence
technology in communicating in a much more effective way.
>> Jeremy Bailenson: Yeah.
>>: Before and after. Is there any research you're doing ->> Jeremy Bailenson: There's about four scholars around the world spending good time
in this.
I did a project -- we got funded by the NIH. A guy named Peter Mundy at UC Davis who
is one of the world's experts in treating autism. And we did a grant -- so this is kind of a
crazy imagine. So as a teacher, I know that me looking at you guys is one of the cues
that I energize my crowd. I pace, I get you guys engaged.
We ran a study where, you know, you guys probably remember the augmented cognition
call from the late '90s. This is augmented social interaction. Imagine that I received
realtime feedback about how much I'm engaging you guys. And the way we ran the
study, this is with adults, normal adults.
If I'm not looking directly at you, your avatar slowly starts to disappear. So what I get is a
realtime cue that's very easy to perceive to give me, as a teacher, an aid to spread my
gaze around and to include you in the conversation.
And what we've demonstrated is when you give the teacher that simple cue, the students
learn better, the teacher spreads her gaze better.
Peter read this paper and said autistic children have a very -- high functioning autistic
children have a very hard time engaging in nonverbal social behavior. What an amazing
tool for them. And we've now run two pilot studies. He just got a new grant on this as
well, showing that when you give high function autistics the abilities to give realtime
feedback about their nonverbal social behavior that they improve.
So it's a huge area. Roz Picard at MIT does some stuff. Send me an e-mail later on, and
I'll give you the list of the four or five scholars doing it. But, you know, in general, to take
a step back, autistic children are comfortable in VR. This is generalization and it's some
opinion. But based from my conversations with Roz Picard, it's a nice technology to help
that population. Yeah?
>> John Tang: So perhaps we should make that the last question. But I encourage you
to check out the demo that's just next door. And Jeremy's around to answer questions
for a while. You have free time before your next [inaudible] but join me in thanking
Jeremy again.
[applause]