>> Eric Horvitz: Okay. We are honored today to have Zoran Popovic with us from the
University of Washington. Zoran is a professor of CS at UW and he is the director of the
new Center of Game Science that you'll be hearing about. He has long-term interests in
computer graphics and interactive games. Over the last few years he has gotten deeply
involved in scientific discovery through game play, learning games, high fidelity, even
modeling in animation older interests, I guess. Many people here are familiar with his
labs. His teams work on Foldit which was a fabulous interactive game around
biochemistry, the folding of proteins that CS people always like getting results published
in Nature. There was a Nature article on this work. There is a second one also there?
Keep me going; this is a crowd source introduction over here. But in general, you can
read his bio online. He has many fabulous awards, and I will say all well deserved and
today he will be talking about solving hard problems in science and education with game
based symbiosis of humans and computers. Zoran.
>> Zoran Popovic: Yeah. That was a mouthful of a title, but I have a lot of things to say.
Thank you for inviting me. So the first thing I will do is I will give you a little bit of
perspective on how I think and what I think the really big problem that I'm trying to
solve; I am sort of thinking way out in the future, and before I start anything I really want
to mention, of course, everything I say today as you all know who have been in academic
enterprises is work of many people and the more people you have, the more you become
less involved in the entire project. It is just impossible to keep everything on track, but it
is also so important to mention a lot of these people.
What I am really trying to do is create a symbiotic human computer architecture,
something that can actually solve problems that we cannot solve by computers alone or
people alone. So that means a game that just tries to harvest people alone only has a
certain amount of ability. Massive amount of computational power has a certain amount
of advantages for different problems, but what happens when we can merge those things
together and actually involve them toward something that simply is a different kind of
power in terms of what it can solve?
So as I was thinking about this for the last maybe three or four years, what became
apparent is that to actually create this kind of architecture what needs to happen is a coadaptation of both people and computers. And what I mean by this is it is impossible for
people to really solve really hard problems just by stumbling upon the problem and
saying oh, I know the solution to this. It just never happens, or maybe it might happen
occasionally, but really the strength of people is not necessarily serendipitously coming
to solutions, but actually persisting on a particular problem and developing expertise in
the process. So the question is how can we have an infrastructure that brings arbitrary
people who perhaps care but don't know much about the topic to the level of experts that
can actually solve those problems.
Similarly, how can we create programs, games, whatever software development you
might want to think of such that it actually becomes the optimal performing thing in this
symbiotic process, not just as a computational engine by itself? So the question is how
can we have a mechanism of co adaptation that can bring programs towards optimal
problem-solving tools when they take people into the account. And so, you know, I
would much rather not have to do this with games and do it in some other framework.
The problem is I can't think of any other framework that can actually keep people
engaged for extended periods of time where they can actually go to the point of actually
been highly capable of solving problems. And similarly, no other place where I can
attract huge numbers of people interacting with a particular software in order to gather
data and in order to optimally adapt the software to be the best performing for that
particular problem. So it is that particular engagement and the amount of data that games
can actually collect that makes games particularly interesting as a vehicle towards solving
these kinds of problems.
So that is exactly what this center is trying to do. We actually now have broken forty
now with a bunch of undergrads over the summer, but I have never been around an
operation this big and it looks like it is going to grow even further, so we might be
moving into a different building at the University as well. But it is a very exciting place
with PhD's, undergraduates, a large number of software developers. We almost had ten
software developers now and there are still more game designers and artists. We
currently have six games currently in development, so I guess when you think of it we
kind of like a little small game development company only we are in a university. We
are not making any money whatsoever [laughter]. Nor will we, we are already public and
we will never go public, but the most amazing thing about it is that we are fearless. We
don't care to fail and we are picking problems that nobody else in their right mind would
pick.
So that is why is exciting to do this thing. So as I mentioned, our challenge has been first
is how to make an entertaining game. Now of course if we knew the answer to this
maybe half of us would be millionaires just making games and collecting money in our
bank. And of course this is a very hard thing to do, and I will get back shortly as to why
and how this is hard. But of course it is even harder to solve actually a real problem
together while making an entertaining game. This is particular because, you know,
reality or what you are actually trying to do puts huge constraints on what the game is. I
cannot now say, oh my boss level is going to do exactly this, because it is irrelevant. It
has to be specifically tailored towards learning or some kind of discovery. How do you
actually ensure that you actually made a real discovery? So there has to be some
feedback on to the lab or some way of knowing that you have really learned something,
rather than just playing this game.
And then finally, you can make money and make an entertaining game for a short burst
and still make a lot of money and not care that nobody plays your game three months
later. This is not good enough here. We are actually looking for long-term involvement,
so even some ways of making money some ways of making money in the gaming
industry is not good enough for this. So that is a challenge, a pretty tall one. And it is
impossible to separate these two objectives. It is really a multi-objective optimization, if
you will, in how to actually solve these two things. And just to demonstrate to you how
little we know about even making the entertaining game, a lot of you have seen books on
how to design games et cetera. A lot of them have postulated a whole bunch of stuff.
But none of them are actually backed up. And here is an example of one of them, a
secondary award effect. So if you play games, the secondary award effects are thought of
as these extra things you need to do in a game. So think of it as the little coins you have
to collect along the way as you actually are trying to find the way out of your labyrinth or
whatever it is. So you will read in many books that of course using secondary award is
good because it actually provides people multiple ways to play the game, to have
multiple challenges so that they are not bored just with a singular goal playing the game.
And the assumption is that you will lose fewer players over time if you actually add this
secondary award process.
Turns out tested over 40,000 people playing on multiple games in different genres for
online games, this is not for games you pay $60 for, this is completely wrong, okay?
What happens is this. So basically there are about 40% of players that you lose sooner if
you have secondary awards than if you didn't have them at all. And only the last 7% of
diehards ever really benefit from them. So if you're actually trying to make a game that
you lose the least amount of people, you actually want to cast the widest possible net to
attract the most number of people to play a game for the longest period of time, using
secondary awards as orthogonal objectives towards the main objective of the game, is a
wrong thing to do.
>>: What did the authors of the study conjecture is the reason behind this?
>> Zoran Popovic: We conjecture that, in fact, we went a little deeper into this exactly to
answer this question, so we actually said so what modality can secondary awards actually
be beneficial? And what came out, and this is interesting because we didn't do an AB
study here, we did a multivariate study, where we basically had sort of a hierarchal way
of asking questions deeper and deeper as we had 40,000 people playing this and
congregate. And so the answer is that the closer your coins get towards supporting the
main mission, the more this red space reduces and you start getting benefits earlier. And
then if you look back and you look at Miyamoto, the guy at Nintendo that makes super
Mario Brothers, what they do with coins as they always put them along the arch of where
they want you to go. So this is basically, so they are supporting the main mission. So
somehow Miyamoto knew that and other people didn't pick up on it. But maybe that is
why he's so much better than most people in design itself.
So basically what that means is there is something about the cognitive load of people-this is now postulation. We have no way to back this up, that basically if you present
them with too many things and there is a challenge in each one of them, they will quit
sooner. However if you provide sort of multiple mechanisms of short-term and long-term
incentives along the way of where you really want them to go, where they will most
likely succeed, you keep them longer. This study itself is not important. We were
actually trying to develop an analysis for tutorials, what are the optimal tutorials for longterm awards with the idea that we can actually create the theory based on a huge amount
of data. Everything has to be statistically significant and developed and understood in a
nuance, because as you can imagine there are so many variables that can eventually offset
this in one way or another. So the question is do you really have true understanding of
what's going on.
So as I said even that first thing is not necessarily no. And when it comes to actually
trying to do something significant by novices, this is sort of the standard approach which
I called trickle-down approach. We know that it doesn't necessarily work in economics, I
hear postulate that it doesn't necessarily work in learning. So the idea is you always look
at the idea of how the people that were really good at this particular task do it. I'm going
to analyze it. I am going to study it, and then I am going to actually create a tool that
somehow makes it easier for novices to do exactly what the experts are doing. And the
assumption is that then give it to everyone and before you know it everybody in the world
is going to be using it because it is so much easier and the fantastic outcomes will come.
Unfortunately somewhere along here, it doesn't happen. People don't end up caring about
this stuff you made. And this is sort of the big problem with, I mean in HCI you see that
a lot; there are huge number of things that make things easier, but it just doesn't go wild
when it is actually released.
Maybe you haven't pushed the boundary all the way down, but maybe something else is
wrong. So here is my approach, and it is a very different way of thinking about things.
Forget about the expertise; create the incentive structure first. Many people really care
about wanting to find things out for themselves, or basically spend time in a process
where they can actually develop expertise. This expertise is going to be different than the
expertise of what people are really doing out there in the field. And I will show you in
Foldit an example of how that actually ends up happening. They're going to have a
completely different language from what the experts are doing. But they are actually
going to be in it for the long run. They are going to have this intrinsic interest in doing it,
and as a result they will persist on it for a long period of time and perhaps that might lead
to discoveries and true learning.
So that is kind of the fundamental way in which I'm thinking about basically enabling a
lot of things to this path from novices to experts. Yes?
>>: [inaudible] implicit? Can't you just have the incentive structure with the tools? Is
there a problem with having both?
>> Zoran Popovic: The problem with the previous approach is that obviously there are
ways in which you can use prior knowledge in many different ways, from people who are
really good at doing it. In fact, people that do really well in the game can inform other
people as to how to play it, but the problem with this is that you don't actually; you're
basing these tools on your interpretation of analysis of what the experts are doing. But it
is not necessarily the case that regular people will follow the same pathway. It is not
necessarily even the case that all people will same the pathways. It is never the case.
And as I will show you, there is a huge amount of variety in how people arrive at their
expertise or what kind of expertise they get such that collectively that expertise now
becomes sort of created as an aggregate and is more powerful than an individual that then
ends up arguing with another individual that says no, I am an expert too. So it is kind of
a different way of doing it.
As I said, I already pitched the games as sort of fundamental things, so I will just give
you some of the key insights into what actually went on in Foldit. How many of you
have played Foldit? Okay. Then I will have to go through a little bit it. So proteins are
basically these large necklaces you can think of them of amino acids strung together. So
it is basically like a very long chain and there are 23 or however many there are amino
acids, so basically there are many ways in which you can combine things, and each one
of those amino acids has a particular letter, thankfully there are more letters than there are
amino acids so you can just encode them with a single letter. So basically the point is
you have a sequence, and actually now the technology is such that I can just send the
sequence to a lab, pay $5000-$10,000 and I get that protein sitting in water. The question
is what is the actual way in which this protein assumes the shape in the cell or in the
water? And the shape itself governs the function and it turns out for those of you don't
know, proteins do everything in a cell. Everything that needs to be done mechanically
from muscles to digesting to antibodies that are defending against disease; everything is
the work of proteins. In fact you can argue that DNA all it is doing is just encoding how
to create proteins to actually do stuff.
So anybody who knows how to take a sequence and create a particular shape that
achieves a particular function knows the secret to life, essentially. So this is why the
problem is actually important. Did you have a question? Okay, no.
So it turns out different sequences produced different structures, and different structures
have different shapes that then interact with molecules in different ways and that is how
the function arises. And of course any sequence can be generated; nobody knows what
the shape is. And then people, there are actually x-ray crystallography methods in the lab
that actually can do this and I think I just read somewhere that the United States itself,
don't quote me on this, but I think it was like $5 billion a year is spent on x-ray
crystallography trying to figure out the particular proteins, and of course the process is
very slow and there is a whole bank of proteins that are unsolved still because experts in
x-ray crystallography cannot fully resolve them. There are actually a huge number of
screensavers around in the world trying to do the same thing computationally by doing
sort of mostly sort of simulated annealing type merge with a little bit of continuous
optimization. In this process they are not very successful, as of now they are actually less
successful than the x-ray crystallography but of course if somebody unhooks this
eventually, it will be a lot cheaper.
These are the two examples. This was at the University of Washington, David Baker, my
collaborator with Foldit has one of them and then Folding at Home is actually not
predicting the shape, it is trying to simulate it. The problem with simulation is, they can
simulate maybe 100th of a second; it takes millions of computers to simulate, so you
basically imagine that they are not even close to actually folding the entire protein, or
projecting how different shapes would end up folding the protein. So here is the
challenge.
>>: So if it's simulated they don't try to solve…
>> Zoran Popovic: So imagine the process where you start with a protein and then you
have to wait for the simulation to finish to see what kind of shape it may or may not have.
It basically, and then you have to experiment with…
>>: [inaudible].
>> Zoran Popovic: In this case it is dynamic, so yeah, so that's the fundamental
difference between these two things. This one tries to predict the whole shape, so it tries
to avoid all of the dynamic stuff, so you might miss something where the final shape is a
good one, but it could not have gotten there from fully opened state. It turns out that in
nature it rarely, rarely happens, so the alternative thing is to study the actual process, but
it is so computationally intensive that it is infeasible.
>>: [inaudible] somebody else [inaudible].
>> Zoran Popovic: This is at Stanford and it is actually running not only computers but
even on PS3s I believe all over the world.
>>: Do you know [inaudible]?
>> Zoran Popovic: That is another very good question. There is an energy function that
people are building, and they basically feel like they can predict to some extent. Of
course it is not perfect, and the only true test is when it actually finally goes to the lab. In
this case with the computers and what everything else is serving as producing the right
candidates in this vast search space to send to the lab.
>>: [inaudible] figure out how they connect?
>> Zoran Popovic: They Foldit and then do x-ray crystallography. So they pay for the
synthesis which is $5-$10,000 and then they do x-ray crystallography, and I think I
don't…
>>: [inaudible] already known the structure risks or how it processes [inaudible]?
>> Zoran Popovic: Because it would take you like several months and I don't know how
long it actually takes. So the idea is--and it requires people; it requires everything else.
>>: [inaudible] collection to just see if it can crystallize a lot of the proteins is very, very
hard…
>> Zoran Popovic: That is correct.
>>: [inaudible].
>> Zoran Popovic: I will touch on that shortly. We actually have a way of making
crystallography more; in fact our second nature paper is exactly that. There was a protein
unsolved for 15…
>>: [inaudible].
>> Zoran Popovic: I will get to that later.
>>: [inaudible] I want to ask you a biology question. So there isn't necessarily just one
configuration, confirmation of these proteins, so you can have several candidates that are
[inaudible] active so you [inaudible].
>> Zoran Popovic: I can make a sequence that just never folds into a coherent structure.
It basically just flops in the water.
>>: In the medical [inaudible] you to get resident structures, right?
>> Zoran Popovic: There are some structures that actually have two modes, yes, and in
fact nature sometimes using that to actually maybe sort of, yeah, yeah. So I mean it is a
great play space, right? But even through optimization you can actually see two big
values I can basically say. Two possible…
>>: [inaudible] that was part of the whole gaming [inaudible] two solutions, not just one.
>> Zoran Popovic: That's really hard. I mean for most of this stuff the synthetic logic
would like to use you want something that has a huge bottom of this energy function that
is very strong we confirmed.
Okay. So that was our challenge; that was our problem. We wanted to solve it by people
who knew nothing about it, while ensuring that they have fun. But at the beginning we
didn't know how to even teach the biochemistry that it needed to know. We didn't know
the appropriate way to present the problem to them was. Because one thing we could
have done is just give them what the scientists look at. And then we didn't know what the
appropriate human centered tools are, because the tools, the automated methods used, are
they the same ones that the people should use, or not? We basically knew nothing. So
the question is how do we then take these proteins and turn them into puzzles to solve
these problems without knowing anything.
And the answer is well, approach it as a science. Assume that you're not going to know
anything and basically start to develop the actual process by discovering along the way.
So what we did is basically we iterated the game design towards these using several
different sorts of avenues and primarily data driven analytics. I am not going to show
you Foldit, but the idea is that this is the actual game. There are a number of other things
that are in it, but suffice to say that the proteins look very different from what the
scientists are looking at now, so this again if we started from experts and just gave them
what the experts had, we wouldn't get to this. You can do all sorts of really weird stuff,
like apply rubber bands between different proteins. So this is how people interact with it;
they say I want to bring these two pieces together. You can freeze some regions so they
turn into ice, so they can actually no longer move while some other regions can move.
There are hydrogen bonds that are actually indicating this, so these are parts that actually
contribute to the energy, so the idea is you want to create more of those to get a better
score. There are some of these things that are greasy, meaning they are hydrophobic,
some amino acids they don't like water. Therefore they should never be on the outside of
the protein; they should be tucked inside. When the protein is in the water, they should
be away from the water. The hydrophilic ones, of course, are the opposite and so they
should be on the outside. So you can basically see the greasy stuff is coded as yellow and
blue stuff which is the stuff that loves water.
There are a whole bunch of these amino acids that are not seen here at all, because we
realized that looking at the backbone without so much extraneous detail was skewed so
that basically there is a dynamic way in which you only see amino acids when you
actually need to see them for a particular purpose. So here is our structure. We take the
open problems; we basically pose them as multiple puzzles. They exist for maybe a week
and are played on many game plans and the solutions get back. They are basically
analyzed for the energy function and for how well they explored the entire space, which
then causes us to produce a new set of puzzles and this iterated process actually
hopefully, leads to discovery.
>>: [inaudible]?
>> Zoran Popovic: There are different ways, but the one that we use most is basically the
connectivity map. So basically have you explored all possible ways in which different
parts of the molecule can interact with other parts of the molecule?
>>: [inaudible] pairwise?
>> Zoran Popovic: Yes, that particular one was pairwise. So a huge amount of player
data is then logged and analyzed and I will tell you a little bit about that. And then we
have basically game updates streaming almost on a weekly basis with new updates based
on what we have learned from this process. Yeah?
>>: Do you have any expert in there or is it all just fully automatic?
>> Zoran Popovic: The expert is actually looking at this stuff and then it is saying okay,
maybe our scoring function was not great. People scored very highly, but it is not related
to the science itself so we have to adjust it to some extent and then creates new puzzles
with different modes of play.
>>: [inaudible]?
>> Zoran Popovic: The scoring is the energy function…
>>: [inaudible] lab.
>> Zoran Popovic: That's right. Scoring is the state-of-the-art energy function that
scientists don't know any other better way to score it. It is still flawed, as I mentioned,
but it's the best we know without actually going through the wet lab.
>>: [inaudible] playing this game. You said how many people…
>> Zoran Popovic: I will get to that. Since this is a scientific visualization. This is our
visualization as I mentioned. We do things like clashes, you know, places where
molecules are at the same place. There are voids, so obviously inside of the protein, most
natural proteins are very tightly packed on the inside so every time you see a void in the
middle, and this is the stuff that you want to eliminate. So that is something that is also
visible to them. I mentioned hydrophobics and the hydrogen bonds already. The
interesting thing that we did is then we said okay, we are going to actually allow them to
change everything in this visualization framework. And then we are basically going to
look at the data that comes out of those who actually perform really well for a particular
subset of puzzles. So now not only do we know whether the most things conducive to
visualizing the solution, but we also know how to adjust that for different kinds of
problems. So it is not a standard way to look at it, but different types of problems have
different ways to look at it. So what we are doing is basically, always analysis of new
emerging experts, or protein savants as we call them, which then informs the
development towards enabling novices to sort of more likely see things the way that those
guys do.
>>: I understand the slide. Are you saying that these would be controls that users are
using and you watch what the great contributors do and look at their configurations?
>> Zoran Popovic: Not only that, but we actually have it-they can change that at any
time, so actually different stages of the game, they actually change the visualization.
This is the stuff that we can see and then we actually know how the visualization is
helping their process. So you can see that all the versions we have these glows. It turns
out that they were actually terrible in terms of helping people see.
The next thing that I will tell you about is interaction, so obviously we took some of the
pieces of the optimization puzzle from the automatic method, and then we actually
exposed them to people. Now of course some of them last for a very long period of time
so they are actually completely useless for people who cannot wait that long and need
immediate feedback. So we basically altered the original optimization method to be
shorter and sort of visually indicative of what is going on, such that people can not only
run an optimization but tug at the same time as the optimization is happening. So we turn
optimization into sort of user guided optimization through considerable space. Nothing is
actually just, well there are some that you just sit and watch; there are actually people out
there that create these very long macros that run overnight, and then there are people that
just totally don't believe in macros and they just want to do everything manual, so it
spends the entire spectrum.
So one way we tested do our interactions actually work, so what we did is we actually
started with a completely unfolded protein and we said now here is the native. It is given
to you as a guide. Can you actually use the tools to fold this into a native? And so
basically that was our test with which we were basically refining the tools and we are
saying which set of tools can let people quickest from intention to what they wanted to do
to the actual realization. And, of course, we wanted to know can they actually get this
solution? So once we have…
>>: Now when you [inaudible] this optimization [inaudible] how interactive was it?
Then what did you get out of that, and actually…
>> Zoran Popovic: You needed like--basically we have 30 frames per second updates.
>>: [inaudible] deposition [inaudible]?
>> Zoran Popovic: That's right. So what is important is that even if the optimization
does not finish, I mean they don't actually know if it finishes or not, right, because it is
always moving somehow. The person should be interacting with it at all times, and that
is the difference. So never are you a passive sort of participant.
>>: It's nice to know that you had [inaudible] interactive rates that a secondary to delay
can actually change the whole experience.
>> Zoran Popovic: Yes. Most of the algorithms that we have ended up using are
actually modifications of the original algorithms with that in mind.
>>: [inaudible] optimization then or is that a function that you can just [inaudible]?
>> Zoran Popovic: Obviously we can, scoring is our grand truth in the game, so you
can't really play with it, I mean, we could optimize it towards engagement [laughter], and
come up with a very different score, but of course the scientists would no longer…
>>: [inaudible] deterministic once you have a particular [inaudible].
>> Zoran Popovic: Yeah. So here's the story about the energy function. This function
does not necessarily guarantee that it is the right function, but for all of the natives when
you verify it, they actually lowest for this function than anything else. The problem with
it might be that there are many things that are very close to the energy function, so
basically they haven't really distinguished that solution way more from the other ones.
But it is a, you know, as I said, there is nothing about the game that doesn't preclude it
from including new state-of-the-art over time.
>>: How the users find out whether or not they [inaudible]?
>> Zoran Popovic: They know it immediately based on the score. So the highest score
wins. Highest score is the lowest energy function, so we basically inverted the energy
function.
>>: But they don't have any guidance about what they should do next in order to lower
their score [inaudible] or it's all in their head?
>> Zoran Popovic: I will tell you a little bit about it. That is the hard problem. So if it
was well structured it would be easier perhaps even computers can do it.
>>: [inaudible] that others have done [inaudible] on that?
>> Zoran Popovic: Yes. They see the best score. There are many different ways in
which they can see. In group play they can share solutions with others. I will tell a little
bit about that. So actually this is the energy function. There is also rank of your score.
There are basically two different ways you can play, as a group, as a soloist. Initially that
was not the case, but then it started being obvious that basically it was the groups that
ended up doing better than the soloists. And then even for individuals initially it was
obvious that we had to do completely different types of scoring, because some people
were very good at starting from zero and evolving. Some people were very good at
taking other people’s solutions and making them better. And they were getting scores on
the shoulders of people who got there first. And they were saying this is not fair; these
guys are stealing my solution. And I'm like oh no, mutiny. And then basically we very
quickly changed the actual scoring such that we completely recognized soloists and
evolvers as different scores. So basically if you take somebody else's solution, you no
longer have a soloist score, you have an evolver score for that particular one. And that
actually stymied that revolution actually.
And then a final thing of course is training. So how do we take this big thick
biochemistry book and many other things that are perhaps unnecessary, and turn it into a
set of levels? And that course took us a very long time. There were some really bad
levels that we had initially. And here's how we did this. This is actually just one of the
visualizations. There are many others. But here you basically see a number of levels.
There are names on there. But really they are in sequence in this particular case. And
this is a percentage of people that actually play. So basically after every level you have
fewer and fewer players. And we get maybe 400 to 500 new players still, everyday.
Somehow that function is linear even though the number of people in the world is finite.
So I don't really understand how that is possible, but this is great for us. So basically this
is big. Everyday is a new notch, and what we are basically seeing is obviously what you
want to do is have most people finish. Of course if you just want to have most people
finish, you could just make it easier, right? But what you want to do is make them finish
and have most of them actually being relevant and useful for the actual challenges that
come afterwards, right? So there is an evaluation function over a longer period of time.
So here, for example, you can see that there is a kind of gap between these levels, way
too many people drop off here, so that is basically our development for the next few days,
is how to actually change these two levels in order to reduce that gap. And then we
basically made a particular change, perhaps we didn't reduce this gap, but there is like
this crazy bad thing we did here even earlier. So we lost almost 15% of people we lose
just on this level. So basically we immediately analyze that to look at the finer detail of
exactly what was going on that level. It turns out that two levels, we hypothesized
reversal of order of two levels, and it was the completely the wrong thing to do.
>>: [inaudible] the bottom curve that matters. I mean isn't it only the people that stay
after all the trials?
>> Zoran Popovic: It is true, right? But the question is why if I reduce this maybe this
curve would've been here. So you always…
>>: You have a larger gap, you know, on the right than you did on the left and yet the
overall trend at the bottom seems to be going up even though that…
>> Zoran Popovic: It's true. It's only going up because there is a whole much higher
stuff here. So that line that you are seeing is an aggregate thing over a longer period of
time.
[laughter].
>> Zoran Popovic: So basically what we did is we immediately worked on this, close it
up and you can basically see. So there is an effect of that, combining with the fact that,
and now we have gone from under 5% of people finishing all levels to, at that point of
time, this was two years ago, maybe 17% of people finishing. Again, just finishing as I
mentioned is not necessarily the right metric. But this is basically how we've approached
everything. We basically have what are we going to work on the next two days is purely
data-driven. And so what actually happened? So this is May 2008. We released the
game as a beta. This is now year ago. Now we have almost 300,000, 400,000 people
playing. We have--this is actually our, the entire player group polled so, everybody who
responded, you can basically see they are playing all over the place. In fact, some people,
some groups have like teams in New Zealand that they cover once the guys in the US go
to sleep, so basically around the clock they are working on puzzles.
You can basically see that they have people that come from all walks of life, from
education, science, technical computer, labor, retired, all sorts of people play, and then
one early thing that was obvious was that there were people who were just unbelievably
good for no apparent reason, certainly not for the reason that our game was good. And
then I flew them over to Seattle and I actually tried to observe some things they were
doing and it was impossible. I mean they just kind of see it. I'm like how can you
possibly--you know, I mean there's no--but there are some particular things that they--it's
almost feels like some people play on they feel, which is really, really weird.
So initially they all outplayed the experts, so you can say okay, experts had real job.
They had the real stuff to do; they didn't have time to play a silly game, maybe that is
why. So over a longer period of time we basically looked at the top 20 players and how
well, where they come from and what kind of knowledge in biochemistry they have. So
basically the largest chunk had actually no knowledge of biochemistry whatsoever. Then
the second largest chunk is high school basic knowledge. How much biochemistry does
anyone have it high school? Or for that matter, one undergraduate course [laughter]. So
basically at that point we were already at three quarters of our top 20 players. In fact,
only two of them are professional and they are number 18 and 19 of the top 20 group. So
this to us was sort of a big success. So basically somehow we were able to get people
who just didn't even know about the topic, and all of a sudden they are actually highly
performing.
Alright so let's dig a little bit deeper about people and what do are they good at what is
actually going on.
>>: [inaudible] space that leads to what your previous slide was showing? I'm trying to
say that can you generalize to lots of different…
>> Zoran Popovic: Yeah. Obviously we're spending a lot of time on that now. I don't
actually know if you can actually conclude anything from a single experiment itself, but
we have some things that we are trying out. Obviously this is a spatial reasoning kind of
3-D puzzle thing, so obviously people with that kind of ability are more likely to solve
those kinds of games. But remember this is, this game is about developing people rather
than just all of a sudden them being able to solve it. For example the savants, they are
good at getting to some point, but then unless they develop the sort of more things behind
it, they are not going to be as useful as they potentially can be.
>>: [inaudible] slide and this one [inaudible] cut the expertise time for these people
versus the dynamics of what happened over time if there is a way…
>> Zoran Popovic: Yeah. That was actually--it is actually even more fascinating. The
expertise does not stay with people. It stays with the game, because basically even our
best players in that staying for maybe 3 to 6 months. Then they get a baby, you know,
life, they no longer play ten hours a day [laughter].
>>: [inaudible].
>> Zoran Popovic: Here's the thing. Some of them come back two months later and try
to play really hard, and they have a hard time breaking the level where they were before.
So what happens, I mean I will show you. There is basically, they create wiki pages that
they talk on and chat over time between them about their new tools that are being
developed in the game by players themselves, and sort of the aggregate knowledge grows
and the kinds of ways that people use the game grows, and somehow if you are not
keeping with the curve, your expertise somehow…
>>: [inaudible].
>> Zoran Popovic: Yeah. So it's a really interesting…
>>: [inaudible] not absolute [inaudible].
>> Zoran Popovic: Yeah. I mean, that is true too. It doesn't change; the ultimate
objective is still the same. The tools have not evolved so much in two months, but
somehow the number of people playing and how they are playing has changed.
>>: What is the motivation for these people? Is a fun or is it hey, I am helping to solve
all of these problems…
>> Zoran Popovic: It is very different for different people. And this is one thing in our
design and maybe a different way scoring that we had to think about it instead of standard
game design languages. You know, if they game designer makes a game that caters to 15
to 35-year-old white males, it is going to do pretty well. There are a lot of people like
that out there, and there are a huge number of companies that make millions of dollars
doing that. But here, that was not our objective. We knew really needed to cast the
widest possible net. So basically what we were thinking about was, what is the sort of
collection of incentive structure we can present at the same time, such that we have
people playing for completely different reasons. And if you look at most games that are
actually successful in that way, they also have those same kinds of mechanisms. So
basically some people play for social effect. We have basically grandmothers with high
school education who stay there, you know, ten hours a day and the greatest enjoyment
for them is staying, basically meeting new people coming to play and helping them
through the introductory levels to understand. That is the reward that they get from the
game.
Other people obviously like to be on top of the hill. Other people really thrive on
collaboration. There are other people who really thrive on creating recipes, which are
really macros in this world since most people really will be able to use it. So there is a
whole number of things, again, evolver or, soloist, it is a different hook for different types
of people.
>>: And recruitment is mostly viral people [inaudible] friends?
>> Zoran Popovic: At this point, yeah. We have a huge peak every time somebody in
the New York Times talks about it and our servers are basically at the mercy of [laughter]
of what is going on, and then it dies off. And then that curve that I mentioned somehow
it's just like, we have this steady underpinning of people who just through ether are
hearing about it.
>>: [inaudible] player? Is like the aggregate score they collect over months, or the best
they do in [inaudible]?
>> Zoran Popovic: There are two new puzzles a day, so this is another really important
thing. Every time you want to have continuous engagement of people, you need to
constantly inject the material. So World of Warcraft does that all the time. If it was just
static, it would be a very different thing. So here we have an advantage of actually
putting new challenges all the time. There are always different sciences. The science
behind it is always different; there is a new type of thing that is different. As you will see
later on we are now going to design so people are making their own proteins rather than
predicting the structure of existing ones. But I will get to that. Let me get to this because
this is sort of a very interesting slide. If you look at how, this is sort of a set of tools or
strategies that people use that we can encode. And basically this is the amount of how
much they use that particular tool is the amount of that color. So you can basically see
the different players use different tools in a different way.
Moreover, for different types of puzzles, people, the same people use the tools in a
different way. So people adapt to a different puzzle and change the actual strategy.
Furthermore, given the same puzzle, but now looking at different points in time for that
puzzle, people change their strategy for different stages of the game. And that in some
sense is the biggest power of people. They search over the space of all possible strategy
as opposed to sort of doing one strategy really well. And this is sort of the standard
protocols that computers do, right? There is iteration of three things that they basically
do for the same amount of time at all different stages of the game, as well. And they
don't adapt to the type of problem and change the strategy based on the problem itself.
So this is the actual amazing thing. Because if you think about it, people can sample way
fewer like several orders of magnitude fewer samples in this large space compared to the
computers. Their advantage is those few samples that they have are very carefully and
judiciously crafted. So they have actually the sense of picking the right places in space to
search. That is their only advantage, because certainly in terms of mass volume of what
they can task, is much smaller.
I mentioned sort of there are a huge number of groups. You can basically see the groups
themselves have their own rank and their score. They have the manager. They have their
own secret way of communicating. You can see how they perform on different sets of
levels. If you look at sort of social competition, this is now energy so the best is low.
For different groups, one thing you can see is one group had a breakthrough and they
found the particular way to refine the problem to be the best performing one. What
happened for the other groups is they were oh crap, these guys had a particularly not--we
don't know if that idea trickled down or what, but you see a huge number of other groups
racing to become better shortly after these guys went to the first place. So there is
basically this weird social effect of actually somebody doing better promotes you from
actually doing better, either by trying harder or by using the sharing mechanisms to learn
what kind of strategy they have used and trying the same thing.
There is also sort of collaboration, so every time you see a vertical line, this is within the
group, somebody actually took somebody else's solution so here you can see there are
two people collaborating. One of them advancing to some extent and then somebody else
pushing it over so they are basically leapfrogging each other to get to the solution.
>>: Was all this done in one day?
>> Zoran Popovic: No, no. Every puzzle--yeah, so this is a week. So basically puzzles
are usually weeklong challenges. So to test how people, how much people are better than
computational methods on some of these problems, we picked ten structures that nobody
knew the solution to except for the scientists who were just about publish. And actually
it was really hard, because we had to wait for scientists to come to that exact stage where
the paper is accepted but delayed for publication. And basically we did a lot of analysis
of what actually happened in those trials. So here is a very interesting thing. What
happened is this is a trace of a person, and what you're seeing here is that the person is
willing for a long period of time to deal with a score that is very low, only to finally get
to the point that they are better than they were originally. And this is equivalent, if you
look at what is happening here, is they are actually unraveling the protein and re-ordering
these helices, and then folding them together again.
>>: [inaudible] the feedback is as they do this they should see the score lowering…
>> Zoran Popovic: There is a score, so the score sucks for awhile and they know it.
>>: But they actually see that they are getting feedback on that [inaudible].
>> Zoran Popovic: Yes, that's right.
>>: How tolerant are people to disentangling it?
>> Zoran Popovic: Well, this is the thing, if they have a hypothesis, in this case I am
going to basically take this thing and I am going to open it up, put it inside and make a
sheet out of it. So that is a hypothesis. I'm going to work on this for several days with a
crappy score, until I actually get to this place that I believe is the right structure. You can
easily imagine the computational methods would have a really hard time having that kind
of sort of first creating visually the idea of what should happen, and then persisting for
that long a period of time.
>>: [inaudible] which is you are dealing with what seems to be a slight modification of
the classical optimization algorithm used by biochemists, if you create an application
package that is built to be complementary [inaudible] such that for example [inaudible] I
don't like that optimization, I want to make a sheet here and then have the system that
moves faster to sort of take the high level commands and do the work. Wouldn’t that be
a better kind of complementary computing scenario?
>> Zoran Popovic: It is. Remember, we are doing exactly that in the game. There is a
period of time where people actually let the optimization explore their particular
hypothesis. So I will get to that shortly.
>>: [inaudible] higher-level and not do all this work. Today I want to make that a sheet,
do your best to put that sheet inside there…
>> Zoran Popovic: If you just say, put that sheet in I think the computational methods
would still even have a hard time with that, because the question is how do you unravel
it, and there is actually a lot of work involved in there. It is not…
>>: [inaudible] for example, you could talk to the program and say, show me another
optimization [inaudible] and I will go from there instead. It's something that will
[inaudible] genre of like high level commands to the optimization algorithm. It is
supposed to [inaudible] point wise.
>> Zoran Popovic: I will give you a little bit of insight into something similar that we
we’re trying. Here is basically a map of that. In this case we know the native so the
closest of the native is here. This is the energy function, so it is not necessarily the
closest to native is the same thing, but you would expect that the native is both here on
this axis and on this axis. And this is that person. So basically the yellow is the
computer expert. It has a hundred times more samples, but they are all in here in this
yellow. People have way fewer samples, but they are spread more to explore the space.
And here is this person going way out off to Neptune into bad space in order to find this
cluster of solutions that is in a new valley that can now be pushed way lower. So it is that
leap of discovery that not everybody jumped over that led to this particular solution of
space.
>>: [inaudible] version of this because you're picking up an interesting person here.
>> Zoran Popovic: Yeah.
>>: Kind of interesting to see what happens…
>> Zoran Popovic: There are several of them that eventually made that leap.
>>: I mean do over 10,000 players what is the [inaudible] look like leaving and coming,
distracting the way the actual details of the protein is there, are they sticking to a certain
region or are they [inaudible]?
>> Zoran Popovic: Remember everybody has their own strategy, and that is the power.
Some of them will just be doing something here. But the key thing is, and the reason that
this is also important is it is never a really good person that is good for all puzzles. It is
the collection of people that are very good at an aggregate set of puzzles, because the
strategy is diversified over many people and they are all good at different aspects of it.
>>: [inaudible] something to do with this too, right? If it's just a few algorithms then I
would be terrified to jump into that space but the next several days maybe I will be
[inaudible] algorithm will be the [inaudible].
>> Zoran Popovic: Yeah. I think that's important. We played with that for a while; I
forget what the duration is. What we ended up doing is actually seven days is a right
time, but then if you wanted to refine it, there are actually different stages you can use
after that for the next seven days. So I will mention that a little bit later. So here are the
ten things that we have done. People did significantly better five out of ten; they did
similar on three out of ten and then worse on two out of those ten things. It turns out not
only people, but computers did really badly on these two. So it is actually irrelevant that
we did worse than them because there were basically both computers and people were
like five to ten angstroms away, which means it is nowhere even close.
>>: [inaudible] so if actual people who actually beat the computer…
>> Zoran Popovic: No, no, this is ten experiments, ten things that nobody knew the
solution for that we ran, and that is actually the report for our Nature paper. So basically
we have identified what are the modalities that people were able to do really well at, so
we basically shown that there are three or four things that computers have a really hard
time doing and that people are very good at, and that was part of that paper.
>>: So these are the ones where people were about to publish, right?
>> Zoran Popovic: Yes.
>>: So were there any cases where people actually got to the correct solution?
>> Zoran Popovic: Yes. Some of these basically you can see, these two, for example,
are basically solutions. They are just basically one angstrom away. That is considered
sort of close enough. Okay. So more exciting stuff that you haven't heard about; nobody
has really heard about, is that there is this thing called monkey AIDS virus that was
unsolved for 15 years. And we said all right let's, in fact, it was x-ray crystallography
and there is something about the way--actually they required that you look at it in many
different silhouettes and you're trying to basically induce from it. So for some particular
shapes it is impossible to figure out the actual structure. So we posed it to two people for
three weeks and three different iterations, and we had a solution which was then sent
these guys who did the x-ray crystallography, and it was actually confirmed now that that
actually is the true shape of that protein. So basically this is the first actually unknown
thing that was discovered by Foldit players and it is another Nature paper that has been
accepted. It was interesting. The fact that it was never solved for a while was not, the
dramatic thing was this monkey virus wasn't as important as some other proteins, so as a
result there were questions as to whether we should publish it or not [laughter]. Like who
cares? The whole point is that it was solved in a completely different way.
>>: So when you look at these end things that the people are doing what computers are
not doing, that has to be really excited about fixing their algorithms to better understand
how to do these things that make them longer-term things, make them less myopic things,
whatever they might be. So where has that gone as far as making better algorithms?
>> Zoran Popovic: Yeah. I am getting there. So in this case, it is interesting. There are
four people in combination that were able to find a solution, and then of course we
originally said should we put your four names in the paper, or should we have the group
that you are part of which is now maybe 30 people. And they all exclusively said the
group. At that point I just sort of realized how un-pure of a scientist that I am. Imagine
how many of you would be okay with just the paper that you write just right MSR and
not your name. Nobody is going to do that, right, because in academia it's even worse,
because it is the only thing you have; you don't have any money; all you have is just your
name on papers and that’s what everybody measures you on, but these guys are pure.
They actually care about discovering stuff. And that is what the drive is. So as a result,
they don't need their names; they are just really into the whole process. So I thought that
was really a hope, a fantastic realization for me personally.
So this is the thing that you wanted to talk about and we wanted to know exactly what is
going on. So originally I logged everything; I said I'm going to do machine learning on
the aspects of this thing and see what's going on. And then I basically started looking at
it and I'm realizing oh my God, there are so many variables that it is impossible. And in
this new world there is a better way to do it. There is a better machine learning
algorithm, machine learning engine that is not a machine; it is a person. So basically
what we did is we looked at basically people started creating strategies themselves. They
started writing their own wiki pages and stuff, and so what we created is a Foldit
cookbook. It is basically both a visual programming tool and a LuaInterface, for them to
actually start scripting different parts of what they are doing in the way that it integrates
very nicely with what they interact with.
And the question is are people able to now not only solve problems but structure these
problems in a way in which we will be able to learn what is actually going on.
>>: [inaudible] scripted, that step is going to get encoded into this cookbook?
>> Zoran Popovic: No. The thing is, yeah, one way you can think of it as here is what I
have done; turn this into a macro. There are some problems of what to leave to a person
still and what should be automatic, right? So we basically let them have a visual tool
where they can actually see what they've done and then also pick different actions from
them. So we see that we actually have creators, but we have a huge number of users as
well, and the strategies are very much spread. So here is, one interesting thing we have
done is we have let people evolve other people's macros and we actually see that
evolution and rating and everything else. So here the size of each one of those circles is
the popularity of each one of those macros, recipes. And then sort of the depth is the, you
can see that some are public, some are groups, so in this case, there is a public one that
then each group secretly has their own particular version that is played by fewer people,
because there are obviously fewer people in the group. But perhaps they think that they
have some particular advantage.
There are some other public ones, that are evolved into even more popular public ones
that are so popular that have many different offsprings but nobody actually can figure out
how to make it significantly better, or perhaps everybody thinks that their private one is
the best. And then there are some that are actually fairly deep. Some particular group
takes a particular version of a public one and then really tries to develop it for a long
period of time, and then when they feel like it is really good, they make it public again.
They want to share it again, which then of course now becomes very popular, perhaps
almost as popular as… What's that?
>>: [inaudible] that group one or?
>> Zoran Popovic: Does it offend it?
>>: Usually the group ones are not public to anybody there if they are within the…
>> Zoran Popovic: That's right. They are shared.
>>: So they are taking a derivative of the group algorithm and…
>>: It's their own, their own…
>> Zoran Popovic: Yeah. But it could be. It could be that one member of the group
decides to make it public. There are a lot of these things recently--recently in analysis we
have been doing another chat with everybody to decide because there are these
defections. People actually take ideas from one group and go to another group and so
how do we prevent that? I actually have no idea. So every time I have no idea how to do
something, I just make a big chat with all of the players, and then we try to figure
something out.
Okay. So then the question is what is the usage of this thing. So here the different
recipes and here are different people in colors and you can see that some set of recipes
are vastly more popular than others. Now that doesn't mean that they are actually solving
solutions. Remember, there is a particular stage in the algorithm where refinement is
good, where these things are particularly useful.
And then we started looking at this algorithm. It is called Blue Fuse and it is one of the
evolutions that was particularly popular. And this is the actual structure. And then we
showed it to biochemists and they were like oh my. This is exactly like this unpublished
algorithm that we are using that is performing really well, but they actually discerned that
it was created by exactly the same process. Now they didn't have the same amount of
parameters to tweak as the scientists did where they can just change anything. They just
had the building blocks that we gave them, but they basically replicated the algorithm
that as yet unpublished but known to be the best performer. So then we actually did the
measurements. So this is the state-of-the-art on thousands and thousands of proteins. So
this is actually really how it generally does this. Again it is not solving from scratch; it is
actually trying to do this go to the bottom of the local minimum kind of optimization.
That is why it is called Fast Relax.
So the state-of-the-art is like this. This is the Blue Fuse and what is interesting is Blue
Fuse in combination with the deep breathing or something, I forgot what it was called,
Deep Breath and this is interesting to know because they called it Deep Breath because
they relaxed opening up and they squeeze it in again. They have this kind of breathing
structure algorithm on it. And so that in combination with that creates this really
favorable curve that is way better than the original. And this is sort of the final tweak
sort of replica of this the scientists themselves are using that are now yet to be published.
So obviously, now we are looking at can the parameters slightly be tweaked to actually
bring this down to exactly what the scientists are doing. But the upshot of this and this
has actually been sent to proceedings of National Academy of Sciences is that people are
actually able to structure things and advance the algorithms, not just the actual process
itself. Huge amount of stuff, I am sure you can think of many different ways in which
this can be analyzed further, or perhaps even suggest that people do things more towards
synthesis perspective, or maybe even have a challenge on automated ways so people
submit their solutions.
>>: [inaudible] interesting to know whether this is all kind of domains relative to the
domain specific heuristics on search and organization or if you discover some new
principal as far as relation from the heuristics. And that would probably take computer
scientists not sort of biologists looking at [inaudible]. Whatever the [inaudible]
>> Zoran Popovic: We did all of this analysis of the algorithms themselves.
>>: In the case of the [inaudible] proteins there been close to publication that you gave
was the iteration involved [inaudible] some stuff in the experts got it, it defined the
problem and gave it…
>> Zoran Popovic: No. Game only.
>>: Game only. So there was no kind of human bias of known solutions [inaudible]
description?
>>: That was important for the study. Otherwise they would shoot us down in the
intermediate. So one thing that we realized was that creativity, Foldit solves problem
solving, but what about creativity. So what we said is what if we can figure out the
sequence that gives rise exactly to the shape that we want that attacks the HIV virus on a
particular part of the virus that has the specific shape. So now we are going into design
space, which now led to a completely new branch of the game. And this is again
injection of new ways in which you can play the game, right? This is, this keeps people
interested over a longer period of time and so here you can basically see we made a lot of
different changes so we could actually take every amino acid and then swap it for another
amino acid in sort of a very intuitive way. And in this case there is no way to tell how
good the design is, because the energy function is not great for completely out of unseen
proteins in this case.
So what we have is now we actually have three labs waiting for the experiments from the
protein design of Foldit to know which is the new set of experiments that they are
actually going to synthesize in the lab. So that to us is exciting, because that means that
they don't see another avenue for new candidates that is actually more, faster producing
than what we are doing in Foldit.
>>: [inaudible] completely specified, that you know precisely what the shape should be
or…
>> Zoran Popovic: You kind of know what the shape, let me give you, here for example.
There is a particular molecule and you want to design a protein that is going to engulf this
protein in the right way such that there is a maximum amount of energy of connections
between these proteins and the thing.
>>: [inaudible] you can send this to the protein but how do you know what's been folded
into that combination?
>> Zoran Popovic: Ah, exactly right. You need to test it in a lab. That is the whole
point. So in the lab you can actually test how many of these little proteins are engulfed in
my thing. And so the first thing we did is by fibronectin design, there was sort of the--so
we designed this core. This stuff on the inside of the protein and this is the new
definition of--what scientists do is they basically try to do simulations over this new
protein to see if it actually spreads in the way that natural proteins do, and if that is the
case, then they send it to the lab. And the first one that was sent to the lab we actually
sent the replica of that protein to Boots McGraw, this guy from Texas. I think he is a
salesman of some sort. It turns out that particular protein didn't actually fold as predicted
by the energy function. So energy function and reality were different. So then, do any of
you know what Diels-Alder Reactions are? It turns out some guy won in the 1950’s a
Nobel prize by inventing this particular reactions, and they are basically really influential
in how people design drugs, because those reactions can be used.
So in this case there is a particular structure that we are trying to design protein around it.
And recently this is the starting position and this is the solution after a while. So you can
see the proteins drastically change. These two new helices engulfing this thing way
better than the original structure has actually done. So then when biologists look at this,
they say wow, this is so radically different. There is no way this is going to fold in a right
way. And then, sort of, well, we don't have anything else to send to the lab so let's just do
it. And it turns out they sent it to the lab and it's actually a new protein that folds exactly
as it was predicted. Not only that but this particular reaction now is 100 times stronger
reaction than anything previously known.
So this just came out of the lab recently. We didn't even send this, but this is probably
our biggest sort of breakthrough here. So creativity now opens a completely new avenue
and what it basically shows you is that in terms of creativity by a chemistry PhD, does it
self select for that? Most likely not. Even if it does, is it a really good selection
mechanism for that? Definitely not. So in some sense we were able to tap into new
scientists that never ever would've touched any of this stuff. Yes?
>>: Can you explain further the noisiness because before you were saying that we have a
pretty good belief in this thing folding. [inaudible] energy proof was a good one, but here
you are saying that there is some more noise in terms of [inaudible] shape and your
energy [inaudible] tells your [inaudible] what it will fold into but there is…
>> Zoran Popovic: For example, if you look at all of the sequences of amino acids you
can see that nature uses some particular patterns. You can see it on machine learning that
it picks particular sequences so that when you create a new protein you use the patterns
that nature has already seen. That is what everybody would do. And then that's what
nature does too. It kind of uses what it has seen…
>>: [inaudible].
>> Zoran Popovic: Yes. The people actually stretch this little bit more, because they
don't actually know the things that nature has seen. So they start doing crazy other stuff
which now is outside of some of the realm of what people have actually--you know, a
PhD student wanting to get their PhD in six years is going to try some stuff that has a
high likelihood of success. These guys don't care about that. They care about making the
score and the function be the best possible and as a result they came up with the…
>>: [inaudible] imagine sending the recreational drug community to design a new line of
drugs just after oh, this is a really cool one.
[laughter].
>>: Let's try it out.
>> Zoran Popovic: What is interesting now is that everybody is asking us who owns
these new things. It is like a new headache for me to deal with. I always say, no, this is
all going to be open to everyone. This is going to be creating generic drugs if we ever get
to it, right? But legally when I start to structure this thing, it's like, oh my God. It's like
impossible.
>>: [inaudible] UW in-your-face with this one.
>> Zoran Popovic: That's right. So UW really owns it, but at the same time we had this
competition that MedImmune promoted, $10,000 for the best protein, only schools can
participate. And there was a huge mutiny, you know, what is this pharma doing taking
over Foldit and I was like oh, I should have never done this. This is total disaster. But it
actually turned out to be a good experiment. So all of these labs were competing between
each other, they got the $10,000 award and then I released the same routine to the general
public and in three days they did better than all of these labs collectively. So for them
that was kind of an award itself. These guys got the money, but we still can do this way
better than them.
I am really running out of time so I will tell you some things that, some of the most
important things that I wanted to talk about and to talk to you about some subsequent
things we are doing especially in the way that Microsoft could potentially be involved.
This is just showing you that we actually have a Kinect version of Foldit now that they
are going to releasing soon. So the idea is what everybody wants to do is grab these
proteins with their hands and start mushing them. So the question is can we actually do
this with a Kinect. So this is kind of the early--now we can have multiple people
grabbing different parts. They don't necessarily need to be in the same space. Perhaps
different Kinects in different parts of the world can basically interact together in such a
way that you have, you know, what can you do with proteins when you have hundreds of
hands on the same protein working at the same time? Maybe nothing. Maybe just a
whole bunch of fights. But we will know the answer to that soon.
It has now actually been used in actually two biophysics, biochemistry books are actually
using Foldit as a reference for subset of things. Where doing drug design game, we are
trying to find novel biofuels and something that I am really excited about, we are doing a
nanotechnology game. This is about creating machines from molecules. And you can
actually see it here. If anybody wants to play it this is a very, very early prototype, but it
is basically how to use synthetic biology with DNA pieces. So how do you create a
walker? In this case it is a substance that has a particular track and it is actually moving
these things in a particular way to walk to a particular thing and clean it out or carry it to
some new place. So you can build these little machines all out of these interactive
mechanisms and there are maybe 50 people in the world that know how to do this now. I
can guarantee you that I can increase this by two orders of magnitude when I release this
game. And so it is almost even a better, this now doesn't even need a full blown
application. It can be a web interface itself. So I am really excited about us releasing this
particular game soon.
The first thing that you should obviously know is that this encompasses education at
large. And this is sort of the crux of my work now is how we can actually solve
educational problems by using this same thing. So I looked at sort of what are the big
questions that everybody is stuck on and so algebra is a big stumbling block and
everybody believes that, more than 80% of kids leaving elementary school do not
understand fractions in a conceptual way. And people have done studies that said if you
don't understand fractions you cannot do algebra. You can't even infer fractions from
algebra. You just have to go back and actually really understand fractions and how they
work. And that is why even in community college there are remedial classes in basic
fractions these days. So they national report came out and said okay, I am going to try to
do fractions because everybody will listen to me if I can actually knock this thing out.
Because of course education is a big Titanic, you know that just even if it sees the
icebergs it cannot turn. It just keeps going towards it.
But one really sobering thing that I realized really early on is that we cannot make an
effective fraction game, because we don't know the optimal pathways to actually lead
people towards understanding, and we don't know how students’ specific adaptations of
those things should actually lead every single kid out there towards understanding. So I
said okay, my idea for games for STEM learning just goes to the dustbin. But, what I can
do instead is actually for the first time turn the education into a data-driven science.
Because I can collect data that no other thing could possibly collect, not just because I
can do large AB testing, but I can do fine-grained nuance adaptive testing of actually
understanding every brief thing about a wide variety of kids. So this is actually my main
goal these days is to actually figure out the platform where I can do both understand what
are the spaces of confusions, and figure out how to lead kids out of each one of those
individual confusions towards the light bulbs that will give them conceptual
understanding.
Okay. So that is my grand challenge. My collaborators are basically people that sort of
literally wrote the book on how people learn. And I have experts both for early math and
sort of learning the sciences themselves. I have a number of people who are actually
interested in helping me--it is amazing how many people in the game industry don't really
like working on shooters and stuff. They really are looking for an alternate way to use
games to help. So I have been using this a lot. I don't have to tell you much about this.
So just like Amazon refines the pages to be ideal for everyone. I know you do for ads on
Bing and everywhere else, so this is the kind of thing that we are doing for education.
Imagine this is basically a path structure through different concepts that is actually
embedded in the game.
My question is, is this the best path in general or how can it be modified for every kid.
So basically what we do is we take different variations of this, and we can change
continuous parameters or we can do discrete changes like this. Maybe we will introduce
a new concept and then test this on hundreds, thousands, tens of thousands of students.
And then basically based on a particular metric of how well they did, maybe we can say
okay, this particular thing seems to be more conducive to actually leading people towards
realization that all of our other methods. So what is exciting is that we can actually do
this on a week or two week basis, if we have a huge number of kids playing. Which
means that we can actually, what takes, you know, a standard Department of Education
grant four years to do sort of a standard test to test only two variables perhaps, we can do
in a month. That can change the landscape of what can actually be known in this entire
space. And that is actually what we are doing.
So of course we continue this in sort of this evolutionary algorithm where we actually
keep generating new sets of things. So how many people do we have? We are actually
working with virtual academies of K-12. They have about 500,000 students in the virtual
academies, but it is not enough. I am now working on a half a million students for the
next two years. And I have had huge numbers of people--I am now actually working
with the states who are actually willing to experiment including consortiums for
assessments that Obama has funded and so the idea is I am looking for all, in fact, I am
going to France to talk to the French education administrators next month and Singapore
who was always interested in innovating education is also interested in doing these trials.
So obviously I need to prime this engine in order to have, because as you can imagine
there are so many possible ways because we are trying to infer both the model of learning
and the specifics of a particular domain and then maybe the model of incentive structures
that varies for different people about how they can learn.
And the idea is to basically have every kid anywhere in the world have a mobile device
where they can have state-of-the-art education. So what are we doing? So we are now
building a huge number of visual data mining tools, so we are logging every single
hesitation of a mouse whatever kid with the idea that actually inferring exactly what is
happening, and we will actually have social games and the social data analysis. And my
question is what can we do all of this data. Here the question is that, you know, every
learning scientist I talk to is drooling. I mean this is something that they spend their
entire livelihood answering, and I am telling them you can now do this in several months.
So everybody is easily convinced that this is the right thing to do. So what is the partial
ordering of concepts that must follow one another? So people define these concepts
common core concepts in math, you can see them right. But there is no ordering. There
are just concepts. How do you know that one thing should proceed the other and how
does that vary for every kid?
And the other thing is obviously, what is the best thing to do at a specific point of
confusion for this kid. So let me give you a little bit of--and then of course there is the
incentive structure behind it to. So here is one thing that we've done. It is just the
beginning but it turns out to be a very powerful visual data analysis tool. It's kind of for
those of you who know what heat maps are; it is a heat map for an arbitrary game that
does not actually have an environment that it is embedded on. So imagine you have a
start on your puzzle and your goal is here on the other side. And imagine, and your state
space is perhaps infinite or maybe countable but still enormous. But you still want to
embed it into a two-dimensional plane in such a way that it can reason about what is
actually going on to people's pathways to sort of solving that specific level.
So the idea is that if somebody gets the level, they have a path that goes directly from
start to the goal and then you put the states or at least features of states where everybody
is everywhere. So the idea is if somebody goes here, it is a confusion according to that
metric, because it is not as close to the goal as some of these pathway stores. So
basically, it is a nonlinear projection with one dimension usurped by the closeness of the
goal, if you will. So basically we have stretched the null linear projection in such a way
that we can actually look at it in this way.
So here's the real example in one of the puzzles. The size of the circles of course is
where how many kids are actually in the particular space. Blue arrows are those kids that
actually solve the levels; red arrows are those that haven't. And here are some exciting
things that come out of this. These guys are experimenters. These guys are trying many
different things to learn sort of the mechanics of how things are working before they
actually figure how things are working and going on. There are others that just really like
to sit, think about it first and then go directly to solution. So you can actually see
different styles of what they are actually doing on this particular projection. Another
thing you can see is that you can see that this particular point is very close to the goal. It
is closer to the goal, but everybody that got to it didn't solve. So that indicates that there
is either something particularly confusing about the way that we designed the level, or
there is a cognitive block that everybody that gets here is no longer able to overcome.
So then what we are able to do is click on this thing and we actually see the state and how
people got to that state. So there is a way in which you can sort of visually explore the
map of this level no matter what kind of game it is. That is the exciting part of this. So
one thing that I am hoping I could talk you about through the second part of this day is
this thing that we are trying to do; we are trying to map the space of all possible
confusions. And that involves two things. Determine the space for every domain by
exploration, and then determine the optimal pathways once you have that exploration
well defined. So think of it as sort of creating all possible clusters of standard ways in
which kids are doing things, and then once you've identified a kid to be in some cluster,
figure out, again through exploration, what is the shortest path I can lead that kid towards
understanding?
Again by multi variant randomized stuff. So we are now actually trying a number of
different--it turns out POMDPs are actually well-suited for these things because they can
and incorporate history which is fundamental for learning, more than sort of the standard
MDP methods can do. But this is the sort of fundamental thing, the standard ways in
which the machine learning is doing things is not, obviously there is always some way in
which history is folded into the current state to make it sort of MacArthian et cetera, but
there are still really unique ways that people actually acquire knowledge that--imagine
that you have a concept. People can understand that concept, but can they use that
concept together with three other concepts to actually solve the problem. It is a different
level of understanding than just verifiably knowing that concept. So how do you
represent different ways in which you can use your information of what you understand
towards solving problems needs to be embedded in this representation. And that is the
challenge that I issue to you, and if you are willing to actually look into solving these
problems, would you definitely let me know?
We also now, and this is something that we know a lot more about, because we have
done it in a number of different instances, but we are trying to use universal
reinforcement training to actually map the intention of people through their play such that
we can sort of know what they are trying to do and as a result sort of custom tailor the
subsequent levels based on what is happening there. Again it is somewhat predetermined
on actually understanding the space first, and that to me, I think at this point, I thought
that this was going to be the hardest thing, now I think it is going to be this. And I guess
it makes sense. In most machine learning things, right, the algorithm is almost never
interesting; it is the actual representation underneath it and the clever crafting of the
structure of it that always is the path to solution.
So I now actually, more and more my students are actually going directly into space, and
we already have actually more data than we know what to deal with. So some of the
games that we have developed have already won a--last year we won a grand prize in the
Disney learning challenge for Refraction. You can go to, you can actually search for the
Refraction game and you'll find it. More recently we won on Congregate. It was one of
the top 20 games on Congregate Mobile so basically we reported on a mobile platform
and it was one of the top 20 games. Remember this is one of the top 20 games of all
games. This is the only educational intentional game, but it is actually on par with the
best games out there. So that is really good. And another thing I want to show you is
these are the common core concepts. What we are building is a teacher portal. So
basically the way in which, it is a way to interpret the machine learning structure of
where people are on the way in which parents and teachers can understand. And you can
sort of imagine how this would be enormously important. The reinforcement learning
and what it wants to do in terms of which concepts it wants to push over which other ones
is here. So this is, we will be pushing more of these concepts than these in the next set of
levels that the people will play.
Here is another exciting thing. People can actually, teachers can now say, I am actually
going to teach simple equivalent fractions next week in my class. I am going to take this
slider I am going to move it up here. And I am going to lower the other one. All of a
sudden my kids as they play at home, they are actually going to be exposed more towards
these things that are actually just about what I am going to teach in class. So there is sort
of a brand-new way of doing this. This is another way; this is a more detailed thing that
comes out of this analysis. So we can actually tell you exactly what kind of mathematical
expressions that you understand and what not. This is actually a very huge list. We have
way more details than any teacher could potentially infer from all of the things that are
actually happening. And that is the thing that we are building as well.
Last thing that I want to tell you about is transferred to the real world. So the big
problem is that even if you play a game, you don't actually know whether they have
actually learned enough to transfer to the real task. So how can that be done? So we are
actually creating parameter trials levels. So everything that we create can actually be
automatically generated in many different ways. So here's an example. So this is a, the
same game but now I can enter several equations. So this can be done by teacher or a
parent or whoever. And I can actually generate the level that makes sure that the kids
need to actually compute that thing in order to complete the level. I can actually add
multiple equations, or I can independently change orthogonal things like spatial
difficulty. So now I have a more complex level that still embeds the mathematical
concepts but now has a completely different structure behind it.
And I think now I am going to deliberately make it a little more complicated by adding
another concept and I am going to have kind of a complex level being generated. This is
all in real time, so it takes a little time. Turns out this is an MPR to ensure that I actually
have to, that I have to go through that equation specifically, but you can see that the
layout and where the obstacles are and what the lasers are and what the source of these
nations are all changes in order to ensure that particular understanding. So that’s exciting
in itself, but it is actually more interestingly for transfer. So what we are doing is we are
actually trying to take multiple, I don't know where my textbook went, but basically take
a math textbook and map every problem into four different games, five different games.
Why is that important? Because if somebody can do something really well in this game,
the question or transfer is can you solve exactly the same problem in a completely
different domain setting? And if the answer is yes, then I feel more confident that you
have a high level construct build that you really understand this stuff, concept, right?
And if the answer is yes, for four different games now I can almost give you, send you
back to really boring terrible test and I can have a very high confidence that you will be
able to do this.
So games here are actually able to test this transfer in a very interesting way and that in
many ways is the biggest weakness of games. There are actually studies that if you--you
all know Tetris right? So if you play Tetris for a while your relational invariance
cognitive ability increases. If you play it for very long period of time there is zero effect.
And the reason is that you are way better, but you no longer use your cognitive tasks to
sort of rotate things and figure out where it will fit. You need to see the shape and you
map it to your fingers; you know exactly what you should be doing. So the question is
how can we design the games that don't go there, that actually make you--so you can
imagine you can make the Tetris such that shapes change all the time. Of course, if you
do that people would like to pull their hair out and just leave the game. So how do you
actually constantly change the challenges in such a way that you always make and
develop cognitive understanding, without overfitting in a standard learning sense.
And so I mentioned preparation for future learning, but here is the platform that I am now
trying to build. I realize I cannot build every single game myself, but there is a way in
which we can do this together. So basically researchers and teachers and maybe
MacArthur foundation will say I want everybody to work on this kind of domain, maybe
global warming. And then you have sort of scientists pose problems et cetera. The
public says no, this is a stupid idea. Don't do it. Or basically, or experts say there is a
better way to do this which then sort of leads to a competition, a $10,000 competition for
all designers to come up with an exciting concept that actually works solving the problem
at that particular angle, or maybe tackles proportional reasoning or biology, which then
students yeah, this is actually fun, or say no, you guys have no clue. Nobody is going to
play this. So basically that decides what is the design that wins which then goes on to a
version on the Top Quarter or something similar. How many of you know what Top
Quarter is? Okay, a few of you. Where you can actually develop these kinds of games
with the API that we are able to provide, and this is what we are working on.
Is the game effective for learning? This is the kind of thing, of course, every company
would probably swipe immediately, because it is much easier to make a game, sell it
without necessarily guarantee that it is actually truly effective. So this is what is
important to me. And this is why I would actually like to create this stuff as a ground
truth of whether it really works is there. Then of course we sort of are providing this way
in which we can generalize, personalize and set the understanding model so they can
apply to any particular domain, and this automated adaptation, imagine scientists say we
don't know of these particular variations, which one is better. Or here is the range for
these variables, but I don't know what is the best value for that range. Turns out you can
basically turn that to an automatic exploration algorithm then that actually automatically
creates varieties and automatically stops generating them once the answer is statistically
significant, and switch to the next set of hypotheses that need to be explored. This could
be automated itself and can actually lead to automatic refinement of the games
themselves towards greater game effectiveness.
Yeah. So that is all I have to say. So basically this loop and the social structure is
something that I am really excited about building, because I see it as the only way to go
from just me tackling fractions of proportional reasoning which now I have funding from
many different sources to do, to really say okay, how can I cover the entire common core,
not just for math but for science. And how can I know that everything together works in
synchrony rather than just a bunch of islands that kind of show individual effectiveness
but there is no rhyme or reason of how they connect together. So that is all I have to say.
>> Eric Horvitz: Thanks very much.
[applause].