>> John Nordlinger: Good morning. Thank you very much for coming to this talk. I'm
John Nordlinger in Microsoft Research, External Research. And I'm very pleased to
announce -- well, to introduce our speakers from Manhattan -- well, Manhattan and
Brooklyn. And speaking of Brooklyn, I'll first introduce Katherine Isbister who comes
from Brooklyn Polytech, now known as NYU Poly. She's a social scientist with a focus
on HGI and of course has collaborated with us on a games usability book. Katherine,
could you raise your hand?
Also presenting will be Jan Plass, a cognitive psychologist who's really making sure that
the games that are produced are both fun and educational. Jan, could you raise your
hand? He's one of the co-PIs at the Games for Learning Institute.
We have no shortage of psychologists. We also have Bruce Homer. Bruce Homer comes
from the [inaudible] center at CUNY. Is that how you say that?
>> Bruce Homer: Graduate center.
>> John Nordlinger: Graduate center at CUNY. Another cognitive psychologist and
expert on how games teach and educational technologies. Is that a fair description? This
is really stretching my abilities to introduce even these people. But it's a great team.
And of course leading this effort would be Ken Perlin from NYU. Ken Perlin is a
well-known SIGGRAPH personality. He's a well-known computer scientist. He's
participated in lots of different projects of Microsoft. He's friends to many of the people
in this room. And he's recently married. He recently got a SIGGRAPH award, he
recently became a recipient of this effort. So Ken's had a great 2008. And let's welcome
Ken Perlin and the other team from the Games For Learning Institute.
Thank you.
[applause]
>> Ken Perlin: I just saw my whole life flash before my eyes here. Whoa.
Okay. So we competed for this grant and then -- oh, are we supposed to -- nothing here
that says -- this is hidden. This is plugged in. I don't know. Do we have an AV person?
>> John Nordlinger: Oh, yeah, we will in a second.
>> Ken Perlin: Okay.
>>: There it is.
>> Ken Perlin: Okay, good. All right. We competed for this. And what I'm going to do
is give you an overview of the proposal that we gave to Microsoft that resulted in them
saying sure, let's give you $1 1/2 million over three years. Then what Jan is going to do
is start filling in what we've been doing to really put some meat on it since we started,
and then Katherine is going to show some of the interviews that we've done with experts
around the field to really -- so they'll tell us exactly what we should be doing.
So first I'll give you the overview. So basically kids want to learn. Kids love learning, as
you know. Everybody here has either been a kid or has kids or both. And you know that
like just kids want to just dive in and learn about new stuff. And games actually are fun
partly because you're sort of tickling your brain and learning more stuff.
How do we tap into this? There's a general understanding that games can help with the
educational process, but there needs to be some solid science to knowing what works and
wasn't doesn't work.
So starting October 7th when Craig Mundie came to NYU -- yeah, Craig Mundie came to
NYU -- it was really cool; I got to shake hands with him and everything -- and sort of
officially launched this thing. The funding is distributed between Microsoft and our own
participating institutions. There are seven participating institutions in the New York area.
And so the total budget is about a million dollars a year, and then we're hoping to
continually raise other funding to continue it for at least ten years.
So there are 14 of us and multiply that by roughly two and you get the number of grad
students, Ph.D. students, almost all the money is going to Ph.D. students because we
think that's the right way do this.
There are basically two halves to what we're doing: we make stuff and then we try to
understand where the learning is. So I'm basically on the left half of that side with a team
of faculty and their students saying let's start creating some games that are scientific
instruments for asking these questions, and I'll show you what the questions are in a bit,
and Jan and his team of psychologists and assessment people and education people are
trying to extract from it, what do we learn. So they're really doing the science of it.
And there are a number of really high-quality faculty involved with this. We're very -there's a lot of participation across culture, across discipline, across gender, across a lot of
lines.
So one thing I say up front here, that I think that the folks who arranged for this grant
really I understand is that it's almost impossible to get a National Science Foundation
grant for a highly interdisciplinary effort in a new area because there is no effective peer
review that's going to actually make it through the various spikes that are awaiting for
you if you try to do something really different.
So Microsoft understood this and they said, hey, something really different, need
psychologists, needs game experts, needs computer scientists, needs the whole
assortment of things. Why don't we kick-start this and after three years if you've got
scientific results, then you have a corpus of journal articles that you can go with and start
getting national funding for. So this hopefully will kick-start the field if we do it right.
So there are a bunch of faculty that I'm working with who are making things and a bunch
of faculty that Jan is working with, several of them are here today who are building the
assessment structure around that.
We have a really great scientific advisory board, that's just the beginning of a scientific
advisory board. Jaron Lanier -- most of you kind of know most of these people. Jaron
Lanier, who's the virtual reality inventor, and Mitch Resnick at MIT, Will Wright. Thee
youngest member of our advisory board is Cailin [phonetic] Doyle. She's 14 and she's
the only person who actually knows what is going on in the world of 14-year-olds among
all of us, and she's been absolutely indispensable.
You know, we said we're going to go -- for example, we said we were going to go into 19
different middle schools and we were going to say we're the Games for Learning
Institute. And she said, Couldn't you go into seven of them and just say you're the Games
for Fun Institute and see if maybe it works better? So she's got this like -- she's sort of
like keeping us honest. She's also a game designer herself, which is very -- which is very
good.
And then we're going to keep adding advisors as we need them.
But why use games for learning? It's a fair question. There are a number of people who
say game -- you know, games, they say, Isn't that that thing that teaches kids violence and
all this stuff? And there's a lot of knee-jerk reaction about games, so you need to at least
ask the question and be able to discuss it.
Well, clearly, they're fun and motivating. Everybody plays games. There are two major
kinds of activities that seem to be wired into our brains biologically: playing games and
telling stories. So it's sort of one-half of all the things we do for fun is these interactive,
kind of competitive bonding experiences.
If you compare a game with a textbook, a game actually -- because it's got this kind of
rule-based computation behind it, it can actually look back at what someone's doing and
start tailoring the experience. It's really hard to do that with paper. But it's actually quite
straightforward if you understand how to approach it, just start saying, oh, this is a social
learner, this is a slow learner, this might be a dyslexic learner, this might be someone
who's borderline ADHD, and so we start switching our learning strategy. You can start
addressing these questions of individualized instruction.
And this is maybe the most important thing. When people learn math, most people -- not
the people in this room, but when most people in this society learn math, you know, they
get to eighth grade or they get to ninth grade and they get to algebra or geometry and
someone says you have to like do these equation things and you get it wrong and they
said guess you don't understand math. You know, and then you get this idea -- oh, I
guess I can't do math, I got it wrong.
So some disciplines like creative writing, perhaps history, have very, very graceful failure
modes. It's like, oh, you wrote an essay, that's kind of cool, you got some things wrong,
but interesting this thing you wrote about "my pet goat." It's really cool. It's like there's
something going on there. But that's not the way math and science tend to be taught.
They tend to be very brittle.
However, games by definition have very graceful failure modes. Oh, you know the
aliens killed me, the aliens killed me, the aliens killed me, oh, I killed the aliens. And so
you just keep working a game. That's where the fun comes from, until you find your way
through the level.
And this is kind of an ideal paradigm for learning. It's, okay, to fail. It's, okay, to break it
and then fix it again. And so we think that some of this mind-set might actually bring in
some of these disciplines -- science, technology, engineering and math disciplines -- that
the NSF worries that our kids aren't learning in a way that's much more graceful for the
kids so it doesn't become this center of fear for our nation.
And so in some sense because of this ability to gracefully approach these things through
games, it may be the best way to learn. We may find that using these interactive, fun,
goal-oriented experiences, whether they're social or individual or networked, may just be
the ideal learning mode most of the time. But we're going to find out.
There are a number of questions that need to be asked. And the basic flow of how we're
approaching it is we have to understand what are the factors that go into games that may
or not -- may or may not make them effective games for learning, how to successfully
integrate that, how these factors interact, whether it matters, whether your -- and what
sense it matters, whether -- are you in a classroom, are you in an after-school club, is the
kid playing it at home, are they playing with their friends, and how do you integrate this
into grown-ups being part of the project.
None of us are thinking that games replace teachers. Teachers, an adult who is really
interested in your learning is very, very important for kids. We can't solve the problem
of, gee, you know, there isn't enough support for teachers in this country, but we can
make the jobs of those teachers easier and we can provide a support structure for them
and we can help the teacher identify where there are kids who need more help and a way
to do that without the teacher getting overwhelmed because they have to do all the
assessment themselves.
So on the left side of this chart is where we start, and the right side of the chart is where
we end. There are different things that might go into a game design. Now, Jan is going
to go over this is much more detail, so I'm just going to kind of give you the overview
here, and then he's going to explain the real science of learning behind this.
A game has factors about how we think, how we feel, how we interact with others, and
they intersect in different ways. And there are many, many, many kinds of learning. But
as we start trying to ask scientific questions about them, we pick a number of axes
through this very high-dimensional space and say these are axes that it may be interesting
for us to start doing testing along so we can get a handle on what knowledge do you have,
what understanding you have of how to think, on how to reason and how to make
something for yourself, you know, understanding of your own learning process, the
metacognitive knowledge, and how well do you feel about it. I mean, there are a number
of studies that have shown conclusively by now that kids who think they learn better do
learn better.
So a lot of the difference between socioeconomic levels of this country have like rich
kids learn better than poor kids. Rich kids think they're smarter than poor kids because
they're told that all the time, and sure enough, when you're told you're smart you end up
being more confident and you end up learning better. And we have to figure out how to
not make that into a vicious cycle.
But how do you get from, okay, here are the things that can go into game design to the
things that can measurably repeatedly help improve learning. So there's an entire engine
here which is mostly designed by Jan, which is while he'll speak to it, where we go
through an iterative process of designing these games and [inaudible] them into a
situation where there are learners and game players, we put the right tools into the games
to be able to figure out what's going on, and then we do a feedback structure to be able to
iterate so we really eventually understand these design factors for these learners in these
situations and these subjects, and these game genres end up feeding into this kind of
learning.
The flow of how we approach this as an institute is we start out by asking people who
know better than us. There are people who've been doing this for years and years and
years. They all have different philosophies, they disagree wildly with each other, they
have like strong debates about it. But we're going and interviewing all of them and
listening to what they have to say. And Katherine's going to speak to some of that.
And then once we sort of get up to speed on what everyone else has been saying, this is
the right way to do it, do you listen to Henry Jenkins or Jin Ji [phonetic] or, you know,
whoever, some of the other usual suspects, then you start watching kids actually playing
games. And we don't just mean games for learning, we mean watch them play games
they like, watch them play popular games that are successful. Before you can deal with
the learnology, you have to kind of have to deal with the funology. If it's not fun, it's not
interesting because they're not going to play it anyway. So we need to figure out what is
engagement before we get too far into this.
And so we're going to review what people have done, we're going to look at kids playing
games, we're going to try to extract some effective features of these games. We're going
to start building game prototypes. And a lot of the core of our building is going to game
prototypes that are amenable to being instrumented so that we can start asking clear
questions as people play these games of what's going on here, how distractible are they
when they play these games, how long do they play it, did they sneak off and try to play
it when no ones around? That's a good thing. Can we make it addictive. You're not
supposed to use the word addictive, but, you know, that's really what we're going for
here. And then try to extract some design principles that we'll say, oh, we -- other people
who are going to do this later will be able to do this effectively.
Now, this is a diagram that I made because I originally had this incredibly boring
diagram, and then Cailin, the 14-year-old I was talking about, the first time she saw me
give this talk and she said, you know, you've got this stupid, boring, abstract diagram and
no one's going to pay any attention to it, so you have to come up with something
interesting and real and personalized. And so I said, okay, great. Like I went to
Microsoft Paint and just sort of worked on it for a while and I said, How's this? Almost.
You know, and so you sort of -- I went around a bit, and so now this is -- this is now the
Games for Learning Institute's official diagram. I'm not sure if Jan approves of it
completely, of like the ideas that came from both of us that we have been discussing back
and forth about not just a game for learning, but any game, this is how we're thinking and
the components that go into the process of game play, and then you can extract from that
this is how you'd use a game for learning.
So, first of all, everything starts with what's going on in the player's head. You can't
directly look inside the player's head. You can only sort of infer for what the player is
doing, when they're doing it, how they're doing it, et cetera, and so the core interaction
that goes on in any game is between the player's understanding and the rules for play, you
know, as Katie Salen calls it, the rules for play, the game mechanic, are they pushing
pieces, are they shooting zombies, are they flipping little blocks, what are they doing?
Whatever they do, you have something that the player is allowed to do each move, and
then what they choose to do gives you insight into their understanding.
Now, before I go on, I want to make sure that you understand; a player's understanding
isn't just "I know the rules." For example, you could know the rules, to take a simple
example, of Tic-Tac-Toe and you still might never win at Tic-Tac-Toe. You'll still put
that X or that O in the middle of the side and you'll lose every time. It's not in the rules
that you're understanding of how this game play works happens. That's not what we're
interested in. We're interested in what happens when you start building learning models
about how to effectively work within the rules that shows an understanding of something
else, the model underlying the reason for the rules.
And all around this, of course, there's the rocket ship, the aesthetics, the design, the
graphics, the sound, the music. Everything that kinds of gives you a sense of an aesthetic
experience, and all of these are things just that are there, but then there are things that
create motion in the experience, that change the nature of the experience itself.
One of them is there's some story: I'm going to get all the real estate on this board or I'm
going to get -- I need to get three in a row or I need to capture the enemy king or I need to
kill all the zombies or I need to become the Level 50 Mage who rules this universe.
Some sort of -- you know, and as I do it, I'm going along, we're a guild or a tribe and
we're going off and like killing all the virtual orks [phonetic]. There's some sort of story.
And that has an interesting property about it because it tends to change the nature of the
game mechanic over time. And it obviously also changes what's going on in the player's
head.
So that's half of what drives the story forward. I call those -- the other half of what drives
the story forward is the reward structure, you know, the carrot at the end of the stick. I'm
getting points, I'm getting a ranking system, I'm the top player in my school and
everybody -- I'm really popular because I got 50,000 points and the next kid got 40,000
points.
And, now, I call these extrinsic rewards because you're basically paying people. This is
the money. When you do a job, there are various reasons you do a job, and one of the
reasons is the money. And I've heard that, for example, apparently people I know at
Microsoft for a long time from the early days, after they vested, people started calling
them volunteers because they're not doing it for the money anymore. So there are all
these phrases like post-economic. It was really cool.
But then there's the other kind of reward. But they didn't quit. They kept doing it. And
the reason is because they're intrinsic awards. And when you play a game, ultimately if
there's only extrinsic rewards, if it's only about points, you get bored really quickly.
You're really playing the game to master something and to get really good at I can flip the
things in Tetris really well or I can -- you know, I'm getting better and better at chess.
And the proof of this is that if you're in a room full of people who totally suck at chess
and you beat them every time, you're not going to have a good experience, even though
you have a 100-0 win/loss record, you're having a bad time because you're bored out of
your skull.
So it's the intrinsic rewards that ultimately are where the real payoff is. It's really why we
play the game. The other stuff is actually just part of the window dressing. And notice
that the arrow here goes in a different direction. The reason the arrow here goes in -there's a little beanie and propellor on top of this player's head. And the reason for this is
as you gain skill, if the game does not change in order to reflect the fact that you have a
greater skill level, then you get bored and you stop playing.
Certain games like chess or Monopoly actually have this sort of levelling up built right
into the course of a single play. And of course most modern computer games have this
notion of levels. When you first start playing Mario, Tomb Raider or anything and you
play the first level, it's like, oh, it's kind of cool, I know how to shoot, I know how to flip,
I know how to like grab the little cherries, but once you're up to level 5 or 6 you can't go
back to level 1, because there's nothing to learn, there are no intrinsic rewards anymore.
So games are designed to be progressive. Games are learning experiences. So one could
actually come up with a definition of a game for learning as in a narrow sense all games
are games for learning, but we say that it's a game for learning in the special case where
the intrinsic rewards, the levelling up, the changing of the model in the player's head to
be a more sophisticated model is one that's transferable.
So, for example, if I'm playing some kind of online community game and I'm becoming
the best necromancer in the world or I'm learning all about some sort of make-believe
physics, it's not necessarily a game for learning, I'm a great alchemist now, but then you
still come out of the game and alchemy doesn't work.
But if you happen to have a structure like that and some of the things that you're learning
actually are usable outside of the game, then that's a game for learning. And a lot of
people try to design games for learning and do a really bad job at it because they're not
looking at systematic like what really works and what doesn't. And our job is to put a
solid structure underneath that.
Now, I just -- I want to get some principles out of the way here. This -- the reason this
looked like a vehicle is that it actually is a vehicle. The whole point of the learning is that
the player and the game and the whole thing are all traveling. They're traveling through
actually a multidimensional space. I'm only showing you two of them on this slide. And
what Mihaly Csikszentmihalyi in his 1990 book calls the theory of flow is also what
other people have called behaviorist learning, operant conditioning. Different schools
have different ways of talking about it.
But the basic concept is if you're -- if you're getting it right most of the time but not all
the time, so you're just kind of challenged just enough, you're in this sort of state that
people find extremely pleasurable. I'm doing the right level of crossword puzzle, I'm
playing tennis with someone who's just a little bit better than I am, I'm learning, I'm
having a really great time.
And this is where good game design is separated from bad game design and good game
designers are actually very, very good at doing this and their skills are sophisticated and
nonobvious. If you rise too quickly and the person has no idea what to do next because
you've just lifted out of what their -- the model in their head was capable of, they're just
frustrated. You know, you immediately jump from the Monday The New York Times
crossword puzzle to the Saturday The New York Times crossword puzzle, that person may
never want to do a crossword puzzle again. And if you go forward too quickly without
levelling up and they're not learning anything, they just get bored and they wander away.
In fact, this -- you want to kind of like stay away from the too hard, you want to stay
away from the too easy, and you want to kind of get to the just right, but the just right is
actually more sophisticated than I'm showing here because those of you who do game -how many people who do game design, game designers? Those of you who do game
design know that there's a much more sophisticated thing going on here.
You don't just give them a continual experience that is the same all the time; you change
the valence of it. You know, you give them reflective, calm moments and then you give
them exciting moments. You do the same thing that you do with good storytelling. You
build them up, you drop them down, you build them up again.
And so in fact there is a third dimension here which is the different kinds of terrain that
you're climbing. You're climbing on a fairly complex path where the internal state of the
player is being engaged differently at all times. So it's important to note that this is not a
simple problem, but it's an approachable problem.
And our goal is to try to figure out what are some of the rules of keeping you on this
maximal terrain, maximum engagement, maximum learning.
One of the things that's been really useful to me as a teacher in the last several years as at
NYU in our computer science department, we've been teaching a graduate course in
computer game design is the ability to jump into Microsoft XNA. This actually isn't a
commercial plug, it's just we actually find it's really good to have a game design
environment that -- do you remember in 1993 Jurassic Park, Ariana Richards sits in front
of the computer and she says, I know this, this is UNIX? Remember, right? And then
she never starred in anything again in the rest of her career. But it was like -- you know,
it was sort of like, great, yes, yes, geeks are cool.
And that moment really spoke to me and it's exactly what happens when I show them
XNA. Because, you know, it looks like an IDE. They, oh, this is software development
design, I know where to put my libraries, I know to write my assets, et cetera, et cetera.
So for teaching computer scientists to be involved, it's really convenient to work with
this. And one of the things that we're hoping to do in this problem is to get those
computer science students pulled into this.
Another nice thing we think about XNA, we've been told about XNA, is that we're going
to be able to transfer it to little handheld devices of the future.
And the reason that this is particularly important to our hearts and minds is that a lot of
the kids we'd like to reach are not going to be able to afford full-fledged computers, and
we want to be able to reach kids in the poor neighborhoods, kids in third-world countries
who will have some sort of handheld device, and it's going to cost much, much less than a
general purpose computer. But we still want to be able to apply the results of these to
those people. And so it seems as though X -- is that true, is XNA going to be able to
work in those environments?
>>: It's strategic for the company to get XNA. We're working on lots more devices
[inaudible].
>> Ken Perlin: Yeah, I'd like to point out, as it's been told to me, that it's strategic for the
company to get XNA working in lots more environments. All right. Okay. Good, I said
that. Thank you. Thank you for the words, yes.
And we're adding to it. We're adding a bunch of procedural models to it so that you
could in-game create all kinds of procedurally defined objects and behaviors, and we're
putting that back into the XNA community.
So the general flow of what we're doing is we've got these first three years which we
divide into four phases, so each one is roughly nine months. And then we've got all the
years after that where we get to build on this initial result. So this first phase that we're in
now is the wide-ranging exploration, the asking experts that Katherine will talk to, the
having kids play lots of different kinds of games. Then we're going to narrow it down
based on what we learn with that.
We're going to start building our own little mini games, we're going to instrument them in
ways -- I think Jan is going to speak in more detail to how we're going to instrument
those games. They're going -- they're not just games, they're also scientific instruments
with both in-game software to monitor what people are doing and then post-game
software. Think of it is as they're playing a game, I'm sending data behind the scenes off
to some server, and then we write analysis software to start asking questions later. And if
we do it right, we can even be flexible later in the kinds of questions we ask.
And then of course the last fourth of this three years is where we're going to do all of the
really, really serious -- you know, you get the IRB approved, we go in with a group of
kids and we have them do these things and ask these scientific questions and say this
group of learners learns this way; that group of learners learns that way, et cetera, et
cetera. And then we'll be able to publish high-quality journal papers that say here's some
real results about what these learners do if you try this or you try that.
The phrase truth or consequences was -- that was the brainchild of -- in this context of
John Nordlinger who wanted me to have a slide called truth and consequences, and so I
just figured I'd go with it. So basically here's the four phases, and here is the external
things that come out of the four phases. First, you know, we give presentations like this,
we start giving publications about our game design, we build our prototypes which we'll
eventually like to make available to other people, and then we end up with all these mini
games.
But the real important thing is the last words on that slide, of course as an institute you
have public outreach, et cetera, et cetera, but you really want to be able to give
scientifically valid results back to the community.
We're going to build a bunch of software, these mini games, individual mini games, the
in-game journaling and the post-game analysis software, and then we're also going to
hope to be a kind of a cultural magnet for fellow travelers -- can I say fellow travelers?
Okay, yeah. And people who are older than a certain age will get like nervous when I say
that.
And so the thing is, what we've discovered, what we've all discovered -- which is a little
bit freaky -- is that since this got announced on October 7th, we've been inundated by
e-mails with educators, with game designers, with people who've been doing this study in
Wisconsin with their students or in Hawaii with their blah, blah, blah. And we're trying
to figure out there's this immense amount of energy of people who want to work with us.
And I just got an e-mail this morning from a woman who's working with a Nobel Peace
Prize Laureate about games for getting people awareness of abuse against women in
Africa and can we help figure out an educational game. And this happens all the time.
So there's a tremendous amount of energy from a larger community that wants to work
with an organization that would be able to say what do we do that's effective. And so
we're trying to figure out how to that deal with that in the best possible way.
But the core nut of what we're doing is that after three years we want to both address the
funology and the learnology. How do we make things that are fun experiences, that are
engaging, even addicting, and the result of it is that kids will learn in a way that is
repeatable and assessable and measurable so that people can go off and have a paradigm
for how to do this.
So that's kind of the overview of why we're doing this. Now I'm going to switch over to
Jan who's going to talk in more detail about the actual science for learning that's been
going on. Okay. Thank you.
>> Jan Plass: Thank you, Ken.
[applause]
>> Jan Plass: Good morning. I'm Jan Plass and I'm, as Ken pointed out, codirector of the
institute and responsible for the education assessment part of the work that we're doing.
And I wanted to give you a little bit of an overview of what that entails and what our
plans are for that and what the next three years and probably several years beyond that
will hold for us.
And it all starts from the observation that we actually know very little about designing
educational games. We know something about designing games and they're becoming
very successful obviously, but designing educational games is a much harder thing to do.
And what we currently know is some heuristics, we know from game designers who have
designed games experience reports that don't have empirical support in the sense of a
broad base of research but where somebody said this really has worked for me or for this
game or for that game.
We have specialized research. There's a lot of work that has been done by Jim Gi
[phonetic], Yasmin Kafai, Constance Steinkuehler, Cort Squire [phonetic], et cetera, on
very specific games and very specific questions within those games. But then there's a
huge leap, and we have the foundational research on learning and general learning with
media, multimedia learning, simulation type of research, research in emotion and
cognition.
But there's a huge gap in between. There's nothing that would provide us general
guidelines about how to design games to make them educationally effective. There's
very little about how to design fun experiences that are at educational. There's very little
about how to design playful kind of activities for learners that are educational that are
implemented in the form of computer-based games. And that's where we're positioning
ourselves.
So what do we need to know about it. We need to bridge this basic research on learning,
on cognition and heuristics from current game designers. We need to broaden the tools
that we have. In many cases we have to invent new tools and appropriate new
methodologies for the research that we're doing. And we need to look for the broad range
of learners in authentic settings.
It's really hard to take that into the lab, even though we're going to do that, but when we
do something in the lab we'll always follow up in the field and make sure that what we
studied wasn't just an artifact that can't be reproduced when you actually go out in the
settings where people are using these games.
And therefore we have this interdisciplinary team of game designers, game developers,
educational, psychological researchers in what I believe is the first time that we're
bringing these together in the format that we're having. And what we want is in the end
theory-based, empirically validated design principles for educational games.
Now, we are not going to have those after three years in a full spectrum that you could
use and design games. But we hope to have made a really good start in some of the areas
that we're working on, and we're seeing this work as kind of setting the research agenda
for the next decade or so, and not just for ourselves but for other people, and providing
the methodology and the tools and the gaming architecture so that they can do that work
kind of following the outline that we're developing.
So it started all out with what makes games educational, and we went on to say, well,
what it is is actually the engaging part of that, so what makes games engaging and
therefore educational. And then we moved on to say, well, it's the fun part that makes it
engaging and educational. And in that are essentially are hiding all of the research
questions that we have.
And those research questions are about design factors and design patterns that are, as Ken
had shown earlier, of a cognitive nature, affective emotional nature and social cultural
nature. And we want to see whether the integration of games, what kind of conditions
there are for the integration of games.
If our work has shown us anything in the past, it is that whatever you design, when you
integrate it into existing environments, you need to understand this environment very,
very well in order for it to work. It's more important for educational materials than for
anything else to understand where they're being used and what the conditions are. If this
goes into schools, what are the conditions in schools, what is the training of teachers and
so on.
So there's a whole set of parameters that could make a very successful -- otherwise
successful game not successful in the educational market because teachers might say I
have nothing to run this on or my students don't want to play this kind of game, they
would have preferred another kind of game. So understanding the culture where this is
being used, what support teachers require, what integration practices are and so on.
And then the impact of learning characteristic. What kind of differences we see in our
learners in gender, age, prior knowledge, cognitive abilities, self-regulation skills, et
cetera, on the educational effectiveness of games. All of those are research questions that
we're going to address. And this is the slide that Ken showed earlier, would we want our
educational games and what we are looking for are the educational game design
principles. And the definition of making games educational is that there's different types
of knowledge outcomes that can be transferred to other settings, and they are factual
knowledge, conceptual knowledge, procedural knowledge of how to do things and
metacognitive knowledge.
And that is probably one of the most important and often completely missed part of
designing games and using games. It is not just what you learn and what procedural
knowledge you acquire of doing things and what factual knowledge and conceptual
knowledge you acquire about the world and concepts within the work, but also how you
learn.
The insights we gain about our own learning process, the understanding of how we set
goals, how we monitor goal achievement, how we adjust our strategies, what strategies
work in what situations, all of that we call metacognitive knowledge. Self-regulation is
one part of that.
And people who are sitting in this room, I can say without any hesitation have the highest
metacognitive knowledge and self-regulation skills that one can have most likely because
you wouldn't have made it here. People who are sitting in the schools they we are trying
to get into and have established contact in New York City often do not have those
metacognitive skills and self-regulation skills. They will read a text and after two
minutes they will say I'm done, I know everything, and when you test them they know
nothing.
So the knowledge about their own learning is something that is very hard to obtain if you
are not trained in that from an early age on, and that's metacognition, which games give
you. There's a lot of feedback in games. If you think you know and you get the next
challenge and you realize you can't solve it, then that's a very nice feedback mechanism
of giving you in a very specific way and detailed way that feedback on metacognition.
So we're looking at these different design factors for that. And the experts we have on
the team specialize in more cognitive aspect, in more emotional factors, in social cultural
factors. So this is how we're splitting up the work of people who are on the assessment
team even though we're all looking at that together in the big picture, but that's how we
kind of separate the work out.
And I want to just talk briefly about how that fits together. So all of those will kind of
form design principles for games that then will lead to the knowledge acquisition we're
looking for and then feedback into building those design principles. Ken pointed that out
already, and he's already talked about this slide here, so I'm going to skip that.
What are cognitive design factors I'm talking about? What examples for that? Here are a
few. There's the educational approach in general. The act of -- aspect of learning,
exploration and discovery interestingly recently has come under a lot of criticism in the
field of education. There are several papers out that say forget discovery learning, forget
active learning and exploration, direct instruction is the only thing that works, especially
for people who have little prior knowledge.
And we're trying to counter that. We're trying -- this is a very myopic view. We're trying
to counter that and we're trying to say because of things like the metacognition that is
typically completely missed in this research, exploration is really what is important in the
active part of learning [inaudible] the narrative structure and model progression, model
progression going from simple models to more complex models and so on are our
questions of the educational approach that need to be implemented. In the game and we
need to have guidelines of how to do that.
Representation of information or information design questions of what is appropriate for
any given task of how to represent information? We currently do research on iconic
versus symbolic representations, when do you want to represent, use icons of let's say a
flame or a burn to represent heat or a weight to represent pressure, et cetera, and how to
integrate that into the design of visual materials for learning, especially science learning.
And we're going into middle schools first and we're going into science and math and
technology, engineering curricula in middle schools, so it is all about the visualization for
science education. And so the question is how do you visualize the elements in that. One
reason why many of the current materials fail is because people visualize for specialists
but not for beginning learners. There are questions of cognitive load that can be design
factors, how do you guide visual attention, how do you use cuing mechanisms to direct
the attention to just the right spot, et cetera. All of those can be design guidelines that
we're looking for.
There are parallels to that in multimedia learning research already. Some of them are
very good. Others are very questionable. And so we have a foundation already that we
can work from, but we have to bring that into the realm of gaming. And the types of
interactivity. When we talk about engagement, when we talk about interaction, we really
need to look at the taxonomy or typology of that and say, well, what does that actually
mean to be engaged?
I can be fully engaged in something that is educational and have learned nothing in the
end. And that would be the case if I have just a behavioral engagement of moving
numbers around, typing results in a box and so on. But I don't have the cognitive
engagement of actually processing that material, connecting it to what I already know,
and then creating new knowledge structures and knowledge representations and making
sense of what I'm looking at.
So if I design an educational game that doesn't foster that, then that won't be a successful
game of transferring skills out of the game. You might have had a lot of fun, you might
have thought you were very successful in the game, but then when you actually come to
applying it elsewhere, you might not have what it takes to do that because the game didn't
foster that.
And we also believe that the affective engagement on an emotional level is something
that games can do, and I'll talk about that in a moment.
Here's an example of work we're doing with the school of medicine right now. We're
building modules for surgery education. And so we're actually applying all of the design
principles, not just to games right now, but we're already going outside of the area of
what is just the game and looking at other materials. And the idea is to say principles that
we're interested in don't necessarily have to just result in games, they could also be used
to enhance other educational materials.
So you could say I have this surgery module for surgery education, this is about
abdominal exam right now [inaudible] what the sectors are that you have to look for, and
how can we enhance an experience like this with design factors that come out of this
research? So it might not be a game in the end, but it still has game-like features in this.
So that's the goal for this.
There are emotional design factors about the visual design, control and feedback, the
musical score and sound, interesting motivation and so on and how that can affect
learning. I've done some research in this area. It's completely underresearched. We have
the link from cognition and emotion, we know that emotion broadens creativity and so
on, but the link to learning is completely underresearched. So we're doing that.
And in our previous research with Ken Perlin and Mary Flanagan on the Rapunzel
project, we've shown that the self-efficacy in games can increase based -- in girls can
increase based on playing in the game and learning how to program in a playful
environment rather than in a typical programming environment.
There are social cultural factors that have to do with social interactions, collaboration
versus computation, social presence and so on that we're looking at. And there are game
integration factors that are on the school level and the teacher level and on the student
level of trying to understand what it takes to make these games once they're designed
actually work in the settings where they're going. So those are examples for those.
And just to give you one example of what we're looking at in terms of engagement,
interactivity, we're looking at triangulating from collecting different types of data. We're
going to have self-reports, we're going to have physiological responses and behavioral
responses.
And what that could look like is self-report during the game play and after the play of
asking players are you engaged; physiological responses like gestures, facial expressions,
heart rate, pupil dilations, et cetera; we have an eye tracker where we can do that with -we, you know, can use other input devices to collect that information while they're
playing the games and synchronizing that with the behavioral responses which is the log
that we're keeping, the journaling from the game and the video of the game player and of
what's happening in the game and look at learning outcomes together.
And so the idea is to collect multiple sources of information and essentially redefine the
field and redefine how we measure certain concepts, which in the field of education,
particular in this setting, is much needed because we're relying a lot on surveys and kind
of post-treatment measures that are not concurrent measures while you're learning, but
measures that happen after the fact and therefore are of much less validity.
And then we're looking at comparison of different logs of and different measures and
matching that up with events where I could say I have a game here and I have certain
events within that game and I see in the user log certain behaviors and activity and I see
that the heart rate is responding, so there might be some in the self-report of activity, so
there might be something happening here that is kind of dropping engagement,
engagement is going up, and if I see there are multiple indicators are going up in the same
direction, then I would take out of that that we can define this as a high-engagement stage
that we would then try to match to learning outcomes. And if we see that there was
actually a learning outcome that we desired, then we have a match of multiple data
sources to show that that was due to certain design features that were implemented there.
Now, this is something that we're going to talk about with the Game Studio folks here,
and I'm really happy to see you guys here and looking forward to our talks, because a lot
of work has already been done in this area, and we're trying to apply that and coordinate
to apply that to the educational research we're doing so that we're benefiting from the
expertise that already exists in-house here.
And that is really it from my side. And I'm going to hand it off to Katherine.
[applause].
>> Katherine Isbister: So if you remember the tiny box at the very, very beginning of the
research process, it was [inaudible] expert interviews. And that's the piece that I'm going
to talk about just a little bit today.
I actually come from a tradition of -- a social science tradition but also a design tradition.
>>: Oh, the mic. Did you turn on ->> Katherine Isbister: I did turn it on. This always happens with ladies' clothing. I
usually find if I hold it to my face it looks a lot better.
Okay. So I come from a tradition that combines social science research with actual
design and improving design. And I've been working with a couple of colleagues, Mary
Flanagan who's at Dartmouth and Colleen Macklin who's at Parsons. And the three of us
decided it would be really valuable to do some formative research at the beginning of this
project and do some bottom-up gathering of people's design intuitions, the kinds of
working practices people already have in terms of games and learning and building
engaging games, and trying to gather that not just from researchers and educators, but
also from players and developers who have insights not unlike our board member that
Ken was referring to.
So here the kinds of questions we've been asking these folks. Essentially what we've
been doing is we've taken the working model, the conceptual model that we have of how
games and learning fit together and how you might begin to think about how outcomes
match the kinds of things you're trying to design toward. And we give that to people.
And then they get a chance to look at it. And then we sit down together and actually I've
been conducting some of the interviews using Skype, using a recorder in Skype so people
can just be at their studio or wherever they are and we can easily record it. I'll show you
a sample of that at the end of this.
And we asked them to think about how that model matches how they think about what
they're doing and what's effective about what they're doing, and then we move into
talking about the design, specific design practices they have, patterns that they think are
especially effective in terms of engaging people or that they think relate to learning.
We also ask them about things we should never, never, never do; in other words, what are
best practices that they've already mastered and things that they've eliminated as things
that are absolute nonsense that one would never do so that we can save ourselves a lot of
time and effort.
And is there anything else that we've completely left out of our model. Because
sometimes insights come out of left field and somebody has a brilliant practice that
they're doing, that if we were able to empirically validate, that would really benefit the
community. And we're doing a sort of a snowball technique where we ask people to lead
us to others who know a lot about the field.
So what we're hoping will come out of this effort is of course -- hmm. Okay. Hit no.
What we're hoping -- first of all, what we're hoping will come out of this is some specific
design patterns that are sort of that middle box that Jan was pointing you to in the model,
that some of the design patterns come from this sort of high-level conceptualization from
social scientists, but some of them may come from actual practitioners who have really
interesting things to say about how you design, that we can link back to that theory and
out to the outcome stuff, right, and sort of jump-start us into that middle ground.
And we're planning to take clips from these interviews. We're videotaping these
interviews with Skype and also some of them in person and put them on the Games for
Learning Institute's Web site, some abbreviated samples of people's insights in their own
words as a resource for the games and learning community at large.
So the idea is that we're building a sort of repository where people can get some insights
and some interesting intuitions about their own practice.
And I have a sample clip. We've only just begin to do interviewing. We're sort of in the
beta phase. We're refining our question set and how we really get at these issues with
people.
I thought I'd show you a super brief clip, and I like this clip because both Ken and Jan
mentioned transference of skills and metacognition and how in essence, you know, all
games are learning engines or learning tools. But what interests educators and
government funders and so forth is what parts of that learning could actually transfer out
into the real world.
And so this is about a two-minute video clip from an interview I did with a guy named
Nick Fortugno. Is there anybody here who's familiar with Nick's work? A couple of
people. Okay. So Nick is actually currently the CEO of his own casual game company.
But before that he worked at Gamelab, which is Eric Zimmerman and Katie Salen's shop.
And he's not only a game designer, he also is an educator. He has taught at schools in the
Bronx and so forth. So he's a person who really bridges all these words. And he had
some really interesting things to say about how playing Dungeons & Dragons changed
his life. So I'm going to play you that clip.
[video playing]
>> Katherine Isbister: Oh, I don't think -- Ken, I don't think this is -- I don't think you
queued it up at the beginning. Let me start it at the start.
[video playing]
>> Katherine Isbister: So that's just one person's anecdotal experience, and he also goes
on in the interview to talk about more specific design tactics that he uses to try to
encourage learning in his games. Maybe it speaks to some of those of you who played
D&D as kids. I don't know.
So the last slide here, if I can figure out how to get the slideshow back on.
>>: [inaudible] want me to do it?
>> Katherine Isbister: Yeah. I hate to admit this, but I'm actually a person who uses a
Mac, so the interface is unfamiliar to me in this platform.
Okay. So my last slide is actually an appeal to all of you, which is that we would really
love to solicit any and all of you who think you have valuable insights around this topic
as soon as possible to work with us and do an interview. It's painless, it's easy, you get to
decide if you don't want anybody to see what you said and so forth. But it would really
benefit us. We're really trying to build a community of practice around this and learn
from what other people have -- what knowledge you have, not just because you know
cognitive or emotional theory but because you've actually made some stuff or even
because you're a player and you've had some really deep insights through your own play.
So if you're interested in doing that, you can contact me, you can send me an e-mail. If
there's somebody you know who you think would be really great person for us to
interview, you could also pass that along, and that would be incredibly helpful. And
we're aiming to have a set of these interviews posted on the Games for Learning Institute
Web site by the time of the Game Developers Conference this year. So if you're just
interested in looking at some of these insights, you should be able to see some interesting
material at that time on the Web site. And I think the URL is in the slides. So that's it.
[applause]
>> Ken Perlin: I guess we answer questions? You want to all get up for the questions?
Is that what we're doing now? We're answering questions?
>> John Nordlinger: Yeah, that'd be great.
>> Ken Perlin: Okay.
>>: I work with teachers [inaudible] teachers also science teachers in an urban setting,
and I'm very excited about these games because we have been trying almost everything,
and it's not really working. So my question is how do teachers access the skills and
support that they will need to facilitate the students when they use these games? Because
currently one of the problems I run into is a really high level of teacher anxiety with
computers. And with the games I don't know -- and I'm very concerned with the teacher
part. I noticed when you spoke about having a student working with the design team, but
I didn't hear about having an educator also working.
>> Jan Plass: I'm going to take this one. We're currently engaged in a large-scale U.S.
Department of Education-funded project where we build simulations for science
education. So we're in the schools right now.
>>: Microphone.
>> Jan Plass: I turned my on, but -- oh, here we go. So we're doing this all the time. We
found actually very early on that if you don't talk to teachers and if you don't build those
materials for teachers, then you may as well not even do the work because it's not worth
it. If you don't enable teachers who are extremely bright, who are extremely passionate
about what they're doing but they're also extremely deprived for time, you know, you
can't say, here, why don't you look at that game, come up with a curriculum, see how it
integrates. It's just not going to work.
So what we're doing is we're working with teachers who help us build lesson plans, who
help us build ideas of how to integrate what we're doing, the interventions, the materials
we're building, how to integrate those into the curriculum, and then when we give people
our materials, we say here's the simulation or game or whatever it is and here's the lesson
plan and here's maybe even a video of showing the teacher how they used it in their
classroom.
We're also facilitating networks of exchanges of teachers, so we have several teachers
who have been working with us in our research so far who kind of connect to one another
and exchange information about that amongst themselves.
So we're working from very early on with the teachers because that's the only way to do
it. You're absolutely right. And there is a person, Catherine Milne, on the team of
investigators who is actually a science educator who trains teachers, so she helps us with
that side of things too.
>>: One of the things that's concerning me about this, I didn't hear a lot of talk about,
you know, kind of the basic assumptions that are going into this in terms of you have a
kid in front of a PC and, you know -- which, you know, there's a long history of gaming
in education. And it almost universally has been group gaming. And it seems like there's
an underlying assumption here this is a one person, one PC, and if there's any group
interaction it's through a network, it's not actually a bunch -- you know, you're limited by
the hardware.
And so I look at this and I say, you know, in some ways I find the effort a little bit
missing the point of what games are used for in the classroom in that it's a social activity
and it's a group thing. And so is there any plans to extend this to, you know,
hardware-like surface or stuff like that, which I think is kind of absolutely critical for
gaming in the classroom.
>> Katherine Isbister: I can't speak to the hardware issue, but I can say that my lab at
NYU Poly is focused on social games and that the early games research we're doing with
popular games that are very appealing to kids include genres of games that are explicitly
co-located social play. So things like party games, right, things that are very appealing,
play formats that, you know, you use on a console and you have a shared screen and you
play together. So it's not addressing the new technology you're talking about.
But in terms of observing and taking into account the social nature of play, it's absolutely
intended to be part of what the factors are for fun engagement and also learning.
>>: [inaudible] then that there would be a gaming platform that's not a PC?
>> Jan Plass: The assumption is that the game platform is not what we're so interested in.
We're interested in different genres of games. And one might lend itself to be played on
the PC, another one might lend itself to be played on the handheld, another one might be
in a kind of browser-based environment where it doesn't matter what platform you use.
It's not the hardware that concerns us; it's what you do with it.
And you absolutely right. There's a lot that we left out here because we would have had
to make it a half-day or a full-day kind of presentation. There is no plan whatsoever to
make that one player/one device. There is -- but what we are doing in this first phase of
broadening out, kind looking at a very broad set of games, is to say let's look at games
that you play individually, let's look at games that you play in groups, let's look at
different genres of games and what do we understand about those different genres, and
we would have to treat each genre differently in the design principles we come up with.
And the one design factor that I called social cultural factor is exactly about the
collaborative or competitive nature of games, not just two players against one another,
but two or three kids around a device or so. We could have augmented reality games
where it's not about the technology so much at all but more about what you actually do.
All of those are open questions for the institute. We might initially look at specific types
of those that we have to select this part of the first phase, but there was never an
assumption from our side that this would be kind of an isolated [inaudible].
>>: So I know that a lot of the -- especially from the analysis work you need a login and
so a lot of this needs to be mediated by a computer in some sense because of all that login
stuff. But there's this huge class of other games that don't do that and then of course also
reduces the costs that are associated with that.
And I'm curious if there's any means, any direction in any of your research that's going to
try to look at those things of, say, that are just pencil-and-paper-based games. I don't
think there's been that much really deep studying of all those classes of games for
learning in there. I mean, like D&D is a great example of that which requires this much
more flexible sort of, you know, on tap kind of stuff and including the math, et cetera. So
I'm just curious if there's a way to study that in the umbrella that you're trying to do.
>> Jan Plass: Yeah, the methodology we're developing can take advantage of journaling
data, but also uses observation by human observers, by videotaping, uses interviews, it
uses kind of asking questions during game play whether that's computer based or not. So
it could very easily be applied to chess or any other game where we don't have journaling
data available yet.
>> Katherine Isbister: And there are people in the interview pool, like Frank Lantz, for
example, who design what he calls big games that are, you know, running around in the
environment or alternative reality games. So we're certainly soliciting information from
people who do that kind of design.
>>: Is your game going to be kind of [inaudible] that you would be taking it out into the
middle schools, testing the games, and then going back and looking at correcting things
[inaudible]?
>> Jan Plass: So there's -- we're going primarily into after-school programs. We're going
into -- we're working with the New York Hall of Science and their educational
department, et cetera. And so we're going to be testing things there. We're going to be
doing portions of that in the lab, and it's [inaudible] back and forth.
But I think one of the difficulties in presenting what we're doing is that we're not building
a game. All right. We're -- many people are building a game and you don't learn so
much from just building one game. We're an institute that tries to define the research
agenda. And we're trying to say what are things that people should be looking at. So
we're starting in a very broad way, and then we're narrowing down. And we could make
speculations of where we would be in three years and what we have decided to focus on,
but that would just be that: speculations.
We essentially want to question all of the assumptions we've made so far about gaming
and say what if we didn't have all of that and what if we just had a fresh look at what it
means to be playing games, why is it engaging, and how can we apply that to learning.
And then, yes, we will [inaudible] will build mini games based on that, but they will be
mini games. They will not be big games because we leave that to the professionals who
actually built big games.
We will be designing and testing and studying games in order to answer our research
questions, and then they will be available for schools. But as an institute you kind of
have to make a choice of what side you want to be on, and we're on the research side of
building research methodology and the gaming architecture and the gaming
instrumentation of the games to enable a whole research agenda that's not just ours, that's
not just people in our institute.
>>: [inaudible] me that one of the problems with educational games is the moment you
apply the label "educational" they become less fun. Would you agree that that is a
psychological hurdle, and how do you imagine you might mitigate it?
>> Ken Perlin: Yeah, there's a New Yorker cartoon that I really like that John hates when
I talk about, which is -- some of you may have seen it. It's from 1925 when The New
Yorker was new. There's a little girl dressed in her cute little flapper outfit and she's
looking at this plate of something in front of her. And her mother is saying, Eat your
broccoli, dear, and she is glaring at it and she's saying, I say it's spinach and I say the hell
with it. And he hates when I say this because he loves spinach and he thinks I'm giving
spinach a bad name.
>> John Nordlinger: I just think it's ironic with Ken's vegan [inaudible].
>> Ken Perlin: No, so the -- so, yeah. So, no, I mean, that speaks to -- that speaks to
Cailin's point that I told you, you know, that the -- one of the research questions we need
to ask is the -- what's the rhetoric around this. And there's this great -- you know, there's
this notion that game designers and storytellers talk about of the magic circle. You know,
when you're in the magic circle and you're plying a game and I'm shooting at you and
killing you, but it's okay because we're just playing a game or, you know, the world
blows up, you know, but it's okay because it's just some stupid Hollywood movie, it's
fine, we're not scared that this world blew up because then, you know, the credits roll.
And Jan actually coined this wonderful phrase for this project, which I really like, there is
the magic circle and there's also what he called the learning circle where it's different. It's
like I know that there's only one consequence. And the one consequence is I'm here to
learn. It's okay. It's all right that I'm learning.
So one of the hypotheses we have is if you present to the -- it's just hypothesis; we have
to test it -- if you present to the kids honestly, look, okay, we know you're learning, you
know you're learning, you know, the alternative is you're all just looking at these stupid
textbooks that aren't responsive to you, maybe this is an okay way to spend your day
when you're in a learning experience. But we have to test it. I'm sorry, I didn't mean to
pre-empt you.
>> Jan Plass: No, that's okay. I like when you quote me. Bruce?
>> Bruce Homer: Yeah, I just wanted to add that [inaudible] involved in the project
[inaudible] earlier where we're developing these simulations to teach chemistry to these
low-prior-knowledge kids in high schools in New York. And we've had a number of
teachers come back to us. And these simulations, we don't say these are fun, we just say,
you know, these are [inaudible] these are where they get to do stuff.
And we've had teachers come back to us and say, wow, I'm amazed, this one kid who
comes in, she always shows up late, she never pays attention, she's always goofing off,
she actually sat down and used the simulation for the whole class.
And so I think compared to -- as Ken was saying, compared to a lot of what we tend to do
in the classroom, even if we -- it has the label, they figure out that we're trying to teach
them, just doing something a little different is going to make it exciting and interesting
and hopefully keep those kids engaged.
>> Ken Perlin: But these aren't just assertions. These are hypotheses that we intend to
test.
>> Jan Plass: But one last comment about this, because we're talking about this all the
time. We may not tell them it's a game and we may not tell them it's educational. We
may just say, here, do you want to use this and learn from it, or, you know, we might say
this teaches you about X.
But we might not call it a game. Because we have this idea that possible -- and we need
to test that too -- once you call it a game, there's assumptions that are violated by what we
give them because you learn from it. But there is no reason to call it a game. You could
just say this is a fun way of learning. So it's definitely something we're thinking about
because we might be in the process of inventing an entirely new genre that we don't have
a name for yet of new educational materials that have game-like features.
>> Ken Perlin: And by analogy, I mean, there is such a thing as, for example,
documentary films. And documentary filmmakers don't need to pretend it's a Hollywood
blockbuster. It's like it's properly labeled and people go see it knowing what they're
going to get. I mean, there are different categories and people are capable of
understanding that. And we just have to understand -- again, we have to understand
where the boundaries of perception are. You had a [inaudible].
>>: Curious about in your observation phase what games have you looked at already or
what games you plan to look at [inaudible]?
>> Jan Plass: Well, this is actually made a little bit difficult by the fact that we're going
into middle schools and you just can't take Halo into the middle school. So we're looking
at one example might be where I had the screen shots from Professor Layton and the
Curious Village, there are the games that -- any Mario game, Mario Galaxy, whatever,
any game that is age-appropriate game for middle schools that commercially was
successful or has critical acclaim. Spores is another example.
So that's what we start with. And then we're going to try and sneak in some of the games
that are not appropriate for the age group or the players tell us we're playing it already.
But we're having problems of getting approval. Every research of this nature has to be
approved by an ethics board. And they don't understand this work. They think that if
there's a label that says mature, then it's actually -- middle schools won't play it, which of
course is not the case. But for researchers it's very hard to convince them to say, well,
they're already doing it, we just want to watch them do it.
But we think we might find a way to also do that. But it's all about commercial
off-the-shelf games right now that engage kids. And it doesn't have to be a kind of highly
visual game. I mean, I'm playing a game right now that is practically just [inaudible]
Tribal Wars, I don't know who else is playing it, but where the emotional and the
engagement factor of that game is just as high as in other games. And so we're looking
for those kind of games also.
>> Ken Perlin: We're actually waiting for Rockstar to come out with Grand Theft Auto
Seventh Grade, but they're not going to do that soon probably. Okay. So you had your
hand up for a while.
>>: One of the things I was going to say was I was going to comment about the teachers,
but before that I think there was a comment about calling the games games that I think
you're already at risk, but there is some, you know, papers that were published -- I think it
was like ten years ago [inaudible] and she was doing some studies with some type of
educational games, simulation [inaudible] and she called it a game. And then when she
brought the kids to play the game, they were saying, well, this is not a game. You said it
was a game, but this is not a game.
And so then there becomes [inaudible] entertainment, movement and people now they -most of the people that was involved in that movement, they've done -- go back to call
anything entertainment anymore just because the term become [inaudible] have such a
bad reputation because he was successful model before, some -- so, yes, in that regard
[inaudible] go with calling this something else [inaudible] regarding with the teachers, I
think was a very important point about the teachers' high level of anxiety. And you
mentioned that it's important to bring the teachers to talk in this conversation.
But from my experience [inaudible] high school teachers in a [inaudible] to create
educational games, the high level of anxiety was definitely there. But one of the things
that helped them to kind of lower the level of anxiety was to [inaudible] teachers that
have kind of similar skills [inaudible] skills and some similar skills, so working teachers
with teachers and talking about issues that have in the classroom was really positive, even
though they didn't get the skill from the [inaudible] to actually develop games, most of
them, there were some that were like -- you always have these strange case that a
software developer that becomes a teacher after ten years of being a software developer,
that he's very successful and have to [inaudible] normal teachers, but how did this
conversation teacher to teacher and given the tools that they think they were very helpful
for the classroom because they develop very simple games that you wouldn't imagine that
would be so helpful for them but they weren't real excited about the games that some of
the teacher [inaudible] and because it's free, because it was something that [inaudible]
potential to use in the classroom, you were sensing that they were very willing to use.
And they were very short, short games that they thought the kids could play like for ten
minutes or something.
So I think having these conversations with teachers and letting them know about what it
takes to create a game, that was very positive, because some of them thought that creating
a game was a little bit simpler than it was. So they have really expectations, huge ideas
about what they wanted to build but [inaudible] haven't had to build it, they realize it
entails quite a bit of skills and quite a bit of time to create something that [inaudible]
smaller games that were simpler and easier to adapt in their regular classrooms.
>> Jan Plass: That's very interesting.
>> Katherine Isbister: [inaudible] dissertation. There's a short version of it.
>> Jan Plass: That's very interesting [inaudible] we have in New York City a very
interesting situation right now where the new chancellor of the school system has started
a lot of model schools where people can say I want to open a new school and I want to try
an entirely new approach. And so we have all these small schools that are just starting
with very enthusiastic principals and teachers who are open to [inaudible] new things. So
teacher anxiety for us is actually nonexistent because we're working with those teachers
who come to us and say can we work with you.
I have -- I'm teaching in a program called educational communication and technology,
and in that program we have as part of our students teachers who are in schools right
now. So all of those students are kind of begging me, can we work with you. So we
have a teacher base that is very enthusiastic, very knowledgeable about this, that we can
take as a seed, and then once other teachers see what's going on and these schools are
visited all the time by other teachers and other principals, it's going to spread in a viral
way.
So we're not trying to convince teachers to use something that they don't want to use.
We're building something for those teachers who are ready to use something new. But
we're also taking the approach that I'm not going to ask a teacher to write a textbook
about their own teaching. Why would I ask them to write a game, unless they really want
to. You know, 1 in 1,000 might want to write the textbook or build the game or build that
Web page.
But teachers are really professionals in integrating that into their curriculum. And so
that's what we're going to give them and we're working with those that are very
enthusiastic about it already. But those are very good points that you've made, and we
found very similar things in our work also.
>>: So I guess the first part is my apologies, but it seems like you're talking with students
and teachers. Well, there is -- and you just highlighted one, the textbooks, there are
people that make the textbooks, like, for example, a Houghton Mifflin, for example,
where they develop the curriculum. Have you been working with them, or maybe I
missed this in the ->> Katherine Isbister: No, you didn't miss it. I forgot to mention that we've also talked a
lot about interviewing subject matter experts once we have a better feel for some of the
principles we know we want to try to design to in our prototypes. And when we dive
down into specific material, that is the knowledge we're trying to imbue from a game,
then I think at that point there are definitely plans to talk to educators who actually teach
that kind of material very successfully, which is somewhat similar to textbook design,
right, but as somebody who's doing that live and what are the principles they use to get
that material across.
>>: [inaudible] and so they -- and I'm actually with several people in this group here
we're in the FlightSim team where we do flight sim in a commercial product where, for
example, we're like, well, can we do Ancient Rome in FlightSim. And so the answer was
absolutely yes.
>> Katherine Isbister: With the wings instead of an aircraft, right?
>>: Actually, it could be ground as well, so they [inaudible] just with an add-on pack in
the FlightSim you could take a tour of Ancient Rome and it's super cost-effective for
them for some of the things that you're talking about. So this doesn't necessarily have the
social interaction, but I did bring up [inaudible] the end of a particular learning sequence,
you could do a chariot race in Circus Maximus, you know, do a multiplayer thing. And
things like that.
So, you know, they're beyond just the -- you know, the Halos and the more popular
games. Take a look in FlightSim, is an E for everybody game, so as far as getting over
some of these hurdles, it's already over several of them. So I'll just kind of toss that into
your bucket as you're looking at future work in things as a platform to kind of take a look
at.
>> Ken Perlin: And also if you put airplanes in Ancient Rome you have a great premise
for a fantasy world.
>>: [inaudible]
>> Ken Perlin: Just don't get too close to the sun or something like that. She's had her
hand up for quite a while.
>>: Have you made any games for like in the two to -- you know, the two to five age
group which can be tested out in day cares? If, you know, Mickey Mouse Club has
like -- kids will endlessly watch it again and again, and it's a great teaching tool.
>> Ken Perlin: Well, we've been focusing -- our initial mandate has been to focus on
middle school and science, technology, engineering, and math. And one of the reasons
we're starting there is we're also addressing a very real bit of a national crisis. At the ages
of roughly 11 through 13 is when a very large proportion of kids, notably underserved
minorities and female learners, just drop out of these topics. And we're sort of hoping we
can give a good kick-start to that.
The methods that we're using in general should be applicable to other groups. But this is
where we're beginning with our limited resources. And Bruce had something to say
about it as well.
>> Bruce Homer: I'm actually a developmental psychologist by training, and that's the
age group that I started with, preschool-age kids. And there's actually very little
empirical research on games with that age group. I found almost nothing. So it's
certainly an area that I'm interested in, but that's not where the focus is.
>> Ken Perlin: And I when I've had conversations with Amy Bruckman, as you had
brought her up, about this, she's reported that it's very difficult, it's much more difficult to
work with children who are under the age where they're thinking linearly. We can work
with 11- to 13-year-olds as collaborators and design things for them; it's much more
difficult to work with children at the younger age. So we'd rather actually first develop a
firm foundation with this age group before we start exploring the inherently more
difficult problem.
>>: Yes. I just have one observation. I work a lot with middle school teachers, and I've
worked with the students as well. And one of the things I'm concerned about, from the
moment you start calling something a game, the middle school students, their perception
about what is learning and their expectations also, from the moment you say it's a game
and students start playing and they will say that doesn't matter, it's a game.
So I'm concerned that students start using these and then they're thinking it's a game and
it really doesn't matter. So what are you -- you're thinking about addressing that with
middle school students?
>> Jan Plass: We'll probably do it incrementally. We'll tell half of them it's a game and
the other half it's something else, and then we'll find out where it matters.
To us those are all empirical questions. What should we call it, how should we introduce
the material. That's all part of the question that I mentioned is the integration question,
how do you integrate it into the setting, what do you call it, how do you prepare the
students that are going to use it, what should teachers do with it. And to us it's an
empirical question and we have no needs to call it a game. We just -- let's just say the
code name is Games For Learning. The internal name for it. And what it will be later
externally, we don't know.
>> Bruce Homer: So we're Games For Learning, but we may not call it games; we may
not call it learning.
>> Jan Plass: That's right.
>> Ken Perlin: We may not even use the word [inaudible].
>> Jan Plass: [inaudible] yeah.
>> Ken Perlin: I think we maybe -- I don't know, do we have time, John? Or how are we
doing with time.
>> John Nordlinger: We have one more question.
>> Ken Perlin: Okay.
>>: What are the -- I know you guys are very early in your research, so it's okay if you
can't answer this, but one of the problems or difficulties I see in developing games for
learning, especially in classroom environments, is the wide discrepancy in learning
ability. What types of game-play mechanics have you found that work well in kind of
tailoring that learning experience for the individual?
>>: [inaudible] scalability.
>> Ken Perlin: Is that part of the question or part of the answer?
>>: Well, no, it's what they're -- it's feelable learning.
>> Jan Plass: [inaudible] yeah. I mean, there are many ways to address that. One is to
give you initial challenge to see where you're at, and there are -- for educational
assessment, there now are new methods that we're developing -- not us, but the
educational community -- of how to assess somebody's prior knowledge in a particular
area rapidly with one question and you know roughly where people are going.
There's ways of doing that by giving you a complex problem and saying what is your
next step of solving that problem, and depending on how much of a leap you could make,
an expert could make the leap all the way to the solution, a novice would make just a
small increment to work the solution, right? And so you can give them that flexibility,
you see roughly where we're at and you can kind of develop and adaptive response to
that.
There are other ways of self-explaining results, so when you let them do something, you
have them self-explain what they're doing. So you can be adaptive to that. But you could
also just filter that out before they even go into the game and integrate.
>> Ken Perlin: And another way of saying that, and this is sort of an obvious point once
you say it, but it's worth mentioning, is that by definition any existing game that's a
successful commercial game is successfully addressing the multiple styles and rates of
learner's problem that you're looking at.
And so one of the reasons we're starting out right at the beginning is looking at successful
commercial games is really trying to tease out what questions we can ask from how they
might be doing that and be able to sort of figure some of that out on a more formal level.
It's a very important question.
Okay. Thank you very much.
[applause]