>> Amy Draves: My name is Amy Draves and I'm here to introduce John Long who is joining us
for the Microsoft Research Visiting Speaker Series. John is here to discuss his book Darwin's
Devices, What Evolving Robots Can Teach Us about the History of Life and the Future of
Technology. Evolving robots can illustrate the power of evolution, illuminating mysteries such
as how the flexible spines of fish and mammals developed or whether or not brains are really
necessary for some species’ survival. These robots mimic and simplify animals that allowed
John and his colleagues to witness evolution in progress. John Long is a professor of biology
and cognitive science at Vassar. He is the director and cofounder of Vassar's Interdisciplinary
Robotics Research Laboratory. Please join me in giving him a very warm welcome. [applause].
>> John Long: Thank you Amy and thank you to Microsoft for inviting me here today. And
when I talk about Darwin's Devices, what I'm really talking about are evolving robots. That's
going to be the focus here and I'm going to talk to you about why we do this crazy thing of
evolving robots. Why go through all the trouble of doing that? And then I'll talk to you about
how we do it and maybe you can get some ideas or have some critiques for us about what our
process is. I want to make sure that we start off with an understanding of what I mean and
what biologists mean by evolution. As you know, evolution is fraught with misunderstanding
out there in the world. When I'm talking about evolution I'm going to talk about the definition
that we use which is a change in the genetics of a population from generation to generation.
Individuals do not evolve; I'm sorry to tell you. You will have to find other ways to improve your
self-esteem. Individuals develop; you can grow, but you cannot evolve. That's something that
is the property of a group of interbreeding individuals who are together at any place in time.
Evolution takes place from generation to generation. So that's where we're going to start in
terms of thinking about our evolutionary process, and I want to give you some bad cartoons of
a fish that I drew to illustrate how, a little bit about how evolution works. I know that you guys
know this, but this is just a little bit of remedial work for you since baby you guys haven't
thought about evolution in a few years. We have a population, a group of interbreeding
organisms. Let's call this generation one. There are a variety of colors here and let's
presuppose that each of these colors of the body has a genetic basis to it. There are genes that
code for it. And let us also suppose that there's a predator out there that really likes blue fish,
or a bunch of these predators so that after a while the predators have selected, that is they
have eaten all of the blue fish and they end up leaving behind the survivors and those survivors
become the parents and notice that the blue and blue-green are gone here. These remaining
individuals in the population are the ones that reproduce. Again, there is a genetic basis to that
color and they produce a population here that when we compare to the parental population is
quite different. So notice I've avoided the awful iconography of evolution of this kind of thing,
standing up, which is, it's horribly racist and other things. That's not the way to think about
evolution. Evolution is this process of filtering if you will from one generation to the next and
the change in the underlying structure. So that's what we're going to start off with here. Now
we're going to talk about why as a biologist I want to evolve robots. I want to evolve robots
because as a biologist I've run out of rope. There's only so much you can study when you have
the animal in front of you, believe it or not, or in the case of what was happening 500 million
years ago and all we have are the fossils. We have some beautiful fossils out of China which I'll
show you in just a second. This is an artist's imagination of a little fish called Haikouichthys
ercaicunensis might've looked like. These guys were about an inch long and they were, we find
them in schools and so we think that they were up in the water column swimming. What's
important about these guys is that they are the first complete vertebrates that we have, so we
are members of the vertebrate subphylum, and so because of that in our universal centrism
about our kind, we want to understand where we come from, right do our family history
studies. So 500 million years ago the first vertebrates were evolving. They are so named, these
vertebrates because of what? Where do we come up with that name? See, I can't help being a
professor. [laughter].
>>: The backbone?
>> John Long: Yes, the backbone exactly. The bones in your back, those individual bones, each
one of those is called vertebra, as plural there called vertebrae. We talk about a vertebral
column or a backbone that means collectively that entire structure. Oddly enough, the first
vertebrates did not have much in the way of vertebrae. It's one of the paradoxes of our kind.
So for me, someone that's really interested in the origins of our group and loves fish and things
like that, I want to know, what was going on early on? What were the selection pressures that
may have driven the origin of the backbone of these actual physical structures in the axial
skeletons of these early vertebrates? The problem is this is the data we have from 500 million
years ago. I don't know if this looks like much to you guys, but if you're used to seeing fossils of
soft tissue, this material here that you see in tan is just gorgeous. Fossils are usually bone. It's
the only thing that survives a mudslide and degradation and actually gets mineralized, that
process called taphonomy to create a fossil. These fossils are actually like right here we have
pretty good evidence that these are actually eyes and the pigment or the cups from eyes. So
here is an artist or a scientist who has drawn a rendering of these little discolorations of the
sand. From China, this is the Chinese let's see there's Lagerstätten in Southeast China where
these come from, so we think we see eyes, maybe some nasal cavities. Over here you can see,
if you've ever eaten fish, the segmented muscles of the fish and internally even we can see
evidence that there's a structure called a notochord. A notochord is the backbone without any
bones and there may even be these little blobs some experiments with vertebrae in that
notochord, so the ancestral condition, meaning the first condition that vertebrates had without
vertebrae was this notochord and then vertebrae evolved. So these are actually gorgeous
fossils. My wife is a botanist and she looks at something like this and she goes looks like a leaf;
what's the big deal? [laughter]. It's not quite as easy to see whatever you want to see, the
Rorschach test of fossils. These are gorgeous fossils, trust me on that, and one of the neat
things about this find coming out of China is that there are so many of them, there have been
50 specimens or so of Haikouichthys that have been and described for us. Given that these are
excellent fossils, it's almost the best we can get, but the problem is that fossils tell no tales. We
are missing really important things about the lives of these animals from fossils. So this is a
dead, beautiful dead, a living species called a Viper moray, but it's dead right here. This picture
was taken by Sandra Raredon and I have this here because you can see each one of those little
vertebrae that are in this Viper moray right here. But the problem is when you're dead, you're
dead. You don't have any behavior. If all you had is a representation of a dead animal, you
don't have its ecology, where the ecology is the interaction of that group of organisms with
themselves and with other kinds of organisms and with the world; all that's missing. Of course
you know that with the fossils, but if you're a biologist and you want to figure out what
happened 500 million years ago, that's the stuff we need to have. We need to have the
behavior and you need to have the ecology available to you. So this is a picture of a robot that
we call Tadro and I think this is homage. I put this into this talk, Biorobotica piscina. I don't
know if you remember the Wile E. Coyote stuff when the coyote is running and then they stop
it, and it says coyote hungricus. They fake the Latin binomial scientific name; I thought I'd
throw that in there for you guys, because I figure you are all nerds like me and you probably
watch still, as I do, those classic Looney Tunes. So there's Biorobotica piscina, fishlike bio robot
and one of the cool things that we can do with our robots, because we can make them
autonomous, and I'll explain what I mean, which you guys probably know in a second, we can
sort of make the models undead representations of these fossils, so we can recover behavior
and we can model the ecology and the ecological interactions. So what we do here in some
ways is replay the game of life, those dynamic interactions that may have occurred. So we are
modeling evolutionary processes with our robots. We can ask them a very specific question. In
my lab we can ask why did the vertebrae evolve. What do we think is going on there? What
was the selection pressure that maybe drove that evolution of that very important axial
skeleton? This is a specific case of a very general question of why did evolution happen the way
that it did? It is the problem with any historical study. You always want to understand what
happened and it's very difficult to do so. There are methods to do it, but you are always left
was saying well, this is our best guess at what's happening. So I'm going to go through all of this
and talk to you about this and I'm going to say well, this is our best guess at what's happening.
But it's a matter of testing ideas and circumscribing the feasible and the stuff that you can't say
is feasible. You say well, it's probably unlikely that it happened in a way that we haven't
described here. So that's the why, and now what I want to do is talk to you about the how of
the evolving robots. What we're doing in our laboratory. We build a special class of robot
that's called a bio robot. Bio robots are in particular and this is an example, Tadro is an
example of it, they are physically embodied. Some people talk about building robots in digital
simulation. We do that also, but we wouldn't call them bio robots. They are meant specifically
to mimic or represent an animal, and they also have to test ideas about how animals behave.
They're not just okay cool, let's build a fish; we're building a fish with certain properties that
relate to fossil fishes so that we can test ideas about how we think fossil fishes may have been
evolving. So that's a bio robot and we're going to talk about six questions that we ask when we
do a study like this. Well, they're not questions. I got thrown off by that. Six things we need to
do. Thing one, ask a question. Thing two, we have to pick an animal. Thing three, we have to
pick an environment. There is a very specific matching of any particular species with the
environment in which it evolved. And this is an important thing. Animals don't just evolve in a
vacuum. Or actually they could; could they evolve in a vacuum? That would be a different kind
of environment, wouldn't it? Anyway, I guess they could, but they don't. We're mimicking
specific features and then we're creating a genetics that we need to talk about what an
evolutionary process is, and then we are applying selection, the creative force of evolution. For
more details you can refer to chapter 5 of Darwin's Devices. So let's start with asking a
question. And by the way this is to remind me that when we do the evolution of these robots,
we have a further restriction of the kind of robots and we call them because we need a silly
name, we call them evolveobots, okay, evolving bio robots is what we're working with here. So
back to our beautiful Viper moray, we can ask a very specific question now that has in it some
assumptions that I'll unpack a little bit later, but here's the question. Did the first vertebrates
evolved vertebrae as an adaptation for improved feeding and fleeing, for eating and not being
eaten? I will explain why we think that's important in a bit. I'm going to pick an animal. Now
this is not Haikouichthys here. This is an animal called Drepanaspis and the reason we picked
Drepanaspis as our model is Haikouichthys are really tiny and we don't know tons about them.
We actually know, because these are large animals, about yay big, we know more about their
external anatomy. This is actually a 3-D model that's based on some three dimensional fossils.
We know, for example, they have, where their eyes are, where the mouth is located and that
thing called the lateral line that I'll explain just a second. So that's the animal we're going to
pick. Drepanaspis is important because it also contains, even though it's 100 million years
younger than Haikouichthys, it has no jaws, no paired fins, and no vertebrae. These are all
ancestral conditions for vertebrates, so they retain many of these ancestral vertebrate
characters. Then we need to pick an environment. What's happening 400 million years ago?
This is the age of fishes if you've ever been to a museum, before the age of dinosaurs and the
Devonian seas had predators, so here are some shark like forms and some bony fishes here
with jaws. Here's a lung fish thing and guess what, they all have, believe it or not, notochords.
They don't have vertebrae here, so here are these notochordial animals swimming around.
Where is Drepanaspis? Well, Drepanaspis isn't here, but a close relative is in this hundred-yearold rendering of what we think was going on in the Devonian, kind of hanging out, has a bony
head shield, wiggly tail and going kind of like Beeker, right, meep, meep, meep, don't eat me
big predators right here. The idea is we see lots of armor in the Devonian because there are
even bigger fishes then this swimming around that can crunch and chew and bite you. And we
also see very large and scary arthropods, so it's like the revenge of all of that lobster you've
eaten. There's a big group of arthropods that literally get, they are 6 feet tall. They are called
eurypterids and they have big claws and if you see a fossil of one you realize you are very glad
that they are extinct. Question back there.
>>: [inaudible] years old, is this really accurate?
>> John Long: No, no no [laughter]. I mean look, I don't want to be too harsh on artists, but if
we were actually doing a class and we weren’t recording this I would say here is an artist’s
misconception of the Devonian, but since we are recording this I won't say that. [laughter]. So
the point of this for our purposes is that it's a marine environment with predators. And then
we want to mimic specific features so we're back to Drepanaspis over here. Here is our Tadro
robot now that we're looking at it from above; it's got a stiff circular like head region and a
flapping propulsive tale. We're going to give our bio robot two different sensory systems, two
eyes and a lateral line. A lateral line is like an external ear. You have hair cells in your ears and
what's actually really cool about this is fishes don't have an inner ear the way we think about it
with accelerometers. We've got accelerometers in our head; did you know that? You guys
gotta, it's like tech; it's like wet tech. It's so cool. We know when we're doing this with our
head we've tiltometers, inclinometers I know they're called and we've got accelerometers as
well. We can do rotational stuff; that's the semicircular canals. Fish actually have the rotational
work, and it's all from a cog side point to subtract out self motion so that the world doesn't feel
like it's moving. We can move our heads and there's a really cool test of this. If you dare, and
you don't have contacts in, you know, we move our eyes and we subtract out the motion so the
room doesn't move. But if you will push on your eyeball, close your eye and push on your
eyeball and you move your eyeball with your finger, the room will move. So the extrinsic
muscles that move your eyeball around are exquisitely innervated and there is a computation
happening in your brain to subtract out that self motion. By the way, I don't have any special
insurance coverage so should you have just given yourself some kind of eye injury [laughter]
don't talk to me about that. So we have eyes. We have this lateral line which I just said is an
external ear. There are hair cells in there. And in our ear there is all this fluid that moves
around. Well evolutionarily, those hair cells were on the body of fishes and they are retained in
fishes. It's a very useful system. You can know in the dark if you are swimming next to
somebody. You can school in the dark. You can know if a predator is coming without having to
see that predator. Anything in the water produces a bow wave moving forward, so it's like
cologne that gets to the room before you do. The fish cologne is this bow wave that goes out
ahead of the critter. In case you want to know, these are photo resistors, so it's not object
forming lenses; it's simple light detection and we use an IR proximity detector as our proxy for
the lateral line there. We're going to focus for a minute here on what's happening down here.
We actually build a biomimetic backbone with variable numbers of vertebrae, so each one of
these light points here represents a vertebra that is in our back bone. The reason that variation
is important is that we natural selection works is like a menu. If there is only one thing on the
menu, you can only order one thing. If there are multiple things on the menu, you get to
choose, so again, it's that screening process that we talked about with evolution. So we need
to be able to do variable numbers. We want to be careful also to match the mechanical
properties of the vertebral column with mechanical properties of some living form that we
know the biomechanics of, and we want to match the anatomy as well. So let me tell you
about that process of building a biomimetic backbone. We start off with molds that allow us to
build a right circular cylinder that’s a hydrogel so here is a hydrogel here that's been crosslinked so it's wet, it's floppy and then to that hydrogel we slide on rings that are our vertebrae
and we can vary the number there. So a little more detailed, the hydrogels are made of 10%
gelatin. The cool thing about gelatin is it's actually made of collagen. Collagen is the primary
structural protein that is in skin, tendons, ligaments, very important, so we're using biological
material that also happens to taste very nice. Gelatin, Jell-O, I don't, nobody cooks anymore,
right? It used to be that you made this horrible stuff called Jell-O. It's fine if it's sweet, but like
my aunt she would put like cream cheese and tomato soup in the Jell-O and like serve it as a
side at dinner. It's like, are you kidding me? What are you trying to get us to do, lose weight
here? I mean what's wrong with you? But it is in fact a protein supplement because of all the
collagen that's in there. So this is hydrogel which we fix with a chemical compound called
glutaraldehyde so that it doesn't rot like Jell-O would. We have to keep them wet because they
will dry out, put on our ring centra and here is the variation. We can make the notochord. We
can vary vertebral columns from 0 to 11. We fix the length of the vertebral column here and
that's not something that we see in nature. It's not that vertebrates have always all been the
same size, but what we're doing here is simplifying our model and allowing, by varying the
number of vertebrae, the inter-vertebral joints change. This is essentially one long joint. For
those of you that have mechanical engineering background, it's a beam of infinite degrees of
freedom. When you start putting in joints, you start restricting those degrees of freedom and
bending. And by the way, we used sharks as our living model that we tried to match the
mechanical properties of. That's stuff I'm not going to tell you about, but we do biomechanical
experiments on isolated shark vertebral columns, so that we know, we're in at least the
biological ballpark of what's biologically feasible. So those are the biomimetic vertebral
columns and they act as the propulsers and allow us to make behavior, and so I want to explain
what I need by behavior before we go on here, because it's very important to what we do.
Remote control is not behavior. This is one of our robots that we call in the book Darwin's
Devices an evolutionary trekker. Yes, it's Star Trek homage, evolutionary trekker, ET for short,
which is another joke. [laughter]. I mean nerds have all the fun don't we, come on [laughter].
So robot Madeline which you see from the top down here is a 4 flipper evolutionary trekker
who in version 1 was remote control, so we could get her to do all sorts of wonderful
acrobatics, test our ideas about flippered locomotion, but I wouldn't say that she was
autonomous or smart. It's not really behavior that we're doing in the sense of autonomous
behavior that we're after. We have a human in the control loop, so it's just like the drones. So
how do we make behavior? What are we after? We go for autonomous behavior. Here's robot
Madeline version 2. Notice she doesn't have a tether coming off of here. She's got a little tail
and inside she's got a PC 104 and she's running her own routines. She's got sensory input. She
is using sonar. She's got a compass. She's got an altimeter. She's pinging with an altimeter.
She's got an internal accelerometer for kinesthetic logging as well, and now the robot is in
control. There's no human in the loop once we turn her on. So this is autonomous behavior
and decisions being made by the robot and this is what I mean when I'm talking about
recovering behavior, using robots to recover behavior. It's not remote control, oh, look, it can
swim. It's it can swim and navigate and do what we need to do hands-off, so when we run our
experiments we literally put the robots in our tank, turn the video camera on and then step
back and let the behavioral interaction unfold. So this is one of the features that we're after is
this autonomous behavior. So just to beat a dead horse here, without sensory input, that's the
equivalent of a windup toy. If the sensory input is from a human, we are talking about the
drones that we currently hear about that are flying in Afghanistan and Yemen last week, robot
Madeline version 1, and when the sensory input is on the robot and the robot is making the
decisions, that's what we call autonomous behavior and Tadro 4 who is going to be the topic of
the rest of the talk today is an autonomous fishlike robot that shows this kind of autonomous
behavior. This is something that we recover. How do we do it? How we get it to behave? I'm
going to diagram in broad strokes what the microcontroller that's on the robot, for those of you
that care, it's a, which when did we use here? MIT Handy Board is what we used for this. And
I'm going to call it loosely brained architecture, knowing that it's really not cortex or anything
like that, but it is doing something brain like. We know that in living fishes they have a very
clear way of responding to stimulus in the world around them. They will cruise along using a
nervous system that's called a chain of central pattern generators which we have central
pattern generators too. Any kind of rhythmic motion is a central pattern generator, so they will
just be a fish do, do, do, do do and if a predator comes, boom, they get the heck out of the way.
They fast start and they escape and within 60 milliseconds they've achieved sometimes
accelerations of up to 10 G, some of the highest accelerations that we see in vertebrates. So
we're actually mimicking that kind of neural control. In the case of light hitting one of the light
sources, we have the sensor that's always asking is there light, is there light, is there light? Our
equivalent and we are in the feeding mode of control, the feeding behavior. In the case where
there is a predator detected, that is the fleeing behavior here and if we didn't have some kind
of arbitration mechanism, you know, what do you do? Do you eat, do you escape? Well we
used Rodney Brooks's assumption hierarchy from robotics and we actually say you know what?
Fleeing is more important than feeding, and this is exactly how the fish brain works too. It
doesn't matter if you're eating if you are dead. I know it seems obvious but there you go. So
that's the basic architecture that we have for the nervous system control here. Put this all
together in a body that makes us capable and we talk about it from a cognitive science point of
view, embodied intelligence, intelligence that includes the body itself. We're actually going to
keep the brain constant and not evolve it and these animals, listen to me, it's my bias, and they
are not animals, John. They are models of animals. These robots will actually get better at
feeding and fleeing without us changing the brain and that's one of the cool surprises when we
allowed the hardware to evolve. So here are two of our Tadro 4s. This is a Tadro called--can
you think of a stupid name? Everybody needs a stupid name for a robot, so the stupid name for
this one is PreyRo. Tadro, prey, PreyRo. You can see it's biomimetic vertebral column here
with a few vertebrae here. This is the predator Tadro, Tadiator, hey, huh, that's pretty good.
Gladiator, Tadro, Tadiator, and here's a light source that's hung over the tank and the PreyRo is
trying to orbit around the light source here and you can see this predator is coming in for the
kill. There is actually no kill. It's just a bump. As you know whenever you actually do this stuff,
you don't want to wreck your machines, right. Even though it would be really cool, like train
wreck stuff when you were a kid. We don't let it happen. So there they are in action. What we
want to do now, we'll get back to some of the trials and what the evolutionary behaviors look
like. What I want to do is just remind you, we've talked about a bunch of features here; some
features we're keeping constant. We're not evolving the brain. We're also not evolving the
body shape and size and the behaviors themselves we're keeping constant. What are we going
to evolve here? We're going to evolve the number of vertebrae right down here and we're
going to evolve the shape of the tail and you can't see the shape of the tail looking down on this
thing, but fish, maybe you know eels have a pointed tail. Tunas have a splayed semi-lunate tail.
There is all this variation in caudal fin shape that is important in different kinds of propulsive
mechanisms. We're evolving the tail shape and then we are evolving the sensitivity of the
lateral line. All three of these features are things that we see changing a lot in early
vertebrates. I'm just going to be focusing on number of vertebrae, because we don't have time
to talk about all of that today. Suffice to say that there are interesting interactions among
those traits as well when we look at the evolution of a group of characters. So that's our
specific features. Next up we need to talk about genetics. How do we get there? We take all
of our evolving features and we assign letters to them, so number of vertebrae we're going to
call n genes. The shape of the tail tip we're going to call b genes so that's the span of the tail.
And then Zeta genes are for the sensitivity of the lateral line. We use many different kinds of
genes. They are simple quantitative characters and so we just use a number from 0 to 11, 0 to
15, and 0 to 50 for our different traits, so they're not Mendelian traits; it's their quantitative
traits that can just be combined arithmetically. You had a question?
>>: Yes. [inaudible] really need to change the…
>> John Long: [inaudible] was monotonic. Try me again.
>>: Yes. [inaudible] sensitive [inaudible] why would [inaudible]?
>> John Long: No. He's asking if we increase sensitivity wouldn't they automatically become
better. If you spend all your time escaping from the predator, you are never going to eat, so I'll
show you why that makes sense when we talk about actual ecologing in a moment. When I
show you the fitness function you'll see that there is a compromise between the different kinds
of behaviors. Evolution is a compromise; is my quick response to you there. It's not a bad
hypothesis right, but when you consider other things that the fish have to do, it's never a single
optimum that's being--evolution doesn't perfect I guess is what I want to say. Evolution
suffices. It gives you the best solution at the time and place, but it might not be some global
optimum at all. Was there another hand up over here?
>>: I was going to ask if you were going to talk about it, what your population size is and do you
have [inaudible]?
>> John Long: Yes. I will talk about it. It's coming up right here. I work with a mathematician
and so we did, the Tadro 3 world, we had three individuals and so he was always critical about
that. John, the mathematics are small; the numbers are going to overwhelm anything that
we're trying to do evolutionarily, so we really worked in Tadro 4 to come up with, we doubled
the population size, so I went to Rob and I said, Rob, we have a population size of six and he
started laughing. He's like, that's still not good enough, you know? Mathematics and small
numbers are really going to dominate, so in genetic systems we talk about the mathematics of
small numbers as effects like genetic drift you may have heard of. There are ways that
evolutionary biologists grapple with that. We know that these effects are present and that they
are part of the system, so we've got six individuals in a population. What I just want to show
you here is about the genetic cycle and what's going on. So pretend for a moment that each of
these colors represents a different genotype, or a different genome that each individual has.
And that's going to be true for the Tadros, anything, we've set up our genetics, so anything that
we can measure and see is directly coded. If you know anything about population genetics, we
would say there was a heritability of one. There is no fabricational noise in our system. And
what we do is, do the selection that we talked about earlier and we reward in our system the
top three performers in any generation are the ones that get to breed. So this sea green color
here, gets, now we're going to make eggs and sperm, so the top winner, the gold-medal winner
gets six egg and sperm that it puts into the gene pool. Our silver medal winner gets four egg
and sperm and our bronze medal winner gets two. So it's a simple ranking scheme here. The
differences of the pie slice, the differences in hue represent the fact that we've mutated, a
random mutational operator, the genome. For example, each one of these gametes is slightly
different in terms of the number that it's giving to the population. It resembles the parent, but
it's slightly different, so we are salting in variation into our system genetically, just like what
happens in real life. This is a genetic algorithm by the way if you are used to this kind of stuff
for your design processes. So in this gene pool we then randomly combine these haploid
gametes into combinations, pairs of genotypes that now give us the instructions for the new
genotype for the next generation. So the next generation, just like we did previously, it
resembles the individuals that actually made it through. So this was our top winner, and then
we had what yellow was in there, so we have a little bit of yellow stuff and some pink as well.
So that's our geodetic process and we have then changed the genetics of the population from
generation one to generation two. Now that's the population. Now we can apply selection. So
we apply selection by going back to this question. I mentioned that I would unpack this for you
and so here we go. Feeding and fleeing. Eat and avoid being eaten. There are a gazillion
different things that we could guess might be the selection pressure that drove the evolution of
early vertebrates. Why did I pick this one? Anybody?
>>: It's common for all vertebrates.
>> John Long: What do you mean it's common for all vertebrates?
>>: We all want to live or eat.
>> John Long: We all want to survive or eat?
>>: It's primal [laughter].
>> John Long: Yeah, right. It's basic stuff. Yep. You gotta do it. You gotta eat and you've got
to avoid being eaten. In fact, we have both fossil data and data from living species in
population that tell us that certainly predation is a very powerful selection pressure. Here is a
great fossil. Check this out. The Green River formation, 50 million years ago, you get lunch and
then you become a fossil. Isn't that like, you get whatever your famous food is; I have a war
with myself. Do I like donuts or sushi better; I don't know. It depends on the day. You eat the
sushi and then the building collapses on you, so you are fossil. So we know that predation is
important and studies on living species as I mentioned are very important for showing that. So
that's why we picked this very basic kind of idea. So for our PreyRo the feeding of PreyRo is
going to involve the searching for light. Light is life in biology, because that's where primary
productivity starts. Photosynthetic organisms convert the energy of sun into glucose.
Everything goes towards the light including in the ocean. And then we have Tadiator and
Tadiator is chasing PreyRo. Light is food for PreyRo and PreyRo is food for Tadiator. I need to
mention here we are not evolving the predator. We are just evolving PreyRo. We could have
evolved the Tadiators too, but we chose not to. It would be an obvious next sort of thing to do,
to co-evolve those two. Generally in biological systems predators are longer lived than pray
and so it's not a bad first approximation. So this is how we're going to apply selection. And I'm
going to show you a video now of one of our evolutionary trials, or a part of our evolutionary
trial and I need to orient you here. Here is PreyRo in the dark. Here is Tadiator in the dark.
There's a camera mounted 12 feet above the tank looking down. Here's the light source.
We've mounted a green bow light and a red stern light on each of the robots so we can track
them in the dark. What you're going to see happen here is PreyRo is going to make its way to
the light source and Tadiator is going to come around here looking for PreyRo and it's going to
sort of detect PreyRo and then sort of move in and sort of nail it in the blind spot only at the
last second right around here are you going to see PreyRo actually sense that Tadiator is
anywhere close. And remember, this is autonomous behavior. We are not remote controlling
this. Oh, by the way, this is a jerky video because it's down sampled from 30 frames per
second. They actually don't move fast enough for 30 frames per second to be anything but
soporific. You're like, if we had to analyze 30 frames per second, we would just be falling
asleep, so we downsample it. That's why it's a little bit jerky. So here we go. Here is PreyRo
moving in. Tadiator over here. Starting to come around, it still actually sort of foraging. It
hasn't really picked up PreyRo yet. PreyRo is going right towards the light now. It's picked up
PreyRo now. It starts to turn, so PreyRo is thinking, great, somebody put out doughnuts. I love
doughnuts. Goes over to the donut table and PreyRo comes up on the side and sneaks up in
the blind spot right here and just at the last moment, right there, the big tail sweep were
PreyRo goes oh shoot, those weren't doughnuts for everybody. And he gets nailed. Like
gotcha, so PreyRo gets it. So it's those kinds of interactions, that's what we'll call kind of the
behavioral ecology interaction that we can replicate. We ran trials for 3 minutes. We run
multiple trials for multiple robots in any particular generation. When we do a run of 10
generations we end up doing about 180 different kinds of tests. In the book I've got a, in figure
6.8, I've got this diagram that takes this point by point average position of each of the robots
and diagrams it for you, so you can see where they start. This is a different trial. You can see
when they get close and they interact and how, what the whole chase pattern looks like there
when they are after each other. It is very consistent behavior across our trials. And now this is
going to get I think back to your question. We have a judging system. This is how we build our
fitness function. For feeding, we are going to reward moving quickly, getting to that donut
table before anybody else does. Closeness to the donut table, I said sushi or donuts; I guess it's
donuts today for me. Closeness to the light, you know this from potlucks, that there's always
the person that is the first one in line for potlucks. Do you guys do potlucks here at Microsoft?
No. Everything's catered; I'm sorry. [laughter]. Like for those catered banquets that you have
every night, you know, the first one in line you can just predict it. And they stay right there.
They let you kind of sneak in and they are shoveling the food in. And so we reward that
behavior here. It's important evolutionarily. Fleeing. Keeping your distance from the predator
and peak acceleration so that when you initiate an escape that you do so and try to generate
high G forces and you actually initiate some escapes. So we reward all five of these features, so
it's a composite fitness function to your point here. This is why it's not simply keeping your
distance from the predator. Although we could have done that, but then we wouldn't be
testing for the combination of feeding and fleeing; it would have just been a kind of fleeing.
Yes?
>>: Has you look at, did it become important like conservation of energy, like…
>> John Long: You can't just do that. You can't just say conservation of energy; of course
everything comes back to conservation of energy.
>>: I'm thinking about like just quickly getting to the feeding, right you could burn a lot of
energy. If you just drift in, then you have more energy for when you want it. You know what
I'm saying.
>> John Long: I think I do.
>>: You want to be fat. You don't want to be thin and [inaudible] efficiency.
>> John Long: But you have, there's something to be said though that one strategy that might
evolve might be to go to the light source and just sit there. But then you're a sitting duck for
the predator. So there is, again, this idea that there is always a compromise with evolution.
You get there quickly, you grab some food, and you’re on the move, so you're harder to track
for the predator. But this is just one possible fitness function. You guys are both exploring the
fact that depending on what fitness function you use, this is really the evolutionary judging
that's applied and that's going to give you a different outcome in your evolutionary modeling.
So there's no doubt that what you guys are saying is correct. Yes?
>>: I'm wondering why you chose the specific attributes as opposed to making sure just that
they got enough light to live on and that they didn't get eaten.
>> John Long: Yeah, so, my college Rob Root who was critical about only six, he was always like
it should just be whether they have energy or not. This is to your point about energy
conservation. Why don't we just measure energy? If one of you can figure out how to do that,
let me know. We tried mounting solar panels on them and charging batteries and we said okay
then will just let them run down. So whoever is continuing to go, we'll just measure how long
they can keep going. And the winners are okay, you were able to harvest the energy. The
problem is the starting conditions. As you guys know every battery is different. Even the same
battery used again is now a different battery because they have histories, so how do you
control for that in a battery? Well so you say, you can start measuring things like current draw
and you can keep track of the watt hours that are being used and things like that. So you could
do that, right? And for just a simple set of robots we decided we will forgo that and save it for
the next round.
>>: That actually leads to my question, but what would be the difference if you just did all of
the simulations instead of actually doing it with the robots?
>> John Long: So what if we did it in simulation? We do do evolutionary experiments in
simulation. The problem is, so I work with an evolutionary computing specialist Chung Wei Lu
and this mathematician Rob, they do the physics engine. They build the physics engine and
they do the evolutionary algorithms and when we were putting together this initial study they
were saying that you know what John, we'll try to slow down so that you guys doing the robots
you're not too many worlds behind us when we do this, because we're going to go so quickly
with the evolutionary simulation that you're not going to know what hit you. The joke is they
are a world behind us in practice and that's because it's really hard to solve the Hydro elastic
equations that govern the behavior of the tail with fluid. There are multiple labs in the world
working on this and it involves Navier-Stokes equations which can be solved with
supercomputers, so everybody wants their approximation of Navier-Stokes, and we use the
Lighthill approximation when we do it but it has its own problems. So the nice thing about
robots is that robots can't violate the laws of physics, so even though they are difficult to build
and we can do fewer generations, they give us a better--we're not simulating the physics; we
are doing it. You get the physics for free. So we actually combine the two processes and the
world that we've done in simulation is the world we started on that. I didn't talk about it today,
but it's in the book. Just feeding, looking at if just feeding was sufficient to change the vertebral
column. So we have done that. And the great thing is you can do thousands of generations and
you can change the fitness function quickly and you can do all of this great stuff, and then it's
always a question of validating that simulation. Yes?
>>: Does having all of these variables in a physical [inaudible] better answer to the original
question as to whether or not a vertebral column helps with evolution. It seems like by
isolating just the number of vertebrae, that might actually just have a [inaudible] much more
simply and then the variables introduced by all of these other components, I mean maybe I
should wait for the results section [inaudible] you know if this actually helps [inaudible].
>> John Long: Let me go on a little bit. His general question is does it help to have the
complexity in the system? And it's always a balance. Our tendency is to apply the kiss
principle, keep it simple stupid and do the simplest stuff first, and this is actually Tadro 4 and
I'm not telling you about some of the earlier Tadro stuff, which are simpler here, so we do have
some reasons and I'll get into it, but we have another question here. Yes?
>>: So your colleagues who are doing the computer simulation have problems due to the
reality gap, right? There like…
>> John Long: Reality gap. I love that. Yeah.
>>: So how bad was it to overcome? I come up with basically an optimal solution that worked
on their computer and then I tried it on a robot and realized it kind of fell flat. How did that
work out?
>> John Long: The way it worked out is we got above a certain speed, we got a behavior that
was generally like what we're finding in the robots, that they could be self-propelled and
autonomous in the sense that they could find the light source, so we could, this was actually
Tadro3 and what we were doing was competing them altogether in the Tadro 3 world. I'm not
saying that simulations are always wrong, and I'm not saying that simulations always violate the
laws of physics, it's just, I don't know how much modeling you've done, but you always have the
ability to make your model better by tweaking the parameters, so you have to be really careful
about it. Our guys are really careful about it and that's why they are slower. They are trying to
do things like okay, we're actually going to go down to the level of the molecules now and we
are going to look at the flux of ions in the muscles and we have to come up with a really good
muscle model because muscle is a dynamic nonlinear force generating system and that needs
to be in there in order to be an accurate forced generating mechanism as we're doing our
coupled force system with the external fluids, so it's not just the elasticity; it's the fact that the
elasticity is then superimposed on this nonlinear force generating system. There is lots of stuff,
cool stuff to do on the simulation side.
>>: So are the trying to reduce the reality [inaudible] results [inaudible]?
>> John Long: Are they trying to reduce the reality…
>>: Like rather than trying to come up with a simple physics engine and…
>> John Long: No, no we have the simple ones. So the LIghthill’s actually fairly simple physics
engine, so what we're working on and immersed boundary layer method that Peskin developed
for heart flow and so where adapting that immersed boundary layer method and we have some
early simulations that we validate in fish swimming at steady speeds, because the steady
speeds are easier than any kind of maneuver or acceleration. So we are working on it. Yes?
>>: I have a question about the environment, how it simulated. Evolution depends a lot on an
environment and the predator you've used here may not actually make the predators that were
there during that time and…
>> John Long: How do you know?
>>: It's just the environment is too simple that…
>> John Long: It's a very simple environment. Yeah.
>>: So it looks like an experiment vacuum where you can say that vertebrae do help in moving
faster, but some of those many other physical things that would help in moving faster…
>> John Long: Electrical or mechanical engineer, which one are you?
>>: I am a computer engineer.
>> John Long: Computer engineer, well, doesn't quite fit my prediction. [laughter]. I work with
engineers all the time and engineers by and large can't stand doing simple stuff. The world is
always more complex than your model. Our problem is we want to understand the
mechanisms that are operating and so as you add complexity, you make it more difficult to
understand what's happening. So your point is correct. It is a very simple environment, right?
But it is more complex than anything that anybody has done so far, so allow us to take that first
step. So I take your point, which is correct, but, you know, somebody else will do the next step
which will be more complicated, and I can tell you that we tossed in a predator because before
we just had the animals competing amongst themselves and we thought that it was going to be
a more complex environment to have a predator in there. All right, we're almost to the end
here. I mentioned to you that the best three in any generation get to breed, and so we talked
about that and so the Tadros are playing the game of life and let's take a look now at the data
that we get. I'm going to show you over 10 generations what happens with just the number of
vertebrae and it's going to be an answer that you would have predicted, so here we go from
generation one, well actually over to 11. We did two different runs. These take weeks and
weeks and weeks to do, by the way, because we have to build all of these tails and things like
that. We have to analyze all the videos. It would be nice if we could automate that. We
haven't done that yet, some of those processes. The number of vertebrae, start at the same
spot. This is the mean of the population. The error bar represents the standard error as a
measure of variance and what you can see in both cases with a constant selection pressure the
number of vertebrae increases and then it stabilizes, which by the way is very similar to what
we found in simulation is that there is a, for that population, for the condition of those
swimmers, there tends to be a kind of simple landscape with maybe a single hill if we're
thinking about hillclimbing algorithms, a simple hill, a single hill when it comes to the number of
vertebrae. I'll refer you to the book for more complexity which I think begins to get to some of
the questions you are bringing about. When we look at the different characters together, we
actually employee analyses that are used in population and quantitative genetics. When we
look at variance covariance matrices and the evolution of the variance covariance matrix itself,
so we do try to take into account those various things. So what we know here is when Tadros
play the game of life, the population changes over generational time and this is evolution. So
people ask me, no, no, you are not really evolving your robots, are you? No, we really are. We
actually know evolution as a process. We've known it for centuries. That's how we do our
domestication of animals, right? That's how we do genetic algorithms for engineering and we
can apply it to robots. It really is evolution. Yes?
>>: So it looks like after a period of time you reach a steady state. Is the implication then to
continue evolution that you have changes in the environment that then push you again to
evolve?
>> John Long: I think this is maybe where you would expect maybe a predator to change in
response and so the fact that our predator was not co-evolving was a simplification to your
point that wasn't realistic. So then the next time we can go back and we can have the predator
actually evolve as well, so that we get into that evolutionary arms race that's going to cause
some of those creative changes. Yes?
>>: Did you try starting at a higher number of vertebrae?
>> John Long: Right. So you can see here. There are all sorts of things that you could do. We
could go down and start at the ancestral condition of no vertebrae. We could go up and start
low. I started us in the middle because I wanted to allow for the possibility that, I didn't want a
floor effect, right? I didn't want to force vertebrae to evolve. So if we're starting in the middle
of a Morpheus space here the number of vertebrae could actually decrease over time. But you
are right, that could be another parameter or starting position that we could change in the
system, and that's why the simulations, to your point, would be useful. And what we do in
simulation is we change the starting point. Okay, so we are back to why vertebrae? Back to our
cool critters from 500 million years ago and suddenly my computer has frozen. I guess
somebody has told me that John you've gone too long and you need to stop. So I will stop with
this blank screen that allows you to imagine all kinds of things and tell you that when we do this
kind of work we are trying to refute a hypothesis, so we were unable to refute the hypothesis
that selection pressure for enhanced feeding and fleeing is sufficient to drive the evolution of
vertebrae. We keep this selection pressure as one of many possible that could account for
what happened 500 million years ago. Is it the answer? No, we'll never know what the answer
is. It is one possible likely mechanism of what may have happened, and is anybody surprised?
Oh jeepers, these are fundamental things. Of course it's going to work, right? But you don't
know until you try it and what I can tell you is that in previous experiments that I talk about in
the book when we just do the feeding without the fleeing, we don't get the same pattern. In
fact, we see a decrease in the number of vertebrae, so in fact, it's not a given that just getting
out there and eating is going to do it for you. It's a combination of things that are going on in
the world. So thank you I will stop right there with my blank screen. [applause]. Do we have
time for questions? Okay. You guys have been asking questions along the way. You have been
a very good set of students today, thank you. Yes?
>>: So using these kind of composite animals [inaudible] animals, I think it's a pretty unique
environment where you can have other interactions that we are not seeing. Did you see any
like emergent properties arise of having say animals that had other controls like a sensitivity to
light or just the number of vertebrae is that you are testing?
>> John Long: So the question is did we see things that we didn't expect to happen because we
actually have these physically embodied robots working in front of us. I will tell you about one
that we think is really cool. Because they are in a tank of finite size, depending on how they
start moving around, they start generating currents. They start structuring their environment
and it's one of the really cool things from a cognitive science point of view about working in
water, is you entrain fluid as you move through it. Imagine if you could walk here and you are
suddenly like pulling the floor with you. And then it's like hey, you're kind of joining me. And
then it's like why are you walking with me? Well, you are pulling the floor. So we've started
doing experiments where we put 10 of these robots together in a tank, in this tank and we vary
them, 10 or 8 or 7 and we start looking if they get behavior as a group that's explainable by
some of these physical constraints that happen just by virtue of moving in that environment.
>>: And that's not something that the simulation might've modeled correctly?
>> John Long: I don't know. We haven't done that simulation that well. It's true that some of
the far field effects to model are a little bit more difficult than some of the near field things, so
it's an interesting question. Okay, I see that we have the same folks, that's great. You guys are
dialed in. Oh, here's somebody new, yes?
>>: Have you tried varying the surface area?
>> John Long: The question is have we tried varying the surface area and in previous
experiments we used slightly smaller tanks and it's absolutely true that the specific
environment matters, so in this particular set of trials we didn't do that.
>>: In the environment was it just straight water that you used or did you change say, the
calcium carbonate or all kinds of things that could have contributed to the [inaudible]
backbone?
>> John Long: So the question is how much did we change the fluid environment, because you
are right. The fluid does matter. We didn't mess around with that at all. People have done
experiments on living fishes where they play with the viscosity for example, and they may try to
tease apart the viscosity effects from the inertial effects in the fluid, and you can do that.
Temperature ends up being important for Reynolds number and things like that. Okay, go.
>>: So since you mentioned you are using like six or larger numbers, I was curious if you just
see schooling without having to program it in, like if it just already comes by simply be aware of
life and temperature and pray, it just happens?
>> John Long: I'm walking away from you because you have predicted what we're actually
trying to write up right now, is that in fact what we see happening is without any
communication among the members in the group, that by virtue of sharing a goal, which is to
orbit around a light source, there is what looks like a coordinated group swarm behavior, so we
are actually trying to write up the paper right now. Hopefully it gets submitted somewhere to
try to strip down what people are ignoring right now with swarm intelligence work, which is
that you don't necessarily need to have communication. You just need to have proximity and
shared goals, so right on. Okay.
>>: Linear sensor, how do you detect friend or foe with that scenario where you have multiple
fish in the tank?
>> John Long: The way we did these trials is it's just having the foe in there. So you are right;
that is a simplification. Everything is assumed to be foe, so that would be another piece of
complexity that we could add. How do you differentiate between a schoolmate as it were, and
a predator? Yep. That is totally valid. Yes?
>>: You mentioned when the genes were being [inaudible] that you added a little bit of noise
[inaudible] like a regularization process, how did you choose that? Did you have a feel about
how quickly you should [inaudible] be affected by changing that [inaudible]?
>> John Long: So the question is for our mutation operator, how much mutation did we put in
each time and did we know how much that was going to affect the? In previous worlds that
we've run we had too much of that going on, so we dialed it down. So it is in part A you can do
your simulations to get a sense of that, but then you actually have to run your experiments and
get a feel for it. All of this stuff like the sensitivity of the eyes to light, it's got to work for that
environment and so you kind of fiddle around to get it to sort of work. You can't have the
predators be too good right, because then you just have the predator going boom, boom,
boom, so you have to balance these out a little bit, so in fact the modeling, we run into the
same kinds of problems with modeling with the embodied robots in this regard as you do with
digital simulation as well. There is a sensitivity analysis or robustness or, what's the word,
where you see if your variables are actually changing anything in your model? What is that
process? It it's a sensitivity analysis is what that is. So you definitely have to do that and work
through that. That's a good point. How did we solve it? We used a poisson distribution for our
randomness and we… Pardon me?
>>: [inaudible] fish.
>> John Long: What?
>>: [inaudible] fish.
>> John Long: Exactly thank you. That was good. Yes, will get back to you in a minute.
>>: So in the video you showed the predator was swimming around in the dark; why didn't the
predator just hang out in the light? Because the predators…
>> John Long: I don't know. You'll have to ask the predators. So the question was why doesn't
the predator just hang around the light source. There are absolutely sit and wait predators who
do just that kind of thing. What I can say is that would be another model to run, is that you
have a predator that hangs out and so then you would have different strategies that could
evolve. So, yes, that is realistic. Sir?
>>: How did you come up with your concept of gold silver bronze in the weighting, when it
seems like just a minimum bar works well as far as appropriation?
>> John Long: The question is about how we do this ranking of the top three and that's again
having sort of done this before we had tried to assign a continuous spectrum of fitness to
individuals and what we found is that sometimes there wouldn't be enough differences among
individuals to allow there to be differential reproduction, so to ensure that we did a ranking. So
when we were doing it mathematically, and this is the mathematics of small numbers again,
right sort of lurking about. So we forced it, the system to give us this ranking, when in fact
there were differences, because sometimes we were covering up the differences before. It
turns out there is a field of evolutionary robotics; I call this field evolutionary bio robotics
because we're doing it specifically to test biological hypotheses. There is a field of evolutionary
robotics that has been promulgated by Nolfi and Floreano, two Italian engineers and they talk
about the various kinds of ways that you can do these mating algorithms as part of it. Each one
has an affect and a time that it's useful or not useful, so these are parameters, or situations that
have been explored, and I would recommend their book if you are interested, Evolutionary
Robotics. It is MIT Press. Yes? I'm not supposed to be selling other books though, sorry. Go
>>: [inaudible] question, I know you enumerated several different variables, how exactly did
you combine these all together to create a number for fitness and how complicated is that
equation? How did you decide on the weights and things?
>> John Long: Oh. I didn't know what you guys were going to be like today, so I gave you the
sort of technical talk, but not a really technical talk. I should've shown you the fitness function
and actually I have it somewhere if my computer wasn't frozen. What we do are Z scores and
so we sum Z scores for each of these things across, within a generation, we sum the Z scores.
So it's a relative fitness function, so it's only good for that generation.
>>: [inaudible] attributes?
>> John Long: Correct.
>>: There are no weights so they're all weighted evenly?
>> John Long: Well, the Z score does normalize those, right.
>>: Yeah, but I mean for each trait. So like, for example, for…
>> John Long: I'm laughing because mathematician my friend brings this up all the time. We
need to have coefficients in here and so we went, the Z score was the next step in the fitness
function. He would like to get us to weighting, but then the question is you can have any
weight in the world, which weights do you choose? So again, that's almost a better thing to
mess around with in simulation and so we just picked weights of one, how's that? Sir?
>>: In your paper are you going to definitely confine your inferences just to the fluid media that
was happening versus saying something like in biological evolution in fluid [inaudible] when you
leave that swarming affect appears, is it [inaudible] specific fluid that constituted the
experiment?
>> John Long: Are you offering to edit the paper? [laughter]. I don't know yet honestly
[laughter] how we're going to frame that.
>>: I can see the temptation. [laughter].
>> John Long: Yeah, no, that's a good point.
>>: So in some of your slides you had the gold, silver, bronze and the gold giving six pairs, four
and two, so you had 12 pairs. How did you pick the six and did that skew your results because
you're artificially taking six out of the multitude of combinations?
>> John Long: So this is a brand of question that you guys are asking and they are all correct.
So the question is does the way that we picked six, four, and two affect the results. You used
the word skew, I would rather use the more neutral, affect the results. It absolutely affects the
results. You would get a different result if you did eight, two and two, something like that and
that's, so all of these choices are sensitive to it. The neat thing is that we are starting to see the
more trials we do some robustness in some of the answers that we get. I was really happy
when we reran the trial, that we sort of replicated it that initial pattern. It was trying to keep all
of the conditions the same. I just wanted to see if we could replicate it. So whatever random
factors are in there to your point, the fact that we could replicate that in the two different trials
says that the randomness isn't just swamping the system or else we would expect very different
patterns to emerge
>>: [inaudible] came to 11 as the optimum [inaudible]?
>> John Long: Yep.
>> Amy Draves: Let's thank the speaker.
>> John Long: Okay. Thank you very much. [applause].