>> Sharad Agarwal: All right. Thank you for coming. We will get started. It's my pleasure to
have George Porter here. George is from UC San Diego. He got his PhD back in 2005>> George Porter: 2008.
>> Sharad Agarwal: 2008, sorry, from Berkeley; and he's spent the majority of his time since
then at UCSD. You know George from his data center work. He's been doing a bunch of work
with a bunch of fabulous students at UC San Diego. He's going to tell us about that. George
also took over pretty much all of the data center research from Amin Bada[phonetic] when he
left and went to Google. And George is also helping manage and run the center for Network
Systems at UCSD. Awesome. Thank you.
>> George Porter: All right. Thanks Sharad. And thank you all for the opportunity to come and
give you a talk today. It's a real pleasure to be here. I don't think I need to tell this audience
this, but please ask questions during my talk if you have any questions or comments.
All right. So my name is George Porter. I'm going to do be talking today about building
resource-efficient data-intensive applications. So it's incredible how much of our lives has
moved online from information gathering, entertainment, collaborative tools, healthcare,
medicine, government, effectively everything we do in some way has moved online; and each
of these applications is fundamentally driven by data. The quality of your user experience using
all of these sites depends on, in some way, the quantity of data that each of these applications
can process over. And it's not just that these things are driven by data. It's sort of like if you
think of the way Amazon uses data to build, say product recommendations, Spotify uses data to
build custom radio stations, Bing uses data to build personalized search. And it isn't just that
they're data driven but they’re data driven on a per user basis. So, for example, if we look at
the way Amazon works, when you visit the main sort of landing page on Amazon.com, the page
that gets generated is customized to you. And, in fact, each time you access this page there's
over 100 underlying applications that are doing things like collaborative filtering, consulting
with ad networks, your previous purchase histories, preferences, etc. And each of these
applications is driven by data. And so there's an enormous amount of data processing at IO
that goes on ahead of time, behind the scenes, before you arrive at these pages in order to
generate all the data that's needed to consult during that request so that this content can be
customized to you.
And so all of these different applications and data processing requirements have driven the
need for very large data center deployments. So in order to scale up to meet the needs of all of
these different users companies like Microsoft, Google, Facebook, and others have developed
these large data centers which are warehouse scale buildings housing sort of tens to hundreds
of thousands of servers, storage, networking gear, cooling power, etc.
And the result of these data centers has been an incredible amount of scalability. And so, for
example, Google served out 100 billion searches per month, Facebook has 1 billion active users,
Amazon has, actually is closer to about 200 million users today. And this incredible scalability
result has had to, has been arrived at in an incredibly short amount of time. So if you, actually
last month was the 25th anniversary of the web; and about 20 years ago the first sort of
mainstream web browser was released. And back then really everything was basically smallscale. So Google's first data center fit on a folding card table, and supposedly the first Facebook
server was a sort of run out of a dorm room type environment. And so those numbers that I
just quoted to you, 100 billion searches a month and 1 billion active users has had to been
developed in about 15 years and 10 years respectfully. And I think even at established
organizations and established companies you're going to see similar scaling results for anything
that’s kind of user-facing.
And so these organizations have really had to be driven by a relentless focus on scalability. And
so in order to close the gap between no users and effectively the Internet connected world's
population in order 10 years, everything that's been developed as had to focus on scalability.
So the data centers I described, the applications that run in those data centers, the
infrastructure that underlie all of those applications, the storage infrastructure, all of that is
really driven to be able to grow as fast as possible and effectively grow at any cost. And the
costs are incredible. I don't have to tell you that there's enormous capital expenses in terms of
building each of these data centers. And so the way you can think of that is of course that
every time you want to roll out a new application or every time you want to grow to a new set
of users you have to stamp out effectively one of these billion-dollar buildings.
But they’re also incredibly expensive to operate as well with sort of industry estimates at tens
of billions of kilowatts, hours, kind of industrywide. And yet, underlying all of these impressive
scalability results lies an enormous amount of inefficiency. So again, industry estimates about 6
to 12 percent of power actually gets translated into productive work. And the question
becomes a sort of why does that gap exist? Why is there all of this inefficiency in terms of
these data intensive applications? And one of the main sources of inefficiency really comes
down to IO. It’s really about input, output.
You can think of this in terms of IO bottlenecks between distributed applications and the
underlying data that lives on the storage layer underneath them, and there's also enormous
amounts of bottlenecks in between nodes sort of in a distributed cluster shuffling between
each other. And these bottlenecks result in servers that end up waiting for data. So this is
referred to as like a pipeline bubble or a pipeline stall wherein one node is waiting for another
node to complete or before it can make progress it has to wait till that data arrives from that
other node. And so this can cause these cascading performance failures wherein large-scale
systems end up spending a lot of time waiting on data.
This can also kind of manifest in terms of requiring a much larger compute in the storage
footprint than you would otherwise need if you just looked at the amount of processing needed
to make these applications work. And so what we really need to do is to focus on recapturing
IO efficiency, and this boils down to kind of a very simple application of Amdahl’s Law which is
we want to, if we look at data-intensive applications IO is really the bottleneck we need to
eliminate any unnecessary IO’s that we can, and for those IO’s that are necessary we want to
make sure they are as efficient as possible. So the rest of the talk I'm going to talk about this in
two different domains which I'll get to in a second.
So stepping back for a second, if we kind of look at the last 25 years we've really been focusing
on this goal of scale and achieving systems that are able to scale, and as we kind of pivot and
look towards the future it's important that we develop systems that are able to scale efficiently
in order to deal with growing user populations and growing data set sizes. And so the work that
I'm going to describe falls into these two domains. The first is on IO efficient data processing;
and I'm going to talk about some work that my students, myself, and my colleagues have
worked on in terms of building very large scale efficient sorting systems and using those to
build large scale data processing systems. And the second domain is on the node to node,
getting rid of node to node bottlenecks in the system, so focusing on IO efficient data-intensive
networking. And we've been looking at data center interconnect designs that rely on circuit
switching in addition to sort of more traditional packet switching models and combining those;
and we've been able to show that this approach will allow you to sort of cut the cost of your
network infrastructure by 2.8X and the power by 6X. And I'll go into these in a little bit in a
second. Yes.
>>: Are there some numbers to show IO is kind of a big problem in [inaudible]?
>> George Porter: So are there numbers, is there a quantitative way to show that IO is
important?
>>: The IO is, let’s say the big problem, one of the big problems in the [inaudible]. Because you
can imagine, for example, [inaudible] rapidly>> George Porter: Yeah. So that's a great question. There are. So when we sort of, I guess you
could look at this in two different ways, and one of them, like you said, is to make systems
somehow power proportional so the amount of power they draw matches the kind of resource
utilization that they are in. And there’s definitely work in that, and that's an important line of
work to do; but it seems like, at least in today's systems even if you, either things aren't very
power proportional, so there’s a lot of overhead, a lot of costs associated with keeping all the
machines running, and so you're really better off trying to drive as much throughput through
your system as you can and sort of max out all of your hardware. But that's a sort of a more
efficient I guess point in this space. And I hope that in kind of describing this work, I'm going to
go through some quantitative analysis of where some of these bottlenecks are, and I think I'll
answer your question that there is in fact, you can actually see where those bottlenecks are.
Okay. I want to talk first about IO efficient data processing, and I want to start by kind of
defining what I mean when I say data-intensive. I've used that term a couple of times already.
Over the last 30 years or so the definition of what makes something data-intensive has changed
quite a bit. So sort of the mid-80s it might be, let’s say 100 megabytes is a data-intensive job,
and today it's maybe 100 terabytes or even a petabyte. And so over this 30 year time span
there's been effectively 1 million fold increase in what we mean by data-intensive. And so over
that time period the types of applications that have been used to solve these jobs has changed
quite a bit. And so today if you talk about data-intensive computing a lot of times what you
mean is, for example, MapReduce. This is a representative example of the sort of a dataintensive framework for doing processing. And MapReduce is not actually not a new idea. If
you're a list programmer you've been using it for some time, but if you're not familiar with it I'll
really briefly describe it right now.
So in the MapReduce program you're given this input set of key value pairs and you start by
applying a user supplies Map function to each of these pairs. You then group the results of that
function application, by keying you sort each group, and finally you apply a user supply Reduce
function to each of these groups. So if we kind of zoom into what's going on in the
implementation of this programming model we see that the application of the Map function
and the application of the Reduce function, which I'm going to call Map tasks and Reduce tasks,
this is what's called embarrassingly parallel meaning that we can execute these functions
entirely node local without any network communication. So what we are really left with in
terms of the building a MapReduce framework is exactly this group I in this sort operation. And
in a lot of ways this is really the hard part about building these large scale systems because it's
almost exactly opposite of embarrassingly parallel. Generally speaking, data from each of these
nodes has to be shuffled, conveyed, and delivered to each of these destinations, and the
application of these functions really is bottlenecked based on all of these IO’s completing. And
so managing and dealing with all of this IO is an enormous challenge.
And we're not the first people to really identify that it’s a huge challenge. Actually, in the mid80s, that now late Jim Gray wanted to focus people's attention on the importance of the IO
subsystem when building data processing systems. So looking beyond sort of how many
floating point operations per second and starting to look at a holistic view of the system where
you see kind of IO's as a part of this. And so the way that he did this was actually really cool. So
he proposed a contest, a sorting contest, in which the idea was to see who could sort 100
megabytes of data the fastest. And this was great for two reasons. One of them is that we all
learn about sorting sort of freshman year of college, and so everyone involved in these industry
and academic efforts kind of had a good sense of what it meant to sort of data. The second
reason this was really cool is that across a variety of different data processing applications we
now have some representative benchmark or stand-in for what the IO performances, what the
resource efficiency of these applications is. So we can kind of do an apples to apples
comparison.
So obviously things have changed quite a bit since the 80s. So by the late 90s this had grown to
a terabyte, tera-sort record or contest which was won by the Berkeley NOW Project. And then
when we started our project in 2009 the idea was sort to sort 100 terabytes of data. And so
this was held by the Hadoop open source MapReduce Project that was being hosted at Yahoo.
So now we've got this really great benchmark of IO efficiency and I was talking about how
systems in practice are not efficient. And so now that we’ve got this benchmark let's see how
deployed systems do in practice. Yeah.
>>: So the thing about the [inaudible] is true with the sort, but aren't there many other jobs
where [inaudible] be part of the data since that is the bottleneck?
>> George Porter: Yeah. So that's a great question too. The kind of selectivity of that Map
operation, it can vary quite a bit. Obviously if you’re searching for data in a very large data set
and you're looking for kind of a needle in a haystack the output of that Map will be a small set
data set size. I guess the thing about sort is that the selectivity of that is one to one. So every
output record, so it's kind of a worst case from the point of view of IO performance. And so for
a lot of jobs that are low CPU to data item ratio it looks a lot like sort. And so that's why we've
been kind of focusing on it is because often times what you're doing is comparing items,
ranking items, things like that that don't require a lot of CPU per item but you do end up having
to convey all this information somewhere else.
>>: [inaudible] some modification for which how much is 1 to 1 [inaudible]?
>> George Porter: So that's a good point too. I'm trying to think, so there are, you can find
evidence of this in various published pieces of work and it varies depending on the
organization. I think the audience in this room would have a much better sense of what exactly
that CDF looks like than I would. So I would love to chat with you about it to the extent you can
talk about it.
Okay. So, great. Now we've got this benchmark of what we mean by resource efficiency and
let's see how well the deployed systems do in practice. So in 2010 these two researchers from
HP labs look at the results of that great sort contest and they look at all the winners, and the
way they analyze that data was as follows, what they did was they took the delivered
performance of each of those sorting systems and they compared it to the inherent capabilities
of the underlying hardware platform and they looked at that difference. And the results were
surprising and incredibly discouraging.
So on average, 94 percent of disc IO was idle, and about a third of CPU capacity was idle. And
this is among the winners. So this is definitely not good. And if we kind of look more
specifically at this 2009 Yahoo result they were able sort 100 terabytes of data with 3452 nodes
in about three hours, which is quite impressive, but if you actually kind of look at what each
node is contributing to that overall result what you see is that each of the discs in the system is
in some sense running at approximately 1 percent efficiency. So we are in this situation where
we are able to achieve these very impressive results via scalability but we are running at sort of
low efficiency. And remember, these data centers are incredibly expensive to build and
operate, and so the goal that we kind of like to set out is to be able to achieve the same data
set size result in the same amount of time but with effectively [inaudible] magnitude, fewer
resources.
>>: This measurement seems strange to me in the sense like I'm not sure [inaudible] your
hardware, especially in the workload, it seems like you always have some resource that is not a
bottleneck and [inaudible] utilized. So in that sense what is measure really showing us?
>> George Porter: So what we'd like to get is a fully perfectly balanced the system, right, where
all our resources are balanced with each other so that no one resource is, or I should say if we
were to reduce the amount of any one resource the entire system should slow down. That's
some goal that's implicit in the work that I'm describing. Now, as you point out, there's a lot of
heterogeneous jobs. And so in one set of workloads or one set of jobs you may end up with
one resource as the bottleneck and then some other particular type of job you might end up
with a different resource that’s the bottleneck. I do think though that focusing on storage as
the bottleneck is the tack that we've taken because a lot of systems really are storage IO
limited, and so using this as a way to start solving some of the system’s problems of getting the
storage IO up is one of the goals that we've had. I think that can be good.
>>: When you use storage IO limited you mean that they are a major source of inefficiency in
the sense that it is idling? Is that what you mean by that?
>> George Porter: So, yeah. Either they’re idling or they’re fully being utilized and there's not
enough discs to actually keep the workload, to keep all the CPU’s busy, or third, they're fully
being utilized but there are extra IO's being issued that aren't necessary. That’s another way
you can look at it. So I hope that in some way addresses your question.
>>: Another metric could be [inaudible] sort of [inaudible] storage or CPUs [inaudible] standard
servers scheme for your 100 terabytes for X [inaudible]. Would you still have similar problems?
Because that's [inaudible] hundred percent as long as I optimize for the metric under some
standard to keep them [inaudible].
>> George Porter: So I think what you're saying is sort of that you’ve sort of settled in a certain
sense on a binding of compute, memory, network, and storage, and then you're sort of
replicating this unit to the data set size that you need, and I think that's the way people build
real systems, right? You sort of provision a server model, you kind of scale that out and that's a
cluster that you build, and what I would say is that inherent in making that binding of compute,
storage, networking, and memory you already have an idea of what that balance is between
CPU and IO. Either IO to the network or IO to the storage. So to a certain extent you've already
kind of had this sense that there's some ratio and that's why you build a platform in a certain
way. And so when we started this work, and I didn't describe it in the slides here, but we knew
that the types of jobs we were going to be working on were low CPU to IO and so we wanted
two-use servers that has as many discs as possible in them. So at the time we were able to get
machines that had 16 discs, but we had 25 discs that would have been even better. Absolutely.
>>: [inaudible] you said that your goal is to build a systems [inaudible] balance [inaudible]. Any
one of these [inaudible] impacts the same performance. I don’t feel that's a good [inaudible]
because resources don’t cost equal. So suppose this IO was much cheaper than CPU, I would
be happy [inaudible] as long as CPU [inaudible]. So, for example, PennySort was essentially
geared towards students [inaudible]. So why don’t use that as a goal frame [inaudible] data at
the least possible price?
>> George Porter: We actually get to that. I think I'm going to revisit this because one of the
things we were focused on was per node efficiency and this sort contest, you mentioned
PennySort, there's also like a JouleSort contest that we entered as well, and I guess what we
found is that the reason that we've been focused so much on disc IO is that otherwise you end
up with all of this CPU in memory that's sort of waiting on basically on discs. And you need a lot
of discs whenever you have data-intensive applications just because of the capacity issue, and
so what we found was that by kind of driving up the efficiency of that resource we end up
getting energy efficiency. And I'll describe that in just a minute.
>>: [inaudible] define efficiency[inaudible].
>> George Porter: Yeah. So, again, when I talk about the evaluation I’ll mention this is a little
bit, the sorting contest is not just an absolute performance contest. There are these different
categories. And I get to this issue so there is work done per watt and there’s work done per
dollar or for penny and there's simply who can do the work the fastest. And those aren't
always, they don't always lead you to the same system design. A case in point of this that’s kind
of interesting is that for this eco-sort, for this JouleSort contest there’s kind of like two solutions
to this equation, and we've got one of the solutions and Dave Anderson’s group at CMU has the
other solution. And so we've been focusing on if we can build a system that could just handle
raw throughput what we end up with is even though our servers are 300 watts each and we've
got 10 gig networking and stuff you end up with a very highly efficient system. On the other
hand, you can focus on atoms and things like that and get a different solution. It just depends
on your assumptions. Absolutely. So I'll expose that in just a second.
And we've been talking a little bit about this balance issue and I just want to mention, not right
now, which is that to a certain extent this project is in the context of a larger set of work on
looking at trying to come up with the right balance of these different resources in terms of
addressing different types of jobs. And one aspect of balance has to do with the data itself, so
in an ideal world we'd like to divide our cluster up into these groups and we would like to
distribute our data in a very uniform way across each of these nodes, and then we've got a
variety of processing elements on each node, which I'm going to represent with these funnels,
which represent in some sense a CPU or disc type resource. And if we are able to kind of very
uniformly distribute all this data on all these resources then all of the data can be processed
uniformly and we don't end up with a lot of pipeline stalls because data gets generated as it’s
needed. And that's a very efficient way to design systems. Of course the real world is not as
kind to us as we would like it to be and data can be incredibly non-uniform.
So if you look at, for example census data, Seattle is going to have a lot more entries in it then
Driftwood, Texas or something like that. And so this imbalance can end up causing some of the
nodes to become bottlenecks which causes this cascading ripple effect. But resources are also
highly heterogeneous as well. So some discs are faster than others, but even if you bought all
exactly identical discs all with the same part numbers, you’re going to end up seeing this very
wide variance in delivered performance based on just the fact that you have so many of these
resources put into a single cluster. And one of the things is that this imbalance, one of the
effects of this imbalance is you end up with not just an inefficient IO but actually wasted IO’s.
So the thing that's interesting about sorting is that any external, there's this well-known lower
bound which is that any external sorting algorithm requires that you read and write each data
item at least twice in the worst case, and what we say is that any system that actually meets
that lower bound has this two IO property; and that's one of the goals that we set for ourselves
when we started this work.
Now, that imbalance that I just showed on the previous slide can result in extra reads and
writes that aren’t necessary, and this is due to what’s called intermediate data materialization,
meaning that you don't have enough memory to, for example, keep your entire working set in
DRAM and so you end up issuing reads and writes to process that data iteratively. And that's
what we mentioned before about you can have your discs running at 100 percent sort of load
even though you end up with extra IO’s that are cutting that effective load down to something
like one percent. So it's not that in that Yahoo cluster the discs were only being operated at
one percent of the time, it’s just that one percent of their performance got delivered into the
aggregate performance of the system. And just like the data can cause imbalance the
imbalanced discs can lead to this exact same problem for the reason I just mentioned.
So we’d like to restore balance; and we do that in two ways, statically, before the job begins,
and then at runtime. So we are going to borrow techniques from the database community to
sample our data to get a sense of where these partition boundaries are going to be. So this is
research, these are things that databases do all the time, and that's how we figure out what
these partition boundaries are. The key thing is that at runtime we still need to impose bounds
because even if our data has been statically allocated correctly into these partitions, because
the on disc layout of the data can have non-uniform data in it we have to handle that at
runtime, and that's what I'm going to describe in this part of the talk right now.
Okay. So we built a 2-IO storage system called TritonSort that we presented in NSDI 2011, and
it is structured as follows. So instead of many fine-grained tasks processing the data in a divide
and conquer approach we have two phases of operation. So in the first phase, the distribution
phase, we divide our data up into these partitions based on those samples, and then we read all
of our data in parallel and we assign each data item to one of these partitions, and in phase 1
we send it over the network to the node it belongs to and we store it in one of these on-disc
partitions. So at the end of phase one all the data is on the right node and it’s in the right
partition, but each of these partitions isn't sorted. And so in phase 2, in parallel across the
cluster, we read in each of these partitions, sort it in memory, and write it back out again.
We've also sized our partitions so that we can ensure that each of these are going to fit into
memory.
So to see that in action we start by reading a buffer data offer input disc. We have a process
that's assigning it to these different partitions and copying it into in-memory buffers designed
for the different destination nodes, and then when these buffers get full we have some code
that sends them over the network to the node they belong to. Then on the receiving side as
data arrives we append it to a variety of these on-disc partitions. And just to give you a sense
of the numbers involved here, we've got eight output discs on our machine and each disc has
about 300-400 of these partitions on it. So you can think of it as 300-400 on-disc files that store
the data. In phase 2 we're going to read one of these unsorted partitions into memory, sort it,
and write it out.
So the bulk of that pipeline, if the details are in our paper, and it’s actually pretty
straightforward to implement, and the real complexity of that system is exactly this partition
appending module because we have to ensure that we're writing out of data to these discs in
large enough batches that the discs deliver good performance. And so I'll just describe very
briefly how we do that now.
So this module has given us input on the left, a buffer of these key value pairs, and on the right
we've got a set of discs, each of them is holding a couple hundred of these partitions and
there's a thread before each one of these that’s ready to write out the data to the disc.
So the first thing we did was we implemented the kind of most straightforward way of doing
this which is to sort of scan through this buffer of key value pairs and rely on the operating
system to deliver and manage the IO for us. So we just issued writes or scattered writes or sort
of the fancier writes. The result was that the system performed, we had low performance. And
the reason for that was really just due to the fact that there wasn't enough buffering handled
automatically by the OS to ensure that the writes getting delivered to these discs were
sufficiently large to run near their sequential speed.
So what we did was we sort of scrounged up as much of the memory as we possibly could,
about 80 percent of the memory on each node which was 20 gigabytes, and we managed all
the buffering ourselves. So we divided this memory up across all are different partitions, we
copied data into these partitions, and when they get full we write them out to disc. But I
mentioned that there's this non-uniformity of the input data. And so what ends up happening
is these partitions are either really hot or very cold. And so taken as a whole our memory was
not particularly well-utilized. So the result of that meant that our writes were not particularly
very large.
So what we ended up doing was we building a load balancer that ran at runtime in front of our
disc and it works as follows, we took that same 20 gigabytes of memory and now we divided it
up into 2 million little ten kilobyte buffers that we stick in a memory pool, and so as data starts
arriving from the network we basically are going to copy it into these little buffers and stick it
into a data structure here. And the way this data structure is organized is that we have a row
for each of our 2500 or so partitions, and each of these rows which corresponds to one
partition, what we're going to do is we grab a buffer, put our data in there, then we add it to a
list or a chain of these buffers per partition. And the nice thing about this data structure is that
as partition’s popularity varies during a run even on short timescales, we can extend some of
these chains to become longer and then some of the less popular partitions have shorter chains
but none of our memory is actually being dedicated that isn’t actively being used.
In parallel with this there is a process that's constantly scanning this data searcher and it’s
looking for the longest length chain which represents at that instant in time the largest write
that we can issue to the disc at a given time. So what we do is once we find this we pull it out
of this table, we send it off to this thread which is going to write it off to the appropriate on-disc
partition, and then it’s going to take all of these buffers, add them back into the pool, and this is
going to be the back pressure mechanism that we use to push back pressure back to the
producing side of this pipeline.
Now this handles non-uniformity in the actual input data, but I mentioned resources can be
non-uniform as well. And so imagine that we have a couple discs that are slower than other
discs. The way, this same data structure actually handles the problem without requiring any
modifications. So this process is constantly scanning for these chains is only actually looking for
the subset of chains that could be issued at that given time. So what that means is if you have a
slower disc, the chains behind it are going to build up and you're going to end up issuing larger
writes to them which is going to a little bit help mitigate this non-uniformity in some way.
Okay. So we’ve looked a little bit of how we handle IO in TritonSort, and I want to talk about
our evaluation. So when we began the project, in terms of the hundred terabyte GraySort, the
Hadoop, MapReduce Project had been able to sort data .578 terabytes per minute, and then
with TritonSort in 2010, 11, and 12 we were able to sort at .725 terabytes per minute using a
cluster of 52 nodes and it was just based on these issues that I just talked about. Now, as is the
case in any particular type of contest, eventually your record is taken back again. And at least
in terms of 100 terabyte GraySort, Hadoop, was able to run at 1.42 terabytes per minute on
2200 nodes in 2013 with a much more recent version of Hadoop. There's some other
categories I didn't describe here like the Indy[phonetic] benchmark which you guys took back
from us. And so we're working I guess to see if we can to retake that.
But I mentioned that it's not just raw performance that we were really interested in. The point
of this project was to focus on resource efficiency. And the community identified that as a
really important metric as well, and so in 2010 they added a JouleSort category which exactly
captures this eco-efficiency and we were able to capture that in 2011, 12, but also maintain it in
2013. And the reason for that is because even though we were beat out in terms of absolute
performance, if you just look at the quotient of the amount of work we did times the number of
nodes we are able to push about two orders of magnitude more throughput through each of
our servers. So that's why we were able to keep that kind of performance.
Now I want to briefly mention fault tolerance at this point because any system you build has to
be fault-tolerant. I don't have a lot of time to go into the details here, but what I would say is
that the fault tolerance approach that you adapt really depends on the failure assumptions that
you have in the system. So if failures are really common you want a pessimistic approach to
fault tolerance, and if failures are rare you want an optimistic approach.
The [inaudible] pipeline I described relies on very aggressive pipelining. So we are not
materializing data at all whereas a something like Hadoop requires materializing intermediate
data in the common case. And so what I would say is that we looked at some published results,
for example from Google in 2010, and kind of what you see is that at sort of a 10,000 node
cluster size you're seeing failures like every two minutes or something like that and so it's really
important that you need to, you actually want tasks to be able to, you want jobs to be able to
survive individual faults. So materializing job state probably a really good idea. We actually
talked to the people at Cloudera and the average Hadoop cluster sizes are order of something
like 30-200 nodes. And even adding an order of magnitude to this number of nodes you end up
with failure rates in this sort of double digit hours, hundreds of hour’s timeframe.
And so here what we are going to argue is that it's actually okay to do job level fault tolerance
where you re-execute jobs on failure as long as the performance improvement you get by
running without fault tolerance is high enough to overcome the occasional job re-execution.
There's no hard and fast rule here, and I think there's sort of a dividing line depending on
exactly what you're failing rule is but this is something that we’ve been looking at. In the future
work that we've been, actually Alex Ratherson[phonetic] which is the lead student on the Trace
and Sort Project, is we've been looking at taking trace data and selectively re-executing parts of
the pipeline that have failed or that parts of the pipeline that depend on a failure to mitigate
the cost of re-executing the sort. Maybe we can talk about that off-line. So, yeah.
>>: How much of your gains in terms of I think efficiency come from resource balance versus
fault tolerance? Like compared to your competitors?
>> George Porter: Well, if we look at Hadoop, for example, there is one extra materialization
that happens after the Map task which we get rid of. But actually there are three different
places in the pipeline where materialization happens, and two of those places are actually just
due to data skew. So what I would say is effectively a third of the IO we get rid of is due to fault
tolerance and two thirds is due to resource imbalance and data set imbalance.
>>: So why doesn’t Hadoop just do this? I mean are there practical reasons why they don't do it
because their developers are jerks?
>> George Porter: No, no, no, no, no.
>>: I guess what I’m wondering is, so you show like all these very nice results and so you would
think the Hadoop guys would say we're just going to nom, nom that, get in there and>> George Porter: The Hadoop people are great. I've worked, so I have a patch that’s been in
there since 2008 for this and part of the problem is that, it’s actually concerning, in a sense they
are doing this. So there has been a huge move to these in-memory, completely in-memory
data processing sort of applications. And in these cases you're getting rid of effectively all the
data materialization at the expense of more cost. But I think that, so that's kind of one
extreme, sort of getting rid of all the data in materialization. I think though that if you look at
actually where Hadoop’s going with projects like Tez, which is like this data flow thing, you're
seeing that they're giving users much more control over what materializations they do. So right
now you kind of fit into the MapReduce model, but if you're doing something like an iterative
job there is now a lot of support for being able to control exactly when those materializations
happen. So I think that's happening.
>>: But just to follow up on this, you take your own skew that you build, you run one job at a
time, so you're really optimized to [inaudible] disc. But if you now have a different cluster with
many jobs of different sizes, maybe not a large sort, how, could this directly be applied, would
you lose some efficiency, would we have to tweak things to make it really work?
>> George Porter: Yeah. So this is great because at a very high level this first part of the talk we
are giving up statistical multiplexing, and what we're doing is we are focusing on individual task
efficiency. So rather than sort of taking lots of different tasks that have heterogeneous
requirements, putting them on a system at a time, and then co-scheduling them, we are
dedicating resources without using [inaudible]. And one thing that I would say is that if you've
got a petabyte cluster and you have a bunch of 10 terabyte jobs you have a lot of opportunities
through doing [inaudible]. But if you're resource constrained, let's say you are a research
group, let's say you're a startup, you are working in biology or something like that, you may
want to solve petabyte scale jobs but you only have the resources to run petabyte scale
clusters. And so the question of is [inaudible] than not [inaudible] is an interesting question
when you’re not resource-constrained. But if you are resource-constrained you can't even ask
that question. So I think that it's good to focus on things like job and cluster scheduling,
obviously if you're doing [inaudible], but it doesn't hurt to also look at can you actually kind of
pull in as much efficiency as you can out of individual tasks when you don't have the
opportunity to do [inaudible].
>>: But it’s not clear how you're actually [inaudible].
>> George Porter: Yeah. I think>>: If you can get better>> George Porter: Yeah. So just to answer this real quick, if you assume that the compute in
the storage are co-located with each other you don't have a ton of choice in that matter. But if
you separate them, for example like with the Blizzard work from NSDI last week where you
actually get to sort of logically separate storage and compute, you could imagine dedicating a
very tightly connected set of machines to storage, getting full bandwidth of that storage, and
then when that job’s done now maybe an order of magnitude more computers can access that
same amount of storage. So you get light binding on that. I guess it depends.
I don't want to run too long, so I'm going to move a little bit forward. Sorting isn’t all we care
about, we also care about the data processing. So we built this system called Themis, I'm sorry,
so we implemented all of these different applications here and what I want to show, I don't
have time to go into the details, what you see on this graph is the performance of our
MapReduce system where the Y axis is throughput in terms of megabytes per second per disc in
each of our phase, the X axis is all the different jobs that we've done and different levels of
skew. What you see is for the vast majority of these jobs we’ve pushed our storage
performance similar to our record settings to our performance. Now I said almost all, there is
this Cloudburst example where in the first phase of Cloudburst it is IO bound and so we see that
performance improvement, but the second phase isn't IO bound which kind of exposes this
point we talked about at the beginning of the talk which is that when you get rid of one
bottleneck you can oftentimes push it somewhere else. And a place that you typically push it is
the network.
Now for us this wasn’t a huge problem because Cisco donated one of these big data center
switches to our group and we only had 52 nodes and we had enough ports to give full bisection
bandwidth, but if you’ve got 150, 2000 nodes that’s not such an easy problem. And that kind of
leads to the second part of my talk which is focusing on the data center interconnect.
So just like applications have changed, the network has changed quite a bit as well, and we've
seen this enormous growth in terms of data rates. So the types of networks that people have
built to address this growth and performance has changed quite a bit. My first exposure to
these kind of networks was in 1994 when I worked at ISP[phonetic] in Houston and the way
that a lot of networks were built then and even today is as these tree type structures. And
you've got nodes along the bottom and then you have the layers of switching getting
increasingly powerful as you move towards the root. And if you imagine let's say one hundred
thousand data center at 10 gigabits a second that's a petabyte of aggregate bandwidth
demands.
Now the real problem is that you simply can't, from a technology point of view, buy core
switches that are fast enough to actually handle all of this bandwidth, and so researchers have
actually looked back to the 1940s and taken ideas from Bell Labs kind of in the 40s and 50s and
adapted them for data center designs. And so this is what is called a folded-Clos multi-rooted
tree, and some version of this is deployed in many types of data centers, and this was proposed
in [inaudible] 2008. The key thing here is that we don't have these really powerful switches in
the middle of the network. Instead, if we have is 10 gigabyte a second servers, all of the
switches on our network are 10 gigabits a second. And we get all of that bandwidth by relying
on multi-pathing to deliver an aggregate amount of bandwidth.
So if you have enough links in the network you can load balance and distribute traffic
appropriately to get a higher amount of aggregate bandwidth, and what you've done is traded
off impossible to by switches with a very challenging but solvable with money problem of
adding lots of links into this network. So a 64,000 node data center has about 200,000 links in
it. And these links are incredibly expensive to kind of deal with installing them and managing
them. They’re also very expensive in terms of cost. And as we move from 10 to 40 to 100
gigabits a second of Ethernet they're going to get disproportionately more expensive. And the
real reason for that is that we can't rely on the copper cables that we know and love and we
have to move to fiber optics. The reason for that is because of a property of copper cables
called the copper skin effect, which roughly speaking, says that the faster the data rate of the
cable the shorter it has to be.
So at a gigabit you can buy spools of hundred meters worth of Ethernet. The second you go to
10 gigabits you're down to order 10 meters, and at 100 gigabits you’re talking about a couple of
meters in length. And remember, these are warehouse scale buildings and so we have to
overcome this length limitation in some way. And so the way people do that is to rely on optics
which don’t have this copper skin effect. So you can send very high-bandwidth, you can create
very high bandwidth links at very long lengths this way
Now the problem with optics is that you have to have some way to convert between the
electrical signals that the switch understands and the optical signals inside the fiber and so you
need a transceiver at either end of this cable that has a laser, a photo receiver in it, which is
used to make this conversion. And these transceivers are sort of ballpark 100 dollars, maybe 10
watts at 100 gigabits a second. And I know that several of you would have much better precise
information about the pricing. This is sort of based on external information and papers and
stuff like that. But the point is that they are not trivial in terms of cost, and you need two of
these for each of these say 200,000 cables. So it adds up to a lot of money and a lot of power.
To look at the implications of that, if we imagine 100 gigabit a second multi-rooted tree here
and we look at the path from a given source to a given destination, what we see is that the
packets transiting this path are constantly being converted to and from optics at each of these
switch hops, at each layer of switching from the leaf up to the core and then from the core back
to the leaf. So the implication of this is that for every device attached to the network there’s
roughly speaking 4-8 of these transceivers in the network kind of conveying the traffic for that
device. And so at 100,000 nodes that's like a megawatt of power and tens of millions of dollars
or more.
And if we step back from that for a second I think it's worthwhile asking why are we doing all of
this packet switching? Why are we doing that? And what I would say is that these folded-Clos
networks, the service model they actually provide is that they allow you to make a different and
a unique fording decision for each packet that you send in the network. But that service model
is, I'm going to argue, too strong for many data centers; and as a result there’s a gap between
the service model we are providing and the service model we could potentially provide, and
this gap is how we are going to get resource efficiency.
And to say what I mean in more specificity, there's a lot of locality in data centers. And actually
Microsoft has been great about publishing actual results from your networks. And this is a
picture that's reproduced from one of these papers. And it’s a little bit dated at the moment,
but it does show kind of the rack to rack traffic at an incident time. And although the details
change over time, what you can see is that a bulk of the traffic is going to a relatively small
number of output ports. So there's a certain amount of spatial locality in these systems.
But if we kind of look bottom up as well, we see that there's a lot of temporal locality as well.
And so my student, Rishi Kapor[phonetic], published a paper in Conex last year where he
looked at a 10 gigabit server, and he deployed a variety of kind of representative applications
on top of it, and he measured the packets leaving that server at micro-second timescales. What
you saw was that because of all the batching the happens in applications, in system calls, in the
operating system kernel, in the NIC hardware, all of that sort of buffering and batching ends up
translating into tens of hundreds of packets that are correlated in nature. So you tend to see
when servers send large amounts of data from one place to another they tend to do it in these
kind of correlated bursts. And so the key idea behind this second part of the work is to use this
temporal and spatial locality to build cost-effective networks by adopting circuit switching in
addition to packet switching.
So if you're not familiar with circuit switching I'll give you a very brief example. This shows a
one input port, two output port circuit switch; and you can think of this as just an empty box
that has some mirrors inside of it. And light enters the input port, it reflects off these mirrors,
and it leaves an output port. If you want to make a circuit switching decision there are tiny
motors underneath these mirrors that can move them and this changes the angle of reflection
and causes the light to leave out of a different port.
Now this is great. You don't need any transceivers. We're not doing this conversion. And it
supports effectively unlimited bandwidth, meaning as we go from 10 to 40 to 100 gigabits a
second this technology doesn't have to be changed. But circuit switching is an incredibly
different service model than packet switching. And so this isn't just a drop in replacement for
your packet switches, you really have to kind of rethink the entire network stack. And to give
you kind of an example of that I'm going to talk about one aspect of the service model that has
changed which is called the reconfiguration delay.
So the reconfiguration delay Delta is the amount of time it takes to change the input to output
mapping of that circuit switch. And it's, roughly speaking, the time to move those little mirrors.
And this determines how much locality you need for these circuit switches to be applicable in
your network. If Delta is really large it means that it's incredibly expensive to change the circuit
mapping, there's a very high overhead, and so you only ever really want to support very large,
highly stable, long-live collections that in the networking world we call elephant flows.
Now on the other hand, if Delta is really small, you could very rapidly reassign circuits on short
timescales and so you can support very highly bursting unpredictable traffic called mice flows.
Now I want to point out Delta is not fundamental, it’s a technology-dependent parameter. It
depends on how you build these mirrors and some other aspects of the technology. But it is a
very important parameter that determines this mixture of circuits to packets. Yeah.
>>: Did you also account for the delay for mirroring the traffic? You have a mirror of something
and make the decision to switch [inaudible]?
>> George Porter: Yes.
>>: The mirroring time can be much longer than this.
>> George Porter: Yes, yes, yes. This is great. So this is talking about the actual data plane.
Now you're talking about a control plane issue about how do you figure out what signals to
send. I'll describe that in a minute, but we started by an observe, analyze, act approach. That
became too slow, and so we ended up with a proactive approach that I'll describe in the talk.
This is an exact problem we dealt with.
Okay. So we have the sense that the majority of the traffic has locality, but not all of it. So the
way you can think about that pictorially is as follows. Imagine that we have been, in a network
with N connected devices there's N squared possible connections. So we could rank order all of
those N squared connections by the amount of traffic per connection; and because of there’s
locality, the picture looks, roughly speaking, like this where a bulk of the traffic is in a relatively
small number of these connections. And so this leads to what we’re saying is a hybrid design
where we are going to rely on both circuit switching and packet switching. So what we like to
do is take the head of this distribution and send it over these circuit switches, and then this
relatively long tail that has a lot of different connections but not a lot of bandwidth we’re going
to send over a less expensive, lower speed packet switch network. And it’s exactly this Delta
value that determines this mixture of packets and circuits.
Okay. So I told you Delta is technology-dependent, and when we started the project we had to
get some sense of what that value was, so we obtained an optical circuit switch that was
developed in the late 90s for the Telecom industry and we characterized it in our lab. And what
we found was that the Delta value is about 30 milliseconds, and what this means is that you
need to keep circuits up for hundreds of milliseconds to seconds or longer to amortize that
overhead. And so it’s really only appropriate for very highly stable, long-live traffic, and the
place you see long-live traffic in the network, generally speaking, at the core we have a lot of
aggregation. And so this led to the development of the Helios Project which was presented in
[inaudible] 2010, and I showed you that multi-rooted tree before. Imagine we're just focusing
on the core switching layer only and we're going to get rid of most of the packet switches in
that switching layer and we're going to replace them with a smaller number of these circuit
switches; and let’s abstract the rest of the network away into these things we call pods, which
are roughly speaking, about 1000 servers or so. So the servers, the links, the switches, these
are in these pods.
The idea behind the Helios Project was to support this type of an environment. And so what we
do is we start by sending traffic over our packet switches and then there's a process that's
looking for these elephant flows, and whenever it finds one of these elephant flows it’s going to
add updated flow roles down here to move it over to the circuit switch. So this is what you
were talking about, about the time it takes to do that. Because we've got 30 milliseconds it’s
like all the time in the world. So it's not a particularly big deal. So we end up, there's details in
the paper about exactly how we do that, but finding those elephant flows, moving them over to
here, we can achieve all of that in this kind of tens of milliseconds time bound. Yeah.
>>: I feel like there's something of a tension between the two halves of your talk which is that
the first part of the talk is about efficiency, which by definition will try to use all things equally,
and it seems like it's exactly the opposite of what you want [inaudible].
>> George Porter: Yeah, yeah. So there's a commonality which is that this is also getting rid of
[inaudible], if you want to think of it that way; but to get to your specific point, I think that one
of the main things here is that what we're doing here is we are in a sense trying to build a
network that matches the average case utilization even though that average case is rapidly
changing and the set of nodes that need high bandwidth is also rapidly changing. Today, you're
really only option is to effectively provision for the worst case. And so in this way we are able
to, as applications, as their communication patterns change we are able to migrate things. But
even in that TritonSort case, if you think about the two phases, in phase 1 we were fully utilizing
the network. But in phase two we actually had no network at all. This model would allow us to
take that resource away and move it to another instance that does need the network. So if we
were over to overlap phase ones and phase twos in two different clusters we could actually
share the network between the two.
All right. So the result of the Helios Project was that we were able to get rid of one of these
transceivers in the core which doesn't seem like a big deal, but it actually represents a very
large cost complexity and power savings because when we looked at this original network we
were looking at 10 gigabit networks and so all of these pods we could entirely interconnect
them internally with electrical cabling. You only really needed optics for these core switch
layers and that's where all the transceivers were. But as we want to start moving to 100
gigabits a second we’re not going to be able to make that assumption anymore because we’re
going to have to start putting optics inside of these pods just because of the length limitation.
So we need to start pushing circuit switching closer to the host into these pods, and that led us
to our second project which is called Mordia. And I want to say that the thing is that if we were
able to aggregate over 1000 servers with this 3-D MEMS technology that was relatively slow, if
we want to put circuit switching down to the host we need a technology that's, roughly
speaking, 1000 times faster. And so we identified such a technology which is a different kind of
circuit switch device called binary MEMS. It's a little bit different, and what I will say is that the
advantage of this binary MEMS technology is it's very fast. It's about two microseconds
seconds, three orders of magnitude faster, but the downside about it is it's not scalable. You
can only buy switches that are maybe four, eight ports in size. Yeah.
>>: [inaudible]. Are you making a strong assumption where predictability [inaudible]. There is
[inaudible] keep informing [inaudible] mislabeled mice and elephant you're getting a
[inaudible]?
>> George Porter: Yeah. Implicit in this particular design is the idea that what we need,
because Delta is so high, we are only going to consider traffic for this that's stable for over a
second. So these little bursts, too fast because it’s 30 milliseconds just to assign a circuit. And
so for this the only kind of traffic that we can actually support this is traffic that’s going to be
stable for order a second or longer. Now with this technology, because we can reconfigure it in
two microseconds, we actually only need traffic that’s stable for about 100 microseconds to
assign a circuit to here. And so that's one of the major issues here is that what we mean by
circuit traffic or locality traffic depends on the Delta value. And so now we can actually support
the burst of a given server using circuits. But, in getting to this point, we can't do things like
measure assign the flow roles, etc. because we only have a couple microseconds to do that. So
we have to be proactive, and that’s what this project deals with. Yeah.
>>: [inaudible] of the flow is very, like a few microseconds. That's going to [inaudible] overhead
on the switches.
>> George Porter: So we don't, so instead of estimating traffic based on the looking at packet
counters and switches what we actually do is we measure demand by looking at hosts. So we
actually look at the hosts, what the send buffers and the hosts are to see what their demand is
going to be in the future. And we can>>: And you can collect the statistics from the host and that could also take longer than
[inaudible]?
>> George Porter: So it takes order microseconds, tens of microseconds let's say, to have the
host send this data out. And I don't have time to get to it in the talk, and I actually don't have
slides to this, but it turns out that in this Mordia design you can think of a pair of ToRs, two
ToRs connected to each other, so they only actually have exchange information on a pairwise
basis. So you don't have to collect this globally, do a decision, and send it back out again.
Maybe we can talk about that off-line.
>>: Sure. So [inaudible]>> George Porter: I have a couple more minutes to go and then we can get to all these
questions. And in fact, there's details in the paper about how we built Mordia, and I'm actually
going to skip over how we did it, but the key idea is that these switches are able to support
multiple wavelengths of light. And by making a copy of the light by tapping some of the light
out of a fiber we can actually replicate the signals across multiple of these stations and each of
these switches can make orthogonal switching decisions. And this is the key idea that we used
to scale up our design. And by adding a variety of these switches into one or more of these ring
networks you're able to support order 600 ToRs or so with this design.
Okay. Now we built Mordia over at UCSD using these switches that we got from this start up,
and we connected them to our servers, and we measured the switching time of the compose
system, and it is in fact these two microsecond results. We are able to keep that fast switching
time even though we scaled up to, in this case, 24 ports. The key idea here is that with two
microsecond switch time we only need 100 microseconds of stability for something to be
circuit-friendly, and that means we can actually support the traffic of a single server. And this
led to our most recent project, which we just presented last Wednesday at NSDI over across the
water there, which is exactly building a top-erect switch, hybrid switch that speaks to both
circuits and packets at the ToR layer.
And this is the premise of that project. So it's very simple. If we've got a 10 gigabit packet
switch network and we overlay a 100 gigabit circuit-switch network into our data center, these
effectively can be put together to build a 100 gigabit packet switch for data center workloads,
meaning that if there is sufficient temporal and spatial locality defined by 100 microseconds
worth of bursts then you can deliver service model akin to this extremely expensive network
using two much less expensive and lower cost network technologies. And we built REACToR
using this as our, we built an eight port REACToR prototype and we hooked it up to our Mordia
network, and I just want to show you one of the graphs from that paper and then I'll sort of
conclude.
So the idea behind REACToR is to give the performance akin to a 100 gigabit packet switch but
using circuit switching. So what we did is we deployed eight nodes and we have seven of the
nodes sending data to the eighth node, and this is the view from the eighth node. So what it's
seeing is that x-axis is time in seconds and the Y axis is throughput, and what you're seeing is
each of the incoming flows is relatively stable, very nicely fair, very uniform, looking like kind of
a very smooth packet switch network, but in reality if we zoom into this at microsecond
timescales what we see is we are actually rapidly multiplexing small bursts of data from all of
these different hosts and delivering them to the end host. The key idea is that we are able to
rapidly multiplex that link fast enough that the transport protocol and the OS doesn't realize
anything’s going on. And the analogy here is to process scheduling where you can just schedule
stuff fast enough nobody notices that they don't have access to that resource.
So the key idea behind this line of work has been to focus on the predominant sources of cost
and power in these networks which is very surprisingly and sort of counter-intuitively cabling
costs and transceiver costs. And so we built a variety of projects that have dropped that
number down to close to one transceiver per host at 100 gigabits a second, and what's nice is
that as we go even behind 100 gigabits a second this same approach should be able to apply as
well. And I just briefly want to mention that what I've described thus far has been taking what
our existing building blocks and prototyping them to build new types of networks, but we are
also wanting to complete that loop to build new building blocks as well. And so we started with
commercial technology, we built some prototype technology, and now we are interested in
building novel devices designed for data center environments. So we are doing that inside of
NSF.
There’s an engineering research center called CIAN which is about 12 institutions and about 30
PIs most of which are photonics and physicists. So they’re photonics people and physicists, and
the idea is that we are taking all of these organizations and actually going back to a lot of the
building blocks we used were designed for the Telecom industry. So what we're doing is we’re
kind of unwinding that decision tree back to the assumptions that were made in building optical
devices, and instead of targeting them towards the Telecom world what we are now doing is
targeting them towards the data center world which has a very different set of assumptions. So
there are people in the center that are building new devices, and this is one example that sort
of has come together in the last month and then I'll conclude.
So I mentioned in the Mordia design that we have these binary MEM switches in the network
that are making switching decisions, and they're basing those decisions on the wavelength of
light that is entering them. Now that's kind of an expensive design actually because these
switches are very expensive. Now what you can do instead is there’s researchers in the center
that have been working on silicon photonics tunable lasers, which sounds very sci-fi but this
actually not much more expensive than building regular transceivers today, but the cool thing
about this is that by changing the frequency at the source you actually don't need those
switches. You can build an entirely passive interconnect network and simply have the sources
change the frequencies that they transmit on. And so, through the center we were able to take
that research and send it to a fab and build it onto a chip and then partner with this company in
Berkeley to package that in an SOP module that we can then reinserted into the Mordia switch.
And so that happened about three weeks ago, and so that's kind of what the future work is for
this project which is to lower the cost of that network.
Okay. So in summary, it's important to sort of pivot away from the question of scaling to
scaling in a resource efficient way, and I’ve talked a little bit about IO efficient data processing,
data-intensive networking. And before I conclude I want to acknowledge the incredible
students that I’ve had the opportunity to work with that have been driving much of this
research. And with that I’d like to thank you for your time and open the floor to questions.
Yeah.
>>: It's good that you got the switching [inaudible] so small, but we’ve also found that if you're
willing to deal with extra traffic hops that you could kind of get away with the need for a lot of
switching by [inaudible] a lot of effort.
>> George Porter: Yeah.
>>: Have you used any of that for>> George Porter: Yeah, yeah, yeah. So this is things like OSA and other projects that rely on
kind of overlay or multi-hop. It's another degree of freedom. So all of the designs that I’ve
talked about have been effectively either zero hop or one hop depending on how you look at it.
And the second that you can start forwarding traffic through intermediaries, maybe using
things like RDMA or something, it gives you this additional degree of freedom where now you
have a scheduling decision which is, do I wait until I get a circuit assigned to me, or do I sort of
send data to some intermediate point which can then send it on my behalf? And it's actually
quite interesting because nothing that we’ve talked about precludes that, it's just that that
hasn’t been something that we've been focusing on.
>>: You mentioned a few microseconds. That doesn't make sense to really use it for them?
>> George Porter: So this is an interesting point because we were servicing in some sense
fundamental here. I think that there's an interesting sweet spot at the kind of 0.1 microsecond
timescale because at 10 gigabits of packets, you know, 1 .2 microseconds or so, and in order to
use circuit switching, I should say we've moved, we haven't looked at optical packet switching
at all. We've really been focusing on circuit switching because it's something that’s practical
and that can be deployed. And so what you really need is a burst size. And if you're talking
about 10 or 40 or even 100 gigabits a second, one microsecond gives you reasonable burst size
that seem to match well with what servers are able to generate. If we were to push that
switching speed much lower we would end up kind of hitting up to the packet boundary and
building effectively a packet switch which isn’t really what we want to do; and if we pushed it to
be a higher value we would end up needing so much burstyness[phonetic] that it would be very
difficult to get servers to be able to generate that. So it happens to be a particularly attractive
spot at sort of 0.1 microsecond or so. So we haven't looked at trying to make switching faster.
There are technologies that could do that. There's SOA, switches and stuff that operate in
nanoseconds, but from our point of view we lose our circuit switching benefit from adopting
those techniques.
>>: So why aren't people building just like, almost like RF-inspired optical switches. Like RF
doesn't need any switching. You could just have the transceiver side they just try things.
>> George Porter: So you mean like wireless?
>>: Yeah.
>> George Porter: Well, I would say, I mean the work on say 60 gigahertz wireless in the data
center in a sense needs switching because you're either physically moving things or you're
somehow choosing a different target to transmit to I guess.
>>: So that’s [inaudible]. I was thinking like an optical switch that instead of like, I think part of
the thing it seems to me like this scheduling control plane idea is [inaudible]. It has complexity.
>> George Porter: Yes.
>>: And that complexity is coming from the fact that you have this kind of switching schedule
like no matter how fast it still [inaudible].
>> George Porter: Yes. That's right.
>>: That doesn't show up like in cellular domain or whatnot. Like my phone can't [inaudible] at
any time without actually having like a global schedule across all transverse>> George Porter: Well, you channel mitigation, so that's one thing. Imagine this network, so
with optics it's like a wireless network where every terminal’s a hidden terminal. So think of it
that way. Imagine a wireless network where every node was hidden. The tunable laser idea
that I just talked about, one of the things that's interesting is that you end up in a situation
where if two, let's say ToRs choose the same frequency to transmit on that interference will
cause data loss. And so what we're going to do to solve that is a very simple kind of brute force
approach which is to create a registry service wherein you opportunistically acquire a channel
and then you register with the service that you get the channel, and if two devices end up
conflicting on that one of them will win and the others will stop sending, and we can bound the
amount of time given the amount of, like the slot time, the contention time can be small. This
is just techniques from the wireless domain, and then we can code over that to make use of, so
we don't lose data. So this is very much like a carrier sense approach, just in the optical
domain. My understanding, and I'm not a physicist or an optics person really I’m a computer
scientist, but my understanding from talking to the optics people, because asked a lot about
like their CDMA, for example, could we do OFDMA or whatever, and my understanding is it's
not a very promising approach. But that's a little bit out of my domain.
>>: Just to be clear what he's saying, couldn’t you choose a different lambda? That's what he's
talking about.
>> George Porter: Oh yeah, that's what we do. We pick>>: [inaudible] do it that way?
>> George Porter: Yeah. So our approach is basically that you give it a list of in preference
order of the lambdas you want and it tells you back you can have three and you can have five or
whatever.
>>: But presumably, there are fewer lambdas than N squared.
>> George Porter: Oh yeah, yeah. Absolutely. So the number of lambdas>>: [inaudible].
>> George Porter: Current technology, the number of lambdas is 0 of 100. Think of it that way.
>>: [inaudible]?
>>: Not for N squared but just [inaudible].
>> George Porter: If you want to support say 600 ToRs, let's say you'd need to add a space
switching into that as well. That's what we're going to do. Because can't really fit more
lambdas into a fiber, but you can have multiple fibers effectively, and what you can do is
choose which fiber you’re going to send data on and then which frequency you're going to send
within that fiber. And that's where I was mentioning that point about how we don't need a
global scheduling decision because we do need a global scheduler that decides what ToRs are
connected to which other ToRs, but once those two ToRs are connected to each other the
specific frequencies that are being used can be negotiated on a pairwise basis which is a much
simpler problem to deal with. Yeah.
>>: [inaudible] hybrid [inaudible]? [inaudible]
>> George Porter: Yeah. So this is actually a great point. There's two things I can say about
that. One is that this stuff is super reliable because it’s built by the Telecom industry; it's very
expensive. So they had these goals of like 10 to the minus 18th bit error rates. The reason that
this stuff is expensive, and I mentioned before that we’re winding back that decision tree to
build devices that are [inaudible] to data centers, it’s just we don't need to 10 to the negative
18th error rate. We have other ways of dealing with errors that if they made the network
substrate significantly cheaper and more integrated it might be a good trade-off.
So I what I would say is the Telecom stuff we are using has been incredibly reliable. And then
there was another point that I was going to say about, so, yeah. The technology we’ve used has
been fine so far. But in terms of handling failures in this model, because of this 100 channels
per fiber, if we moved to a multi-fiber or multi-ring type network the way you can think of that
is that I'm now spreading traffic over multiple rings; and so if one of these rings were to fail in
some way that proportionally reduces my bandwidth by N minus one. And so there’s kind of
nice failure recovery model there as well. So you don't lose, it's all or nothing. You can imagine
degrading our service based on the number of failures that you get. So if a laser fails or
something like that you might lose a fifth of your bandwidth or something like that. But you
don't lose 100 percent of your bandwidth.
>>: Have you also seen [inaudible] same thing. [inaudible]?
>> George Porter: I don't have any concrete quantitative data on failure rates between the two
of them but what I would say is that the optical stuff, especially the stuff in the Telecom
industry, is a really reliable. So there are failures that occur, of course, but it hasn't been a big
problem for us. Now one of the things that I will say is that these tunable lasers, the way they
actually tune is that they have very small heaters next to the laser that change the temperature
because the frequency that they transmit depends on the temperature. Now the flip side of
that is in the data center if you changes in temperature you have to have some way to stabilize
that temperature. And so one of the areas that's really, that a lot of the optics people are
working on is on devices that don't require active cooling and stabilization of temperature. But,
yeah. We didn’t run into that problem at all. So we have a sort of a data center like the size of
this room at UCSD and it’s got some chillers and stuff in it, and we've not experienced any
failures in three years. But we are very small scale.
>>: For example, like vibration effects [inaudible]?
>> George Porter: No. Nothing like that. We also, just as a side point, for the sorting record we
have 1000 spinning discs. We never saw any vibration effects at all in the time we used them.
Yeah.
>>: You measure the scale effect copper?
>> George Porter: Yes.
>>: At lower frequencies they’re fixed or making error. So replacing a single cable by a grade of
[inaudible] cable [inaudible] or does that get too ridiculous as the frequency?
>> George Porter: My understanding is that it does. And again, you all are the experts here,
but the labor involved in the plugging all these cables in is very nontrivial. It’s gotten to the
point where organizations like Google are building these robots to build cable assemblies so
they can plug one layer of switching into another. And the second you say I want to take 10 big
fat cables and put 500 of them together into a bundle this big, I think people don't like that.
>>: All right. Let's thanks George once again.
>> George Porter: Thank you.