>> Shaz Qadeer: Okay. It's my great pleasure to welcome David Walker in our midst. He's a professor at Princeton University. He works in the area of programming languages, type systems, and other related topics. He is very well known in that community. He did early work on type assembly language with some colleagues of his, and for that work he got 10-year retrospective award for the best POPL 1998 paper, and he has received other awards in addition to that. Lately, he has turned his attention to making network programming simpler and applying programming languages ideas to it, and today he's going to tell us about that work.
>> David Walker: Thanks. Right. So networking from the perspective of a programming language researcher. And perhaps though, a better title is what I'm really going to tell you about is NetCore which is a new domain specific language that we are developing to help people to find network configurations in a modular and high-level way. So my research in this area starts with a quick story back in 2009, I often bends theoretical, and when one does that one sometimes begins to run out of grant money; and when I do, I typically go to one of my more practical colleagues and ask what's up so that we can join forces and solve some real problems and get some real money. So my question to Jen Rexford, a well-known networking professor, is what's networking? I had no idea at the time. And she said oh, it's fantastic and you don't need to know the lambda calculus. At the same time we hired a fantastic post doc,
Nate Foster, to come help us. He comes from the programming languages community as well.
He was disappointed like I was about the lack of lambda calculus, but we decided to go for it nonetheless. So Jen told us that what happens is you have a couple computers, and they need to talk to one another, so you might link them up with collection of Internet, Ethernet switches.
And these Ethernet switches have two parts. One part you might call the control plane, and that part runs a bunch of algorithms to decide how packets should be forwarded, and the second part you might call the data plane, that's typically some hardware that is very good at forwarding packets very, very fast. So it implements the directives of the control plane.
All right, sounds easy. Is that it? Well, actually no. There's a little bit more. Still no lambda calculus though. Darn. All right. So we might want to add to our network of computers a whole bunch of servers, and they'll be connected by routers. The routers are in some ways similar to the Ethernet switches, but a little different. The Ethernet switches are designed for plug-and-play; the routers are structured and optimized so they scale to a much greater degree.
So they have different control planes, but the underlying forwarding hardware is fairly similar.
In addition, in order to connect these two groups of computers, we might need a bridge that connects the Ethernet islands to the IP core, and we might want to add a load balancer that will help balance the load between our backend servers, and, of course, there are other ISPs, and those other ISPs need to be connected to our network using inter-domain routers; and if we want to make sure that none of the malicious traffic from the other guys gets to us we may

want to add a firewall box; and everyone has a Windows phone these days, and so in order to connect them to your network, you’re going to need some wireless basestations-
>>: [inaudible] short leash from [inaudible].
>> David Walker: I see. What are you talking about? I don't know. And then of course, in order to build those Windows phone users we'll need some more middleboxes, we might need some boxes to intercept the packets or do de-packet inspection for security. So things are getting pretty complicated. At this point I cannot keep track of all the three letter and four letter acronyms. The lambda calculus is much similar. There's only three primitives. I'm very disappointed with Jen. But, she says there is a new way to do things. And the idea is that this mess of different kinds of boxes is largely a control plane issue. So each color represents some different set of control plane protocols and algorithms for communicating between the boxes.
However, the data planes, the underneath, the gray part underneath is largely, fairly similar.
And so some recent ideas that were actually, had their intellectual genesis with Jen Rexford and were later expanded on by some people at Stanford, was to decouple the data plane from the control plane and move all of those different control plane algorithms onto one centralized computer or collection of computers. And so we move from a situation where you have all these independent boxes that are sold to you by different vendors who are communicating along fixed protocols, move from that situation to a situation which you’re programming applications on top of a centralized platform that has a global view of the network. And this new idea has gained a tremendous amount of prominence in the networking community recently, and the name of the protocol that allows this generic controller platform to communicate with basically just the underlying data plane hardware is OpenFlow.
So OpenFlow provides you a very simple abstraction of one of these boxes which I'll call a switch from now on. And this abstraction is one that we’ll call a flow table. So a flow table is a list of rules. Each rule has a pattern that can match some bits in a packet and also an action.
And actions can be things like drop a packet, forwarded packet at a particular port, or send the packet to the controller. So in addition, these various different rules will have priorities, and the highest priority rule will match the packet. And there are some counters that are associated with each rule. So every time a packet matches a rule you’ll increment a counter that will say the number of packets that match the rule and maybe the number of bytes that have been processed.
So at a high-level, the operations on these flow tables, sort of the messages that can be sent from the controller platform to one of these switches, are incredibly simple. It's just install a rule, uninstall a rule, or maybe ask for the values of certain counters so that you can do some measurement of the load on the network. So overall, I have to say that the networking people

did an amazing job at developing this abstraction. It's incredibly simple. It's so simple that someone who knows literally nothing about networking, like myself, can learn about this abstraction, understand it in a couple days, and begin to think about how it works. It's also very general, so we implement many of the boxes that were shown on the previous slides very efficiently.
Okay. So the programming language’s perspective on this situation is that we've got now a new piece of critical infrastructure that we can begin to think about how to program. It's an interesting kind of heterogeneous distributed system where you have a bunch of dumb boxes that have very simple functionality in terms of forwarding packets, and you have a centralized application for controlling those dumb boxes. It's a simple, clean, narrow interface. We can think of it as a new kind of assembly language that needs some domain specific abstractions on top. Moreover, there's lots of research to be done here. We still need all of the application logic that we needed from before, we still need things like firewall, security, we need loadbalancing, we need routing. And in order to compose those applications, we need to think about how to program this kind of system in a modular way, when you think about how to compose the different modules, when you think about abstraction and information hiding. It's also the case that one of these systems, they need to be up, of course, 24-7. So we need to design new correct-by-construction abstractions, we need to apply things that we know about program analysis to detect defects, think about new ways to do testing and verification, and think about fault tolerance.
In addition, a network can be shared by many different entities. Think of a data center in which you have multiple different clients, a data center such as Amazon, ZZ2 or something like that when you think about security in such a situation. There are also resource constraints on all of these switches in terms of the number of rules that they can, you can place on one of the switches and the bandwidth that the switches can process. So there's optimization problems for compilers people to handle. Last, there's, although this interface started quite simple, clean, and narrow, it's becoming more complex as people tried to export the various different kinds of underlying hardware that they have. It's a little bit more complicated. And that just leads us to more compilation problems. Lastly, from the programming languages perspective area important component is the money component. This is drawing a lot of attention from industry. Google has recently deployed this kind of technology for linking up their data centers.
Jen Rexford and my partners are both interested from the perspective of multiple startups being sold for hundreds of millions or billions of dollars. Anyway, my grant funding issues are now fixed.
Okay. So with that as a backdrop, I wanted to tell you that basically some of the ideas that we've been developing at Princeton and also with Nate Foster, who is now our colleague at

Cornell University, in the frenetic project. So in particular, in this talk I'll tell you little bit about
NetCore, which is this domain specific language for putting together networking configurations; and if I have time, I might talk a little bit about technology for, shifting from one configuration of a network to another, so the idea of how to perform a consistent global update of a network.
And, as always, a lot of the, most of the key research is being done by our students; and we have a number of students, three in particular, that I've been really instrumental in pushing this project forward, Chris Monsanto, Mark Reitblatt, and Cole Schlesinger. I should also make special, pay special attention to Arjun Guha, who is a postdoc that's been working with Nate
Foster and has done incredible things implementing our system; Rod Harrison was a student at
Princeton and now he's gone into West Point; Josh Reich is another postdoc. Any others are faculty helping.
Okay. So the big problem that we wanted to tackle with NetCore was really the problem of modularity. We still need all the functionality of the old networks, all these different kinds of components, the only way to introduce this is through modular designing. But, when we first started looking at OpenFlow, we noticed that it's incredibly anti-modular. So if you just take the simplest situation where you have a repeater module, so I just want to program this switch here to take all the packets coming inport one and send them out port two and program it so that all the packets coming inport two go out port one; if at the same time as running that program I get some different programmer to write a monitoring module, perhaps one that monitors web traffic, we'll see that attempting to do those tasks separately just doesn't work in
OpenFlow. So in particular, my repeater module, what it might do is it might issue two messages to say, I want to install a rule that for all of the packets on inport one, forward them out port two; and I want to install a second rule that says for all the packets coming inport two, forward them inport one.
Now what this guy would like to do is this guy would like to issue a query for the counters associated with certain packets. So in particular, you might want to say hey, for all the packets coming in on port one, whose dstport is 80 indicating web traffic, tell me how many packets there were. Unfortunately, this isn't a query that you can issue an OpenFlow. You can only ask for the counters that are associated with preinstalled rules. So if this guy installed these two rules, I could only ask for all of the packets going from port one to port two, or all the packets from port to port one, but not this more refined information about say, all the packets with port 80.
So basically, the repeater rules are too coarse-grained for the desired monitoring, and if I wanted to try and install new moderating rules, that would potentially clobber the repeater actions. So to get around this, what we propose to do is develop a higher level abstraction.
Yep.

>>: So in the OpenFlow rules you showed earlier, you had some priorities. Multiple rules matched as only the highest priority one fire or do all of them fire in that?
>> David Walker: Only in the higher highest priority one fires. So, yeah. So one other thing I wanted to say is that this isn't just a theoretical problem, but the problem actually shows up when you start to write code; and here's some example code that you don't really have to read from one of the early controller platforms that really just to mirrors the underlying OpenFlow interface. And again, if one tried to write a repeater separately from a web monitor and then put them together, when you put them together you might get some code that looks like this; and the take away basically is the blue lines come from the repeater and the red lines come from the web monitor and then there's also little bit of green in the middle that shows some new things that you had to add to disambiguate between the two different policies. So the take away is: OpenFlow is desperately anti-modular. You can't easily put together a billing service in a repeater, a firewall in a switch, a load balancer and a router, or all kinds of other little bits of functionality that you'd like to program independently.
So our attempt to design a language that did allow you to put these different configurations together is called NetCore, and it’s really evolved over time. And we're still trying to understand it, and what we're doing, and certainly there's elements of continuing work in progress in this talk that I'd be happy to discuss with you more afterwards. But the basic idea with NetCore is that rather than thinking about installing concrete rules on switches, programmers are going to specify what the network does at a higher level abstraction using pure functions. So we're not going to worry about how those functions implemented, we're just going to specify the function that implements the switch. And each function we can think of as a function that takes a located packet, which means a packet at a particular location in the network, and generates a set of located packets.
So if we want to drop a particular kind of packet, we’re going to generate the empty set. If we want to broadcast a particular kind of packet, we might take a packet in and generate a set of several different packets. So here's a little picture that just shows a packet, its location might be at a switch and a port, and the network is going to implement a function F, and in implementing that function, it takes the packet as an input and produces a new packet with possibly modified fields at a new location. So in addition to having real concrete locations that are places in the network, we also have some abstract locations in our little language. And I'm going to call these abstract locations buckets. The idea is that you have a place, a data structure if you will, that sits on your controller platform. And one kind of function you can have is a function that takes a packet in the network and moves it into one of your buckets.
Once you have one or more packets inside a bucket you can ask some questions about the contents of that bucket. You can ask how many packets are there in that bucket? Or how

many bytes have been transferred into that bucket? Or perhaps, what are the packet contents in that bucket?
So there's different kinds of buckets, and for different kinds of buckets you can ask different questions, and they're going to be implemented in different ways. A bucket that just allows counting will be implemented just as the counters on the switches. A bucket that allows you to ask what the packet contents will actually be implemented by installing rules on your switches that forward packets to the controller. Whoops. Yep.
>>: [inaudible] the packets in the buckets are actually consumed? Or-
>> David Walker: What do you mean by consumed?
>>: So this, if you have those functions and you use the one packet [inaudible] and then you don't [inaudible] consumed you don’t have a packet anymore, right? It has been processed.
>> David Walker: Yeah, so that's right. So we're going to, at every step we will apply our function F that takes a packet and it generates a new set of packets in different places, and then we’ll basically use the function that represents the topology of the network to push the packet along the topology, and then we’ll apply F again and etc. So yeah, you can think about it as if a packet gets consumed, but you don't necessarily have to worry about that because you can construct functions that return sets of packets. So if you want a packet to go both along the wire and into a bucket then you'll generated function that has two outputs instead of just one.
Yeah.
>>: Have you looked at the group [inaudible]?
>> David Walker: Have you looked at the group tables in OpenFlow?
>>: Right. They look very similar to this. We can forward a packet to a group [inaudible] second actions, forward it to another group and then the packet [inaudible]. It looks very similar to [inaudible].
>> David Walker: So I haven't specifically looked at that. But OpenFlow won't supply the means for constructing these functions that I'm going to show you in a second that allow you to take multiple different functions and put them together in various different ways. Although everything I'm going to say can in fact be implemented using OpenFlow. So it's not that I’m going to provide you with something that is impossible to do, it's that I'm just going to provide you with a high-level language that allows you to put things together in various different ways, and it's all going to be compiled down into vanilla OpenFlow.

Right. So as I said before, so NetCore is going to be this language basically for specifying and putting together these functions. And you are going to think of each network policy as a snapshot of network behavior at one point in time. Okay? And in a frenetic program, so frenetic is really a bigger language than NetCore, what happens is that there is events that come up from the network, things like topology changes, network statistics, packets coming in, those get processed by our platform and the frenetic application that you as the programmer might write will receive those events, and then on the receiving an event it might decide to produce a new global NetCore policy. Each time this new global NetCore policy gets produced it will be sent to our runtime system that's going to compile the policy to OpenFlow and then spread the policy across the switches. Yep.
>>: So I have a question. This network policy, global network policy is basically that function for every switch in the network?
>> David Walker: Yep. So for every location and packet, some new location and set of packets.
Yep.
>>: And there's more than one application program?
>> David Walker: You can build an application by having different parts of your application, different functions that generate different parts of the policy and then you can put together those different parts. So you think of it like you can write a Java program that has lots of different methods and you can put together those different methods-
>>: You imagine just sort of one person [inaudible]. It's not like I'm writing a program and someone completely disconnected from me is running a different program.
>> David Walker: Well, I think I know what you're saying. So you-
>>: [inaudible].
>> David Walker: It’s not like that at the moment. However, I am doing a project with some students right now that's looking at taking this in exactly that direction. So you can have multiple difference controller programs written by completely different people written on different, running on a different machines and basically have a hypervisor that stitches together the directives from those different applications. But that's very much work in progress.
Okay. So let me show you some of the simple policies to start off with, and I'll tell you about their semantics as well. So I’m going to define the semantics of a policy just as an evaluation function. So it's going to be an evaluated policy on a packet and return a collection of packets.
For PL people, you can think of this as a denotational semantics, but for rest of us, we can just

think about it as an evaluator. Okay. So the simplest policy is just the policy that drops all packets. So if I evaluate the drop policy on any packet, I just return the empty set of results.
>>: [inaudible].
>> David Walker: Okay. Yes. So if I have a policy that's the ID policy, it takes in a packet and it returns the singleton set. So it just does nothing the packet and returns it in place. The forwarded to the ports V policy looks at a packet, and if the packet’s at a switch in a particular port, then it's going to return the singleton set of the packet where the new location is at the switch in the different port, and if the packet’s already in a bucket, it's not going to do anything to the packet.
>>: What does that mean? It’s in a bucket? What is this bucket?
>> David Walker: So this bucket is this abstract location that is a data structure that sits on the controller. So there's two kinds of locations: one is concrete locations and one is abstract locations.
>>: So you can have many abstract locations on the controller?
>> David Walker: Yeah.
>>: Not just one.
>> David Walker: Many different buckets. That's right. So, in general, if you want to write different kinds of queries you can, yeah. For different properties you send packets in different places. Yeah.
>>: Is the location of a different packet, does that mean that the incoming location or the outgoing location or does that make no sense?
>> David Walker: It's going to be where the packet is right now. But there's another field that might be the inport of the packet which is where it came in to your switch.
>>: Okay.
>>: But what I don't understand is it generating a set of packets but where will they go?
>> David Walker: So they are going to, so you’re going to take a packet in here and you’re going to, the output of this function you think of as oh, there's a packet that maybe goes over here and that goes over there, and then what's going to happen is if after you process the function, packet one’s up here, it's going to go across the link and then you’re going to run the function again.

>>: So what if they are multiple links? Then which, supposing you-
>> David Walker: So it always goes device, link, device, link, device link, device link. So you go switch function, topology function, switch function, topology function, switch function, topology function, switch function, topology function, etc.
>>: Topology function?
>> David Walker: Yeah.
>>: So that's the topology function. Every packet that is inside, or no, put it on the corresponding link?
>> David Walker: It takes, so basically take a packet, I might send it across here, then the topology function basically says oh, this port is connected to this port so I’m going to, the topology’s going to take it across the link, and then at the next time step I'm going to apply this function and maybe it sends it down here.
>>: So the information about which port the packet is currently sitting in in, is that encoded in the packet itself? I mean how would that-
>> David Walker: Abstractly, that's the way you think about it. Concretely, the packet is actually somewhere in the network. And it actually runs along a wire.
>>: But here-
>> David Walker: In terms of-
>>: [inaudible] take away that information is [inaudible].
>> David Walker: Yeah, that's right. Exactly. So it's right here. I've said oh, it's inside the packet’s location.
>>: Okay. So all packets [inaudible] topology, then do some packets stay inside the node
[inaudible]?
>> David Walker: They all move. Or they get dropped.
>>: I don’t understand some of the forwarding policy, which it seems like you want your forwarding policies to be specific to a switch. Forwarding-
>> David Walker: Good point. That's going to be the next slide.
>>: It’s very different from forwarding ports on a different switch.

>> David Walker: That's right. That's going to be on the next slide.
>>: Like your policy’s not parameterized by S here. It sends forward to V. And you use the same forwarded to port two it doesn't mean anything unless you know what [inaudible] switch you’re talking about.
>> David Walker: Give me the next slide. I mean it means something here, right? So it says whatever switch it has, go to the port two on that switch.
>>: It’s a ridiculous policy.
>> David Walker: Yes, it’s a ridiculous policy. I agree. So you’d never write a policy just like this, but I'm showing you the building blocks, and you’re going to put together the building blocks to make reasonable policies. Okay. And here's a little policy that modifies a packet, a particular field of a packet. But let's get it to the next policy that sends it to a bucket. Oh, and there's some silly animations. Okay. So here's this ridiculous policy that you just said where I have the generic policy forward to port three, and it’s a ridiculous thing to do because it means that on every switch you might forward out some port three which is never what you're going to want to do. So we need a way to constrain when and where to apply various different atomic policies. So we do this using constructor called, well, a restriction. So this policy here says hey, if the switch is 101 and the packet is currently at port one and the source IP is 1, dot, 1 dot anything, then forward out port three. And here is another fragment of a policy that says oh, if the switch is 101 and the port is one and the source IP is not 1, dot 1 dot star, then go ahead and drop the packet.
>>: But if the precondition is not satisfied then what does it do?
>> David Walker: Then the packet gets dropped. So in general, we have some predicates on packets, and we have a new kind of policy that is beginning to allow you to build some more interesting things, and it says if the predicate is true on the packet, then do the underlying policy P. So again, the semantics is just what I said. If the predicate is true on the packet, then evaluate the policy P on that packet, otherwise drop. So do nothing.
>>: So why don't you just [inaudible] ahead of time on the packet, some fields and so on, and then ask them to write whatever Boolean crap you want to write on the fields?
>> David Walker: That's what this is. This is just Boolean crap, as you say, on the fields. That's all that is.
>>: [inaudible] buckets which-

>> David Walker: That's a couple of little specialized at things that I threw in to make you annoyed.
>>: But what about the fields? I mean, the packet [inaudible], right?
>> David Walker: So here's a predicate that says oh, a field F has a particular value. Sorry, I didn't say that. And a switch has a switch ID. Yeah. So it is basically conditions on fields and a
Boolean algebra over those conditions, and that's it.
>>: What happens if multi-core rules apply?
>> David Walker: So I'll tell you in a second. So these are independent policies, and we’ll need to put them together in various different ways. Okay? So I’m going to slow for the audience.
So the next operator takes two policies and puts them together in what we call in parallel. So here is how I might define the simplest possible router that says hey, for a single switch if destination IP is 1, dot, 2, dot, star and the destination or, sorry. The destination IP is 1, dot, two, then forward out port one. If the destination IP is 3, dot, four, forward out port two.
Okay? And here's another policy that specifies some monitoring. So the source IP is five, dot, six, dot, seven, dot, eight, then forward the bucket B1; if the source IP is five, dot, six, dot, seven, dot, nine, forward to bucket two. And once I’ve defined both of those policies, I can define another application that puts them together that says do both the monitor and the router.
>>: Can ask you a question?
>> David Walker: Yeah.
>>: So you told me that supposing I had only one of those implications inside the monitor, then that, the condition of the [inaudible] not satisfied, it would've dropped it.
>> David Walker: Yeah.
>>: But now, once you put it, do a plus with this other thing, now the semantics is that it's not going to get dropped, but it will try the condition for the other guy. Right?
>> David Walker: Yep.
>>: I see.
>> David Walker: Yep. The semantics of this plus one operator is if I evaluate P1 plus P2 on a packet, then what I'm going to do is I'm going to evaluate P1 on the packet, it's going to return a set of results’ and I'm going to evaluate P2 on that packet, and it's going to return a set of results and I just take the union of the two sets.

>>: Where'd you get like, how do you get like a priority or along this prefix match?
>> David Walker: Right. So you can get priority using the following: so here's a kind of priority, and if you don't like this priority we’ll do another one. So here's an example where we are going to stitch together two bits of policies. The goal is going to be to find the simplest possible load balancer. So we're going to spread client traffic over a set of replicas. We’re doing that by advertising a public IP address for the service. So the way our little load balancer’s going to work is it's going to split traffic based on client IP and rewrite the server IP address, and then we’re going to route to the appropriate replica.
So here is a little bit of code to, well, it looks to see if the source IP is zero, and the destination
IP is 1, 2, 3, 4, then we’re going to modify the destination IP to 10, dot, O, dot, O, dot, one; and if the source IP starts with one, then we’re going to modify that destination IP to be 10, dot, O, dot, O, dot, two. And now we actually have to get the packet to the right spot, so we might have a forwarder component that says oh, if the destination is 10, dot, O, dot, one, then forward out port one; and if it’s 10, dot, O, dot, two, then forward out port two. And we’ll create our little load balancer component by using what we call sequential compositions to put together those two different pieces. Okay? So the selector first modifies the IP address, then based on the modified IP address, we will go ahead and pipe it into the forwarder.
>>: The interesting thing is the output is a set.
>> David Walker: Yes.
>>: So you apply point wise and the union of all the point wise sets that we've created.
>> David Walker: Exactly. So Shaz described the semantics of this operator for us very accurately. So the sequential composition, P1 followed by P2, is evaluate P1 on the packet P, generate a set of outputs, then on each one of those outputs, evaluate P2 and take the union of all those outputs to give you a final set of results. Okay? So, and I'll get, I have one more example that you might like in a second. So this is basically all there is to this little algebra that we call NetCore.
>>: No loop?
>> David Walker: Sorry?
>>: No loop?
>> David Walker: No. There's no loop. We have, let's call recursion a work in progress. There's one small application of it, but you don't really need general looping. But let's talk about that afterwards sometime.

>>: I have a question about the bucket. So the bucket is [inaudible] per node, like if you put something into a bucket, is it still at that node? So if like, you’re moving from bucket back onto port-
>> David Walker: You never move things that are in buckets back to anywhere else. They just stay put. So, and the buckets are on the controller, the centralized controller. So there's this centralized, there's this higher-level frenetic program that reacts to events. And one kind of event is oh, what packet appeared in my bucket? Let's go look at the packet and maybe change the policy based on that.
>>: All right.
>> David Walker: Okay, right. So I've really told you what NetCore is. It's very, very simple. I think its simplicity is a bonus. Yeah.
>>: [inaudible] network [inaudible]. What else does the bucket can do?
>> David Walker: The bucket is basically designed to do network statistics. That's really its purpose so that you can monitor load on the network and then react and change the policy based on load. That’s the primary purpose of the buckets. Can be used to implement, well let's just call it, let's leave it at that for now. Yep.
>>: So my understanding is that the controller is just an ordinary box, a standard PC, right?
>> David Walker: Yeah.
>>: So in that case, is there any advantage to using frenetic rather than just a standard programming language? CEO or Java, or ML, if you want?
>> David Walker: Right. So frenetic is, so the frenetic language is really, it really is just an ordinary program language. We have versions of Python, we have versions in a O-
Camel[phonetic] that we’re building right now, I have version Haskell. So it’s just an ordinary programming language that does ordinary things to react to events. The only special part is that it produces a series of these NetCore policies and then it just hands it off to our compiler that compiles these policies and generates a bunch of OpenFlow messages. So really, if you don't want to call frenetic a language, you can call it just a library that has some nice structure and it’s implemented in a Python or whatever you want.
Okay. So one small bit of evidence that we've designed, a nice little languages that has a particularly simple equational theory that has lots of very familiar laws, the standard OpenFlow messages, they have no equational theory. So just a very simple things like hey, parallel composition is commutative. Yey. If this were not true, we would've done something wrong.

So it's just a small amount of evidence that we haven't done something completely broken quite yet. You know, parallel composition is associated. If, this is maybe Shaz’s question, if you composed drop with any policy, you get back the policy P. There are some more laws for sequential composition, right? Again, it's associative. ID is that the unit for policy’s P. And drop is like the zero element. So drop sequentially composed with P is just drop. And if you have an if statement and you sequentially compose it with R, then that’s the same as doing the condition first and sequentially composing the two branches.
>>: [inaudible]?
>> David Walker: We have lots more axioms. P plus P is equal to P? That's a good question. So
I have slightly lied to you in this talk in that I've said that the semantics of policies generates a sets of, it actually generates a multi-set.
>>: Okay.
>> David Walker:. But we could-
>>: So P is equal to 2P.
>> David Walker: Yeah. But you never want to write that. We could change the language and make it a set. I’m just going to go with the lie. So if you believe the lie, then yes, that would be true.
>>: Is there reason to why [inaudible]?
>> David Walker: It’s because OpenFlow allows you to admit the same packet twice on the same port. So OpenFlow supplies that. But I don't know when you would ever want to use that.
>>: [inaudible]?
>> David Walker: No, I don't think so.
>>: So then why do you have multi-set? You want to hide that, right? From the users.
>> David Walker: Probably. So that's why I give the talk saying it's a set.
>>: So the fact that the drop, the unit is [inaudible] set in your operational model, the packets could be dropped anywhere, anytime.
>> David Walker: It means that, so this policy is equal to that policy.

>>: Yeah, so if the policy says the set of locations so that you can produce, it also means it could also, you know, show up in none of those locations. Is that right?
>> David Walker: No. So it means that drop doesn't think what you, doesn't actually mean what you think it means.
>>: Okay.
>> David Walker: It means that, so again, drop generates the empty set of packets, plus is the union of results, so empty set union, some good results that P produces is just the good results that P produces. So in parallel, composing drop with something else does not force a drop. It these things alone which might be counterintuitive.
>>: Right. So if you write a policy that has an empty output set, then it gets forced to drop, but if it does not, if it does at least one element you're guaranteed that it doesn't drop.
>> David Walker: If you used parallel composition to compose it with something else, then yes.
That's right. Okay. So let me, on this kind of issue, let me just show you a little example of some nice reasoning that you can do using this equation theory. It's very simple, but it just shows that you can, it’s easy to think about these programs whereas it's not easy to think about what happens to OpenFlow.
>>: [inaudible] actually go to the real network, it can drop packets without you being in control of that.
>> David Walker: I mean, well there are, not in our model. But in real networks packets get dropped. So our-
>>: When they get dropped [inaudible] you install a rule that allows it to be dropped, right?
>>: [inaudible].
>> David Walker: Yeah, exactly.
>>: [inaudible] or the cable is cut?
>>: They're all best effort.
>> David Walker: We don't model that.
>>: Is that bad? That means that your model is different from reality.
>> David Walker: That means that a subset of the packets that we say will arrive actually do arrive. And every network protocol has to deal with this because-

>>: [inaudible].
>>: If you can say that, right, that’s good.
>> David Walker: So we are only software, and we can't ensure something that can't be insured by the underlying hardware. And we can’t prevent someone from cutting a cable in the underlying hardware. But that doesn't mean that there's not research to be done on fault tolerance and other such issues, and there certainly is.
So let me just go on with this. So let's suppose I define a little firewall, what does a firewall look like? Well, it looks like something like this. If the source IP is some value, then drop it.
Otherwise, do nothing with the packet. And now I have some router. And I'm not going to tell you how the router is defined, but it's when the forward packets somehow. Right?
So my application I might write, it's nice, it’s [inaudible] nice modularly, I want to sequentially compose my firewall with my router. Now, does this actually do what we think it does? Does it actually go ahead and drop all of those packets and leave the rest alone? Well, we could do a little bit of equational reasoning without this definition over here to find that out. So my app is just a firewall sequentially composed with the router; we can expand the definition of the firewall, that gives us this; we can go ahead and then use one of the axioms on the previous page to push the router inside the two branches of the F; we know that drop sequentially composed with anything is just drop; we know that ID sequentially composed with anything is just that other thing; so in the end, we deduce that the composition of our firewall with any router is equal to the program that looks at the source IP; if it’s this, drops, otherwise does whatever the router does. So that's also how you kind of get some prioritization, if that makes any sense.
Okay. So our language supports a nice kind of modular reasoning. Okay. 11:22. So now I wanted to, I’ve given you this high-level language, we have to compile it to something that actually works, that actually forwards packets around a network-
>>: You actually didn’t show a constructor that constructs more than one packet from a single packet.
>> David Walker: That's true.
>>: That's where you would get the power of using a set, right?
>> David Walker: Well, a sequential composition can do that. So I showed like non-sequential, the parallel composition. It will forward something along the line, plus it will forward into a bucket.

>>: Okay.
>> David Walker: There is an example somewhere back there. So that will give us two packets in two different places. We also have though, like OpenFlow supports like a flood action that we also have that in a set of actions. I just didn't show it, which will flood a packet at all the ports except the incoming port.
>>: So the same bucket in a different places is just a multi-set of two buckets with the locations?
>> David Walker: It's a set of packets. So the output is a set of located packets where a located packet is a pair of a location and a packet.
>>: So you always have the set [inaudible]?
>> David Walker: So that's why it's really never, you're never going to have the multi-set in practice. I just, anyway. Okay. So what we need to do, I'm going to sketch to you a little bit of how the compilation works. And basically, the main thing that needs to happen is you need to take a high-level policy that looks something like this and you need to generate what I’m going to call an action classifier, or what I called before a flow table, or part of a flow table. So this is basically a list of rules, and each rule has a pattern and some list of actions that occur in sequence. Okay? So here is an example of such a classifier. I'm always going to draw the higher priority rules at the top and the lower priority rules at the bottom. And it has the semantics of the first rule that matches the packet gets, is the one that gets activated.
Okay. So the first step that I'm going to tell you about in compiling these policies will involve compiling predicates. So we have basically patterns on the fields and a Boolean algebra on top of that. And so what we're going to do to compile a predicate is we’re going to generate what
I'll call a Boolean classifier, and that is a list of rules, and each rule is going to have a pattern and instead of an action, it’s going to have just true or false. And the idea is that if a packet matches this predicate, then when you sent the packet through this Boolean classifier it's going to match a pattern that's associated with true. But if packet does not match this predicate, it's going to match a pattern associated with false or it's going to fall through and not match any of the patterns. So the default, if you want, is false. Okay. So a simple basic pattern that looks at a field like this one gets compiled into basically that same pattern and the Boolean true. So it says, I match. If you had a predicate like no packets, you would generate a classifier with an empty set of rules. Nothing matches. If you wanted to match all the packets, you generate a classifier which has wildcards for all of the fields and the Boolean true.
If you're trying to compile the negation of some predicate, what you'll first to do is you'll compile that predicate into a Boolean classifier, and then what you’ll doing is you’ll take that

classifier, you'll negate all of the conditions, and you will return true. Right? So if this one up here said oh, [inaudible] a packet matches and that's true, then down here it going to be false, it’s going to be dropped. Okay. If I have a conjunction of two predicates, Q1 and Q2, then what
I'm going to go ahead and do is I'm going to take the first predicate and I'm going to compile it,
I’m going to take the second predicate and compile it, suppose I get it two rules in each of those, and then what I'm going to do is an operation that we often call the cross product. It's going to take at the first of pattern of there and conjoin it with both of these. And it's going to take the second pattern up there and conjoin it with both of those. And it's going to conjoin the Booleans.
>>: It's not symmetric?
>> David Walker: Excellent question. So it's not symmetric, and it should be. So, challenge problem. Figure out why it actually is symmetric, or why if I flip the two, it doesn't matter.
Basically it doesn't matter because the topmost one quotients out the overlap. But it always stumps me when I have to explain it. So that's why it's a challenge left for the audience. Okay.
What this means though is it means these patterns have to support certain operations on them so they have to support basically intersection. They sort of have to form a lattice with a couple of technical conditions on them. And again, I mentioned this is the cross product and it shows up in multiple different places.
Okay. So that's a sketch of how to compile some of the predicates. Here's a sketch of how to compile some of the policies. So if we take a conditional policy like Q implies P, then the first thing we have to do is we have to compile Q. We might get a Boolean classifier that has the end that has one pattern associated with true, another pattern associated with false. And you can think of this as saying okay, pattern one because it's true, I'm going to say the packet’s basically okay and do nothing with it; pattern two, I'm going to say I’m going to drop any packets that hit that. If I then go ahead and compile the policy, get some pattern three and some action, then the resulting classifier, both pattern and one and pattern three must be true; and if that's true I'm going to generate basically the action A that I got out here. If both pattern two and pattern three are true, then I'm going to basically wind up dropping the packet.
In order to compile a parallel conjunction, we compile each of the two policies. We get a, perhaps in this simple case, a pattern and an action, a pattern and an action for each. And then what we do is we do a similar kind of cross product operation that we did last time. We take the conjunction of the two patterns, and if that is true, then we’re going to have to do both actions. And if the conjunction is not true, but just the first pattern is true, then we’re going to have to do the first action. And if the second pattern is true and not the first pattern, then we’ re going to do, well, then we’re to do action A too. Okay. In order to make this work, by the way, it has to be possible to take the actions from two different policies and put them in a

linear sequence that OpenFlow can do. And this is actually not always possible with general modifications. And I can show you why afterwards. It's not always possible, but actually Arjun, who's a postdoc working with Nate, has a nice little fix that slightly restricts the actions that you're allowed to do so that compiling this is always possible and it looks like, for almost all policies, this restriction will work.
>>: Can I ask you a question?
>> David Walker: Yep.
>>: Why can't you compile to current networks as they are?
>> David Walker: Why can't we compile to current networks as they are? So the languages for configuring current networks are extremely complicated, extremely poorly designed, and extremely vendor specific. And, for example, both Jen Rexford and my former student, Issak
Mandelbaum[phonetic], they both went to work at AT&T and they both did separate projects in which their goal was merely to figure out what the syntax of the Cisco configuration languages for some router. And Jen just said oh, she just failed. And Issak[phonetic] I think did some inference stuff and he maybe partially succeeded.
But the problem is that even figuring out what the syntax of these languages is is very difficult.
It's also the case that the mechanisms for controlling the network are, the switches are sometimes very indirect. So they may run some shortest path routing algorithm, and the way you might change the policy might be to do something like change the weights that the algorithm uses on the various links in the path and thereby indirectly reconfigure the network by changing those weights of knowing how the shortest path algorithm works. So it's a very indirect mechanism for controlling these switches. And so, and basically I mean, but really I think the most pressing thing is the simplicity aspect. OpenFlow provides an incredibly simple interface. There's no way that I would've bothered to take the time to learn the vigarities[phonetic] of Cisco's vendor specific-
>>: So what you are saying is that is right now you are dependent, your success depends on the success of OpenFlow. [inaudible].
>> David Walker: So a couple weeks ago I was at the open networking Summit, and you might be surprised, but it was bigger than POPL. There were 1600 people there all talking about
OpenFlow and the software defined networks, and you know many of them did not know what they were talking about, but some fraction of them did. You know, again, so recently Google has deployed some of its technology in their backbone network. The Sierra is a startup that was bought by VMware for 1.2 billion dollars. It's a series of other smaller startups bought for hundreds of millions of dollars. It could collapse, but it might workout. And I guess what we're

doing here is we've managed to get in at the beginning of this initiative and we're hoping that we can influence the community, provide some language design ideas, some principles, in terms of how to design these systems, and we’re hoping that we can make it, make an impact on this area that has a sort of this fresh slate, we are starting over again kind of situation. So
I'm not going to say that OpenFlow is a guaranteed thing, but there's some promise and I'm going to take that risk, if you want. Yeah.
>>: I don't think I understand the semantics of linearized. So-
>> David Walker: It's kind of vague. It’s semantics [inaudible].
>>: Suppose A, 1 is forward one, and A, 2 is forward two. It’s a copy of the packet?
>> David Walker: Yeah. It will do both. The OpenFlow basically does each action in series. So the reason why it's difficult with modifications is because-
>>: So you will have two packets of [inaudible] and one-
>> David Walker: Yeah.
>>: Okay. And what is this difficulty of applying both actions?
>> David Walker: So if the first action does something like change the source IP to seven and then forward out port one, and the second action here does change different field like the dstIP to five and then forward out port two, then what you want to do is do the first action with the old source, with the old destination IP, do the second action with the old source IP, and you can't put them in any particular order and make both happen.
>>: I see.
>>: [inaudible] the second option queries the ID if you just changed?
>> David Walker: Say that one again?
>>: So the first action, change the source ID, and the second action, if source IP equals something-
>> David Walker: Yeah. The second action has that the form and then it will work. So there's, if modification, let me tell you afterwards so I don't get to-
>>: So it’s a limitation to language [inaudible]. Like if you have instructions for copy this packet into two, you would-

>> David Walker: Yeah. But there is more one way to get around this limitation. Okay. Right.
So just one other thing I wanted to say about compilation was that as we compile these policies, they can get very large. So you have to think a little bit about optimization. And so as a really simple example, if I tried to sequentially compose these two policies here, what would happen is I get a series of rules, and these two intermediate rules would turn out to be redundant. They'd come from basically taking the intersection of this guy with the guy and taking the intersection of that guy with that guy. And both of those would have a prefix one star and instead of, and, you know, do one star. Anyway, so you get a very large and significant blow up unless actually every time you do one of these cross product steps you need to optimize it as you go to lead all the redundant rules that you might otherwise generate. And if you don't do that then you blow up very fast.
>>: [inaudible]?
>> David Walker: No, we don't do anything as sophisticated as that. One technique is just do a cross product and just run through all the rules and delete all of the shadowed rules and then go on. But there's another one where you can, as you go, cross out things.
>>: [inaudible]?
>> David Walker: Well, you might have some more smarter ideas about how to do this than we do.
>>: Is this a known [inaudible] optimization of these tables? You know, problem?
>> David Walker: There is a lot of literature on taking tables that are big and compressing them.
>>: What about finding an optimal table that is going to [inaudible]?
>> David Walker: Yeah. So depending on the kinds of patterns you have, I think it's, the status is it's an open problem as to whether or not it's MP complete or MP hard. Basically, people believe that it is; but certainly there's lots of those things. So the only thing I'm saying here is that we’re at a little bit of a different setting because we’re compiling from a high-level language to a low level language, and you can't actually do something like wait until the end of compilation and then do minimization, you have to do it as you go. That's the only thing that
I'm saying.
>>: So the core, NetCore table, my understanding is that these tables were written by hand,
[inaudible] right?

>> David Walker: Or you would, in this SDN world, you would craft messages to install one rule after another rule after another rule. Yeah, and they're written by hand. But there has been some, like people would write very long lists of firewall table rules by hand, for instance. And then there's has been past work on doing things like minimizing those rules.
>>: Okay.
>> David Walker: But, yeah. One pushback I get often from networking community is oh, you're going to, it’s easy to write these policies that will generate lots and lots and lots of rules, isn't that a bad thing? And I think it's probably likely that a compiler will be able to do a better job minimizing these rules than a human will. Our compilers might not be there yet, but they will.
>>: [inaudible] compile at intermediate stages you have large tables that you cannot compress, but then you get all the way to the end and then [inaudible] compress too small. Is that happening or is that-
>> David Walker: We haven't looked deeply enough into that. We haven't looked at that too much. We basically have been oh, we did the first [inaudible] compilation. Oh, doesn't work because it's way too slow. So we did enough optimization to make it work and then we haven't focused as much on that as on other topics.
>>: [inaudible] best time. The first column is not necessarily just be in synch with predicate. It might be more compressed, right?
>> David Walker: Yeah. This can be like there's, you know, like nine different fields that have various different patterns, yeah.
>>: Because [inaudible] techniques using [inaudible]?
>> David Walker: Okay, cool. No, I’d love to hear about various different algorithms people know for doing this. I had more things, but I think I've been talking for a while, so I might just, you know, I'm here [inaudible] and I would love to chat more about various different algorithms that we use. I'd love to talk more about new kinds of abstractions that we could build. I think that, you know, what we have here is a good solid core, but there’s lots more work to do. I’d very much like to talk about abstractions for quality of service. I'd like to talk about abstractions that, I'd like to understand more fully what, well, a data center for instance has,
I've been told, at least three parts. It has network, it has entose[phonetic] that do compute, and it has storage. So I'd very much like to know what the assembly language is for those other different components of the system that isn't the network part because if some systems people who can tell me what the assembly language is that they work with, maybe we can find new

higher-level constructs to make it easier to program those things. And I'd love to, you know, there is a metaphor that people talk about, the data center as the computer. But if it is a computer, then it's got to have an instruction set, I'd love to know what this instruction set is, and I'd love to, you know, as sort of PL theorists like I often do, bake this construction set down to some really small core. So someone who doesn't really know very much, like me, can understand it, can figure it-
>>: Small like [inaudible].
>> David Walker: Sure. I don't know. Anyway, yeah. So I’d love to talk about all these things, and-
>>: So David’s going to [inaudible]. And if anybody wants to meet him, send me an email. I’m managing his calendar.
>> David Walker: Yeah. And I'll just leave it there.