SDN: Extensions
Middleboxes
Ack: Vyas Sekar, Aaron Gember, Felipe Huici, Zafar Qazi
1
Need for Network Evolution
New applications
Evolving
threats
Performance,
Security,
Compliance
Policy
constraints
New devices
2
Network Evolution today: Middleboxes!
Type of appliance
Data from a large enterprise:
>80K users across tens of sites
Just network security
$10 billion
Number
Firewalls
166
NIDS
127
Media gateways
110
Load balancers
67
Proxies
66
VPN gateways
45
WAN Optimizers
44
Voice gateways
11
Total Middleboxes
Total routers
636
~900
3
How many middleboxes do you deploy?
Typically on par with # routers and switches.
How do administrators spend their
time?
Most administrators spent 1-5 hrs/week dealing with failures;
9% spent 6-10 hrs/week.
Firewalls
Proxies
IDS
Misconfig.
Overload
67.3%
63.2%
54.45%
16.3%
15.7%
11.4%
Physical/
Electrical
16.3%
21.1%
34%
Key “pain points”
Narrow
Management Management Management
interfaces
Specialized
boxes
“Point”

solutions!
Increases capital expenses & sprawl
Increases operating expenses
Limits extensibility and flexibility
6
SDN Stack
App
Runtime
Applications
Controller
Control Flow, Data Structures, etc.
Controller Platform
Switch API
(OpenFlow)
Switches
7
Outline
• Why middleboxes?
• SIMPLE
• OpenMB
• Slick
8
Can SDN simplify middlebox management?
Centralized Controller
Web
Firewall
IDS
Proxy
“Flow”
FwdAction
…
…
Proxy
OpenFlow
“Flow”
FwdAction
…
…
IDS
Scope: Enforce middlebox-specific steering policies
Necessity + Opportunity:
Incorporate functions markets views as important
9
What makes this problem challenging?
Centralized Controller
Web
Firewall
IDS
Proxy
“Flow”
FwdAction
…
…
Proxy
OpenFlow
“Flow”
FwdAction
…
…
IDS
Middleboxes introduce new dimensions beyond L2/L3 tasks.
Achieve this with unmodified middleboxes and existing SDN APIs
10
SIMPLE overview
Web
Firewall
IDS
Proxy
Policy enforcement layer for
middlebox-specific “traffic steering”
Legacy
Middleboxes
Flow
Action
…
…
Flow
Action
…
…
OpenFlow
capable
11
Challenge: Policy Composition
Policy Chain:
Firewall
S1
*
IDS
Proxy
Firewall
IDS
Proxy
Oops!
Forward Pkt
to IDS or Dst?
S2
Dst
“Loops”
Traditional flow rules may not suffice!
12
Challenge: Resource Constraints
Firewall
Proxy
Space for
traffic split?
S2
S1
S3
S4
IDS1 = 50%
IDS2 = 50%
Can we set up “feasible” forwarding rules?
13
Challenge: Dynamic Modifications
User1: Proxy  Firewall
User2: Proxy
User 1
Proxy
S1
User 2
Proxy may
modify
flows
S2
Firewall
Are forwarding rules at S2 correct?
14
New dimensions beyond Layer 2-3 tasks
1) Policy Composition  Potential loops
2) Resource Constraints  Switch + Middlebox
3) Dynamic Modifications  Correctness?
Can we address these with unmodified
middleboxes and existing SDN APIs?
15
SIMPLE System Overview
Web
Firewall
IDS
Proxy
Modifications Handler
Resource Manager
Rule Generator
Legacy
Middleboxes
Flow
Action
…
…
Flow
Action
…
…
OpenFlow
capable
16
Composition  Tag Processing State
Policy Chain:
Firewall
*
Firewall
IDS
Proxy
IDS
Proxy
Fwd to
Dst
S1
ORIGINAL
S2
Dst
Post-Firewall
Post-Proxy
Post-IDS
Insight: Distinguish different instances of the same packet
17
SIMPLE System Overview
Web
Firewall
IDS
Proxy
Modifications Handler
Resource Manager
Rule Generator
Legacy
Middleboxes
Flow
Action
…
…
Flow
Action
…
…
OpenFlow
capable
18
Resource Constraints Joint Optimization
Topology &
Traffic
Middlebox
Capacity +
Footprints
Switch
TCAM
Policy
Spec
Resource Manager
Optimal & Feasible
load balancing
Theoretically hard!
Not obvious if some configuration is feasible!
19
Offline + Online Decomposition
Policy
Spec
Network
Topology
Switch
TCAM
Mbox Capacity
+ Footprints
Traffic
Matrix
Resource Manager
Offline Stage
Deals with Switch constraints
Online Step
Deals with only load balancing
20
Offline Stage: ILP based pruning
• Feasible
• Sufficient freedom
Set of all possible middlebox
Set
loadPruned
distributions
Balance the middlebox load
21
SIMPLE System Overview
Web
FW
IDS
Proxy
Modifications Handler
Resource Manager
Rule Generator
Legacy
Middleboxes
Flow
Action
…
…
Flow
Action
…
…
OpenFlow
capable
22
Modifications  Infer flow correlations
Correlate
flows
Install
rules
Payload
Similarity Proxy
User 1
S1
User 2
S2
Firewall
User1: Proxy  Firewall
User2: Proxy
23
SIMPLE Implementation
Web
Resource Manager
(Resource Constraint)
FW
IDS
Proxy
Modifications Handler
(Dynamic modifications)
CPLEX
Rule Generator
(Policy Composition)
POX
extensions
OpenFlow 1.0
Flow
…
Tag/Tun
nel
Action
…
Flow
…
Tag/Tun
nel
Action
…
24
Evaluation and Methodology
•
•
•
•
•
What benefits SIMPLE offers? load balancing?
How scalable is the SIMPLE optimizer?
How close is the SIMPLE optimizer to the optimal?
How accurate is the dynamic inference?
Methodology
– Small-scale real test bed experiments (Emulab)
– Evaluation over Mininet (with up to 60 nodes)
– Large-scale trace driven simulations (for convergence times)
25
Summary of SIMPLE
• Middleboxes: Necessity and opportunity for SDN
• Goal: Simplify middlebox-specific policy enforcement
• Challenges: Composition, resource constraints, modifications
• SIMPLE: policy enforcement layer
– Does not modify middleboxes
– No changes to SDN APIs
– No visibility required into the internal of middleboxes
• Scalable and offers 4-7X improvement in load balancing
26
Outline
• Why middleboxes?
• SIMPLE
• OpenMB
• Slick
27
Middlebox Deployment Models
• Arbitrary middlebox placement
• New forms of middlebox deployment
(VMs, ETTM [NSDI 2011], CoMB [NSDI 2012])
28
Live Data Center Migration
• Move between software-defined data centers
Data Center A
Data Center B
• Existing VM and network migration methods
– Unsuitable for changing underlying substrate
Programmatic control over middlebox state
29
Middlebox Scaling
• Add or remove middlebox VMs based on load
• Clone VM (logic, policy, and internal state)
– Unsuitable for scaling down or some scaling up
Fine-grained control
30
Contributions
• Classify middlebox state, and discuss what
should be controlled
• Abstractions and interfaces
– Representing state
– Manipulating where state resides
– Announcing state-related events
• Control logic design sketches
31
Software-Defined Middlebox Networking
Today
SDN-like Middleboxes
IPS
App
App
Controller
Middlebox
Middlebox
32
Key Issues
1. How is the logic
divided?
2. Where is state
manipulated?
3. What interfaces
are exposed?
Middlebox
App
App
Controller
Middlebox
33
Middlebox State
• Configuration input + detailed internal records
Src: HostA
Server: B
Server: B
CPU: 50%
Proto: TCP
Port: 22
State: ESTAB
Seq #: 3423
Hash: 34225
Content: ABCDE
Balance Method:
Round Robin
Cache size: 100
Significant state diversity
34
Classification of State
Action
Src: HostA
Server: B
Proto: TCP
Port: 22
Supporting
Server: B
CPU: 50%
State: ESTAB
Seq #: 3423
Hash: 34225
Content: ABCDE
Internal & dynamic
Tuning
Balance Method:
Round Robin
Only affects
performance,
not correctness
Cache size: 100
Many forms
35
How to Represent State?
Per flow
Src: HostA
1010110
Server: B
Server:
B
1000101
CPU: 50%
Proto: TCP
1111000
Port: 22
State:
ESTAB
1101010
Seq #: 3423
Policy
Language
Hash:
34225
0101001
Content: ABCDE
Significant
diversity
May be
shared
Unknown
structure
Sharedmiddlebox operations
Commonality among
36
State Representation
Key
Field1 = Value1
…
FieldN = ValueN
Action
Offset1 → Const1
…
OffsetN → ConstN
Supporting
Binary
Blob
• Key: •protocol
header for
field/value
Only suitable
per-flowpairs
stateidentify
traffic• subsets
to vendor
which state
applies
Not fully
independent
• Action: transformation function to change
parts of packet to new constants
• Supporting: binary blob
37
How to Manipulate State?
• Today: only control some state
– Constrains flexibility and sophistication
• Manipulate all state at controller
– Removes too much functionality from middleboxes
Controller
Middlebox
38
State Manipulation
Determine where
state resides
Controller
IPS 1
IPS 2
Create and
update state
• Control over state placement
1. Broad operations interface
2. Expose state-related events
39
Operations Interface
get (
Filter
SrcIP = 10.10.54.41
,
)
Key
SrcIP = 10.10.0.0/16
DPort = 22
Key
SrcIP = 10.10.54.41
DstIP = 10.20.1.23
SPort = 12983
DPort = 22
Action
*
Supporting
State =
ESTAB
Action
• Need
atomic
blocks
add
( Key
, operations
)
DstIP = 10.20.1.0/24
DROP of
• Potential for invalid manipulations of state
remove(
Filter
…
,
Source
Destination
Proto
*
10.20.1.0/24 TCP
Other Action
*
DROP
)
40
Events Interface
Controller
Firewall
• Triggers
– Created/updated state
– Require state to
complete operation
Contents
Balance visibility• and
overhead
– Key
– Copy of packet?
– Copy of new state?
41
Conclusion
• Need fine-grained, centralized control over
middlebox state to support rich scenarios
• Challenges: state diversity, unknown semantics
Key
Field1 = Value1
…
Action
Offset1 → Const1
…
get/add/remove (
…
,
Supporting
Binary Blob
)
42
Open Questions
•
•
•
•
•
•
Encoding supporting state/other action state?
Preventing invalid state manipulations?
Exposing events with sufficient detail?
Maintaining operation during state changes?
Designing a variety of control logics?
Providing middlebox fault tolerance?
43
Outline
• Why middleboxes?
• SIMPLE
• OpenMB
• Slick
44
Network Policies
• Reachability
– Alice can not send packets to Bob
• Application classification
– Place Skype traffic in the gold queue
Limitations of SDN Data Plane
Match
Action
10.2.3.4:10.2.3.3
Fwd Port 1
A2:e3:f1:ba:ea:23:*
Drop
• Limited actions and matching
– Match: Ethernet, IP, TCP/UDP port numbers
– Action: forward, drop, rewrite header, etc.
Extending SDN’s Data Plane
• Expand the OpenFlow standards
– Requires hardware support
• Implement richer data plane in controller
– Introduces additional latency to packets
• Add new devices (Middleboxes)
Example: Detecting Network Attacks
• Inspect all DNS traffic with a DPI device
• If suspicious lookup takes place, send to traffic scrubber
Example: Detecting Network Attacks
• Inspect all DNS traffic with a DPI device
• If suspicious lookup takes place, send to traffic scrubber
Example: Detecting Network Attacks
• Inspect all DNS traffic with a DPI device
• If suspicious lookup takes place, send to traffic scrubber
Example: Detecting Network Attacks
• Inspect all DNS traffic with a DPI device
• If suspicious lookup takes place, send to traffic scrubber
Challenges
• Specify network policies across middleboxes
– Difficult to automatically react to middlebox events
• Dynamically place sophisticated middleboxes
– Difficult to determine efficient placement
– Difficult to adjust placement to traffic patterns
• Support for arbitrary middlebox functionality
– Difficult to capture hardware requirements
Slick Contributions
• Abstraction for programming middleboxes
– Simplifies the development of network policies
– Separates specification of intent from implementation
• Dynamic placement of middlebox functionality
– Online resource allocation algorithm
• Support for heterogeneous devices
– Maintains performance profiles of middlebox
Slick Architecture
Application
Your network
operator
3rd party element
developers
• Encodes network policy
• Provides handlers for
triggers
Slick Controller
Triggers from
elements
Middlebox
Element
Middlebox
Element
Virtual Switch
Programmable device:
NetFPGA, x86 server
• Piece of code encapsulating
middlebox functions
Slick Architecture
Application
Slick Controller
Deploy
Middlebox
code
Middlebox
Element
Middlebox
Element
Virtual Switch
Programmable device:
NetFPGA, x86 server
• Runs applications
• Runs resource allocation algo.
• Places middlebox elements
• Steers traffic through middleboxes
• Configures switches
• Installs/uninstalls
middlebox functions
Resource Allocation Heuristic
Traffic matrix
And topology
Network policies in
applications
Middlebox perf
profile
Hardware
constraints
Resource allocation heuristic
Objective: minimize latency (path lengths)
Placement
Decisions
Traffic Steering
OpenFlow
Controller
Virtual Switch
Virtual Switch
Programmable
device
Programmable
device
Summary
• Slick: control plane for middleboxes
– Presented an initial architecture
– Discussed algorithmic challenge
• Slick is implemented in python
– Slick controller as a module on NoX 0.5.0
– Developed 2 applications and 3 middlebox elements
• Open questions
– How can developers help guide placement?
– What is the optimal solution for resource allocation?
Discussion: Likes/dislikes?
58