Enterprise Networks
Jennifer Rexford
COS 461: Computer Networks
Lectures: MW 10-10:50am in Architecture N101
http://www.cs.princeton.edu/courses/archive/spr12/cos461/
Networking Case Studies
Data Center
Enterprise
Backbone
Cellular
Wireless
2
Simple Enterprise Design
• Single layer-two subnet
• Local services
– Hubs and switches
– Gateway to the Internet
– Single IP address block
S
1.2.3.1
S
– DHCP
– DNS
DHCP server
1.2.3.0/24
Internet
G
1.2.3.76
0.0.0.0/0
S
1.2.3.5
1.2.3.150
S
DNS server
3
Limitations of Simple Design
• Ethernet scalability and
performance
• Single ISP reliability and
performance
• Limited IP address space
S
1.2.3.1
S
• Unwanted Internet traffic
• Privacy and isolation within
the enterprise
• Detecting and preventing
bad behavior from inside
DHCP server
1.2.3.0/24
Internet
G
1.2.3.76
0.0.0.0/0
S
1.2.3.5
1.2.3.150
S
DNS server
4
Beyond Ethernet Switching
5
Scalability Limitations of Ethernet
• Spanning tree
– Paths that are longer than necessary
– Bandwidth wasted for links not in the tree
• Forwarding tables
– Bridge tables grow with number of hosts
• Broadcast traffic
– ARP, DHCP, and broadcast applications
• Flooding
– Frames sent to unknown destinations
6
Hybrid of Switches and Routers
1.2.3.0/26
R
Ethernet Bridging
- Flat addressing
- Self-learning
- Flooding
- Forwarding along a tree
R
IP Routing
- Hierarchical addressing
- Subnet configuration
- Host configuration
- Forwarding along shortest paths
1.2.3.192/26
R
R
1.2.3.128/26
R
1.2.3.64/26
7
Internet
Limitations of Hybrid Design
• No plug-and-play and mobility between subnets
• Need consistency between IP addressing & routing
1.2.3.0/26
R
1.2.3.192/26
R
R
R
1.2.3.128/26
R
1.2.3.64/26
8
Internet
Virtual Local Area Networks
9
Early Days of Ethernet LANs
• Thick cables snaked through cable ducts
– Every computer they passed was plugged in
• All people in adjacent offices on the same LAN
– Whether they belonged together or not
• Users grouped based on physical layout
– Rather than organizational structure
• Security, privacy, and scalability limitations…
10
Today’s Ethernet LANs
• Changes introduced by hubs and switches
– Every office connected to central wiring closets
– Often multiple LANs (k hubs) connected by switches
– Flexibility in mapping offices to different LANs
• Can group by organizational structure
– Better privacy: snooping in promiscuous mode
– Separate IP addresses: one IP subnet per LAN
– Better security: access control at IP routers
– Better load management: isolate broadcast/flooding
11
People Move, and Roles Change
• Organizational changes are frequent
– E.g., faculty office becomes a grad-student office
– E.g., graduate student becomes a faculty member
• Physical rewiring is a major pain
– Requires unplugging the cable from one port
– … and plugging it into another
– … and hoping the cable is long enough to reach
• Would like to “rewire” the building in software
– The resulting concept is a Virtual LAN (VLAN)
12
Example: Two Virtual LANs
R
RO
O
R
R
O
R
R
O
O
R
O
O
O
O
R
R
Red VLAN and Orange VLAN
Switches forward traffic as needed
13
Making VLANs Work
• Changing the Ethernet header
– Adding a field for a VLAN tag
– Implemented on the bridges/switches
– … but can still interoperate with old Ethernet cards
• Bridges/switches trunk links
– Saying which VLANs are accessible via which interfaces
• Approaches to mapping access links to VLANs
– Each interface has a VLAN color
– Each MAC address has a VLAN color
14
Uses of VLANs (See the Survey
Paper)
• Scoping broadcast traffic
• Simplifying access control policies
• Decentralizing network management
• Enabling host mobility
15
Problem: Limited Granularity
• Limited number of VLANs
– Placing multiple groups in the same VLAN
– Reusing limited VLAN
• Limited number of hosts per VLAN
– Divide a large group into multiple VLANs
• One VLAN per access port
– Supporting VLANs on the end host
– Supporting multiple groups at the router
16
Problem: Complex Configuration
• Host address assignment
– Wasting IP addresses
– Complex host address assignment
• Spanning tree computation
– Limitation of automated trunk configuration
– Enabling extra links to survive failures
– Distributing load over the root bridges
Open question: can we do better than VLANs?
17
Multiple Internet Connections
18
Motivation for Multi-Homing
• Benefits of multi-homing
– Extra reliability, e.g., survive single ISP failure
– Financial leverage through competition
– Better performance by selecting better path
– Gaming the 95th-percentile billing model
ISP 1
ISP 2
1.2.3.0/24
19
Multi-Homing Without BGP
Inbound Traffic
• Ask each ISP to originate the
IP prefix
• … to rest of the Internet
ISP 1
Outbound Traffic
• One ISP as a primary, the
other as a backup
• Or simple load balancing of
all traffic
ISP 2
1.2.3.0/24
20
Multi-Homing With BGP
• Inbound traffic
• Outbound traffic
– Originate the prefix to
both providers
– Do not allow traffic from
one ISP to another
ISP 1
– Select the “best” route for
each remote prefix
– Define BGP policies based
on load, performance, cost
ISP 2
BGP sessions
1.2.3.0/24
“Intelligent route
control” or “multi-homed
traffic engineering”.
21
Interconnecting Multiple
Enterprise Sites
22
Challenges
• Challenges of interconnecting multiple sites
– Performance
– Reliability
– Security
– Privacy
• Solutions
– Connecting via the Internet using secure tunnels
– Virtual Private Network (VPN) service
– Dedicated backbone between sites
23
Connecting Via the Internet
• Each site connects to the Internet
– Encrypted tunnel between each pair of sites
– Packet filtering to block unwanted traffic
– But, no performance or reliability guarantees
Site 1
Internet
Site 2
Site 3
24
Virtual Private Network (VPN)
• Each site connects to a common VPN provider
– Provider allows each site to announce IP prefixes
– Separate routing/forwarding table for each customer
– Performance guarantees
Site 1
VPN Provider
Site 2
Site 3
25
Middleboxes
26
Enterprise Internet Connection
• Multiple middleboxes
– Intrusion prevention system
– Network address translator
– Firewall
– Traffic shaper
• Handling bad internal users
– Filtering IP packets with
spoofed source IP addresses
– Logging which MAC address
has each IP address
Internet
M
link
M
M
R
27
Internal Middleboxes
• Network divided into regions
– E.g., departments within a campus
– E.g., public computers (servers, WiFi) vs. private
• Network divided by roles
– E.g., human resources vs. engineering
– E.g., faculty vs. students
• Sometimes physically separate networks
– E.g., ATM machines, campus safety, media streaming
28
Princeton Campus Network
http://www.net.princeton.edu/index.html
http://www.net.princeton.edu/statistics/
http://www.net.princeton.edu/whatsnew.html
29
Internet Connections
Comcast
(full routes)
WindStream
(full routes)
ESnet
1 Gbps
MAGPI/Internet2
1 Gbps
1 Gbps
vgate1
1 Gbps
Princeton Campus
• Two commercial ISPs: Comcast and WindStream
• Two research networks: ESnet and Internet2
• Non-profits: McCarter Theater, Princeton Public
Library, and Princeton Regional Schools
30
Princeton Public Internet Traffic
• Traffic volumes over the past week
– Green: traffic from the Internet
– Blue: traffic to the Internet
31
Three Internal Networks
• Campus Data Network
– Connects dorms, academic and administrative
buildings, campus WiFi, etc.
• Princeton Private Network
– Environmental systems, power, security cameras,
building locks
• VoIP Network
– VoIP phones in data center, chemistry, neuroscience,
Forrestal campus, and all new construction
– Separate for disaster recovery & traffic management
32
Campus Data Network
Internet
vgate1
RIP
routing
Dormnet
Academic,
administrative
gigasw2
gigasw4
gigagate1
Applications
Data Center
33
Data Center (Forrestal Campus)
• 40,000 square feet with 1800 computers
• Multiple tiers of backup power
• Minimizes energy for cooling and power
34
Virtual Private Network (VPN)
• Online campus resources
– E.g., some Princeton University library resources
– Not available from outside of campus
• External resources with Princeton subscription
– E.g., digital libraries from ACM and IEEE
– Accessible from a Princeton IP address
• Princeton VPN service (vpn.princeton.edu)
– Secure network connection layered over IP network
– … connects you to an internal Princeton machine
35
Aruba WiFi Access Points
• Adaptive radio management
– Automatically assigns channel and power settings
– Channel load balancing to distribute clients
– Coverage hole detection
36
WiFi Anecdote (puwireless)
• Single large VLAN
– Enabling seamless mobility on campus
• Limited address space
– 16K or 32K IP addresses
– 3 hour DHCP leases
• Frequently a large number of users
– Several thousand to up to 10,000
– … may soon run low on IP addresses
37
WiFi Anecdote (puwireless), Continued
• Bug in Android and IOS smart phones
– Don’t release DHCP lease on IP addresses
– Offloads ARP processing to the chipset, to avoid
waking up sleeping device on ARP requests
– … but DHCP timeout is handled by the processor
• So, can have IP address collisions
– DHCP lease expires, but the phone doesn’t know
– DHCP server gives the IP address to someone else
– … and both devices respond to ARP requests!
http://www.net.princeton.edu/android/android-stopsrenewing-lease-keeps-using-IP-address-11236.html
38
WiFi Anecdote (puwireless), Continued
• Working with Google and Apple on the problem
• Longer-term solution
– Move to larger, private address block (10.0.0.0/8)
– Use network address translation (NAT) to
communicate with the public Internet
• Benefits
– Avoids running out of IP addresses
– Introduces long delay before reusing an address
– Seems like a good solution, right?
39
WiFi Anecdote (puwireless), Continued
• Solution makes troubleshooting harder
– Public IP addresses shared by many users
– … due to network address translation
• Example: DMCA violations
– Student downloads copyrighted material on WiFi
– Company comes to Princeton to complain
• Given IP address, can OIT identify the student?
– With NAT, cannot pinpoint a unique MAC address
– … without much more detailed (flow-level) logs
40
Conclusions
• Enterprise networks
– Campuses and companies
– Access to local services and the Internet
• Challenges
– IP address limitations
– Hybrid switch and routed network
– Load balancing over upstream ISPs
– Protecting users and the Internet from each other
• Next time: data-center networks
41