Multilayer Campus
Architecture
and Design Principles
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
1
Enterprise-Class Availability
Resilient
R
ili t C
Campus C
Communication
i ti F
Fabric
bi
Campus Systems Approach to High Availability
 Network-level redundancy
 System-level
y
resiliency
y
 Enhanced management
 Human
u a ea
ear notices
ot ces tthe
e
difference in voice within
150–200 msec—10
consecutive G711 packet loss
Ultimate Goal……………..100%
Next-Generation Apps
Video conf., Unified Messaging,
Global Outsourcing,
E-Business
E
Business, Wireless Ubiquity
Mission Critical Apps.
Databases, Order-Entry,
CRM, ERP
 Video loss is even more
noticeable
 200 msec end-to end-campus
convergence
Desktop Apps
E-mail, File & Print
APPLICATIONS DRIVE REQUIREMENTS
FOR HIGH AVAILABILITY NETWORKING
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
2
Next Generation Campus Design
U ifi d C
Unified
Communications
i ti
E
Evolution
l ti
 VoIP is now a mainstream technology
 Ongoing evolution to the full spectrum of Unified Communications
 High
High-Definition
Definition Executive Communication Application requires
stringent Service-Level Agreement (SLA)
Reliable Service—High Availability Infrastructure
Application Service Management—QoS
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
3
Agenda
Data Center
Services
Block
 Multilayer Campus
Design Principles
 Foundation Services
 Campus Design
Best Practices
 IP Telephony
Considerations
Si
Si
 QoS Considerations
 Security
Considerations
Si
 Putting It All Together
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Si
Si
Si
Si
Distribution Blocks
 Summary
BRKRST-2031
14457_04_2008_c2
Si
Si
Cisco Public
4
High-Availability Campus Design
Structure Modularity
Structure,
Modularity, and Hierarchy
Access
Si
Distribution
Si
Core
Distribution
Access
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Si
Si
WAN
Cisco Public
Si
Si
Si
Si
Si
Si
Si
S
Data Center
Si
Si
Internet
5
Hierarchical Campus Network
Structure, Modularity and Hierarchy
Not This!!
Si
Si
Si
Si
Si
Si
Si
Si
Si
Si
Si
Si
Server Farm
WAN
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Internet
PSTN
6
Hierarchical Network Design
Without a Rock Solid Foundation the Rest Doesn’t Matter
Access
 Offers hierarchy—each layer has
specific role
 Modular topology—building
topology building blocks
Distribution
 Easy to grow, understand, and
troubleshoot
Si
Si
 Creates small fault domains—
domains
clear demarcations and isolation
Core
 Promotes load balancing and
y
redundancy
Si
Si
 Promotes deterministic traffic
patterns
Distribution
Access
BRKRST-2031
14457_04_2008_c2
p
balance of both Layer
y 2
 Incorporates
and Layer 3 technology, leveraging
the strength of both
 Utilizes Layer 3 routing for load
balancing fast convergence,
balancing,
convergence
scalability, and control
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Si
Si
Building Block
7
Access Layer
Feature Rich Environment
 It
It’ss not just about connectivity
 Layer 2/Layer 3 feature rich environment;
convergence, HA, security, QoS, IP
multicast, etc.
Core
Si
 Intelligent network services: QoS,
trust boundary, broadcast suppression,
IGMP snooping
 Intelligent
g
network services: PVST+,
Rapid PVST+, EIGRP, OSPF, DTP,
PAgP/LACP, UDLD, FlexLink, etc.
Si
Distribution
Si
Si
 Cisco Catalyst integrated security
features IBNS (802.1x), (CISF):
port security, DHCP snooping,
DAI, IPSG, etc.
 Automatic phone discovery,
conditional trust boundary
boundary,
power over Ethernet, auxiliary
VLAN, etc.
Access
 Spanning tree toolkit: PortFast,
UplinkFast BackboneFast
UplinkFast,
BackboneFast, LoopGuard
LoopGuard,
BPDU Guard, BPDU Filter, RootGuard, etc.
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
8
Distribution Layer
Policy, Convergence, QoS, and High Availability
 Availability
Availability, load balancing
balancing,
QoS and provisioning are the
important considerations at
this layer
y
Core
Si
Si
 Aggregates wiring closets
(access layer) and
uplinks to core
 Protects core from high
density peering and
problems in access layer
 Route summarization, fast
convergence, redundant
path load sharing
Distribution
Si
Si
Access
 HSRP or GLBP to provide
first hop
p redundancy
y
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
9
Core Layer
Scalability, High Availability, and Fast Convergence
 Backbone for the
network—connects
network building blocks
Core
Si
Si
 Performance and
stability vs. complexity—
less is more in the core
Distribution
 Aggregation point for
distribution layer
Si
 Separate core layer
helps in scalability
during future growth
Si
Access
 Keep the design
technology-independent
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
10
Do I Need a Core Layer?
It’s Really a Question of
Scale Complexity
Scale,
Complexity, and Convergence
No Core
 Fully meshed distribution layers
 Physical
Ph i l cabling
bli
requirement
 Routing complexity
Second Building
Block–4 New Links
4th Building Block
12 New Links
24 Links Total
3rd Building Block
8 New Links
12 Links Total
8 IGP Neighbors
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
5 IGP Neighbors
Cisco Public
11
Do I Need a Core Layer?
It’s Really a Question of
Scale Complexity
Scale,
Complexity, and Convergence
Dedicated Core Switches




Easier to add a module
Fewer links in the core
Easier bandwidth upgrade
Routing protocol peering
reduced
 Equal
q
cost Layer
y 3 links
for best convergence
2nd Building Block
8 New Links
4th Building Block
4 New Links
16 Links Total
3rd Building Block
4 New Links
12 Links Total
3 IGP Neighbors
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
3 IGP Neighbors
Cisco Public
12
Design Alternatives Come Within a
Building (or Distribution) Block
Layer 2 Access
Routed Access Virtual Switching System
Access
Si
Distribution
Si
Core
Distribution
Access
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Si
Si
Si
Si
Si
Si
WAN
Cisco Public
Si
S
Data Center
Si
Si
Internet
13
Layer 3 Distribution Interconnection
Layer 2 Access—No VLANs Span Access Layer
 Tune CEF load balancing
 Match CatOS/IOS EtherChannel
settings and tune load balancing
 Summarize routes towards core
 Limit redundant IGP peering
 STP Root and HSRP primary
g or GLBP to load balance
tuning
on uplinks
 Set trunk mode on/no-negotiate
 Disable EtherChannel
unless needed
 Set port host on access
layer ports:
Disable
Trunking
Di bl T
ki
Disable EtherChannel
Enable PortFast
 RootGuard or BPDU-Guard
 Use security features
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Si
Layer 3
Si
Point
to
Point
Link
Core
Distribution
Si
Access
VLAN 20 Data
10.1.20.0/24
VLAN 120 Voice
10.1.120.0/24
Cisco Public
VLAN 40 Data
10.1.40.0/24
VLAN 140 Voice
10.1.140.0/24
14
Layer 2 Distribution Interconnection
Layer 2 Access—Some VLANs Span Access Layer
 Tune CEF load balancing
 Match CatOS/IOS EtherChannel
settings and tune load balancing
 Summarize routes towards core
 Limit redundant IGP peering
 STP Root and HSRP primary or
GLBP and STP port cost tuning to
load balance on uplinks
 Set trunk mode on/no-negotiate
 Disable EtherChannel
unless needed
 RootGuard on downlinks
 LoopGuard on uplinks
 Set port host on access
Layer ports:
Disable Trunking
Disable EtherChannel
Enable PortFast
 RootGuard or BPDU-Guard
 Use security features
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Si
Layer 2
Si
Trunk
Si
VLAN 20 Data
VLAN 40 Data
10.1.20.0/24
10.1.40.0/24
VLAN 120 Voice
VLAN 140 Voice
10.1.120.0/24
10.1.140.0/24
VLAN 250 WLAN
10.1.250.0/24
Cisco Public
Core
Distribution
Access
15
Routed Access and
Virtual Switching System
Evolutions of and Improvements to Existing Designs
Si
Si
Si
Si
Core
VSS Link
Si
Layer 3
New
Concept
Si
Distribution
P-t-P Link
VLAN 20 Data
10.1.20.0/24
VLAN 120 Voice
10.1.120.0/24
VLAN 20 Data
10.1.20.0/24
VLAN
40 Data
10.1.40.0/24
VLAN 120 Voice
10.1.120.0/24
VLAN
140 Voice
10
10.1.140.0/24
1
140
0/24
VLAN 250 WLAN
10.1.250.0/24
VLAN 40 Data
10.1.40.0/24
VLAN 140 Voice
10.1.140.0/24
Access
See RST-3035—Advanced Enterprise Campus Design Alternatives: Routed Access and Virtual Switching System (VSS)
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
16
Virtual Switch System (VSS)
Hub and Spoke VLANs can Span Access Layer
 Tune CEF load balancing
 Match CatOS/IOS EtherChannel
settings and tune load balancing
 Summarize routes towards core
 Set trunk mode on/nonegotiate
Multi Chassis
 Use PaGP and Multi-Chassis
EtherChannel
 RootGuard on downlink (MEC)
 LoopGuard on uplink (MEC)
 Set port host on access
Layer ports:
Si
Si
VSS Link
New
Concept
Si
Si
– Disable trunking
Disable EtherChannel
Enable
PortFast
E bl P
tF t
 RootGuard or BPDU-Guard on VLAN 20 Data
10.1.20.0/24
access ports
VLAN 120 Voice
 Use securityy features
10 1 120 0/24
10.1.120.0/24
VLAN 40 Data
10.1.40.0/24
VLAN 140 Voice
10 1 140 0/24
10.1.140.0/24
VLAN 250 WLAN
10.1.250.0/24
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Core
Distribution
Access
17
Virtual Switching System 1440
Network System Virtualization
Core/Distribution
Si
Si
Data Center Access
Si
Si
Si
Si
Si
Si
Features
Benefits of VSS
Network System Virtualization
Increased Operational Efficiency
via
i Simplified
Si lifi d Network
N t
k
Inter-Chassis Stateful Switch
Over ((SSO))
Boost Non-stop Communication
Multi-Chassis EtherChannel
(MEC)
Scale the System Bandwidth
Capacity to 1.4 Tbps
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
18
Agenda
Data Center
Services
Block
 Multilayer Campus
Design Principles
 Foundation Services
 Campus Design
Best Practices
 IP Telephony
Considerations
Si
Si
 QoS Considerations
 Security
Considerations
Si
 Putting It All Together
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Si
Si
Si
Si
Distribution Blocks
 Summary
BRKRST-2031
14457_04_2008_c2
Si
Si
Cisco Public
19
Foundation Services
 Layer 1 physical things
 Layer 2 redundancy—spanning tree
 Layer 3 routing protocols
 Trunking protocols—(ISL/.1q)
 Unidirectional link detection
 Load balancing
g
EtherChannel link aggregation
CEF equal cost load balancing
HSRP
 First hop redundancy protocols
VRRP, HSRP, and GLBP
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Routing
Spanning
Tree
20
Best Practices—
Layer 1 Physical Things
 Use point
point-to-point
to point
interconnections—no
L2 aggregation points
between nodes
 Use fiber for best
convergence
(debounce timer)
Si
Si
Si
Si
Si
Si
Cisco Public
Si
Si
WAN
Si
Layer 3 Equal
Cost Links
Si
 Use configuration on
the physical interface
not VLAN/SVI when
possible
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Layer 3 Equal
Cost Links
 Tune carrier
delay timer
BRKRST-2031
14457_04_2008_c2
Si
Si
Si
Data Center
Internet
21
Redundancy and Protocol Interaction
Li k N
Link
Neighbour
i hb
F
Failure
il
D
Detection
t ti
 Indirect link failures are harder
to detect
 With no direct HW notification of link
loss or topology change convergence
times are dependent on SW notification
Hellos
Si
Si
Hub
Si
 Indirect failure events in a bridged
environment are detected by Spanning
Tree Hellos
 In certain topologies the need for TCN
updates or dummy multicast flooding
(uplink fast) is necessary for
convergence
 You should not be using hubs in a high
availability
il bilit d
design
i
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
BPDU
BPDUs
Si
Si
Hub
Si
S
22
Redundancy and Protocol Interaction
Li k Redundancy
Link
R d d
and
dF
Failure
il
D
Detection
t ti
 Direct point-to-point fiber provides for fast
failure detection
 IEEE 802.3z and 802.3ae link negotiation
define the use of Remote Fault Indicator and
Link Fault Signaling mechanisms
 Bit D13 in the Fast Link Pulse (FLP) can
be set to indicate a physical fault to the
remote side
 Do not disable auto-negotiation on GigE and
10GigE interfaces
 The default debounce timer on GigE and
10GigE fiber linecards is 10 msec
 The minimum debounce for
f copper is
300 msec
 Carrier-Delay
3560, 3750 and 4500—0 msec
6500—leave it set at default
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
3
2
Cisco IOS Throttling:
C i D
Carrier
Delay
l Ti
Timer
Linecard
Throttling:
Debounce Timer
1
1
Si
Remote IEEE
Fault Detection
Mechanism
Si
23
Redundancy and Protocol Interaction
L
Layer
2 and
d 3—Why
3 Wh Use
U R
Routed
t d IInterfaces
t f
 Configuring L3 routed interfaces provides for faster convergence
than an L2 switch port with an associated L3 SVI
L2
L3
Si
Si
Si
1. Link Down
1.
Link Down
2. Interface Down
2.
Interface Down
3. Routing Update
3.
Autostate
4
4.
SVI Down
5.
Routing Update
~ 8 msec
loss
~ 150
150-200
200
msec loss
21:38:37.042 UTC: %LINEPROTO-5-UPDOWN: Line
protocol on Interface GigabitEthernet3/1, changed
state to down
21:38:37.050 UTC: %LINK-3-UPDOWN: Interface
GigabitEthernet3/1, changed state to down
21:38:37.050 UTC: IP-EIGRP(Default-IP-RoutingTable:100): Callback: route_adjust
GigabitEthernet3/1
BRKRST-2031
14457_04_2008_c2
Si
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
21:32:47.813 UTC: %LINEPROTO-5-UPDOWN:
%LINEPROTO 5 UPDOWN: Line
protocol on Interface GigabitEthernet2/1, changed state
to down
21:32:47.821 UTC: %LINK-3-UPDOWN: Interface
GigabitEthernet2/1, changed state to down
21:32:48.069 UTC: %LINK-3-UPDOWN: Interface Vlan301,,
changed state to down
21:32:48.069 UTC: IP-EIGRP(Default-IP-RoutingTable:100): Callback: route, adjust Vlan301
24
Best Practices—
Spanning Tree Configuration
 Only span VLAN across
multiple access layer
switches when you have to!
 Use Rapid PVST+ for best
convergence
 More common in the
d
data
center
 Required to protect against
‘user side’ loops
p
 Required to protect against
operational accidents
(misconfiguration or
hardware failure)
 Take advantage of the
spanning tree toolkit
Layer 2 Loops
Si
Si
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Si
Si
Si
Si
Si
Si
Si
Si
Layer 3 Equal
Cost Links
Layer 3 Equal
Cost Links
Si
Si
WAN
BRKRST-2031
14457_04_2008_c2
Same VLAN
Same VLAN
Same VLAN
Si
Si
Data Center
Internet
25
Multilayer Network Design
Layer 2 Access with Layer 3 Distribution
Si
Vlan 10
Si
Vlan 20
Si
Vlan 30
 Each access switch has
unique VLANs
 No layer 2 loops
 Layer 3 link between distribution
 No blocked links
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Vlan 30
Si
Vlan 30
Vlan 30
 At least some VLANs span
multiple access switches
 Layer 2 loops
 Layer 2 and 3 running over
link between distribution
 Blocked links
26
Optimizing L2 Convergence
PVST+, Rapid PVST+ or MST
 Rapid-PVST+ greatly improves the restoration times for any VLAN that
requires a topology convergence due to link UP
 Rapid-PVST+ also greatly improves convergence time over backbone fast
for any indirect link failures
Traditional spanning
p
g tree
implementation
 Rapid PVST+ (802.1w)
Scales to large size
(~10,000 logical ports)
Easy to implement, proven, scales
 MST (802.1s)
(802 1s)
Permits very large scale STP
implementations
(~30,000 logical ports)
Time to Restore Data
T
a Flows (se
ec)
 PVST+ (802.1d)
Not as flexible as Rapid PVST+
http://www.cisco.com/en/US/products/hw/switches/ps708/products_co
nfiguration_example09186a00807b0670.shtml
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
35
30
Upstream
25
Downstream
20
15
10
5
0
PVST+
Rapid PVST+
27
Layer 2 Hardening
Spanning Tree Should Behave
the Way You Expect
 Place the root where you
want it
LoopGuard
STP Root
Root primary/secondary macro
 The root bridge should stay
where you put it
Si
Si
RootGuard
RootGuard
LoopGuard
LoopGuard
UplinkFast
UDLD
 Only end-station traffic
should be seen on
an edge
g p
port
UplinkFast
BPDU Guard
BPDU Guard or
RootGuard
PortFast
Port Security
RootGuard
PortFast
Port-security
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
28
Best Practices—
Layer 3 Routing Protocols
 Typically deployed in distribution
to core, and core to core
interconnections
 Used to quickly re-route
around
d ffailed
il d node/links
d /li k while
hil
providing load balancing over
redundant paths
g
not squares
q
for
 Build triangles
deterministic convergence
 Only peer on links that you intend
to use as transit
 Insure redundant L3 paths to
avoid black holes
 Summarize distribution to core
to limit EIGRP query diameter or
OSPF LSA propagation
 Tune CEF L3/L4 load balancing
hash to achieve maximum
tili ti off equall costt paths
th
utilization
(CEF polarization)
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Si
Si
Si
Si
Si
Layer 3 Equal
Cost Links
Layer 3 Equal
Cost Links
Si
Si
Si
Si
Si
Si
WAN
Si
Si
Si
Data Center
Internet
29
Best Practice—
Build Triangles Not Squares
Deterministic vs. Non-Deterministic
Triangles: Link/Box Failure Does NOT
Require Routing Protocol Convergence
Si
Si
Si
Si
Model A
Squares: Link/Box Failure Requires
Routing Protocol Convergence
Si
Si
Si
Si
Model B
 Layer 3 redundant equal cost links support fast convergence
 Hardware based—fast recovery to remaining path
 Convergence is extremely fast (dual equal-cost paths: no need for OSPF or
EIGRP to recalculate a new path)
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
30
Best Practice—
Passive Interfaces for IGP
Limit OSPF and EIGRP Peering
Through the Access Layer
 Limit unnecessary peering using
passive interface:
Distribution
Si
Si
Routing
Updates
F
Four
VLANs
VLAN per wiring
i i closet
l
t
12 adjacencies total
Memory and CPU requirements
increase with no real benefit
Access
Creates overhead for IGP
OSPF Example:
EIGRP Example:
Router(config)#routerospf 1
Router(config-router)#passiveinterfaceVlan 99
Router(config)#routereigrp 1
Router(config-router)#passiveinterfaceVlan 99
Router(config)#routerospf 1
Router(config-router)#passiveinterface default
Router(config-router)#no passiveinterface Vlan 99
Router(config)#routereigrp 1
Router(config-router)#passiveinterface default
Router(config-router)#no passiveinterface Vlan 99
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
31
Why You Want to Summarize
at the Distribution
Limit EIGRP Queries and OSPF LSA Propagation
 It is important to force
summarization at the
distribution towards the core
 For return path traffic an
OSPF or EIGRP re-route
t
is required
 By limiting the number of peers
query
y or
an EIGRP router must q
the number of LSAs an OSPF
peer must process we can
optimize this re-route
 EIGRP example:
interface Port-channel1
description to Core#1
ip address 10.122.0.34
255.255.255.252
ip hello-interval eigrp 100 1
ip hold-time eigrp 100 3
ip summary-address eigrp 100
10.1.0.0 255.255.0.0 5
No Summaries
Q
Queries
i Go
G Beyond
B
d the
th Core
C
Rest of Network
Core
Si
Distribution
Si
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Si
Access
10.1.1.0/24
BRKRST-2031
14457_04_2008_c2
Si
10.1.2.0/24
32
Why You Want to Summarize
at the Distribution
Reduce the Complexity of IGP Convergence
 It is important to force
summarization at the distribution
towards the core
 For return path traffic an OSPF
or EIGRP re-route
t is
i required
i d
 By limiting the number of peers
an EIGRP router must query or
the number of LSAs an OSPF
|peer must process we can
optimize his re-route
 For EIGRP if we summarize at
the distribution we stop queries
at the core boxes for an access
layer ‘flap’
 For OSPF when we summarize
at the distribution (area border
or L1/L2 border) the flooding of
LSAs is limited to the distribution
switches;; SPF now deals with
one LSA not three
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Summaries
St Q
Stop
Queries
i att the
th Core
C
Rest of Network
Core
Si
Si
Summary:
10.1.0.0/16
Si
Distribution
Si
Access
10.1.1.0/24
10.1.2.0/24
33
Best Practice—
Summarize at the Distribution
Gotcha—Distribution-to-Distribution Link Required
 Best practice—summarize at
the distribution layer to limit
EIGRP queries or OSPF
LSA p
propagation
p g
 Gotcha:
Upstream: HSRP on left
distribution takes over when
link fails
Core
Si
Si
Summary:
10.1.0.0/16
Distribution
Return path: old router still
advertises summary to core
Si
Return traffic is dropped on
right distribution switch
Si
 Summarizing requires a link
between the distribution switches
 Alternative design:
Use the access layer for transit
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Access
10.1.1.0/24
10.1.2.0/24
34
Provide Alternate Paths
 What happens if
fails?
 No route to the core
anymore?
Si
Core
Si
Single
Si
l Path
P th
to Core
 Allow the traffic to go
through the access?
Do you want to use your access
switches
it h as ttransit
it nodes?
d ?
How do you design for
scalability if the access used
for transit traffic?
Si
Distribution
Si
 Install a redundant link
to the core
 Best practice: install
redundant link to core
and utilize L3 link
between
bet
ee d
distribution
st but o Layer
aye
(summarization—coming)
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Access
A
B
35
EIGRP Design Rules in the Campus
L
Leverage
th
the T
Tools
l P
Provided
id d
 The greatest advantages of EIGRP are
gained when the network has a
structured addressing plan that allows
for use of summarization and stub
routers when appropriate
10.10.0.0/16
 EIGRP provides the ability to
implement multiple tiers of
summarization and route filtering
 Minimize the number and time for
query response to speed up
convergence
Si
Si
10.10.128.0/17
10.10.0.0/17
 Summarize distribution block routes
upstream to the core
 If routing in the access configure
all access switches as EIGRP
stub routers
Si
Si
Si
Si
 If routing in the access layer filter
routes sent down to access switches
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
36
OSPF Design Rules in the Campus
Wh
Where
Are
A th
the Areas?
A
?
 Area design based on address
summarization
 Area boundaries should define buffers
between fault domains
 Summarize routes from the distribution
block upstream into the core
 Minimize the number of LSAs and
routes in the core
 Reduce the need for SPF calculations
due to internal distribution block
changes
 ABR for a regular area forwards
Summary LSAs (Type 3)
ASBR summary (Type 4)
Specific externals (Type 5)
Area 100
Area 110
Area 120
Si
Si
Si
Si
Si
Si
Area 0
Si
Si
Si
Si
Si
Si
Si
Si
 Stub area ABR forwards
Summary LSAs (Type 3)
Summary default (0.0.0.0)
 A totally stubby area ABR forwards
WAN
Data Center
Internet
Summary default (0.0.0.0)
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
37
Equal Cost Multi
Multi-Path
Path
Optimizing CEF Load-Sharing
 Depending on the traffic flow patterns and IP
Addressing in use one algorithm may provide
better load-sharing results than another
 Be careful not to introduce polarization in a multimulti
tier design by changing the default to the same
thing in all tiers/layers of the network
Si
30%
of
Flows
Si
70%
of
Flows
Si
C t l t 4500 Load-Sharing
Catalyst
L d Sh i Options
O ti
Original
Src IP + Dst IP
Universal*
Src IP + Dst IP + Unique ID
Include
P t
Port
Src IP + Dst IP + (Src or Dst Port) + Unique ID
Load-Sharing
Simple
Catalyst 6500 PFC3** Load-Sharing Options
Default*
Src IP + Dst IP + Unique ID
Full
Src IP + Dst IP + Src Port + Dst Port
Full Exclude Port
Src IP + Dst IP + (Src or Dst Port)
Simple
Src IP + Dst IP
Full Simple
Src IP + Dst IP + Src Port + Dst Port
Load-Sharing
Full Simple
Si
Si
Si
Si
Load-Sharing
Simple
Si
* = Default Load-Sharing Mode
** = PFC3 in Sup720 and Sup32 Supervisors
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
38
CEF Load Balancing
Avoid Underutilizing Redundant Layer 3 Paths
Redundant Paths Ignored
Distribution
Default L3 Hash
Si
L
Core
Default L3 Hash
Distribution
Default L3 Hash
BRKRST-2031
14457_04_2008_c2
Si
Si
R
Si
© 2008 Cisco Systems, Inc. All rights reserved.
 Imbalance/overload
could occur
 Redundant paths are
ignored/underutilized
g
 The default CEF hash
‘input’ is L3
L
R
Si
 CEF polarization: without
some tuning CEF will select
the same p
path left/left or
right/right
Si
Cisco Public
 We can change the default to
use L3 + L4 information as
‘input’
input to the hash derivation
39
CEF Load Balancing
Avoid Underutilizing Redundant Layer 3 Paths
All Paths Used
Distribution
L3/L4 Hash
Si
L R
Core
Default L3 Hash
Distribution
L3/L4 Hash
BRKRST-2031
14457_04_2008_c2
Si
L
Si
L R
Si
R
Si
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Cisco Public
 The default will for
Sup720/32 and latest
hardware ((unique
q ID added to
default). However, depending
on IP addressing, and flows
imbalance could occur
 Alternating L3/L4 hash and L3
hash will give us the best load
lts
balancing res
results
 Use simple in the core and
full simple in the distribution
to add L4 information to the
algorithm at the distribution
and maintain differentiation
tier-to-tier
40
Best Practices
Practices—Trunk
Trunk Configuration
 Typically deployed on
interconnection between
access and distribution layers
 Use VTP transparent mode
to decrease potential for
operational error
 Hard set trunk mode to on and
encapsulation negotiate off for
optimal convergence
 Change the native VLAN to
something unused to avoid
VLAN hopping
 Manually prune all VLANS
exceptt those
needed
th
d d
 Disable on host ports:
CatOS: set port host
Cisco IOS: switchport host
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
802.1q Trunks
Si
Si
Si
Si
Si
Layer 3 Equal
Cost Links
Layer 3 Equal
Cost Links
Si
Si
Si
Si
Si
Si
WAN
Si
Si
Si
Data Center
Internet
41
VTP Virtual Trunk Protocol
 Centralized VLAN
management
Set
VLAN 50
Trunk
 VTP server switch
propagates VLAN
database to VTP
client switches
A
F
Server
Trunk
Transparent
Trunk
 Runs only on trunks
Ok, I Just
Learnt
VLAN 50!
 Four modes:
Server: updates clients
and servers
Client
Transparent: let updates
pass through
Off: ignores VTP updates
© 2008 Cisco Systems, Inc. All rights reserved.
Client
Ok, I Just
L
Learnt
t
VLAN 50!
B
Trunk
Client: receive updates—
cannot make changes
BRKRST-2031
14457_04_2008_c2
Pass
Through
Update
Cisco Public
Drop
VTP
Updates
Off
C
42
DTP Dynamic Trunk Protocol
 Automatic formation of
trunked switch-to-switch
interconnection
On: always
O
l
b
be a ttrunk
k
Desirable: ask if the other side can/will
Auto: if the other sides asks I will
Off: don’t become a trunk
Si
On/On
Trunk
Si
Si
Auto/Desirable
Trunk
Si
Si
Off/Off
NO Trunk
 Negotiation of 802.1Q or
ISL encapsulation
p
ISL: try to use ISL trunk encapsulation
802.1q: try to use 802.1q
encapsulation
p
Negotiate: negotiate ISL or 802.1q
encapsulation with peer
Non-negotiate: always use
encapsulation that is hard set
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Si
Si
Si
Off/On, Auto, Desirable
NO Trunk
43
Optimizing Convergence: Trunk Tuning
Trunk Auto/Desirable Takes Some Time
 DTP negotiation tuning improves link up convergence time
CatOS> (enable) set trunk <port> nonegotiate dot1q <vlan>
IOS(config-if)# switchport mode trunk
IOS(config-if)# switchport nonegotiate
Time to Co
onverge in S
Seconds
25
2.5
2
1.5
Two Seconds
of Delay/Loss
T
Tuned
d Away
A
1
0.5
Voice Data
0
BRKRST-2031
14457_04_2008_c2
Si
Trunking Desirable
© 2008 Cisco Systems, Inc. All rights reserved.
Trunking Nonegotiate
Cisco Public
44
Trunking/VTP/DTP Quick Summary
Trunking/VTP/DTP—Quick
 VTP Transparent should be used; there is a trade off
between administrative overhead and the temptation to
span existing VLANS across multiple access layer switches
 Emerging technologies that do VLAN assignment
b name (IBNS
by
(IBNS, NAC
NAC, etc.)
t ) require
i a unique
i
VLAN
database per access layer switch if the rule: A VLAN
= A Subnet = AN access layer switch is going to
be followed
 One can consider a configuration that uses DTP
ON/ON and NO NEGOTIATE; there is a trade off
between performance/HA impact and maintenance
and operations implications
 An ON/ON and NO NEGOTIATE configuration is
faster from a link up (restoration) perspective than
a desirable/desirable alternative. However, in this
configuration DTP is not actively monitoring the
state of the trunk and a misconfigured trunk is not
easily identified.
g
 It’s reallyy a balance between fast convergence
and your ability to manage configuration and
change control …
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
45
Best Practices
Practices—UDLD
UDLD Configuration
 Typically deployed
on any fiberoptic
interconnection
 Use UDLD aggressive
mode for best protection
 Turn on in global
configuration to
avoid operational
error/“misses”
Si
Si
Si
Si
Si
Fiber Interconnections
Layer 3 Equal
Cost Links
Layer 3 Equal
Cost Links
Si
Si
 Config example
Si
Si
Si
Si
Si
Si
Si
Cisco IOS:
udld aggressive
WAN
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Data Center
Internet
46
Unidirectional Link Detection
Protecting Against One Way Communication
 Highly-available
Highly available networks require UDLD to
protect against one-way communication or
partially failed links and the effect that they could
have on protocols like STP and RSTP
 Primarily used on fiberoptic links where patch
panel errors could cause link up/up with
mismatched transmit/receive pairs
Si
 Each switch port configured for UDLD will send
UDLD protocol packets (at L2) containing the
port’s own device/port ID, and the neighbor’s
device/port IDs seen by UDLD on that port
Are You
‘Echoing’
M Hellos?
My
H ll ?
 Neighboring ports should see their own
device/port ID (echo) in the packets received
from the other side
 If the port does not see its own device/port ID in
the incoming UDLD packets for a specific
duration of time, the link is considered
unidirectional and is shutdown
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Si
47
UDLD Aggressive and UDLD ‘Normal’
Normal
Si
Si
 Timers are the same—15
same 15 second hellos by default
 Aggressive Mode—after aging on a previously bi-directional link—
tries 8 times (once per second) to reestablish connection then
err-disables port
 UDLD—Normal Mode—Only err-disable the end where UDLD
detected other end just sees the link go down
 UDLD—Aggressive—err-disable BOTH ends of the connection
due to err-disable when aging
g g and re-establishment of UDLD
communication fails
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
48
Best Practices—
EtherChannel Configuration
 Typically deployed in
distribution to core, and core
to core interconnections
 Used to provide link
redundancy—while reducing
peering complexity
 Tune L3/L4 load balancing
g
hash to achieve maximum
utilization of channel members
 Deploy in powers of 2 (2, 4, or 8)
 Match CatOS and Cisco IOS
PAgP settings
p
 802.3ad LACP for interop
if you need it
 Disable unless needed
CatOS: set port
host
p
Cisco IOS: switchport host
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Si
Si
Si
Si
Si
Layer 3 Equal
Cost Links
Layer 3 Equal
Cost Links
Si
Si
Si
Si
Si
Si
WAN
Si
Si
Si
Data Center
Internet
49
Understanding EtherChannel
Li k Negotiation
Link
N
ti ti O
Options—PAgP
ti
PA P and
d LACP
Packet Aggregation Protocol
Si
Si
Si
Si
Si
On/On
Channel
On/Off
No Channel
Auto/Desirable
Channel
Si
Si
Si
Si
Si
Si
On/Off
No Channel
Active/Passive
Channel
Si
Si
Si
Si
Passive/Passive
N Ch
No
Channell
On: always be a channel/bundle member
Desirable: ask if the other side can/will
Auto: if the other side asks I will
Off: don’t become a member of a
channel/bundle
© 2008 Cisco Systems, Inc. All rights reserved.
On/On
Channel
Si
Off/On, Auto, Desirable
No Channel
BRKRST-2031
14457_04_2008_c2
Link Aggregation Protocol
Cisco Public
On: always be a channel/bundle member
Active: ask if the other side can/will
Passive: if the other side asks I will
Off: don’t become a member of a
channel/bundle
50
PAgP Tuning
PAgP Default Mismatches
Matching EtherChannel Configuration on Both Sides
p
Link Restoration Convergence
g
Times
Improves
set port channel <mod/port> off
Time
e to Converrge in
Seconds
7
6
4
3
2
1
0
BRKRST-2031
14457_04_2008_c2
As Much As
Seven Seconds
of Delay/Loss
Tuned Away
5
PAgP Mismatch
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
PAgP Off
51
EtherChannels or Equal Cost Multipath
10/100/1000 How Do You Aggregate It?
10GE and
10GE
10GE
10
GE channels
Core
T i l 4 :1
Typical
Data OverOverSubscription
Si
Distribution
Si
Typical 20
20::1
Data OverOverSubscription
BRKRST-2031
14457_04_2008_c2
Si
Si
Access
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
52
EtherChannels or Equal Cost Multipath
Reduce Complexity/Peer Relationships
 More links = more routing
peer relationships and
associated overhead
Si
Si
Si
Si
Si
 EtherChannels allow you to
reduce peers by creating single
logical interface to peer over
Si
 On single link failure in a bundle
Layer 3 Equal
Cost Links
Layer 3 Equal
Cost Links
Si
Si
OSPF running on an IOS-based switch
will reduce link cost and re-route
re route traffic
OSPF running on a hybrid switch will
not change link cost and may overload
remaining links
Si
Si
Si
Si
WAN
BRKRST-2031
14457_04_2008_c2
Si
EIGRP may not change link cost and
may overload remaining links
Si
Data Center
© 2008 Cisco Systems, Inc. All rights reserved.
Internet
Cisco Public
53
EtherChannels or Equal Cost Multipath
Why 10-Gigabit Interfaces
 More links = more routing peer
relationships and associated
overhead
Si
Si
Si
Si
Si
Layer 3 Equal
Cost Links
Layer 3 Equal
Cost Links
Si
Si
Si
Si
Si
Si
WAN
BRKRST-2031
14457_04_2008_c2
 EtherChannels allow you to
reduce peers by creating single
logical interface to peer over
Si
Si
Si
Data Center
© 2008 Cisco Systems, Inc. All rights reserved.
Internet
Cisco Public
 However, a single link failure is
not taken into consideration by
protocols.
routing
out g p
otoco s Overload
O e oad
possible.
 Single 10-Gigabit links address
both problems.
problems Increased
bandwidth without increasing
complexity or compromising
routing
protocols ability
gp
y to select
best path.
54
EtherChannels Quick Summary
EtherChannels—Quick
 For Layer
Layer-2
2 EtherChannels: Desirable/Desirable is the recommended
configuration so that PAgP is running across all members of the bundle
insuring that an individual link failure will not result in an STP failure
 For Layer
Layer-3
3 EtherChannels: One can consider a configuration that uses
ON/ON. There is a trade-off between performance/HA impact and
maintenance and operations implications.
 An ON/ON configuration
g
is faster from a link-up
p ((restoration)) p
perspective
p
than a Desirable/Desirable alternative. However, in this configuration PAgP
is not actively monitoring the state of the bundle members and a
misconfigured bundle is not easily identified.
 Routing protocols may not have visibility into the state of an individual
member of a bundle. LACP and the minimum links option can be used to
bring the entire bundle down when the capacity is diminished.
OSPF has visibility to member loss (best practices pending investigation)
investigation). EIGRP does not
not…
 When used to increase bandwidth—no individual flow can go faster than the
speed of an individual member of the link
 Best
B t used
d to
t eliminate
li i t single
i l points
i t off ffailure
il
(i
(i.e. lilink
k or port)
t) d
dependencies
d
i
from a topology
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
55
Best Practices
Practices—First
First Hop Redundancy
 Used to provide a resilient
default gateway/first hop
address to end-stations
1st Hop Redundancy
 HSRP
HSRP, VRRP
VRRP, and
GLBP alternatives
Si
 VRRP, HSRP and GLBP
provide
id millisecond
illi
d titimers
and excellent convergence
performance
Si
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Si
WAN
Si
Si
Si
Si
Cisco Public
Si
Layer 3 Equal
Cost Links
Si
Si
 Preempt timers need
to be tuned to avoid
black-holed traffic
Si
Layer 3 Equal
Cost Links
d
 VRRP if you need
multivendor interoperability
 GLBP facilitates uplink
load balancing
Si
Si
Si
Data Center
Internet
56
First Hop Redundancy with VRRP
IETF Standard RFC 2338 (April 1998)
 A group of routers function
as one virtual router by
sharing one virtual
IP address and one
virtual MAC address
R1—Master, Forwarding
g Traffic; R2,—Backup
p
VRRP ACTIVE
R2
R1
Si
Si
Distribution-B
VRRP Backup
Distribution-A
VRRP Active
 The rest of the routers
act as “back
back up”
up in case
the master router fails
Access-a
 Backup routers stay idle
as far
f as packet forwarding
f
from the client side
is concerned
© 2008 Cisco Systems, Inc. All rights reserved.
IP:
10.0.0.253
MAC: 0000.0C78.9abc
vIP:
vMAC:
IP:
10.0.0.254
MAC: 0000.0c12.3456
vIP: 10.0.0.10
vMAC: 0000.5e00.0101
 One (master) router
performs packet
forwarding for local hosts
BRKRST-2031
14457_04_2008_c2
VRRP BACKUP
IP:
MAC
MAC:
GW:
ARP:
Cisco Public
10.0.0.1
aaaa.aaaa.aa01
01
10.0.0.10
0000.5e00.0101
IP:
MAC
MAC:
GW:
ARP:
10.0.0.2
aaaa.aaaa.aa02
02
10.0.0.10
0000.5e00.0101
IP:
MAC
MAC:
GW:
ARP:
10.0.0.3
aaaa.aaaa.aa03
03
10.0.0.10
0000.5e00.0101
57
First Hop Redundancy with HSRP
RFC 2281 (March 1998)
 A group of routers function
as one virtual router by
sharing one virtual
IP address and one
virtual MAC address
R1—Active, Forwarding Traffic;
R2—Hot Standby, Idle
HSRP ACTIVE
© 2008 Cisco Systems, Inc. All rights reserved.
R2
R1
Si
Si
Distribution-B
HSRP Backup
Distribution-A
HSRP Active
 The rest of the routers
provide “hot
hot standby”
standby in
case the active router fails
BRKRST-2031
14457_04_2008_c2
IP:
10.0.0.253
MAC: 0000.0C78.9abc
vIP:
vMAC:
IP:
10.0.0.254
MAC: 0000.0c12.3456
vIP: 10.0.0.10
vMAC: 0000.0c07.ac00
 One (active) router
performs packet
forwarding for local hosts
 Standby routers stay idle
as far
f as packet forwarding
f
from the client side is
concerned
HSRP STANDBY
Access-a
IP:
MAC
MAC:
GW:
ARP:
Cisco Public
10.0.0.1
aaaa.aaaa.aa01
01
10.0.0.10
0000.0c07.ac00
IP:
MAC
MAC:
GW:
ARP:
10.0.0.2
aaaa.aaaa.aa02
02
10.0.0.10
0000.0c07.ac00
IP:
MAC
MAC:
GW:
ARP:
10.0.0.3
aaaa.aaaa.aa03
03
10.0.0.10
0000.0c07.ac00
58
Why You Want HSRP Preemption
 Spanning Tree Root and
HSRP Primary aligned
 When Spanning Tree Root
is re
re-introduced,
introduced traffic will
take a two-hop path to
HSRP Active
Si
Spanning Tree
Root
HSRP
HSRP Preempt
Active
Core
Si
HSRP
Active
Si
 HSRP Preemption will
allow HSRP to follow
Spanning Tree topology
Si
Distribution
Spanning Tree
Root
Access
Without Preempt Delay HSRP Can Go Active Before Box Completely Ready to
Forward Traffic: L1 (Boards), L2 (STP), L3 (IGP Convergence)
standby 1 preempt delay minimum 180
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
59
First Hop Redundancy with GLBP
Cisco Designed, Load Sharing, Patent Pending
 All the benefits of HSRP
plus load balancing of
default gateway  utilizes
all available bandwidth
 A group of routers function
as one virtual router by
sharing one virtual IP
address but using multiple
virtual MAC addresses
g
for traffic forwarding
 Allows traffic from a single
common subnet to go
through multiple redundant
gateways using a single
virtual IP address
R1- AVG; R1, R2 Both Forward Traffic
R1
© 2008 Cisco Systems, Inc. All rights reserved.
IP:
10.0.0.253
MAC: 0000.0C78.9abc
vIP: 10.0.0.10
vMAC: 0007.b400.0102
IP:
10.0.0.254
MAC: 0000.0c12.3456
vIP: 10.0.0.10
10 0 0 10
vMAC: 0007.b400.0101
R1
Si
Distribution-A
GLBP AVG/
AVF, SVF
IP:
MAC
MAC:
GW:
ARP:
BRKRST-2031
14457_04_2008_c2
GLBP AVF, SVF
GLBP AVG/AVF, SVF
Cisco Public
10.0.0.1
aaaa.aaaa.aa01
01
10.0.0.10
0007.B400.0101
Si
Distribution-B
GLPB AVF, SVF
Access-a
IP:
MAC
MAC:
GW:
ARP:
10.0.0.2
aaaa.aaaa.aa02
02
10.0.0.10
0007.B400.0102
IP:
MAC
MAC:
GW:
ARP:
10.0.0.3
aaaa.aaaa.aa03
03
10.0.0.10
0007.B400.0101
60
First Hop Redundancy with
Load Balancing
Cisco Gateway Load Balancing Protocol (GLBP)
 Each member of a GLBP redundancy group owns a unique virtual MAC address
for a common IP address/default gateway
 When end-stations ARP for the common IP address/default gateway they are
given a load balanced virtual MAC address
 Host A and host B send traffic to different GLBP peers but have the same
default gateway
GLBP 1 ip 10.88.1.10
vMAC 0000.0000.0001
vIP
10.88.1.10
R1
.1
GLBP 1 ip 10.88.1.10
vMAC 0000.0000.0002
R2
.2
2
ARP
Reply
10.88.1.0/24
.4
ARPs for 10.88.1.10
Gets MAC 0000.0000.0001
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
.5
A
B
Cisco Public
ARPs for 10.88.1.10
Gets MAC 0000.0000.0002
61
Optimizing Convergence:
VRRP HSRP
VRRP,
HSRP, GLBP
Mean, Max, and Min—Are There Differences?
 VRRP not tested with sub-second
sub second timers and all flows go through
a common VRRP peer; mean, max, and min are equal
 HSRP has sub-second timers; however all flows go through same
peer so there is no difference between mean,, max,, and min
HSRP p
Si
Si
 GLBP has sub-second timers and distributes the load amongst
the GLBP peers; so 50% of the clients are not affected by an
uplink failure
Time in Seconds to Converge
e
Distribution to Access Link Failure
Access to Server Farm
1.2
VRRP
HSRP
GLBP
50% of Flows
Have ZERO
Loss W/ GLBP
1
0.8
GLBP Is 50%
Better
0.6
0.4
0.2
0
BRKRST-2031
14457_04_2008_c2
Longest
© 2008 Cisco Systems, Inc. All rights reserved.
Shortest
Cisco Public
Average
62
If You Span VLANS,
VLANS Tuning Required
By Default, Half the Traffic Will Take a Two-Hop L2 Path
 Both distribution switches act as default gateway
 Blocked uplink caused traffic to take less than optimal path
Core
Layer 3
Distribution
Layer 2/3
Core
Distribution-A
GLBP Virtual
MAC 1
Distribution-B
GLBP Virtual
MAC 2
Si
Si
Access
Layer 2
BRKRST-2031
14457_04_2008_c2
F: Forwarding
B: Blocking
Access-a
Access-b
VLAN 2
VLAN 2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
63
Agenda
Data Center
Services
Block
 Multilayer Campus
Design principles
 Foundation Services
 Campus Design
Best Practices
 IP Telephony
Considerations
Si
Si
 QoS Considerations
 Security
Considerations
Si
 Putting It All Together
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Si
Si
Si
Si
Distribution Blocks
 Summary
BRKRST-2031
14457_04_2008_c2
Si
Si
Cisco Public
64
Daisy Chaining Access Layer Switches
Avoid Potential Black Holes
Return Path Traffic Has a 50/50 Chance of Being ‘Black
Black Holed’
Holed
Core
Layer 3
Si
Si
50% Chance That Traffic
Will Go Down Path with
No Connectivity
Layer 3 Link
Distribution
Layer 2/3
Distribution-A
Distribution-B
Si
Si
Access
Layer 2
BRKRST-2031
14457_04_2008_c2
Access-a
Access-n
Access-c
VLAN 2
VLAN 2
VLAN 2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
65
Daisy Chaining Access Layer Switches
New Technology Addresses Old Problems
 Stackwise/Stackwise-Plus technology eliminates the concern
Loopback links not required
No longer forced to have L2 link in distribution
 If you use modular (chassis-based) switches, these problems
are not a concern
Forwarding
Si
HSRP
Active
Layer 3
Forwarding
Si
HSRP
Standby
Catalyst 3750-E
or Catalyst 2975
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
66
What Happens if You Don’t
Link the Distributions?
 STPs slow convergence can
cause considerable periods
of traffic loss
 STP could cause
non-deterministic traffic
flows/link load engineering
 STP convergence
g
will
cause Layer 3 convergence
 STP and Layer 3 timers
are independent
 Unexpected Layer 3 convergence
and re-convergence could occur
 Even if yyou do link the distribution
switches dependence on STP
and link state/connectivity can
cause HSRP irregularities and
unexpected state transitions
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Core
STP Root and
HSRP Active
STP Secondary
Root and HSRP
Standby
Hellos
Si
Si
F 2
B
2
Access-a
VLAN 2
Traffic
Dropped Until
Transition to
Forwarding;
As much as 50
Seconds
Access-b
VLAN 2
Traffic
Dropped Until
MaxAge
Expires Then
Listening and
Learning
67
What if You Don
Don’t?
t?
Black Holes and Multiple ‘Transitions’ …
Core
Layer 3
Core
STP Root and
HSRP Active
Distribution
Layer 2/3
STP
Secondary
Root and
HSRP Standby
Hellos
Si
Si
 Aggressive HSRP
timers limit black
hole #1
 Backbone fast limits
time (30 seconds)
HSRP Active
to event #2
(Temporarily)
 Even with Rapid
PVST+ at least
one second
before event #2
F: Forwarding
B: Blocking
Access
Layer 2
Access-a
Access-b
VLAN 2
VLAN 2
MaxAge
Seconds Before
Failure Is
Detected…
Then Listening
and Learning
 Blocking link on access-b will take 50 seconds to move to forwarding  traffic black hole until HSRP
goes active on standby HSRP peer
 After MaxAge expires (or backbone fast or Rapid PVST+) converges HSRP preempt causes another
transition
 Access-b used as transit for access-a’s traffic
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
68
What if You Don
Don’t?
t?
Return Path Traffic Black Holed …
Core
Layer 3
Core
STP Root and
HSRP Active
Distribution
Layer 2/3
STP
Secondary
Root and
HSRP Standby
Hellos
Si
Si
Access
Layer 2
 802
802.1d:
1d: up to
50 seconds
 PVST+: backbone
fast 30 seconds
 Rapid PVST+:
address by the
protocol (one
second)
F: Forwarding
g
B: Blocking
Access-a
Access-b
VLAN 2
VLAN 2
 Blocking link on access-b
access b will take 50 seconds to move to forwarding 
return traffic black hole until then
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
69
Asymmetric Routing (Unicast Flooding)
 Affects redundant
topologies with
shared L2 access
 One path upstream
and two paths
downstream
 CAM table entry
ages out on
standby HSRP
 Without a CAM
entry packet is
flooded to all ports
in the VLAN
Asymmetric
Equal Cost
Return Path
CAM Timer Has
Aged out on
Standby HSRP
Si
Upstream Packet
Unicast to
Active HSRP
Downstream
Packet
Flooded
VLAN 2
BRKRST-2031
14457_04_2008_c2
Si
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
VLAN 2
VLAN 2
VLAN 2
70
Best Practices Prevent Unicast Flooding
 Assign one unique
data and voice VLAN
to each access switch
 Traffic is now only
flooded down
one trunk
 A
Access switch
it h
unicasts correctly;
no flooding to
all ports
 If you have to:
Asymmetric
Equal Cost
Return Path
Downstream
Packet
Flooded on
Single Port
Si
Si
Upstream Packet
Unicast to
Active HSRP
Tune ARP and CAM
aging timers; CAM
timer exceeds
ARP timer
Bias routing metrics
t remove equall
to
cost routes
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
VLAN 3
Cisco Public
VLAN 4
VLAN 5
VLAN 2
71
Agenda
Data Center
Services
Block
 Multilayer Campus
Design Principles
 Foundation Services
 Campus Design
Best Practices
 IP Telephony
Considerations
Si
Si
 QoS Considerations
 Security
Considerations
Si
 Putting It All Together
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Si
Si
Si
Si
Distribution Blocks
 Summary
BRKRST-2031
14457_04_2008_c2
Si
Si
Cisco Public
72
Building a Converged Campus Network
Infrastructure Integration, QoS, and Availability
 Access layer
Auto phone
detection
Inline power
Q S scheduling,
QoS:
h d li
trust boundary and
classification
Fast convergence
Access
Si
Si
Si
Si
Si
Si
Distribution
 Distribution
Di t ib ti llayer
High availability,
redundancy,
fast convergence
Policy enforcement
QoS: scheduling,
trust boundary
and classification
 Core
Core
Distribution
High availability,
redundancy,
fast convergence
QoS: scheduling,
trust boundary
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Layer 3
Equal Cost
Links
Si
Si
Si
Si
Layer 3
Equal Cost
Links
Si
Si
Si
Si
A
Access
WAN
Cisco Public
Data Center
Internet
73
Infrastructure Integration
Extending the Network Edge
Switch Detects IP Phone and Applies Power
CDP Transaction Between Phone and Switch
IP Phone Placed in Proper VLAN
DHCP Request
Req est and Call Manager Registration
 Phone contains a three-port
three port switch that is configured in conjunction
with the access switch and CallManager
BRKRST-2031
14457_04_2008_c2
1.
Power negotiation
2.
VLAN configuration
3.
802.1x interoperation
4
4.
QoS configuration
5.
DHCP and CallManager registration
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
74
Infrastructure Integration: First Step
Power Requirement Negotiation
 Cisco pre
pre-standard
standard devices initially receive 6.3
6 3 watts
and then optionally negotiate via CDP
 802
802.3af
3af devices initially receive 12.95
12 95 watts unless PSE
able to detect specific PD power classification
Class
Usage
Minimum Power Levels
Output at the PSE
Maximum Power Levels
at the Powered Device
0
Default
15.4W
0.44 to 12.95W
1
Optional
4.0W
0.44 to 3.84W
2
Optional
7.0W
3.84 to 6.49W
3
Optional
15.4W
6.49 to 12.95W
4
Reserved for
Future
Use
Treat as Class 0
Reserved for Future Use: a Class 4
Signature Cannot Be Provided by a
Compliant Powered Device
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
75
Enhanced Power Negotiation
802.3af Plus Bi-Directional CDP (Cisco 7970)
PD Plugged in
PSE—Power
Source Equipment
Cisco 6500,4500,
3750 3560
3750,
Switch Detects IEEE PD
PD Is
I Classified
Cl
ifi d
Power Is Applied
Phone Transmits a CDP Power Negotiation
Packet Listing Its Power Mode
Switch Sends a CDP Response
with a Power Request
PD—Powered
Device Cisco 7970
Based on Capabilities
p
Exchanged
g
Final Power Allocation Is Determined
 Using bi-directional
bi directional CDP exchange exact power
requirements are negotiated after initial power-on
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
76
Design Considerations for PoE
Power Management
 Switch manages power by what is allocated not by what is currently used
 Device power consumption is not constant
 A 7960G requires
q
7W when the p
phone is ringing
g g at maximum volume and
requires 5W on or off hook
 Understand the power behavior of your PoE devices
 Utilize static power configuration with caution
Dynamic allocation:
power inline auto max 7200
Static allocation:
power inline static max 7200
 Use power calculator to determine power requirements
http://www.cisco.com/go/powercalculator
Discover Cisco Enhanced PoE at the World of Solutions
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
77
Infrastructure Integration: Next Steps
VLAN, QoS and 802.1x Configuration
Phone VLAN = 110
(VVID)
802.1Q encapsulation
with 802.1p
Layer 2 CoS
PC VLAN = 10
(PVID)
Native VLAN (PVID) No
Configuration Changes
Needed on PC
 During initial CDP exchange phone is configured with a Voice
VLAN ID (VVID)
 Phone also supplied with QoS configuration via CDP TLV fields
 Additionally switch port currently bypasses 802.1x authentication
for VVID if detects Cisco phone
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
78
Agenda
Data Center
Services
Block
 Multilayer Campus
Design principles
 Foundation Services
 Campus Design
Best Practices
 IP Telephony
Considerations
Si
Si
 QoS Considerations
 Security
Considerations
Si
 Putting It All Together
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Si
Si
Si
Si
Distribution Blocks
 Summary
BRKRST-2031
14457_04_2008_c2
Si
Si
Cisco Public
79
Best Practices
Practices—Quality
Quality of Service
 Must be deployed end-toend to be effective; all layers
play different but equal roles
End to End QoS
 Ensure that mission critical
applications are not
impacted by link or transmit
queue congestion
 Aggregation and rate
transition points must
enforce QoS policies
Si
Si
Si
Si
Si
Si
Si
Si
Cisco Public
Si
Layer 3 Equal
Cost Links
Si
WAN
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Layer 3 Equal
Cost Links
 Multiple queues with
configurable admission
criteria and scheduling
are required
BRKRST-2031
14457_04_2008_c2
Si
Si
Si
Data Center
Internet
80
Transmit Queue Congestion
10/100m
Queued
128k Uplink
p
WAN
R
Router
100 Meg in 128 Kb/S out—Packets Serialize in Faster than They Serialize out
P k t Queued
Packets
Q
d as They
Th Wait
W it to
t Serialize
S i li outt Slower
Sl
Link
Li k
1 Gig Link
Distribution Switch
Queued
100 Meg Link
Access Switch
1 Gig In 100 Meg out—Packets Serialize in Faster than They Serialize out
Packets Queued
Q
as They Wait to Serialize
S
out Slower
S
Link
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
81
Auto QoS VoIP—Making
VoIP Making It Easy …
Configures QoS for VoIP on Campus Switches
Access-Switch(config-if)#auto qos voip ?
cisco-phone
cisco
phone
Trust the QoS marking of Cisco IP Phone
cisco-softphone Trust the QoS marking of Cisco IP SoftPhone
trust
Trust the DSCP/CoS marking
Access-Switch(config-if)#autoqosvoipcisco-phone
(
g )
q
p
p
Access-Switch(config-if)#exit
!
interface FastEthernet1/0/21
srr-queue bandwidth share 10 10 60 20
srr-queue bandwidth shape 10 0 0 0
Mls qos trust device cisco-phone
Mls qos trust cos
auto qosvoipcisco-phone
end
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
82
Agenda
Data Center
Services
Block
 Multilayer Campus
Design principles
 Foundation Services
 Campus Design
Best Practices
 IP Telephony
Considerations
Si
Si
 QoS Considerations
 Security
Considerations
Si
 Putting It All Together
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Si
Si
Si
Si
Distribution Blocks
 Summary
BRKRST-2031
14457_04_2008_c2
Si
Si
Cisco Public
83
Best Practices
Practices—Campus
Campus Security
 New
e stu
stuff tthat
at we
e will co
cover!
e
Catalyst Integrated Security Feature Set!
Dynamic Port Security, DHCP Snooping,
Dynamic ARP Inspection, IP Source Guard
End-to-End Security
 Things you already know—
we won’t cover…
Si
Use SSH to access devices instead of Telnet
Enable AAA and roles-based access control
(RADIUS/TACACS+) for the CLI on all devices
Enable SYSLOG to a server.
server Collect and
archive logs
When using SNMP use SNMPv3
Disable unused services:
no service tcp-small-servers
i udp-small-servers
d
ll
no service
Use FTP or SFTP (SSH FTP) to move images
and configurations around—avoid TFTP
when possible
Install VTY access-lists to limit which addresses
can access managementt and
d CLI services
i
Enable control plane protocol authentication
where it is available (EIGRP, OSPF, BGP,
HSRP, VTP, etc.)
Apply basic protections offered by implementing
filt i on external
t
l edge
d inbound
i b
d
RFC2827 filtering
interfaces
Si
Si
Si
Si
Si
Si
WAN
Si
Si
Si
Si
Si
Si
Si
Internet
For More Details, See BRKSEC-2002 Session, Understanding and Preventing Layer 2 Attacks
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
84
BPDU Guard
Prevent Loops via WLAN (Windows XP Bridging)
 Problem:
WLAN APs do not
forward BPDUs
STP Loop
Formed
BPDU Guard
Disables Port
Multiple Windows XP
machines can create a
l
loop
iin th
the wired
i d VLAN
via the WLAN
BPDU
Generated
 Solution:
BPDU Guard configured
on all end-station
end station switch
ports will prevent loop
from forming
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Win XP
Bridging
Enabled
BPDU
Discarded
Win XP
Bridging
Enabled
85
Problem: Prevalence of Rogue APs
Example: 59 APs in Seven Miles in SJ Commute
 The majority of WLAN deployments
are unauthorized
by well intended employees (rogue
APs)—many are insecure
 A daily drive to work taken within the
car at normal speeds with
an IPAQ running a freeware
application (mix of residences
and enterprises)
Insecure
APs
 Insecure enterprise rogue
AP’s are a result of:
– Well intentioned staff install due
to absence of sanctioned WLAN deployment
– An infrastructure that is not
“wireless ready” to protect
against rogue AP’s
59 APs Found
War Chalking
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
86
Basic 802.1x
802 1x Access Control
Controlling When and Where APs Are Connected
Who Are You?
I Am Joe Cisco
Authorized
User
No
802.1x
H
Here
802.1x Enabled on
“user” Facing
Ports
o ts
Who Are You?
Rogue AP
Disabled
D on Authorized
WLAN AP Ports
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco IOS Configuration Example
radius-server host 10.1.125.1
radius-server key cisco123
aaa new-model
aaa authentication dot1x default group
radius
aaa authorization default group radius
aaa authorization config-commands
dot1x system-auth-control
Cisco IOS Per-Port configuration
int range fa3/1 - 48
dot1x port-control auto
A h i d AP
Authorized
BRKRST-2031
14457_04_2008_c2
CatOS Configuration Example
set dot1x system-auth-control enable
set dot1x guest-vlan 250
set radius server 10.1.125.1 auth-port
1812 primary
set radius key cisco123
set port dot1x 3/1-48 port-control
auto
Cisco Public
87
Securing Layer 2 from
Surveillance Attacks
Cutting off MAC-Based Attacks
Only 3 MAC
Addresses
Allowed on
the Port:
Shutdown
00:0e:00:aa:aa:aa
00
0 00
00:0e:00:bb:bb:bb
250,000
Bogus MACs
per Second
SOLUTION:
PROBLEM:
“Script Kiddie” Hacking Tools Enable
Attackers Flood Switch CAM Tables
with Bogus Macs; Turning the VLAN
into a “Hub” and Eliminating Privacy
Switch CAM Table Limit Is Finite
Number of Mac Addresses
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Port Security Limits MAC Flooding
Attack and Locks down Port and
Sends an SNMP Trap
switchport
switchport
switchport
switchport
switchport
port-security
port-security
port-security
port-security
port-security
maximum 10
violation restrict
aging time 2
aging type inactivity
88
DHCP Snooping
Protection Against Rogue/Malicious DHCP Server
1
DHCP
Server
1000s of DHCP
R
Requests
t tto
Overrun the
DHCP Server
2
 DHCP requests (discover) and responses (offer) tracked
 R
Rate-limit
t li it requests
t on ttrusted
t d interfaces;
i t f
limits
li it D
DoS
S attacks
tt k on
DHCP server
 Deny responses (offers) on non trusted interfaces; stop malicious
or errant DHCP server
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
89
Securing Layer 2 from
Surveillance Attacks
Protection Against ARP Poisoning
 Dynamic ARP inspection
protects against ARP
poisoning (ettercap,
dsnif arpspoof)
dsnif,
 Uses the DHCP snooping
binding
g table
 Tracks MAC to IP from
DHCP transactions
Gateway = 10.1.1.1
MAC=A
Gratuitous
G
i
ARP
10.1.1.50=MAC_B
Gratuitous ARP
10.1.1.1=MAC_B
 Rate-limits ARP requests
from client ports; stop port
scanning
 Drop BOGUS gratuitous
ARPs; stop ARP
poisoning/MIM
i
i /MIM attacks
tt k
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Si
Attacker
Att
k = 10.1.1.25
10 1 1 25
MAC=B
Victim
Vi
ti = 10.1.1.50
10 1 1 50
MAC=C
90
IP Source Guard
Protection Against Spoofed IP Addresses
 IP source guard
Gateway = 10.1.1.1
protects against
MAC=A
p
IP addresses
spoofed
Si
 Uses the DHCP
p g binding
g table
snooping
 Tracks IP address to
port associations
 Dynamically programs
port ACL to drop traffic
not originating from
IP address assigned
via
i DHCP
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Hey, I’m 10.1.1.50 !
Att k = 10.1.1.25
Attacker
10 1 1 25
Vi ti = 10.1.1.50
Victim
10 1 1 50
91
Catalyst Integrated Security Features
Summary Cisco IOS
ipdhcp snooping
IP Source
S
Guard
G
d
ipdhcp snooping vlan 2-10
iparp inspection vlan 2-10
Dynamic ARP Inspection
!
interface fa3/1
DHCP Snooping
switchport port-security
Port Security
switchport port-security max 3
switchport port-security violation
restrict
 Port security prevents MAC
g attacks
flooding
 DHCP snooping prevents client
attack on the switch and server
 Dynamic
y
ARP Inspection
p
adds
security to ARP using DHCP
snooping table
 IP source guard adds security
t IP source address
to
dd
using
i
DHCP snooping table
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
switchport port-security aging time 2
switchport port
port-security
security aging type
inactivity
iparp inspection limit rate 100
ipdhcp snooping limit rate 100
ip verify source vlandhcp-snooping
!
Interface gigabit1/1
ipdhcp snooping trust
iparp inspection trust
92
Agenda
Data Center
Services
Block
 Multilayer Campus
Design principles
 Foundation Services
 Campus Design
Best Practices
 IP Telephony
Considerations
Si
Si
 QoS Considerations
 Security Considerations
Si
Si
 Putting It All Together
Si
 Summary
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Si
Si
Si
Si
Distribution Blocks
Cisco Public
93
Hierarchical Campus
Access
Si
Si
Si
Si
Si
Si
Distribution
Core
Si
Si
Si
Si
Si
Si
Si
Distribution
Si
Access
WAN
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Data Center
Cisco Public
Internet
94
Layer 3 Distribution Interconnection
Layer 2 Access—No VLANs Span Access Layer
 Tune CEF load balancing
 Match CatOS/IOS EtherChannel
settings and tune load balancing
 Summarize routes towards core
 Limit redundant IGP peering
 STP Root and HSRP primary
tuning or GLBP to load balance
on uplinks
li k
 Set trunk mode on/nonegotiate
 Disable EtherChannel
unless needed
 Set port host on access
layer ports:
Si
Si
Layer 3
Si
Point
to
Point
Link
Distribution
Si
sab e Trunking
u
g
Disable
Disable EtherChannel
Enable PortFast
 RootGuard or BPDU-Guard
 Use security features
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Core
Access
VLAN 20 Data
10.1.20.0/24
VLAN 120 Voice
10.1.120.0/24
Cisco Public
VLAN 40 Data
10.1.40.0/24
VLAN 140 Voice
10.1.140.0/24
95
Layer 2 Distribution Interconnection
Layer 2 Access—Some VLANs Span Access Layer
 Tune CEF load balancing
 Match CatOS/IOS EtherChannel
settings and tune load balancing
 Summarize routes towards core
 Limit redundant IGP peering
 STP Root and HSRP primary or
GLBP and STP port cost tuning
to load balance on uplinks
 Set trunk mode on/nonegotiate
 Disable EtherChannel
unless needed
 RootGuard on downlinks
 LoopGuard on uplinks
 Set port host on access
Layer ports:
Disable Trunking
Disable EtherChannel
Enable PortFast
 RootGuard or BPDU-Guard
 Use security features
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Si
Layer 2
Si
Trunk
Si
VLAN 20 Data
VLAN 40 Data
10.1.20.0/24
10.1.40.0/24
VLAN 120 Voice
VLAN 140 Voice
10.1.120.0/24
10.1.140.0/24
VLAN 250 WLAN
10.1.250.0/24
Cisco Public
Core
Distribution
Access
96
Routed Access and
Virtual Switching System
Evolutions of and Improvements to Existing Designs
Si
Si
Si
Si
Core
VSS Link
Si
Layer 3
New
Concept
Si
Distribution
P-t-P Link
VLAN 20 Data
10.1.20.0/24
VLAN 120 Voice
10.1.120.0/24
VLAN 20 Data
10.1.20.0/24
VLAN
40 Data
10.1.40.0/24
VLAN 120 Voice
10.1.120.0/24
VLAN
140 Voice
10
10.1.140.0/24
1
140
0/24
VLAN 250 WLAN
10.1.250.0/24
VLAN 40 Data
10.1.40.0/24
VLAN 140 Voice
10.1.140.0/24
Access
See RST-3035—Advanced Enterprise Campus Design Alternatives: Routed Access and Virtual Switch System (VSS)
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
97
High Availability Campus Design
Simplified with VSS
Access
Si
Si Si
Si Si
Si
Si Si
Si
Si Si
Si Si
Si Si
Distribution
Core
Si Si
Si Si
Si
Si
Distribution
Access
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Si Si
Si
Si Si
Si Si Si
WAN
WAN
WAN
Cisco Public
Si Si
Data
Data
Data
Center
Center
Center
Si Si
Si Si
Internet
Internet
Internet
98
SmartPorts—Predefined Configurations
Access-Switch#show parser macro brief
default global : cisco-global
default interface: cisco-desktop
p
default interface: cisco-phone
default interface: cisco-switch
default interface: cisco-router
default interface: cisco-wireless
Si
Si
Access-Switch(config-if)#$ macro apply cisco-phone
$access_vlan 100 $voice_vlan 10
Access-Switch#show run int fa1/0/19
!
interface FastEthernet1/0/19
switchport access vlan 100
switchport mode access
switchport voice vlan 10
switchport port-security maximum 2
switchport port-security
switchport port-security aging time 2
switchport port-security violation restrict
switchport port-security
port security aging type inactivity
srr-queue bandwidth share 10 10 60 20
srr-queue bandwidth shape 10 0 0 0
mls qos trust device cisco-phone
mls qos trust cos
macro description cisco
cisco-phone
phone
auto qosvoipcisco-phone
spanning-tree portfast
spanning-tree bpduguard enable
BRKRST-2031
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
14457_04_2008_c2
end
Si
Si
99
Agenda
Data Center
Services
Block
 Multilayer Campus
Design principles
 Foundation Services
 Campus Design
Best Practices
 IP Telephony
Considerations
Si
Si
 QoS Considerations
 Security Considerations
Si
Si
 Putting It All Together
Si
 Summary
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Si
Si
Si
Si
Si
Distribution Blocks
Cisco Public
100
Summary
 Off
Offers hierarchy—each
hi
h
h llayer
has specific role
Access
 Modular topology—
building blocks
 Easy to grow, understand,
and troubleshoot
Si
Si
Si
Si
Si
Si
Distribution
 Creates small fault domains—
Clear demarcations and
isolation
 Promotes load balancing
and redundancy
 Promotes deterministic
traffic patterns
 Incorporates balance of
both Layer 2 and Layer 3
technology, leveraging
the strength of both
 Utilizes Layer 3 Routing
for load balancing,
g, fast
convergence, scalability,
and control
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Layer 3
Equal Cost
s
Links
Si
Si
Si
Si
Layer 3
Equal Cost
Links
Si
Si
Si
Core
Distribution
Si
Access
WAN
Cisco Public
Data Center
Internet
101
Q and A
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
102
Hierarchical Network Design
Without a Rock Solid Foundation the Rest Doesn’t Matter
Access
Distribution
Si
Si
Core
Si
Si
HSRP
Distribution
Si
Access
Routing
g
Si
Spanning
Tree
Building Block
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
103
Reference Materials
http://www.cisco.com/go/srnd
 High Availability Campus Design Guide
 High Availability Campus Convergence Analysis
 High Availability Campus Design Guide—
Routed Access EIGRP and OSPF
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
104
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
105
Optimal Redundancy
When Is More Less?
 Core and distribution
engineered with
redundant nodes
and links to
provide maximum
redundancy and
optimal convergence
 Network bandwidth
and capacity
engineered to
ith t d node
d
withstand
or link failure
 120–200ms to
converge around
most events
Access
Si
Distribution
Si
Si
Si
Si
Si
Redundan
t
Nodes
Core
Si
Distribution
Si
Si
Si
Si
Si
Si
Si
Access
WAN
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Data Center
Internet
106
Single Points of Termination
SSO/NSF Avoiding Total Network Outage
L2 = SSO
L3 =
SSO/NSF
Access
Distribution
Si
Si
Si
Si
Si
Si
Core
Si
Si
 The access layer is candidate for supervisor redundancy
 L2 access layer SSO
 L3 access layer SSO and NSF
 Network outage until physical replacement or reload vs.
one to three seconds
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
107
Supervisor Processor Redundancy
Stateful Switch Over (SSO)
 Active/standby supervisors
run in synchronized mode
SP
RP
PFC
 Redundant supervisor is in ‘hotstandby’ mode
standby
Active Supervisor
SP
RP
 Switch processors synchronize L2
port state information,
((e.g.,
g , STP,, 802.1x,, 802.1q)
q)
 PFCs synchronize L2/L3 FIB,
NetFlow and ACL tables
PFC
 DFCs are populated with L2/L3
FIB, NetFlow and
ACL tables
Standby Supervisor
Line Card—DFC
Line
e Ca
Card—DFC
d
C
Line Card—DFC
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
108
Non Stop Forwarding (NSF)
Non-Stop
NSF Recovery

DFC enabled line cards continue to
forward based on existing FIB entries

Following SSO recovery and activation
of standby Sup synchronized PFC
continues
ti
tto forward
f
d traffic
t ffi based
b
d
on existing FIB entries

“Hot-Standby” MSFC RIB is detached
from the FIB isolating FIB from
RP changes

“Hot-Standby” MSFC activates routing
processes in NSF recovery mode

MSFC re-establishes adjacency
indicating this is an NSF restart

Peer updates restarting MSFC with
it’s routing information

Restarting MSFC sends routing
updates to the peer

RIB reattaches to FIB and PFC
C and
DFCs updated with new FIB entries
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Si
Si
No Route Flaps During Recovery
109
Non Stop Forwarding (NSF)
Non-Stop
NSF Capable vs. NSF Awareness
 Two roles in NSF neighbor graceful
restart
NSF-Aware
– NSF Capable
– NSF Aware
 An NSF-Capable router is ‘capable’ of
continuous forwarding while
undergoing a switchover
Si
Si
 An NSF-Aware
NSF Aware router is able to assist
NSF-Capable routers by:
– Not resetting adjacency
– Supplying routing information for verification
after switchover
 NSF capable and NSF aware peers
cooperate using Graceful Restart
extensions to BGP, OSPF, ISIS and
EIGRP protocols
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Si
NSF-Capable
110
Design Considerations for NSF/SSO
NSF and Hello Timer Tuning?
 NSF is intended to provide
availability through route
convergence avoidance
 Fast IGP timers are intended to
provide availability through fast
route convergence
Neighbor Loss,
Loss No
Graceful Restart
Si
Si
 In an NSF environment dead
timer must be greater than SSO
Recovery + RP restart + time
to send first hello
 Switches running Native IOS
– OSPF 2/8 seconds for hello/dead
– EIGRP 1/4 seconds for hello/hold
 Switches running Hybrid
– OSPF 3/12 seconds for hello/dead
– EIGRP 2/8 seconds for hello/hold
BRKRST-2031
14457_04_2008_c2
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
111
Design Considerations for NSF/SSO
Where Does It Make Sense?
 Redundant topologies with equal cost
paths provide sub-second convergence
 NSF/SSO provides superior
availability in environments with
non-redundant paths
?
Second
ds of Los
st Voice
6
5
4
RP Convergence Is
Dependent
on IGP and Tuning
Si
Si
3
2
Si
1
0
BRKRST-2031
14457_04_2008_c2
Link
Failure
Node
Failure
© 2008 Cisco Systems, Inc. All rights reserved.
NSF/SSO
Cisco Public
OSPF
Convergence
112