Catalyst 6800 Series
Switch Architectures
BRKARC-3465
Shawn Wargo
Technical Marketing Engineer
BRKARC-3465
Abstract
Is your Campus network facing some, or all, of these challenges?
•
•
•
•
Host Mobility (w/o stretching VLANs)
Network Segmentation (w/o implementing MPLS)
Roles-based Access Control (w/o end-to-end TrustSec)
Common Policy for Wired & Wireless (w/o multiple tools)
Using Cisco technologies already available today, you can overcome
these challenges and build an evolved Campus network to better meet
your business objectives.
Come to this session to get a deeper insight into the key technologies,
designs and configurations (e.g. LISP with VXLAN and TrustSec) that
bring this evolution to life!
We highly recommend attendees of this session already be familiar with:
Enterprise Campus Design (BRKCRS-2031), Location ID Separation
Protocol (BRKRST-3045), and Cisco Trust Security (BRKCRS-2891).
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
3
Session Objectives
What we will cover…
• Catalyst 6807-XL
• Catalyst 6880-X
• Catalyst 6840-X
•
Chassis Architectures
•
Supervisor Engines
•
Ethernet LAN Modules
•
Forwarding Behaviors
•
Basic Packet Walks
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
4
Why is this session important?
Catalyst 6800 is an Architectural Foundation for Next Generation Campus
• Increased Port Speeds – To support the advent of next-generation integrated
Wireless (e.g. 802.11ac) and integrated HD Video Conferencing.
• Increased Port Density – To support the increased number of directly-connected
hosts (Wired & Wireless) and adjacent network devices.
• Increased Fabric Capacity – To support the increased amount of Speeds & Density
• Improved Control-Plane – To support the increased number of directly-connected
hosts (e.g. ARP/ND) and adjacent network devices.
• New Hardware Capabilities – To off-load traditional software (CPU) capabilities
(e.g. BFD & NDE), as well as to enable new features (e.g. VNtag & VxLAN).
•
Innovation + Investment Protection – To support an infrastructure that is capable of
using older technologies (software & hardware), while enabling new ones.
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
5
Agenda
 Background



History of Catalyst 6500
Evolution of Catalyst 6800
The 4 Basic Elements
 C6807-XL
 C6880-X
 C6840-X
 L2 Forwarding
 L3 Forwarding
 Packet Walks
Cisco Catalyst 6500 Series
A history of Innovation & Investment Protection
Years of
Innovation
VS-S2T-10G
WS-SUP720
WS-SUP32
WS-SUP2
WS-SUP1/1A
MSFC1/2
PFC1
32G Bus
1999
MSFC2
PFC2
256G SFM
or 32G Bus
2001
Catalyst 6000
3.5G (Bus) or
8G (SFM) per Slot
MSFC3
PFC3A/B
720G Fabric
or 32G Bus
VS-S720-10G
MSFC3
PFC3C
720G Fabric
MSFC5
PFC4
2T Fabric
2011
2007
2003
Catalyst 6500-E
Catalyst 6500
40G (Sup720) or
80G (Sup2T) Per Slot
40G (Sup720) or
3.5G (Sup32) Per Slot
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
7
Catalyst 6800 Series
Evolution of the Catalyst 6500-E
Catalyst 6807-XL
Catalyst 6880-X
Catalyst 6840-X
NEW
•
4 Fixed Form Factor Models
•
Semi Modular 5-Slot Chassis
•
Fully Modular 7-Slot Chassis
•
Only 2 RU height (smaller than 6503-E)
•
Only 4.5 RU height (smaller than 6504-E)
•
Only 10 RU height (smaller than 6506-E)
•
16 or 32 x 1/10GE Base Models
•
80 to 220Gbps per Half Slot capable
•
80 to 440Gbps per Slot capable
•
24 or 40 x 1/10GE + 2 x 40GE Models
•
16 to 80 x 1/10GE Port Density
•
Supports all Sup2T enabled Modules
ME-6524
C6503-E
C6504-E
C6509-E
C6506-E
BRKARC-3465
C6513-E
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
8
Catalyst 6800 Series
The 4 Basic Elements
Software Control-Plane
Hardware Control-Plane
The System CPU and DRAM,
where IOS runs (MSFC)
Intelligent Switching ASICs
(PFC/DFC), programmed by IOS
Catalyst 6800
Series
The Back-Plane (Fabric),
where Packets are transferred
Data-Plane ASICs for Ports,
receives and transmits Packets
Hardware Data-Plane
Forwarding ASICs
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
9
Catalyst 6800 Series
3500+ Catalyst 6500 Features
MPLS, VPLS & EVN
Next-Gen Solutions













15 Year MPLS Maturity
L2 VPN
L3 VPN
L2/L3 VPN over mGRE
MPLS TE
VPLS/A-VPLS/H-VPLS
Label Switch Multicast
MACsec over EoMPLS
MPLS at Access with I-Sat
VRF-Lite
EVN
VRF-Aware Multicast
VRF-Aware NetFlow
IPV4 Unicast
Security & QoS





Instant Access Controller
L3 Campus LISP
Software Defined Access
SDN - OpenFlow/APIC
Smart Install Director
VSS & HA






10 Year VSS Maturity
MACsec over VSL
1G/10G/40G VSL
SSO / NSF & NSR
BGP PIC & IP FRR
BFD on SVI & MEC
Catalyst 6800
Series
MacSec & NDAC

DHCP Snooping

SGT & SGACL

Dynamic ARP Inspection

Ingress/Egress ACL

SPAN with ACL

Time-Based ACL

Identity 2.0

ACL Statistics

ACLQoS Classification

Port Security

Marking (DSCP/COS)

IPv6 FH Security

Microflow Policing

ACL Atomic Commit/Dry Run

Advanced CoPP

IPv6 uRPF

IPv6 VLAN ACL

Large Tables & Scalability
IPV4 Multicast
OSPFv2/v3

IGMPv3 & MLDv2 Snooping in HW

OSPFv3 VRF-Lite

PIM-SM “Dual-RPF” in HW

VRF-Aware Unicast

PIM-DM, PIM-Bidir

IP Tunnel HA

L3 PIM Snooping

BFD SVI-GRE

PIM Register in HW

BGP PIC

mVPN, MSR, mcast BFD

IPV6 Features
IPv6 - IPv4 HW Parity

IPv6 in IPv4 Tunnels,
VRF-Aware IPv6 Tunnels

OSPFv3 VRF PE-CE

BGPv6, IS-ISv6

MLD, MLD Snooping

6VPE and 6PE

IPv6 Mcast HA

PIM Sparse Mode (PIM-SM)
IPv6 SLA, TCL, LLDP

IPv6 QoS, PACL, RACL
IPv4 Routing Capability
256K-2M
Multicast Routes (IPv4)
64K
Number of Adjacencies
1M
MAC Addresses
128K
ECMP (v4 and v6)
16
Security & QoS ACL
64K-256K
Flexible Netflow
128K-1M
MPLS Label Push/Pop in 1pass
5/3

Aggregate Policers
8K

Number of VPNs
16K
BRKARC-3465
Management &
Services
Rich Media
Flexible NetFlow

WCCPv3

Egress NetfFow

PBR IPv4/IPv6

Sampled NetFlow

NAT/PAT

NDE (Full & Sampled)

GRE/mGRE

Video Monitoring

ERSPAN

Mediatrace

GOLD

Metadata QoS

Cisco ISE

Multicast Service Reflect (MSR)

Cisco Prime


Multicast VPN (MVPN)
Mini Protocol Analyzer
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
10

Agenda
 Background
 C6807-XL



Chassis & Power
Supervisor Architectures
Module Architectures
 C6880-X
 C6840-X
 L2 Forwarding
 L3 Forwarding
 Packet Walks
Catalyst 6807-XL
For Your
Information
Modular Chassis Overview
Up to 880G/Slot capable
7 Slots
10 RU
Catalyst 6000 DNA
Next-Generation
ASIC Ready
Investment Protection
Compatible with Sup2T,
6700, 6800, 6900 and
Latest Service Modules
Low-Power & Noise
High-Efficiency Fans
Backwards compatible
backplane connectors
Up to 4 (N+1)
3000 Watt PSU
Redundancy
4 x 220VAC
Power Inputs
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
12
Catalyst 6807-XL
For Your
Information
Mechanical View
Slot 1: Line Card
Fan Tray
Slot 2: Line Card
Extra Fabric
Connectors
Slot 3: Supervisor *
Slot 4: Supervisor
*
Slot 5: Line Card
Slot 6: Line Card
Slot 7: Line Card
Power
Supplies
Power
Inputs
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
13
Catalyst 6807-XL
Environmental Overview
Height
17.5 in (10RU)
High Efficiency
4500 RPM
Redundant Fans
Depth
18.10 in
Platinum Efficient
3000W
Power Supplies
Width
17.36 in
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
14
Catalyst 6807-XL
Air Flow:
Side to Side
Fan Redundancy & Air Flow
New
Locking
Mechanism
Individual Fans
(3 columns of 3)
Fan-Tray LEDs
Fan-Tray Handle
Front-Service
Fan-Tray
Fan-Tray Highlights:
• Has 9 variable-speed High-Efficiency Fans (850 CFM)
LED
• Supports 4 speeds between 3000 & 4500 RPM per Fan
Color
Status
Description
FAN
Solid
Fan-Tray OK
• Capable of cooling Slots operating up to 800W per Slot
FAN
Solid
Fan-Tray Fault
• Can still operate with up to 3 individual Fan failures
ID
Solid
Identifies Fan-Tray
• Supports Fan-Tray “OIR” for minimum of 120 seconds
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
15
Catalyst 6807-XL
C6800-XL-3KW-AC
PSU Redundancy & Inputs
Dual Variable
Speed Fans
AC
ONLY
Power Supply
Status LEDs
Power Switch
PSU Lock
& Ejector
PSU 4
AC Input 3
PSU 3
AC Input 4
PSU 1
AC Input 2
PSU 2
AC Input 1
Power Supply Highlights:
• Max output is 3000W @ 220V (or 1300W @ 110V)
LED
Color
Status
Description
IN
Solid
Input OK
• Up to 92% Power Efficiency @ 50-100% of load
IN
Blinking
Under-Current
• Power Hold-up Time is ~20 msec @ 100% load
OUT
Solid
Output OK
• Dual “Front to Back” Variable-Speed Cooling Fans
OUT
Blinking
Over-Current
• Supports Combined or Redundant (N+1) mode
Fault
Solid
Malfunction
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
16
Power Supply Redundancy
For Your
Information
Catalyst 6807-XL
Catalyst 6807-XL Can Utilize Four Power Supplies in Either Redundant or Combined Mode
N+1 Redundant Mode
Combined Mode
PSU3
PSU4
PSU3
PSU4
90%
0%
90%
90%
100%
90%
90%
90%
PSU1
PSU2
PSU1
PSU2
• Adds +1 to Total # of Redundant PSU
• Same operational behavior as 6500-E
• First PSU operates @ 100% of capacity
• Each PSU provides ~90% of capacity
• Each Additional PSU @ 90% (100+90*N),
with the +1 Redundant @ 0%
• The total system power is ~360% of the
capacity of a single PSU
• With 1+1, 2+1 & 3+1 redundancy: if one
PSU fails, then the +1 PSU will take over
• Pseudo-redundant behavior, but this is
not equivalent to 1:1 or N+1 redundancy.
• This mode is Recommended (Default)
• This mode is NOT Recommended
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
17
Agenda
 Background
 C6807-XL



Chassis & Power
Supervisor Architectures
Module Architectures
 C6880-X
 C6840-X
 L2 Forwarding
 L3 Forwarding
 Packet Walks
Supervisor Engine 2T
Bridging the Catalyst 6500 & 6800
QUICK FACTS
2Tbps
Switch Fabric
PFC4 (XL)
 Integrated 2Tbps Switch Fabric
Forwarding Engine
 Policy Feature Card 4
supports L2 / L3+ hardware acceleration
 Multilayer Switch Feature Card 5
single CPU for both L2 & L3
 2 x 10GE & 3 x 1GE Uplink Ports
MSFC5
Control Plane
Processor
2 x 10G (X2) and
x 1G (SFP) Uplinks
1G (RJ45) Mgmt Port
3
 Connectivity Management Processor
 Hardware support for L2, IPv4, IPv6,
Multicast, MPLS / VPLS, GRE, NAT,
VSS & Instant Access
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
19
Supervisor 2T
Block Diagram
Shared Bus Connector
Local-Bus
Fabric Connector
20G
2Tbps
Fabric ASIC
PFC4 (XL)
Fabric
ASIC 1
Bus
Replication
ASIC
20G
DRAM
Compact
Flash
1GE FDX
Serial
Port
Control Plane
CPU 1.5GHz
MGMT
Port
MAC
Fabric
Replication ASIC
MSFC5
Connectivity
MGMT
Processor
Layer 2 Engine
Fabric
ASIC 0
28 x 40G Channels
Bootdisk
Layer 3 Engine
USB
Port
Front Panel
SFP 1
Port ASIC 0
Port ASIC 1
CTS ASIC
CTS ASIC
SFP 2
BRKARC-3465
X2 1
SFP 3
X2 2
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
20
Supervisor Engine 6T
NEW
Taking Catalyst 6800 to the Next Level
Scalability &
Performance
6Tbps
SUP2T
SUP6T
6807-XL Bandwidth
220G /Slot
440G /Slot
6500-E Bandwidth
80G /Slot
80G /Slot
RP CPU
1.5Ghz
MPC8572
2.5Ghz
X86 Dual Core
Memory
2 - 4GB
DDR2 667Mhz
4 GB
DDR3 1333Mhz
Uplinks
2 x 10G (X2) &
3 x 1G (SFP)
2 x 40G (QSFP) &
8 x 1/10G (SFP+)
Advanced
Uplink Features
VSS,
MACSEC, SGT
VSS, IA,
MACSEC, SGT,
LISP, UCI
RJ45
CMP
RJ45 / SFP
Mgmt0
PFC4-E (XL)
Switch Fabric
Forwarding Engine
MSFC6
2 x 40G (QSFP) and
10G (SFP+) Uplinks
Control Plane
Processor
8x
2 x 1G (RJ45 / SFP) Mgmt
Mgmt Port
* No CFC or Service Module Support
Feature Parity with Sup2T from Day 1: 4000+ Features
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
21
Supervisor 6T
FIRE = Fabric Interface & Replication Engine
Block Diagram
6Tbps Fabric ASIC
Chassis
Mgmt (I2C)
EOBC
Switch
28 x 110G Channels
PCIe
2 x 20G
Bootdisk
eUSB
2 x 20G
PFC4E
Inband
Inband
(XL)
D
D
DD
R
DD
R3
DD
R33
R3
Control-Plane
CPU 2.5 GHz
FIRE
ASIC 0
4GB
D
D
R
3
FIRE
ASIC 1
D
D
R
3
2GB
2GB
RTC
1GE Inband
40G
Keystore
NVRAM
40G
Port / CTS
ASIC 0
I/O
FPGA
PHY
Port / CTS
ASIC 1
PHY XPT
40G
40G
PHY
MSFC6
USB
Console
USB
Drive
RJ45
& SFP
RJ45
Console
SFP
01-04
PHY
Baseboard
Front Panel
QSFP
09
BRKARC-3465
SFP
05-08
QSFP
10
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
22
MSFC = Multilayer Switch Feature Card
The “Software” Control Plane for the System, where IOS runs…
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
23
Multilayer Switch Feature Card
Sup2T - MSFC5 Recap
• MSFC5 Control-Plane
• Combines functionalities of
the Switch Processor (SP) and
the Router Processor (RP)
• 1.5GHz PPC CPU
• 2 - 4GB DDR2 DRAM
• Internal Bootdisk & External Disk0
Compact Flash File System
• Built-in Connectivity Management
Processor (CMP) RJ45 Port
• Serial Console via RJ45 or USB
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
24
Multilayer Switch Feature Card
NEW
Sup6T - MSFC6 Introduction
• MSFC6 Control-Plane
• Integrated onto Baseboard
• Same behavior as MSFC5
• Higher Performance & Scale
• 2.5GHz IBC CPU
• 4GB DDR3 DRAM
• Internal Bootdisk & External Disk0
USB File System
• Built-in Out-of-band Management
RJ45 / SFP Port
• Serial Console via RJ45 or USB
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
25
MSFC – Control-Plane Processor
The combined Control-Plane Processor combines all L2 & L3+ system functions
MSFC Complex
EOBC
Bootdisk
DRAM
Control-Plane
Processor
SP
RP
Functions
Functions
Inband Intf
Runs Layer 2 & System functions:
• System operations like Boot, EOBC,
Chassis & Power Management, etc.
• L2 features like VTP, DTP, STP, CDP,
IGMP, EtherChannel, etc.
Run Layer 3 & User Interface functions:
• System management via Console,
Telnet/SSH, SNMP, TCL, etc.
• L3 features like HSRP, OSPF, BGP,
MPLS, PIM, etc.
1Gbps Full-Duplex “Inband” CPU interface
Communicates with HW via EOBC interface
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
26
MSFC3 vs. MSFC5 vs. MSFC6
For Your
Information
Feature
MSFC3 (Sup720-10G)
MSFC5 (Sup2T)
MSFC6 (Sup6T)
CPU Speed
SP CPU – PPC 600Mhz
RP CPU – PPC 600Mhz
RP CPU – PPC 1.5Ghz
RP CPU – IBC 2.5Ghz
DRAM
SP – 512MB (1GB upgrade)
RP – 512MB (1GB upgrade)
2GB default
(4GB upgrade)
4GB default
NVRAM
2MB
4MB
4MB
OBFL Flash
No
4MB
4MB
Bootdisk
SP – 1GB (CF)
RP – 64MB (flash)
1GB (CF)
2GB USB
Disk0:
Up to 1GB CF
Up to 8GB CF
Up to 8GB USB
EOBC
100Mbps HDX Bus
100Mbps HDX Bus
1Gbps FDX Switch
No
Yes - CMP
Yes – Mgmt0
Dedicated
Management Port
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
27
PFC = Policy Feature Card
The “Hardware” Control Plane, based on information learned by MSFC…
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
28
Policy Feature Card 4
Sup2T - PFC4 Recap
Also applies to DFC4
 Increased Hardware Performance
 60Mpps L2 / L3+ Forwarding
 New IFE / OFE Lookup Process
 Increased Hardware Scalability
 256K or 1M FIB TCAM Entries
 128K MAC Address CAM Entries
 64K or 256K Security & QoS ACL Entries
 512K or 1M Flexible NetFlow (FNF) Entries
 16K Virtual Routing & Forwarding (VRF) Instances
 New & Enhanced Feature Capabilities
EARL8
 SGT & MACSEC for Cisco Trustsec (CTS)
 IPv4 & IPv6 RPF check for up to 16 Paths
 Improved 8-bit EtherChannel Load-Balancing
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
29
Policy Feature Card 4
NEW
Sup6T - PFC4-E Introduction
Also applies to DFC4
 PFC4-E merges L2 & L3+ ASICs
 60Mpps L2 / L3+ Forwarding
 256K, 1M or 2M FIB TCAM Entries
 128K MAC Address CAM Entries
 Enhanced Performance & Equal Scale
 Improved Intra-ASIC RW Bandwidth
 Integrates external SRAMs
 Uses 4 sets of 32K x 96bit eDRAM
 Full ECC with additional 8 bits on RW
 Reduces pin count & block size
 Uses 3 RLDRAM3 chips @ 600 MHz
 Support for 2M FIB entries* (4 TCAMs)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
30
Policy Feature Card 4
Block Diagram
Contains CEF IPV4, IPV6
prefixes & MPLS entries
Contains location of
flow in Netflow Table
FIB TCAM
Contains Layer 2 rewrite
information & pointers
Collection of ADJ
statistics for each
active flow
Netflow TCAM
Netflow Table
Adjacency Table
Adjacency Statistics
Contains table of
“Exception” cases
& action to take
Exception Table
Collection of NF
statistics for each
active flow
Layer 3+
Netflow Statistics
Forwarding
Engine
LIF Map Table
Contains Logical
Interface Mapping info
RPF Map Table
Table of Src-Port info
for Multicast & uRPF
Classification ACL
Table #1
Classification ACL
Table #2
Contains the Ingress
ACL entries (128K)
Contains the actual LIF
Database entries
128K CAM contains
MAC address table
CAM Table
ACE Counters
Contains several key
packet fields for flow
Contains the Egress
ACL entries (128K)
LIF Table
Layer 2
Forwarding Engine
Collection of ACL “hit”
statistics & other info
LIF Statistics
Contains per-LIF
Usage statistics
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
31
*also applies to each DFC4
Policy Feature Card 4
EARL8 IFE / OFE Processing
The L3 Forwarding Engine ASIC has 2 processing pipelines @ 60Mpps:
1. Input Forwarding Engine (IFE)
2. Output Forwarding Engine (OFE)
STEP 1 - As each Header enters the L3 ASIC, the “IFE” pipeline performs an L3 Lookup and Ingress Security, QoS & NetFlow processing
STEP 2 - The Header is merged with IFE result and passed to the “OFE” pipeline, which does Egress Security, QoS & NetFlow
processing
Headers
From L2 Engine
1: Ingress
ACL
2: Ingress
NetFlow
3: DST L3
Lookup
4: Ingress
QoS
IFE Process
OFE Process
Headers
To L2 Engine
8: Rewrite
Result
7: Egress
QoS
6. Egress
NetFlow
BRKARC-3465
5: Egress
ACL
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
32
PFC3 vs. PFC4
For Your
Information
Feature
PFC3B/BXL
PFC3C/CXL
PFC4/XL
IPv4 Forwarding
Up to 30Mpps
Up to 48Mpps
Up to 60Mpps
IPv6 Forwarding
Up to 15Mpps
Up to 24Mpps
Up to 30Mpps
FIB TCAM (IPv4)
256K / 1M
256K / 1M
256K / 1M
FIB TCAM (IPv6)
128K / 500K
128K / 500K
128K / 500K
Adjacency Table
1M
1M
1M
Netflow Table
Up to 256K (XL)
Up to 256K (XL)
Up to 1M (XL)
(Ingress 512K : Egress 512K)
MAC Table
64K (32K)
96K (80K)
128K
Egress Netflow
No
No
Yes
Flexible Netflow
No
No
Yes
MPLSoGRE
No
No
Yes
IPv6 uRPF
No
No
Yes
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
33
PFC3 vs. PFC4
Feature
For Your
Information
PFC3B/BXL
PFC3C/CXL
PFC4/XL
ACL Labels
4K
4K
16K
Security ACEs
Up to 32K
Up to 32K
Up to 192K (XL Default)
QoS ACEs
Up to 32K
Up to 32K
Up to 64K (XL Default)
Port ACLs
2K
2K
8K
Aggregate Policers
1023
1023
6K
Shared Microflow Policers
63
63
512
Egress Microflow Policing
No
No
Yes
Distributed Policers
No
No
Yes
Packet or Byte Based Policing
No
No
Yes
RPF Interfaces
2
2
16
Native VPLS
No
No
Yes
VSS
No
Yes
Yes
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
34
Switch Fabric = Hardware Data Plane
A dedicated set of Crossbar Channels that interconnect all Slots…
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
35
2T Switch Fabric
Recap
• Integrated 2Tbps Switch Fabric
• 28 Channels for C6513-E & C6807-XL
• Dual Queues per Fabric Channel
• Redundant Channel to Standby Fabric
for sub-second convergence, during SSO
• Provides Backplane Interconnects
• 1 to 4 Fabric Channels supplied to each Slot
• Each Fabric Channel can independently
operate @ 20Gbps or 40Gbps
• Mixing old and new modules (e.g. 20G & 40G)
does not affect the speeds of other modules
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
36
Catalyst 6807-XL
Supports
up to 220G
per slot
How Sup2T operates
For Your
Information
Supervisor2T:
• Using the same XBAR Fabric ASIC
• Supports 1 to 4 channels (per Slot)
• Increased per Channel bandwidth
28 x 55G Channels
• New Clock Frequencies (7.5Ghz = 55G)
• New Line Encodings (24/26b or 64/66b)
• Applicable only to new C6800 Cards
• NO changes to the MSFC5 or PFC4
• Local Channel for Uplinks @ 20Gbps
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
37
6T Switch Fabric
NEW
Introduction
• Integrated 6Tbps Switch Fabric
• Same design & behavior as 2T Fabric
• 28 Channels for C6513-E & C6807-XL
• Up to 4 Channels operating @ 110Gbps
per Slot
• Provides Backplane Interconnects
• 1 to 4 Fabric Channels supplied to each Slot
• Each Fabric Channel can independently
operate @ 20Gbps, 40Gbps or 110Gbps*
• Mixing old and new modules (e.g. 40G & 110G)
does not affect the speeds of other modules
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
38
Crossbar Switch Fabric
A Closer Look
The Catalyst 6500 & 6800 series eliminated the earlier Bus-based
limits by creating a new “Crossbar” Switch Fabric for its backplane.
A Crossbar Architecture is essentially 2*N busses (where N is the number
of ASICs connected to the Switch Fabric) connected by N*N cross-points.
0
1
2
3
4
5
6
7
0
1
2
3
4
5
6
BRKARC-3465
7
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
39
Crossbar Switch Fabric
For Your
Information
More Details
This allows multiple LC’s to transmit and receive data simultaneously
A scheduler is responsible for selecting which LCs can transmit, and which
LCs receive data, during any given fabric cycle.
This can be viewed as (1 or more) dedicated Fabric channels to each LC
Per Channel bandwidth is defined by 2 factors:
0
1
2
3
4
5
6
7
•
Clock Speed (in Hz) defines the maximum BPS
•
Line Encoding (e.g. 8/10b) defines usable bits
Per Slot bandwidth is defined by # of Channels:
0
1
2
3
4
5
6
7
•
X bps * Y channels = Z bandwidth
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
40
Crossbar Switch Fabric
Logical Architecture
20G
Supervisor
20G
6800 Series
20G
20G
Service Module
20G
20G
40G
Switch
Fabric
6900 Series
40G
40G
40G
New 10G Series
6700 Series
20G
40G
40G
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
41
Switch Fabric Redundancy
For Your
Information
High Availability
•
When using a redundant Supervisor, 2 fabric channels are connected "back to back" *
• Standby Supervisor uplink connection to Active Supervisor
• Active Supervisor bus connection to Standby Supervisor
•
The Standby Supervisor is connected in DFC Mode, with its bus connection disabled
•
The redundant Standby Supervisor enables it’s fabric channels to dCEF2T or newer modules for ~50ms failover...
Line Card
Slot 1
Line Card
Slot 13
Active
Fabric
* “Back-to-Back”
Fabric Channels, for
sub-second (~50ms)
SSO switchover
Standby
Fabric
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
42
720G vs 2T vs. 6T Fabric
Feature
Number of Channels
Sup720
18
(20 on S720-10G)
For Your
Information
Sup2T
Sup6T
28
28
Aggregate Bandwidth
720 Gbps
2 Tbps
6 Tbps
Channel Speeds (bps)
8G - 20G
20G - 40G
20G - 40G - 110G
Fabric Redundancy
Yes
Yes
Yes
Yes
Yes
SSO Fabric Hot Synch
No
(Yes on S720-10G)
Redundant Channels
No
Yes
Yes
Fabric Priority (QoS)
8Q Priority
8Q Hi Priority
8Q Lo Priority
8Q Hi Priority
8Q Lo Priority
Clear Block Support
Yes
Yes
Yes
Bus, Truncated, Compact
Truncated, Compact
Compact Only
Switching Modes
(DBUS Header Size)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
43
Catalyst 6807-XL
Fabric Channel Distribution
1
2
3
4
Active
Fabric
Slot 1
1
2
3
4
1
2
3
4
Slot 2
1
2
3
4
1
2
3
4
Slot 3
1
2
3
4
1
2
3
4
Slot 4
1
2
3
4
1
2
3
4
Slot 5
1
2
3
4
1
2
3
4
Slot 6
1
2
3
4
Slot 7
1
2
3
4
1
2
3
4
Current Fabric Channels
Sup (HA) Fabric Channels
Extra Fabric Channels
(for future use)
Standby
Fabric
Each “Channel” can
operate at the following
Clock Frequencies:
• 3.13 GHz for 20 Gbps
• 6.25 GHz for 40 Gbps
• 7.50 GHz for 55 Gbps
• 15.0 GHz for 110 Gbps
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
44
Agenda
 Background
 C6807-XL



Chassis & Power
Supervisor Architectures
Module Architectures
 C6880-X
 C6840-X
 L2 Forwarding
 L3 Forwarding
 Packet Walks
Catalyst 6500-E & 6807-XL Line Cards
6700 & 6800 Series
6900 Series
with CFC or DFC4
with DFC4
New C6800 1G Series
New C6800 10G Series
with DFC4-E
with DFC4-E
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
46
For Your
Information
CFC = Centralized Forwarding Card
CFC connects to DBUS/RBUS, so the PFC can perform Forwarding Lookup
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
47
For Your
Information
Centralized Forwarding Card (CFC)
The Centralized Forwarding Card (CFC) provides BUS connectivity
for centralized (via Supervisor PFC) forwarding lookups ONLY…
The CFC comes on
legacy 6700 modules
to provide connection
to the DBUS & RBUS
All L2 / L3 Forwarding
“decisions” are made by
the PFC and “results” are
returned on the RBUS
Actual DATA Switching
is via the Switch Fabric...
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
48
For Your
Information
6700 Series (with CFC)
WS-X6704-10G, WS-X6748-SFP/GETX
Supervisor
Linecard
20G
The 6700 modules
connect to the BUS
using a Centralized
Forwarding Card (CFC)
FIRE ASIC
They connect to the
Switch Fabric via Dual
20G Fabric channels
Switch Fabric
20G
CFC
FIRE = Fabric Interface & Replication Engine
FIRE ASIC
PORT
ASIC
PORT
ASIC
PORT
ASIC
PORT
ASIC
BRKARC-3465
NOTE: The DBUS is
NOT used. Its ONLY
for control traffic
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
49
DFC = Distributed Forwarding Card
DFC enables Local (Distributed) Forwarding Lookup on each Module
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
50
Distributed Forwarding Card 4
The DFC4 is an option for 6700 Series, and pre-installed on 6800 & 6900 Series
Each DFC4 stores a “Local Copy” of PFC4 forwarding info, as well as NetFlow, Security & QoS ACL’s
The DFC4 supports local
forwarding rates up to 60Mpps
Two models of the DFC4
are supported:
The DFC4 includes same IFE / OFE
capabilities & increased table sizes
• DFC4-A (XL) for 1G
• DFC4-E (XL) for 10G+
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
51
6700 & 6800 Series (with DFC)
WS-X6748-SFP/TX or WS-X6848-SFP/TX
Supervisor
DFC
Linecard
FIRE = Fabric Interface & Replication Engine
Switch Fabric
20G
FIRE ASIC
PORT
ASIC
PORT
ASIC
For Your
Information
20G
The 6700 Series
supports a Distributed
Forwarding Card (DFC)
FIRE ASIC
The 6800 Series has
preinstalled Distributed
Forwarding Card 4
PORT
ASIC
PORT
ASIC
BRKARC-3465
They connect to the
Switch Fabric via
Dual 20G Channels
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
52
WS-X6704-10G
For Your
Information
6700 Series Line Cards
• 4 ports 10G Xenpak
• Supports CFC or DFC4-A
• Up to 60Mpps with DFC installed
• 2 x 20G Channels to Switch Fabric
• Connection to the Shared Bus
• 16MB Packet Buffers per port
• Supports Strict Priority queue on TX
• Supports 2 receive queues per port
• Supports 8 transmit queues per port
• Supports Weighted Round Robin
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
53
WS-X6748-GETX & 6848-TX
For Your
Information
6700 and 6800 Series Line Cards
• 48 ports 10/100/1000M RJ45
• Supports CFC or DFC4-A
• Up to 60Mpps with DFC installed
• 2 x 20G Channels to Switch Fabric
• Connection to the Shared Bus
• 1.2MB Packet Buffers per port
• Supports Strict Priority queue on TX
• Supports 2 receive queues per port
• Supports 4 transmit queues per port
• Supports Weighted Round Robin
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
54
WS-X6748-SFP & 6848-SFP
For Your
Information
6700 and 6800 Series Line Cards
• 48 ports 1G SFP
• Supports CFC or DFC4-A
• Up to 60Mpps with DFC installed
• 2 x 20G Channels to Switch Fabric
• Connection to the Shared Bus
• 1.2MB Packet Buffers per port
• Supports Strict Priority queue on TX
• Supports 2 receive queues per port
• Supports 4 transmit queues per port
• Supports Weighted Round Robin
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
55
WS-X6716-10G & 6816-10G
For Your
Information
6700 and 6800 Series Line Cards
• 16 ports 10GE (X2) in MUX mode
• 4 Programmable “port-groups” in 2 modes
• Requires integrated DFC4-E
• Up to 60Mpps with DFC installed
• 2 x 20G Channels to the Switch Fabric
• VSL support on 4 ports in Transparent mode
• 256MB Packet Buffers per-port
• Supports Strict Priority Queue on TX
• Supports 2 receive queues per port
• Supports 8 transmit queues per port
• Supports Weighted Round Robin &
Shaped Round Robin
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
56
WS-X6716-10T & 6816-10T
For Your
Information
6700 and 6800 Series Line Cards
• 16 ports 10GE (RJ45) in MUX mode
• 4 Programmable “port-groups” in 2 modes
• Requires integrated DFC4-E
• Up to 60Mpps with DFC installed
• 2 x 20G Channels to the Switch Fabric
• VSL support on 4 ports in Transparent mode
• 256MB Packet Buffers per-port
• Supports Strict Priority Queue on TX
• Supports 2 receive queues per port
• Supports 8 transmit queues per port
• Supports Weighted Round Robin &
Shaped Round Robin
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
57
6700/6800 Series Cards
For Your
Information
WS-X6716-10G/T or 6816-10G/T
Supervisor
Switch Fabric
20G
EOBC
20G
BACKPLANE INTF
Fabric
Interface
FPGA
Fabric
Interface
FPGA
FIRE
ASIC
FIRE
ASIC
PORT
ASIC
The 6716 & 6816 use
preinstalled Distributed
Forwarding Card 3 or
can upgrade to DFC4
DFC3/4
PORT
ASIC
MUX
PORT
ASIC
MUX
PORT
ASIC
MUX
They connect to the
Switch Fabric via
Dual 20G Channels
MUX
MUX
MUX
MUX
MUX
MUX
MUX
MUX
MUX
1
3
5
7
9
11
13
15
2
4
6
8
10
12
14
BRKARC-3465
16
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
58
WS-X6908-10G
For Your
Information
6900 Series Line Cards
• 8 ports 10GE (X2 based)
• Pre-Installed with DFC4-E
• Up to 60Mpps with DFC installed
• 2 x 40G Channels to Switch Fabric
• Supports Cisco TrustSec (CTS)
• MacSec encryption on all ports
• Supports VSL on all ports
• 256MB Packet Buffers per port
• Supports Strict Priority Queue on TX
• Supports 2 receive queues per port
• Supports 8 transmit queues per port
• Supports Deficit Weighted Round Robin
& Shaped Round Robin
X2-10G-SR/LR
CVR-SFP10G
(OneX)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
59
For Your
Information
6900 Series Cards
WS-X6908-10G
FIRE = Fabric Interface & Replication Engine
Supervisor
Switch Fabric
40G
EOBC
The 6900 Series use
preinstalled Distributed
Forwarding Card 4
40G
FABRIC INTERFACE
They connect to the
Switch Fabric via
Dual 40G Channels
DFC4
FPGA
FPGA
FIRE
ASIC
FIRE
ASIC
FIRE
ASIC
FIRE
ASIC
PORT
ASIC
PORT
ASIC
PORT
ASIC
PORT
ASIC
PORT
ASIC
PORT
ASIC
PORT
ASIC
PORT
ASIC
CTS
ASIC
CTS
ASIC
CTS
ASIC
CTS
ASIC
CTS
ASIC
CTS
ASIC
CTS
ASIC
CTS
ASIC
BRKARC-3465
They come with a
Cisco Trust Security
(CTS) ASIC built in
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
60
WS-X6904-40G
For Your
Information
6900 Series Line Cards
• 4 ports 40GE (CFP)
• 16 ports 10GE (FourX & SFP+)
• Pre-Installed with DFC4-E
• Up to 60Mpps with DFC installed
• 2 x 40G Connections to Switch Fabric
• Supports Cisco TrustSec (CTS)
• MacSec encryption on all ports
• Supports VSL & IA on all ports
• 10MB Packet Buffers per 10G port
• 40MB Packet Buffers per 40G port
• Supports Dual Strict Priority Queue on TX
• Supports 2 receive queues per port
• Supports 8 transmit queues per port
• Supports Deficit Weighted Round Robin
& Shaped Round Robin
CFP-40G-LR4
CFP-40G-SR4
CVR-4SFP10G
(FourX)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
61
For Your
Information
6900 Series Cards
WS-X6904-10G
FIRE = Fabric Interface & Replication Engine
Supervisor
Switch Fabric
EOBC
40G
40G
20G
20G
Fabric ASIC
Fabric Interface
ASIC
Fabric Interface
ASIC
Replication
Engine
Replication
Engine
RX MUX
FPGA
Inband
FPGA
Inband
FPGA
DFC4
TX MUX
FPGA
Port ASIC
20G
20G
Fabric Interface
ASIC
Fabric Interface
ASIC
Replication
Engine
Replication
Engine
RX MUX
FPGA
PHY / CTS ASIC
40 G CFP - Port 1
S
F
P
5
S
F
P
6
S
F
P
7
PHY / CTS ASIC
40 G CFP - Port 2
S
F
P
8
S
F
P
9
TX MUX
FPGA
Port ASIC
S
F
P
1
0
S
F
P
1
1
S
F
P
1
2
CFP
Daughter Card
40 G CFP - Port 3
S
F
P
1
3
BRKARC-3465
S
F
P
1
4
S
F
P
1
5
40 G CFP - Port 4
S
F
P
1
6
S
F
P
1
7
S
F
P
1
8
S
F
P
1
9
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
S
F
P
2
0
62
Catalyst 6500-E & 6807-XL
For Your
Information
High Performance Integrated L4-L7 Service Modules
Integrate Wired &
Wireless Management
BYOD
Next Generation
Wireless Controller: WISM2
Accelerate & Balance
Application Performance
Next Generation
Load Balancer: ACE-30
Performance 20 Gbps
16 Gbps
Performance
Access Points 500 - 1000
6 Gbps
Compression
30,000
Transactions per Second
Wireless Clients 15,000 per WISM2
Concurrent AP Upgrades Up to 500
Mobility, Domain Size Up to 18,000 APs
Enhance Visibility,
Accelerate Troubleshooting
NMS
Next Generation
Network Analysis: NAM3
Monitoring Performance 16 Gbps
Capture to External Disk 6 Gbps
Performance Analytics 1588 Timestamps
Hardware Filters & SPAN, FnF, SNMP
Packet Captures Port Monitoring
250
Virtual Context
4000
VLANs
SLB
Deliver Robust,
Integrated, Streamlined Security
Next Generation
Firewall & DPI: ASA-SM
64 Gbps
16 Gbps
10,000,000
300,000
250
1,000
BRKARC-3465
SEC
System Performance
Performance per ASA-SM
Concurrent Sessions
Connections per Second
Security Contexts
VLANs
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
63
Network Analysis with NAM-3
WS-SVC-NAM3-6G-K9
20G Fabric Connector
Memory (24 GB)
SAS Controller
Internal
Hard Disk
(600 GB SAS)
miniSAS
1GE /
1588 Sync
2 X86 (6 Core)
2GHz CPU
10GE SFP+
/ FCoE
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
64
Wireless Services with WISM-2
WS-SVC-WISM2-K9
One device for converged Wireless and Wired Services
supporting next-generation wiring closet infrastructures…
Reduced Operational Costs
20G Fabric Channel
• Scalability
1000 Access Points
15,000 Clients
2 x 6-Core 2Ghz
Control Processor
2 x 6-Core 2Ghz
Data Processor
• Central Maintenance
Troubleshooting
Simultaneous AP Upgrade
• Wireless Mobility
PRIME
36,000 AP in Domain
Fast Roaming
• Performance
16+ Gbps Throughput
• New Features
Wireless AVC
Flexible NetFlow
Bonjour Gateway
NMSP Location Services
Stateful AP Failover with VSS
ISE
Status LEDs
Serial & USB
Console Ports
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
65
Firewall & VPN Services with ASA-SM
WS-SVC-ASA-SM1-K9
Multi-Gigabit Fabric
Multiple Contexts (250)
• 20G Connection
• High Capacity
• Virtualized interfaces
• Memory for high session counts
• Module-to-module
• 24 GB of memory
communications
Dual-Crypto Accelerators
• Hardware Processing
• Accelerated Virtual Private Networking
• Unified Communications Encryption
Security Service Processors
• Multi-Services Capable
• Dedicated 64-bit multi-core Processors
• Future-proof Hardware
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
66
C6800-48P-TX
Based on
WS-X6848-TX
Updated 1GE Line Cards
48 ports of RJ45
• 40Gbps Capacity per Slot
• Single DFC4 for 60Mpps
• Integrated DFC-4A (XL)
• Updated Port ASICs
• 1.5MB per Port Buffers
• New Hardware RFID
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
67
C6800-48P-TX
Block Diagram
FIRE = Fabric Interface & Replication Engine
FIRE = Fabric Interface & Replication Engine
Supervisor
12 x 1G Ports per Port ASIC
• DFC4-A supports 48 x 1G Ports
Switch Fabric
20G
20G
EOBC
Fabric
ASIC
• Improved FIRE & Fabric ASICs
• 20Gbps per Fabric Channel
Fabric
ASIC
BACKPLANE INTF
20G
FPGA
20G
FPGA
FIRE
ASIC
• 20Gbps per FIRE ASIC
• 1.2:1 Oversubscribed @ Port ASIC
10G
• New Port ASIC with more Buffers
FIRE
ASIC
DFC4
10G
Port
ASIC
Port
ASIC
Port
ASIC
10G
10G
Port
ASIC
• 1.5MB RX and 1.5MB TX
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
25-28
29-32
33-36
37-40
41-44
45-48
Front
Panel
01-04
05-08
09-12
13-16
17-20
BRKARC-3465
21-24
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
68
C6800-48P-SFP
Based on
WS-X6848-SFP
Updated 1GE Line Cards
48 ports of SFP
• 40Gbps Capacity per Slot
• Single DFC4 for 60Mpps
• Integrated DFC-4A (XL)
• Updated Port ASICs
• 1.5MB per Port Buffers
• New Hardware RFID
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
69
C6800-48P-SFP
Block Diagram
FIRE = Fabric Interface & Replication Engine
FIRE = Fabric Interface & Replication Engine
Supervisor
12 x 1G Ports per Port ASIC
• DFC4-A supports 48 x 1G Ports
Switch Fabric
20G
20G
EOBC
Fabric
ASIC
• Improved FIRE & Fabric ASICs
• 20Gbps per Fabric Channel
Fabric
ASIC
BACKPLANE INTF
20G
FPGA
20G
FPGA
FIRE
ASIC
• 20Gbps per FIRE ASIC
• 1.2:1 Oversubscribed @ Port ASIC
10G
• New Port ASIC with more Buffers
FIRE
ASIC
DFC4
10G
Port
ASIC
Port
ASIC
Port
ASIC
10G
10G
Port
ASIC
• 1.5MB RX and 1.5MB TX
PHY
PHY
PHY
PHY
PHY
PHY
ODD
ODD
ODD
ODD
ODD
ODD
01-07
09-15
17-23
25-31
33-39
41-47
BRKARC-3465
PHY
Front
Panel
PHY
PHY
PHY
PHY
PHY
EVEN
EVEN
EVEN
EVEN
EVEN
EVEN
02-08
10-16
18-24
26-32
34-40
42-48
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
70
C6800-32P10G
Based on
WS-X6904-40G
New High Density 10GE Line Cards
32 ports of SFP/SFP+
Up to 8 ports of QSFP*
• 160Gbps Capacity per Slot
• Dual DFC4 for 120Mpps
• 2 x Integrated DFC-4E (XL)
• Support for 1/10GE or 40GE*
• 250 / 500MB Per Port TX Buffers
• VSL and IA capable on all ports
• New Hardware RFID
* with new CVR-4SFP-QSFP adapter cable
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
71
C6800-32P10G
Block Diagram
FIRE = Fabric Interface & Replication Engine
FIRE = Fabric Interface & Replication Engine
Fabric ASIC of Active Sup
4 x 8 SFP+ Port-Groups
40G
40G
40G
40G
40G
2x20G
• New FIRE & Fabric ASICs
40G
Backplane
2x20G
DFC
4
DFC
4
Inband
• 40Gbps per Fabric Channel
Inband
• 40Gbps per FIRE ASIC
FIRE
ASIC
• 2:1 Oversubscribed @ Port ASIC
FIRE
ASIC
• Performance Mode per Port-Group
• 1.25 / 2.5MB RX and 250 / 500MB TX
40G
40G
Local Fabric ASIC
• DFC4-E supports 16 x SFP Ports
• 1/10 or 40GE Mode per Port-Group*
Fabric ASIC of Standby Sup
40G
40G
Port ASIC
Port ASIC
Inband
Inband
FIRE
ASIC
FIRE
ASIC
40G
40G
Port ASIC
Port ASIC
PHY
PHY
PHY
PHY
PHY
PHY
PHY
SFP
SFP
SFP
SFP
SFP
SFP
SFP
SFP
01,03,05,07
09,11,13,15
17,19,21,23
25,27,29,31
02,04,06,08
10,12,14,16
18,20,22,24
26,28,30,32
Front
Panel
PHY
* with new CVR-4SFP-QSFP adapter cable
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
72
C6800-16P10G
Based on
WS-X6904-40G
New High Density 10GE Line Cards
16 ports of SFP/SFP+
Up to 4 Ports of QSFP*
• 80Gbps Capacity per Slot
• Single DFC4 for 60Mpps
• Integrated DFC-4E (XL)
• Support for 1 / 10GE or 40GE*
• 250 / 500MB Per Port TX Buffers
• VSL and IA capable on all ports
• New Hardware RFID
* with new CVR-4SFP-QSFP adapter cable
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
73
C6800-16P10G
Block Diagram
FIRE = Fabric Interface & Replication Engine
FIRE = Fabric Interface & Replication Engine
Fabric ASIC of Active Sup
2 x 8 SFP+ Port-Groups
• DFC4-E supports 16 x SFP Ports
40G
40G
40G
40G
Backplane
2x20G
• 40Gbps per Fabric Channel
• 40Gbps per FIRE ASIC
DFC
4
Inband
FIRE
ASIC
• 2:1 Oversubscribed @ Port ASIC
• 1.25 / 2.5MB RX and 250 / 500MB TX
40G
40G
2x20G
Inband
• 1/10 or 40GE Mode per Port-Group*
40G
40G
Local Fabric ASIC
• New FIRE & Fabric ASICs
• Performance Mode per Port-Group
Fabric ASIC of Standby Sup
FIRE
ASIC
40G
40G
Port ASIC
PHY
Port ASIC
PHY
SFP
SFP
01-04
05-08
PHY
Front
Panel
PHY
SFP
SFP
09-12
13-16
* with new CVR-4SFP-QSFP adapter cable
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
74
C6800-8P10G
Based on
WS-X6904-40G
New High Density Multi-Rate Line Cards
8 ports of SFP/SFP+
Up to 2 Ports of QSFP*
• 80Gbps Capacity per Slot
• Single DFC4 for 60Mpps
• Integrated DFC-4E (XL)
• 500MB Per Port TX Buffers
• Support for 1 / 10GE or 40GE*
• VSL and IA capable on all ports
• New Hardware RFID
* with new CVR-4SFP-QSFP adapter cable
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
75
C6800-8P10G
Block Diagram
FIRE = Fabric Interface & Replication Engine
Fabric ASIC of Active Sup
2 x 4 SFP+ Port-Groups
• DFC4-E supports 8 x SFP Ports
40G
40G
40G
• 1/10 or 40GE Mode per Port-Group*
• 2.5MB RX and 500MB TX
40G
40G
Backplane
2x20G
• 40Gbps per Fabric Channel
• No Need for Performance Mode
40G
2x20G
Inband
• Non Oversubscribed (1:1)
40G
40G
Local Fabric ASIC
• New FIRE & Fabric ASICs
• 40Gbps per FIRE ASIC
Fabric ASIC of Standby Sup
DFC
4
FIRE
ASIC
Inband
FIRE
ASIC
40G
40G
Port ASIC
Port ASIC
PHY
PHY
SFP
Front
Panel
01-04
SFP
05-08
* with new CVR-4SFP-QSFP adapter cable
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
76
Port ASIC
C6800 10G Modules
PHY
Port-Groups & Performance Mode
For Your
Information
PHY
SFP
SFP
01-04
05-08
C6800-32P10G
Port-Group 1
Port-Group 3
1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
32
14
15
16
Port-Group 2
Port-Group 4
C6800-16P10G
Port-Group 1
1
2
3
4
5
Port-Group 2
6
7
8
9
10
11
12
13
C6800-8P10G
Port-Group 1
1
2
3
4
Port-Group 2
5
6
7
BRKARC-3465
8
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
77
Agenda
 Background
 C6807-XL
 C6880-X



Chassis & Power
Fixed Supervisor
Modular Port Card
 C6840-X
 L2 Forwarding
 L3 Forwarding
 Packet Walks
Catalyst 6880-X
For Your
Information
Semi-Fixed Chassis Overview
Catalyst 6000
Feature Set
5 RU
(4 Half Slots)
Up to 220G/Slot
Backplane
(e.g. MPLS/VPLS,
LISP, Multicast, IPv6,
CTS, VSS, FEX)
Low-Power & Noise
High-Efficiency Fans
Powerful X86
2.0GHz CPU
& 4GB DRAM
Fixed RP Slot with
Built-in 16P10G
Up to 2M FIB
Up to 5M Netflow
Up to 256K ACL
3000W (AC/DC)
Power Supplies
1:1 Redundant PS
(Front Serviceable)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
79
Catalyst 6880-X
For Your
Information
Mechanical View
Fan-Tray
4 Modular Half-Slots
Fixed
Uplink Ports
Slot 5: Fixed
Supervisor
Power Supplies
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
80
Catalyst 6880-X
Environmental Overview
Selectable
In-Reach Depth
21.4” to 11.7”
Height
8.5” (5RU)
High Efficiency
4500 RPM
Redundant Fans
Depth
23.0”
Platinum Efficient
3000W
Power Supplies
Width
17.35”
Two-Post Rack
Custom Mount Kit
Flexible Mounting Brackets
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
81
Catalyst 6880-X
Air Flow:
Side to Side
Fan Redundancy & Air Flow
Fan-Tray LEDs
Fan-Tray
Thumb-Screws
Individual
Fans
(1 column of 4)
Fan-Tray Handle
Front-Serviced
Fan-Tray Card
Fan-Tray Highlights:
• Has 4 variable-speed High-Efficiency Fans (250 CFM)
LED
• Supports 4 speeds between 3000 & 4500 RPM per Fan
Color
Status
Description
FAN
Solid
Fan-Tray OK
• Capable of cooling Slots operating up to 800W per Slot
FAN
Solid
Fan-Tray Fault
• Can still operate with up to 2 individual Fan failures
ID
Solid
Identifies Fan-Tray
• Supports Fan-Tray “OIR” for minimum of 120 seconds
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
82
Catalyst 6880-X
PSU Redundancy & Inputs
Power Supply
Status LEDs
C6880-X-3KW-AC (DC)
AC Input
AC
or DC
Power Switch
PSU Fans
PSU Lock
& Ejector
PSU Handle
PSU 1
PSU 2
Status
Description
IN
Solid
Input OK
IN
Blinking
Under-Current
• Up to 92% Power Efficiency @ 100% of load
OUT
Solid
Output OK
• Power Hold-Up Time is ~20 msec @ 100% load
OUT
Blinking
Over-Current
• Dual “Front to Back” Variable-Speed Cooling Fans
Fault
Solid
Malfunction
• Supports Combined or Redundant (1:1) mode
ID
Solid
Identifies PSU
Power Supply Highlights:
• Max output is 3000W @ 220V (or 1300W @ 110V)
LED
BRKARC-3465
Color
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
83
Power Supply Redundancy
For Your
Information
Catalyst 6880-X
The Catalyst 6880-X Utilizes Two Power Supplies in Either Redundant or Combined Mode
1:1 Redundant Mode
50%
PSU 1
50%
PSU 2
Combined Mode
90%
PSU 1
90%
PSU 2
• Each PSU provides ~50% of power needs
• Each PSU provides up to 90% of its capacity
• Neither PSU operates at >60% or <40% capacity
• Total power available is 180% of a single supply
• Either PSU can power the system on its own
• A single PSU may not have power for the system
• This mode is Recommended (Default)
• This mode is NOT Recommended
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
84
Catalyst 6880-X
Based on
WS-X6904-40G
Fixed Supervisor Base Board
System
Base Board
Forwarding
Daughter Board
Two HW Options
6880-X-LE
6880-X
IPv4 Routes
256K
2M
IPv6 Routes
128K
1M
16 x SFP+ Ports:
Multicast Routes
64K
128K
MAC Addresses
128K
128K
Security ACL Entries
128K
128K
QoS ACL Entries
64K
256K
VSS, IA (FEX),
LISP, MPLS, HQoS,
MACSEC, SGT,
available on Every Port
Flexible NetFlow Entries
512K
1M
USB Host (Type A)
USB Console (Type B)
RJ-45 Console and
Management Ports
Enhanced Control-Plane Scale with new X86 2GHz RP CPU
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
85
Catalyst 6880-X:
Fixed Supervisor Design
3 Main Components:
• RP Complex
• Baseboard
• Switch Fabric
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
86
Catalyst 6880-X
C6880X-SUP - RP Complex
Based on Sup2T MSFC5
• 2.0GHz X86 IBC CPU
• 2 or 4GB of DDR3 DRAM
• EOBC & PCIe Switch Interface
• 2GB eUSB Bootdisk
• Direct RJ45 Ethernet Port (Mgmt0)
• USB Type A File System (Disk0)
• USB Type B Serial Console
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
87
Catalyst 6880-X
C6880-X-SUP – Uplink Ports
20Gbps
20Gbps
FIRE = Fabric Interface & Replication Engine
20Gbps
20Gbps
Based on Sup2T + 6904-40G
•
16 ports of 1/10G SFP+
•
PHY
PHY
SFP
01-04
SFP
05-08
Front
Panel
PHY
PHY
SFP
09-12
SFP
13-16
2 x 8 SFP+ Port-Groups
•
Enhanced PFC4-E Forwarding Engine
•
80Gbps to Switch Fabric (2 Modes)
•
Improved 40Gbps Fabric/Replication ASIC
•
New 40Gbps Port Interface MUX FPGA
•
RLDRAM Packet Buffers on MUX FPGA
•
1.25 or 2.5MB RX per Port (10MB per Port ASIC)
•
24 or 48MB TX per Port (192MB per MUX FPGA)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
88
Catalyst 6880-X:
Fabric Channel Distribution
Slot 2
Slot 1
1
2
3
4
1
2
3
4
Current Fabric Channels
1
4
1
2
3
Slot 4
2
3
2
1
Slot 3
4
Switch
Fabric
3
Extra Fabric Channels
(for future use)
Each “Channel” can
use any of the following
Clock Frequencies:
• 6.25 GHz for 40 Gbps
• 7.50 GHz for 55 Gbps
4
Slot 5
Baseboard
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
89
Catalyst 6880-X
Based on
WS-X6904-40G
C6880-X-16P10G - 16-port SFP+ Multi-Rate Port Card
Port Status
LED
Port Card
Status LED
Port Card
Base Board
Ejector
Lever
16 x 10/1G
SFP Ports
Port Card
ID LED
Forwarding Engine
Daughter Board
Two Versions
Standard (LE)
Large Tables
Port Speed & Type
Number of Ports
FIB Table v4/v6
256K/128K
2M/1M
10/100/1000 Mb/s Copper
16 (GLC-T)
NetFlow Table
512K
1M
1 Gb/s Fiber
16 (SFP)
Security ACL Table
64K
256K
10 Gb/s Fiber
16 (SFP+)
Port Buffering
48MB / Port
48MB / Port
40 Gb/s Fiber
4 (SFP-QSFP)
MacSec, FEX, VSS, LISP, SGT, 1588 Capable on Every Port
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
90
Catalyst 6880-X
C6880-X-16P10G Architecture
FIRE = Fabric Interface & Replication Engine
Same as Baseboard + 1.2Ghz LCP
•
16 ports of 1/10G SFP+
•
PHY
PHY
SFP
01-04
SFP
05-08
Front
Panel
2 x 8 SFP+ Port-Groups
•
Enhanced DFC4-E Forwarding Engine
•
80Gbps to Switch Fabric (2 Modes)
•
Improved 40Gbps Fabric/Replication ASIC
•
New 40Gbps Port Interface MUX FPGA
•
RLDRAM Packet Buffers on MUX FPGA
PHY
PHY
•
1.25 or 2.5MB RX per Port (10MB per Port ASIC)
SFP
09-12
SFP
13-16
•
24 or 48MB TX per Port (192MB per MUX FPGA)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
91
Agenda
 Background
 C6807-XL
 C6880-X
 C6840-X


Chassis & Power
Fixed Models
 L2 Forwarding
 L3 Forwarding
 Packet Walks
NEW
Catalyst 6840-X Chassis
For Your
Information
High-Level Overview
Catalyst 6000
Feature Set
2 RU
(Fixed Chassis)
Low-Power & Noise
High-Efficiency Fans
(e.g. MPLS/VPLS,
LISP, Multicast, IPv6,
CTS, VSS, FEX)
Up to 256K FIB
Up to 1.5M NetFlow
Up to 64K ACL
Powerful X86
2.0GHz CPU
& 4GB DRAM
4 Models with
up to 48 x 10G
and 12 x 40G
750/1100W (AC/DC)
Power Supplies
1:1 Redundant PS
(Front Serviceable)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
93
Catalyst 6840-X
For Your
Information
Mechanical View
Slot 1: Fixed
Supervisor
Rear-Serviced
Fan-Tray
Fixed
Uplink
Ports
1
3
5
7
9
11
13 15
17 19
2
4
6
8
10 12
14 16
18 20 22
29 31
33 35
37 39
25 27
21 23
24
41
42
43-46
47-50
PSU1
26 28
30 32
34 36
PSU2
38 40
Power Supplies
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
94
Catalyst 6840-X
NEW
Environmental Overview
Height
3.5 in (2RU)
High Efficiency
6500 RPM
Redundant Fans
Platinum Efficient
750 / 1100W
Power Supplies
Depth
21.8 in
Width
17.35 in
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
95
Catalyst 6840-X
Air Flow:
Front to Back
Fan Redundancy & Air Flow
Fan-Tray
Thumb-Screws
Individual
Fans
(2 set of 2)
Fan-Tray Handle
Airflow
Intake Vents
Fan-Tray Highlights:
• Has 4 variable-speed High-Efficiency Fans (200 CFM)
LED
• Supports 4 speeds between 6500 & 11000 RPM per Fan
• Capable of cooling of ALL 4 C6804-X chassis models
Color
Status
Description
FAN
Solid
Fan-Tray OK
FAN
Solid
Fan-Tray Fault
• Can still operate with up to 2 individual Fan failures
• Supports Fan-Tray “OIR” for minimum of 120 seconds
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
96
Catalyst 6840-X
PSU Redundancy & Inputs
AC or DC
PSU Fans
C6840-X-750W-AC (DC)
C6840-X-1110W-AC (DC)
Power Supply
Status LEDs
AC Input
PSU Handle
PSU Lock
& Ejector
PSU 2
PSU 1
Power Supply Highlights:
LED
• Two AC or DC models @ 110V: 750W or 1100W
Status
Description
IN
Solid
Input OK
• Up to 92% Power Efficiency @ 100% of load
IN
Blinking
Under-Current
• Power Hold-Up Time is ~20 msec @ 100% load
OUT
Solid
Output OK
• “Front to Back” Variable-Speed Cooling Fans
OUT
Blinking
Over-Current
Fault
Solid
Malfunction
• Supports Combined or Redundant (1:1) mode
BRKARC-3465
Color
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
97
Power Supply Redundancy
For Your
Information
Catalyst 6840-X
The Catalyst 6840-X Utilizes Two Power Supplies in Either Redundant or Combined Mode
1:1 Redundant Mode
50%
50%
PSU 1
PSU 2
Combined Mode
90%
90%
PSU 1
PSU 2
• Each PSU provides ~50% of power needs
• Each PSU provides up to 90% of its capacity
• Neither PSU operates at >60% or <40% capacity
• Total power available is 180% of a single supply
• Either PSU can power the system on its own
• A single PSU may not have power for the system
• This mode is Recommended (Default)
• This mode is NOT Recommended
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
98
Catalyst 6840-X Series
NEW
Fixed Chassis Portfolio
15.2(1)SY
C6832-X-LE
Q4CY2014
C6816-X-LE
15.2(1)SY
C6824-X-LE-40G
Q4CY2014
C6840-X-LE-40G
SFP/SFP+ and QSFP
Native Optics
SFP/SFP+
SFP/SFP+
SFP/SFP+ and QSFP
# of 10G Ports
16
32
24
40
+8 using breakout cable
+8 using breakout cable
# of 40G Ports
2
2
+4 using reverse adapter
+8 using reverse adapter
+6 using reverse adapter
+10 using reverse adapter
IPv4 / v6 Routes
256K / 128K
256K / 128K
256K / 128K
256K / 128K
Multicast Routes
128K / 64K
128K / 64K
128K / 64K
128K / 64K
MPLS Labels
256K
256K
256K
256K
MAC Addresses
128K
128K
128K
128K
Security ACL
64K (Shared)
64K (Shared)
64K (Shared)
64K (Shared)
QoS ACL
64K (Shared)
64K (Shared)
64K (Shared)
64K (Shared)
Flexible NetFlow
512K
1M
1M
1.5M
Enhanced Control-Plane Scale with X86 2GHz RP CPU
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
99
Catalyst 6840-X:
Fixed Chassis Design
3 Main Components:
• RP Complex
• Baseboard
• Switch Fabric
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
100
Catalyst 6840-X
Fixed Chassis - RP Complex
Based on C6880-X-SUP
• 2.0GHz X86 IBC CPU
• 4GB of DDR3 DRAM
• EOBC & PCIe Switch Interface
• 2GB eUSB Bootdisk
• Direct RJ45 Ethernet Port (Mgmt0)
• USB Type A File System (Disk0)
• USB Type B Serial Console
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
101
Catalyst 6840-X:
Fabric Channel Distribution
PG 3-4
PG 4-5
1
2
Fabric Channels
2
1
Switch
Fabric
Each “Channel” uses
the following Clock
Frequency:
• 6.25 GHz for 40 Gbps
1
2
PG 1-2
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
102
Catalyst 6840-X
NEW
C6816-X-LE
FIRE = Fabric Interface & Replication Engine
Based on C6800-16P10G
Local Fabric ASIC
2x20G
2x20G
Inband
DFC4
FIRE
ASIC
•
16 ports of 1/10G SFP+
•
Inband
FIRE
ASIC
40G
2 x 8 SFP+ Port-Groups
•
Enhanced DFC4-E Forwarding Engine
•
80Gbps to Switch Fabric (2 Modes)
•
New 40Gbps Fabric & Replication ASIC
•
Combines FIRE ASIC & MUX FPGA of 6880-X
•
DDR3 Packet Buffers on FIRE & Port ASIC
40G
Port ASIC
Port ASIC
PHY
PHY
PHY
PHY
SFP
01-04
SFP
05-08
SFP
09-12
SFP
13-16
Front
Panel
•
1.25 or 2.5MB RX per Port (10MB per Port ASIC)
•
250 or 500MB TX per Port (2GB per FIRE ASIC)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
103
Catalyst 6840-X
NEW
C6832-X-LE
FIRE = Fabric Interface & Replication Engine
Based on C6800-32P10G
Local Fabric ASIC
2x20G
•
2x20G
Inband
DFC4
•
Inband
FIRE
ASIC
40G
40G
Port ASIC
Port ASIC
PHY
PHY
PHY
4 x 8 SFP+ Port-Groups
Inband
DFC4
FIRE
ASIC
32 ports of 1/10G SFP+
Inband
FIRE
ASIC
PHY
SFP
SFP
SFP
SFP
01-04
05-08
09-12
13-16
FIRE
ASIC
40G
40G
Port ASIC
Port ASIC
PHY
Front
Panel
PHY
PHY
•
2 x DFC4-E Forwarding Engines
•
80Gbps to Switch Fabric (2 Modes)
•
New 40Gbps Fabric & Replication ASIC
•
Combines FIRE ASIC & MUX FPGA of 6880-X
•
DDR3 Packet Buffers on FIRE ASIC
PHY
SFP
SFP
SFP
SFP
17-20
21-24
25-28
29-32
•
1.25 or 2.5MB RX per Port (10MB per Port ASIC)
•
250 or 500MB TX per Port (2GB per FIRE ASIC)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
104
Catalyst 6840-X
NEW
C6824-X-LE-40G
FIRE = Fabric Interface & Replication Engine
Based on C6800-32P10G
Local Fabric ASIC
2x20G
•
2x20G
Inband
DFC4
•
Inband
FIRE
ASIC
40G
40G
Port ASIC
Port ASIC
PHY
PHY
PHY
3 x 8 SFP+ Port-Groups / 1 x 2 QSFP Port-Group
Inband
DFC4
FIRE
ASIC
24 ports of 1/10G SFP+ / 2 ports of 40G QSFP
Inband
FIRE
ASIC
PHY
SFP
SFP
SFP
SFP
01-04
05-08
09-12
13-16
FIRE
ASIC
40G
40G
Port ASIC
Port ASIC
PHY
Front
Panel
PHY
PHY
PHY
SFP
SFP
QSFP
QSFP
17-20
21-24
25
26
•
2 x DFC4-E Forwarding Engines
•
80Gbps to Switch Fabric (2 Modes)
•
New 40Gbps Fabric/Replication ASIC
•
Combines FIRE ASIC & MUX FPGA of 6880-X
•
DDR3 Packet Buffers on FIRE ASIC
•
1.25 or 2.5MB RX per Port (10MB per Port ASIC)
•
250 or 500MB TX per Port (2GB per FIRE ASIC)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
105
Catalyst 6840-X
NEW
C6840-X-LE-40G
FIRE = Fabric Interface & Replication Engine
Local Fabric ASIC
2x20G
2x20G
Based on C6800-32P10G
2x20G
Inband
Inband
DFC4
Inband
•
Inband
DFC4
FIRE
ASIC
40G
FIRE
ASIC
FIRE
ASIC
40G
Port ASIC
Port ASIC
PHY
PHY
PHY
PHY
SFP
01-04
SFP
05-08
SFP
09-12
SFP
13-16
Front
Panel
FIRE
ASIC
40 ports of 1/10G SFP+, 2 ports of 40G QSFP
•
40G
40G
Port ASIC
Port ASIC
PHY
PHY
PHY
PHY
SFP
17-20
SFP
21-24
SFP
25-28
SFP
29-32
Inband
Inband
DFC4
FIRE
ASIC
FIRE
ASIC
40G
40G
Port ASIC
Port ASIC
PHY
PHY
PHY
PHY
SFP
33-36
SFP Front QSFP
37-40 Panel 41
QSFP
42
5 x 8 SFP+ Port-Groups, 1 x 2 QSFP Port-Group
•
3 x DFC4-E Forwarding Engines
•
80Gbps to Switch Fabric (2 Modes)
•
New 40Gbps Fabric/Replication ASIC
•
Combines FIRE ASIC & MUX FPGA of 6880-X
•
DDR3 Packet Buffers on FIRE ASIC
•
1.25 or 2.5MB RX per Port (10MB per Port ASIC)
•
250 or 500MB TX per Port (2GB per FIRE ASIC)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
106
Agenda
 Background
 C6807-XL
 C6880-X
 C6840-X
 L2 Forwarding
 L3 Forwarding
 Packet Walks
Layer 2 Switching
For Your
Information
L2 Forwarding Steps
Frame received
1
L2 Table
Source MAC
Lookup
Destination MAC L2 Table
Lookup
2
Learn
New MAC?
Router MAC?
Yes
L2 Table
Yes
No
L3 forwarding
No
3
Update entry
L2 Table
L2 forwarding
Known MAC?
Yes
No
L2 flooding
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
108
Layer 2 Switching
Layer 2 MAC Table
MAC Table
Table
MAC
Port
PFC & DFC
has an CAM with
N pages x 4096 rows =
MAC address space
PFC
A
B
C
D
E
F
32
Pages
1
2
3
4
5
6
4096
Rows
MAC Table
PFC4 = 128K Entries
(32 x 4096)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
109
Layer 2 Switching
Layer 2 Forwarding Operation
Frame
32
Pages
VLAN
1
PFC
MAC
Hash
0000.2222.7777 | 20
0000.1111.cccc | 10
0000.dddd.a112 | 30
2
MAC Table Row
4096
Rows
0000.bbbb.ac1c | 30
HIT!
MAC Table
1. Hash result identifies the starting Page and Row in the MAC table
2. Lookup Key (VLAN + MAC) compared to contents on each page (sequentially)
- DST MAC Lookup: Match returns Destination interface(s) OR Miss results in Flood
- SRC MAC Lookup: Match updates age of current entry OR Miss installs new entry
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
110
Displaying the Layer 2 Table
For Your
Information
SUP2T#show mac address-table
Legend: * - primary entry
age - seconds since last seen; n/a - not available; S - secure entry;
R - router's gateway mac address entry; D - Duplicate mac address entry
Displaying entries from active supervisor:
vlan
mac address
type
learn
age
ports
----+----+---------------+-------+-----+----------+----------------------------*
192 00d0.0053.bc00 dynamic Yes
5
Gi7/3
R
205 0024.c4dc.d740
static
No
Router
R
20 0024.c4dc.d740
static
No
Router
*
192 0014.5e31.4220 dynamic Yes
65
Gi7/3
*
60 00d0.2bfc.23f5 dynamic Yes
30
Gi5/14
*
192 00e0.1e5d.e9ff dynamic Yes
30
Gi7/3
…
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
111
Catalyst 6500/6800
For Your
Information
Understanding EtherChannels
•
Combines Multiple physical ports into One logical port
•
Deterministic Hash-based Channel Load-Balancing
•
Configurable Hash uses SRC, DST, L2 and / or L3
•
Load Sharing is always Per Flow (Not Per Packet)
•
PFC3 hash algorithm supports 8 results (3 bits)
•
PFC4 hash algorithm supports 256 results (8 bits)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
112
EtherChannel Load-Balancing
For Your
Information
PFC3 Flow Distribution
Frame
1
2
3
4
5
6
7
8
EtherChannel Hash 3 bit Result
Channel
Bundle
Link1
Link2
Link3
Link4
Link5
Link6
Link7
Link8
2 Links
50%
50%
--
--
--
--
--
--
3 Links
37.5%
37.5%
25%
--
--
--
--
--
4 Links
25%
25%
25%
25%
--
--
--
--
5 Links
25%
25%
25%
12.5%
12.5%
--
--
--
6 Links
25%
25%
12.5%
12.5%
12.5%
12.5%
--
--
7 Links
25%
12.5%
12.5%
12.5%
12.5%
12.5%
12.5%
--
8 Links
12.5%
12.5%
12.5%
12.5%
12.5%
12.5%
12.5%
12.5%
Even distribution ONLY for Hash Combinations highlighted in RED!
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
113
EtherChannel Load-Balancing
For Your
Information
PFC4 Flow Distribution
Frame
1
2
3
4
…
256
EtherChannel Hash 8 bit Result
Channel
Bundle
Link1
Link2
Link3
Link4
Link5
Link6
Link7
Link8
2 Links
50%
50%
--
--
--
--
--
--
3 Links
33.6%
33.2%
33.2%
--
--
--
--
--
4 Links
25%
25%
25%
25%
--
--
--
--
5 Links
20.4%
19.9%
19.9%
19.9%
19.9%
--
--
--
6 Links
16.8%
16.8%
16.8%
16.8%
16.4%
16.4%
--
--
7 Links
14.5%
14.5%
14.5%
14.5%
14%
14%
14%
--
8 Links
12.5%
12.5%
12.5%
12.5%
12.5%
12.5%
12.5%
12.5%
Nearly Even distribution for ODD & EVEN Hash Combinations!
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
114
PFC4 EtherChannel Inputs
For Your
Information
EtherChannel Uses a Hash Algorithm to Determine which Link in the Bundle to Use The Inputs to the Algorithm Are a Combination of L2, L3 and / or L4 addresses
dst-ip
dst-mac
dst-mixed-ip-port
dst-port
mpls
src-dst-ip
src-dst-mac
src-dst-mixed-ip-port
src-dst-port
src-ip
src-mac
src-mixed-ip-port
src-port
vlan-dst-ip
vlan-dst-mixed-ip-port
vlan-src-dst-ip
vlan-src-dst-mixed-ip-port
vlan-src-ip
vlan-src-mixed-ip-port
Dst IP Addr
Dst Mac Addr
Dst IP Addr and TCP/UDP Port
Dst TCP/UDP Port
Load Balancing for MPLS packets
Src XOR Dst IP Addr
Src XOR Dst Mac Addr
Src XOR Dst IP Addr and TCP/UDP Port
Src XOR Dst TCP/UDP Port
Src IP Addr
Src Mac Addr
Src IP Addr and TCP/UDP Port
Src TCP/UDP Port
Vlan, Dst IP Addr
Vlan, Dst IP Addr and TCP/UDP Port
Vlan, Src XOR Dst IP Addr
Vlan, Src XOR Dst IP Addr and TCP/UDP Port
Vlan, Src IP Addr
Vlan Src IP Addr and TCP/UDP Port
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
115
Agenda
 Background
 C6807-XL
 C6880-X
 C6840-X
 L2 Forwarding
 L3 Forwarding
 Packet Walks
Interface Management
For Your
Information
VLANs & Interfaces
Supervisor 720
4K VLAN POOL
VLANs
L3 Ports
SVIs
Tunnels
CoPP
Etc…
Supervisor 2T
16K Bridge
Domains
128K Logical
Interfaces
VLAN 1 - 4K
L3 Ports
VLAN 1 - 4K
SVIs
Tunnels
VLAN 1 - 4K
CoPP
Etc…
• VLANs used for both L2 Bridging
and L3 Routing
• Separate L2 Bridging and L3 Routing
• Each L3 Interfaces consumes
an internal VLANs from total 4K
VLAN pool
• Allows VLAN reuse on Per Port basis
• Breaks the 4K VLAN barrier
• Massive scale for L3 interfaces
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
117
L3 Forwarding
For Your
Information
High-Level SW & HW Process
Control Plane (RP)
Routing Protocols
1
OSPF, EIGRP, RIP, BGP, etc
RP receives routing updates
from attached neighbors
2
FIB (on PFC/DFC)
FIB & ADJ tables are used
by EARL to perform L3
lookups & forwarding
Stores routing details,
from Static Routes &
Routing Protocols in
Routing Information Base
(RIB)
Software CEF 3
Takes RIB and builds a
Forwarding Information
Base (FIB) containing
IP/mask prefixes
Hardware CEF
5
Loads FIB into PFC
& distributes to DFC’s
4
Hardware-based CEF Process
1. FIB lookup based on Destination prefix (longest-match)
2. FIB “Hit” returns an Adjacency pointer
3. Adjacency contains Rewrite (next-hop) information
4. ACL, QoS & NetFlow lookups occur IN PARALLEL (may effect final result)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
118
L3 Forwarding
FIB & Adjacency Overview
Each PFC/DFC stores a copy of the “FIB” & “Adjacency Table”…
172.20.45.1
FIB contains:
10.1.1.100
CEF entries from MOST to LEAST specific
based on /Mask
FIB TCAM shared by:
– IPv4 Unicast
– IPv4 Multicast
– IPv6 Unicast
– IPv6 Multicast
– MPLS
IF1, MAC, MTU
IF2, MAC, MTU
MASK (/32)
…
IF3, MAC, MTU
10.1.3.0
IF4, MAC, MTU
10.1.2.0
MASK (/24)
…
10.1.0.0
172.16.0.0
…
…
Adjacency
Table
MASK (/16)
…
Adjacency Table:
– L2 “Re-Write” information and / or pointers for replication
– Hardware Adjacency table also shared among protocols
0.0.0.0
MASK (/0)
FIB TCAM
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
119
L3 Forwarding
FIB & ADJ Lookup in PFC/DFC
Lets assume a lookup needs to be performed for a packet with
a destination of 10.1.5.2 /24, then the following would occur…
1
172.20.45.1
Packet
10.1.1.100
MASK (/32)
2
IF1, MAC, MTU
…
Key Gen
IF2, MAC, MTU
10.1.3.0
3
10.1.2.0
MASK (/24)
Lookup Key
…
HIT!
Load-Sharing
Hash
7
IF3, MAC, MTU
IF4, MAC, MTU
4
10.1.0.0
172.16.0.0
6
…
5
MASK (/16)
…
…
Adjacency
Table
0.0.0.0
MASK (/0)
FIB TCAM
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
120
PFC/DFC FIB TCAM
For Your
Information
Defaults and Changes
IPv4 Unicast & MPLS require only 1 entry
IPv6 Unicast & IPv4 Multicast require 2 entries
NON-XL
PFC
XL
PFC
Standard PFC/DFC = 256K entries
IPv4, MPLS
192k
512k
The “XL” PFC/DFC = 1M entries
IPv6, Multicast
32k
256k
Default TCAM allocation shown below
SUP2T-XL Example
Changing default (requires Reboot!)
SUP2T#sh platform cef maximum-routes
FIB TCAM maximum routes :
=======================
Current :------IPv4 + MPLS
- 512k (default)
IPv6 + IP Multicast - 256k (default)
SUP2T(config)#platform cef maximum-routes ?
ip
number of ip routes
ip-multicast number of multicast routes
ipv6
number of ipv6 routes
mpls
number of MPLS labels
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
121
Displaying IPv4 Forwarding Summary
SUP2T#show platform hardware capacity forwarding
...
L3 Forwarding Resources
FIB TCAM usage:
Total
72 bits (IPv4, MPLS, EoM)
196608
144 bits (IP mcast, IPv6)
32768
detail:
Protocol
IPv4
MPLS
EoM
IPv6
IPv4 mcast
IPv6 mcast
Adjacency usage:
Total
1048576
BRKARC-3465
For Your
Information
Used
28
7
%Used
1%
1%
Used
28
0
0
%Used
1%
0%
0%
1
3
3
1%
1%
1%
Used
171
%Used
1%
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
122
Displaying Hardware IPv4 Prefix Entries
For Your
Information
SUP2T#show platform hardware cef
Codes: decap - Decapsulation, + - Push Label
Index Prefix
Adjacency
68
255.255.255.255/32 receive
75
10.10.1.1/32
receive
76
77
78
3200
3201
3202
10.10.1.0/32
10.10.1.255/32
10.10.1.2/32
224.0.0.0/24
10.10.1.0/24
10.100.0.0/24
receive
receive
Gi1/1,
receive
glean
Gi1/1,
3203
3204
3205
10.100.1.0/24
10.100.2.0/24
10.100.3.0/24
Gi1/1,
Gi1/1,
Gi1/1,
0030.f272.31fe
0030.f272.31fe
0030.f272.31fe
0030.f272.31fe
0030.f272.31fe
...
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
123
Finding the Longest-Match Prefix Entry
For Your
Information
SUP2T#show platform hardware cef 171.1.1.0
Codes: decap - Decapsulation, + - Push Label
Index
Prefix
Adjacency
SUP2T#show platform hardware cef lookup 171.1.1.0
Codes: decap - Decapsulation, + - Push Label
Index
Prefix
3531584 171.0.0.0/8
Adjacency
Vl192
,00d0.0053.bc00
SUP2T#show platform hardware cef ipv6 lookup FF00::
Codes: + - Push label
Index
Prefix
512 FF00::/8
Adjacency
glean
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
124
Agenda
 Background
 C6807-XL
 C6880-X
 C6840-X
 L2 Forwarding
 L3 Forwarding
 NetFlow
 Access Control
 Packet Walks
Catalyst Hardware NetFlow
Cisco NetFlow is a process designed to collect information about traffic “flows” that pass through a switch
Netflow
Collection
Server
Data Flow (PFC)
Exported Netflow
Record (MSFC)
NetFlow
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
126
Catalyst Hardware NetFlow
For Your
Information
NetFlow Flow Masks
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
127
Hardware NetFlow
For Your
Information
Supervisor 2T Enhancements
Optimal CPU Utilization with
Yielding NDE & Direct Export
from a Line Card
Flexible
NetFlow
CPU Friendly
Export
Allow to use Netflow after
ingress lookup is done
(NetFlow on CoPP)
Allow to account for
IP Multicast traffic per
destination instead of per group
Increased customization and scale by
selecting the fields to Match and
Collect for both IPv4 and IPv6
Egress
Netflow
Up to 13M
Flows per
System
Sampled
Netflow in
Hardware
BRKARC-3465
Bigger tables mean
MORE entries per
system, giving you
better visibility into
your network (up to
13 million NF entries
with a 13 slot chassis)
Optimize the Netflow
Tables utilization and
minimize load on Analyzers
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
128
Hardware NetFlow
TCAM Lookup on PFC4
1
DST IP
SRC IP
Proto
DST Port
SRC Port
10.1.2.11
10.1.1.10
0x6
80
33992
2
Flow Key
Key
Flow
3
Hash Function Compare
all pages
5
Lookup Key
4
DataKey
Key Index
Index
Data
Data
Key Index
Index
Data
Key
DataKey
Key Index
Index
Data
Data
Key Index
Index
Data
Key
Data
Key
Index
DataKey
Key Index
Index
Data
Data
Key
Index
DataKey
Key Index
Index
Data
Data
Key Index
Index
Data
Key
DataKey
Key Index
Index
Data
Data
Key Index
Index
Data
Key
Data
Key
Index
DataKey
Key Index
Index
Data
Data
Key
Index
DataKey
Key Index
Index
HIT!
Data
Data
Key Index
Index
Data
Key
DataKey
Key Index
Index
Data
Data
Key Index
Index
Data
Key
DataKey
Key Index
Index
Data
Data
Key Index
Index
Data
Key
Data
Key
Index
DataKey
Key Index
Index
Data
Data
Key
Index
DataKey
Key Index
Index
Data
Data
Key Index
Index
Data
Key
Compare
Flow Data
7
6
Index to
NF Data
Table
Flow Data
Flow Data
Flow Data
Flow Data
Flow Data
Flow Data
Flow
HIT! Data
Flow Data
Flow Data
Flow Data
Flow Data
8
Update
Stats
Statistics
Statistics
Statistics
Statistics
Statistics
Statistics
Statistics
Statistics
Statistics
Statistics
Statistics
Data Key
Key
Data
512K
entries
Indexes row in Lookup Table
NetFlow Data Table
NetFlow Statistics
NetFlow Lookup Table
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
129
Hardware NetFlow
Sup2T Flexible NetFlow
For Your
Information
Flow Export
Flow Record
Key Field
Non-Key Field
Key Field
Non-Key Field
…
…
Multiple Exporters
can be associated
with a single FNF
monitor
Export Profile
Export Profile
…
Key Fields trigger the creation of a new Flow entry
every time their value change
Flow Profile
Non-Key Fields are data that is indexed by the Key Fields.
Flow
Monitor
Key Fields are defined using the “match” statement
Non-Key-Fields are defined using the “collect” statement
Ingress
and / or
Ingress
….
and / or
Egress
Egress
Interfaces
BRKARC-3465
Same Flow Monitor
can be associated with
multiple Interfaces
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
130
For Your
Information
PFC4 Key & Non-Key Fields
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
131
Hardware NetFlow
NetFlow Data Export Process
VS-S2T-10G
Netflow
Export
Netflow
Data
Netflow Collector
WS-X6848-TX-2T
EOBC
Netflow
Data
WS-X6908-10G-2T
Netflow Direct
Export
Data
Direct Export with Sup2T:
•
•
•
•
•
•
•
C6800-32P10G
C6800-16P10G
C6800-8P10G
WS-X6904-40G
WS-X6908-10G
WS-X6816-10X
WS-X6716-10X (DFC4-E)
Also supported on C6880-X:
• C6880-X-16P10G
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
132
Hardware NetFlow
For Your
Information
Sup2T “CPU Friendly” Netflow Export
CPU
Utilization
NDE increases
export rate until
threshold reached
When threshold reached,
NDE quickly backs off
20% export rate
70%
Yielding NDE
CPU threshold
30%
CPU before
NDE begins
Wait 5 seconds and then
step up export rate again
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
133
Displaying NetFlow Utilization
For Your
Information
SUP2T#show platform hardware capacity netflow
Netflow resources:
Netflow table size: 515032 entries total
Netflow table usage: Module/Instance
Input flows
Output flows
3
10%
10%
7
25%
25%
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
134
For Your
Information
PFC3 vs. PFC4 - Flexible Netflow
Feature
PFC3
PFC4
256 K (Ingress Only)
512 K Ingress – 512 K Egress
Shared Netflow Policers
N/A
512
Netflow Samplers
N/A
1K
Class Maps per Policy Map
1K
4K
Aggregate Policers
1K
6K
Distributed Policers
N/A
4K
Microflow Policers
63
128
Flexible Netflow
N/A
Yes
Egress Netflow
N/A
Yes
VRF-aware Netflow
N/A
Yes*
Netflow Export Enhancements
Direct Export
Direct Export,
CPU Yield, EEM
Packet or Byte Policing
Byte-based
Packet or Byte-based
Netflow Entries
BRKARC-3465
* Available in future IOS software
releases
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
135
Agenda
 Background
 C6807-XL
 C6880-X
 C6840-X
 L2 Forwarding
 L3 Forwarding
 NetFlow
 Access Control
 Packet Walks
Access Control Lists
Hardware Support
Full Hardware Support
DFC
Create an ACL or classification
policy, using the CLI or Network
Management System (NMS)
DFC
PFC
PFC
DFC
1
ip access-list extended Internet
permit ip any host 10.2.2.4
permit ip any host 10.5.2.33
permit ip any host 10.11.0.0
permit ip any host 10.4.0.0
Policy Feature Card
Distributed Forwarding Card
•
•
•
•
Router ACLs
VLAN ACLs
Port Based ACLs
Role Based ACLs
2
DFC
DFC
Hardware- Assisted
ACL Features
•
•
•
•
•
•
BRKARC-3465
3
NetFlow
NAT & PAT
PBR
WCCP
Reflexive ACLs
Cisco Trust Sec
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
137
Access Control Lists
For Your
Information
Three Forms of Security ACLs
The PFC / DFC supports three forms of Security ACLs: the RACL, VACL and PACL…
Router ACL (RACL)
VLAN ACL (VACL)
Port ACL (PACL)
Used to permit or deny the
movement of traffic
between Layer 3 Subnets
Used to permit or deny the
movement of traffic
between Layer 3 Subnets
& VLANs or within a VLAN
Used to permit or deny the
movement of traffic
between Layer 3 Subnets
& VLANs or within a VLAN
Applied as an input or output
policy to a Layer 3 interface
Applied as a policy to a
VLAN - is inherently applied
to both inbound and
outbound traffic
Applied as a policy to a
Layer 2 Switch port interface
- is applied for inbound traffic
only
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
138
Access Control Lists
ACL Order of Processing
3
4
Output RACL
Input RACL
2
VACL
1
VACL
5
Input PACL
Note: NO
Output PACL
exists
Destination
Source
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
139
Access Control Lists
PFC4 TCAM Lookup
TCAM A
BANK 0
BANK 1
PFC4 / DFC4
Forwarding Engine
TCAM B
BANK 2
BANK 3
VACL
QoS
RACL
SGACL
RACL
PACL
ACE
Counters
(L2 ASIC)
3
TCAM Controller
2
2X
Lookup Keys
Packet Header Information
1
ACL
Labels
4X
Results
ACL
LOUs
Classification Module 1
4
7
4 X Results
& Priority
6
5
Classification Module 2
BRKARC-3465
Final Result
to Netflow
8
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
140
Access Control Lists
For Your
Information
Mask Utilization: PFC3 vs PFC4
permit ip 10.1.1.0 0.0.0.255 any
permit ip 10.2.1.0 0.0.0.255 any
permit ip 10.3.0.0 0.0.255.255 any
PFC3 ACL TCAM
MASK
0.0.0.255
MASK
0.0.255.255
PFC4 ACL TCAM
10.1.1.0
permit
Mask 0.0.0.255
10.1.1.0
permit
10.2.1.0
10.3.0.0
-
permit
permit
-
Mask 0.0.0.255
10.2.1.0
permit
Mask 0.0.255.255
-
10.3.0.0
-
permit
-
3 ACEs
used
16 ACEs
used
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
141
Access Control Lists
For Your
Information
PFC4 “ACL Dry Run” Feature
•
Make sure an ACL can fit in the
TCAM before you apply the ACL
- ACLs that do not fit can cause
software forwarding and possible
high CPU utilization
• Special configuration session
- Create and Edit the ACL
- Verifies if the changes will fit within
the hardware resources
•
The actual changes are not
programmed into the hardware
during the dry-run session
•
Configuration changes can be
verified step by step…
SUP2T-E# show configuration session test status
====================================
Status of last config validation:
Timestamp: 2013-09-20@17:27:06
======================================
SLOT = [1]
Result = Configuration will fit in TCAM
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
142
Access Control Lists
For Your
Information
PFC4 “ACL Hitless Update” Feature
•
Allows updates to an ACL
interrupting traffic
•
Multiple features updated at once
•
IPv4, IPv6, MAC…
•
RACL, VACL, PBR…
IPv4
without
IPv6
MAC
ACL
Updates
•
Global configuration option
•
Feature does consume double the number of
TCAM entries
(default is on)
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
143
Access Control Lists
For Your
Information
PFC4 ACL Hitless Update
 Each ACL feature is initially
programmed into 2 different
spaces into the TCAM
Primary Space (Label-1)
TCAM B
TCAM A
BANK 0
BANK 1
 Once the ACL changes have been
completed the then PFC4 will then
use the original label again
BANK 3
SGT-1
RACL-1
SGT-2
RACL-2
VACL-1
QoS-1
VACL-2
QoS-2
Shadow Space (Label-2)
 While an ACL is being updated
the PFC4 will use a temporary
label that points to the shadow
TCAM space
BANK 2
TCAM Controller
2 X Lookup Keys
ACL
Labels
1, 2
4 X Results & Priority
ACL LOUs
Classification Module 1
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
144
For Your
Information
PFC3 vs. PFC4 - Security
Feature
PFC3
PFC4
32K
64K or 256K
(Programmable)
ACL Labels
4K
16K
ACE Mask Ratio
8:1
1:1
ACL LOU’s
64
208
Port ACL’s
2K
8K
Software
Hardware
Per-Port Per-VLAN ACL’s
N/A
Yes
Security Group ACL’s
N/A
32K
Security Group Tagging (SGT)
CTS 1.5 (SXP)
CTS 2.0
802.1ae Encryption Support
N/A
Yes (Line-Rate)
IPv4 (2 paths)
IPv4 & IPv6 (16 paths)
Layer3: 8
Layer2: 4
Layer3: 31
Layer2: 26
ACL TCAM Size
MAC ACL Support
Unicast RPF
Hardware Rate Limiters
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
145
Agenda
 Background
 C6807-XL
 C6880-X
 C6840-X
 L2 Forwarding
 L3 Forwarding
 Packet Walks





6700/6800 & 6900
C6800 1G Series
C6800 10G Series
C6880-X Series
C6840-X Series
For Your
Information
6700 to 6700 - Centralized Forwarding
Slot 1
1
Port ASIC A
2
Port ASIC B
R
6
Port ASIC A
CFC
FIRE ASIC AH
P
Slot 2
Port ASIC B
CFC
FIRE ASIC B
FIRE ASIC A
7
FIRE ASIC B
3
Dbus
Rbus
Michael
Engineering
Switch Fabric
Amanda
Marketing
5
R
H
H
Fabric / Bus
Interface &
Replication ASIC
4
5
R
Layer 2 Engine
PFC4
P = Packet
H = Header
Layer 3 Engine
R = Result
Supervisor Engine 2T
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
147
1
Port ASIC A
6700 (CFC)
15
Slot 2
Port ASIC B
14
Local Forwarding
CFC
4 10
FIRE ASIC A 3
2
11
10
CPU (MSFC)
10
FIRE ASIC B
13
5
E-DBUS
10
E-RBUS
12
Switch Fabric
Port ASIC
5
6
10
6
5
9
Fabric / Bus
Interface &
Replication ASIC
Layer 2 Engine 7 P
F
Layer 3 Engine 8 C
Supervisor Engine
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
148
6800 to 6800 - Distributed Forwarding (DFC)
Slot 1
1
Port ASIC A
2
P
Slot 2
Port ASIC B
3
DFC4
R
FIRE ASIC A H
L
2
For Your
Information
Port ASIC A
DFC4
L
3
Port ASIC B
L
2
L
3
6
4
FIRE ASIC B
FIRE ASIC A
FIRE ASIC B
5
Dbus
Rbus
Michael
Engineering
Switch Fabric
Amanda
Marketing
PFC4
Fabric / Bus
Interface &
Replication ASIC
Layer 2 Engine
P = Packet
H = Header
Layer 3 Engine
R = Result
Supervisor Engine 2T
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
149
1
Slot 2
Port ASIC A
6800 (DFC)
13
Port ASIC B
2
Local Forwarding
12
5
3
4
FIRE ASIC A
7
DFC
6
10
11
FIRE ASIC B
9
E-DBUS
E-RBUS
Switch Fabric
Port ASIC
CPU (MSFC)
8
Fabric / Bus
Interface &
Replication ASIC
Layer 2 Engine
Layer 3 Engine
Supervisor Engine
BRKARC-3465
P
F
C
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
150
For Your
Information
6900 to 6900 - Distributed Forwarding (DFC)
1a
1b
CTS ASIC A
Slot1
CTS ASIC B
CTS ASIC A
Port ASIC A
3
Port ASIC B
Port ASIC A
DFC4
2
P
R
FIRE ASIC A H
L
2
Slot2
Port ASIC B
DFC4
L
3
CTS ASIC B
L
2
6b
6a
L
3
4
FIRE ASIC B
FIRE ASIC A
FIRE ASIC B
5
Dbus
Rbus
Michael
Engineering
Switch Fabric
Amanda
Marketing
PFC4
Fabric / Bus
Interface &
Replication ASIC
Layer 2 Engine
P = Packet
H = Header
Layer 3 Engine
R = Result
Supervisor Engine 2T
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
151
1
6900 (DFC)
CTS
ASIC A
CTS
ASIC B
Local Forwarding
Port
ASIC A
Port
ASIC B
2
Slot 2
CTS
ASIC C
Port
ASIC C
DFC 5
3
4
FIRE ASIC A
7
6
10
14
CTS
ASIC D
13
Port
ASIC D
12
11
FIRE ASIC B
9
E-DBUS
E-RBUS
Switch Fabric
Port ASIC
CPU (MSFC)
8
Fabric / Bus
Interface &
Replication ASIC
Supervisor Engine
Layer 2 Engine
Layer 3 Engine
BRKARC-3465
P
F
C
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
152
For Your
Information
Multicast Ingress Replication Model
Egress Linecards
Host
Ingress Linecard
RE
Host
Host
Switch
Fabric
RE
1
3
Host
4
RE
2
Host
Host
Host
RE
Host
Host
Host
Ingress Replication Engine (RE) responsible
for replication to ALL OIF’s
Host
Host
Replicates for each egress linecard
Switch Fabric sends copies for each OIF
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
153
For Your
Information
Multicast Egress Replication Model
Egress Linecards
Host
Ingress Linecard
RE
Host
Host
Switch
Fabric
RE
1
3
2
Host
4
RE
5
Host
Host
Host
RE
Host
Host
Host
Ingress Replication Engine responsible
for replication to local OIF’s
Replicates a SINGLE copy to Fabric
Host
Host
Switch Fabric replicates for each OIF
Local Linecard RE replicates for local OIF’s
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
154
Agenda
 Background
 C6807-XL
 C6880-X
 C6840-X
 L2 Forwarding
 L3 Forwarding
 Packet Walks





6700/6800 & 6900
C6800 1G Series
C6800 10G Series
C6880-X Series
C6840-X Series
C6800 1G - Packet Walks
Remote Forwarding (Ingress)
Supervisor
Switch Fabric
20G
EOBC
Step 4: FIRE ASIC stores data
payload in local buffer, and then it
sends only the Internal Header to
Forwarding Engine for Lookup
20G
BACKPLANE INTF
FABRIC
INTF
20G
FPGA
20G
FPGA
FIRE
ASIC
10G
10G
PORT
ASIC
PORT
ASIC
10G
10G
DFC4
Step 5: Inband FPGA parses
Internal Header, and then it sends
to Forwarding Engine
PORT
ASIC
PORT
ASIC
Step 2: PHY converts the signal
& serializes the bits, and then it
sends to Port ASIC
Step 1: Packet Arrives
@ Ingress Port 13
Step 8: Ingress FIRE ASIC uses
lookup result to determine
the Fabric Port mapped to Egress
Port, and converts Internal Header
to Fabric Header.
Then it sends to Fabric ASIC
FABRIC
INTF
FIRE
ASIC
Step 3: Port ASIC parses packet
to derive VLAN, CoS, etc. and
perform Ingress QoS.
Then it applies Internal Header
and sends to FIRE ASIC
Step 9: Fabric ASIC uses
Fabric Header to determine
Egress Fabric Port and then it
sends to Switch Fabric
Step 6-7: Forwarding Engine
performs L2, L3, ACL and
Netflow IFE & OFE processing
to determine the Egress Port
&
Rewrite Info.
Then it returns result to FIRE ASIC
(via Inband FPGA)
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
01-04
05-08
09-12
13-16
17-20
21-24
25-28
29-32
33-36
37-40
41-44
45-48
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
156
C6800 1G Packet Walks
Remote Forwarding (Egress)
Step 10: Switch Fabric transmits
frame to Egress Fabric ASIC, which
sends to Egress FIRE ASIC
Supervisor
Switch Fabric
20G
EOBC
Step 11: FIRE ASIC uses Fabric
Header to derive new Internal header,
which it sends to Forwarding Engine
(Egress Lookup)
BACKPLANE INTF
FABRIC
INTF
20G
FPGA
FABRIC
INTF
FPGA
FIRE
ASIC
Step 12: Inband FPGA parses
Internal Header, and then it sends
to Forwarding Engine
Step 13: Forwarding Engine
performs an Egress (L2) Lookup
to learn SRC MAC address.
Then it returns Internal Header to
FIRE ASIC (via Inband FPGA)
20G
Step 14: FIRE ASIC uses
Internal Header to determine Egress
20G
Port and reassemble the packet.
Then it sends to Port ASIC
FIRE
ASIC
10G
10G
PORT
ASIC
PORT
ASIC
10G
10G
DFC4
PORT
ASIC
Step 15: Port ASIC removes
Internal Header and rewrites VLAN,
CoS, etc. and perform Egress QoS.
Then it sends to PHY
PORT
ASIC
Step 16: PHY serializes the bits
& converts signal, and then
transmits the packet
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
01-04
05-08
09-12
13-16
17-20
21-24
25-28
29-32
33-36
37-40
41-44
45-48
BRKARC-3465
Step 17: Packet Leaves
@ Egress Port 36
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
157
Agenda
 Background
 C6807-XL
 C6880-X
 C6840-X
 L2 Forwarding
 L3 Forwarding
 Packet Walks





6700/6800 & 6900
C6800 1G Series
C6800 10G Series
C6880-X Series
C6840-X Series
C6800 10G - Packet Walks
Remote Forwarding (Ingress)
Fabric ASIC of Active Sup
Fabric ASIC of Standby Sup
40G
40G
40G
40G
40G
40G
40G
Step 9: Fabric ASIC uses
Fabric Header to determine
Egress Fabric Port and then it
sends to Switch Fabric
40G
Step 8: Ingress FIRE ASIC uses
lookup result to determine
the Fabric Port mapped to Egress
Port, and converts Internal Header
to Fabric Header.
Then it sends to Fabric ASIC
Local Fabric ASIC
Step 4: FIRE ASIC stores data
payload in local buffer, and then it
sends only the Internal Header to
Forwarding Engine for Lookup
2x20G
2x20G
Inband
Inband
DFC4
Inband
Step 3: Port ASIC Decrypts
CTS, then it parses packet to
derive VLAN, CoS, etc. and
perform Ingress QoS.
Then it applies Internal Header
and sends to FIRE ASIC
Inband
FIRE
ASIC
FIRE
ASIC
FIRE
ASIC
FIRE
ASIC
40G
40G
Step 2: PHY converts the signal
& serializes the bits, and then it
sends to Port ASIC
Step 1: Packet Arrives
@ Ingress Port 17
Step 5: Inband FPGA parses
Internal Header, and then it sends
to Forwarding Engine
DFC4
40G
40G
Port ASIC
Port ASIC
Port ASIC
Port ASIC
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
SFP
01,03,05,07
SFP
09,11,13,15
SFP
17,19,21,23
SFP
25,27,29,31
SFP
02,04,06,08
SFP
10,12,14,16
SFP
18,20,22,24
SFP
26,28,30,32
Front
Panel
BRKARC-3465
Step 6-7: Forwarding Engine
performs L2, L3, ACL and
Netflow IFE & OFE processing
to determine the Egress Port
&
Rewrite Info.
Then it returns result to FIRE ASIC
(via Inband FPGA)
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
159
C6800 10G Packet Walks
Remote Forwarding (Egress)
Step 10: Switch Fabric transmits
frame to Egress Fabric ASIC, which
sends to Egress FIRE ASIC
Fabric ASIC of Active Sup
Fabric ASIC of Standby Sup
40G
40G
40G
40G
40G
40G
40G
Local Fabric ASIC
Step 11: FIRE ASIC uses Fabric
Header to derive new Internal header,
which it sends to Forwarding Engine
(Egress Lookup)
2x20G
2x20G
Inband
Step 14: FIRE ASIC uses
Internal Header to determine
Egress Port, reassemble the packet
and perform Egress QoS.
Then it sends to Port ASIC
Inband
DFC4
Inband
Inband
DFC4
Step 12: Inband FPGA parses
Internal Header, and then it sends
to Forwarding Engine
Step 13: Forwarding Engine
performs an Egress (L2) Lookup
to learn SRC MAC address.
Then it returns Internal Header to
FIRE ASIC (via Inband FPGA)
40G
FIRE
ASIC
FIRE
ASIC
FIRE
ASIC
FIRE
ASIC
Step 15: Port ASIC removes Internal
Header, rewrites VLAN, CoS, etc. and
adds Encryption. Then it sends to PHY
40G
40G
40G
40G
Port ASIC
Port ASIC
Port ASIC
Port ASIC
PHY
PHY
PHY
PHY
PHY
PHY
PHY
PHY
SFP
01,03,05,07
SFP
09,11,13,15
SFP
17,19,21,23
SFP
25,27,29,31
SFP
02,04,06,08
SFP
10,12,14,16
SFP
18,20,22,24
SFP
26,28,30,32
Front
Panel
BRKARC-3465
Step 16: PHY serializes the bits
& converts signal, and then
transmits the packet
Step 17: Packet Leaves
@ Egress Port 24
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
160
Agenda
 Background
 C6807-XL
 C6880-X
 C6840-X
 L2 Forwarding
 L3 Forwarding
 Packet Walks





6700/6800 & 6900
C6800-1G Series
C6800-10G Series
C6880-X Series
C6840-X Series
C6880-X & C6840-X Packet Walk
Remote Forwarding (Egress)
Step 5: FIRE ASIC stores data
payload in local buffer, and then it
sends only the Internal Header to
Forwarding Engine for Lookup
Step 9: Fabric ASIC uses
Fabric Header to determine
Egress Fabric Port and then it
sends to Switch Fabric
Step 4: MUX ASIC stores
packet in local buffer, and
may perform special packet
encap. Then it sends to
(1 of 2) FIRE ASIC
Step 8: Ingress FIRE ASIC uses
new Internal Header to determine
the Fabric Port mapped to
Egress Port, and converts Internal
Header to Fabric Header.
Then it sends to Fabric ASIC
Step 6: Inband FPGA parses
Internal Header, and then it sends
to Forwarding Engine
Step 3: Port ASIC removes
Decryption, then it parses packet
to derive VLAN, CoS, etc. and
perform Ingress QoS.
Then it applies Internal Header
and sends to MUX ASIC
Step 7: Forwarding Engine
performs L2, L3, ACL and Netflow
IFE & OFE processing and
determines the Egress Port &
Rewrite Info. Then it returns new
Internal Header to FIRE ASIC
(via Inband FPGA)
Step 2: PHY converts the signal
& serializes the bits, and then it
sends to Port ASIC
Step 1: Packet Arrives
@ Ingress Port 1
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
162
C6880-X & C6840-X Packet Walk
Remote Forwarding (Egress)
Step 10: Fabric ASIC transmits frame
to Egress Fabric Port, which is
received by Egress FIRE ASIC
Step 11: FIRE ASIC uses Fabric
Header to derive new Internal header,
which it sends to Forwarding Engine
(egress lookup)
Step 12: Inband FPGA parses
Internal Header, and then it sends
to Forwarding Engine
Step 14: FIRE ASIC uses new Internal
Header to determine Egress Port and
reassemble the packet, and then it
sends to MUX ASIC
Step 15: MUX ASIC uses Internal
Header to determine Egress Port
and perform Egress QoS.
Then it sends to Port ASIC
Step 13: Forwarding Engine
performs an egress (L2) lookup to
learn MAC address.
Then it returns Internal Header to FIRE
ASIC (via Inband FPGA)
Step 16: Port ASIC removes Internal
Header, rewrites VLAN, CoS, etc. and
adds Encryption. Then it sends to PHY
Step 17: PHY serializes the bits
& converts signal, and then
transmits the packet
Step 18: Packet Leaves
@ Egress Port 16
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
163
Agenda
 Background
 C6807-XL
 C6880-X
 C6840-X
 L2 Forwarding
 L3 Forwarding
 Packet Walks





6700/6800 & 6900
C6800-1G Series
C6800-10G Series
C6880-X Series
C6840-X Series
C6840-X - Packet Walks
Same as C6800 10G Series
C6800-32P10G
C6832-X-LE
Fabric ASIC of Active
Sup
40G
40G
40G
Local Fabric ASIC
2x20G
DFC
4
Inband
FIRE
ASIC
40G
40G
Port ASIC
Port ASIC
PHY
PHY
PHY
PHY
SFP
SFP
SFP
SFP
01-04
05-08
09-12
13-16
40G
40G
2x20G
Inband
Inband
Inband
Inband
DFC
4
Inband
Inband
DFC
4
FIRE
ASIC
FIRE
ASIC
Front
Panel
40G
2x20G
DFC
4
FIRE
ASIC
40G
Local Fabric ASIC
2x20G
Inband
Fabric ASIC of Standby Sup
40G
FIRE
ASIC
FIRE
ASIC
40G
40G
Port ASIC
Port ASIC
Port ASIC
PHY
PHY
PHY
PHY
PHY
SFP
SFP
SFP
21-24
25-28
29-32
40G
40G
Port ASIC
PHY
SFP
17-20
SFP
01,03,05,07
PHY
SFP
09,11,13,15
SFP
17,19,21,23
BRKARC-3465
FIRE
ASIC
PHY
SFP
25,27,29,31
Front
Panel
FIRE
ASIC
40G
40G
Port ASIC
Port ASIC
PHY
PHY
PHY
PHY
SFP
02,04,06,08
SFP
10,12,14,16
SFP
18,20,22,24
SFP
26,28,30,32
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
165
Summary
Innovation & Investment Protection
The Catalyst 6800 Series are based on the “Gold Standard”
Catalyst 6500 Series, leveraging the same ASICs and IOS
Software, while providing a foundation for the next-generation.
Hardware Multi-Layer Switching
L2 and L3 forwarding, network policies and statistics collection are
performed by the ASIC hardware, so there is minimal difference in
scale and performance.
Combined Features & Performance
Providing advanced features, in hardware, such as L2, IPv4, IPv6,
MPLS, NetFlow, QoS and Security, etc. without impacting overall
scale or performance.
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
166
Complete Your Online Session Evaluation
• Give us your session feedback
to be entered into a Daily Survey
Drawing.
• One daily winner will receive a
$750 Amazon gift card.
• Complete your session surveys
through the Cisco Live mobile
app or the Session Catalog on
CiscoLive.com/us.
Don’t Forget: Cisco Live sessions are available
for viewing on-demand after the event at
CiscoLive.com/Online
BRKARC-3465
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
167
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Lunch & Learn
• Meet the Engineer 1:1 meetings
• Related sessions
Presentation ID
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
168