Contents
Introduction
Background Information
Problem
Solution
Related Cisco Support Community Discussions
Introduction
This document describes how to adjust the routing Ternary Content Addressable Memory (TCAM)
allocations on 3BXL modules for the Cisco Catalyst 6500 and 7600 Series routers and switches.
Background Information
On the Catalyst 6500 and 7600 Series platforms, all of the routing information is stored in special
high-speed memory called TCAM. More specifically, the Catalyst 6500 and 7600 Series platforms
have three different types of TCAM:
●
●
●
Forwarding Information Base (FIB), or routing TCAM
Access Control List (ACL) TCAM
Netflow TCAM
When a route is programmed into the Cisco Express Forwarding (CEF) table in the main memory
(RAM), a second copy of that route is stored in the hardware TCAM memory on the Supervisor as
well as any Distributed Forwarding Card (DFC) modules on the linecards.
Problem
This document focuses on the FIB TCAM; however, the information in this document can also be
used in order to resolve these error messages:
%MLSCEF-SP-4-FIB_EXCEPTION_THRESHOLD: Hardware CEF entry
usage is at 95% capacity for IPv4 unicast protocol
%MLSCEF-DFC4-7-FIB_EXCEPTION: FIB TCAM exception, Some
entries will be software switched
%MLSCEF-SP-7-FIB_EXCEPTION: FIB TCAM exception, Some
entries will be software switched
It is important to be aware that the aforementioned errors are reported only once when the TCAM
limit is exceeded. Even if the total number of routes withdraws below the configured limit, the
switch remains in the exception state until it is cleared:
7600#show mls cef exception status
Current IPv4 FIB exception state = TRUE
Current IPv6 FIB exception state = FALSE
Current MPLS FIB exception state = FALSE
As a result of this TCAM exception condition, connectivity is affected and might result in elevated
CPU usage due to software switching.
On August 8th, 2014, the Classless Inter-Domain Routing (CIDR) Report, which provides statistics
on the global Internet routing table, reported that the global Internet routing table had passed
512,000 routes.
Most platforms have more than enough TCAM space to support larger routing tables, but the
default configurations might require adjustment. As the Internet routing table approaches 512,000
routes, it can cause the Catalyst 6500 and 7600 3BXL-based modules to exceed the default
routing TCAM allocations.
Note: The VS-S2T-10G-XL supervisor engine and DFC4XL modules support 1,000,000
routes that are dynamically shared between IPv4 and IPv6 by default.
This table shows the Supervisor modules and DFCs that are affected by the continued growth of
the IPv4 Internet routing table:
Product Name
WS-SUP720-3BXL
WS-F6700-DFC3BXL
VS-S720-10G-3CXL
RSP720-3CXL-GE
WS-F6700-DFC3CXL
Default IPv4 TCAM Space
512,000
512,000
512,000
512,000
512,000
Maximum IPv4 TCAM Space
1,000,000
1,000,000
1,000,000
1,000,000
1,000,000
On the 3BXL Series modules, the default FIB TCAM space for the IPv4 routes is 512k routing
entries. You can enter the show mls cef maximum-routes command in order to view this
information:
7600#show mls cef max
FIB TCAM maximum routes :
=======================
Current :------IPv4 + MPLS
- 512k (default)
IPv6 + IP Multicast - 256k (default)
Note: Each IPv6 route consumes two TCAM entries. Thus, 256,000 IPv6 routes is equal to
512,000 IPv4 routes.
The FIB TCAM is a single block of memory that is shared between the IPv4 routes/Multiprotocol
Label Switching (MPLS) labels and the IPv6 routes/Multicast routes.
The total memory space cannot be changed unless you replace the Supervisor and the DFC
module, but you can adjust the amount of memory that is allocated to either the IPv4/MPLS or the
IPv6/Multicast.
Solution
You can enter the mls cef maximum-routes ip <number in thousands> command in order to
adjust the number of routing entries that are allocated to the IPv4. This does not increase the
overall size of the FIB TCAM, but it reduces the number of routing entries that are allocated to the
IPv6 in order to increase the amount of TCAM space for the IPv4.
It is very important to check the number of MPLS, IPv6, and Multicast routes present before you
increase the allocation for IPv4 routes. Enter the show mls cef summary command in order to
verify the total amount of routes per protocol:
7600# show mls cef summary
Total routes: 513525
IPv4 unicast routes: 513507
IPv4 non-vrf routes: 513507
IPv4 vrf routes: 0
IPv4 Multicast routes: 3
MPLS routes: 1
IPv6 unicast routes: 5
IPv6 non-vrf routes: 5
IPv6 vrf routes: 0
IPv6 multicast routes: 3
EoM routes: 1
7600(config)# mls cef maximum-routes ip 1000
Maximum routes set to 1024000. Configuration will be effective on reboot.
Note: This example reduces the total number of available MPLS labels, IPv6 routes, and
IPv4 multicast routes to only 8,000. An increase in the size of the IPv4 TCAM space always
reduces the amount of TCAM space that is available to other protocols. Consider the current
and future MPLS, IPv6, and multicast needs of your network before you perform TCAM
adjustments.
After you adjust the maximum-routes, you must save the running configuration and reboot the
switch before the changes become active (there is no operational impact before the reboot). After
the reboot, you can enter the show mls cef maximum-routes command in order to view the new
TCAM allocations:
7600# show mls cef maximum-routes
FIB TCAM maximum routes :
=======================
Current :------IPv4
- 1000k
MPLS
- 8k (default)
IPv6 + IP Multicast - 8k (default)
After the adjustment, the total FIB TCAM size is unchanged. The total number of MPLS labels for
the IPv6 routes that can be used is reduced, which now allows for 1,000,000 IPv4 routes.