Nexus 2248PQ FEX Connection Problem with a
Nexus 6000 40G QSFP Port
Document ID: 116348
Contributed by Austin Peacock and Carlo Schmidt, Cisco TAC
Engineers.
Jul 23, 2013
Contents
Introduction
Prerequisites
Requirements
Components Used
Background Information
Problem
Solution
Related Information
Introduction
This document describes a problem with the Cisco Nexus 6000 connection to the Cisco Nexus 2248PQ Fabric
Extender (FEX) and offers a solution.
Prerequisites
Requirements
Cisco recommends that you have basic knowledge of these topics:
• Cisco Nexus 2000 Series FEX
• Cisco Nexus 6000 Series FEX Configuration
Components Used
The information in this document is based on these software and hardware versions:
• Cisco Nexus 2248PQ−10GE
• Cisco Nexus 6004
The information in this document was created from the devices in a specific lab environment. All of the
devices used in this document started with a cleared (default) configuration. If your network is live, make sure
that you understand the potential impact of any command.
Background Information
The Cisco Nexus 2248PQ FEX has sixteen 10−Gigabit Ethernet fabric interfaces via four Quad (4−channel)
Small Form−factor Pluggable (QSFP) ports and forty−eight 10−Gigabit Ethernet host interfaces. All host
interfaces use all of the available fabric interfaces. (Static pinning is not supported. Port−channel mode is only
supported on fabric interfaces.) When all host interfaces send traffic to all fabric interfaces, the maximum
oversubscription ratio for the Cisco Nexus 2248PQ is 3:1.
Problem
When you bring up the link on the Cisco Nexus 6000 FEX fabric ports that connects to the Cisco Nexus
2248PQ−10GE, the link on the Cisco Nexus 6000 shows as Link not connected, and the link lights on the
port of the Cisco Nexus 2248PQ are green or off:
Nexus6004−A# show int eth 2/7
Ethernet2/7 is down (Link not connected)
Dedicated Interface
Belongs to Po125
Hardware: 40000 Ethernet, address: 5897.1ef0.4856 (bia 5897.1ef0.4856)
MTU 1500 bytes, BW 40000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is fex−fabric
auto−duplex, 40 Gb/s, media type is 40g
Solution
In order to bring this link up, the Cisco Nexus 2248PQ needs to see the QSFP cable as four separate 10G links
(recall the previous description of the uplinks as sixteen 10G links via four QSFP ports). The same cable is
used, but you need to enter this command on the Cisco Nexus 6000 ports that you plan to break into four 10G
connections:
(config)# interface breakout slot 2 port 7−9 map 10g−4x
Note: You have to divide the ports into groups of three, and you must divide the entire group, not just one
member. So your options are 1−3, 4−6, 7−9, 10−12, or any combination that includes all members of the
previous groups. For example, 1−6, 7−12, 4−9.
Once you enter this command, you need to reload the module for the changes to take effect:
(config)# poweroff module 2
(config)# no poweroff module 2
It takes a few minutes for the module to come online. When it does, you should see your link divide into four
10G subports:
Nexus6004−A# show int status | i 2/7
Eth2/7/1
−−
connected 1
Eth2/7/2
−−
connected 1
Eth2/7/3
−−
connected 1
Eth2/7/4
−−
connected 1
full
full
full
full
10G
10G
10G
10G
−−
−−
−−
−−
At this point, you need to bundle the ports into a port−channel, and the normal FEX configuration can be
applied:
(config)# fex 270
(config)# int ethernet 2/7/1−4
(config−if−range)# switchport mode fex−fabric
(config−if−range)# channel−group 270
(config)# interface port−channel 270
(config−if)# switchport mode fex−fabric
(config−if)# fex associate 270
(config−if)# shut
(config−if)# no shut
Related Information
• Breakout Command in the Configuration Guide
• Nexus 6000 FEX Configuration Guide
• Nexus 2000 Data Sheet
Updated: Jul 23, 2013
Document ID: 116348