© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
1
• Physical Uplinks and Connectivity Verification
• UCSM Configuration – Boot Policy, VHBAs, End-Host Mode, VSANs
• Upstream 5k VSAN configuration and verification
• 5k FCalias and Zoning Configuration and Verification
• Verification via Lunlist and FLOGI
• EMC VNX side LUN Masking
• Boot from SAN
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
2
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
3
Physical Uplinks and Connectivity Verification
We already cabled from the 5k’s to the EMC VNX on interfaces fc1/30 and
fc1/32 so you don’t need to do that. HOWEVER, you do need to cable your
FI’s to the 5k’s upstream (FIA to 5kA and FIB to 5kB) with FC uplinks.
FI A side:
interface fc2/13
switchport mode NP
switchport trunk mode on
no shutdown
© 2010 Cisco and/or its affiliates. All rights reserved.
FI B side:
interface fc2/13
switchport mode NP
switchport trunk mode on
no shutdown
Cisco Confidential
4
Physical Uplinks and Connectivity Verification
We already cabled from the 5k’s to the EMC VNX on interfaces fc1/30 and
fc1/32 so you don’t need to do that. HOWEVER, you do need to cable your
FI’s to the 5k’s upstream (FIA to 5kA and FIB to 5kB) with FC uplinks.
AGR012-5K-A# show run interface fc1/29-32
(fc1/29 and 1/31 removed)
AGR012-5K-B# show run interface fc1/29-32
(fc1/29 and 1/31 removed)
interface fc1/30
switchport description emcVnx_B2_port1
no shutdown
interface fc1/30
switchport description emcVnx_A2_port1
no shutdown
interface fc1/32
switchport description emcVnx_A2_port0
no shutdown
interface fc1/32
switchport description emcVnx_B2_port0
no shutdown
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
5
UCSM Configuration – Boot Policy, VHBAs, EndHost Mode, and VSANs
Boot Policy
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
6
UCSM Configuration – Boot Policy, VHBAs, EndHost Mode, and VSANs
VHBAs - You need two VHBAs configured
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
7
UCSM Configuration – Boot Policy, VHBAs, EndHost Mode, and VSANs
You must be running in FC End-Host Mode (B side shown here)
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
8
UCSM Configuration – Boot Policy, VHBAs, EndHost Mode, and VSANs
Create VSANs for both A and B side in UCSM and label them to reflect
your name, VLAN wiki number and A/B side. My VLAN wiki is 28.
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
9
UCSM Configuration – Boot Policy, VHBAs, EndHost Mode, and VSANs
Create VSANs for both A and B side in UCSM and label them to reflect
your name, VLAN wiki number and A/B side. My VLAN wiki is 28.
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
10
UCSM Configuration – Boot Policy, VHBAs, EndHost Mode, and VSANs
Create VSANs for both A side (128) and B side (228) in UCSM and
label them appropriately.
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
11
UCSM Configuration – Boot Policy, VHBAs, EndHost Mode, and VSANs
After you create VSANs for both A side (128) and B side (228) in UCSM
it should look something like this.
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
12
Upstream 5k VSAN Configuration and Verification
Now you will need to create VSANs on the upstream 5kA and 5kB to
match your new VSANs you made in UCSM. (Done for you already)
A side:
(config)#vsan database
(config-vsan-db)#vsan 128 name VNX3949_FabA
B side:
(config)#vsan database
(config-vsan-db)#vsan 228 name VNX3949_FabB
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
13
Upstream 5k VSAN Configuration and Verification
To verify the vsan configuration on the 5k just do a show interface on
one of the 5k uplinks.
A Side Config:
B Side Config:
AGR012-5K-A# show int fc1/31
AGR012-5K-B# show int fc1/31
fc1/31 is trunking
fc1/31 is trunking
Port description is profx_fiA_port2_13
Port description is profx_fiB_port2_13
Hardware is Fibre Channel, SFP is short wave laser w/o Hardware is Fibre Channel, SFP is short wave laser
OFC (SN)
w/o OFC (SN)
Port WWN is 20:1f:8c:60:4f:05:d3:80
Port WWN is 20:1f:8c:60:4f:05:d6:40
Admin port mode is auto, trunk mode is on
Admin port mode is auto, trunk mode is on
snmp link state traps are enabled
snmp link state traps are enabled
Port mode is TF
Port mode is TF
Port vsan is 1
Port vsan is 1
Speed is 4 Gbps
Speed is 4 Gbps
Transmit B2B Credit is 16
Transmit B2B Credit is 16
Receive B2B Credit is 16
Receive B2B Credit is 16
Receive data field Size is 2112
Receive data field Size is 2112
Beacon is turned off
Beacon is turned off
Trunk vsans (admin allowed and active) (1,128)
Trunk vsans (admin allowed and active) (1,228)
Trunk vsans (up)
(1,128)
Trunk vsans (up)
(1,228)
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
14
5k FC Aliases and Zoning Configuration and
Verification
Now it’s time to create your FC aliases and zoning on the upstream 5k
for your service profile/server and HBA WWPNs.
FC Alias Config Side A:
AGR012-5K-A(config)# fcalias name Tray_Gambit_ESX-55-03_fc0 vsan 128
AGR012-5K-A(config-fcalias)# member pwwn 20:00:00:15:B5:0A:00:03
AGR012-5K-A(config)# show fcalias
fcalias name Tray_Gambit_ESX-55-03_fc0 vsan 128
pwwn 20:00:00:15:b5:0a:00:03 matches VHBA fc0 WWPN
FC Alias Config Side B:
AGR012-5K-B(config)# fcalias name Tray_Gambit_ESX_55-03_fc1 vsan 228
AGR012-5K-B(config-fcalias)# member pwwn 20:00:00:15:B5:0B:00:03
AGR012-5K-B(config-fcalias)# show fcalias
fcalias name Tray_Gambit_ESX_55-03_fc1 vsan 228
pwwn 20:00:00:15:b5:0b:00:03  matches VHBA fc1 WWPN
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
15
5k FC Aliases and Zoning Configuration and
Verification
Now it’s time to create your FC aliases and zoning on the upstream 5k
for your service profile/server and HBA WWPNs.
Zone Config Side A:
AGR012-5K-A(config)# zone name z_Tray_Gambit_ESX_55_03_VNX3949_SPA0 vsan 128
AGR012-5K-A(config-zone)# member pwwn 20:00:00:15:B5:0A:00:03
AGR012-5K-A(config-zone)# member pwwn 50:06:01:60:36:e0:0f:26
AGR012-5K-A(config)# zone name z_Tray_Gambit_ESX_55_03_VNX3949_SPB1 vsan 128
AGR012-5K-A(config-zone)# member pwwn 20:00:00:15:B5:0A:00:03
AGR012-5K-A(config-zone)# member pwwn 50:06:01:69:36:e0:0f:26
Zone Config Side B:
AGR012-5K-A(config)# zone name z_Tray_Gambit_ESX_55_03_VNX3949_SPA1 vsan 228
AGR012-5K-A(config-zone)# member fcalias Tray_Gambit_ESX_55-03_fc1
AGR012-5K-A(config-zone)# member pwwn 50:06:01:61:36:e0:0f:26
AGR012-5K-A(config)# zone name z_Tray_Gambit_ESX_55_03_VNX3949_SPB0 vsan 228
AGR012-5K-A(config-zone)# member pwwn 20:00:00:15:B5:0b:00:03
AGR012-5K-A(config-zone)# member pwwn 50:06:01:68:36:e0:0f:26
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
16
5k FC Aliases and Zoning Configuration and
Verification
Verify your FC Alias configurations.
FC Alias Verification Side A:
AGR012-5K-A# show fcalias | inc Tray
fcalias name Tray_Gambit_ESX-55-03 vsan 128
FC Alias Verification Side B:
AGR012-5K-B# show fcalias | inc Tray
fcalias name Tray_Gambit_ESX_55-03 vsan 228
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
17
5k FC Aliases and Zoning Configuration and
Verification
Verify your Zoning configurations.
Zoning Verification Side A:
AGR012-5K-A(config)# show zone vsan 128
zone name z_Tray_Gambit_ESX_55_03_VNX3949_SPA0 vsan 128
pwwn 20:00:00:15:b5:0a:00:03
pwwn 50:06:01:60:36:e0:0f:26
zone name z_Tray_Gambit_ESX_55_03_VNX3949_SPB1 vsan 128
pwwn 20:00:00:15:b5:0a:00:03
pwwn 50:06:01:69:36:e0:0f:26
Zoning Verification Side B:
AGR012-5K-B(config)# show zone vsan 228
zone name z_Tray_Gambit_ESX_55_03_VNX3949_SPA1 vsan 228
pwwn 20:00:00:15:b5:0b:00:03
pwwn 50:06:01:61:36:e0:0f:26
zone name z_Tray_Gambit_ESX_55_03_VNX3949_SPB0 vsan 228
pwwn 20:00:00:15:b5:0b:00:03
pwwn 50:06:01:68:36:e0:0f:26
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
18
5k FC Aliases and Zoning Configuration and
Verification
Add the zone you created to the existing zoneset VNXLAB
DoNotDelete_FabX.
Adding my zone side A:
AGR012-5K-A(config)# zoneset name VNXLAB_DoNotDelete_FabA vsan 128
AGR012-5K-A(config-zoneset)# member z_Tray_Gambit_ESX_55_03_VNX3949_SPA0
AGR012-5K-A(config-zoneset)# member z_Tray_Gambit_ESX_55_03_VNX3949_SPB1
Adding my zone side B:
AGR012-5K-B(config)# zoneset name VNXLAB_DoNotDelete_FabB vsan 228
AGR012-5K-B(config-zoneset)# member z_Tray_Gambit_ESX_55_03_VNX3949_SPA1
AGR012-5K-B(config-zoneset)# member z_Tray_Gambit_ESX_55_03_VNX3949_SPB0
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
19
5k FC Aliases and Zoning Configuration and
Verification
Activate the zoneset: this will make your zones part of the active set.
AGR012-5K-A(config-zoneset)# zoneset activate name VNXLAB_DoNotDelete_FabA vsan 128
Zoneset activation initiated. check zone status
AGR012-5K-A(config)# show zoneset active
zoneset name VNXLAB_DoNotDelete_FabA vsan 128
zone name z_Tray_Gambit_ESX_55_03_VNX3949_SPA0 vsan 128
pwwn 20:00:00:15:b5:0a:00:03
* fcid 0x1c00ef [pwwn 50:06:01:60:36:e0:0f:26]
zone name z_Tray_Gambit_ESX_55_03_VNX3949_SPB1 vsan 128
pwwn 20:00:00:15:b5:0a:00:03
* fcid 0x1c01ef [pwwn 50:06:01:69:36:e0:0f:26]
AGR012-5K-B(config-zoneset)# zoneset activate name VNXLAB_DoNotDelete_FabB vsan 228
Zoneset activation initiated. check zone status
AGR012-5K-B(config)# show zoneset active
zoneset name VNXLAB_DoNotDelete_FabB vsan 228
zone name z_Tray_Gambit_ESX_55_03_VNX3949_SPA1 vsan 228
pwwn 20:00:00:15:b5:0b:00:03
* fcid 0xb200ef [pwwn 50:06:01:61:36:e0:0f:26]
zone name z_Tray_Gambit_ESX_55_03_VNX3949_SPB0 vsan 228
pwwn 20:00:00:15:b5:0b:00:03
* fcid 0xb200ef [pwwn 50:06:01:61:36:e0:0f:26]
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
20
Verification via FLOGI and Lunlist
With ESXi already installed, you should see FLOGI on the 5ks
FLOGI Side A:
AGR012-5K-A# show flogi database | inc 128
fc1/30
128 0x1c01ef 50:06:01:69:36:e0:0f:26 50:06:01:60:b6:e0:0f:26
fc1/31
128 0x1c0000 20:00:00:15:b5:0a:00:03 20:00:00:15:b5:00:00:03
fc1/32
128 0x1c00ef 50:06:01:60:36:e0:0f:26 50:06:01:60:b6:e0:0f:26
The FLOGI output matches previous zoning config on side A
pwwn 20:00:00:15:b5:0a:00:03 – Initiator UCS VHBA
pwwn 50:06:01:60:36:e0:0f:26 - Target EMC
pwwn 50:06:01:69:36:e0:0f:26 – Target EMC
FLOGI Side B:
AGR012-5K-B# show flogi database | inc 228
fc1/30
228 0xb200ef 50:06:01:61:36:e0:0f:26 50:06:01:60:b6:e0:0f:26
fc1/31
228 0xb20001 20:00:00:15:b5:0b:00:03 20:00:00:15:b5:00:00:03
fc1/32
228 0xb201ef 50:06:01:68:36:e0:0f:26 50:06:01:60:b6:e0:0f:26
The FLOGI output matches previous zoning config on side B
pwwn 20:00:00:15:b5:0b:00:03 – Initiator UCS VHBA
pwwn 50:06:01:61:36:e0:0f:26 – Target EMC
pwwn 50:06:01:68:36:e0:0f:26 – Target EMC
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
21
Verification via FLOGI and Lunlist
If you restart ESXi, during boot, you should see lunlist info on the 5ks
ProfX-A# connect adapter 2/3/1
adapter 2/3/1 # connect
No entry for terminal type "dumb";
using dumb terminal settings.
adapter 2/3/1 (top):1# attach-fls
No entry for terminal type "dumb";
using dumb terminal settings.
adapter 2/3/1 (fls):1# vnic
---- ---- ---- ------- ------vnic ecpu type state lif
---- ---- ---- ------- ------14 1
fc active 3
16 2
fc active 5
© 2010 Cisco and/or its affiliates. All rights reserved.
adapter 2/3/1 (fls):2# lif 3
lifid : 3
state : active
wwpn : 20:00:00:15:b5:0a:00:03
lif_mac : 00:15:b5:0a:00:03
port : 0
flogi state : flogi est  FLOGI established!
adapter 2/3/1 (fls):4# lif 5
lifid : 5
state : active
wwpn : 20:00:00:15:b5:0b:00:03
lif_mac : 00:15:b5:0b:00:03
port : 1
flogi state : flogi est  FLOGI established!
Cisco Confidential
22
Verification via FLOGI and Lunlist
If you restart ESXi, during boot, you should see lunlist info on the 5ks
adapter 2/3/1 (fls):5# lunlist
vnic : 14 lifid: 3
- FLOGI State : flogi est (fc_id 0x1c0000)
- PLOGI Sessions
- WWNN 50:06:01:69:36:e0:0f:26 WWPN 50:06:01:69:36:e0:0f:26 fc_id 0x1c01ef  Boot policy configured targets
- LUN's configured (SCSI Type, Version, Vendor, Serial No.)
LUN ID : 0x0000000000000000 (0x0, 0x4, DGC , APM00151123949)  Boot lun access granted
- REPORT LUNs Query Response
LUN ID : 0x0000000000000000  Boot lun reported
LUN ID : 0x0001000000000000  dataTest lun reported
- WWNN 50:06:01:60:36:e0:0f:26 WWPN 50:06:01:60:36:e0:0f:26 fc_id 0x1c00ef
- LUN's configured (SCSI Type, Version, Vendor, Serial No.)
LUN ID : 0x0000000000000000 (0x0, 0x4, DGC , APM00151123949)  Boot lun access granted
- REPORT LUNs Query Response
LUN ID : 0x0000000000000000
LUN ID : 0x0001000000000000
- Nameserver Query Response
- WWPN : 50:06:01:60:36:e0:0f:26  Nameserver response matches boot policy config
- WWPN : 50:06:01:69:36:e0:0f:26
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
23
Verification via FLOGI and Lunlist
If you restart ESXi, during boot, you should see lunlist info on the 5ks
adapter 2/3/1 (fls):5# lunlist (continued)
vnic : 16 lifid: 5
- FLOGI State : flogi est (fc_id 0xb20001)
- PLOGI Sessions
- WWNN 50:06:01:68:36:e0:0f:26 WWPN 50:06:01:68:36:e0:0f:26 fc_id 0xb201ef  Boot policy configured targets
- LUN's configured (SCSI Type, Version, Vendor, Serial No.)
LUN ID : 0x0000000000000000 (0x0, 0x4, DGC , APM00151123949)  Boot lun access granted
- REPORT LUNs Query Response
LUN ID : 0x0000000000000000  Boot lun reported
LUN ID : 0x0001000000000000  dataTest lun reported
- WWNN 50:06:01:61:36:e0:0f:26 WWPN 50:06:01:61:36:e0:0f:26 fc_id 0xb200ef
- LUN's configured (SCSI Type, Version, Vendor, Serial No.)
LUN ID : 0x0000000000000000 (0x0, 0x4, DGC , APM00151123949)  Boot lun access granted
- REPORT LUNs Query Response
LUN ID : 0x0000000000000000
LUN ID : 0x0001000000000000
- Nameserver Query Response
- WWPN : 50:06:01:61:36:e0:0f:26  Nameserver response matches boot policy config
- WWPN : 50:06:01:68:36:e0:0f:26
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
24
Verification via FLOGI and Lunlist
If you restart ESXi, during boot, you can get lunmap off the 5ks
adapter 2/3/1 (fls):6# lunmap
lunmapid: 0 port_cnt: 1
lif_id: 3
PORTNAME
NODENAME
LUN
PLOGI
50:06:01:60:36:e0:0f:26 00:00:00:00:00:00:00:00 0000000000000000 Y
lunmapid: 1 port_cnt: 1
lif_id: 3
PORTNAME
NODENAME
LUN
PLOGI
50:06:01:69:36:e0:0f:26 00:00:00:00:00:00:00:00 0000000000000000 Y
lunmapid: 0 port_cnt: 1
lif_id: 5
PORTNAME
NODENAME
LUN
PLOGI
50:06:01:61:36:e0:0f:26 00:00:00:00:00:00:00:00 0000000000000000 Y
lunmapid: 1 port_cnt: 1
lif_id: 5
PORTNAME
NODENAME
LUN
PLOGI
50:06:01:68:36:e0:0f:26 00:00:00:00:00:00:00:00 0000000000000000 Y
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
25
EMC VNX Side LUN Masking
The EMC VNX will already see your PWWN initiators so all we need to
do is add the necessary LUN masking.
The pwwns show up in the EMC management software. Here we have registered one already. (B side not shown)
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
26
EMC VNX Side LUN Masking
The EMC VNX will already see your PWWN initiators so all we need to
do is add the necessary LUN masking.
Register A side Initiator
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
27
EMC VNX Side LUN Masking
The EMC VNX will already see your PWWN initiators so all we need to
do is add the necessary LUN masking.
Register B side Initiator
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
28
EMC VNX Side LUN Masking
The EMC VNX will already see your PWWN initiators so all we need to
do is add the necessary LUN masking.
All initiators are now registered!
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
29
EMC VNX Side LUN Masking
The EMC VNX will already see your PWWN initiators so all we need to
do is add the necessary LUN masking.
Add a new storage group.
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
30
EMC VNX Side LUN Masking
The EMC VNX will already see your PWWN initiators so all we need to
do is add the necessary LUN masking.
Add a new storage group. (cont’d)
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
31
EMC VNX Side LUN Masking
The EMC VNX will already see your PWWN initiators so all we need to
do is add the necessary LUN masking.
Add existing LUNs to new sg_Gambit-ESX-55-03 storage group
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
32
EMC VNX Side LUN Masking
The EMC VNX will already see your PWWN initiators so all we need to
do is add the necessary LUN masking.
Storage Group Add Lun’s (cont’d)
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
33
EMC VNX Side LUN Masking
The EMC VNX will already see your PWWN initiators so all we need to
do is add the necessary LUN masking.
Storage Group Add Lun’s (cont’d)
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
34
EMC VNX Side LUN Masking
The EMC VNX will already see your PWWN initiators so all we need to
do is add the necessary LUN masking.
New storage group created and LUNs associated.
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
35
Boot From SAN
Now that everything is setup you should be able to boot from SAN.
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
36
Boot From SAN
Now that everything is setup you should be able to boot from SAN.
Targets Match!
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
37
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
38