Storage connectivity and options
5.3
Unit objectives
After completing this unit, you should be able to:
• List the internal storage options for the blade server
• Select the appropriate I/O switch module for external storage access
• List the components required for Fibre Channel connectivity
• Select the components required for Fibre Channel boot from SAN
• Describe the functions implemented on the iSCSI expansion card
2
Storage options for BladeCenter: Internal storage
• Internal storage
• Fibre Channel
• iSCSI
• SAS
• Boot from SAN
3
Common components: Storage
• Hard drives
– SAS
– SATA
– SCSI
• Hot swap on selected blade servers
Solid State Disk (SSD)
• RAID (0,1)
• Solid state disk (SSD)
• Modular flash drive
Modular Flash Drive
Hard Disk
4
Common components: Storage configurations
Two internal hard drives, front-mounted hot swap
Two internal hard drives, non-hot swap
Modular Flash Drive
5
IBM expansion blades: Storage and I/O
RAID 5 (optional battery
cache)
Storage and
I/O
Expansion
Blade
I/O Expansion slot
Three SAS hot-swap
hard drive bays
I/O Expansion slot
Blade expansion
connector
IBM Blade (HS21, LS21, or LS41)
Storage and I/O Expansion Blade Assembly
6
Storage options for BladeCenter: Fibre Channel
• Internal storage
• Fibre Channel
• iSCSI
• SAS
• Boot from SAN
7
Fibre Channel components: Expansion cards
• IBM BladeCenter Fibre Channel expansion cards
– Provide blade server connectivity to remote storage
– BIOS support for boot from SAN
– Provide multi-port redundancy
1. Blade Server
2. Fibre Channel Expansion Card
8
Fibre Channel expansion cards: QLogic
• QLogic Fibre Channel expansion card models
– 8 Gb CIOv form factor
– 4 Gb CIOv form factor
– 4 Gb SFF form factor
– 4 Gb CFFv form factor
QLogic 4Gb Ethernet and FC CFFh Card
• QLogic combination Fibre Channel and Ethernet models
– 8 Gb CFFh form factor
– 4 Gb CFFh form factor
QLogic 8Gb Ethernet and FC CFFh Card
QLogic 8Gb FC CIOv Expansion Card QLogic 4Gb FC CIOv Expansion Card
QLogic 4Gb FC CFFv Expansion Card
9
Fibre Channel expansion cards: Emulex
•
Emulex Fibre Channel expansion card models
– Emulex 8 Gb Fibre Channel expansion card
• Two 8 Gb ports
• CIOv form factor
– Emulex 4 Gb Fibre Channel expansion card
• Two 4 Gb ports
• CFFv form factor
Emulex 8 Gb FC Expansion Card (CIOv)
Emulex 4 Gb FC Expansion Card (CFFv)
10
Fibre Channel storage components: Switch I/O modules
• IBM BladeCenter Fibre Channel Switch I/O Modules
– QLogic 20-port 8 Gb SAN Switch Module
– QLogic 10-port and 20-port 4Gb SAN Switch Module
– Brocade 10 and 20 port SAN Switch Module
• IBM BladeCenter Fibre Channel Pass-Thru I/O Modules
– QLogic Intelligent 8 Gb Pass-Thru Fibre Channel Module
– QLogic Intelligent 4 Gb Pass-Thru Fibre Channel Module
3. BladeCenter Chassis
4. Fibre Channel Switch I/O Modules
11
Definition of 10-port and 20-port
For the 10-port offerings, each switch module has three
external ports enabled and seven internal ports enabled
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
1
2
3
4
5
6
7
Internal Parts
For the 20-port offerings, each switch module has all six external
ports enabled and all fourteen internal ports enabled
Internal Parts
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
Blade
Bay
1
2
3
4
5
6
7
8
9
10
11
12
13
14
12
QLogic 10 and 20 Port FC switch modules
• QLogic 8 Gb SAN Switch I/O Module
– 20 Ports (14 internal, 6 external
– Supports 2, 4 and 8 Gb speeds
• QLogic 4 Gb SAN Switch I/O Module
– Affordable 10-port and 20-port offerings
available - ideal for Small, Medium and
Enterprise business needs
– Easy-to-use software upgrade doubles
10-port switch connectivity to 20-ports
for on demand scalability
– Included with every switch, QLogic's
SANsurfer Management Suite eases
installation, configuration and
management of your SAN infrastructure
all from one GUI
13
QLogic intelligent pass-thru Fibre Channel modules
• QLogic 8 Gb and 4 Gb Intelligent Pass-Thru Modules
– 20 port modules provide standards-based NPIV
interface
– Zoning and LUN Masking
– Alternative to traditional optical pass-thru
– 6 External auto-sensing ports
– 14 Internal auto-sensing ports
14
Brocade 10 and 20 port Fibre Channel switch module
• 1, 2 and 4 Gigabits per second
bandwidth
• Automatic failover support
• 10-port and 20-port
• Non-disruptive upgrade from 10port switch connectivity to 20ports
• Utilizes Brocades Silkworm and
interoperable with Silkworm and
IBM TotalStorage b-type SAN
switches
15
Managing the Fibre Channel switch modules
16
Interaction with the management module
• The switch interacts with the Blade Center Management
Module via an Inter-Integrated Circuit (I2C)* device to transfer
Vital Product Data (VPD)
• User can perform “write erase” using the “reset to factory
defaults” control on the MM’s GUI.
• User can disable ALL external ports via the MM GUI.
• The switch can be power cycled via the MM’s GUI.
• The switch provides some information to the MM.
–
–
–
–
–
–
Temperature alarms
Switch name
Port License State
SFP information
POST status
URL to launch the Device manager
17
BladeCenter AMM Interface: Admin/Power/Restart
18
BladeCenter AMM interface: Configuration
19
Establishing a remote session
The default IP addresses and
commands to open a Telnet commandprompt session open are:
• For the switch module in I/O bay 3:
– telnet 192.168.70.129
• For the switch module in I/O bay 4:
– telnet 192.168.70.130
20
SAN Browser for the QLogic switch module
• SAN Browser graphical
(Java) user interface:
– Allows for viewing and
configuration Ports, Zoning and
Network (including SNMP
alerting) settings.
– Troubleshooting through
Diagnostic functionalities
– Multiplatform support:
– Requires 128 MB RAM, 150
MB of disk space for
installation, 300 MHz CPU and
either Microsoft Internet
Explorer or Netscape Navigator
• ANSI Command Line
Interface
– Telnet directly onto IP address
of the module
– Functionally equivalent to the
Java application
21
SAN Browser topological view
Switch Icons appear in
the Graphic Window
Fabric Tree
Data Window
Data Window Tabs
for different displays
22
SAN Browser faceplate
Switch Name Status
Data Window Tabs
for different displays
23
SAN Browser zoning
• Zoning can be accomplished from Enterprise Fabric Connectivity Manager
(EFCM) for the Switch Modules
• Since Zoning is a Fabric Service, it can also be managed from EFCM
24
SAN Browser functions
25
Establishing a Web interface session for Brocade
To establish a Web-interface session through a Web browser, open a
supported Web browser on the network-management workstation and type
one of the following default IP addresses in the Web address field and press
Enter.
• For I/O bay 3: http://192.168.70.129
• For I/O bay 4: http://192.168.70.130
The I/O-module Advanced Web Tools interface window will then open.
26
Brocade switch administration
To access the Switch Admin
window:
• Select the desired switch from
the Fabric Tree.
The selected switch appears in
the Switch view.
• Click the Admin icon from the
Switch view.
The login dialog box displays.
Enter the admin user name
and password.
• Click the desired tab.
27
Brocade switch zone administration
To Create a Zone
• Click Create
• The Create New Config dialog
box appears
• Enter a name for the new
configuration and click OK
• Click on “+” signs in the
Member Selection List to
view the nested elements
• Highlight an element in the
Member Selection List that
you want to include
• Click Add Member to add
configuration members.
28
Configuring the expansion card
• Boot the blade server.
• Press Ctrl+Q to enter the
BIOS configuration
utility.
• Select the first Fibre
Channel adapter port
(port 1 correlates to the
top Fibre Channel
Switch in slot 3 of the
BladeCenter chassis).
• Select Configuration
Settings and Host
Adapter Settings.
29
Fibre Channel storage components: Disk systems
• IBM
BladeCenter Boot Disk System
• IBM System Storage DS3400
• IBM System Storage DS3200
30
Storage options for BladeCenter: iSCSI
• Internal storage
• Fibre Channel
• iSCSI
• SAS
• Boot from SAN
31
What is iSCSI?
• iSCSI: Internet SCSI (Small
Computer System Interface), SCSI
commands sent across a network in
TCP/IP packets. It was developed
as a storage networking standard for
linking data storage facilities
• Initiator: System making the iSCSI
requests for data
• Target: System receiving the iSCSI
requests for data
32
iSCSI expansion card overview
The iSCSI expansion card offers a connection to
an iSCSI storage device, via iSCSI host
connections, to leverage the available standard
Ethernet infrastructure to offer storage area
network (SAN) solutions
Blade Server
iSCSI Storage Device
Via TCP
33
Storage options for BladeCenter: SAS
• Internal storage
• Fibre Channel
• iSCSI
• SAS
• Boot from SAN
34
IBM BladeCenter storage: SAS expansion cards
• SAS Expansion Card
–
–
–
–
Configurable SAS device
Form factor CFFv card
Enables Boot from IBM BladeCenter Boot Disk System
Connects through SAS I/O Modules in module bays 3 and 4
• SAS Connectivity Card
– Uses SAS controller ports on Blade server
– Form factor CFFv
– Connects through SAS I/O Modules in module bays 3 and 4
SAS Expansion Card
SAS Connectivity Card
35
IBM BladeCenter storage: SAS connectivity module
• Provides SAS connectivity from Blade
server SAS expansion card to external SAS
storage
– 14 Internal connections to BladeServer blade bays
– 4 External connections to SAS infrastructure
Sample connection using dual SAS Connectivity Modules
IBM BladeCenter SAS Connectivity Module
36
IBM BladeCenter storage: DS3200 SAS controller
• 3 Gbps Serial Attached
SCSI (SAS) host
interface and drive
expansion technology
• Combination of 12 SAS
or SATA 3.5" drives per
enclosure
IBM DS3200
• Scalable to 5.4 TB of
storage
IBM DS3200 Rear Chassis, Dual SAS Controllers
37
Storage options for BladeCenter: Boot from SAN
• Internal storage
• Fibre Channel
• iSCSI
• SAS
• Boot from SAN
38
Booting from SAN
In th is e x a m p le , w e c o n n e c t
th e B la d e C e n te r to a n IB M
To ta lS to ra g e D S 4 5 0 0
th ro u g h tw o F ib re C h a n n e l
S w itc h e s . B o th F C S w itc h
m o d u le s a reBoot
in uLUN
se.
Blade Server
39
Benefits to booting from SAN
• Improving disaster tolerance If a server becomes faulty,
unavailable or compromised, it can be swapped out for a new
server with the same configuration, without having to:
– Reconstruct the booting, operating and application environments for
the new machine when it is most urgently needed.
– Swap out hard drives, reconfigure arrays, and restore data and
applications from backup.
– The ability to replace and add servers in minutes provides a significant
return on investment (ROI) by preventing lengthy downtime of core
servers.
• Enabling centralized administration
• Reducing TCO through diskless servers
• Using high availability (HA) storage ensuring business
continuance
40
Implementation considerations of booting from SAN
• The server must have an FC HBA equipped with a boot BIOS that can support
booting from the storage device attached to the SAN
• Typically the host needs to have a boot order of CD-ROM, diskette, and then drive 0
• The SAN must be installed and configured, and the storage visible to the host
• The storage system must have at least one available LUN for booting the server(s)
• The storage must be configured to allow the server exclusive access to the LUN;
access is typically assigned via the FC HBA’s WorldWide Port Name (WWPN) on
the storage
• To minimize data interruptions between hosts, it is best to keep hosts separated
from each other by way of zoning to minimize the Registered State Change
Notification (RSCN) interruptions
41
Booting from SAN: Creating the LUNs
9 GB Logical Drive
18 GB
18 GB
Array A (RAID 5)
18 GB
1. Create an array
2. Create a logical drive
(LUN)
3. From the blade,
broadcast the WWPN
4. Map the LUN to the
host via the WWPN
5. Assign the Boot LUN to
HBA Port A
6. Install the operating
system
42
Booting from SAN: Zoning the switch
• SAN Zoning provides partitioning of
resources
• Zoning is done at the fabric switch
level
• Zones can be created at port level
– HBA
– Switch
– SAN Controller
• Zones can also be created at the
WWN level
– HBA
– Switch
– SAN Controller
Zoning by WWN
Zoning by Port
43
Configuring the expansion card BIOS
• Boot the blade server.
• Press Ctrl+Q to enter the
BIOS configuration
utility.
• Select the first Fibre
Channel adapter port
(port 1 correlates to the
top Fibre Channel
Switch in slot 3 of the
BladeCenter chassis).
• Select Configuration
Settings and Host
Adapter Settings.
44
Broadcasting the world wide name (WWN)
In order for the storage device to see
the HBA ports, you need to open the
port by doing a Scan Fibre Devices.
Do this for both ports.
45
Mapping the logical drive to the WWN
46
Assign the boot LUN to an HBA port
We want to assign the boot
LUN to port B also, but we will
disable this path during the
initial OS installation
47
Installing the operating system
• You must use the Fibre card Driver diskette during the installation phase (press F6 for
boot disk during install)
• During the POST sequence notice the following text:
Drive letter C: is moved to the Drive letter D:
Loop ID 129,0 is installed as Drive C:
• This indicates that the blade is now booting from the LUN 0 associated to the first HBA
adapter
48
Key words
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Access Control List (ACL)
Basic Input/Output System (BIOS)
Boot from SAN
Fibre Channel
Fabric
Just a bunch of disks (JBOD)
InfiniBand
Internet Small Computer System Interface (iSCSI)
Logical Unit Number (LUN)
Serial-attached SCSI (SAS)
Small Computer System Interface (SCSI)
Small Form Factor (SmFF)
Standard Form Factor (StFF)
Storage Area Network (SAN)
Zoning
LUN masking
World Wide Name (WWN)
49
Checkpoint (1 of 2)
1. True/False: Blade server local disk options include hot-swap capability.
2. To enable fibre channel (FC) SAN connectivity, which of the following
statements is true?
a. The BladeCenter chassis must have a fibre channel controller
installed in I/O bay 2
b. Internal cabling must be performed from the Blade server to the
fibre channel switch
c. The Blade server fibre channel expansion card (HBA) World Wide
Name (WWN) must be known to the external fibre channel
controller
d. The BladeCenter Ethernet Switch I/O module must be connected to
the external fibre channel controller
3. True/False: The QLogic iSCSI TOE Expansion card contains an iSCSI
Initiator
50
Checkpoint solutions (1 of 2)
1. True/False: Blade server local disk options include hot-swap
capability.
Answer: True
2. To enable fibre channel (FC) SAN connectivity, which of the following
statements is true?
a. The BladeCenter chassis must have a fibre channel controller
installed in I/O bay 2
b. Internal cabling must be performed from the Blade server to the
fibre channel switch
c. The Blade server fibre channel expansion card (HBA) World
Wide Name (WWN) must be known to the external fibre
channel controller
d. The BladeCenter Ethernet Switch I/O module must be connected
to the external fibre channel controller
Answer: c
3. True/False: The QLogic iSCSI TOE Expansion card contains an iSCSI
Initiator
Answer: True
51
Checkpoint (2 of 2)
4. Select the correct statement regarding Boot from SAN.
a. The storage controller requires a minimum of (4) available boot LUNs
b. A parallel Ethernet connection must exist for each Blade server using fibre
channel
c. The I/O Module Advanced Configuration page in the AMM is used to enable
the Blade server fibre channel expansion card BIOS
d. The components required for Boot from SAN include: Blade server fibre
channel expansion card, fibre channel switch I/O module and external fibre
channel controller
5. True/False: To connect from the BladeCenter chassis to an external
Fibre Channel fabric requires either a Fibre Channel switch module or
an SAS Connectivity module.
6. True/False: Boot from SAN required steps include: Configuring the
fibre channel expansion card BIOS and configuring a LUN at the fibre
channel storage controller.
52
Checkpoint solutions (2 of 2)
4. Select the correct statement regarding Boot from SAN.
a. The storage controller requires a minimum of (4) available boot LUNs
b. A parallel Ethernet connection must exist for each Blade server using fibre
channel
c. The I/O Module Advanced Configuration page in the AMM is used to enable
the Blade server fibre channel expansion card BIOS
d. The components required for Boot from SAN include: Blade server fibre
channel expansion card, fibre channel switch I/O module and external
fibre channel controller
Answer: d
5. True/False: To connect from the BladeCenter chassis to an external
Fibre Channel fabric requires either a Fibre Channel switch module or
an SAS Connectivity module.
Answer: False
6. True/False: Boot from SAN required steps include: Configuring the fibre
channel expansion card BIOS and configuring a LUN at the fibre channel
storage controller.
Answer: True
53
Unit summary
Having completed this unit, you should be able to:
• List the internal storage options for the blade server
• Select the appropriate I/O switch module for external storage access
• List the components required for Fibre Channel connectivity
• Select the components required for Fibre Channel boot from SAN
• Describe the functions implemented on the iSCSI expansion card
54