Storage
Fundamentals
October 2014
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Introduction
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Course overview
After completing this training, you should be able to:
• Explain hard drive types, interconnect technologies, and RAID levels.
• Explain Fibre Channel storage area networks (SANs) and components, and compare SANs to direct
attached storage (DAS) and network-attached storage (NAS).
• Describe the Fibre Channel architecture, characteristics, and operation, including naming and
addressing.
• Describe the Fibre Channel Arbitrated Loop topology and its operation and benefits.
• Describe a switch topology with fabric operation and zoning concepts.
• Describe fiber optic technology and Fibre Channel cabling options and connectors.
3
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Defining the storage
technology
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Hard drives
5
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Characteristics of drives
• Form factor
− Small form factor (SFF)—2.5-inch
− Large form factor (LFF)—3.5-inch
• Drive capacity
− Depends on number of platters the drive contains, the surface area of each platter, and the areal
density (the number of bits that can be stored per unit area)
− Expressed in gigabytes
• Disk drive performance
− Depends on the rotational speed of the platters, the seek performance, the mechanical latency, the
read/write bandwidth, the queuing strategies, and the interface technologies
• Reliability
− Measured in terms of Annual Failure Rates (AFRs)
6
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Drive interconnect technologies
The technology to connect one or more drives to a computer system has transitioned from
parallel bus data interfaces to serial interfaces
• Parallel interfaces:
− ATA—Advanced Technology Attachment
− IDE—Integrated Drive Electronics, also called PATA, Parallel Advanced Technology Attachment
− SCSI—Small Computer System Interface
• Serial interfaces:
− SATA—Serial ATA
− SAS—Serial Attached SCSI
7
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Serial ATA and Serial Attached SCSI drives
Serial ATA
Serial Attached SCSI
• SAS uses a point-to-point, full-duplex serial
• SATA uses a half-duplex serial connection and
connection and the SCSI command set
ATA uses a command set
• Two generations of SAS drives:
• Three generations of SATA drives:
− First-generation SAS supported a link speed
− 1.5 Gb/s
of
• Targeted at replacing ATA in the desktop and
3 Gb/s
consumer markets
− The current generation supports a link speed
− 1.5 Gb/s with extensions
of up to 6 Gb/s
• Targeted for workstations and low-end servers
• This generation added native command
queuing
− 3 Gb/s
• Targeted for workstations and low-end servers
• This generation increased the data transfer
© Copyright
8
rate 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Improving performance and reliability with
RAID
• Storing data on the single drive creates the risk of
losing data
• To achieve better performance and fault
tolerance, it is recommended to store data across
multiple drives
• Disks can be combined to form an Redundant
Array of Independent Disks (RAID)
• RAID strategies vary
− How they achieve data reliability
− How many drives they require
− How efficient they are at data storage
9
Raid
level
Description
RAID 0
Striping
RAID 1
Mirroring
RAID 1+0
Striping and mirroring
RAID 5
Block striping with distributed
parity
RAID 6
Block striping with distributed
parity
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
RAID 0
• Minimum of 2 disks
• Excellent performance (as blocks are striped)
• No redundancy (no mirror, no parity)
• Do not use this for any critical system
NOTICE: RAID 0 provides no data
redundancy.
10
RAID0
A1
A3
A5
A7
A2
A4
A6
A8
Disk 0
Disk 1
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
RAID 1
• Minimum of 2 disks
• Good performance (no striping, no parity)
• Excellent redundancy (blocks are mirrored)
• Provides 50% of usable disk space
NOTE: For more information about mirroring,
go to:
http://en.wikipedia.org/wiki/Disk_mirroring
11
RAID1
A1
A2
A3
A4
A1
A2
A3
A4
Disk0
Disk1
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
RAID 1+0
• Minimum of 4 disks
• RAID 1+0 is also called “stripe of mirrors”
• Excellent redundancy (blocks are mirrored)
• Excellent performance (blocks are striped)
• This is the best option for any mission-critical
applications (especially databases)
• Provides 50% of usable drive space
12
RAID1+0
RAID0
RAID1
RAID1
A1
A3
A5
A7
A1
A3
A5
A7
A2
A4
A6
A8
A2
A4
A6
A8
Disk0
Disk1
Disk2
Disk3
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
RAID 5
• Minimum of 3 disks
• Good performance (blocks are striped)
• Good redundancy (distributed parity)
• The most cost-effective option, providing both
performance and redundancy
• Use this for a database that is heavily read
oriented
• Write operations will be slow
• Provides 67% to 93% of usable drive space
13
RAID5
A1
B1
C1
Dp
A2
B2
Cp
D1
A3
Bp
C2
D2
Ap
B3
C3
D3
Disk0
Disk1
Disk2
Disk3
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
RAID 6
• Just like RAID 5, this does block-level striping
− However, it uses dual parity
− It creates two parity blocks for each data block
• Can handle two disk failures
• Requires a minimum of 4 drives
• This RAID configuration is complex to implement
in a RAID controller because it has to calculate
two parity data for each data block
14
RAID6
A1
B1
C1
Dp
A2
B2
Cp
Dq
A3
Bp
Cq
D1
Ap
Bq
C2
D2
Disk0
Disk1
Disk2
Disk3
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Aq
B3
C3
D3
Disk4
Storage arrays
• Multiple drives combined to increase overall
storage capacity, data availability, and
performance
• Drives are combined to form RAID groups
• Available disk space is arranged in the form of
logical (virtual) drives
• Clients (hosts) access the available storage
space using available communication channels
such as:
− iSCSI (SCSI over TCP/IP)
− FC (Fibre Channel)
− FCoE (Fibre Channel over Ethernet)
15
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Switches, Fibre Channel, iSCSI technologies
• The fabric for a SAN provides the connectivity
between the host servers and the storage
devices
• The dominant architecture for SANs is based on
Fibre Channel (FC)
• Compared to SCSI devices, many more storage
devices can be connected over much larger
distances with higher data transfer rates
• In Fibre Channel topologies, the host server can
be connected to the storage directly, or by means
of a hub or a switch
16
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Introduction to DAS, NAS, and
SAN
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
DAS, NAS, and SAN
18
DAS
NAS
SAN
Advantages
Speed and security
Simple implementation
Unrestricted distance
over the LAN
Performance
Scalability
Manageability
Disadvantage
s
Distance restrictions
High network overhead
Limited scalability
High network
overhead and limited
scalability
Greater initial
investment support
expertise
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Direct attached storage
• The traditional method of locally attaching storage to servers through a dedicated SCSI
communication channel between the server and storage
• Storage for each server is managed separately and cannot be shared
• DAS supports disk drives, a RAID subsystem, or another storage device
19
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Network-attached storage
• NAS provides a file-level access to storage
systems
• NAS devices are:
− Server-independent
− Used to off-load storage traffic to a single,
dedicated storage device
Application
Database
Server
Server
Clients
Clients
NAS
Internal SCSI or
SAN attached storage
20
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Fibre Channel storage area network
Dedicated network that provides access to
consolidated, block-level data storage
• Special switches are used to connect storage
arrays with servers and with each other
• Network communication uses the Fibre Channel
protocol, which was specially developed for the
transport of files
− This protocol is reliable, with speeds up to 16
Gbit/s
• FC SAN components allow for high levels of
redundancy and resiliency
21
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
SAN considerations
When designing SAN solutions, consider the following:
• Scalability (number of FC ports and expansion capability)
• Storage capacity, efficiency, and cost
• Availability of the fabric, systems, and data
• Performance
• Remote replication of data
22
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Comparing SAN and NAS
SAN benefits
NAS
SAN
• Network speed
• Reliability
• Centralization
• Data protection
NAS benefits
• Interoperability
• Lower TCO
• Simplicity
23
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Comparing DAS, NAS, and SAN
24
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Choosing between SAN, NAS, and DAS
DAS
NAS
SAN
Applications
Any
File serving
Storage for application
servers
Server and Operating
System
General purpose
Optimized
General purpose
Storage Devices
Internal or external dedicated
External direct-attached
External shared
Management
Labor intensive
Centralized
Centralized
Data Centers
Workgroup or departmental
Workgroup or departmental
Small workgroup to
enterprise data centers
Performance
Network traffic
Increased network
performance
Higher bandwidth
Distance
None
Limited distance
Greater distances
Speed
Bottlenecks
Improved bottlenecks
Greater speeds (up to 16
Gbit/s)
Availability
Limited
Limited
No Single Point of Failure
(NSPOF)
25
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Cost
Low cost
Affordable
High host, but great benefits
Tiered storage
Performance and cost
Policy-based Data Migration
26
Tier 1
Tier 2
Online
Tier 3
Near-online
Reference information
Faster recovery
Instant data access and
recovery
Near line,
Searchable
File recovery
Indexed online archive
Frequently accessed
data
Backup devices,
Tapes and Tape
Libraries
NAS or DAS based
solutions
Tier 4
High Performance, FC
based Disk Arrays
Mid-Range FC based
Disk Arrays
Scalability and availability
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
SAN components
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Identifying SAN components
Host
• Servers
• HBAs
Fabric
• Hubs or switches
• Routers
• SAN software
• Fibre Channel cables
Storage
• Storage devices
• Backup devices
28
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Host component (initiator)
Consists of servers and components that
enable servers to connect to the SAN
• HBAs
− In-server components that perform digital-tooptical signal conversion
• HBA drivers
− System software that enables the operating
system of a server to communicate with the HBA
• Multipath software
− A software component that enables faulttolerance and performance enhancements
(MPIO)
29
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
HBAs
Fibre Channel HBAs
• Can address more devices than the SCSI or
NIC counterparts
• Provide I/O connectivity to more devices over
longer distances than SCSI
• Enable Fibre Channel frames to relay over
gateways
Mezzanine HBA
PCI HBA
30
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Disk array (target)
Disk array characteristics
• Multiple port connections
• Up to 99.999% uptime—about 5 minutes of
downtime per year!
• Battery-backed controller cache for protected
“write-back” caching
• Snapshot and cloning capabilities
• Remote, controller-based replication for data
integrity and disaster recovery
31
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Interconnect devices
Fibre Channel switches
Two types:
• Fabric switches—Smaller fixed
configurations
• Directors—High port count in a modular
(slot-based) chassis with no single point of
failure
Fabric switch
32
SAN director switch
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
SAN boot order
3
33
1
2
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Fibre Channel basics
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Open System Interconnection
What is OSI?
• A reference model is a framework for understanding relationships
• Open System Interconnection (OSI) is a reference model for how messages should be
transmitted between any two points in a telecommunications network
• The purpose of the OSI reference model is to guide vendors so the digital
communication products they create will interoperate
35
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
OSI layers
36
Layer 7
Application
Layer 6
Presentation
Layer 5
Session
Layer 4
Transport
Layer 3
Network
Layer 2
Data-link
Layer 1
Physical
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
OSI layers in the Fibre Channel stack
Layer Title
Fibre Channel
7
Application
6
Presentation
5
Session
4
Transport
FC – 4 Protocol Interface ULP
3
Network
FC – 3 Encryption Authentication
2
Data Link
FC – 2 Framing Flow Control Class of
Service
SCSI-3, IPI, HIPPI, IP
FC – 1 Encoding Link Control
1
37
Physical
FC – 0 Physical
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
World Wide Name
The definition
What is WWN?
• A World Wide Name is a 64-bit IEEE structured address
• Example: 21:11:00:02:AC:00:08:EB
• It is used to preserve the identity of a node if its FC – 2 (Data Link) or FC – 3 (Network) layer address
is changed
• The WWN is unique worldwide, and it is assigned for the life of a connection device
• A WWN consists of three sections:
− Section 1: Identifies the WWN as a standard format WWN
• Only one of the 4 digits is used, the other three must be zeroes
− Section 2: TheOrganizationally Unique Identifier (OUI)
21:00
00:e0:8b
00:e0:8b
or “company_id” that identifies the vendor
Section 1
Section 2
Section 3
− Section 3: A unique identifier created by the vendor
38
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
WWN Port Name and Node Name
Two types of WWNs
• World Wide Node Name (WWNN)—Assigned to the node (server or storage array)
• World Wide Port Name (WWPN)—Assigned to the port of the Fibre Channel device
2xWWPN
1xWWN
N
1xWWN
N
4xWWPN
39
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Fibre Channel WWN
A WWN can be used for:
• Zoning—To identify zone members
• LUN masking—To identify entities that are permitted or denied access to LUN resources within an
array
A WWN is not used for:
• Frame delivery
• Inter-switch (fabric) traffic delivery
40
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Nodes, ports, and links
Device ports
Link
• N_Port—Node port
− A device directly
attached to a fabric
• NL_Port—Node loop port
− A device connected to a
hub
• Connects ports together
• U_Port—Universal port
• Can be a copper or a fiber
− A port that is waiting to become a
optic cable
different port type
• F_Port—Fabric port
− A port that is attached to an N_Port
• FL_Port—Fabric loop port
− A switch connected to a hub
• E_Port—Expansion Port
− A port that is connected to another
switch using an inter-switch link
• G_Port—Generic Port
− A port that is waiting to become an
E_Port or an F_Port
41
Switch ports
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
SAN topologies
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Fibre Channel topologies
Switched fabric (FC-SW)
Arbitrated loop (FC-AL)
N_Port
L_Port
L_Port
F_Port
L_Port
FC
0
FC
1
Fabric
HBA
F_Port
Host
L_Port
L_Port
NL_Port
FL_Por
t
L_Port
FL_Por
L_Port
t
L_Port
L_Port
L_Port
N_Port
L_Port
L_Port
Point-to-point (FC-P2P)
N_Por
t
43
N_Por
t
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Point-to-point topology
• FC-P2P is inexpensive
• Uses full bandwidth and has limited scalability
• Only connects two devices
• A separate P2P configuration must be created for each new storage device, requiring a new HBA for
each one
Node A
Node A
Receiver
Transmitte
r
44
Transmitte
r
Receiver
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Arbitrated loop topology
• A serial, full-duplex data transfer architecture
• Each port (NL_Port) on the loop has a transmit (TX) and receive (RX) lines
• The TX line of the upstream device connects to the RX line of the downstream device
• Only one port at a time can transmit data—the bandwidth is divided among all devices on the loop
• Because of the loop arbitration, performance degrades when the number of devices in the loop
exceeds 35
L_Port
L_Port
L_Port
L_Port
L_Port
L_Port
45
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Private arbitrated loop
• If there is no active FL_Port on the arbitrated loop, it is referred to as a private loop
• The private loop can accommodate up to 126 NL_Ports
• A private loop is not connected to a switch, so communication and bandwidth are limited to the ports
in the loop
NL_Por
t
NL_Por
t
NL_Por
t
L_Port
NL_Por
t
L_Port
L_Port
L_Port
L_Port
L_Port
NL_Por
t
NL_Por
t
46
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Public arbitrated loop
• A public loop has at least one active FL_Port on the loop
• The public loop can accommodate up to 126 NL_Ports and one FL_Port
• The FL_Port extends the number of ports for communication and introduces the loop identifier, which
is common to all NL_Port addresses in the loop
47
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Switched fabric topology
Switched Fabric (FC-SW)
NL_Port
N_Port
F_Port
FL_Port
Fabric
F_Port
L_Port
FL_Port
L_Port
L_Port
L_Port
L_Port
N_Port
L_Port
48
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Comparing topologies
Point to point
Arbitrated loop
Switched fabric
Advantages:
Advantages:
Advantages:
• Full bandwidth for the link
• Scalability
• Good topology for disk drive
I/O
• Multiple devices communicate
at the same time
• Loss of one component does
not interrupt the link
• Full bandwidth for each switch
port
• Performance only minimally
depends on length
Disadvantages:
• High cost for hardware
• No scalability
Disadvantages:
• All ports share bandwidth
• Maximum of 126 ports per
loop
• The failure of one port forces
loop initialization
• Performance depends on the
loop length and the number of
NL_Ports
49
Disadvantages:
• Higher initial cost compared to
Arbitrated Loop
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Fibre Channel port types
(1 of 2)
Name
Description
N_Port
A port on the node (storage device or host).
NL_Port
A port on the node used in the FC-AL topology.
F_Port
A fabric port on the switch that connects to the N_Port.
FL_Port
A fabric loop port on the switch that connects to the FC-AL loop.
E_Port
An expansion port; the connection between two Fibre Channel switches. When ports
between two switches form a link, that link is referred to as an inter-switch link (ISL).
B_Port
A Bridge Port is a fabric inter-element port that is used to connect bridge devices with
E_Ports on a switch. The B_Port provides a subset of the E_Port functionality.
D_Port
A diagnostic port, used for the purpose of running link-level diagnostics.
50
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Fibre Channel port types
(2 of 2)
Name
Description
EX_Port
The connection between a Fibre Channel router and a Fibre Channel switch. On the side of the
switch, it looks like a normal E_Port, but on the side of the router it is an EX_Port.
TE_Port
An extended inter-switch link (ISL) that is used for virtual SANs. Also known as a trunking E_Port.
Auto
An auto-sensing port that can automatically become an E_, TE_, F_, or FL_Port as needed.
Fx_Port
A generic port that can become an F_Port (when connected to a N_Port) or an FL_Port (when
connected to an NL_Port).
GL_Port
A port on a switch that can operate as an E_Port, FL_Port, or F_Port. Found on QLogic switches.
G_Port
A generic port; a port waiting to be used as an E_Port or F_Port. Found on Brocade, McData, and
QLogic switches.
L_Port
A loose term used for any arbitrated loop port, NL_Port, or FL_Port. Also known as a loop port.
U_Port
A loose term used for any arbitrated port or a port waiting to become another port type. Also known
as a universal port. Found only on Brocade switches.
51
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Establishing a new link between ports
When a new link is established between ports, the switch effectively poses 3 questions to
the newly connected port:
• Loop initialization process (LIP)—Do you support loop functions?
• Fabric Login (FLOGI)—Do you support 24-bit addressing?
• All others—Send Link Service frames to establish an “E” port connection?
All other connections will be ignored by the switch port
52
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Fibre Channel architecture
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Fibre Channel function levels
FC-4
• Channels (SCSI-3, HIPPI, SBCCS)
• Networks (802.2, IP, ATM)
FC-3
• Common services
Node level:
• FC-4
• FC-3
FC-2
• Signaling, Framing Protocol, and flow control
FC-1
• Encode and decode
Port level:
• FC-2
• FC-1
• FC-0
FC-0
• Available at 1, 2, 4, 8, 10, 16, and 20 Gbit/s
54
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
FC-0—Physical level
Defines the physical link in the Fibre Channel system
• Transceivers
• Connection
• Media type
Available data rates
• 133 Mbit/s
• 266 Mbit/s
• 531 Mbit/s
• 1062 Mbit/s
55
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Transceivers
Transceiver
Data rate
Distance
SFP
155M/622M/
1.25G/
2.5G/3G/
4.25G
300m/2km/
10km/15km/
20km/40km/
60km/80km/
100km/120km/150km
SFP+
6G/8.5G/10G
220m/300m/
2km/10km/
20km/40km/
60km/80km
XFP
10G
220m/300m/
2km/10km/
20km/40km/
60km/80km/
120km
SFP+ - 10 Gigabit
Application:
• Switches
• Disk controllers
• FCIP/iSCSI bridges
56
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Fibre Channel connectors
• SFP, SFP+, and XFP transceivers are
compatible with the Lucent Connector (LC) type
of connectors
• Cables containing LC connectors on both sides
are known as LC-LC cables
57
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Fibre Channel cabling
58
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Multimode fiber
• Multiple streams of light to travel different paths
• Most popular for networking
• Fibre Channel uses single wavelength
− Example: 850nm
59
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Single-mode fiber
Highest bandwidth and lowest performance loss
• One stream of light travels a single path
• Long wave lasers
• Single-mode, step-index fiber
60
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Single-mode step-index fiber
Best for long-distance communication
61
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Fiber-optic class signal loss—Attenuation
Attenuation
• The reduction in power of the light signal as it is transmitted
• Caused by passive media components such as cables, cable splices, and connectors
62
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Fiber-optic class signal loss—Dispersion
Dispersion
• Spreading of the signal over time
• Two types of dispersion can affect an optical data link:
− Chromatic dispersion—Resulting from the different speeds of light rays
− Modal dispersion—Resulting from the different propagation modes in the fiber
63
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Cable bending and damage
Micro bending
Macro bending
• Difficult to diagnose
• Causes bit transport errors
• Can reduce the effective data transport
distance
• Causes signal degradation
64
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
FC-1 coding layer
FC-1 8b/10b encode/decode
• FC-1 defines the transmission protocol including:
− Serial encoding and decoding rules
− Special characters
− Error control
• The information transmitted over a fiber is encoded 8 bits at a time into a 10-bit transmission
character
Also used in:
• PCI Express
• IEEE 1394b
• Serial ATA
• SSA
• Gigabit Ethernet
• Infiniband
65
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
FC-2 signaling protocol level
The transport mechanism of
Fibre Channel
• Framing rules
• Payload
• Service classes and control mechanisms
• Management of the data transfer sequence
66
Building blocks
• Ordered sets
• Frames
• Sequences
• Exchanges
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
FC-3 common services
• The FC-3 layer covers functions that can span
multiple N-ports
• FC-3 defines the common services necessary for
the higher level capabilities
67
FC-3 provides features such as:
• Port striping
• RAID
• Virtualization
• Compression
• Encryption
• Hunt groups
• Multicast
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
FC-4 ULP mappings
Each upper-level protocol supported by the Fibre Channel transport requires a mapping
for its Information Units to be presented to the lower levels for transport
The FC-4 layer provides these mappings for:
• SCSI-3
• IP
• High-Performance Peripheral Interface (HIPPI)
• FC-AV—A high-bandwidth video link for video networks, up to 500m
• FC-VE—Fibre Channel Virtual Interface Architecture
• FC-AE—Fibre Channel Avionics Environment
• Ficon, IEEE 802.2 LLC, ATM, Link Encapsulation, SBCCS, IPI
A Fibre Channel SAN is almost exclusively concerned with using the SCSI-3 mapping
68
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Introduction to iSCSI
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
IP storage
Meeting storage challenges with IP-based network storage
• Increased utilization
• Reduced management cost
• Increased reliability
• Simplified backup and recovery
70
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
IP storage protocols
71
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Overview of the iSCSI protocol
What is iSCSI?
• iSCSI is a transport layer protocol that describes how SCSI packets should be transported over a
TCP/IP network
• iSCSI works on top of the TCP
• It allows the SCSI command to be sent end-to-end over LANs, WANs, or the Internet
72
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
The iSCSI protocol
• The SCSI protocol has been mapped over various transports such as Parallel SCSI, Firewire, and
Fibre Channel
• These transports are I/O specific and have limited distance capabilities
• The iSCSI protocol uses TCP/IP, which can take advantage of existing Internet infrastructure and
management facilities and address distance limitations
73
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Comparing iSCSI and Fibre Channel
74
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
iSCSI protocol stack
Initiator
Target
SCSI
SCSI
iSCSI
iSCSI
TCP
TCP
IP
IP
IPSec
IPSec
Link
Link
IP
network
75
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
iSCSI encapsulation
76
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
iSCSI drivers and offload engines
iSCSI
TCP
IP
Fabric adapter
TCP
IP
Network
hardware
Network
hardware
77
Other
protocols
SCSI
iSCSI
iSCSI HBA
SCSI
TOE cards
NIC cards
Other
protocols
Apps/file
systems
Apps/file
systems
Apps/file
systems
Other
protocols
SCSI
iSCSI
TCP
IP
Network
hardware
Apps/file
systems
Other
protocols
SCSI
iSCSI
TCP
IP
Network
hardware
Processed in
the network
card
Processed in
the server
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
iSCSI names
• iSCSI names:
− Are used for identification
− Are used for authentication
− Enable iSCSI resources to be managed regardless of their location
• Each iSCSI initiator and target must have an iSCSI name
• The iSCSI name consists of two parts: a “type designation” followed by a unique name string
• The three type designators for iSCSI are:
− iqn.
iSCSI qualified name (iqn.2003-02.com.hp:server3)
− eui.
IEEE EUI-64 identifier in ASCII-encoded hexadecimal (eui.02004567A425678D)
− NAA.
T11 Network Address Authority Format NASA 64 or 128 bit identifier
(naa.52004567BA64782D)
78
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Internet Storage Name Service
An iSNS implementation provides four primary services:
• Name Registration and Storage Resource Discovery
• Discovery Domains and Login Control
• State Change Notification
• Bidirectional Mappings Between Fibre Channel and iSCSI Devices
79
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
iSCSI target discovery
• Before establishing the iSCSI connection, the iSCSI initiator needs to find (discover) targets to which
it has access
• The four discovery methods are:
− iSCSI targets are configured on the initiator
• The initiator uses a configuration file containing the target information
− The iSCSI initiator queries the target
• The initiator issues a SendTargets message to request the list of targets
− The initiator uses the Service Location Protocol (SLP)
• It locates iSCSI targets or SNS without specifying the address
− The initiator queries a Storage Name Server (SNS)
• It locates iSCSI targets without specifying the address
80
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
iSCSI operations
iSCSI login request to
initiate a session over TCP
iSCSI initiator
iSCSI target
Persistent session carrying
the authentication and
exchange of certificates
NOTE: After the persistent state is initialized, iSCSI will use multiple parallel sessions to aggregate
bandwidth and improve performance. The iSCSI session terminates when its TCP session is closed.
81
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
iSCSI security
Authentication
• iSCSI initiators and targets prove their identity to each other using the Challenge Handshake Authentication
Protocol (CHAP).
Logical network isolation
• This is the deployment architecture, to mitigate the authentication risk.
• It is usually provided through the VLAN capability of network equipment.
Physical network isolation
• This is used to prevent cabling mistakes.
Authorization
• iSCSI aims for storage consolidation. Authentication is used to prevent unrelated initiators from accessing storage
resources.
Confidentiality and integrity
• The IPsec protocol provides standards-based cryptographic protection for the iSCSI traffic.
82
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
iSCSI advantages and limitations
iSCSI advantages
iSCSI limitations
A separate network for SAN is not required. You can use
existing IP networks and components.
The IP network is currently a “best effort” network. The
packages might drop or be delivered out of order
because of network congestion.
The iSCSI SAN can coexist with a Fibre Channel-based
SAN.
The server CPU might be burdened with TCP/IP SAN
traffic.
The iSCSI SAN does not have distance limitations.
Running iSCSI on the same network as production might
lead to congestion.
You can use specialized HBAs or standard NICs.
iSCSI operates on a clear text protocol, so the traffic
must be encrypted.
iSCSI is suitable for virtualized environments because it
supports software-based initiators.
It provides a means of direct backup to tape or disks,
even from certain virtual servers.
83
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Fibre Channel over Ethernet
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
What is FCoE?
• Fibre Channel over Ethernet is a mapping of Fibre Channel over selected full-duplex IEEE 802.3
networks
• The goal is to provide I/O consolidation over Ethernet, reducing network complexity in the data center
• Customer benefits of a unified fabric:
− Fewer NICs, HBAs, and cables
− Lower capital expenditures and operating expenses
85
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
FCoE I/O consolidation
86
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
FCoE mapping
• FCoE maps the Fibre Channel commands and data directly into Ethernet frames to create FCoE
− Fibre Channel frames are encapsulated in Ethernet frames
• The mapping is 1:1, meaning there is no segmentation or compression of the Fibre Channel frames
FC- 4
FC- 4
FC- 3
FC- 3
FC- 2
FC- 2
FC- 1
FCoEmapping
MAC
PHY
FC- 0
Ethernet
Header
87
FCoE
FCHeader
Header
SCSI Commands / Data
FCLevel
(Unchanged)
IEEE802.3
Layers
CRC
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
FCoE lossless Ethernet infrastructure
• FCoE has to create a lossless Ethernet environment to ensure the reliability of large-scale data
transportation
• Two standards enable lossless Ethernet
− Data Center Bridging (DCB)
− Converged Enhanced Ethernet (CEE)
• In addition to DCB and CEE, the standard introduces three enhancements to the Ethernet to make it
lossless:
− Priority Flow Control (IEEE 802.1Qbb)
− Congestion Notification (IEEE 802.1Qau)
− Enhanced Transmission Selection (IEEE 802.1Qaz)
88
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Priority Flow Control
Priority Flow Control (IEEE 802.1Qbb)
• IEEE 802.1Qbb is an enhanced QoS service
• Traffic is classified in 8 lanes, each of which could be assigned a priority level
• Priority Flow Control issues a “Pause” command to manage and prioritize traffic when there is
congestion
• The administrators can create lossless (virtual) lanes for FCoE traffic and lossy (virtual) lanes for
normal
IP traffic
89
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Congestion Notification
Congestion Notification (IEEE 802.1Qau)
Congestion is measured at the congestion point, but link rate limiting is taken at the point
of origin
• Example: An aggregation switch can ask an edge switch to stop (or limit) its traffic from a particular
port, if congestion occurs
90
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Enhanced Transmission Selection
Enhanced Transmission Selection (IEEE 802.1Qaz)
• High-priority traffic such as FCoE is allocated with a minimum guaranteed bandwidth
• If the FCoE traffic does not fully utilize its reserved capacity, the extra bandwidth can be used by other
types of traffic, and this can be controlled dynamically
91
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
FCoE components
Fiber Channel
Network (Storage)
1
2
HBA
Converged Network
Adapter (CNA)
3
Ethernet Network
(LAN)
NIC
FCoE Switch / Ethernet Switch
Supporting FCoE
92
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
FCoE advantages and limitations
FCoE advantages
FCoE limitations
FCoE reduces the two network adapters (HBA for
storage connectivity and NIC for network connectivity)
and two individual cables to just one.
The only Ethernet component that is currently compatible
with FCoE is the cables.
FCoE can carry traffic over the Ethernet medium.
The cost of a Unified CNA (although the price is coming
down) might be more than the cost of the HBA and NIC
combined.
Having one network adapter instead of two results in
some power savings for the server.
FCoE is currently restricted to access networks only
(server-to-switch connections).
FCoE can be used in virtualized environments.
Security on FCoE networks might have to be re-evaluated
because the network is now running over Ethernet, which
is more accessible than Fibre Channel.
Unlike iSCSI, FCoE is reliable. It can scale up to
thousands of servers.
93
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.
Thank you
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential – For training purposes only.