iSER on IB
draft-hufferd-iser-ib-00.txt/pdf
Generalizations to iSER specification
John Hufferd
Mike Ko
Yaron Haviv
Abstract
• The iSCSI Extensions for RDMA
document currently specifies the RDMA
data transfer capability for iSCSI over
iWARP/TCP
• This new document generalizes the iSER
document to permit it to be used with other
RDMA capable protocols such as
– InfiniBand
2
Motivation
• Current wordage in iSER is only applicable to iSER on
iWARP/TCP
• It was felt that the protocol should be made generic for
RDMA LLPs, and specifically for:
– InfiniBand
• Makes an iSCSI based protocol apply across both IB
and iWARP Networks thus
– Enabling the same basic iSCSI protocol on TCP/IP, iWARP, & IB
– Eliminating some of the management and discovery protocols
that would otherwise be needed
• For example, would not have to fix the short comings of SRP (SCSI
RDMA Protocol) on IB
Goals
• To specify changes/adjustments to the
wordage in the iSER document to make it
more General and specifically applicable
to IB
• These changes should not modify the
basic operation of iSCSI/iSER when
operating on iWARP/TCP
Examples of changes (1)
• The term “iWARP protocol suite” is generally
replaced by “RDMA-Capable Protocol” (RCP)
– The protocol or protocol suite that provides the RDMA
functionality, e.g., iWARP, InfiniBand
• Wherever the “iWARP” term is specific to the TCP
implementation, it is replaced with “iWARP Layer”
• The term “RNIC” is replaced with “RDMA-Capable
Controller” (RCC)
– The clause “such as an RNIC” will be used when an
iWARP implemented RCC is intended
Examples of changes (2)
• The Steering Tag (STag) term has its definition extended
such that the IB “Local Steering Tag (L-Key)” and the
“Remote Steering Tag (R-Key)” are included in the STag
definition by way of example
• The Definitions for IRD and ORD terms will have their
definition extended such that the IB “Responder
Resources”, and Initiator Depth” are included in the
definitions by way of example
• The Term RDMA-capable protocol (RCP) is used when
ever any RDMA wire protocol or RDMA protocol stack is
applicable
– RDMAP is used only when it explicitly refers the iWARP protocol
Examples of changes (3)
• The Term “RDMAP Stream” is replaced by the
term “RCP Stream” and defined as:
– RCP Stream - A single bidirectional association
between the peer RDMA-capable protocol layers on
two Nodes over a single transport-level stream.
• For TCP an RCP Stream is also known as an RDMAP
Stream
• For iSER/TCP, the association is created when the
connection transitions to iSER-assisted mode following a
successful Login Phase during which iSER support is
negotiated
– Needed since IB start its RCP stream mode at
connection time
Examples of changes (4)
• The term “RCP Message” is defined and used
as a replacement for the term “RDMAP
Message” the definition is:
– RCP Message – The sequence of packets of the
RDMA-capable protocol which represent a single
RDMA operation or a part of RDMA Read Operation.
For TCP, an RCP Message is also known as an
RDMAP Message
• When discussing the iSER Hello and HelloReply
Messages the term "iSER Message" is used
– Instead of “RDMAP Message”
– This distinction is needed in order to accommodate
LLPs that have native message delivery capability,
such as or IB
Example of Changes (5)
• The following is placed/updated in
the Acronym Section
• HCA Host Channel Adapter
• IB InfiniBand
• IPoIB IP over InfiniBand
• LLP Lower Layer Protocol
• TCA Target Channel Adapter
Example of Changes (6)
• We permit the iSCSI layer (if appropriate) to use the
RCP message mode capability immediately after
connection establishment before enabling iSER-assisted
mode
– Appropriate for IB
– In this case the iSER Hello and HelloReply Messages are not
the first RCP Messages, but they are the first iSER Messages
• Added a discussion of connection establishment along
with the use of the “RCP messaging protocol”, for
exchanging Login Request and Login Response
Messages
– Examples are discussed that are appropriate for IB, along with
the transitioning of the connection to iSER mode
Examples of changes (7)
• The discussion of Security specifies that all non
IP protocols will define their own requirements
for IPsec
– However the iSCSI requirements for IPsec are still
required:
• Wherever an iSER Message enters an IP environment from a
non IP one (such as IB)
– There is now a SHOULD implement (optional to use)
requirement on non IP networks for a packet by
packet security facility that is at least as strong as that
required by iSCSI
Note: “Should” is used instead of “MUST” since currently IB does not have such
a capability. Also, since some implementations use IB like an internal PCIe Bus, some felt it inappropriate to have a “MUST”
Generic example of iSCSI/iSER
layering in Full Feature Mode
IB Informative Appendix (B)
• Explains how an InfiniBand RC connection can
be used to carry the iSER protocol
– Various IB Network topologies are shown
• Host Side IB Network, including Gateways
• Storage Side that includes iSER/IB
– IB iSER discovery process is described
• Use of IP over IB (IPoIB)
– SendTargets
– SLP
– iSNS
• Conversion of IP address into IB GID via ARP processes
– Discussion of what the IBTA are expected to define
Recommendations
• Accept this Draft as an IPS Draft
Backup Charts
Host Side iSER/IB Topology
Storage Side with iSER/IB
Info: What is Expected that IBTA will define
• The iSER ServiceID
• Means for permitting a Host to establish an iSCSI/iSER connection
with a peer InfiniBand end-node
– Indicating when that end node does not support iSER
• So the Host would be able to fall back to iSCSI/TCP over IPoIB
• Means for permitting the Host to establish connections with
– IB iSER connections on Storage Controllers in preference to a Gateway
path
or
– IB iSER connected Gateways in preference to IPoIB connected
Gateways/Bridges
or
– Connections to Target Storage Controllers that accept iSCSI via IPoIB
• How the ServiceID can be added to the IP port number during the
connection process