iSER on SCTP & IB
draft-hufferd-ips-iser-sctp-ib-00.txt
Generalizations to iSER specification
John Hufferd
Mike Ko
Yaron Haviv
Abstract
• The iSCSI Extensions for RDMA document currently
specifies the RDMA data transfer capability for iSCSI
over iWARP/TCP
• This new document generalizes the iSER document to
permit it to be used with other RDMA capable protocols
such as
– iWARP/SCTP
– InfiniBand
– etc.
• It also describes what should be defined in the InfiniBand
Trade Association and what things are appropriate for
specification in the IETF
2
Motivation
• Current wordage in iSER is only applicable to iSER on
iWARP/TCP
• It was felt that the protocol should be made generic for
RDMA LLPs, and that includes:
– iWARP/SCTP
– InfiniBand
– Etc.
• Makes an iSCSI based protocol apply across more
Networks, and eliminates some of the management and
discovery protocols that would otherwise be needed
– For example, would not have to fix the short comings of SRP
(SCSI RDMA Protocol) on IB
Goals
• To specify changes/adjustments that should be
considered to the wordage in the iSER
document to make it more General
• These changes should not modify the basic
operation of iSCSI/iSER when operating on
iWARP/TCP
• Some of the terminology needed to be clarified
as to the applicability of the terms to the actual
LLP used
Examples of changes (1)
• The term “iWARP protocol suite” is replaced by
“RDMA-Capable Protocol”
• The term “iWARP layer” will be replaced by
“RDMA-capable protocol layer”
• Wherever the “iWARP” term is specific to the TCP
implementation, it will be replaced with
“iWARP/TCP”
• The term “RNIC” will be replaced with “RDMACapable Controller”
– The clause “such as an RNIC” will be added as needed
Examples of changes (2)
• The Steering Tag (STag) term will have its definition
extended such at the IB “Local Steering Tag (L-Key)” and
the “Remote Steering Tag (R-Key)” are included in the
STag definition by way of example
• The Definitions for IRD and ORD terms will have their
definition extended such that the IB “Responder
Resources”, and Initiator Depth” are included in the
definitions by way of example
• The Term RDMA-capable protocol (RCP) will be defined
and used when ever any RDMA wire protocol or RDMA
protocol stack is applicable
– RDMAP should be used only when it explicitly refers the iWARP
protocol (TCP or SCTP)
Examples of changes (3)
• The Term “RDMAP Stream” will be replaced by
the term “RCP Stream” and defined as:
– RCP Stream - A single bidirectional association
between the peer RDMA-capable protocol layers on
two Nodes over a single transport-level stream.
• For TCP or SCTP, an RCP Stream is also known as an
RDMAP Stream.
• For iSER/TCP, the association is created when the
connection transitions to iSER-assisted mode following a
successful Login Phase during which iSER support is
negotiated.
– Needed since SCTP and IB start their RCP stream
mode at connection time
Examples of changes (4)
• The term “RCP Message” should be defined and
used as a replacement for the term “RDMAP
Message” the definition will be:
– RCP Message – The sequence of packets of the
RDMA-capable protocol which represent a single
RDMA operation or a part of RDMA Read Operation.
For TCP or SCTP, an RCP Message is also known as
an RDMAP Message
• When discussing the iSER Hello and HelloReply
Messages the term "iSER Message" will be used
– Instead of “RDMAP Message”
– This distinction is needed in order to accommodate
LLPs that have native message delivery capability,
such as SCTP or IB
Example of Changes (5)
• We permit the iSCSI layer (if appropriate) to use the
RCP message mode capability immediately after
connection establishment before enabling iSER-assisted
mode
– Appropriate for SCTP or IB
– In this case the iSER Hello and HelloReply Messages are not
the first RCP Messages, but they are the first iSER Messages
• Added a discussion of connection establishment along
with the use of the “RCP messaging protocol”, for
exchanging Login Request and Login Response
Messages
– Examples are discussed that are appropriate for SCTP and IB,
along with the transitioning of the connection to iSER mode
Examples of changes (6)
• The discussion of Security specifies that
all non IP protocols will define their own
requirements for IPsec
– However the iSCSI requirements for IPsec are
still required:
• Wherever an iSER Message enters an IP
environment from a non IP one (such as IB)
– The iSCSI/iSER requirement for IPsec on IP
based protocols such as TCP and SCTP
• Will continue to require IPsec as a must implement
Generic example of iSCSI/iSER
layering in Full Feature Mode
IB Informative Section
• Information on how iSER would be used in an IB
network is included in the Draft
– Not intended to be included in the iSER draft unless
included in an informational Appendix
– Various Network topologies are shown
• Host Side IB Network, including Gateways
• Storage Side that includes iSER/IB
– IB iSER discovery process is described
• Use of IP over IB (IPoIB)
– SendTargets
– SLP
– iSNS
• Conversion of IP address into IB GID via ARP processes
– Discussion of what the IBTA needs to define
Host Side iSER/IB Topology
Storage Side with iSER/IB
Info: What the IBTA needs to define
• Means for permitting a Host to establish an iSCSI/iSER connection
with a peer InfiniBand end-node
– Indicating when that end node does not support iSER
• So the Host would be able to fall back to iSCSI/TCP over IPoIB
• Means for permitting the Host to establish connections with
– IB iSER connections on Storage Controllers
or
– IB iSER connected Gateways in preference to IPoIB connected
Gateways/Bridges
or
– Connections to Target Storage Controllers that accept iSCSI via IPoIB
• How to operate in an environment where ZBTO, and SendInvSE are
optional
• iSER ServiceID
• How the ServiceID can be added to the IP port number during the
connection process
Info: Key Implementation areas that
do NOT require IBTA specification
• How implementations determine which
iSCSI/iSER portal group to use
– Basing the decision on new information that may be
placed in the iSCSI discovery information is
Not required
– Simple trial selection is acceptable
• How implementations determine how to best
handle the concept of MC/S as it deals with
multiple IB Addresses:Ports per Portal Group
Preferred Change to Discovery Data
• Useful to have a connection type associated with the
Portal Group Tag
– Will permit the most appropriate connections to be made without
needless connection tries and failures
– Useful for iSER/iWARP/TCP
– Useful for iSER/iWARP/SCTP
– Useful for iSER/IB
– Proposed Syntax
• IP Address:Port[, PG#[, Type#]]
Portal Group Type
iSCSI
iSER/iWARP/TCP
iSER/iWARP/SCTP
iSER/iWARP/IB
IANA Portal Group Type Value
0, or blank
1
2
3
Recommendations
• Have Mike Ko update the iSER wordage
as indicated by this draft
• Include the IB sections as Informational
Appendices in iSER draft
• Send Discovery Data changes to the
authors of iSCSI, iSNS, & SLP for iSCSI