iSER - iSCSI RDMA

About iSER - iSCSI Extension for RDMA Transports

iSCSI is an IETF standard for transferring block storage over networks. It has a comprehensive set of storage networking capabilities enabling such features as:

• Plug and Play deployment,
  
• Name services,
  
• Automated discovery,
  
• Security,
  
• Automated error and failure recovery,
  
• Change modifications,
  
• Load balancing, and
  
• Standard management interfacing.
  

Storage vendors, end users and leaders in the open source community have stepped forward to support Voltaire's development and contribution of iSER.

A critical issue with iSCSI technology is performance. The implementation requires placing a layer over the slow TCP/IP transport and using software-based TCP processing. A new IETF standard called iSER was developed, adding a direct access RDMA-based transport under iSCSI. The result is much better in performance over TCP. iSER is also an official IBTA protocol.

The iSER (iSCSI Extension to RDMA) transport resides under a newly defined API layer called Datamover Architecture (DA). The new layer separates the iSCSI implementation from the transport. This enables iSCSI to dynamically select the best available transport (TCP, iSER, TCP Offload) for any given connection based on hardware and path availability, thus improving performance transparently without compromising features or data management. The new iSCSI architecture separates the actual SCSI commands and data that are delivered through the fast Datamover, from the management traffic (discovery, iSNS, SLP, etc) that use standard TCP and UDP protocols.





The combination of iSCSI and iSER takes the best of both worlds: the simplicity, industry momentum and comprehensive standardization of iSCSI with the performance achieved by RDMA and hardware accelerated protocols. The iSCSI /iSER combination enables simpler solutions than FC, better functionality, and better performance at much lower costs.

By supporting iSER over InfiniBand end-users automatically gain all the features and usability of iSCSI already incorporated into most operating systems and management platforms. Combined with the performance of InfiniBand, the iSCSI and DA enable using the same driver for Ethernet NICs, Ethernet RDMA (RNICs), and InfiniBand HCAs. The driver can automatically select the fastest transport per destination transparently to the application. In all cases the iSCSI discovery and management are delivered through standard iSCSI TCP/IP protocols such as iSNS and SLP (over IPoIB in the InfiniBand cases). The standard iSCSI management platforms are not aware of the transport layer that was selected. This enables InfiniBand storage to fully leverage on existing storage management tools and protocols, and be considered as yet another very fast iSCSI device (or iSCSI to FC gateway in case of a router) without the need to purchase or deploy any new management software.

The iSCSI specifications also define standard mapping from iSCSI to FC networks, this allows FC resources to be mapped to iSCSI or iSER devices, and iSCSI or iSER devices to be mapped to FC as standard FC devices with valid WWPN number, this allows an administrator to use a single console from his preferred storage management provider to have visibility of InfiniBand, FC, and iSCSI storage. This greatly simplifies storage management in heterogeneous environments, and allows storage systems to become true Plug & Play.

Voltaire recently developed an Open Source implementation of DA and iSER. The Datamover API (DA) is incorporated into the Linux-iSCSI code, allowing the selection of different transports (Datamovers) and the adaptation of the TCP transport to the architecture. The iSER Datamover code was contributed to the OpenIB OpenSource project.

The iSER Datamover implementation uses kDAPL as its lower layer API, allowing it to run on different RDMA transports as well as different RDMA/InfiniBand implementations.

Voltaire provides an iSER target reference implementation kit for storage and RAID vendors who are interested in accelerating iSER support for their existing storage products.

Voltaire switch/router products provide transparent high-performance routing between iSER over InfiniBand, Fiber Channel Protocol (FCP), and iSCSI/TCP, as well as embedded switch based storage virtualization, which facilitates true storage network consolidation. By leveraging on the above mentioned industry standards Voltaire storage routing products can be administered and provisioned out of the box by any standard storage management platform for iSCSI or FC networks.

Links and Resources

iSCSI Specification
iSER and DA Production
iSER Specification
iSER over IB Overview
iSER extended to IB
iSCSI-Linux Project
OpenIB Project