FreeBSD iSER initiator

[Sagi Grimberg]

Hello everyone,

Mellanox Technologies is pleased to announce the availability of a new
iSCSI Extensions for RDMA (iSER) initiator mode support for FreeBSD.

What is it ?

iSER is a computer network protocol that extends the iSCSI storage
networking protocol to use Remote Direct Memory Access (RDMA). RDMA is
provided by either RoCE (RDMA over converged Ethernet), iWARP or
InfiniBand to enable efficient data movement using hardware offload
capabilities.

Why doing this ?

RDMA infrastructure provides benefits such as Zero-Copy, CPU offload,
Reliable transport, Fabric consolidation and many more. The iSER
protocol eliminates some of the bottlenecks in the traditional
iSCSI/TCP stack, provides low latency and high throughput and is well
suited for latency aware workloads.

What was added ?

The iSER transport was added as another icl provider module, and it
opens and RDMA channel to the target (icl_conn_connect). Other than
that the addition of the iser module is pretty straight-forward.

Some minor additions and changes were added to the iscsi layer for iser 
support:
- Not relying on io_received when handling a scsi response since
   io_received is not maintained in iser (data is transferred via RDMA).
- Added a PDU pointer to task_setup so iser can reference the scsiio in
   its private pdu context.
- Added a -T <transport> flag to user-space to indicate using iser.
- Added iSER relevant information to the login request.


The code is available at https://github.com/sagigrimberg/iser-freebsd

The code maturity level is considered as alpha. It has gone through
limited testing on top of 11-current using Mellanox ConnectX-3 adapters
(Login, Logout, I/O). We tested against Linux LIO and TGT targets and
we have several known issues in error flows and session recovery areas
(Documented in release_notes.txt).

We also did some initial performance measurements and we can see that
iser achieves x20 BW (~5.4 GB/s) and x2 IOPs (~100 KIOPs) compared to
iscsi/tcp for a single LUN device.

We would like to get some feedback on the code in general, and
specifically on some of the framework changes that we did in the iscsi
common layers (and user-space). Moreover, feel free to try out the
code, report BUGs (to Max or me at this point) and even provide fixes.

Cheers,

Max Gurtovoy & Sagi Grimberg

Mellanox Technologies