svn commit: r274853 - in head: sys/dev/iscsi usr.sbin/ctld usr.sbin/iscsid

[Alexander Motin]

On 22.11.2014 20:03, Navdeep Parhar wrote:
> On Sat, Nov 22, 2014 at 03:09:19PM +0000, Alexander Motin wrote:
>> Author: mav
>> Date: Sat Nov 22 15:09:18 2014
>> New Revision: 274853
>> URL: https://svnweb.freebsd.org/changeset/base/274853
>>
>> Log:
>>    For both iSCSI initiator and target increase socket buffer sizes before
>>    establishing connection.
>>
>>    This is a workaround for Chelsio TOE driver, that does not update socket
>>    buffer size in hardware after connection established, and unless that is
>>    done beforehand, kernel code will stuck, attempting to send/receive full
>>    PDU at once.
>
> cxgbe's TOE driver does update the rcv window in the hardware in
> response to changes in the socket buffer size (see t4_rcvd in
> cxgbe/tom/t4_cpl_io.c).
>
> The bug is that the driver tries to avoid sending too many small updates
> to the chip, and instead tries to build up a balance of credits before
> an update.  This doesn't interact well with some applications that set
> the sockbuf size to a high fixed value and then wait for the entire
> buffer to fill up before draining it.  The driver ends up waiting for
> the application to drain the socket's receive buffer so that pru_rcvd
> (t4_rcvd) can then send a big batch of credits to the chip, the
> application is waiting for the receive to reach its high water mark
> before draining the buffer, but the TOE on the chip has shut down the
> receive window because it's out of credits.  If you see the peer sending
> zero window probes and the TOE refusing to budge, then the problem
> you've observed is definitely the problem I just described.

Yes, it looks like this. You may find the packet dump with the problem 
here: https://people.freebsd.org/~mav/rcvbuf.dump

> I'll fix this in the TOE driver.  I can send you an early patch in case
> you'd like to try out the fix before it gets committed.

Thank you. I'll be happy to try it.

-- 
Alexander Motin