bogus 0 len IP packet, was: Hang in VOP_LOCK1_APV on 8-STABLE with NFS.

[Ronald Klop]

On Mon, 07 Feb 2011 01:22:36 +0100, Pyun YongHyeon <pyunyh at gmail.com>  
wrote:

> On Sun, Feb 06, 2011 at 11:54:49PM +0100, Ronald Klop wrote:
>> On Sat, 22 Jan 2011 00:01:47 +0100, Ronald Klop
>> <ronald-freebsd8 at klop.yi.org> wrote:
>>
>> >On Tue, 18 Jan 2011 09:38:04 +0100, <sthaug at nethelp.no> wrote:
>> >
>> >>>> So, does anyone have an idea why the IP length field would be set  
>> to
>> >>>0
>> >>>> for these TCP/IP packets?
>> >>>>
>> >>>> Here's some info from Ronald w.r.t. his hardware. (All I can think
>> >>>of is
>> >>>> that he could try disabling TSO, etc?)
>> >>>>
>> >>>> Thanks in advance for any help with this, rick
>> >>>>
>> >>>
>> >>>It seems that issue came from TSO. Driver will set ip_len and
>> >>>ip_sum field to 0 before passing the TCP segment to controller.
>> >>>The failed length were 4446, 5858, 3034 and 4310 and the total
>> >>>number of such frames are more than 35k within 90 seconds. Since
>> >>>failed length 4310 is continuously repeated I guess there is edge
>> >>>case where em(4) didn't free failed TCP segment for TSO.
>> >>>I remember there was commit to HEAD(r217295) which could be related
>> >>>with this issue.
>> >>
>> >>I'm seeing the same problem with Broadcom NetXtreme (bce) cards:
>> >>
>> >>bce0 at pci0:3:0:0:        class=0x020000 card=0x03421014 chip=0x164c14e4
>> >>rev=0x12 hdr=0x00
>> >>    vendor     = 'Broadcom Corporation'
>> >>    device     = 'Broadcom NetXtreme II Gigabit Ethernet Adapter
>> >>(BCM5708)'
>> >>    class      = network
>> >>    subclass   = ethernet
>> >>
>> >>This is with 8.2-PRERELEASE. Turning off TSO (ifconfig bce0 -tso)
>> >>removes the problem.
>> >>
>> >>Steinar Haug, Nethelp consulting, sthaug at nethelp.no
>> >>_______________________________________________
>> >>freebsd-net at freebsd.org mailing list
>> >>http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> >>To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>> >
>> >I tried -tso and -txcsum in various combinations, but it didn't solve
>> >the problem. I wil look for another brand of network card to try. But
>> >this has to wait till monday when I'm at the office again.
>>
>> I also used another network card (rl0) and it has the same problem with
>> NFS. I'm going to change some network cables to see if that helps. I  
>> have
>> some hints that there might be something wrong with that.
>>
>
> Hmm, given that rl(4) also shows the issue it seems the issue could
> be in TCP/IP stack, not in driver side. rl(4) is dumb device so
> network stack should do segmentation and checksum computation.
> I highly doubt the issue came from faulty cable since other users
> also reported the same issue.
> Unfortunately I have no clue yet and I was not able to reproduce it
> on my box. I vaguely guess some code in kernel changed the ip_len
> to 0 in the middle of transmission. Rick's captured traffic looks
> normal except 0 ip_len given that controller is computing checksum
> on the fly. If mbuf chain was corrupted(e.g. m_len == 0) driver
> would have failed to send those frames.

Changing the cable didn't help indeed. I'm glad the issue is seen by  
others too. I will try to downgrade to an older version of FreeBSD to try  
to find the commit which broke it. But that can take a while, because it  
is time consuming and I have to do some real work also at work. :-)

Thanks for taking the time for it and I hope we will find the cause  
someday,

Ronald.