RELENG_8 ignoring TCP window size? [Was: Re: Help for TCP understanding wanted, ACK-MSS-Window [Was: Re: best practice to watch TCP parms of established sockets]]

[Harald Schmalzbauer]

Nikos Ntarmos schrieb am 24.02.2010 16:37 (localtime):
> On Fri, Feb 19, 2010 at 11:55:39AM +0200, Nikos Ntarmos wrote:
>> On Thu, Feb 18, 2010 at 10:41:28PM +0100, Harald Schmalzbauer wrote:
>>> Kevin Oberman schrieb am 18.02.2010 20:23 (localtime):
>>> ...
>>>> window allows for many packets to be in flight and with a 3 Gbps flow,
>>>> that is a LOT of data. While an ACK is sent every two packets of
>>>> received data, the transmitting side does not wait for the ACKs. It
>>> ...
>>>> That is a VERY simple and incomplete explanation of what is happening
>>>> with the window, but most of that is irrelevant in local transfers with
>>> Thanks a lot, then I understood it at least half correct ;) My
>>> missunderstanding was that I thought the receiver would reduce
>>> ACKs... Now I know more :)
>>> But unfortunately that makes it more mysterious where the throughput
>>> problem lies...
>>>
>>> Thanks to everyone so far,
>> Hi there.
>>
>> This is a long shot but have you tried disabling checksum and
>> segmentation offloading? I've found that they cause trouble with some
>> NICs. FYI on FreeBSD the first is done through 'ifconfig -rxcsum' while
>> the latter through 'sysctl net.inet.tcp.tso=0'. If you try this out,
>> remember to disable these features on both communicating boxes for the
>> period of the test, just to be sure that it's not the other box causing
>> these issues. You mentioned samba so if one of them is a win32 box, you
>> can access these settings through the hardware options of your NIC.
> 
> Hi again.
> 
> Just a friendly nudge. :) Did you find the root of these issues yet?

I don't have solid conclusions. But I ruled out some things.
First, the em driver has no problems in my setup. Neither disabling 
rx/txcsum nor disabling TSO made any differenz in trhoughput, though no 
direct recognizable load behaviour on my 2x3GHz machine. Mybe checsum 
offload is beneficial on slower machines.
One major problem was that one of the two FreeBSD Boxes was on a VMware 
which slowed things down a bit, although the FreeBSD System showed 
plenty free resources.

Disabling rfc1323 defnetily increases the throughput on gigabit 
ethernet. I can rsync between two (native) FreeBSD machines with 72-92 
MB/s averaging at 80+MB/s.
What I couldn't investigeate yet is why I always get 10% more throughput 
when one side is windwos, no matter which direction, no matter what 
application (cifs, rsync, ftp).
Another big point on my todo list is to find out why tcp.inflight brakes 
my webserver downloads really often to less than a quarter of the 
available bandwith (client bw, server bw is 100mb). I saw many 10mb/s E3 
pipes with 15ms delay, but limited transfer rates to 200kb/s. Disabling 
tcp.inflight opens that brake.

Since I could only capture "bad IP length" packets with checsum 
offloading enabled on the em, I guess it does also IP checksum 
offloading, not only TCP. I'll have to set up a port mirroring test bed 
to get the line packets. I hope I'll find the rfc1323 slowdown and the 
difference between FreeBSD clients and Windows clients. But at the 
moment I don't have spare equipment.

Greets,

-Harry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20100225/0b4f2ff4/signature.pgp