Automatic TCP send and receive socket buffer sizing

[Andre Oppermann]

Chris wrote:
> On 12/12/06, Andre Oppermann <andre at freebsd.org> wrote:
>> This is a patch adding automatic TCP send and receive socket buffer 
>> sizing.
>> Normally the socket buffers are static (either derived from global 
>> defaults
>> or set with setsockopt) and do not adapt to real network conditions.  Two
>> things happen: a) your socket buffers are too small and you can't 
>> reach the
>> full potential of the network between both hosts; b) your socket 
>> buffers are
>> too big and you waste a lot of kernel memory for data just sitting 
>> around.
>>
>> With automatic TCP send and receive socket buffers we can start with a 
>> small
>> buffer and quickly grow it in parallel with the TCP congestion window 
>> to match
>> real network conditions.
>>
>> FreeBSD has a default 32K send socket buffer.  This supports a maximal
>> transfer rate of only slightly more than 2Mbit/s on a 100ms RTT trans-
>> continental link.  Or at 200ms just above 1Mbit/s.  With TCP send buffer
>> auto scaling and the default values below it supports 20Mbit/s at 100ms
>> and 10Mbit/s at 200ms.  That's an improvement of factor 10, or 1000%.
>> For the receive side it looks slightly better with a default of 64K 
>> buffer
>> size.
>>
>> The automatic send buffer sizing patch is currently running on one 
>> half of
>> the FTP.FreeBSD.ORG cluster w/o any problems so far.  Against this 
>> machine
>> with the automatic receive buffer sizing patch I can download at 
>> 5.7MBytes
>> per second.  Without patch it maxed out at 1.6MBytes per second as the 
>> delay
>> bandwidth product became equal to the static socket buffer size 
>> without hitting
>> the limits of the physical link between the machines.  My test machine 
>> is about
>> 35ms from that FTP.FreeBSD.ORG and connected through a moderately 
>> loaded 100Mbit
>> Internet link.
>>
>> New sysctl's are:
>>
>>  net.inet.tcp.sendbuf_auto=1      (enabled)
>>  net.inet.tcp.sendbuf_inc=8192    (8K, step size)
>>  net.inet.tcp.sendbuf_max=262144   (256K, growth limit)
>>  net.inet.tcp.recvbuf_auto=1      (enabled)
>>  net.inet.tcp.recvbuf_inc=16384           (16K, step size)
>>  net.inet.tcp.recvbuf_max=262144   (256K, growth limit)
>>
>> The patch is available here (it may apply with some fuzz):
>>
>>  http://people.freebsd.org/~andre/tcp_auto_buf-20061212.diff
>>
>> Any tests and test reports are very welcome.
>>
>> -- 
>> Andre
> 
> Hi does this patch work on 6.x? I used the send patch on 6.x and works
> great please make a 6.x patch thank you and I will happily test.

No, this patch doesn't work on 6.x.  It makes changes to struct tcpcb
to add two additional fields.  This requires netstat(1) to be recompiled
and is a ABI change.  However I've got a number of requests for 6.x
patch so I may make one anyway.

-- 
Andre